CIstat is a Python package supporting to fetch, analyze and present automation test data from common CI services as CircleCI, Travis, Bamboo and Jenkins (preview version has implemented CircleCI to verify the idea)
Installation
CIstat with preview version could be installed with pip CVS interface from github directly.
Install cistat Pypi dist with dev branch latest code from Github:
pip install https://github.com/maxwu/cistat/archive/dev.zip |
Install cistat from master (release) branch on Github:
pip install https://github.com/maxwu/cistat/archive/master.zip |
Usage
Sample Charts
Pie Chart
report.get_class_rpt().plot_piechart_casenum(project, "Case Num per Class") |
The Pie Chart for case numbers per class level is:
Bar Chart
report.plot_barchart_rate(project, "Pass Rate per case") |
The Bar Chart of Pass Rate for cistat project is:
Bubble Chart
The ROI chart is more considerable for nightly build result analysis in a testing project within XUnit test framework to control the resources as case, docker/cpu, feature integration.
In the sample height of bubble represents the pass rate and the size of bubble shows the test efforts (Currently it is simply the case number; It will be updated to a formula on test time and case number). The lower the bubble positions, the bigger it shall grow to get more resource and invest to float up to 1.0 the stable status.
report.get_class_rpt().plot_scatter_roi(project, "Test ROI per Class") |
The Bubble Chart to present case ROI per class as below:
Console Entry Point
Quick step to run above sample: cistat-cli. Since circleci.com has throttling limit on RESTful API, it is recommended to configure an API-Key in environmental variable or local config.yaml.
This console command is installed with Pypi dist. cistat-cli is the console entry point planted by PIP installer setup.py.
Configuration
Configuration item preference order is:
Cli Options >> Environment Variables >> Local Config.yaml
Configuration items:
- circleci_api_token: The 40 token characters to avoid throttling.
To test string length with shell,
printf $STR | wc -c
Design
Data Models
Now CIstat has implemented XUnit model. Which is a widely used test report XML format. CIstat wrappered it with full functional Python type with +, +=, getter/setter with [key] and len() to simplify the further applications.
As a practice to prepare for future Cloud related statD thoughts (as DataDog service), XUnit is firstly picked up as XUnit has counter, scalar and ratio types of data. These data in various types can be accumulated or aggregated with +, += and speficied with [] operator.
The future step on data model is to expand to white-box parts as Lint scores and code coverage.
Requests
So far only circleci RESTful interfaces are supported. TravisCI and Jenkins are on following step while the preview version is under more tests.
References
[1] The README document with samples, https://github.com/maxwu/cistat
[2] My Gitbook chapter about Pip installation from Github, https://maxwu.gitbooks.io/python-tips/content/pip-supports-installation-from-github.html
Change Logs
May 10, 2017: Init post.