We can’t control things which we can’t measure.
When software is under development, testers need tocontinually test and measure its quality and report the status to the concerned people. They also need to evaluate test effectiveness as a guide for improvement. None of these tasks would be possible without measurement and metrics in test management. To meet set expectations in terms of software quality, you simply can’t ignore their importance.
Importance of Measurement and Metrics in testing
With the help of well-established measurements and metrics, a test analyst can clearly track and communicate quality outcomes to the management. Improper establishment of these indicators lead to a purely subjective assessment of the software quality which in turn can develop some major disputes at the end of the SDLC. Some of the common areas where measurement and metrics can be applied are:
- Schedule of the project
- Coverage
- Proposed and actual cost
- Workload vis-à-vis resource usage
- Project risk and product risk
- Inadequacies
Measurement and metrics are simple, easily collectable, easily communicable and only focused on products and processes. These indicators act as advance warning systems that supply information on when to make a pause and look back for any necessary corrective measure.
Key Metrics & how they are calculated
Test metrics help us to measure and subsequently perform the following aspects:
- Understand the required corrective measures
- Decide on the subsequent phase of activities
- Decide whether a procedural or technological change is required
Key test metrics are classified as the following types:
- Base Metrics: These are obtained from the data by test analyst during test case development and execution.
- Calculated Metrics: These are obtained from the data collected in base metrics and are generally tracked for reporting purpose.
Proper setting of these metrics help you get a good control of both quality and estimates. The number of defects can be measured and remedial actions can be taken right away. Though Prototype and Agile models are used in testing most projects these days, the Waterfall model still holds good for smaller projects with straightforward and testable requirements in a stable (not dynamic) setting.
Tracking and sharing significant agile metrics greatly help in enlightening the progress as well as setbacks of the development team and in lowering confusion throughout the SDLC. Some key metrics in an agile environment are:
- Sprint Burndown: It tracks the work completed throughout a sprint.
- Epic and Version Burndown: These delivery vehicles track the advancement of development over significantly greater bodies of work than sprint Burndown.
- Velocity: It stands for the average volume of work completed by a scrum team during a sprint.
- Control Chart: It measures cycle time of issues.
- Cumulative Flow Diagram: It helps to ensure the consistency of the work flow across the team.
Following factors should be considered before identifying correct business and agile metrics:
- Define the objective for metrics
- Examine the cost benefit facets of each metrics in accordance with the SDLC
- Introduce all relevant metrics based on the project requirement
To understand the calculation process of test metrics, the following example of obtaining percentage of “passed test cases” can be taken:
Test cases passed = (No. of test cases passed / No. of total test cases written) x 100
Similarly, other parameters like test cases executed, test cases failed etc can be calculated as well.
Measurement Criteria
Ambiguity should be avoided as much as possible to obtain correct measurement. Following standardized criteria should be considered to attain unambiguous measurement:
- Existence of a common baseline
- Measurers should be allowed to use the associated reference concept thus be able to be on same page
- A point of reference for measurers to substantiate their measurement results
Key Benefits
Measurement and metrics provide a myriad of benefits in test management. Some of the most important ones are:
- Improve project planning
- Help in cost savings by averting defects
- Help to analyze associated risks in a better manner
- Improve ROI of test automation
- Facilitate the team to understand whether the expected quality is achieved
- Enhance coordination between testing coverage, complexities and risks of the systems
gomadeindiaInc: A Different Approach to Software Testing
With gomadeindia, you can leverage the defined metrics to measure quality, cost, reliability and speed of testing service delivery accurately. This will let you set up a software quality measurement culture across your company. Once your quality measurement processes reach satisfactory maturity levels, test specialists at gomadeindia can help you in implementing more relevant metrics dynamically, which would facilitate collection of additional information that would come handy in carrying our further testing improvements.
Are you ready to measure the quality and ROI of your testing efforts?
