Measuring Goals for DevOps Test Succcess

Each organization will have to define what their quality goals are for integrated testing based on the business and customer requirements for speed and outcomes.  The fact is that these goals must be quantified or in other words measurable. If you are not measuring your testing activities in alignment with the strategic goals then success becomes subjective and it will be very difficult to show value for your effort.

Some examples of measurements might be in the form of Run-To-Plan and Pass-To-Plan.
  • Run-To-Plan (RTP) is the number of the total planned tests that have completed and typical goals are to have 95% RTP
  • Pass-TopPlan is the number of tests that have passed and a typical goal for this metric is to have a 90% PTP
The criteria for determining when testing is complete is agreed by all stakeholders.   It will be impossible to have all stakeholders on the sprint team, but certainly input and validation from key stakeholders will have to be included before acceptance of the test results.

When considering goals for reliability an S-curve diagram for pass rates could be used to show when the pass rate stabilizes to meet your goal. 
  • Example:  95% pass rate for 3 consecutive test cycle
The recommendation is to consider your strategic and customer goals.   Determine the metrics and the measurements like those examples above for testing and how measuring these things can help you to meet those higher-level goals.  Other goals and metrics will need to be aligned and none more so than the release criteria which will include metrics to determine whether the Entity Under Test (EUT) is ready for release.  What are your “goals”?  What metrics and measurements from integrated testing will be required for you to meet them?  Share with us what you are measuring and what goals these metrics might help others to meet.

For DevOps education and certification:
…educate and inspire!


Comments

Popular posts from this blog

The Role of Process Practitioner

The Difference between Change and Release Management

What is the difference between Process Owner, Process Manager and Process Practitioner?

Search This Blog