Skip to main content

Measuring Goals for DevOps Test Succcess

Each organization will have to define what their quality goals are for integrated testing based on the business and customer requirements for speed and outcomes.  The fact is that these goals must be quantified or in other words measurable. If you are not measuring your testing activities in alignment with the strategic goals then success becomes subjective and it will be very difficult to show value for your effort.

Some examples of measurements might be in the form of Run-To-Plan and Pass-To-Plan.
  • Run-To-Plan (RTP) is the number of the total planned tests that have completed and typical goals are to have 95% RTP
  • Pass-TopPlan is the number of tests that have passed and a typical goal for this metric is to have a 90% PTP
The criteria for determining when testing is complete is agreed by all stakeholders.   It will be impossible to have all stakeholders on the sprint team, but certainly input and validation from key stakeholders will have to be included before acceptance of the test results.

When considering goals for reliability an S-curve diagram for pass rates could be used to show when the pass rate stabilizes to meet your goal. 
  • Example:  95% pass rate for 3 consecutive test cycle
The recommendation is to consider your strategic and customer goals.   Determine the metrics and the measurements like those examples above for testing and how measuring these things can help you to meet those higher-level goals.  Other goals and metrics will need to be aligned and none more so than the release criteria which will include metrics to determine whether the Entity Under Test (EUT) is ready for release.  What are your “goals”?  What metrics and measurements from integrated testing will be required for you to meet them?  Share with us what you are measuring and what goals these metrics might help others to meet.

For DevOps education and certification:
…educate and inspire!


Comments

Popular posts from this blog

The Four Ps of Service Design - It’s not all about Technology

People ask me why I think that many designs and projects often fail. The most common answer is from a lack of preparation and management. Many IT organizations just think about the technology (product) implementation and fail to understand the risks of not planning for the effective and efficient use of the four Ps: People, Process, Products (services, technology and tools) and Partners (suppliers, manufacturers and vendors). A holistic approach should be adopted for all Service Design aspects and areas to ensure consistency and integration within all activities and processes across the entire IT environment, providing end to end business-related functionality and quality. (SD 2.4.2) People:   Have to have proper skills and possess the necessary competencies in order to get involved in the provision of IT services. The right skills, the right knowledge, the right level of experience must be kept current and aligned to the business needs. Products:   These are the technology managem

What Is A Service Offering?

The ITIL4 Best Practice Guidance defines a “Service Offering” as a description of one or more services designed to address the needs of a target customer or group .   As a service provider, we can’t stop there!   We must know what the contracts of our service offering are and be able to put them into context as required by the customer.     Let’s explore the three elements that comprise a Service Offering. A “Service Offering” may include:     Goods, Access to Resources, and Service Actions Goods – When we think of “Goods” within a service offering these are the items where ownership is transferred to the consumer and the consumer takes responsibility for the future use of these goods.   Example of goods that are being provided in the offering – If this is a hotel service than toiletries or chocolates are yours to take with you.   You the consumer own these and they are yours to take with you.               Note: Goods may not always be provided for every Service

What is the difference between Process Owner, Process Manager and Process Practitioner?

I was recently asked to clarify the roles of the Process Owner, Process Manager and Process Practitioner and wanted to share this with you. Roles and Responsibilities: Process Owner – this individual is “Accountable” for the process. They are the goto person and represent this process across the entire organization. They will ensure that the process is clearly defined, designed and documented. They will ensure that the process has a set of Policies for governance. Example: The process owner for Incident management will ensure that all of the activities to Identify, Record, Categorize, Investigate, … all the way to closing the incident are defined and documented with clearly defined roles, responsibilities, handoffs, and deliverables.  An example of a policy in could be… “All Incidents must be logged”. Policies are rules that govern the process. Process Owner ensures that all Process activities, (what to do), Procedures (details on how to perform the activity) and th