Measuring Goals for DevOps Test Succcess

Each organization will have to define what their quality goals are for integrated testing based on the business and customer requirements for speed and outcomes. The fact is that these goals must be quantified or in other words measurable. If you are not measuring your testing activities in alignment with the strategic goals then success becomes subjective and it will be very difficult to show value for your effort.

Some examples of measurements might be in the form of Run-To-Plan and Pass-To-Plan.

Run-To-Plan (RTP) is the number of the total planned tests that have completed and typical goals are to have 95% RTP

Pass-TopPlan is the number of tests that have passed and a typical goal for this metric is to have a 90% PTP

The criteria for determining when testing is complete is agreed by all stakeholders. It will be impossible to have all stakeholders on the sprint team, but certainly input and validation from key stakeholders will have to be included before acceptance of the test results.

When considering goals for reliability an S-curve diagram for pass rates could be used to show when the pass rate stabilizes to meet your goal.

Example: 95% pass rate for 3 consecutive test cycle

The recommendation is to consider your strategic and customer goals. Determine the metrics and the measurements like those examples above for testing and how measuring these things can help you to meet those higher-level goals. Other goals and metrics will need to be aligned and none more so than the release criteria which will include metrics to determine whether the Entity Under Test (EUT) is ready for release. What are your “goals”? What metrics and measurements from integrated testing will be required for you to meet them? Share with us what you are measuring and what goals these metrics might help others to meet.

For DevOps education and certification:

DevOps Foundation

DevOps Test Engineer

DevOps Leaders

…educate and inspire!

Comments

Four Service Characteristics

Recently I came across several articles by researchers and experts that laid out definitions and characteristics of services. ITIL provides us with a definition that can help drive the creation of value-laden services: A means of delivering value to customers by facilitating outcomes customers want to achieve without the ownership of specific costs and risks. An area that ITIL is not so clear is in terms of service characteristics. Several researchers and experts put forth that services have four basic characteristics (IHIP): Intangibility—Services are the results of actions not things. They have no physical presence and represent a logical set of elements. One way to think of service is “work done for others.” Heterogeneity—Also known as “variability”; services are unique items because of the mechanisms used to deliver services, which is people. Because the people element adds variability, the service is variable. This holds true, especially for the value proposition—not eve...

What Is A Service Offering?

The ITIL 4 Best Practice Guidance defines a “Service Offering” as a description of one or more services designed to address the needs of a target customer or group. As a service provider, we can’t stop there! We must know what the contracts of our service offering are and be able to put them into context as required by the customer. Let’s explore the three elements that comprise a Service Offering. A “Service Offering” may include: Goods, Access to Resources, and Service Actions 1. Goods – When we think of “Goods” within a service offering these are the items where ownership is transferred to the consumer and the consumer takes responsibility for the future use of these goods. Example of goods that are being provided in the offering – If this is a hotel service then toiletries or chocolates are yours to take with you. You the consumer own these and they are yours to take with you. ...

What is the difference between Process Owner, Process Manager and Process Practitioner?

This article was originally published in 2015. With the Introduction of ITIL 4, some of this best practice has changed. See ITIL 4 and the Evolving Role of Roles . Updated Definitions in ITIL 4: Process Owner: In ITIL 4, the concept of 'processes' has expanded into broader 'practices.' Consequently, the Process Owner is now often referred to as the 'Practice Owner.' This individual is accountable for the overall design, performance, integration, and improvement of a specific practice within the organization. They ensure that the practice achieves its intended outcomes and aligns with the organization's objectives. Process Manager: Now commonly known as the 'Practice Manager' in ITIL 4, this role is responsible for the day-to-day management of the practice. The Practice Manager ensures that activities are carried out as intended, manages resources assigned to the practice, and oversees the practitioners performing the work. Process Practit...

Search