Skip to main content

DevOps - Measuring Success

While you might not be able to get your head around how to measure DevOps when it is defined as a cultural and professional movement, there are some key factors to consider that will help us to measure the success of shifting to a collaborative and integrated team throughout the value stream that includes Dev and Ops.

It is clear that DevOps goes beyond the integration of the development and operation staff but includes breaking down those silos between all parties and stakeholders including the Business, external partners, and 3rd party suppliers and IT.  If you think it is difficult to get your own internal teams to play in the same sandbox consider the challenges when 3rd party vendors and suppliers are then thrown into the mix. Being able to demonstrate proof that DevOps practices benefit the organization requires examining factors that influence overall performance.  Practices that enable your organization to improve the flow of work between the Business and Operations enables improved IT performance.

Important Factors

IT Performance in measured in terms of throughput and stability.  
  • Throughput is measured by deployment frequency and lead time for change
  • Stability is measured by mean time to recover and the ability to preemptively detect and mitigate problems

Simply put, you cannot measure Development outcomes separately from Operational outcomes. 

Key Metrics

Showing proof that DevOps practices benefit the organization also requires baselining key metrics before and after making improvements and can include:
  • Deployment Frequency
  • Change Lead Time – Time from request to delivery
  • Cycle Time – Time from start of work to ready for delivery to customer
  • Change Failure Rate – Backed out, Rescheduled or unforeseen impact/incidents
  • MTTD – Mean time to detect incidents
  • MTTR - Mean Time to Recover or repair - Component
  • MTRS - Mean Time to Restore - Service

Change lead time is one of the most important metrics as it represents what the customer sees.  Cycle time is a more mechanical measure of process capability. Lead time depends on cycle time, but also depends on your willingness to keep a backlog, the customer’s patience, and the customer’s readiness for delivery.  MTTR is measured from when a component fails until it is repaired. MTTR does not include the time required to recover or restore service. e.g., data may need to be recovered before service is fully restored and delivering its normal functionality.



Comments

Popular posts from this blog

The Four Ps of Service Design - It’s not all about Technology

People ask me why I think that many designs and projects often fail. The most common answer is from a lack of preparation and management. Many IT organizations just think about the technology (product) implementation and fail to understand the risks of not planning for the effective and efficient use of the four Ps: People, Process, Products (services, technology and tools) and Partners (suppliers, manufacturers and vendors). A holistic approach should be adopted for all Service Design aspects and areas to ensure consistency and integration within all activities and processes across the entire IT environment, providing end to end business-related functionality and quality. (SD 2.4.2) People:   Have to have proper skills and possess the necessary competencies in order to get involved in the provision of IT services. The right skills, the right knowledge, the right level of experience must be kept current and aligned to the business needs. Products:   These are the technology managem

What Is A Service Offering?

The ITIL4 Best Practice Guidance defines a “Service Offering” as a description of one or more services designed to address the needs of a target customer or group .   As a service provider, we can’t stop there!   We must know what the contracts of our service offering are and be able to put them into context as required by the customer.     Let’s explore the three elements that comprise a Service Offering. A “Service Offering” may include:     Goods, Access to Resources, and Service Actions Goods – When we think of “Goods” within a service offering these are the items where ownership is transferred to the consumer and the consumer takes responsibility for the future use of these goods.   Example of goods that are being provided in the offering – If this is a hotel service than toiletries or chocolates are yours to take with you.   You the consumer own these and they are yours to take with you.               Note: Goods may not always be provided for every Service

What is the difference between Process Owner, Process Manager and Process Practitioner?

I was recently asked to clarify the roles of the Process Owner, Process Manager and Process Practitioner and wanted to share this with you. Roles and Responsibilities: Process Owner – this individual is “Accountable” for the process. They are the goto person and represent this process across the entire organization. They will ensure that the process is clearly defined, designed and documented. They will ensure that the process has a set of Policies for governance. Example: The process owner for Incident management will ensure that all of the activities to Identify, Record, Categorize, Investigate, … all the way to closing the incident are defined and documented with clearly defined roles, responsibilities, handoffs, and deliverables.  An example of a policy in could be… “All Incidents must be logged”. Policies are rules that govern the process. Process Owner ensures that all Process activities, (what to do), Procedures (details on how to perform the activity) and th