Skip to main content

DevOps - Measuring Success

While you might not be able to get your head around how to measure DevOps when it is defined as a cultural and professional movement, there are some key factors to consider that will help us to measure the success of shifting to a collaborative and integrated team throughout the value stream that includes Dev and Ops.

It is clear that DevOps goes beyond the integration of the development and operation staff but includes breaking down those silos between all parties and stakeholders including the Business, external partners, and 3rd party suppliers and IT.  If you think it is difficult to get your own internal teams to play in the same sandbox consider the challenges when 3rd party vendors and suppliers are then thrown into the mix. Being able to demonstrate proof that DevOps practices benefit the organization requires examining factors that influence overall performance.  Practices that enable your organization to improve the flow of work between the Business and Operations enables improved IT performance.

Important Factors

IT Performance in measured in terms of throughput and stability.  
  • Throughput is measured by deployment frequency and lead time for change
  • Stability is measured by mean time to recover and the ability to preemptively detect and mitigate problems

Simply put, you cannot measure Development outcomes separately from Operational outcomes. 

Key Metrics

Showing proof that DevOps practices benefit the organization also requires baselining key metrics before and after making improvements and can include:
  • Deployment Frequency
  • Change Lead Time – Time from request to delivery
  • Cycle Time – Time from start of work to ready for delivery to customer
  • Change Failure Rate – Backed out, Rescheduled or unforeseen impact/incidents
  • MTTD – Mean time to detect incidents
  • MTTR - Mean Time to Recover or repair - Component
  • MTRS - Mean Time to Restore - Service

Change lead time is one of the most important metrics as it represents what the customer sees.  Cycle time is a more mechanical measure of process capability. Lead time depends on cycle time, but also depends on your willingness to keep a backlog, the customer’s patience, and the customer’s readiness for delivery.  MTTR is measured from when a component fails until it is repaired. MTTR does not include the time required to recover or restore service. e.g., data may need to be recovered before service is fully restored and delivering its normal functionality.



Comments

Popular posts from this blog

What is the difference between Process Owner, Process Manager and Process Practitioner?

I was recently asked to clarify the roles of the Process Owner, Process Manager and Process Practitioner and wanted to share this with you.

Roles and Responsibilities:
Process Owner – this individual is “Accountable” for the process. They are the goto person and represent this process across the entire organization. They will ensure that the process is clearly defined, designed and documented. They will ensure that the process has a set of Policies for governance.Example: The process owner for Incident management will ensure that all of the activities to Identify, Record, Categorize, Investigate, … all the way to closing the incident are defined and documented with clearly defined roles, responsibilities, handoffs, and deliverables. An example of a policy in could be… “All Incidents must be logged”. Policies are rules that govern the process. Process Owner ensures that all Process activities, (what to do), Procedures (details on how to perform the activity) and the policies (r…

How Does ITIL Help in the Management of the SDLC?

I was recently asked how ITIL helps in the management of the SDLC (Software Development Lifecycle).  Simply put... SDLC is a Lifecycle approach to produce the software or the "product".  ITIL is a Lifecycle approach that focuses on the "service".
I’ll start by reviewing both SDLC and ITIL Lifecycles and then summarize:
SDLC  -  The intent of an SDLC process is to help produce a product that is cost-efficient, effective and of high quality. Once an application is created, the SDLC maps the proper deployment of the software into the live environment. The SDLC methodology usually contains the following stages: Analysis (requirements and design), construction, testing, release and maintenance.  The focus here is on the Software.  Most organizations will use an Agile or Waterfall approach to implement the software through the Software Development Lifecycle.
ITIL  -  is a best practice for IT service management (ITSM) that focuses on aligning IT services with the needs …

Incidents when a Defect is Involved

Question: We currently track defects in a separate system than our ticket management system. With that said, my question is does anyone have suggestions and/or best practices on how to handle incidents when a defect is involved? Should the incident be closed since the defect is being worked on in another defect tracking system if it is noted in the incident ticket? I am considering creating an incident statuses of 'closed-unresolved' so the incident can still be reported on in our ticket management system but know it is being worked on/tracked in the defect system. With defects, it is possible that we may never work on them because they are very low priority and the impact is low to the user. However, in some cases a defect is being worked on. Should we create a problem ticket instead?
Thanks, René W.

Answer: RenĂ©. In ITIL, the activity you are describing is handled by the Problem Management process. ITIL does not use the term “defect” but it does use the term “known error” to…