Team Performance

[kinlane]

This is a discussion to move forward with the blueprint to define team performance, building upon existing metrics like DORA metrics to help define what team performance is, but also begin to measure, report, and begin to steer things in the desirable direction.

Reference: DORA metrics are used by DevOps teams to measure their performance and find out whether they are “low performers” to “elite performers”. The four metrics used are deployment frequency (DF), lead time for changes (LT), mean time to recovery (MTTR), and change failure rate (CFR).

• Deployment Frequency: Refers to the frequency of successful software releases to production.
• Lead Time for Changes: Captures the time between a code change commit and its deployable state.
• Mean Time to Recovery: Measures the time between an interruption due to deployment or system failure and full recovery.
• Change Failure Rate: Indicates how often a team’s changes or hotfixes lead to failures after the code has been deployed.

What is the consideration when mapping this to a platform approach to the API lifecycle, thinking about the specifics of how organizations are producing APIs, as opposed to other types of software?

[kinlane]

Could use some help mapping these standards metrics @kevinswiber @meenakshi-dhanani @arno-di-loreto -- leave all random thoughts here.

[kinlane]

Additional questions to consider:

• What metrics should we be recommending they track?
• What are we seeing in the industry?
• On a more tactical level, how can we track improvements against said metrics with Postman?

[kinlane]

Current blueprint deck (available on README of this repo too)

[kinlane]

Charity Majors’ Recipe for High-Performing Teams

[kevinswiber]

A 5th metric was added in the 2021 DORA report:

• Reliability: the degree to which a team can keep promises and assertions about the software they operate.

https://cloud.google.com/blog/products/devops-sre/announcing-dora-2021-accelerate-state-of-devops-report

[kevinswiber]

Another significant report regarding how to measure team performance is The SPACE of Developer Productivity by Dr. Nicole Forsgren, et al.

• Satisfaction & Well-Being: How fulfilled, happy, and healthy one is.
• Performance: An outcome of a process.
• Activity: The count of actions or outputs.
• Communication & Collaboration: How people talk and work together.
• Efficiency & Flow: Doing work with minimal delays or interruptions.

[kinlane]

So if we were take this list of elements/metrics considered in team performance:

• Deployment Frequency
• Lead Time for Changes
• Mean Time to Recovery
• Change Failure Rate
• Reliability
• Satisfaction & Well-Being
• Performance
• Activity
• Communication & Collaboration
• Efficiency & Flow

How do we map to a platform approach supported by multiple vendors:

• Deployment Frequency (ie. JIra, GitHub, Jenkins)
• Lead Time for Changes (ie. Jira)
• Mean Time to Recovery (ie. Postman)
• Change Failure Rate (ie. Github, Jenkins)
• Reliability (Postman)
• Satisfaction & Well-Being (?? Surveys?)
• Performance
• Activity
• Communication & Collaboration (ie. Github, Slack)
• Efficiency & Flow (????)

Then how do we provide coverage across this spectrum with PlatofrmOps collections + native integrations?

[arno-di-loreto]

Starting a Twitter thread with my random thoughts inspired by this question https://twitter.com/apihandyman/status/1525091655866540032?s=20&t=X27AWmy8DHFKI8hZTCHQHQ

[kevinswiber]

Focusing on a vendor map might bury the lede.

Team performance starts with culture. Culture informs process. Process requires iterative improvement. There are many formulas for iterative improvement, and yet a high level, most of them go a little like this:

• Build metrics from observations.
• Identify an opportunity.
• Propose a hypothesis with a stated goal.
• Run an experiment.
• Gather new observations.
• Repeat.

A platform approach has a number of benefits for high performing teams. One area where it shines is in centralizing key processes and making them observable.

Back to tooling, can we actually pull the observations we need to ascertain current maturity and ensure meaningful results from running experiments?

• How long does it take to go from git push to production? (Lead time for changes)
• What's the average time to first API call? (Efficiency and flow)
• What's the maximum request latency during peak hours? (Reliability)

As an organization, while tools like GitHub, Slack, and Postman feel extremely effective at improving team performance, how do we access the information we need to know they're effective?

[kinlane]

I like it. Agreed. Helps me stay high level on this. II am just looking for one or two layers deep right now, not the actual implementations until we know more. Have a scaffolding to think about. Start with the people, but then also the enablement through tooling. Gonna process Arnaud's tweets too, and think about it some more. Thanks, y'all!!