Measure the journey to the first production call
Platform metrics are more useful when they cover how quickly a new team gets started, not only how the platform performs once it's running.
The idea in brief
Operational metrics such as uptime and latency describe a platform once teams are using it. Onboarding metrics describe how hard it is to get there. Two public sources support measuring both: the CNCF white paper on platforms and DORA's research on software delivery and platform engineering.
What the CNCF white paper suggests
The CNCF Platforms White Paper, from the TAG App Delivery Platforms Working Group, groups platform success metrics into three categories:
| Category | Example measures in the paper |
|---|---|
| User satisfaction and productivity | Active users and retention, satisfaction surveys such as NPS, SPACE framework measures |
| Organisational efficiency | Time to fulfil a request, time to build and deploy a new service, time for a new user to submit their first code change |
| Product and feature delivery | The DORA metrics: deployment frequency, lead time for changes, time to restore service, change failure rate |
The paper also suggests that simple surveys and usage analysis are a reasonable place to start.
How the scorecard maps to these sources
Each item in this scorecard corresponds to a measure in one of the public frameworks:
| Scorecard item | Closest published measure | Source |
|---|---|---|
| Time to first production call | Onboarding speed; time to build and deploy a new service | CNCF |
| Onboarding without a support ticket | Developer independence | DORA 2024 |
| Deployment lead time | Lead time for changes | DORA |
| Failed deployment rate | Change failure rate | DORA |
| Time to diagnose a blocked request | Time to fulfil a request | CNCF |
| Weekly active projects | Active users and retention | CNCF |
| Developer confidence | Satisfaction surveys | CNCF |
What DORA's 2024 research found
DORA's 2024 report included a chapter on platform engineering. Among its findings:
- Respondents using an internal developer platform reported higher individual productivity and team performance.
- The same group showed lower software delivery throughput (about 8%) and change stability (about 14%), which the report described as unexpected.
- Developer independence, being able to complete work without waiting on an enabling team, was associated with about a 5% productivity improvement.
- The infographic notes that the downsides were more pronounced when the platform was mandated for the whole application lifecycle.
These results point towards measuring how independently teams can work, alongside delivery speed and stability.
Limitations
DORA's findings are correlational.
They show associations between platform use and outcomes, not causes.
The mapping is approximate.
The scorecard items are close to the published measures, not identical to them.
Small samples are noisy.
When only a few teams onboard each quarter, individual journeys say more than percentages.
Any metric can be gamed.
Pairing speed measures with stability and satisfaction measures, as DORA and CNCF both do, reduces that risk.
Sources
Cite this post
Adam, Ammar. (Oct 2026). Measure the journey to the first production call. www.ammaradam.dev. https://www.ammaradam.dev/posts/time-to-first-production-call/.
BibTeX
@article{adam2026timetofirstproductioncall,
title = {Measure the journey to the first production call},
author = {Adam, Ammar},
journal = {www.ammaradam.dev},
year = {2026},
month = {Oct},
url = {https://www.ammaradam.dev/posts/time-to-first-production-call/}
} 
Comments
Sign in with GitHub to like this post or join the discussion.