Measuring real output
Start with how current AI-assisted work performs; compare relevant outcomes after the selected intervention, whether context, coaching, tooling or skills. Separately assess skill fitness and health only where skills exist. Registry inclusion does not require a metric to be used, collected or baselined. More adoption does not prove improved work.
Target operating model — candidate definitions, not installed telemetry or a collection plan. Proposed owners and cadences in the contracts apply only if selected and agreed for a workflow.
Read a candidate contract
For the EXP-01 Delivery hypothesis, start with three questions, not all 16 contracts. Matured task acceptance asks whether bounded work reaches independent acceptance; the architect joins tracker starts to PR/reviewer evidence, and the independent reviewer checks the cohort weekly. Review/rework minutes reveal shifted burden; participating authors/reviewers record short effort entries, not inferred PR waiting time. Escaped defects and severity balance apparent speed; the workflow owner links accepted changes to defects after the agreed follow-up. Agree these proposed responsibilities and lawful access in month 2. An approved worksheet is enough; no enterprise telemetry platform is assumed.
Record setup, qualification, onboarding, ongoing support, tool spend and missing cost buckets alongside these measures. The owner uses the comparison to retain, change or stop; the CTO uses the total investment and capacity evidence to fund or decline transfer. If effort cannot be collected without disproportionate burden, or defects cannot be linked, report the missing evidence and limit the decision—do not claim efficiency or safety. Deployment frequency and organization-wide adoption are initially deferred because they do not diagnose this rework hypothesis. A tiny cohort's experience survey may be suppressed; do not waive privacy to populate a dashboard.
Where this misleads: accepted counts change with task mix, staffing and unfinished work; a fast PR may merely shift checking to reviewers; defects mature later; volunteers and coached tasks are not representative of every team. Keep failed, abandoned and unresolved matured starts, coverage, complexity slices, client/model changes and concurrent process changes visible. Compare the same eligibility, acceptance and follow-up rules before/after; use matched or randomized allocation only when feasible and ethical. Report uncertainty, not a causal ROI claim. Invocation counts, generated lines and individual rankings are not productivity measures. Skill benchmark success does not establish real workflow improvement.
Each entry explains meaning and applicability, calculation, required inputs and evidence, exclusions, missing-evidence behavior, limitations and implementation status. The two groups below classify every contract by its primary question; conditional reuse never merges populations. A delivery metric need not fit a non-delivery workflow.
To select a relevant subset and establish actual starting evidence, use Baseline. Pilot application belongs to Adoption; ongoing interpretation belongs to Maintenance.
In the illustrative Delivery workflow from Stage 01, incomplete regression evidence is the improvement hypothesis. The workflow owner and independent Quality Engineering reviewer select contracts that can test it, then carry the same definitions into the baseline and pilot comparison. They can leave unrelated metrics unselected. This reference capability supports the whole journey; opening this page is not an execution gate.
Metric definitions
Each contract is a prospective definition, not a rollout obligation. No live source or observed baseline is established. N and D refer to the numerator and denominator shown for that metric.
Implementation status—not live telemetry
There are 16 contracts: 9 prototype calculations and 7 unimplemented metrics. The prototype has no network collectors. None of these counts establishes live telemetry.
The nine calculations cover task success, selection precision and recall, change fail rate, deployment rework, escaped quality, policy effects, cost and the primary human-experience item. The seven omissions are supported bundle coverage, approved-workflow adoption, change lead time, deployment frequency, failed deployment recovery time, substantive-review coverage and workflow-specific business acceptance.
Workflow effectiveness
Is the work getting better? Baseline relevant current-workflow outcomes even before standardized skills exist, then compare the same outcomes during adoption and maintenance. Use comparable task mix, acceptance rules, follow-up periods and source coverage. Before/after trends alone are not causal evidence of a skills benefit.
- Validated task success — Prototype calculation. Independently accepted outcomes among matured workflow starts; failures, abandonment and unresolved outcomes remain in the denominator.
- Change lead time — Unimplemented metric. Delivery time for production changes in one service, not general task duration.
- Deployment frequency — Unimplemented metric. Production delivery frequency for one service; frequency alone is not value.
- Change fail rate — Prototype calculation. Production deployments causing failures; requires joined change/incident evidence.
- Failed deployment recovery time — Unimplemented metric. Recovery from failed production deployments; conditional on eligible incidents.
- Deployment rework rate — Prototype calculation. Unplanned corrective production deployments, not all forms of task rework.
- Changes with escaped quality issues — Prototype calculation. Accepted changes with confirmed downstream issues after follow-up.
- Confirmed policy effects — Prototype calculation. Safety of supported workflow attempts; governed skill trials are a separate evaluation population, never pooled with live work.
- Cost per accepted outcome — Prototype calculation. Cost of the matured workflow cohort, including unsuccessful attempts and recorded review/rework effort. Missing cost buckets remain partial.
- Verification confidence and cognitive load — Prototype calculation. Anonymous cohort experience: confidence, workload and usefulness separately, not measured time savings.
- Workflow-specific business acceptance — Unimplemented metric. Acceptance against an agreed departmental rubric; requires a defined workflow and independent evidence.
Human effort is recorded as review/rework inputs where the cost contract supports it; the experience survey describes perception. Neither elapsed delivery time nor token counts establish effort saved. If the current workflow lacks the event semantics required by a contract (for example, governed attempts), record it as not applicable or unavailable with a reason; do not relabel unrelated records to manufacture a baseline.
Skills effectiveness & health
Are the skills fit for use and kept fit? Qualify named skills and versions, observe supported use during adoption, and review or retire them in maintenance. Without applicable skills, bundles or enrolled use, skill-specific measures may be N/A, not zero. Once a population exists, missing observations are unknown, not evidence of non-use or good health.
Fitness comes from Quality's evaluation evidence, not health proxies. Compare named candidate and approved revisions against declared cases, independent artifact acceptance and mandatory safety checks. Retain regression results, failed trials and coverage limits. The five registry contracts below complement that report: selection tests relevance; support, use and review measures describe operating health. None replaces evidence that a skill produces useful, safe artifacts. Maintenance sends real failures and revision changes back to Quality for requalification.
- Skill selection precision — Prototype calculation. Did the evaluated selection choose an applicable skill? Requires labeled qualification cases.
- Skill selection recall — Prototype calculation. Were expected skills selected? Requires the same labeled qualification population.
- Supported bundle coverage — Unimplemented metric. Are expected active installations on supported, non-revoked bundles/adapters?
- Approved-workflow adoption — Unimplemented metric. Is the enrolled cohort using approved skills in eligible workflows? Installation/loading is not use.
- Skills within substantive review window — Unimplemented metric. Do active approved revisions have current owners and substantive review evidence?
Task-success population matters. The canonical validated-task-success contract above measures workflow outcomes, not an evaluation benchmark. A named skill/version can be examined as a predeclared live-workflow slice under that same contract, without counting retries or multiple skills as extra tasks or attributing causality. Sandbox/evaluation cases remain excluded: a named skill/version evaluation must be reported separately with its own declared cases, acceptance evidence and limits. It is not an additional implemented calculation or a second conflicting task-success formula. The canonical page explains this boundary. Likewise, keep policy-effect evaluation trials separate from live attempts.
Use both tracks together to decide whether to improve, retain or retire a skill. Higher selection scores, wider adoption or current reviews cannot substitute for accepted work, safety, cost and human experience.
Read Collection and safeguards for shared interpretation, event and privacy rules. Run the synthetic measurement example for reproducible arithmetic and limits.