Skill selection precision
Target operating model — recommended contracts, not installed collection. All owner and cadence entries below are proposals to confirm before the pilot, not assigned responsibilities or approved schedules.
Prototype calculation — no live telemetry. The synthetic example calculates this metric; it has no network collectors.
Correct activations among activations on an independently labeled evaluation corpus.
| Contract | Definition |
|---|---|
| Numerator | True-positive activations for the requested workflow |
| Denominator | True-positive plus false-positive activations |
| Expression | N / D * 100 |
| Aggregation | percent |
| Unit | % |
| Eligible population | All candidate/client labeled evaluation task-trials with known activation and relevance. |
| Exclusions | Unlabeled/unknown activations excluded with coverage; no cherry-picked failed trials. |
| Events and source | eval.selection; versioned labeled corpus and runner |
| Owner / cadence | Data & AI / Each skill/client qualification |
| Missing evidence | Unknown activation/relevance is unscorable; publish missing count and deny qualification if planned coverage incomplete. |
| Gaming and limits | Choose easy labels or hide negative cases; freeze corpus and independently review labels. Descriptive signal, not causal AI productivity attribution. Source completeness, task mix and sampling limit inference; report uncertainty. Does not show execution success; precision depends on corpus prevalence. |
Read the shared collection and safeguards before collecting data; see the synthetic example and limits.
Return to Skills Measurement.
Use this labeled-case result in Quality's candidate evaluation. Selecting the right skill is only one input: independently test artifact fitness, safe execution and revision regression before the owning approver considers a bounded pilot. It is not the live Delivery workflow's accepted-task rate.