Skill selection recall
Target operating model — recommended contracts, not installed collection. All owner and cadence entries below are proposals to confirm before the pilot, not assigned responsibilities or approved schedules.
Prototype calculation — no live telemetry. The synthetic example calculates this metric; it has no network collectors.
Relevant task-trials in which the intended skill activates.
| Contract | Definition |
|---|---|
| Numerator | True-positive activations |
| Denominator | True-positive plus false-negative task-trials |
| Expression | N / D * 100 |
| Aggregation | percent |
| Unit | % |
| Eligible population | Independently labeled relevant evaluation task-trials for a skill/client tuple. |
| Exclusions | Irrelevant tasks; missing relevance or activation reported as unscorable, not negative. |
| Events and source | eval.selection; versioned labeled corpus and runner |
| Owner / cadence | Data & AI / Each skill/client qualification |
| Missing evidence | Report evaluation coverage; null if no relevant labeled tasks. |
| Gaming and limits | Overtrigger to maximize recall; always pair with precision and negative cases. Descriptive signal, not causal AI productivity attribution. Source completeness, task mix and sampling limit inference; report uncertainty. Loading the right skill is not adherence. |
Read the shared collection and safeguards before collecting data; see the synthetic example and limits.
Return to Skills Measurement.
Use this labeled-case result alongside precision in Quality's candidate evaluation. Finding expected skills does not establish useful artifacts or safe execution. Preserve missing-case coverage and revision-regression evidence before a bounded pilot; do not pool evaluation cases with live Delivery tasks.