Measurement quality

Measured attainment and the quality of its evidence answer different questions.

• Nines keeps the SLI visible while explaining coverage, freshness, eligible traffic and measurement-contract verification.
• It does not assign a statistical confidence score.

Read the evidence

• Open an SLO to see Measurement quality.
  • The report includes the evaluated cutoff, definition revision, source fingerprint, observed event count and configured count requirement.
  • The same report is available through the Go client, slo_quality MCP tool and:

nines quality api --expected-revision 3

• This reads saved evidence and makes no provider query. Its states describe quantitative evidence:

• Missing data and zero traffic never establish success.
  • Counts and coverage do not establish statistical independence, representative sampling or statistical confidence.
  • Custom queries and imported counts do not receive automatic structured metadata verification.
  • A service-side observation cannot prove that requests never reaching the service succeeded.
• Collection evidence includes an opaque source fingerprint and the contiguous time span verified from that source.
  • Route or credential identity changes invalidate old lineage.
  • For interval-count measurements, backfill the complete SLO window or wait for full fresh coverage before a budget gate can proceed.
  • Whole-window collection verifies its requested full windows directly.
  • Manual imports into a queried SLO invalidate collector lineage until re-collected.
  • A failed query never stamps old samples with a new source identity.

Set requirements in the SLO

Add this optional section alongside measurement, target and policies in an existing SLO. The portal exposes the same fields under Measurement quality requirements.

spec:
  quality:
    minEvents: 100
    maxSampleAgeSeconds: 180
    cohortLabels: [region, customer_tier]

• minEvents is a minimum eligible-event count for the rolling SLO window and an annotation on low-volume cohort rows.
  • It is separate from a burn policy's minEvents, which gates both alert windows.
  • Choose counts for the service; these example values are not universal defaults.
• An omitted or zero sample age uses twice the measurement interval plus the collector scheduling allowance when applicable.
  • An explicit age must be at least stepSeconds and no more than 31 days.
  • These requirements create a reviewed configuration revision without reinterpreting or discarding existing samples.
  • They also pass through intent compilation and Kubernetes CRDs.

Inspect cohorts

Configure one to three metric labels, then choose Inspect cohorts or run:

nines quality api --expected-revision 3 --cohorts

• This performs an explicit read against the configured provider using the SLO's population, rolling window and completed cutoff. Prometheus and Mimir support provider-window measurements. Interval custom queries and providers without the optional cohort capability report unsupported.
• The report lists observed attainment and event counts for each group, ordered with missing numerators first and then lowest attainment.
  • A healthy aggregate can therefore coexist with a failing region or user tier.
  • Missing good-event series remain unknown; they are never zero-filled.
  • Missing label values appear as an explicit missing-value group.
• Results are limited to 100 cohorts.
  • Higher cardinality is rejected as a complete inspection rather than presenting a partial worst-cohort ranking. Requests are time and response-size bounded.
  • Definition or source changes during the query invalidate the response.
  • No samples, definitions, rules or alert states are written.
• Cohort estimates have no independently verified coverage or confidence interval.
  • The report separately states whether their good/total sums reconcile with the saved aggregate.
  • A mismatch can reflect missing numerator series, collection timing, late data or instrumentation semantics. Review it before drawing conclusions.
• Cohort analysis is diagnostic.
  • It does not silently introduce per-cohort targets or modify aggregate SLI arithmetic, burn alerts or budget decisions.
  • Define explicit SLOs for populations that need separate operational guarantees, and link them to the relevant critical user journey.

APIs

• GET /api/slos/{id}/quality reads stored evidence.
  • Add cohorts=true for explicit provider inspection and expectedRevision=N to reject stale definition reads.
  • Existing tenant authorization applies; viewers and agents can read their tenant's evidence. Source credentials remain server-side.
• See error-budget policies for operational decisions built on these checks, and provider adapters for extension boundaries.