Service dependencies

A service dependency map links callers to the services they depend on and associates each service with its SLOs.

• Use it to review reliability risks across a critical user journey.
• A map describes relationships; it does not prove causality or calculate end-to-end journey availability.

Create and edit a map

• Open Service dependencies → Define dependency map.
  • Set the map identifier, project and display name.
  • The editor has three views of the same draft:

• Visual editor: add services, drag their title handles to arrange them, and drag an arrow handle onto another service to draw a caller-to-dependency link. Select a service to edit its name, display name and SLO references. Mark a dependency critical only after reviewing its effect on the caller.
• YAML: edit the native ServiceDependencyMap resource. Valid changes appear in the visual editor.
• Discover from Kubernetes: import a local resource export or preview inventory from a configured cluster, then add the candidates to the draft.
• Keyboard users can focus a service's move handle and press arrow keys to reposition it.
  • The caller/dependency form creates the same links without dragging.
  • Removing a service removes its draft links. Reset layout returns to automatic positioning.
  • Saved positions belong to the map's revision and do not alter dependency semantics.
• Select Validate and plan, inspect the revision and previous definition, then Apply reviewed definition. Every edit invalidates the plan.
  • Owners and editors can apply; viewers and agent credentials can preview and plan. Concurrent changes require a fresh plan.
  • Nothing is saved or deployed by moving a node or previewing discovery.
• SLO references must identify existing SLOs in the current tenant, with the same service identity as the node.
  • Renaming a node can therefore require updating its SLO references.
  • Discovery merges retain existing SLO links, criticality, manual edges, positions and observed-source settings.

Import Kubernetes resources

Import works without granting the Nines server any Kubernetes access. Export only the resources needed for the map:

kubectl get services,deployments,statefulsets,daemonsets,ingresses \
  -n commerce -o yaml > commerce-resources.yaml

• Choose Discover from Kubernetes → Local resource YAML or JSON, open that file, select Preview discovery, review the evidence and warnings, then Add discovery to draft.
  • The browser projects the export locally before sending a bounded inventory to Nines.
  • Only resource identities, selectors, labels, supported service annotations, host names and backend names enter that request.
  • Full URLs, user information, paths, unrelated environment values, image names, commands, status and Secret/ConfigMap references are discarded.
  • Treat the original export as sensitive and keep it out of version control.

The CLI performs the same projection locally and can run entirely offline:

nines dependencies import-kubernetes --offline \
  --file commerce-resources.yaml --namespace commerce \
  --name commerce-services --project production \
  --out commerce-services.yaml
nines validate --offline -f commerce-services.yaml
# After inspecting the candidate map, use the intended tenant context:
nines plan -f commerce-services.yaml --context production --agent
nines apply -f commerce-services.yaml --expected-revision 0 --context production --agent

• Omit --out to receive a JSON preview with the manifest, inventory digest, evidence and warnings.
  • Output files are created exclusively and never overwritten.
  • Agent credentials cannot bypass server-side write permissions.
  • A namespace flag supplies the default for resources with no namespace; explicitly namespaced resources retain their identities, including cross-namespace references in a multi-namespace export.
• Supported imports include Services, Deployments, StatefulSets, DaemonSets, Jobs, CronJobs, Pods and Ingress resources in a YAML stream, JSON document or Kubernetes List.
  • The default identities are namespace/service-name for Services and namespace/kind/name for unmatched workloads and Ingress.
  • Long workload identities receive a deterministic hash suffix to fit the 200-character map limit.

What discovery infers

For logical names that already appear in SLOs, annotate the Service or workload:

metadata:
  annotations:
    nines.dev/service: api
    nines.dev/depends-on: database.commerce.svc.cluster.local, commerce/cache

• Pod-template annotations override workload annotations.
  • Dependency annotations accept comma-separated service identities or Kubernetes service DNS names.
  • A workload matching multiple different Service identities is skipped with a warning unless its identity is explicit. All candidate links default to noncritical.
  • NetworkPolicy permissions, co-location in a namespace and shared labels do not establish calls.
  • External hosts, variable substitution, Secrets, ConfigMaps, IP addresses and mesh configuration are not resolved.
  • Custom cluster DNS suffixes require explicit dependency annotations.
• The preview is limited to 2 MiB and 1,000 resources; the resulting map supports 40 services and 100 directed links.
  • Evidence is capped at 200 records with a truncation warning.
  • Narrow larger exports instead of accepting a partial map silently.
  • Discovery runs on demand and does not continuously reconcile a saved map.

Enable connected-cluster discovery

• This optional reader runs inside the Nines server's Kubernetes cluster.
  • It uses the mounted service-account CA and rotating token, verifies API TLS and refuses redirects.
  • It does not use kubeconfig or accept a cluster URL or credentials from the browser.
  • Local Compose and -demo mode use resource imports instead.

Add an explicit namespace allowlist to each authorized tenant's server-owned configuration:

tenants:
  - id: commerce
    name: Commerce
    access:
      - tokenEnv: COMMERCE_API_TOKEN
        role: owner
    source:
      kind: prometheus
      url: http://prometheus.monitoring.svc:9090
    kubernetesDiscovery:
      namespaces: [commerce]

• Create a dedicated ServiceAccount and a Role/RoleBinding for each allowed application namespace.
  • The reference discovery RBAC manifest grants list only on Services, Deployments, StatefulSets, DaemonSets and Ingress.
  • Adapt its nines installation namespace, commerce application namespace and ServiceAccount name before applying it.
  • It grants no Secret, ConfigMap, Pod, exec, write or cluster-wide permissions.
  • Kubernetes list responses still include workload configuration and literal environment values, so only enable namespaces whose configuration may be processed by this Nines installation.

In the Nines Deployment, explicitly set:

spec:
  template:
    spec:
      serviceAccountName: nines-discovery
      automountServiceAccountToken: true

• The normal deployment disables token mounting.
  • Keep it disabled when discovery is not used.
  • Mount the trusted tenant file, set TENANTS_FILE to its path, restart the single Nines replica, then choose Connected Kubernetes cluster in the discovery tab.
  • Network policy must permit the Kubernetes API. See Kubernetes deployment.

nines dependencies discover --namespace commerce \
  --name commerce-services --project production \
  --context production --agent

• The CLI asks Nines to read only that tenant's configured namespace and returns a preview; it does not save a map. Requests are time-bounded, paginated and size-limited.
  • An unavailable API, missing RoleBinding or forbidden namespace produces an error, not an empty successful discovery.
  • Tenants sharing a namespace can see the same sanitized inventory; use distinct namespaces where inventory isolation is required.
• These permissions follow Kubernetes' namespace-scoped RBAC guidance.
  • Run make test-dependencies to verify the reader against a disposable Kubernetes API with the shipped Role, verified TLS, tenant boundaries, revision checks and permission revocation.
  • The fixture does not load your kubeconfig or schedule workloads.

Observed dependencies

• Observed service-graph metrics can complement the reviewed map through its existing spec.observed source, labels and window.
  • Declared edges use solid lines and observed edges use dashed lines.
  • Configuration discovery never relabels a candidate as observed traffic.
  • Missing trace coverage or an absent edge cannot establish that a service is independent or healthy.

Read observed evidence

• Each observed link records the configured telemetry connection, a nonsecret connection fingerprint, the counter name, query window and read time. Fresh sample refers to the latest actual counter sample inspected inside that window, rather than the timestamp of an instant query result. Stale sample, Sample age unverified, Source unavailable and Not seen in window describe different gaps and remain visible in the evidence table. Expand Evidence details to inspect the timestamps and limits.
• Configure scrapeIntervalSeconds to match the exporter scrape cadence and maxSampleAgeSeconds to set the tolerated sample age.
  • Defaults are 15 seconds and three scrape intervals (at least 30 seconds).
  • Sample continuity compares observed samples with the expected samples of the series present in the window.
  • It does not measure instrumentation or trace-sampling coverage.
  • Those remain unknown: Nines cannot know about services that were never instrumented.
  • A counter can be fresh while reporting no new calls; freshness alone does not establish active traffic.
• Declared and observed directed links are matched by caller and dependency.
  • A declared link absent from the selected window remains declared; an observed-only link suggests a relationship to review. Kubernetes candidates retain their configuration provenance.
  • Renamed services appear as different identities until the reviewed map is updated.
  • Stale or age-unverified observed links are excluded from inferred upstream impact. Existing declared critical links still participate.
  • Impact identifies possible exposure; linked SLOs determine health.
• Queries return at most 100 observed edges and views at most 80 nodes.
  • Metadata is bounded independently, uses an inspection resolution of at least one scrape interval and at most 240 steps, and reports missing metadata as unverified. Truncation is explicit.
  • Narrow the labels or window when necessary.
  • Journey links show the affected service steps where present, with bounded details; open the journey for its full definition.

Run an instrumented example

• The optional topology profile adds three small HTTP services and a traffic generator.
  • Each service exposes request counters; outbound HTTP wrappers expose traces_service_graph_request_total. This example measures real calls directly.
  • It does not run a trace processor or claim tracing coverage.
  • Production deployments can supply the same service-graph counter through a tracing pipeline such as Tempo's metrics generator.

# Prepare .env and its generated secrets as described in the playground guide.
./playground up --topology

nines apply --server http://localhost:18080 --tenant prometheus \
  -f examples/service-graph/gateway.slo.yaml --expected-revision 0
nines apply --server http://localhost:18080 --tenant prometheus \
  -f examples/service-graph/checkout.slo.yaml --expected-revision 0
nines apply --server http://localhost:18080 --tenant prometheus \
  -f examples/service-graph/payments.slo.yaml --expected-revision 0
nines apply --server http://localhost:18080 --tenant prometheus \
  -f examples/service-graph/journey.yaml --expected-revision 0
nines apply --server http://localhost:18080 --tenant prometheus \
  -f examples/service-graph/dependencies.yaml --expected-revision 0

• Use your configured tenant context or NINES_API_KEY; do not put API keys in shell arguments.
  • For Mimir, add --mimir when starting the playground.
  • The existing Prometheus remote-write route forwards the example counters to Mimir organization playground.
  • Use --tenant mimir when applying the examples to the Mimir workspace; tenant identifiers and provider organization IDs are separate.
  • Open Service dependencies → Checkout services after at least two scrapes.
  • The two observed links should match their declared links.
  • Linked SLOs need enough retained samples before reporting complete-window attainment.
• make test-topology runs a separate disposable fixture: real HTTP traffic, real Prometheus scraping, journey-step links, tenant isolation, then a paused exporter to verify stale evidence. make test-operator additionally checks cleanup across provider outage and credential rotation, plus independent critical/warning pending, firing and recovery transitions using the actual operator-generated rules on Prometheus and Mimir. The agentless fixture bridges the Prometheus Operator's selected ConfigMap to a disposable evaluator; it does not schedule a Prometheus Pod.