Observability

The pipeline

The API emits all three OpenTelemetry signals to an OpenTelemetry Collector, which fans them out to dedicated backends, all explored through Grafana:

.NET API

Serilog + OTel SDK

OTel Collector

receives · routes

Traces Tempo

Metrics Prometheus

Logs Loki

Grafana

dashboards · alerts

Signal	Backend
Traces	Tempo
Metrics	Prometheus
Logs	Loki

The collector, the backends and Grafana are all in docker-compose.yml, with Grafana dashboards provisioned from deploy/. Open Grafana at localhost:3010 once the stack is up.

Logs and audit

Application logs go through Serilog and out over OTLP to Loki. Audit events use the same pipeline rather than a separate table, so audit retention is a Loki configuration, and the API stays free of purge jobs.

Tracing a request

Because the whole request path is instrumented, a single browser action produces one trace spanning the endpoint, the Wolverine handler, the database call and any published messages. When something is slow, open the trace, find the expensive span, and jump to its correlated logs.

Instrumenting your own slice

Most of the time the built-in instrumentation is enough. When a slice does something worth measuring, add a span or a metric explicitly:

private static readonly ActivitySource Activity = new("Slicekit.Search");
private static readonly Counter<long> Rebuilds =
    new Meter("Slicekit.Search").CreateCounter<long>("search.index.rebuilds");

using var span = Activity.StartActivity("rebuild-search-index");
span?.SetTag("project.id", projectId);
Rebuilds.Add(1);

Spans you start nest automatically under the request trace, so custom work shows up in context.

Alerts

Alertmanager is included in the stack. Define alert rules against the Prometheus metrics your product cares about (error rate, queue depth, latency) and route notifications where your team will see them.