KBkilterKB
Log in
dev

Observability (SigNoz)

SigNoz is OpenTelemetry-native: your app exports OTLP, SigNoz stores and displays traces, metrics, and logs. kilter provisions the collector and injects two env vars — confirm with kilter env, never hardcode:

  • OTEL_EXPORTER_OTLP_ENDPOINT — the OTLP HTTP collector (:4318). Where the SDK exports.
  • SIGNOZ_URL — the SigNoz UI (:8080). Where you read the traces.

Setup

Three steps regardless of language:

  1. Install the OTel SDK.
  2. Initialize tracing at process start, before any instrumented library loads.
  3. Set a service name so the app is identifiable in SigNoz's Services list.

Node / Next.js

npm install @opentelemetry/sdk-node @opentelemetry/auto-instrumentations-node @opentelemetry/exporter-trace-otlp-http

Next.js calls register() in a root-level instrumentation.ts once at server startup — the correct init point. Guard on the Node runtime (the Edge runtime can't run the Node SDK):

// instrumentation.ts
export async function register() {
  if (process.env.NEXT_RUNTIME === 'nodejs') {
    await import('./otel.node');
  }
}
// otel.node.ts
import { NodeSDK } from '@opentelemetry/sdk-node';
import { getNodeAutoInstrumentations } from '@opentelemetry/auto-instrumentations-node';
import { OTLPTraceExporter } from '@opentelemetry/exporter-trace-otlp-http';
import { resourceFromAttributes } from '@opentelemetry/resources';
import { ATTR_SERVICE_NAME } from '@opentelemetry/semantic-conventions';
 
const sdk = new NodeSDK({
  resource: resourceFromAttributes({ [ATTR_SERVICE_NAME]: 'my-app' }),
  traceExporter: new OTLPTraceExporter(),   // reads OTEL_EXPORTER_OTLP_ENDPOINT
  instrumentations: [getNodeAutoInstrumentations()],
});
sdk.start();

Plain Node: load the same init via --require ./otel.node.js or as the first import in the entry file. Go: otlptracehttp.New(ctx) reads the endpoint from env; set OTEL_SERVICE_NAME. Python: run under opentelemetry-instrument with OTEL_SERVICE_NAME set.

Init order is everything

Auto-instrumentation patches libraries as they load. If the SDK starts after your HTTP/DB clients are imported, those traces silently never exist. The init must be the very first thing the process runs.

Verify

  1. Generate traffic — hit a route, run a job.
  2. Open the SigNoz UI at SIGNOZ_URL → Services. The app appears within ~30 seconds of the first exported span.
SymptomCause / fix
App absent from ServicesNo spans exported yet — generate traffic; log after sdk.start() to confirm it ran
ECONNREFUSED to the collectorWrong endpoint — re-read OTEL_EXPORTER_OTLP_ENDPOINT from kilter env
Spans start, nothing arrivesProtocol/port mismatch — this stack is OTLP HTTP on :4318; the gRPC exporter wants :4317
Library traces missingSDK initialized too late — move init to the entry point / instrumentation.ts
Nothing on Edge/serverlessNode SDK can't run on the Edge runtime — guard on NEXT_RUNTIME === 'nodejs'

Custom spans

Auto-instrumentation covers HTTP, DB, and framework calls. Add spans for business logic:

import { trace } from '@opentelemetry/api';
const tracer = trace.getTracer('my-app');
await tracer.startActiveSpan('checkout', async (span) => {
  span.setAttribute('order.id', orderId);
  try { /* work */ } finally { span.end(); }
});