Skip to content
SQA Cockpit

ADR-0019 — Pipeline boundary: structured JSON to stdout, transport via external Collector

ADRsUpdated 6 min readEdit on GitHub ↗
4 sections··

ADR-0019 — Pipeline boundary: structured JSON to stdout, transport via external Collector #

  • Status: Accepted
  • Date: 2026-05-08
  • Deciders: Natan
  • Source: Charity Majors / Liz Fong-Jones / George Miranda,

Observability Engineering ↗ (O'Reilly, 2022), Chapter 18 — "Telemetry Management with Pipelines" ↗. Audit and synthesis captured in PRD-05 ↗.

Context

Context #

SQA emits structured events today via pino to stdout. The current shape (stdout-JSON in non-TTY mode, pretty rendering in TTY mode) is described in ADR-0016 and the event-shape concept doc ↗.

The May 2026 audit against Observability Engineering ↗ surfaced a question we hadn't named: where does the seam between SQA and a real telemetry pipeline live? OE Ch. 18 describes the canonical pipeline shape:

text
receiver → buffer → processor → exporter

…and explicitly recommends that applications stop at the receiver boundary:

"It does not make economic sense to spend engineering resources developing in-house software to make up the basis of your telemetry management pipeline." — OE Ch. 18 ↗

Today's SQA flow is:

text
step() → pino → stdout

There is no buffer, no processor, no exporter inside SQA. That's already aligned with OE's recommendation, but the commitment isn't documented anywhere. Without it, three drift hazards exist:

  1. A future contributor adds a Tempo HTTP exporter directly inside

src/lib/step.ts because "we already have the data, why pipe it through stdout?". The codebase grows transport logic.

  1. A future contributor adds in-process buffering / retry / batching

for resilience against backend outages. SQA grows a state machine that duplicates what OTel Collector already does.

  1. The pretty-mode summary block (a view, not the source of truth)

becomes confused with the JSON envelope, and someone adds business logic to the renderer.

We considered three options:

  1. Status quo, undocumented. Cheap, but drift-prone — every PR

that touches step() is a chance to grow transport.

  1. Build a sink interface inside SQA (Sink.emit(event)) with

pluggable adapters (stdout, file, OTLP, Kafka). Maximally flexible. But replicates OTel Collector's job and pulls SQA toward the "build versus buy" failure OE warns against.

  1. Commit to stdout-JSON as the only egress. External Collector

handles transport. SQA becomes a no-op for telemetry: emit one record per event, let the pipeline do everything else.

Decision

Decision #

Adopt option 3. SQA emits structured JSON to stdout and nothing else. The seam between SQA and any real backend (Tempo, Honeycomb, Loki, ClickHouse) lives outside this codebase, in a telemetry pipeline (OTel Collector / Vector / Fluent Bit / Cribl / a home-rolled tee).

Concretely:

What this ADR commits SQA to #

  • One JSON record per event, on stdout, newline-delimited. Every

field listed in the event-shape concept doc ↗ is present.

  • No transport code in SQA. No HTTP exporter, no Kafka producer,

no OTLP client, no batching, no retry, no backpressure handling. All of those are the pipeline's job.

  • No in-process buffering for backend outages. If a backend is

down, the pipeline buffers (or drops); SQA doesn't notice.

  • Sampling decisions live in the pipeline. SQA emits with

sample_rate: 1.0 always (per event-shape § Required fields ↗). The pipeline applies any per-key, per-trace, or dynamic sampling using the samplingId SQA emits.

  • Backend changes don't touch SQA. Switching from Tempo to

Honeycomb is a Collector config change.

What this ADR explicitly does not adopt #

  • Direct OTel SDK integration in SQA. The OTel TypeScript SDK is

a transport library; it belongs in the pipeline, not in SQA. We emit OTel-format IDs (traceId 16 hex, spanId 8 hex per ADR-0014) so a Collector can consume our JSON and re-emit OTLP, but we don't import the SDK.

  • A pluggable sink interface inside SQA. Tempting but premature.

The day a real reason exists (e.g., we need to write to two pipelines simultaneously from one process), revisit. Until then, one egress = one shape.

The pretty-mode exception #

ADR-0016 establishes that pretty mode renders a human-readable summary block to stdout via console.log. That's a view of the JSON envelope, not a separate channel. Pipelines parse JSON-mode (which is what runs in CI / cron / non-TTY contexts); pretty mode is exclusively for interactive use. The two modes are mutually exclusive (per ADR-0016) and neither violates this ADR's commitment.

The pipeline shape SQA assumes #

Operators wiring SQA to a real backend should expect this shape:

text
SQA (this codebase)            external telemetry pipeline
─────────────────────          ──────────────────────────────
step() / runner                OTel Collector / Vector / Fluent Bit
  │                              │
  └─ pino → stdout (JSON) ──────▶│
                                 ├─▶ Tempo / Honeycomb / Loki (traces)
                                 ├─▶ ClickHouse / Druid (events)
                                 └─▶ Prometheus / VictoriaMetrics (derived metrics)

This matches OE Ch. 18's reference architecture exactly. The collector reads stdout (or a log file in containerized deploys) and fans out.

Falsifiability #

This ADR holds if and only if:

  • (a) The next time we want to send SQA events to a new backend,

the change lives in pipeline config — not in src/lib/step.ts or src/lib/logger.ts. Predicted because that's the whole point.

  • (b) No transport-related code (HTTP exporter, retry buffer,

backpressure handler) accumulates in src/. make standards can partially enforce this by flagging non-pino, non-stdout writes from lib/.

  • (c) The CI pipeline can swap from "drop events on the floor"

(current state) to "send to Honeycomb" without an SQA code change.

Revisit if (a) or (c) breaks within twelve months. If (a) fails the seam isn't where we thought; we either document a real reason for in-process transport or supersede this ADR with one that defines a sink interface.

Consequences

Consequences #

  • Pro: SQA stays small. The lib has zero transport surface.
  • Pro: Backend choice is a deploy-time decision, not a code-time

one. We can run the same SQA build with three different pipelines (dev/staging/prod) by changing the Collector config.

  • Pro: When backends fail, SQA doesn't notice. The pipeline

absorbs the outage; SQA just keeps emitting.

  • Pro: The OTel-format IDs (ADR-0014) mean the

Collector can transform JSON→OTLP without remapping.

  • Con: Operators must run a real Collector. There's no

zero-config "send to Honeycomb" path. Mitigated: zero-config means "we drop events on the floor," which is what runs today and is still correct for a CLI.

  • Con: A SQA crash mid-write can lose events that haven't yet

flushed to stdout. Mitigated: pino's default sync mode (used in pretty), and node's stdout is line-buffered in JSON mode for the same crash window any unix tool has.

  • Con: Sampling decisions made downstream means SQA cannot do

head-based sampling (which OE Ch. 17 says is the cheapest kind). Mitigated: SQA's volume is low (12 events per run, runs are infrequent), so sampling isn't a near-term concern. When it is, the Collector handles it via the samplingId field SQA already emits.

See also

See also #

contract this ADR's egress carries.

the Collector can consume.

view that's the documented exception to this ADR.

ratified the books-as-standards triangle this ADR is one node of.

that landed this commitment plus the new event fields.

source of the pipeline shape.

the canonical implementation operators will most often pick.

  • Vector ↗ — alternative pipeline tool OE Ch. 18

endorses.

Was this page helpful?