Skip to content
SQA Cockpit

Concepts & how it works

LearnUpdated 3 min readEdit on GitHub ↗

Concepts & how it works #

Diátaxis form: explanation. The one page that defines SQA's core nouns and shows how they connect. For the full vocabulary see the Glossary; for the motivation see Why SQA exists.

SQA verifies that a system delivers the value it promised its users - not just that its code runs. The whole model is nine nouns. Read them in order; each builds on the one before it.

The core concepts #

ConceptWhat it is
SUTThe system under test - the thing you're verifying (e.g. metaintro-chat). One folder under src/systems/.
ComponentA dependency the SUT relies on - Mongo, S3, the browser. SQA probes each one to know the SUT can even work.
ContractThe SUT's promise to its users, written down. One per scenario. It describes value (what users get), not shape (an API spec).
ClaimOne falsifiable sentence in the contract - "the jobs returned are relevant to the query." The unit of value, and what a run grades.
ScenarioThe synthetic transaction that drives the SUT to exercise a claim. One file = one composite result.
StepOne atomic action inside a scenario: drive the system, observe what came back, verify it against the claim. Steps nest (1, 1.1).
RunOne execution of a scenario - a trace, a result tree, an exit code.
Outcome + EvidenceEach claim's result - an outcome (pass / fail / ...) or a 0-100 score with a band - plus the artifacts (screenshots, returned data, judge scores) that back it.
ReportThe run made readable: the run's verdict (a one-line human summary like "Relevancy 67/100 - yellow"), every claim's outcome, and the evidence underneath each one.

How they connect #

text
SUT  ──promises──▶  Contract  ──a set of──▶  Claims
                                                │ each verified by
                                                ▼
Report  ◀──assembled into──  Run  ◀──executes──  Scenario
   ▲                          │                     │ made of
   │ verdict + outcomes       │ grades each claim    ▼
   └────────────────────  Outcome + Evidence    Steps: drive → observe → verify

In one sentence: a SUT makes its users a promise; you write that promise as a Contract of Claims; each claim is verified by a Scenario - a sequence of Steps (drive → observe → verify) that exercise the live SUT and its Components; one execution is a Run, which grades each claim into an Outcome backed by Evidence, summarized as a Verdict and assembled into a Report.

How everything works, end to end #

  1. Understand the SUT. What the system is, the components it leans on, and

how to drive it, observe it, and collect evidence from outside.

  1. Write the contract. Capture what the system owes its users as falsifiable

claims. You author the promise; SQA is the witness that tries to falsify it (ADR-0032) - it never grades a promise it wrote itself.

  1. Build the scenario. Wire the steps that drive the live system, read

what came back, and verify it against each claim - keeping evidence at every step.

  1. Run it. One execution produces a trace, a result tree, and a score or

pass/fail per claim - locally (make run) or from the cockpit.

  1. Read the report. The run's verdict up top, every claim's outcome,

who triggered the run, and the evidence underneath each result - so a green board can still tell you the product fell short, before your users do.

Where to go next #

Was this page helpful?