The Correctness Layer: Deterministic Validation Outside the LLM Loop

You build or buy a data agent, and you have to decide where its checks run. A check the model calls is one design. A check the harness runs without asking the model is another. A check in CI that no model can skip is a third. Each position catches a different failure. A complete correctness layer uses all three positions.

Deterministic validation means code that returns the same result for the same input on every run. A parser, a schema validator and a dbt test all qualify. A second model that reviews the first model's SQL does not qualify, because its answer can change between runs.

This post is for the engineer who wires the loop in an agentic data engineering stack. It shows the three positions, what each one returns to the model, and how the loop terminates. The examples use Altimate Code. Its source is public, so you can check each behavior in the code.

Three checkpoints for one agent session. The first two sit inside the loop, and the third runs after it.

Deterministic Validation Sits at Three Points in an Agent Loop

Three positions cover the path from a draft to a merged change. The three positions answer different questions.

The correctness layer inside Altimate Code explains the three software layers behind these checks. That post covers how the dispatcher routes a call to compiled Rust. This post covers when each check fires and what happens next.

The Harness Classifies Every Statement Before It Reaches the Warehouse

The first checkpoint runs on each tool call that touches data. In Altimate Code, the sql_execute tool classifies the SQL before it runs. The classifier in sql-classify.ts uses the engine's AST-based statement types.

The classifier applies three rules:

• It treats only a plain query as a known safe read.
• It routes any write through the sql_execute_write permission, which asks you first.
• It refuses DROP DATABASE, DROP SCHEMA and TRUNCATE in every case.

For those three statements, the tool throws a fixed error that no permission setting lifts. No prompt changes that result, because the model never makes the decision. If the native engine fails to load, a regex fallback treats every statement it cannot prove is a read as a write.

Agent modes add a second boundary. The agent modes reference lists Analyst as a read-only mode. In Analyst mode, the classifier denies INSERT, UPDATE, DELETE and DROP outright, with no approval prompt.

At the same checkpoint, the model can call altimate_core_validate on a draft before it runs anything. That tool checks syntax and schema references against a schema file or an inline table map. A draft that names a missing column fails in the engine, before the warehouse bills a single second. A worked example of a zero-token check runs this validator on one column rename.

The model decides when to call the validator. So this checkpoint catches errors only when the agent's instructions or skills tell it to validate. The second checkpoint does not depend on that choice.

A Completion Gate Blocks the Agent's Done Until Tests Pass

The second checkpoint fires when the model declares a clean stop. Altimate Code calls these checks completion validators. The docs state that the validators "are not visible to the agent." The harness runs them after the model's last message ends with a stop.

At commit 024e800, the harness registers seven dbt validators. They run in this order, cheapest first:

1. dbt-nothing-built
2. dbt-build-green
3. dbt-deliverable-names
4. dbt-incremental-config
5. dbt-dialect-guard
6. dbt-schema-verify
7. dbt-tests-pass

The docs describe the last two in detail. The dbt-schema-verify validator compares each modified model's columns with its schema.yml spec. The dbt-tests-pass validator runs the dbt tests of each modified model. The docs say it "refuses to terminate if any model's tests fail or error."

A failed validator does not end the session. The harness writes one synthetic user message that lists every failure, and the model gets another turn. The message format comes from prompt.ts:

[altimate-validator: dbt-schema-verify] <reason naming the failing models>
<fixHint>

The docs give an example reason: "2 of 3 models you edited have a column-shape mismatch against schema.yml: foo, bar". The model reads the model names and the fix hint, then edits those files.

The Loop Ends on a Pass or on a Spent Retry Budget

A correctness layer needs an exit rule, or a model that cannot fix a failure loops forever. Altimate Code bounds the loop with ALTIMATE_VALIDATORS_MAX_RETRIES, which defaults to 3. The simplified loop below follows prompt.ts and sql-execute.ts:

retries = 0
loop:
  step = model.next(messages)                  # draft SQL, pick tools
  for call in step.tool_calls:
    if call.tool == "sql_execute":
      kind = classify(call.sql)                # AST statement types
      if kind is DROP DATABASE, DROP SCHEMA or TRUNCATE: refuse
      if kind is write: ask for sql_execute_write permission
    messages.append(run_tool(call))            # native handler, no model call
  if step.finish == "stop" and no tool calls are outstanding:
    if validators_on:                          # ENABLED=1 or SHADOW=1
      failures = [v for v in validators if v.applies() and not v.check().ok]
      if failures and enforcement_on and retries < MAX_RETRIES:
        messages.append(user_turn(format(failures)))
        retries = retries + 1
        continue
    break

The loop has three exits:

• All validators pass. The session ends, and the agent's claim of done holds for every applied check.
• The retry budget runs out. The harness emits a validator_retries_exhausted event and marks the session completed with unresolved failures.
• Validators are off. The session ends when the model stops, and only the first and third checkpoints protect you.

The second exit is the one to plan for. A spent budget means the agent could not fix its own work. The session record says so, and a person has to take over.

CI Runs the Same Checks With No Model in the Job

The third checkpoint runs after the session, on the pull request. Nothing in the job calls a model, so the result depends only on the files. The check command docs say altimate-code check needs no model provider and no API key. It exits 1 when findings reach the --fail-on level.

The GitHub Actions job below runs three checks in order: parse, compile, then the Altimate Code check. It uses the DuckDB adapter, like the Altimate Code sample project. Swap in your own adapter and credentials.

name: dbt deterministic checks
on: [pull_request]

jobs:
  checks:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Install dbt and Altimate Code
        run: |
          pip install dbt-duckdb
          npm install -g altimate-code
      - name: Parse the project
        run: dbt parse
      - name: Compile models to plain SQL
        run: dbt compile
      - name: Run deterministic checks
        run: |
          altimate-code check "target/compiled/**/*.sql" \
            --checks lint,validate,safety \
            --schema schema.yml \
            --format json --fail-on error

The steps run in this order for a reason. The dbt parse command validates the project without a warehouse connection. So a broken ref fails first, before any step connects to the warehouse. The dbt compile command then renders Jinja into plain SQL, which the engine can parse. The last step lints, validates and scans the compiled files.

The JSON output ends with a summary object that holds errors, warnings and pass. A later step can read summary.pass with jq and post the counts on the pull request. The check docs show that pattern with actions/github-script.

For a faster loop on your own machine, the docs show a pre-commit hook:

repos:
  - repo: local
    hooks:
      - id: altimate-sql-check
        name: SQL Check
        entry: altimate-code check --fail-on warning
        language: system
        types: [sql]
        pass_filenames: true

Point the hook at compiled SQL if your models use Jinja, because the engine parses SQL text.

The Result Format Decides Whether the Model Can Fix the Error

A check helps the loop only if the model can act on its result. Each checkpoint in Altimate Code returns a short, structured result.

• The validator tool returns valid, has_schema and a list of findings with a category such as missing_column.
• The completion gate returns ok, a reason that names the failing models, and a fixHint.
• The CI check returns JSON findings with file, line, rule, severity, message and an optional suggestion.

Each result names a location and a rule. A model that reads "line 6, missing column" edits line 6. A model that reads a reviewer's paragraph has to guess which line the paragraph meant. The CI result is the same for the model and for a person, so a reviewer can read the finding the agent read.

Five reasons deterministic tooling beats an LLM-only stack covers why a fixed answer matters for trust. Why human review stops at scale covers the reviewer's side of the same problem. Why agents fail after merge covers the failures that appear only once a change reaches production.

Deterministic Validation Has Five Limits You Should Plan Around

A correctness layer catches only what its rules describe. These five limits come from the Altimate Code source and docs.

The completion gate is off by default. Enforcement needs ALTIMATE_VALIDATORS_ENABLED=1, and shadow mode needs ALTIMATE_VALIDATORS_SHADOW=1. A comment in prompt.ts warns against enforcement beyond a shadow soak. It says duplicated heuristics already caused a blocking false positive.

The completion gate costs time. The docs put each per-model subprocess at 5 to 30 seconds. Five touched models take about 1 to 2 minutes after the agent says done.

The completion gate misses some models. It scans only .sql files under a models/ folder. Python models and custom model-paths are outside its scan.

The anti-pattern benchmark is synthetic. The benchmark file reports F1 of 1.00 on 19 rules across 1,077 queries, with 0 false positives. A script generated those queries from a fixed seed, in the Snowflake dialect only. The file says 100% on synthetic queries "does not guarantee 100% on production SQL."

Rules check shape, not intent. The docs show policy rules as regex patterns over SQL text. A validator confirms that a column exists, not that the join returns the right grain. A dbt test on your real data covers that gap, which is why dbt-tests-pass runs last. The CI job also reads files, not data. It validates against the schema file you pass, so a stale schema.yml gives a stale result.

Add the CI Check Before You Enable the Completion Gate

Start with the checkpoint that has no model in it. Add the CI job above to one dbt repository, with --fail-on error. Read the findings on several pull requests before you tighten the threshold.

Then run the completion gate in shadow mode with ALTIMATE_VALIDATORS_SHADOW=1. Shadow mode runs every validator and records whether it would have fired, without blocking the session. Each run emits a validator_check event with the validator name, the ok result and the retry count. Turn on enforcement only when those events show few false positives.

For a whole team, the agentic data engineering platform adds shared context and governance. To try the checks today, install Altimate Code and run altimate-code check on your compiled models. Add a lineage diff before merge when column-level changes matter to your reviewers.