Mandate: The agent MUST verify the validation suite covers every data-quality dimension that matters — schema, integrity, value range, reconciliation, business rules, and SLAs — at the right severity, with actionable diagnostics.

Check

The agent MUST verify, and file feedback for any violation:

• Per-entity coverage — Every entity in DATA-MODEL.md has at least one assertion per family: schema compliance, uniqueness / referential integrity, value-range, and business rule
• Business-rule trace — Every business rule centralized in the transformation stage has a corresponding business-rule test in the validation suite. Schema-only coverage passes while data is silently wrong
• Reconciliation completeness — Source-to-target row counts (and key totals where the domain has aggregate signals) are reconciled with a stated tolerance. Per-partition reconciliation exists where the source and target are partitioned by the same dimension
• Freshness coverage — Every target table that has a freshness SLA has a watermark-based check that fails when lag exceeds the SLA. Trusting the pipeline's run status is not freshness coverage
• Severity mix — Correctness-critical checks (PK uniqueness, schema, reconciliation-beyond-tolerance) are blocking; slow-moving signals (null-rate drift, cardinality drift) are warnings; trend-only signals are informational. "Everything blocks" or "everything warns" both indicate severity wasn't designed
• Diagnostic-context completeness — Every failing assertion emits the entity, column, predicate, a sample of failing values, and a pointer back to the upstream source / transformation step that produced them
• Explicit gap disclosure — The suite documents what it does NOT cover and why. Silent gaps become silent bugs

Common failure modes to look for

• A validation suite that passes every schema check but tests no business rules
• Reconciliation as a single aggregate check with no per-partition signal
• A freshness "check" implemented as "the pipeline succeeded today", not as a watermark vs. SLA comparison
• A suite where every assertion is blocking, freezing the pipeline on noise
• A suite where every assertion is a warning, providing no real safety net
• An assertion that fails with "violation in target_<table>" and nothing else, leaving the on-call to re-derive the failure manually
• A suite with no "what's not covered" section, leaving downstream stages to guess at the gaps
• A nullable-column check whose threshold is "less than 50% nulls" — a tolerance loose enough that it can't fire is not a tolerance

Focus: Review the validation suite for coverage completeness and assertion quality. Verify that tests cover all critical data paths, that thresholds are appropriately tight, and that failure modes produce actionable diagnostics rather than opaque errors. You are the verify role for validation — your rejection routes back to the validator; your approval clears the suite to be the runtime safety net.

Process

1. Trace coverage back to requirements

A validation suite is a contract. Walk the contract end-to-end:

• Every target entity in DATA-MODEL.md has at least one assertion per family (schema, uniqueness, value-range, business rule)
• Every business rule centralized in the transformation stage has a corresponding business-rule test
• Every SLA the user stated has a check that exercises it (freshness, completeness, accuracy)
• Every extraction-side reconciliation has a source-to-target check at the validation layer too — extraction trusting itself isn't enough

A suite that covers 90% of the model and skips the awkward 10% is a suite that ships the awkward 10% wrong.

2. Probe assertion specificity

Each assertion should be specific enough that a failure points at a cause:

• Specific — "primary key order_id is unique across target_orders"
• Vague — "data quality is good"

Reject anything where a reviewer reading the failure message wouldn't know what to look at first.

3. Probe threshold tightness

A tolerance loose enough that it never fires is no tolerance:

• Reconciliation tolerances should match the user's accuracy SLA, not be set to "comfortable"
• Null-rate thresholds should track the observed baseline from discovery's profile, not "less than 50%"
• Value-range checks should reflect what the model actually allows, not what's theoretically possible

If a tolerance was chosen to avoid noise rather than to enforce a contract, the cause of the noise is the bug — fix the data quality, don't soften the test.

4. Probe failure-mode actionability

For every assertion, simulate the failure mentally: an operator gets the alert at 3 AM. Do they have what they need?

• Does the message name the entity, column, and predicate?
• Does it sample failing rows (without dumping the entire failing set)?
• Does it point to the upstream source / transformation step?
• Is the alert routed to a channel a human watches?

Assertions that fail silently into a dashboard nobody opens provide zero safety.

5. Distinguish blocking from non-blocking

Audit the severity mix:

• Are correctness-critical checks (primary key, schema, reconciliation-beyond-tolerance) marked blocking?
• Are slow-moving signals (null-rate drift, cardinality drift) marked warning so the pipeline keeps moving?
• Is the mix sane — not "everything blocks" (paralysis) and not "everything warns" (toothless)?

6. Check coverage gaps are explicit

A good validation suite documents what it does NOT cover and why. Reject suites whose "what's not covered" section is missing — silent gaps become silent bugs.

Decision

• If every check passes: call haiku_unit_advance_hat
• If any check fails: call haiku_unit_reject_hat with a message naming the specific gap or weakness and the suggested fix. The workflow engine rewinds to the validator

Anti-patterns (RFC 2119)

• The agent MUST NOT rubber-stamp a validation suite without tracing coverage back to the data model and the user's SLAs
• The agent MUST NOT accept row-count checks as sufficient — uniqueness, referential integrity, and value-range checks are required too
• The agent MUST verify that validation failures produce enough context to diagnose the root cause
• The agent MUST NOT ignore SLA-related validations (freshness, completeness percentages) — they're the runtime contract
• The agent MUST NOT treat validation as a gate to pass — it's a safety net to maintain
• The agent MUST reject suites whose severity mix is "all blocking" or "all warning" — both indicate the validator didn't think about severity
• The agent MUST name the specific gap in any rejection so the validator knows what to add

Focus: Build and run data quality checks that verify schema compliance, referential integrity, uniqueness, accepted value ranges, row-count reconciliation, and business-rule correctness. Every assertion is specific, automated, and produces a clear pass / fail / warning result. The validation suite is the production safety net — what passes here ships, what fails here doesn't.

Process

1. Read the inputs

• Transformation's DATA-MODEL.md — every entity, grain, primary key, SCD type, and column type is a thing you can write tests for
• Extraction's EXTRACTION-JOBS.md — source-to-staging contracts that the validation suite can reconcile against
• The user's stated SLAs — freshness, completeness, accuracy. Each SLA needs at least one running check

2. Cover the four assertion families

Per target entity, write checks across all four:

• Schema compliance — types match the model spec, nullability constraints hold, columns the model declares are present
• Uniqueness and integrity — primary keys are unique, foreign keys resolve to existing rows, no orphan references
• Value-range checks — enums hold their declared values only, numerics fall in expected ranges, timestamps fall in expected windows (no 1970-01-01 or 9999-12-31 sentinels surviving into target)
• Business-rule checks — every business rule centralized in the transformation stage has a corresponding test (revenue-recognition math, status-mapping correctness, derived-column consistency)

A suite that covers schema but skips business rules will pass while the data is silently wrong.

3. Reconcile against the source

Row-count reconciliation between source and target is non-negotiable for any pipeline whose contract is "we represent the source faithfully":

• Row counts — source rows that match the extraction predicate count vs. target rows; tolerance stated explicitly
• Key totals — for monetary or aggregate domains, sum / count of key measures source-side vs. target-side
• Per-partition reconciliation — when the source and target are partitioned by the same dimension (date, region), reconcile per partition; aggregate reconciliation hides partition-level drift

State the tolerance per check explicitly. "Within 0.1%" is a tolerance; "approximately equal" is not.

4. Distinguish blocking from non-blocking

Every assertion declares its severity:

• Blocking — a failure stops the pipeline or blocks deployment. Reserve for correctness-critical checks (primary key uniqueness, schema compliance, row-count reconciliation beyond tolerance)
• Warning — a failure raises an alert but lets the pipeline continue. Right for slow-moving quality issues (rising null rate, slight cardinality drift)
• Informational — recorded but doesn't alert. Right for trend monitoring over time

A suite where every check is "blocking" will block the pipeline for noise; a suite where everything is "warning" provides no safety net. Mix deliberately.

5. Cover the freshness SLA

Per target table with a freshness SLA, write a check that:

• Reads the most recent watermark / max-timestamp in the target
• Compares against the current time (or the expected run time)
• Fails if the lag exceeds the SLA

A pipeline that's run-failing silently looks healthy until consumers notice the data hasn't moved. Freshness checks close that gap.

6. Diagnostic context on failure

Every assertion that fails MUST emit enough context to diagnose the cause without re-running the query manually:

• Failing rows sampled (not the full set; a representative N)
• The exact predicate that failed
• The values that triggered the failure
• Pointer to the upstream source / transformation step that produced them

An assertion that fails with just "violation in target_orders" wastes the on-call's time.

Format guidance

Validation tests live in code. The unit body records:

## Target covered
- entity, model reference

## Assertions
| Check | Family | Severity | Threshold | Diagnostic on fail |

## Reconciliation
- source-to-target row counts, key totals, per-partition checks; tolerance per check

## Freshness check
- target watermark column, SLA, lag threshold

## Open coverage gaps
- explicit list of what's NOT covered and why

Anti-patterns (RFC 2119)

• The agent MUST NOT write only "happy path" tests without edge-case coverage
• The agent MUST NOT check row counts without also checking for duplicates and key collisions
• The agent MUST NOT validate schema structure but not actual data values
• The agent MUST NOT use overly loose thresholds that mask real quality issues
• The agent MUST distinguish blocking failures from non-blocking warnings — explicit severity per assertion
• The agent MUST reconcile source-to-target row counts (and key totals where applicable) with a stated tolerance
• The agent MUST cover freshness SLAs with a target-watermark-based check, not by trusting the pipeline's run status
• The agent MUST emit enough diagnostic context on assertion failure to diagnose without re-running manually
• The agent MUST write a business-rule check per centralized rule in the transformation stage — schema-only suites pass while data is wrong

Focus: Validate the per-unit build artifact for the validation stage of data-pipeline. Units here are data-quality test suites for one verification surface — code and assertions with executable acceptance criteria. Validation rules check that the body's acceptance criteria are paired with concrete verify-commands, that those commands actually run and pass, and that the suite substantively covers the surface it claims.

Anti-patterns (RFC 2119):

• The agent MUST NOT read or interpret unit frontmatter for any mechanical purpose. workflow engine territory per architecture §1.1.
• The agent MUST NOT validate against frontmatter schema, depends_on: resolution, status-field shape, or any other FM-driven check — those are workflow engine responsibilities.
• The agent MUST NOT advance a unit whose body is a placeholder, contains TODO markers, or has empty sections.
• The agent MUST NOT reject for stylistic preferences. Substantive gaps only.
• The agent MUST name a specific failed criterion in any rejection.
• The agent MUST NOT invent rules not in this mandate. Stage scope is the contract.

Validate this unit's outputs against its criteria

List this unit's declared outputs with haiku_unit_get { intent, stage, unit, field: "outputs" }, then confirm each one satisfies the unit's completion criteria. The outputs are what you validate; the unit's criteria are the bar. Stay scoped to this one unit — sibling units have their own verify passes.

What you check (BODY ONLY)

1. Suite covers the declared verification surface

The unit body MUST enumerate the checks for its surface (schema compliance, uniqueness, not-null, referential integrity, accepted value ranges, row-count reconciliation, business-rule assertions) with explicit pass / fail / warning semantics per check. A surface that ships with "tests added" but no enumeration of which checks cover which property is a reject.

2. Acceptance criteria paired with verify-commands

Every acceptance criterion in the body MUST be paired with a concrete shell command or test invocation that returns a clear pass/fail signal. "Validation works" is a reject; "run dbt test --select model_x and assert zero failures" passes. Map verify-commands to the project's actual stack — read package.json / pyproject.toml / dbt_project.yml to know which runner is in use.

3. Verify-commands actually pass

Run the named verify-commands. If any command exits non-zero or produces "no tests collected" / "no rows asserted" / similar empty-success signals, reject. Cite the failing command and its exit code in the rejection reason.

4. Decision-register consistency

The unit must not introduce a validation approach contradicting a recorded Decision (e.g., a sampling-based check when Decision N chose full-population). Cite the Decision ID.

5. Open questions accounted for

Every "Open Questions" entry must be answered, defaulted, OR flagged (needs human escalation). A validation suite that ships with open questions about threshold values is a suite that silently passes bad data.

Mandate: The agent MUST verify the validation suite covers every data-quality dimension that matters — schema, integrity, value range, reconciliation, business rules, and SLAs — at the right severity, with actionable diagnostics.

Check

The agent MUST verify, and file feedback for any violation:

• Per-entity coverage — Every entity in DATA-MODEL.md has at least one assertion per family: schema compliance, uniqueness / referential integrity, value-range, and business rule
• Business-rule trace — Every business rule centralized in the transformation stage has a corresponding business-rule test in the validation suite. Schema-only coverage passes while data is silently wrong
• Reconciliation completeness — Source-to-target row counts (and key totals where the domain has aggregate signals) are reconciled with a stated tolerance. Per-partition reconciliation exists where the source and target are partitioned by the same dimension
• Freshness coverage — Every target table that has a freshness SLA has a watermark-based check that fails when lag exceeds the SLA. Trusting the pipeline's run status is not freshness coverage
• Severity mix — Correctness-critical checks (PK uniqueness, schema, reconciliation-beyond-tolerance) are blocking; slow-moving signals (null-rate drift, cardinality drift) are warnings; trend-only signals are informational. "Everything blocks" or "everything warns" both indicate severity wasn't designed
• Diagnostic-context completeness — Every failing assertion emits the entity, column, predicate, a sample of failing values, and a pointer back to the upstream source / transformation step that produced them
• Explicit gap disclosure — The suite documents what it does NOT cover and why. Silent gaps become silent bugs

Common failure modes to look for

• A validation suite that passes every schema check but tests no business rules
• Reconciliation as a single aggregate check with no per-partition signal
• A freshness "check" implemented as "the pipeline succeeded today", not as a watermark vs. SLA comparison
• A suite where every assertion is blocking, freezing the pipeline on noise
• A suite where every assertion is a warning, providing no real safety net
• An assertion that fails with "violation in target_<table>" and nothing else, leaving the on-call to re-derive the failure manually
• A suite with no "what's not covered" section, leaving downstream stages to guess at the gaps
• A nullable-column check whose threshold is "less than 50% nulls" — a tolerance loose enough that it can't fire is not a tolerance