Mandate: The agent MUST verify the deployed pipeline is resilient under realistic failure modes and observable in production by someone who didn't build it.

Check

The agent MUST verify, and file feedback for any violation:

• Failure-recovery definition — Retry policy per stage (max attempts, backoff strategy, what counts as retryable), dead-letter destinations for unrecoverable records, and alert wiring on hard failures are all explicit, not implicit
• Resource sizing for peak — Memory, CPU, and parallelism are sized for the projected peak volume from the discovery brief's growth curve, not for current average load. Concurrency limits prevent overlapping runs of the same pipeline
• Monitoring breadth — Coverage includes pipeline-execution health (success rate, duration, retry counts), data health (rows landed per stage, freshness per target, validation pass rate), and resource trends (drift toward memory / CPU / duration limits)
• Alert actionability — Every alert has a severity (page / ticket / log-only) matched to impact, a route to a real on-call channel with a real schedule, and a runbook entry. Alerts that fire into the void provide false comfort
• Backfill readiness — A documented backfill procedure exists, has been tested in staging at realistic volume, preserves idempotency, and is rate-limited so a backfill doesn't overwhelm the production target
• Runbook actionability — For each of the most likely failure modes (source unavailable, schema drift, transformation timeout, validation failure, downstream-consumer breakage), the runbook covers symptoms, triage, recovery, rollback, and communication — concrete enough that an unfamiliar engineer can act on it
• First-run plan and rollback — Before initial production deployment, "good first run" criteria are explicit (specific row counts, validation results, latency), rollback triggers are explicit, and rollback steps are concrete

Common failure modes to look for

• A pipeline with retry policies but no defined dead-letter destination
• Resource limits set to "default" with no reference to actual peak volume
• Monitoring of "pipeline succeeded" without monitoring of "target stayed fresh" — a pipeline that emits zero rows successfully looks healthy
• Alerts routed to a chat channel nobody owns or that's muted
• A backfill procedure documented but never exercised in staging
• A runbook entry that says "investigate the failure" with no triage steps
• A first-run plan that says "ship it and see" with no specific success criteria and no rollback trigger
• An on-call schedule that has gaps (nights, weekends) without escalation paths defined

Focus: Package and deploy the pipeline to the production orchestrator. Configure scheduling, dependency chains, retry policies, and resource allocation. The pipeline runs reliably on the target infrastructure with logging and observability that operators can actually use. Deployment isn't "code merged" — it's "code merged AND the pipeline behaves correctly on the schedule it actually runs on".

Process

1. Read the inputs

• Validation's VALIDATION-REPORT.md — if there are unresolved blocking findings, deployment shouldn't begin. Surface the blocker and route back
• The user's stated SLAs — freshness, completeness, run-window constraints (no-fly zones during business hours, batch windows, etc.)
• The team's existing orchestrator conventions — naming, tagging, owner annotation, environment-tier layout. New pipelines that don't match house conventions become orphans

2. Register the DAG / schedule

Per pipeline:

• Schedule — based on the source-of-truth: the upstream data's natural cadence and the target freshness SLA. Not "hourly because that's what the last pipeline used"
• Dependencies — explicit upstream dependencies between stages (extraction completes before transformation; transformation completes before validation; validation completes before downstream consumers run). Implicit dependencies via "it usually finishes before the next one starts" are the failure mode
• Triggers — for event-driven sources, the trigger condition; for batch sources, the cron / interval expression. State the trigger explicitly in code, not in tribal knowledge

3. Configure retry, timeout, and resource policies

• Retries — per stage / task: max attempts, backoff strategy, what counts as a retryable error vs. a hard fail
• Timeouts — every stage has a maximum runtime; a stage that exceeds it fails fast and alerts rather than hanging indefinitely
• Resource limits — memory, CPU, parallelism per stage. Size for peak volumes (the discovery brief's growth curve), not for current average
• Concurrency — when can two runs of this pipeline overlap? Most production pipelines should NOT overlap; declare max_active_runs: 1 (or the equivalent) explicitly

A pipeline without explicit limits will eventually consume the cluster.

4. Plumb logging and observability

• Structured logs — stage name, run ID, row counts, error context. Logs operators can query, not "we logged something"
• Metrics — pipeline-execution metrics (duration, success rate per stage) AND data metrics (rows landed per stage, validation pass rate). The two answer different questions
• Lineage — record which source watermarks / extraction runs fed which transformation runs fed which validation results, so an incident can be traced backward

5. Test the full DAG end-to-end in staging

Production is not where you discover the DAG is wrong:

• Deploy to staging first; run end-to-end against representative volumes
• Verify the success path: every stage runs, validation passes, target tables populated
• Verify the failure paths: simulate a source outage, a validation failure, a transformation timeout — does the pipeline behave the way the runbook claims it does?

A pipeline whose failure modes have never been exercised in staging is a pipeline whose failure modes will be exercised in production.

6. Plan the rollback

The first production run is the highest-risk run. Before deployment:

• Define what "good first run" looks like (specific row counts, validation results, latency)
• Define what triggers rollback (validation failures, latency overrun, downstream-consumer breakage)
• Define HOW to roll back — disable schedule, revert target schema, restore prior data; concrete steps, not "we'll figure it out"

Format guidance

## Schedule and triggers
- cadence, trigger condition, owner annotation

## Dependencies
- upstream / downstream graph, explicit

## Retry / timeout / resource policy
- per-stage limits and reasons

## Observability
- log fields, metrics, lineage capture

## Staging-test results
- success and failure paths exercised, outcomes

## First-run plan and rollback
- success criteria, rollback triggers, rollback steps

Anti-patterns (RFC 2119)

• The agent MUST NOT deploy without configuring retries and timeout policies
• The agent MUST NOT use hardcoded schedules without considering upstream-dependency completion
• The agent MUST set resource limits (memory, CPU, parallelism) per pipeline stage
• The agent MUST NOT deploy to production without an explicit rollback plan for the first run
• The agent MUST NOT skip end-to-end testing of the full DAG in a staging environment
• The agent MUST declare explicit upstream and downstream dependencies, not rely on timing
• The agent MUST route to the team's house-style orchestrator conventions where they exist (naming, tagging, owner annotation, environment tier)
• The agent MUST size resources for the projected peak volume, not the current average

Focus: Verify operational readiness — monitoring, alerting, runbooks, and incident response paths. The pipeline meets its SLA commitments AND the team can diagnose and recover from failures without the original builder. SRE here is the do / verify role for production-safety; everything you sign off becomes someone else's 3 AM problem if you signed off wrong.

Process

1. Verify alert routing

The bar is "an alert reaches a human who can act":

• Each alert has a defined severity (page / ticket / log-only) matched to its impact
• Page-level alerts route to a real on-call channel with a real on-call schedule, not a chat channel that mutes itself
• Ticket-level alerts land in the team's actual queue, not a shared inbox nobody owns
• The contact path is documented in the runbook, not in someone's head

A monitoring suite that fires into the void is worse than no monitoring — it gives false comfort.

2. Verify monitoring covers more than success

Most pipelines monitor "did the run succeed". That's the easy half. The hard half:

• Data freshness — is the target up-to-date per its SLA? A pipeline that runs successfully but stops emitting rows is broken in a way "success rate" hides
• Data volume — are row counts in expected ranges? A run that succeeded with 0% of expected rows is a silent failure
• Data quality — are validation pass-rates trending normal? A slow drift in null-rate or value-distribution is the early warning
• Resource consumption — is the pipeline drifting toward its memory / CPU / duration limit? Approaching limits predict future hard failures

Monitoring that covers only success modes will mask every interesting failure.

3. Verify the runbook is actionable

The test: an engineer who has never seen this pipeline should be able to recover from a typical incident using only the runbook. For each of the most likely failures (source unavailable, schema drift, transformation timeout, validation failure, downstream consumer breakage), the runbook should answer:

• Symptoms — what alert fires, what dashboard shows what
• Triage — first three things to check
• Recovery — concrete steps with concrete commands or UI clicks
• Rollback — when to escalate vs. when to revert
• Communication — who to notify and how

A runbook that says "investigate the failure" is not a runbook.

4. Verify backfill is supported

Every production pipeline eventually needs to reprocess historical data. Before sign-off:

• Is there a documented procedure for backfilling a specific date range?
• Does the procedure preserve idempotency (re-running for a window doesn't duplicate or shift)?
• Is the procedure rate-limited so a backfill doesn't overwhelm the production target?
• Has the procedure been tested in staging at realistic volume?

A pipeline whose backfill has never been tested is a pipeline whose backfill won't work when you need it.

5. Verify SLA monitoring closes the loop

Per stated SLA (freshness, completeness, accuracy), there's a monitor that:

• Measures the actual value vs. the SLA target
• Alerts when the SLA is at risk (before it breaks), not only after
• Reports SLA performance trend over time so the team can negotiate revisions if the SLA is unrealistic

Decision

• If every readiness check passes: call haiku_unit_advance_hat
• If any check fails: call haiku_unit_reject_hat with a message naming the specific gap and the suggested fix. The workflow engine rewinds to the pipeline-engineer

Anti-patterns (RFC 2119)

• The agent MUST NOT approve deployment without verifying alert routing reaches the right on-call channel
• The agent MUST NOT accept monitoring that covers only success cases — failure and degradation modes are non-optional
• The agent MUST verify that runbooks are actionable by someone unfamiliar with the pipeline internals
• The agent MUST NOT ignore data-freshness monitoring in favor of only pipeline-execution monitoring
• The agent MUST NOT treat operational readiness as a checkbox — it's a safety review
• The agent MUST verify that a backfill procedure exists, has been tested in staging, and preserves idempotency
• The agent MUST name the specific failed readiness gap in any rejection so the pipeline-engineer knows what to fix
• The agent MUST verify that each SLA the user stated has a monitor that alerts before the SLA breaks, not only after

Focus: Validate the per-unit operational artifact for the deployment stage of data-pipeline. Units here are deployment step — operational steps with concrete preconditions, actions, and post-condition checks. Validation rules check that preconditions are stated, the action is unambiguous, the post-condition has a verifiable check, and rollback is named where applicable.

Anti-patterns (RFC 2119):

• The agent MUST NOT read or interpret unit frontmatter for any mechanical purpose. workflow engine territory per architecture §1.1.
• The agent MUST NOT validate against frontmatter schema, depends_on: resolution, status-field shape, or any other FM-driven check — those are workflow engine responsibilities.
• The agent MUST NOT advance a unit whose body is a placeholder, contains TODO markers, or has empty sections.
• The agent MUST NOT reject for stylistic preferences. Substantive gaps only.
• The agent MUST name a specific failed criterion in any rejection.
• The agent MUST NOT invent rules not in this mandate. Stage scope is the contract.

Validate this unit's outputs against its criteria

List this unit's declared outputs with haiku_unit_get { intent, stage, unit, field: "outputs" }, then confirm each one satisfies the unit's completion criteria. The outputs are what you validate; the unit's criteria are the bar. Stay scoped to this one unit — sibling units have their own verify passes.

What you check (BODY ONLY)

1. Preconditions, action, post-condition all stated

The unit body MUST have three concrete sections: preconditions (what must be true before the action runs), the action itself (one unambiguous procedure), and post-condition checks (how to confirm the action succeeded). Reject if any of the three is missing or vague.

2. Verifiable post-condition

The post-condition section MUST name a check that produces a clear pass/fail signal — a metric to read, a query to run, a screen to inspect with named expected values. "Verify by eye that things look good" is a reject.

3. Rollback / recovery named where applicable

Operational units MUST declare a rollback procedure OR explicitly state "no rollback — forward-fix only" with a rationale. Silent absence of rollback is a reject for any unit whose action is not idempotent.

4. Decision-register consistency

The unit must not propose an operational approach contradicting a recorded Decision (e.g., blue-green deploy when Decision N chose canary). Cite the Decision ID.

5. Open questions accounted for

Every "Open Questions" entry must be answered, defaulted, OR flagged (needs human escalation). Operational open questions left to runtime are how outages happen.

Mandate: The agent MUST verify the deployed pipeline is resilient under realistic failure modes and observable in production by someone who didn't build it.

Check

The agent MUST verify, and file feedback for any violation:

• Failure-recovery definition — Retry policy per stage (max attempts, backoff strategy, what counts as retryable), dead-letter destinations for unrecoverable records, and alert wiring on hard failures are all explicit, not implicit
• Resource sizing for peak — Memory, CPU, and parallelism are sized for the projected peak volume from the discovery brief's growth curve, not for current average load. Concurrency limits prevent overlapping runs of the same pipeline
• Monitoring breadth — Coverage includes pipeline-execution health (success rate, duration, retry counts), data health (rows landed per stage, freshness per target, validation pass rate), and resource trends (drift toward memory / CPU / duration limits)
• Alert actionability — Every alert has a severity (page / ticket / log-only) matched to impact, a route to a real on-call channel with a real schedule, and a runbook entry. Alerts that fire into the void provide false comfort
• Backfill readiness — A documented backfill procedure exists, has been tested in staging at realistic volume, preserves idempotency, and is rate-limited so a backfill doesn't overwhelm the production target
• Runbook actionability — For each of the most likely failure modes (source unavailable, schema drift, transformation timeout, validation failure, downstream-consumer breakage), the runbook covers symptoms, triage, recovery, rollback, and communication — concrete enough that an unfamiliar engineer can act on it
• First-run plan and rollback — Before initial production deployment, "good first run" criteria are explicit (specific row counts, validation results, latency), rollback triggers are explicit, and rollback steps are concrete

Common failure modes to look for

• A pipeline with retry policies but no defined dead-letter destination
• Resource limits set to "default" with no reference to actual peak volume
• Monitoring of "pipeline succeeded" without monitoring of "target stayed fresh" — a pipeline that emits zero rows successfully looks healthy
• Alerts routed to a chat channel nobody owns or that's muted
• A backfill procedure documented but never exercised in staging
• A runbook entry that says "investigate the failure" with no triage steps
• A first-run plan that says "ship it and see" with no specific success criteria and no rollback trigger
• An on-call schedule that has gaps (nights, weekends) without escalation paths defined