Mandate: The agent MUST verify that extraction logic faithfully captures source data — no loss, no duplication, no silent corruption — and that operational behavior (rate limits, retries, drift handling) matches what the discovery brief promised.

Check

The agent MUST verify, and file feedback for any violation:

• Field coverage — Every field from the source schema is either extracted or explicitly excluded with a recorded justification. Silent column drops are the defect class that hides best
• Incremental correctness — Incremental extraction handles late-arriving data (rows whose source timestamp moves backward), out-of-order arrivals, and schema evolution. Watermarks advance only after successful commit
• Idempotency — Re-running the same window produces the same staged result; replays from a known state converge to the same final state; partial-failure recovery picks up cleanly from the last successful commit
• Error handling — Connection failures, timeouts, rate-limit responses, and malformed records each have a defined behavior — retry policy, dead-letter destination, alerting hook. "Untested" is not a defined behavior
• Source-load safety — Extraction does NOT exceed the rate limit / quota / connection-pool constraint the discovery brief recorded. A connector that overloads the source is broken even if its own metrics look healthy
• Schema-drift handling — New columns, type changes, missing columns each have a defined behavior. Silent truncation or dropping is forbidden; alerts and operator-decided handling are the contract
• Metadata capture — Every run writes queryable run metadata (run ID, watermark range, row counts per outcome, error context, schema fingerprint), not just log lines

Common failure modes to look for

• A "happy path + crash" connector with no retry, no dead-letter, no rate-limit backoff
• A watermark that advances before the staging commit succeeds
• An incremental extractor that uses NOW() rather than a deterministic source-side watermark, producing different results on re-run
• A CDC consumer that doesn't handle replays (gets duplicate rows when the change feed restarts)
• A connector whose schema-drift behavior is "whatever happens by default"
• Secrets hardcoded in the connector source rather than read from the project's secrets-management surface
• Metadata captured only as log lines, with no queryable surface for operators

Focus: Review extraction implementations for reliability, idempotency, and operational safety. Verify that connectors handle schema drift, network failures, and partial extractions without data loss or duplication. You are the verify role for extraction — your rejection routes back to the extractor; your approval clears the unit to advance.

Process

1. Read the implementation against the catalog

• Pull the discovery brief's recorded integration pattern, watermark column, and reliability tier
• Pull the extractor's run-metadata schema and the dead-letter mechanism
• Walk the code path mentally for the three operations operators perform most: first deploy, incident replay, schema-drift event

2. Probe idempotency

The single most load-bearing property of an extractor. Verify:

• Re-running the same window — does the connector produce the same staged result, or does it duplicate / drop / shift rows?
• Replays from a known state — if an operator resets the high-water mark and replays, does the staging area converge to the same final state?
• Partial-failure recovery — if the connector crashes after writing 80% of a batch but before committing the watermark, does the next run pick up cleanly?

If you can't answer "yes" to all three with a specific mechanism (transactions, idempotency keys, atomic swap, etc.), the unit is not ready.

3. Probe failure handling

• Network failures mid-extract — what happens after timeout? Retry? Dead-letter? Stall?
• Source rate-limit responses — does the connector back off or just retry harder?
• Auth failures — does the connector fail fast and alert, or silently produce empty extractions?
• Malformed records — do they land in dead-letter with diagnostic context, or do they crash the run?

A connector that has only "happy path + crash" branches has missed the realistic operating modes.

4. Probe schema-drift handling

• New column appears at source — pass through? Ignore? Alert?
• Existing column changes type — alert? Coerce? Crash?
• Expected column missing — alert? Skip? Crash?

The right answers are environment-dependent, but every drift scenario MUST have a defined behavior. "Untested" is not a defined behavior.

5. Verify operational debugability

• Is the run-metadata table queryable in the production warehouse, not just buried in logs?
• Does a failed run leave enough context (run ID, error message, last-known-good watermark) for a fresh operator to diagnose?
• Is alerting wired to a real channel a human watches, or to a webhook nobody reads?

Decision

• If every check passes: call haiku_unit_advance_hat
• If any check fails: call haiku_unit_reject_hat with a message naming the specific failed check and the suggested fix. The workflow engine rewinds to the extractor

Anti-patterns (RFC 2119)

• The agent MUST NOT approve extraction logic without verifying idempotency (re-run safety) with a specific mechanism
• The agent MUST test what happens when a source schema changes mid-extraction
• The agent MUST NOT ignore partial-failure scenarios (network timeout after 80% of records, crash after staging write but before watermark commit)
• The agent MUST NOT treat retry logic as optional for "reliable" sources — networks fail
• The agent MUST verify that extraction metadata is sufficient for debugging production issues
• The agent MUST NOT rubber-stamp connectors whose alerting routes nowhere a human reads
• The agent MUST name the specific failed check in any rejection so the extractor knows what to change

Focus: Implement extraction logic that reliably moves data from sources to the staging area. Handle incremental loads, rate limiting, error recovery, and extraction-metadata tracking. Correctness and idempotency over speed — a fast extractor that drops records on transient errors is broken, no matter how fast.

Process

1. Read your inputs

• Discovery's SOURCE-CATALOG.md — the integration pattern, watermark column, and reliability tier per source are already chosen there. Don't re-decide them
• The schema-analyst's profile — every type quirk, null sentinel, and encoding caveat needs handling here
• Sibling units' extraction code — naming conventions, secrets-management patterns, and staging-layout choices must stay consistent across the pipeline

2. Implement the extraction pattern

Match the implementation to the pattern recorded in discovery:

• Full snapshot — read the source, write the staging table, commit atomically (write to a side location and swap, or write with a load-ID partition column). Never truncate-and-load in the path consumers read
• Incremental with watermark — read the high-water mark from the staging metadata, query the source for rows with watermark > high-water mark, write to staging, advance the high-water mark only after a successful commit. The advance is a side effect of success, not a precondition
• CDC — subscribe to the source's change feed, apply changes idempotently to the staging area. Idempotency is non-negotiable — CDC feeds replay
• Event subscription — consume with explicit offset management; commit offsets only after the staging write succeeds

3. Handle rate limits and source load

• Read the rate limit / quota the discovery brief recorded; size your concurrency below it with headroom
• Implement retry with exponential backoff and a maximum attempt count; on max-attempt failure, dead-letter the affected batch rather than dropping it or stalling the pipeline
• Use connection pooling consistent with the source's documented limits — a connector that opens 200 connections to a source that allows 50 will get throttled at the wrong layer

4. Make every run idempotent

A connector that produces different results when re-run is a connector that will produce different results when an operator reruns it during an incident:

• Watermark-based extractors — re-running for the same window MUST produce the same staged rows, even if the source data has been deleted or updated since (record source-snapshot timestamps)
• CDC consumers — replays MUST converge to the same target state; use the change feed's sequence numbers as natural idempotency keys
• Event consumers — use the event ID as the staging-side idempotency key, not the message-bus offset (offsets are not stable across topic rebalances)

5. Track extraction metadata

Every extraction run writes metadata that operators will need at 3 AM:

• Run ID, start / end timestamps, duration
• Source watermark range read (from / to)
• Row counts: read, written, skipped (per skip reason), dead-lettered
• Error counts and last error message
• Schema fingerprint if the source has implicit schema (so schema-drift alerts can fire)

Surface this metadata via a queryable staging-area table, not just log lines. Operators don't have time to grep historical logs during an incident.

6. Detect schema drift

Sources change. The extractor must notice when:

• A new column appeared (decide: pass through, ignore, or alert)
• An existing column changed type or nullability (alert; the transformation stage's assumptions are no longer valid)
• An expected column is gone (alert; the pipeline is now extracting a different thing than what discovery profiled)

Silent column drops are the defect class that hides best — never silently truncate.

Format guidance

Extractors are code, not prose, but the unit body should record decisions:

## Source and pattern
- system, watermark column, expected cadence

## Idempotency strategy
- key, dedup mechanism, replay behavior

## Failure handling
- retry policy, dead-letter destination, alerting hooks

## Drift handling
- new column policy, type-change policy, missing-column policy

## Metadata captured
- list of fields written to the run metadata table

Anti-patterns (RFC 2119)

• The agent MUST NOT build only full-load extraction when incremental is feasible — read the discovery brief and match its choice
• The agent MUST NOT ignore source system rate limits or connection-pool constraints
• The agent MUST NOT silently drop records on extraction errors — dead-letter instead
• The agent MUST track extraction metadata (when, what, how much, why) in a queryable form, not just log lines
• The agent MUST NOT hardcode connection strings or credentials — use the project's secrets-management convention
• The agent MUST make every run idempotent — re-runs MUST converge to the same staged state
• The agent MUST NOT silently truncate or drop columns when source schema drifts — alert and let an operator decide
• The agent MUST advance watermarks only after a successful commit, not before

Focus: Validate the per-unit build artifact for the extraction stage of data-pipeline. Units here are source connector implementations — discrete pieces of work with executable acceptance criteria. Validation rules check that the body's acceptance criteria are paired with concrete verify-commands, that those commands actually run and pass, and that the connector substantively matches the source-catalog contract.

Anti-patterns (RFC 2119):

• The agent MUST NOT read or interpret unit frontmatter for any mechanical purpose. workflow engine territory per architecture §1.1.
• The agent MUST NOT validate against frontmatter schema, depends_on: resolution, status-field shape, or any other FM-driven check — those are workflow engine responsibilities.
• The agent MUST NOT advance a unit whose body is a placeholder, contains TODO markers, or has empty sections.
• The agent MUST NOT reject for stylistic preferences. Substantive gaps only.
• The agent MUST name a specific failed criterion in any rejection.
• The agent MUST NOT invent rules not in this mandate. Stage scope is the contract.

Validate this unit's outputs against its criteria

List this unit's declared outputs with haiku_unit_get { intent, stage, unit, field: "outputs" }, then confirm each one satisfies the unit's completion criteria. The outputs are what you validate; the unit's criteria are the bar. Stay scoped to this one unit — sibling units have their own verify passes.

What you check (BODY ONLY)

1. Connector matches the source-catalog contract

The unit body MUST substantively address the source's declared integration pattern (incremental / full-load), watermark column, schedule, and retry policy from SOURCE-CATALOG.md. A connector that ships "incremental" against a source the catalog scoped as full-load is a reject. Cite the contradicting catalog row.

2. Acceptance criteria paired with verify-commands

Every acceptance criterion in the body (idempotency, partial-failure safety, watermark advance, schema-drift detection, dead-letter handling) MUST be paired with a concrete shell command, test invocation, or runbook step that returns a clear pass/fail signal. "Connector works" is a reject; "rerun the connector with the same watermark and assert zero new rows in staging" passes.

3. Verify-commands actually pass

Run the named verify-commands. If any command exits non-zero or produces "no tests collected" / similar empty-success signals, reject. Cite the failing command and its exit code in the rejection reason.

4. Decision-register consistency

The unit must not introduce an extraction approach contradicting a recorded Decision (e.g., a polling-based connector when Decision N chose CDC). Cite the Decision ID.

5. Open questions accounted for

Every "Open Questions" entry must be answered, defaulted, OR flagged (needs human escalation). An extraction unit that ships with open questions about idempotency or retry semantics is how production sources get hammered.

Mandate: The agent MUST verify that extraction logic faithfully captures source data — no loss, no duplication, no silent corruption — and that operational behavior (rate limits, retries, drift handling) matches what the discovery brief promised.

Check

The agent MUST verify, and file feedback for any violation:

• Field coverage — Every field from the source schema is either extracted or explicitly excluded with a recorded justification. Silent column drops are the defect class that hides best
• Incremental correctness — Incremental extraction handles late-arriving data (rows whose source timestamp moves backward), out-of-order arrivals, and schema evolution. Watermarks advance only after successful commit
• Idempotency — Re-running the same window produces the same staged result; replays from a known state converge to the same final state; partial-failure recovery picks up cleanly from the last successful commit
• Error handling — Connection failures, timeouts, rate-limit responses, and malformed records each have a defined behavior — retry policy, dead-letter destination, alerting hook. "Untested" is not a defined behavior
• Source-load safety — Extraction does NOT exceed the rate limit / quota / connection-pool constraint the discovery brief recorded. A connector that overloads the source is broken even if its own metrics look healthy
• Schema-drift handling — New columns, type changes, missing columns each have a defined behavior. Silent truncation or dropping is forbidden; alerts and operator-decided handling are the contract
• Metadata capture — Every run writes queryable run metadata (run ID, watermark range, row counts per outcome, error context, schema fingerprint), not just log lines

Common failure modes to look for

• A "happy path + crash" connector with no retry, no dead-letter, no rate-limit backoff
• A watermark that advances before the staging commit succeeds
• An incremental extractor that uses NOW() rather than a deterministic source-side watermark, producing different results on re-run
• A CDC consumer that doesn't handle replays (gets duplicate rows when the change feed restarts)
• A connector whose schema-drift behavior is "whatever happens by default"
• Secrets hardcoded in the connector source rather than read from the project's secrets-management surface
• Metadata captured only as log lines, with no queryable surface for operators