Execute Tests

Auto review

Execute tests and log defects

Hats
2
Review Agents
1
Review
Auto
Unit Types
Test Execution, Defect Logging
Inputs
Design Tests, Plan

Dependencies

Design Teststest-suite-spec
Plantest-strategy

Hat Sequence

1

Reporter

Focus: Log defects with complete reproduction information and maintain test execution metrics.

Responsibilities:

  • Create defect reports with reproduction steps, environment details, and severity classification
  • Include root cause hypothesis where determinable
  • Track test execution progress and coverage metrics
  • Categorize defects for efficient triage and assignment

Anti-patterns (RFC 2119):

  • The agent MUST NOT fil defects without reproduction steps or environment context
  • The agent MUST NOT misclassify defect severity based on personal judgment rather than defined criteria
  • The agent MUST track execution progress metrics during the test cycle
  • The agent MUST NOT fil duplicate defects without checking for existing reports
2

Tester

Focus: Execute test cases systematically, document results with evidence, and ensure test environment fidelity.

Responsibilities:

  • Execute test cases according to the test suite spec and test strategy priorities
  • Verify the test environment matches production conditions before execution
  • Capture evidence (screenshots, logs, output) for each test result
  • Identify and document any blocked or unexecutable tests with reasons

Anti-patterns (RFC 2119):

  • The agent MUST NOT execute tests in an environment that does not match production
  • The agent MUST NOT record pass/fail without capturing supporting evidence
  • The agent MUST NOT skip tests without documentation and approval
  • The agent MUST retest after environment issues are resolved

Review Agents

Evidence

Mandate: The agent MUST verify test execution is complete with evidence-backed results and properly documented defects.

Check:

  • The agent MUST verify that all planned tests are executed or have documented justification for skip
  • The agent MUST verify that test results include evidence (screenshots, logs) for failures
  • The agent MUST verify that defect reports have reproduction steps, environment details, and severity classification
  • The agent MUST verify that test environment matched production conditions during execution

Execute Tests

Criteria Guidance

Good criteria examples:

  • "Test results document pass/fail status for every test case with evidence (screenshots, logs, or output) for each failure"
  • "Defect reports include reproduction steps, environment details, severity classification, and root cause hypothesis"
  • "Coverage report confirms execution percentage against the planned test suite with justification for any unexecuted tests"

Bad criteria examples:

  • "Tests are run"
  • "Defects are logged"
  • "Testing is complete"

Completion Signal (RFC 2119)

Test results exist with all planned tests executed or justified as skipped, defects logged with severity and reproduction details, and coverage metrics documented. Tester MUST have confirmed test environment matched production conditions. Reporter MUST have categorized all defects and confirmed each MUST have sufficient information for resolution.