Analyze

Ask review

Analyze test results and compute quality metrics

Hats
2
Review Agents
1
Review
Ask
Unit Types
Results Analysis, Quality Metrics
Inputs
Execute Tests, Plan

Dependencies

Execute Teststest-results
Plantest-strategy

Hat Sequence

1

Analyst

Focus: Analyze test results to identify quality patterns, root causes, and actionable improvement opportunities.

Responsibilities:

  • Compute quality metrics (defect density, severity distribution, pass rates)
  • Identify defect patterns and clusters that indicate systemic issues
  • Perform root cause analysis grouping defects by category
  • Produce a quality report with findings and improvement recommendations

Anti-patterns (RFC 2119):

  • The agent MUST NOT report metrics without analyzing what they mean
  • The agent MUST NOT treat each defect in isolation without looking for patterns
  • The agent MUST NOT comput averages that mask important variation
  • The agent MUST NOT produce analysis that is descriptive but not actionable
2

Statistician

Focus: Validate quality metrics with statistical rigor and ensure trend analysis is sound.

Responsibilities:

  • Validate metric calculations and confirm data integrity
  • Apply statistical methods to trend analysis to distinguish signal from noise
  • Compare current release quality against historical baselines with significance testing
  • Identify whether quality differences between components or releases are statistically meaningful

Anti-patterns (RFC 2119):

  • The agent MUST NOT present trends without enough data points to be meaningful
  • The agent MUST NOT draw conclusions from metrics without considering sample sizes
  • The agent MUST NOT compar releases without controlling for scope or complexity differences
  • The agent MUST NOT use complex statistics when simple descriptive metrics would be more useful

Review Agents

Insight

Mandate: The agent MUST verify quality analysis produces actionable insights, not just metrics.

Check:

  • The agent MUST verify that quality metrics are accurately computed with verified underlying data
  • The agent MUST verify that defect pattern analysis identifies root causes, not just categories
  • The agent MUST verify that trend comparisons use appropriate baselines and account for scope differences
  • The agent MUST verify that improvement recommendations are specific and prioritized by impact

Analyze

Criteria Guidance

Good criteria examples:

  • "Quality report includes defect density, severity distribution, and trend analysis compared to previous releases"
  • "Root cause analysis groups defects into categories (design, code, environment, data) with distribution percentages"
  • "Risk assessment maps unresolved defects to business impact with recommendation for release, defer, or block"

Bad criteria examples:

  • "Results are analyzed"
  • "Metrics are computed"
  • "Quality is assessed"

Completion Signal (RFC 2119)

Quality report MUST exist with metrics computed, defect patterns analyzed, and risk assessment complete. Analyst MUST have identified systemic quality issues and their root causes. Statistician MUST have validated metric calculations and confirmed trend analysis is statistically sound.