add bmad techniques to the project

This commit is contained in:
2026-03-12 13:34:41 +01:00
parent ca80c3e6c7
commit e8052abf79
516 changed files with 88575 additions and 0 deletions

View File

@@ -0,0 +1,74 @@
# TEA Workflow Step Files
This folder contains the Test Architect (TEA) workflows converted to step-file architecture for strict LLM compliance. Each workflow is tri-modal (create, edit, validate) and uses small, ordered step files instead of a single monolithic instruction file.
## Why Step Files
- Enforces sequential execution and prevents improvisation
- Keeps context small and focused per step
- Makes validation and edits deterministic
## Standard Layout (per workflow)
```
<workflow>/
├── workflow.md # Mode routing (create / edit / validate)
├── workflow-plan.md # Design reference for step order and intent
├── workflow.yaml # Installer metadata
├── instructions.md # Short entrypoint / summary
├── checklist.md # Validation criteria for outputs
├── steps-c/ # Create mode steps
├── steps-e/ # Edit mode steps
├── steps-v/ # Validate mode steps
├── templates/ # Output templates (if applicable)
└── validation-report-*.md # Validator outputs (latest run)
```
## Modes
- **Create (steps-c/):** Primary execution flow to generate outputs
- **Edit (steps-e/):** Structured edits to existing outputs
- **Validate (steps-v/):** Checklist-based validation of outputs
## Execution Rules (Summary)
- Load **one step at a time**. Do not preload future steps.
- Follow the **MANDATORY SEQUENCE** exactly in each step.
- Do not skip steps, reorder, or improvise.
- If a step writes outputs, do so **before** loading the next step.
## Step Naming Conventions
- `step-01-*.md` is the init step (no menus unless explicitly required).
- `step-01b-*.md` is a continuation/resume step if the workflow is continuable.
- `step-0X-*.md` are sequential create-mode steps.
- `steps-v/step-01-validate.md` is the validate mode entrypoint.
- `steps-e/step-01-assess.md` is the edit mode entrypoint.
## Validation
- Each workflow has a latest `validation-report-*.md` in its folder.
- Validation uses the BMad Builder workflow validator (workflow-builder).
- The goal is 100% compliance with no warnings.
## References
- Step-file architecture: `docs/explanation/step-file-architecture.md`
- Subagent patterns: `docs/explanation/subagent-architecture.md`
## TEA Workflows
- test-design
- automate
- atdd
- test-review
- trace
- framework
- ci
- nfr-assess
## Notes
- `workflow.md` is the canonical entrypoint. `instructions.md` is a short summary for quick context.
- Output files typically use `{test_artifacts}` or `{project-root}` variables.
- If a workflow produces multiple artifacts (e.g., system-level vs epic-level), the step file will specify which templates and output paths to use.

View File

@@ -0,0 +1,371 @@
---
stepsCompleted: []
lastStep: ''
lastSaved: ''
workflowType: 'testarch-atdd'
inputDocuments: []
---
# ATDD Checklist - Epic {epic_num}, Story {story_num}: {story_title}
**Date:** {date}
**Author:** {user_name}
**Primary Test Level:** {primary_level}
---
## Story Summary
{Brief 2-3 sentence summary of the user story}
**As a** {user_role}
**I want** {feature_description}
**So that** {business_value}
---
## Acceptance Criteria
{List all testable acceptance criteria from the story}
1. {Acceptance criterion 1}
2. {Acceptance criterion 2}
3. {Acceptance criterion 3}
---
## Failing Tests Created (RED Phase)
### E2E Tests ({e2e_test_count} tests)
**File:** `{e2e_test_file_path}` ({line_count} lines)
{List each E2E test with its current status and expected failure reason}
-**Test:** {test_name}
- **Status:** RED - {failure_reason}
- **Verifies:** {what_this_test_validates}
### API Tests ({api_test_count} tests)
**File:** `{api_test_file_path}` ({line_count} lines)
{List each API test with its current status and expected failure reason}
-**Test:** {test_name}
- **Status:** RED - {failure_reason}
- **Verifies:** {what_this_test_validates}
### Component Tests ({component_test_count} tests)
**File:** `{component_test_file_path}` ({line_count} lines)
{List each component test with its current status and expected failure reason}
-**Test:** {test_name}
- **Status:** RED - {failure_reason}
- **Verifies:** {what_this_test_validates}
---
## Data Factories Created
{List all data factory files created with their exports}
### {Entity} Factory
**File:** `tests/support/factories/{entity}.factory.ts`
**Exports:**
- `create{Entity}(overrides?)` - Create single entity with optional overrides
- `create{Entity}s(count)` - Create array of entities
**Example Usage:**
```typescript
const user = createUser({ email: 'specific@example.com' });
const users = createUsers(5); // Generate 5 random users
```
---
## Fixtures Created
{List all test fixture files created with their fixture names and descriptions}
### {Feature} Fixtures
**File:** `tests/support/fixtures/{feature}.fixture.ts`
**Fixtures:**
- `{fixtureName}` - {description_of_what_fixture_provides}
- **Setup:** {what_setup_does}
- **Provides:** {what_test_receives}
- **Cleanup:** {what_cleanup_does}
**Example Usage:**
```typescript
import { test } from './fixtures/{feature}.fixture';
test('should do something', async ({ {fixtureName} }) => {
// {fixtureName} is ready to use with auto-cleanup
});
```
---
## Mock Requirements
{Document external services that need mocking and their requirements}
### {Service Name} Mock
**Endpoint:** `{HTTP_METHOD} {endpoint_url}`
**Success Response:**
```json
{
{success_response_example}
}
```
**Failure Response:**
```json
{
{failure_response_example}
}
```
**Notes:** {any_special_mock_requirements}
---
## Required data-testid Attributes
{List all data-testid attributes required in UI implementation for test stability}
### {Page or Component Name}
- `{data-testid-name}` - {description_of_element}
- `{data-testid-name}` - {description_of_element}
**Implementation Example:**
```tsx
<button data-testid="login-button">Log In</button>
<input data-testid="email-input" type="email" />
<div data-testid="error-message">{errorText}</div>
```
---
## Implementation Checklist
{Map each failing test to concrete implementation tasks that will make it pass}
### Test: {test_name_1}
**File:** `{test_file_path}`
**Tasks to make this test pass:**
- [ ] {Implementation task 1}
- [ ] {Implementation task 2}
- [ ] {Implementation task 3}
- [ ] Add required data-testid attributes: {list_of_testids}
- [ ] Run test: `{test_execution_command}`
- [ ] ✅ Test passes (green phase)
**Estimated Effort:** {effort_estimate} hours
---
### Test: {test_name_2}
**File:** `{test_file_path}`
**Tasks to make this test pass:**
- [ ] {Implementation task 1}
- [ ] {Implementation task 2}
- [ ] {Implementation task 3}
- [ ] Add required data-testid attributes: {list_of_testids}
- [ ] Run test: `{test_execution_command}`
- [ ] ✅ Test passes (green phase)
**Estimated Effort:** {effort_estimate} hours
---
## Running Tests
```bash
# Run all failing tests for this story
{test_command_all}
# Run specific test file
{test_command_specific_file}
# Run tests in headed mode (see browser)
{test_command_headed}
# Debug specific test
{test_command_debug}
# Run tests with coverage
{test_command_coverage}
```
---
## Red-Green-Refactor Workflow
### RED Phase (Complete) ✅
**TEA Agent Responsibilities:**
- ✅ All tests written and failing
- ✅ Fixtures and factories created with auto-cleanup
- ✅ Mock requirements documented
- ✅ data-testid requirements listed
- ✅ Implementation checklist created
**Verification:**
- All tests run and fail as expected
- Failure messages are clear and actionable
- Tests fail due to missing implementation, not test bugs
---
### GREEN Phase (DEV Team - Next Steps)
**DEV Agent Responsibilities:**
1. **Pick one failing test** from implementation checklist (start with highest priority)
2. **Read the test** to understand expected behavior
3. **Implement minimal code** to make that specific test pass
4. **Run the test** to verify it now passes (green)
5. **Check off the task** in implementation checklist
6. **Move to next test** and repeat
**Key Principles:**
- One test at a time (don't try to fix all at once)
- Minimal implementation (don't over-engineer)
- Run tests frequently (immediate feedback)
- Use implementation checklist as roadmap
**Progress Tracking:**
- Check off tasks as you complete them
- Share progress in daily standup
---
### REFACTOR Phase (DEV Team - After All Tests Pass)
**DEV Agent Responsibilities:**
1. **Verify all tests pass** (green phase complete)
2. **Review code for quality** (readability, maintainability, performance)
3. **Extract duplications** (DRY principle)
4. **Optimize performance** (if needed)
5. **Ensure tests still pass** after each refactor
6. **Update documentation** (if API contracts change)
**Key Principles:**
- Tests provide safety net (refactor with confidence)
- Make small refactors (easier to debug if tests fail)
- Run tests after each change
- Don't change test behavior (only implementation)
**Completion:**
- All tests pass
- Code quality meets team standards
- No duplications or code smells
- Ready for code review and story approval
---
## Next Steps
1. **Share this checklist and failing tests** with the dev workflow (manual handoff)
2. **Review this checklist** with team in standup or planning
3. **Run failing tests** to confirm RED phase: `{test_command_all}`
4. **Begin implementation** using implementation checklist as guide
5. **Work one test at a time** (red → green for each)
6. **Share progress** in daily standup
7. **When all tests pass**, refactor code for quality
8. **When refactoring complete**, manually update story status to 'done' in sprint-status.yaml
---
## Knowledge Base References Applied
This ATDD workflow consulted the following knowledge fragments:
- **fixture-architecture.md** - Test fixture patterns with setup/teardown and auto-cleanup using Playwright's `test.extend()`
- **data-factories.md** - Factory patterns using `@faker-js/faker` for random test data generation with overrides support
- **component-tdd.md** - Component test strategies using Playwright Component Testing
- **network-first.md** - Route interception patterns (intercept BEFORE navigation to prevent race conditions)
- **test-quality.md** - Test design principles (Given-When-Then, one assertion per test, determinism, isolation)
- **test-levels-framework.md** - Test level selection framework (E2E vs API vs Component vs Unit)
See `tea-index.csv` for complete knowledge fragment mapping.
---
## Test Execution Evidence
### Initial Test Run (RED Phase Verification)
**Command:** `{test_command_all}`
**Results:**
```
{paste_test_run_output_showing_all_tests_failing}
```
**Summary:**
- Total tests: {total_test_count}
- Passing: 0 (expected)
- Failing: {total_test_count} (expected)
- Status: ✅ RED phase verified
**Expected Failure Messages:**
{list_expected_failure_messages_for_each_test}
---
## Notes
{Any additional notes, context, or special considerations for this story}
- {Note 1}
- {Note 2}
- {Note 3}
---
## Contact
**Questions or Issues?**
- Ask in team standup
- Tag @{tea_agent_username} in Slack/Discord
- Refer to `./bmm/docs/tea-README.md` for workflow documentation
- Consult `./bmm/testarch/knowledge` for testing best practices
---
**Generated by BMad TEA Agent** - {date}

View File

@@ -0,0 +1,374 @@
# ATDD Workflow Validation Checklist
Use this checklist to validate that the ATDD workflow has been executed correctly and all deliverables meet quality standards.
## Prerequisites
Before starting this workflow, verify:
- [ ] Story approved with clear acceptance criteria (AC must be testable)
- [ ] Development sandbox/environment ready
- [ ] Framework scaffolding exists (run `framework` workflow if missing)
- [ ] Test framework configuration available (playwright.config.ts or cypress.config.ts)
- [ ] Package.json has test dependencies installed (Playwright or Cypress)
**Halt if missing:** Framework scaffolding or story acceptance criteria
---
## Step 1: Story Context and Requirements
- [ ] Story markdown file loaded and parsed successfully
- [ ] All acceptance criteria identified and extracted
- [ ] Affected systems and components identified
- [ ] Technical constraints documented
- [ ] Framework configuration loaded (playwright.config.ts or cypress.config.ts)
- [ ] Test directory structure identified from config
- [ ] Existing fixture patterns reviewed for consistency
- [ ] Similar test patterns searched and found in `{test_dir}`
- [ ] Knowledge base fragments loaded:
- [ ] `fixture-architecture.md`
- [ ] `data-factories.md`
- [ ] `component-tdd.md`
- [ ] `network-first.md`
- [ ] `test-quality.md`
---
## Step 2: Test Level Selection and Strategy
- [ ] Each acceptance criterion analyzed for appropriate test level
- [ ] Test level selection framework applied (E2E vs API vs Component vs Unit)
- [ ] E2E tests: Critical user journeys and multi-system integration identified
- [ ] API tests: Business logic and service contracts identified
- [ ] Component tests: UI component behavior and interactions identified
- [ ] Unit tests: Pure logic and edge cases identified (if applicable)
- [ ] Duplicate coverage avoided (same behavior not tested at multiple levels unnecessarily)
- [ ] Tests prioritized using P0-P3 framework (if test-design document exists)
- [ ] Primary test level set in `primary_level` variable (typically E2E or API)
- [ ] Test levels documented in ATDD checklist
---
## Step 3: Failing Tests Generated
### Test File Structure Created
- [ ] Test files organized in appropriate directories:
- [ ] `tests/e2e/` for end-to-end tests
- [ ] `tests/api/` for API tests
- [ ] `tests/component/` for component tests
- [ ] `tests/support/` for infrastructure (fixtures, factories, helpers)
### E2E Tests (If Applicable)
- [ ] E2E test files created in `tests/e2e/`
- [ ] All tests follow Given-When-Then format
- [ ] Tests use `data-testid` selectors (not CSS classes or fragile selectors)
- [ ] One assertion per test (atomic test design)
- [ ] No hard waits or sleeps (explicit waits only)
- [ ] Network-first pattern applied (route interception BEFORE navigation)
- [ ] Tests fail initially (RED phase verified by local test run)
- [ ] Failure messages are clear and actionable
### API Tests (If Applicable)
- [ ] API test files created in `tests/api/`
- [ ] Tests follow Given-When-Then format
- [ ] API contracts validated (request/response structure)
- [ ] HTTP status codes verified
- [ ] Response body validation includes all required fields
- [ ] Error cases tested (400, 401, 403, 404, 500)
- [ ] Tests fail initially (RED phase verified)
### Component Tests (If Applicable)
- [ ] Component test files created in `tests/component/`
- [ ] Tests follow Given-When-Then format
- [ ] Component mounting works correctly
- [ ] Interaction testing covers user actions (click, hover, keyboard)
- [ ] State management within component validated
- [ ] Props and events tested
- [ ] Tests fail initially (RED phase verified)
### Test Quality Validation
- [ ] All tests use Given-When-Then structure with clear comments
- [ ] All tests have descriptive names explaining what they test
- [ ] No duplicate tests (same behavior tested multiple times)
- [ ] No flaky patterns (race conditions, timing issues)
- [ ] No test interdependencies (tests can run in any order)
- [ ] Tests are deterministic (same input always produces same result)
---
## Step 4: Data Infrastructure Built
### Data Factories Created
- [ ] Factory files created in `tests/support/factories/`
- [ ] All factories use `@faker-js/faker` for random data generation (no hardcoded values)
- [ ] Factories support overrides for specific test scenarios
- [ ] Factories generate complete valid objects matching API contracts
- [ ] Helper functions for bulk creation provided (e.g., `createUsers(count)`)
- [ ] Factory exports are properly typed (TypeScript)
### Test Fixtures Created
- [ ] Fixture files created in `tests/support/fixtures/`
- [ ] All fixtures use Playwright's `test.extend()` pattern
- [ ] Fixtures have setup phase (arrange test preconditions)
- [ ] Fixtures provide data to tests via `await use(data)`
- [ ] Fixtures have teardown phase with auto-cleanup (delete created data)
- [ ] Fixtures are composable (can use other fixtures if needed)
- [ ] Fixtures are isolated (each test gets fresh data)
- [ ] Fixtures are type-safe (TypeScript types defined)
### Mock Requirements Documented
- [ ] External service mocking requirements identified
- [ ] Mock endpoints documented with URLs and methods
- [ ] Success response examples provided
- [ ] Failure response examples provided
- [ ] Mock requirements documented in ATDD checklist for DEV team
### data-testid Requirements Listed
- [ ] All required data-testid attributes identified from E2E tests
- [ ] data-testid list organized by page or component
- [ ] Each data-testid has clear description of element it targets
- [ ] data-testid list included in ATDD checklist for DEV team
---
## Step 5: Implementation Checklist Created
- [ ] Implementation checklist created with clear structure
- [ ] Each failing test mapped to concrete implementation tasks
- [ ] Tasks include:
- [ ] Route/component creation
- [ ] Business logic implementation
- [ ] API integration
- [ ] data-testid attribute additions
- [ ] Error handling
- [ ] Test execution command
- [ ] Completion checkbox
- [ ] Red-Green-Refactor workflow documented in checklist
- [ ] RED phase marked as complete (TEA responsibility)
- [ ] GREEN phase tasks listed for DEV team
- [ ] REFACTOR phase guidance provided
- [ ] Execution commands provided:
- [ ] Run all tests: `npm run test:e2e`
- [ ] Run specific test file
- [ ] Run in headed mode
- [ ] Debug specific test
- [ ] Estimated effort included (hours or story points)
---
## Step 6: Deliverables Generated
### ATDD Checklist Document Created
- [ ] Output file created at `{test_artifacts}/atdd-checklist-{story_id}.md`
- [ ] Document follows template structure from `atdd-checklist-template.md`
- [ ] Document includes all required sections:
- [ ] Story summary
- [ ] Acceptance criteria breakdown
- [ ] Failing tests created (paths and line counts)
- [ ] Data factories created
- [ ] Fixtures created
- [ ] Mock requirements
- [ ] Required data-testid attributes
- [ ] Implementation checklist
- [ ] Red-green-refactor workflow
- [ ] Execution commands
- [ ] Next steps for DEV team
- [ ] Output shared with DEV workflow (manual handoff; not auto-consumed)
### All Tests Verified to Fail (RED Phase)
- [ ] Full test suite run locally before finalizing
- [ ] All tests fail as expected (RED phase confirmed)
- [ ] No tests passing before implementation (if passing, test is invalid)
- [ ] Failure messages documented in ATDD checklist
- [ ] Failures are due to missing implementation, not test bugs
- [ ] Test run output captured for reference
### Summary Provided
- [ ] Summary includes:
- [ ] Story ID
- [ ] Primary test level
- [ ] Test counts (E2E, API, Component)
- [ ] Test file paths
- [ ] Factory count
- [ ] Fixture count
- [ ] Mock requirements count
- [ ] data-testid count
- [ ] Implementation task count
- [ ] Estimated effort
- [ ] Next steps for DEV team
- [ ] Output file path
- [ ] Knowledge base references applied
---
## Quality Checks
### Test Design Quality
- [ ] Tests are readable (clear Given-When-Then structure)
- [ ] Tests are maintainable (use factories and fixtures, not hardcoded data)
- [ ] Tests are isolated (no shared state between tests)
- [ ] Tests are deterministic (no race conditions or flaky patterns)
- [ ] Tests are atomic (one assertion per test)
- [ ] Tests are fast (no unnecessary waits or delays)
### Knowledge Base Integration
- [ ] fixture-architecture.md patterns applied to all fixtures
- [ ] data-factories.md patterns applied to all factories
- [ ] network-first.md patterns applied to E2E tests with network requests
- [ ] component-tdd.md patterns applied to component tests
- [ ] test-quality.md principles applied to all test design
### Code Quality
- [ ] All TypeScript types are correct and complete
- [ ] No linting errors in generated test files
- [ ] Consistent naming conventions followed
- [ ] Imports are organized and correct
- [ ] Code follows project style guide
---
## Integration Points
### With DEV Agent
- [ ] ATDD checklist provides clear implementation guidance
- [ ] Implementation tasks are granular and actionable
- [ ] data-testid requirements are complete and clear
- [ ] Mock requirements include all necessary details
- [ ] Execution commands work correctly
### With Story Workflow
- [ ] Story ID correctly referenced in output files
- [ ] Acceptance criteria from story accurately reflected in tests
- [ ] Technical constraints from story considered in test design
### With Framework Workflow
- [ ] Test framework configuration correctly detected and used
- [ ] Directory structure matches framework setup
- [ ] Fixtures and helpers follow established patterns
- [ ] Naming conventions consistent with framework standards
### With test-design Workflow (If Available)
- [ ] P0 scenarios from test-design prioritized in ATDD
- [ ] Risk assessment from test-design considered in test coverage
- [ ] Coverage strategy from test-design aligned with ATDD tests
---
## Completion Criteria
All of the following must be true before marking this workflow as complete:
- [ ] **Story acceptance criteria analyzed** and mapped to appropriate test levels
- [ ] **Failing tests created** at all appropriate levels (E2E, API, Component)
- [ ] **Given-When-Then format** used consistently across all tests
- [ ] **RED phase verified** by local test run (all tests failing as expected)
- [ ] **Network-first pattern** applied to E2E tests with network requests
- [ ] **Data factories created** using faker (no hardcoded test data)
- [ ] **Fixtures created** with auto-cleanup in teardown
- [ ] **Mock requirements documented** for external services
- [ ] **data-testid attributes listed** for DEV team
- [ ] **Implementation checklist created** mapping tests to code tasks
- [ ] **Red-green-refactor workflow documented** in ATDD checklist
- [ ] **Execution commands provided** and verified to work
- [ ] **ATDD checklist document created** and saved to correct location
- [ ] **Output file formatted correctly** using template structure
- [ ] **Knowledge base references applied** and documented in summary
- [ ] **No test quality issues** (flaky patterns, race conditions, hardcoded data)
---
## Common Issues and Resolutions
### Issue: Tests pass before implementation
**Problem:** A test passes even though no implementation code exists yet.
**Resolution:**
- Review test to ensure it's testing actual behavior, not mocked/stubbed behavior
- Check if test is accidentally using existing functionality
- Verify test assertions are correct and meaningful
- Rewrite test to fail until implementation is complete
### Issue: Network-first pattern not applied
**Problem:** Route interception happens after navigation, causing race conditions.
**Resolution:**
- Move `await page.route()` calls BEFORE `await page.goto()`
- Review `network-first.md` knowledge fragment
- Update all E2E tests to follow network-first pattern
### Issue: Hardcoded test data in tests
**Problem:** Tests use hardcoded strings/numbers instead of factories.
**Resolution:**
- Replace all hardcoded data with factory function calls
- Use `faker` for all random data generation
- Update data-factories to support all required test scenarios
### Issue: Fixtures missing auto-cleanup
**Problem:** Fixtures create data but don't clean it up in teardown.
**Resolution:**
- Add cleanup logic after `await use(data)` in fixture
- Call deletion/cleanup functions in teardown
- Verify cleanup works by checking database/storage after test run
### Issue: Tests have multiple assertions
**Problem:** Tests verify multiple behaviors in single test (not atomic).
**Resolution:**
- Split into separate tests (one assertion per test)
- Each test should verify exactly one behavior
- Use descriptive test names to clarify what each test verifies
### Issue: Tests depend on execution order
**Problem:** Tests fail when run in isolation or different order.
**Resolution:**
- Remove shared state between tests
- Each test should create its own test data
- Use fixtures for consistent setup across tests
- Verify tests can run with `.only` flag
---
## Notes for TEA Agent
- **Preflight halt is critical:** Do not proceed if story has no acceptance criteria or framework is missing
- **RED phase verification is mandatory:** Tests must fail before sharing with DEV team
- **Network-first pattern:** Route interception BEFORE navigation prevents race conditions
- **One assertion per test:** Atomic tests provide clear failure diagnosis
- **Auto-cleanup is non-negotiable:** Every fixture must clean up data in teardown
- **Use knowledge base:** Load relevant fragments (fixture-architecture, data-factories, network-first, component-tdd, test-quality) for guidance
- **Share with DEV agent:** ATDD checklist provides implementation roadmap from red to green

View File

@@ -0,0 +1,45 @@
<!-- Powered by BMAD-CORE™ -->
# Acceptance Test-Driven Development (ATDD)
**Workflow ID**: `_bmad/tea/testarch/atdd`
**Version**: 5.0 (Step-File Architecture)
---
## Overview
Generates **failing acceptance tests** before implementation (TDD red phase), plus an implementation checklist. Produces tests at appropriate levels (E2E/API/Component) with supporting fixtures and helpers.
---
## WORKFLOW ARCHITECTURE
This workflow uses **step-file architecture**:
- **Micro-file Design**: Each step is self-contained
- **JIT Loading**: Only the current step file is in memory
- **Sequential Enforcement**: Execute steps in order without skipping
---
## INITIALIZATION SEQUENCE
### 1. Configuration Loading
From `workflow.yaml`, resolve:
- `config_source`, `test_artifacts`, `user_name`, `communication_language`, `document_output_language`, `date`
- `test_dir`
### 2. First Step
Load, read completely, and execute:
`{project-root}/_bmad/tea/workflows/testarch/atdd/steps-c/step-01-preflight-and-context.md`
### 3. Resume Support
If the user selects **Resume** mode, load, read completely, and execute:
`{project-root}/_bmad/tea/workflows/testarch/atdd/steps-c/step-01b-resume.md`
This checks the output document for progress tracking frontmatter and routes to the next incomplete step.

View File

@@ -0,0 +1,226 @@
---
name: 'step-01-preflight-and-context'
description: 'Verify prerequisites and load story, framework, and knowledge base'
outputFile: '{test_artifacts}/atdd-checklist-{story_id}.md'
nextStepFile: './step-02-generation-mode.md'
knowledgeIndex: '{project-root}/_bmad/tea/testarch/tea-index.csv'
---
# Step 1: Preflight & Context Loading
## STEP GOAL
Verify prerequisites and load all required inputs before generating failing tests.
## MANDATORY EXECUTION RULES
- 📖 Read the entire step file before acting
- ✅ Speak in `{communication_language}`
- 🚫 Halt if requirements are missing
---
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
- 💾 Record outputs before proceeding
- 📖 Load the next step only when instructed
## CONTEXT BOUNDARIES:
- Available context: config, loaded artifacts, and knowledge fragments
- Focus: this step's goal only
- Limits: do not execute future steps
- Dependencies: prior steps' outputs (if any)
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
## 1. Stack Detection
**Read `config.test_stack_type`** from `{config_source}`.
**Auto-Detection Algorithm** (when `test_stack_type` is `"auto"` or not configured):
- Scan `{project-root}` for project manifests:
- **Frontend indicators**: `package.json` with react/vue/angular/next dependencies, `playwright.config.*`, `vite.config.*`, `webpack.config.*`
- **Backend indicators**: `pyproject.toml`, `pom.xml`/`build.gradle`, `go.mod`, `*.csproj`/`*.sln`, `Gemfile`, `Cargo.toml`
- **Both present** = `fullstack`; only frontend = `frontend`; only backend = `backend`
- Explicit `test_stack_type` config value overrides auto-detection
- **Backward compatibility**: if `test_stack_type` is not in config, treat as `"auto"` (preserves current frontend behavior for existing installs)
Store result as `{detected_stack}` = `frontend` | `backend` | `fullstack`
---
## 2. Prerequisites (Hard Requirements)
- Story approved with **clear acceptance criteria**
- Test framework configured:
- **If {detected_stack} is `frontend` or `fullstack`:** `playwright.config.ts` or `cypress.config.ts`
- **If {detected_stack} is `backend`:** relevant test config exists (e.g., `conftest.py`, `src/test/`, `*_test.go`, `.rspec`)
- Development environment available
If any are missing: **HALT** and notify the user.
---
## 3. Load Story Context
- Read story markdown from `{story_file}` (or ask user if not provided)
- Extract acceptance criteria and constraints
- Identify affected components and integrations
---
## 4. Load Framework & Existing Patterns
- Read framework config
- Inspect `{test_dir}` for existing test patterns, fixtures, helpers
## 4.5 Read TEA Config Flags
From `{config_source}`:
- `tea_use_playwright_utils`
- `tea_use_pactjs_utils`
- `tea_pact_mcp`
- `tea_browser_automation`
- `test_stack_type`
---
### Tiered Knowledge Loading
Load fragments based on their `tier` classification in `tea-index.csv`:
1. **Core tier** (always load): Foundational fragments required for this workflow
2. **Extended tier** (load on-demand): Load when deeper analysis is needed or when the user's context requires it
3. **Specialized tier** (load only when relevant): Load only when the specific use case matches (e.g., contract-testing only for microservices, email-auth only for email flows)
> **Context Efficiency**: Loading only core fragments reduces context usage by 40-50% compared to loading all fragments.
### Playwright Utils Loading Profiles
**If `tea_use_playwright_utils` is enabled**, select the appropriate loading profile:
- **API-only profile** (when `{detected_stack}` is `backend` or no `page.goto`/`page.locator` found in test files):
Load: `overview`, `api-request`, `auth-session`, `recurse` (~1,800 lines)
- **Full UI+API profile** (when `{detected_stack}` is `frontend`/`fullstack` or browser tests detected):
Load: all Playwright Utils core fragments (~4,500 lines)
**Detection**: Scan `{test_dir}` for files containing `page.goto` or `page.locator`. If none found, use API-only profile.
### Pact.js Utils Loading
**If `tea_use_pactjs_utils` is enabled** (and `{detected_stack}` is `backend` or `fullstack`, or microservices indicators detected):
Load: `pactjs-utils-overview.md`, `pactjs-utils-consumer-helpers.md`, `pactjs-utils-provider-verifier.md`, `pactjs-utils-request-filter.md`
**If `tea_use_pactjs_utils` is disabled** but contract testing is relevant:
Load: `contract-testing.md`
### Pact MCP Loading
**If `tea_pact_mcp` is `"mcp"`:**
Load: `pact-mcp.md`
## 5. Load Knowledge Base Fragments
Use `{knowledgeIndex}` to load:
**Core (always):**
- `data-factories.md`
- `component-tdd.md`
- `test-quality.md`
- `test-healing-patterns.md`
**If {detected_stack} is `frontend` or `fullstack`:**
- `selector-resilience.md`
- `timing-debugging.md`
**Playwright Utils (if enabled and {detected_stack} is `frontend` or `fullstack`):**
- `overview.md`, `api-request.md`, `network-recorder.md`, `auth-session.md`, `intercept-network-call.md`, `recurse.md`, `log.md`, `file-utils.md`, `network-error-monitor.md`, `fixtures-composition.md`
**Playwright CLI (if tea_browser_automation is "cli" or "auto" and {detected_stack} is `frontend` or `fullstack`):**
- `playwright-cli.md`
**MCP Patterns (if tea_browser_automation is "mcp" or "auto" and {detected_stack} is `frontend` or `fullstack`):**
- (existing MCP-related fragments, if any are added in future)
**Traditional Patterns (if utils disabled and {detected_stack} is `frontend` or `fullstack`):**
- `fixture-architecture.md`
- `network-first.md`
**Backend Patterns (if {detected_stack} is `backend` or `fullstack`):**
- `test-levels-framework.md`
- `test-priorities-matrix.md`
- `ci-burn-in.md`
**Pact.js Utils (if enabled):**
- `pactjs-utils-overview.md`, `pactjs-utils-consumer-helpers.md`, `pactjs-utils-provider-verifier.md`, `pactjs-utils-request-filter.md`
**Contract Testing (if pactjs-utils disabled but relevant):**
- `contract-testing.md`
**Pact MCP (if tea_pact_mcp is "mcp"):**
- `pact-mcp.md`
---
## 6. Confirm Inputs
Summarize loaded inputs and confirm with the user. Then proceed.
---
## 7. Save Progress
**Save this step's accumulated work to `{outputFile}`.**
- **If `{outputFile}` does not exist** (first save), create it with YAML frontmatter:
```yaml
---
stepsCompleted: ['step-01-preflight-and-context']
lastStep: 'step-01-preflight-and-context'
lastSaved: '{date}'
---
```
Then write this step's output below the frontmatter.
- **If `{outputFile}` already exists**, update:
- Add `'step-01-preflight-and-context'` to `stepsCompleted` array (only if not already present)
- Set `lastStep: 'step-01-preflight-and-context'`
- Set `lastSaved: '{date}'`
- Append this step's output to the appropriate section.
**Update `inputDocuments`**: Set `inputDocuments` in the output template frontmatter to the list of artifact paths loaded in this step (e.g., knowledge fragments, test design documents, configuration files).
Load next step: `{nextStepFile}`
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Step completed in full with required outputs
### ❌ SYSTEM FAILURE:
- Skipped sequence steps or missing outputs
**Master Rule:** Skipping steps is FORBIDDEN.

View File

@@ -0,0 +1,96 @@
---
name: 'step-01b-resume'
description: 'Resume interrupted workflow from last completed step'
outputFile: '{test_artifacts}/atdd-checklist-{story_id}.md'
---
# Step 1b: Resume Workflow
## STEP GOAL
Resume an interrupted workflow by loading the existing output document, displaying progress, and routing to the next incomplete step.
## MANDATORY EXECUTION RULES
- 📖 Read the entire step file before acting
- ✅ Speak in `{communication_language}`
---
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
- 📖 Load the next step only when instructed
## CONTEXT BOUNDARIES:
- Available context: Output document with progress frontmatter
- Focus: Load progress and route to next step
- Limits: Do not re-execute completed steps
- Dependencies: Output document must exist from a previous run
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly.
### 1. Load Output Document
Read `{outputFile}` and parse YAML frontmatter for:
- `stepsCompleted` — array of completed step names
- `lastStep` — last completed step name
- `lastSaved` — timestamp of last save
**If `{outputFile}` does not exist**, display:
"⚠️ **No previous progress found.** There is no output document to resume from. Please use **[C] Create** to start a fresh workflow run."
**THEN:** Halt. Do not proceed.
---
### 2. Display Progress Dashboard
Display progress with ✅/⬜ indicators:
1. ✅/⬜ Preflight & Context (step-01-preflight-and-context)
2. ✅/⬜ Generation Mode (step-02-generation-mode)
3. ✅/⬜ Test Strategy (step-03-test-strategy)
4. ✅/⬜ Generate Tests + Aggregate (step-04c-aggregate)
5. ✅/⬜ Validate & Complete (step-05-validate-and-complete)
---
### 3. Route to Next Step
Based on `lastStep`, load the next incomplete step:
- `'step-01-preflight-and-context'` → load `./step-02-generation-mode.md`
- `'step-02-generation-mode'` → load `./step-03-test-strategy.md`
- `'step-03-test-strategy'` → load `./step-04-generate-tests.md`
- `'step-04c-aggregate'` → load `./step-05-validate-and-complete.md`
- `'step-05-validate-and-complete'`**Workflow already complete.** Display: "✅ **All steps completed.** Use **[V] Validate** to review outputs or **[E] Edit** to make revisions." Then halt.
**If `lastStep` does not match any value above**, display: "⚠️ **Unknown progress state** (`lastStep`: {lastStep}). Please use **[C] Create** to start fresh." Then halt.
**Otherwise**, load the identified step file, read completely, and execute.
The existing content in `{outputFile}` provides context from previously completed steps.
---
## 🚨 SYSTEM SUCCESS/FAILURE METRICS
### ✅ SUCCESS:
- Output document loaded and parsed correctly
- Progress dashboard displayed accurately
- Routed to correct next step
### ❌ SYSTEM FAILURE:
- Not loading output document
- Incorrect progress display
- Routing to wrong step
**Master Rule:** Resume MUST route to the exact next incomplete step. Never re-execute completed steps.

View File

@@ -0,0 +1,125 @@
---
name: 'step-02-generation-mode'
description: 'Choose AI generation or recording mode'
outputFile: '{test_artifacts}/atdd-checklist-{story_id}.md'
nextStepFile: './step-03-test-strategy.md'
---
# Step 2: Generation Mode Selection
## STEP GOAL
Choose the appropriate generation mode for ATDD tests.
## MANDATORY EXECUTION RULES
- 📖 Read the entire step file before acting
- ✅ Speak in `{communication_language}`
---
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
- 💾 Record outputs before proceeding
- 📖 Load the next step only when instructed
## CONTEXT BOUNDARIES:
- Available context: config, loaded artifacts, and knowledge fragments
- Focus: this step's goal only
- Limits: do not execute future steps
- Dependencies: prior steps' outputs (if any)
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
## 1. Default Mode: AI Generation
Use AI generation when:
- Acceptance criteria are clear
- Scenarios are standard (CRUD, auth, API, navigation)
- **If {detected_stack} is `backend`:** Always use AI generation (no browser recording needed)
Proceed directly to test strategy if this applies.
---
## 2. Optional Mode: Recording (Complex UI)
**Skip this section entirely if {detected_stack} is `backend`.** For backend projects, use AI generation from API documentation, OpenAPI/Swagger specs, or source code analysis instead.
**If {detected_stack} is `frontend` or `fullstack`:**
Use recording when UI interactions need live browser verification.
**Tool selection based on `config.tea_browser_automation`:**
If `auto`:
> **Note:** `${timestamp}` is a placeholder the agent should replace with a unique value (e.g., epoch seconds) for session isolation.
- **Simple recording** (snapshot selectors, capture structure): Use CLI
- `playwright-cli -s=tea-atdd-${timestamp} open <url>``playwright-cli -s=tea-atdd-${timestamp} snapshot` → extract refs
- **Complex recording** (drag/drop, wizards, multi-step state): Use MCP
- Full browser automation with rich tool semantics
- **Fallback:** If preferred tool unavailable, use the other; if neither, skip recording
If `cli`:
- Use Playwright CLI for all recording
- `playwright-cli -s=tea-atdd-${timestamp} open <url>`, `snapshot`, `screenshot`, `click <ref>`, etc.
If `mcp`:
- Use Playwright MCP tools for all recording (current behavior)
- Confirm MCP availability, record selectors and interactions
If `none`:
- Skip recording mode entirely, use AI generation from documentation
---
## 3. Confirm Mode
State the chosen mode and why. Then proceed.
---
## 4. Save Progress
**Save this step's accumulated work to `{outputFile}`.**
- **If `{outputFile}` does not exist** (first save), create it with YAML frontmatter:
```yaml
---
stepsCompleted: ['step-02-generation-mode']
lastStep: 'step-02-generation-mode'
lastSaved: '{date}'
---
```
Then write this step's output below the frontmatter.
- **If `{outputFile}` already exists**, update:
- Add `'step-02-generation-mode'` to `stepsCompleted` array (only if not already present)
- Set `lastStep: 'step-02-generation-mode'`
- Set `lastSaved: '{date}'`
- Append this step's output to the appropriate section.
Load next step: `{nextStepFile}`
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Step completed in full with required outputs
### ❌ SYSTEM FAILURE:
- Skipped sequence steps or missing outputs
**Master Rule:** Skipping steps is FORBIDDEN.

View File

@@ -0,0 +1,110 @@
---
name: 'step-03-test-strategy'
description: 'Map acceptance criteria to test levels and priorities'
outputFile: '{test_artifacts}/atdd-checklist-{story_id}.md'
nextStepFile: './step-04-generate-tests.md'
---
# Step 3: Test Strategy
## STEP GOAL
Translate acceptance criteria into a prioritized, level-appropriate test plan.
## MANDATORY EXECUTION RULES
- 📖 Read the entire step file before acting
- ✅ Speak in `{communication_language}`
- 🚫 Avoid duplicate coverage across levels
---
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
- 💾 Record outputs before proceeding
- 📖 Load the next step only when instructed
## CONTEXT BOUNDARIES:
- Available context: config, loaded artifacts, and knowledge fragments
- Focus: this step's goal only
- Limits: do not execute future steps
- Dependencies: prior steps' outputs (if any)
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
## 1. Map Acceptance Criteria
- Convert each acceptance criterion into test scenarios
- Include negative and edge cases where risk is high
---
## 2. Select Test Levels
Choose the best level per scenario based on `{detected_stack}`:
**If {detected_stack} is `frontend` or `fullstack`:**
- **E2E** for critical user journeys
- **API** for business logic and service contracts
- **Component** for UI behavior
**If {detected_stack} is `backend` or `fullstack`:**
- **Unit** for pure functions, business logic, and edge cases
- **Integration** for service interactions, database queries, and middleware
- **API/Contract** for endpoint validation, request/response schemas, and Pact contracts
- **No E2E** for pure backend projects (no browser-based testing needed)
---
## 3. Prioritize Tests
Assign P0P3 priorities using risk and business impact.
---
## 4. Confirm Red Phase Requirements
Ensure all tests are designed to **fail before implementation** (TDD red phase).
---
## 5. Save Progress
**Save this step's accumulated work to `{outputFile}`.**
- **If `{outputFile}` does not exist** (first save), create it with YAML frontmatter:
```yaml
---
stepsCompleted: ['step-03-test-strategy']
lastStep: 'step-03-test-strategy'
lastSaved: '{date}'
---
```
Then write this step's output below the frontmatter.
- **If `{outputFile}` already exists**, update:
- Add `'step-03-test-strategy'` to `stepsCompleted` array (only if not already present)
- Set `lastStep: 'step-03-test-strategy'`
- Set `lastSaved: '{date}'`
- Append this step's output to the appropriate section.
Load next step: `{nextStepFile}`
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Step completed in full with required outputs
### ❌ SYSTEM FAILURE:
- Skipped sequence steps or missing outputs
**Master Rule:** Skipping steps is FORBIDDEN.

View File

@@ -0,0 +1,334 @@
---
name: 'step-04-generate-tests'
description: 'Orchestrate adaptive FAILING test generation (TDD red phase)'
nextStepFile: './step-04c-aggregate.md'
---
# Step 4: Orchestrate Adaptive FAILING Test Generation
## STEP GOAL
Select execution mode deterministically, then generate FAILING API and E2E tests (TDD RED PHASE) with consistent output contracts across agent-team, subagent, or sequential execution.
## MANDATORY EXECUTION RULES
- 📖 Read the entire step file before acting
- ✅ Speak in `{communication_language}`
- ✅ Resolve execution mode from config (`tea_execution_mode`, `tea_capability_probe`)
- ✅ Apply fallback rules deterministically when requested mode is unsupported
- ✅ Generate FAILING tests only (TDD red phase)
- ✅ Wait for required worker steps to complete
- ❌ Do NOT skip capability checks when probing is enabled
- ❌ Do NOT generate passing tests (this is red phase)
- ❌ Do NOT proceed until required worker steps finish
---
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
- 💾 Wait for subagent outputs
- 📖 Load the next step only when instructed
## CONTEXT BOUNDARIES:
- Available context: config, acceptance criteria from Step 1, test strategy from Step 3
- Focus: orchestration only (mode selection + worker dispatch)
- Limits: do not generate tests directly (delegate to worker steps)
- Dependencies: Steps 1-3 outputs
---
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
### 1. Prepare Execution Context
**Generate unique timestamp** for temp file naming:
```javascript
const timestamp = new Date().toISOString().replace(/[:.]/g, '-');
```
**Prepare input context for both subagents:**
```javascript
const parseBooleanFlag = (value, defaultValue = true) => {
if (typeof value === 'string') {
const normalized = value.trim().toLowerCase();
if (['false', '0', 'off', 'no'].includes(normalized)) return false;
if (['true', '1', 'on', 'yes'].includes(normalized)) return true;
}
if (value === undefined || value === null) return defaultValue;
return Boolean(value);
};
const subagentContext = {
story_acceptance_criteria: /* from Step 1 */,
test_strategy: /* from Step 3 */,
knowledge_fragments_loaded: /* list of fragments */,
config: {
test_framework: config.test_framework,
use_playwright_utils: config.tea_use_playwright_utils,
use_pactjs_utils: config.tea_use_pactjs_utils,
pact_mcp: config.tea_pact_mcp, // "mcp" | "none"
browser_automation: config.tea_browser_automation,
execution_mode: config.tea_execution_mode || 'auto', // "auto" | "subagent" | "agent-team" | "sequential"
capability_probe: parseBooleanFlag(config.tea_capability_probe, true), // supports booleans and "false"/"true" strings
provider_endpoint_map: /* from Step 1/3 context, if use_pactjs_utils enabled */,
},
timestamp: timestamp
};
```
---
### 2. Resolve Execution Mode with Capability Probe
```javascript
const normalizeUserExecutionMode = (mode) => {
if (typeof mode !== 'string') return null;
const normalized = mode.trim().toLowerCase().replace(/[-_]/g, ' ').replace(/\s+/g, ' ');
if (normalized === 'auto') return 'auto';
if (normalized === 'sequential') return 'sequential';
if (normalized === 'subagent' || normalized === 'sub agent' || normalized === 'subagents' || normalized === 'sub agents') {
return 'subagent';
}
if (normalized === 'agent team' || normalized === 'agent teams' || normalized === 'agentteam') {
return 'agent-team';
}
return null;
};
const normalizeConfigExecutionMode = (mode) => {
if (mode === 'subagent') return 'subagent';
if (mode === 'auto' || mode === 'sequential' || mode === 'subagent' || mode === 'agent-team') {
return mode;
}
return null;
};
// Explicit user instruction in the active run takes priority over config.
const explicitModeFromUser = normalizeUserExecutionMode(runtime.getExplicitExecutionModeHint?.() || null);
const requestedMode = explicitModeFromUser || normalizeConfigExecutionMode(subagentContext.config.execution_mode) || 'auto';
const probeEnabled = subagentContext.config.capability_probe;
const supports = {
subagent: runtime.canLaunchSubagents?.() === true,
agentTeam: runtime.canLaunchAgentTeams?.() === true,
};
let resolvedMode = requestedMode;
if (requestedMode === 'auto') {
if (supports.agentTeam) resolvedMode = 'agent-team';
else if (supports.subagent) resolvedMode = 'subagent';
else resolvedMode = 'sequential';
} else if (probeEnabled && requestedMode === 'agent-team' && !supports.agentTeam) {
resolvedMode = supports.subagent ? 'subagent' : 'sequential';
} else if (probeEnabled && requestedMode === 'subagent' && !supports.subagent) {
resolvedMode = 'sequential';
}
subagentContext.execution = {
requestedMode,
resolvedMode,
probeEnabled,
supports,
};
if (!probeEnabled && (requestedMode === 'agent-team' || requestedMode === 'subagent')) {
const unsupportedRequestedMode =
(requestedMode === 'agent-team' && !supports.agentTeam) || (requestedMode === 'subagent' && !supports.subagent);
if (unsupportedRequestedMode) {
subagentContext.execution.error = `Requested execution mode "${requestedMode}" is unavailable because capability probing is disabled.`;
throw new Error(subagentContext.execution.error);
}
}
```
Resolution precedence:
1. Explicit user request in this run (`agent team` => `agent-team`; `subagent` => `subagent`; `sequential`; `auto`)
2. `tea_execution_mode` from config
3. Runtime capability fallback (when probing enabled)
If probing is disabled, honor the requested mode strictly. If that mode cannot be executed at runtime, fail with explicit error instead of silent fallback.
---
### 3. Dispatch Worker A: Failing API Test Generation
**Dispatch worker:**
- **Subagent File:** `./step-04a-subagent-api-failing.md`
- **Output File:** `/tmp/tea-atdd-api-tests-${timestamp}.json`
- **Context:** Pass `subagentContext`
- **Execution:**
- `agent-team` or `subagent`: launch non-blocking
- `sequential`: run blocking and wait before next dispatch
- **TDD Phase:** RED (failing tests)
**System Action:**
```
🚀 Launching Subagent A: FAILING API Test Generation (RED PHASE)
📝 Output: /tmp/tea-atdd-api-tests-${timestamp}.json
⚙️ Mode: ${resolvedMode}
🔴 TDD Phase: RED (tests will fail until feature implemented)
⏳ Status: Running...
```
---
### 4. Dispatch Worker B: Failing E2E Test Generation
**Dispatch worker:**
- **Subagent File:** `./step-04b-subagent-e2e-failing.md`
- **Output File:** `/tmp/tea-atdd-e2e-tests-${timestamp}.json`
- **Context:** Pass `subagentContext`
- **Execution:**
- `agent-team` or `subagent`: launch non-blocking
- `sequential`: run blocking and wait before next dispatch
- **TDD Phase:** RED (failing tests)
**System Action:**
```
🚀 Launching Subagent B: FAILING E2E Test Generation (RED PHASE)
📝 Output: /tmp/tea-atdd-e2e-tests-${timestamp}.json
⚙️ Mode: ${resolvedMode}
🔴 TDD Phase: RED (tests will fail until feature implemented)
⏳ Status: Running...
```
---
### 5. Wait for Required Worker Completion
**If `resolvedMode` is `agent-team` or `subagent`:**
```
⏳ Waiting for subagents to complete...
├── Subagent A (API RED): Running... ⟳
└── Subagent B (E2E RED): Running... ⟳
[... time passes ...]
├── Subagent A (API RED): Complete ✅
└── Subagent B (E2E RED): Complete ✅
✅ All subagents completed successfully!
```
**If `resolvedMode` is `sequential`:**
```
✅ Sequential mode: each worker already completed during dispatch.
```
**Verify both outputs exist:**
```javascript
const apiOutputExists = fs.existsSync(`/tmp/tea-atdd-api-tests-${timestamp}.json`);
const e2eOutputExists = fs.existsSync(`/tmp/tea-atdd-e2e-tests-${timestamp}.json`);
if (!apiOutputExists || !e2eOutputExists) {
throw new Error('One or both subagent outputs missing!');
}
```
---
### 6. TDD Red Phase Report
**Display TDD status:**
```
🔴 TDD RED PHASE: Failing Tests Generated
✅ Both subagents completed:
- API Tests: Generated with test.skip()
- E2E Tests: Generated with test.skip()
📋 All tests assert EXPECTED behavior
📋 All tests will FAIL until feature implemented
📋 This is INTENTIONAL (TDD red phase)
Next: Aggregation will verify TDD compliance
```
---
### 7. Execution Report
**Display performance metrics:**
```
🚀 Performance Report:
- Execution Mode: {resolvedMode}
- API Test Generation: ~X minutes
- E2E Test Generation: ~Y minutes
- Total Elapsed: ~mode-dependent
- Parallel Gain: ~50% faster when mode is subagent/agent-team
```
---
### 8. Proceed to Aggregation
**Load aggregation step:**
Load next step: `{nextStepFile}`
The aggregation step (4C) will:
- Read both subagent outputs
- Verify TDD red phase compliance (all tests have test.skip())
- Write all test files to disk
- Generate ATDD checklist
- Calculate summary statistics
---
## EXIT CONDITION
Proceed to Step 4C (Aggregation) when:
- ✅ Subagent A (API failing tests) completed successfully
- ✅ Subagent B (E2E failing tests) completed successfully
- ✅ Both output files exist and are valid JSON
- ✅ TDD red phase status reported
**Do NOT proceed if:**
- ❌ One or both subagents failed
- ❌ Output files missing or corrupted
- ❌ Subagent generated passing tests (wrong - must be failing)
---
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Both subagents launched successfully
- Both worker steps completed without errors
- Output files generated and valid
- Tests generated with test.skip() (TDD red phase)
- Fallback behavior respected configuration and capability probe rules
### ❌ SYSTEM FAILURE:
- Failed to launch subagents
- One or both subagents failed
- Output files missing or invalid
- Tests generated without test.skip() (wrong phase)
- Unsupported requested mode with probing disabled
**Master Rule:** TDD RED PHASE requires FAILING tests (with test.skip()). Mode selection changes orchestration, never red-phase requirements.

View File

@@ -0,0 +1,286 @@
---
name: 'step-04a-subagent-api-failing'
description: 'Subagent: Generate FAILING API tests (TDD red phase)'
subagent: true
outputFile: '/tmp/tea-atdd-api-tests-{{timestamp}}.json'
---
# Subagent 4A: Generate Failing API Tests (TDD Red Phase)
## SUBAGENT CONTEXT
This is an **isolated subagent** running in parallel with E2E failing test generation.
**What you have from parent workflow:**
- Story acceptance criteria from Step 1
- Test strategy and scenarios from Step 3
- Knowledge fragments loaded: api-request, data-factories, api-testing-patterns
- Config: test framework, Playwright Utils enabled/disabled, Pact.js Utils enabled/disabled (`use_pactjs_utils`), Pact MCP mode (`pact_mcp`)
- Provider Endpoint Map (if `use_pactjs_utils` enabled and provider source accessible)
**Your task:** Generate API tests that will FAIL because the feature is not implemented yet (TDD RED PHASE).
---
## MANDATORY EXECUTION RULES
- 📖 Read this entire subagent file before acting
- ✅ Generate FAILING API tests ONLY
- ✅ Tests MUST fail when run (feature not implemented yet)
- ✅ Output structured JSON to temp file
- ✅ Follow knowledge fragment patterns
- ❌ Do NOT generate E2E tests (that's subagent 4B)
- ❌ Do NOT generate passing tests (this is TDD red phase)
- ❌ Do NOT run tests (that's step 5)
---
## SUBAGENT TASK
### 1. Identify API Endpoints from Acceptance Criteria
From the story acceptance criteria (Step 1 output), identify:
- Which API endpoints will be created for this story
- Expected request/response contracts
- Authentication requirements
- Expected status codes and error scenarios
**Example Acceptance Criteria:**
```
Story: User Registration
- As a user, I can POST to /api/users/register with email and password
- System returns 201 Created with user object
- System returns 400 Bad Request if email already exists
- System returns 422 Unprocessable Entity if validation fails
```
### 2. Generate FAILING API Test Files
For each API endpoint, create test file in `tests/api/[feature].spec.ts`:
**Test Structure (ATDD - Red Phase):**
```typescript
import { test, expect } from '@playwright/test';
// If Playwright Utils enabled:
// import { apiRequest } from '@playwright-utils/api';
test.describe('[Story Name] API Tests (ATDD)', () => {
test.skip('[P0] should register new user successfully', async ({ request }) => {
// THIS TEST WILL FAIL - Endpoint not implemented yet
const response = await request.post('/api/users/register', {
data: {
email: 'newuser@example.com',
password: 'SecurePass123!',
},
});
// Expect 201 but will get 404 (endpoint doesn't exist)
expect(response.status()).toBe(201);
const user = await response.json();
expect(user).toMatchObject({
id: expect.any(Number),
email: 'newuser@example.com',
});
});
test.skip('[P1] should return 400 if email exists', async ({ request }) => {
// THIS TEST WILL FAIL - Endpoint not implemented yet
const response = await request.post('/api/users/register', {
data: {
email: 'existing@example.com',
password: 'SecurePass123!',
},
});
expect(response.status()).toBe(400);
const error = await response.json();
expect(error.message).toContain('Email already exists');
});
});
```
**CRITICAL ATDD Requirements:**
- ✅ Use `test.skip()` to mark tests as intentionally failing (red phase)
- ✅ Write assertions for EXPECTED behavior (even though not implemented)
- ✅ Use realistic test data (not placeholder data)
- ✅ Test both happy path and error scenarios from acceptance criteria
- ✅ Use `apiRequest()` helper if Playwright Utils enabled
- ✅ Use data factories for test data (from data-factories fragment)
- ✅ Include priority tags [P0], [P1], [P2], [P3]
### 1.5 Provider Source Scrutiny for CDC in TDD Red Phase (If `use_pactjs_utils` Enabled)
When generating Pact consumer contract tests in the ATDD red phase, provider scrutiny applies with TDD-specific rules. Apply the **Seven-Point Scrutiny Checklist** from `contract-testing.md` (Response shape, Status codes, Field names, Enum values, Required fields, Data types, Nested structures) for both existing and new endpoints.
**If provider endpoint already exists** (extending an existing API):
- READ the provider route handler, types, and validation schemas
- Verify all seven scrutiny points against the provider source: Response shape, Status codes, Field names, Enum values, Required fields, Data types, Nested structures
- Add `// Provider endpoint:` comment and scrutiny evidence block documenting findings for each point
- Wrap the entire test function in `test.skip()` (so the whole test including `executeTest` is skipped), not just the callback
**If provider endpoint is new** (TDD — endpoint not implemented yet):
- Use acceptance criteria as the source of truth for expected behavior
- Acceptance criteria should specify all seven scrutiny points where possible (status codes, field names, types, etc.) — note any gaps as assumptions in the evidence block
- Add `// Provider endpoint: TODO — new endpoint, not yet implemented`
- Document expected behavior from acceptance criteria in scrutiny evidence block
- Wrap the entire test function in `test.skip()` and use realistic expectations from the story
**Graceful degradation when provider source is inaccessible:**
1. **OpenAPI/Swagger spec available**: Use the spec as the source of truth for response shapes, status codes, and field names
2. **Pact Broker available** (when `pact_mcp` is `"mcp"`): Use SmartBear MCP tools to fetch existing provider states and verified interactions as reference
3. **Neither available**: For new endpoints, use acceptance criteria; for existing endpoints, use consumer-side types. Mark with `// Provider endpoint: TODO — provider source not accessible, verify manually` and set `provider_scrutiny: "pending"` in output JSON
4. **Never silently guess**: Document all assumptions in the scrutiny evidence block
**Provider endpoint comments are MANDATORY** even in red-phase tests — they document the intent.
**Example: Red-phase Pact test with provider scrutiny:**
```typescript
// Provider endpoint: TODO — new endpoint, not yet implemented
/*
* Provider Scrutiny Evidence:
* - Handler: NEW — not yet implemented (TDD red phase)
* - Expected from acceptance criteria:
* - Endpoint: POST /api/v2/users/register
* - Status: 201 for success, 400 for duplicate email, 422 for validation error
* - Response: { id: number, email: string, createdAt: string }
*/
test.skip('[P0] should generate consumer contract for user registration', async () => {
await provider
.given('no users exist')
.uponReceiving('a request to register a new user')
.withRequest({
method: 'POST',
path: '/api/v2/users/register',
headers: { 'Content-Type': 'application/json' },
body: { email: 'newuser@example.com', password: 'SecurePass123!' },
})
.willRespondWith({
status: 201,
headers: { 'Content-Type': 'application/json' },
body: like({
id: integer(1),
email: string('newuser@example.com'),
createdAt: string('2025-01-15T10:00:00Z'),
}),
})
.executeTest(async (mockServer) => {
const result = await registerUser({ email: 'newuser@example.com', password: 'SecurePass123!' }, { baseUrl: mockServer.url });
expect(result.id).toEqual(expect.any(Number));
});
});
```
**Why test.skip():**
- Tests are written correctly for EXPECTED behavior
- But we know they'll fail because feature isn't implemented
- `test.skip()` documents this is intentional (TDD red phase)
- Once feature is implemented, remove `test.skip()` to verify green phase
### 3. Track Fixture Needs
Identify fixtures needed for API tests:
- Authentication fixtures (if endpoints require auth)
- Data factories (user data, etc.)
- API client configurations
**Do NOT create fixtures yet** - just track what's needed for aggregation step.
---
## OUTPUT FORMAT
Write JSON to temp file: `/tmp/tea-atdd-api-tests-{{timestamp}}.json`
```json
{
"success": true,
"subagent": "atdd-api-tests",
"tests": [
{
"file": "tests/api/user-registration.spec.ts",
"content": "[full TypeScript test file content with test.skip()]",
"description": "ATDD API tests for user registration (RED PHASE)",
"expected_to_fail": true,
"acceptance_criteria_covered": [
"User can register with email/password",
"System returns 201 on success",
"System returns 400 if email exists"
],
"priority_coverage": {
"P0": 1,
"P1": 2,
"P2": 0,
"P3": 0
}
}
],
"fixture_needs": ["userDataFactory"],
"knowledge_fragments_used": ["api-request", "data-factories", "api-testing-patterns"],
"test_count": 3,
"tdd_phase": "RED",
"provider_scrutiny": "completed",
"summary": "Generated 3 FAILING API tests for user registration story"
}
```
**On Error:**
```json
{
"success": false,
"subagent": "atdd-api-tests",
"error": "Error message describing what went wrong",
"partial_output": {
/* any tests generated before error */
}
}
```
---
## EXIT CONDITION
Subagent completes when:
- ✅ All API endpoints from acceptance criteria have test files
- ✅ All tests use `test.skip()` (documented failing tests)
- ✅ All tests assert EXPECTED behavior (not placeholder assertions)
- ✅ JSON output written to temp file
- ✅ Fixture needs to be tracked
**Subagent terminates here.** Parent workflow will read output and proceed to aggregation.
---
## 🚨 SUBAGENT SUCCESS METRICS
### ✅ SUCCESS:
- All API tests generated with test.skip()
- Tests assert expected behavior (not placeholders)
- JSON output valid and complete
- No E2E/component/unit tests included (out of scope)
- Tests follow knowledge fragment patterns
- Every Pact interaction has `// Provider endpoint:` comment (if CDC enabled)
- Provider scrutiny completed or TODO markers added for new endpoints (if CDC enabled)
### ❌ FAILURE:
- Generated passing tests (wrong - this is RED phase)
- Tests without test.skip() (will break CI)
- Placeholder assertions (expect(true).toBe(true))
- Did not follow knowledge fragment patterns
- Invalid or missing JSON output
- Pact interactions missing provider endpoint comments (if CDC enabled)

View File

@@ -0,0 +1,244 @@
---
name: 'step-04b-subagent-e2e-failing'
description: 'Subagent: Generate FAILING E2E tests (TDD red phase)'
subagent: true
outputFile: '/tmp/tea-atdd-e2e-tests-{{timestamp}}.json'
---
# Subagent 4B: Generate Failing E2E Tests (TDD Red Phase)
## SUBAGENT CONTEXT
This is an **isolated subagent** running in parallel with API failing test generation.
**What you have from parent workflow:**
- Story acceptance criteria from Step 1
- Test strategy and user journey scenarios from Step 3
- Knowledge fragments loaded: fixture-architecture, network-first, selector-resilience
- Config: test framework, Playwright Utils enabled/disabled
**Your task:** Generate E2E tests that will FAIL because the feature UI is not implemented yet (TDD RED PHASE).
---
## MANDATORY EXECUTION RULES
- 📖 Read this entire subagent file before acting
- ✅ Generate FAILING E2E tests ONLY
- ✅ Tests MUST fail when run (UI not implemented yet)
- ✅ Output structured JSON to temp file
- ✅ Follow knowledge fragment patterns
- ❌ Do NOT generate API tests (that's subagent 4A)
- ❌ Do NOT generate passing tests (this is TDD red phase)
- ❌ Do NOT run tests (that's step 5)
---
## SUBAGENT TASK
### 1. Identify User Journeys from Acceptance Criteria
From the story acceptance criteria (Step 1 output), identify:
- Which UI flows will be created for this story
- User interactions required
- Expected visual states
- Success/error messages expected
**Example Acceptance Criteria:**
```
Story: User Registration
- As a user, I can navigate to /register page
- I can fill in email and password fields
- I can click "Register" button
- System shows success message and redirects to dashboard
- System shows error if email already exists
```
### 2. Browser Interaction (Selector Verification)
**Automation mode:** `config.tea_browser_automation`
If `auto` (fall back to MCP if CLI unavailable; if neither available, generate from best practices):
- Open the target page first, then verify selectors with a snapshot:
`playwright-cli -s=tea-atdd-{{timestamp}} open <target_url>`
`playwright-cli -s=tea-atdd-{{timestamp}} snapshot` → map refs to Playwright locators
- ref `{role: "button", name: "Submit"}``page.getByRole('button', { name: 'Submit' })`
- ref `{role: "textbox", name: "Email"}``page.getByRole('textbox', { name: 'Email' })`
- `playwright-cli -s=tea-atdd-{{timestamp}} close` when done
If `cli` (CLI only — do NOT fall back to MCP; generate from best practices if CLI unavailable):
- Open the target page first, then verify selectors with a snapshot:
`playwright-cli -s=tea-atdd-{{timestamp}} open <target_url>`
`playwright-cli -s=tea-atdd-{{timestamp}} snapshot` → map refs to Playwright locators
- ref `{role: "button", name: "Submit"}``page.getByRole('button', { name: 'Submit' })`
- ref `{role: "textbox", name: "Email"}``page.getByRole('textbox', { name: 'Email' })`
- `playwright-cli -s=tea-atdd-{{timestamp}} close` when done
> **Session Hygiene:** Always close sessions using `playwright-cli -s=tea-atdd-{{timestamp}} close`. Do NOT use `close-all` — it kills every session on the machine and breaks parallel execution.
If `mcp`:
- Use MCP tools for selector verification (current behavior)
If `none`:
- Generate selectors from best practices without browser verification
### 3. Generate FAILING E2E Test Files
For each user journey, create test file in `tests/e2e/[feature].spec.ts`:
**Test Structure (ATDD - Red Phase):**
```typescript
import { test, expect } from '@playwright/test';
test.describe('[Story Name] E2E User Journey (ATDD)', () => {
test.skip('[P0] should complete user registration successfully', async ({ page }) => {
// THIS TEST WILL FAIL - UI not implemented yet
await page.goto('/register');
// Expect registration form but will get 404 or missing elements
await page.fill('[name="email"]', 'newuser@example.com');
await page.fill('[name="password"]', 'SecurePass123!');
await page.click('button:has-text("Register")');
// Expect success message and redirect
await expect(page.getByText('Registration successful!')).toBeVisible();
await page.waitForURL('/dashboard');
});
test.skip('[P1] should show error if email exists', async ({ page }) => {
// THIS TEST WILL FAIL - UI not implemented yet
await page.goto('/register');
await page.fill('[name="email"]', 'existing@example.com');
await page.fill('[name="password"]', 'SecurePass123!');
await page.click('button:has-text("Register")');
// Expect error message
await expect(page.getByText('Email already exists')).toBeVisible();
});
});
```
**CRITICAL ATDD Requirements:**
- ✅ Use `test.skip()` to mark tests as intentionally failing (red phase)
- ✅ Write assertions for EXPECTED UI behavior (even though not implemented)
- ✅ Use resilient selectors: getByRole, getByText, getByLabel (from selector-resilience)
- ✅ Follow network-first patterns if API calls involved (from network-first)
- ✅ Test complete user journeys from acceptance criteria
- ✅ Include priority tags [P0], [P1], [P2], [P3]
- ✅ Use proper TypeScript types
- ✅ Deterministic waits (no hard sleeps)
**Why test.skip():**
- Tests are written correctly for EXPECTED UI behavior
- But we know they'll fail because UI isn't implemented
- `test.skip()` documents this is intentional (TDD red phase)
- Once UI is implemented, remove `test.skip()` to verify green phase
### 4. Track Fixture Needs
Identify fixtures needed for E2E tests:
- Authentication fixtures (if journey requires logged-in state)
- Network mocks (if API calls involved)
- Test data fixtures
**Do NOT create fixtures yet** - just track what's needed for aggregation step.
---
## OUTPUT FORMAT
Write JSON to temp file: `/tmp/tea-atdd-e2e-tests-{{timestamp}}.json`
```json
{
"success": true,
"subagent": "atdd-e2e-tests",
"tests": [
{
"file": "tests/e2e/user-registration.spec.ts",
"content": "[full TypeScript test file content with test.skip()]",
"description": "ATDD E2E tests for user registration journey (RED PHASE)",
"expected_to_fail": true,
"acceptance_criteria_covered": [
"User can navigate to /register",
"User can fill registration form",
"System shows success message on registration",
"System shows error if email exists"
],
"priority_coverage": {
"P0": 1,
"P1": 1,
"P2": 0,
"P3": 0
}
}
],
"fixture_needs": ["registrationPageMock"],
"knowledge_fragments_used": ["fixture-architecture", "network-first", "selector-resilience"],
"test_count": 2,
"tdd_phase": "RED",
"summary": "Generated 2 FAILING E2E tests for user registration story"
}
```
**On Error:**
```json
{
"success": false,
"subagent": "atdd-e2e-tests",
"error": "Error message describing what went wrong",
"partial_output": {
/* any tests generated before error */
}
}
```
---
## EXIT CONDITION
Subagent completes when:
- ✅ All user journeys from acceptance criteria have test files
- ✅ All tests use `test.skip()` (documented failing tests)
- ✅ All tests assert EXPECTED UI behavior (not placeholder assertions)
- ✅ Resilient selectors used (getByRole, getByText)
- ✅ JSON output written to temp file
- ✅ Fixture needs tracked
**Subagent terminates here.** Parent workflow will read output and proceed to aggregation.
---
## 🚨 SUBAGENT SUCCESS METRICS
### ✅ SUCCESS:
- All E2E tests generated with test.skip()
- Tests assert expected UI behavior (not placeholders)
- Resilient selectors used (getByRole, getByText)
- JSON output valid and complete
- No API/component/unit tests included (out of scope)
- Tests follow knowledge fragment patterns
### ❌ FAILURE:
- Generated passing tests (wrong - this is RED phase)
- Tests without test.skip() (will break CI)
- Placeholder assertions (expect(true).toBe(true))
- Brittle selectors used (CSS classes, XPath)
- Did not follow knowledge fragment patterns
- Invalid or missing JSON output

View File

@@ -0,0 +1,370 @@
---
name: 'step-04c-aggregate'
description: 'Aggregate subagent outputs and complete ATDD test infrastructure'
outputFile: '{test_artifacts}/atdd-checklist-{story_id}.md'
nextStepFile: './step-05-validate-and-complete.md'
---
# Step 4C: Aggregate ATDD Test Generation Results
## STEP GOAL
Read outputs from parallel subagents (API + E2E failing test generation), aggregate results, verify TDD red phase compliance, and create supporting infrastructure.
---
## MANDATORY EXECUTION RULES
- 📖 Read the entire step file before acting
- ✅ Speak in `{communication_language}`
- ✅ Read subagent outputs from temp files
- ✅ Verify all tests are marked with test.skip() (TDD red phase)
- ✅ Generate shared fixtures based on fixture needs
- ✅ Write all generated test files to disk
- ❌ Do NOT remove test.skip() (that's done after feature implementation)
- ❌ Do NOT run tests yet (that's step 5 - verify they fail)
---
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
- 💾 Record outputs before proceeding
- 📖 Load the next step only when instructed
## CONTEXT BOUNDARIES:
- Available context: config, subagent outputs from temp files
- Focus: aggregation and TDD validation
- Limits: do not execute future steps
- Dependencies: Step 4A and 4B subagent outputs
---
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
### 1. Read Subagent Outputs
**Read API test subagent output:**
```javascript
const apiTestsPath = '/tmp/tea-atdd-api-tests-{{timestamp}}.json';
const apiTestsOutput = JSON.parse(fs.readFileSync(apiTestsPath, 'utf8'));
```
**Read E2E test subagent output:**
```javascript
const e2eTestsPath = '/tmp/tea-atdd-e2e-tests-{{timestamp}}.json';
const e2eTestsOutput = JSON.parse(fs.readFileSync(e2eTestsPath, 'utf8'));
```
**Verify both subagents succeeded:**
- Check `apiTestsOutput.success === true`
- Check `e2eTestsOutput.success === true`
- If either failed, report error and stop (don't proceed)
---
### 2. Verify TDD Red Phase Compliance
**CRITICAL TDD Validation:**
**Check API tests:**
```javascript
apiTestsOutput.tests.forEach((test) => {
// Verify test.skip() is present
if (!test.content.includes('test.skip(')) {
throw new Error(`ATDD ERROR: ${test.file} missing test.skip() - tests MUST be skipped in red phase!`);
}
// Verify not placeholder assertions
if (test.content.includes('expect(true).toBe(true)')) {
throw new Error(`ATDD ERROR: ${test.file} has placeholder assertions - must assert EXPECTED behavior!`);
}
// Verify expected_to_fail flag
if (!test.expected_to_fail) {
throw new Error(`ATDD ERROR: ${test.file} not marked as expected_to_fail!`);
}
});
```
**Check E2E tests:**
```javascript
e2eTestsOutput.tests.forEach((test) => {
// Same validation as API tests
if (!test.content.includes('test.skip(')) {
throw new Error(`ATDD ERROR: ${test.file} missing test.skip() - tests MUST be skipped in red phase!`);
}
if (test.content.includes('expect(true).toBe(true)')) {
throw new Error(`ATDD ERROR: ${test.file} has placeholder assertions!`);
}
if (!test.expected_to_fail) {
throw new Error(`ATDD ERROR: ${test.file} not marked as expected_to_fail!`);
}
});
```
**If validation passes:**
```
✅ TDD Red Phase Validation: PASS
- All tests use test.skip()
- All tests assert expected behavior (not placeholders)
- All tests marked as expected_to_fail
```
---
### 3. Write All Test Files to Disk
**Write API test files:**
```javascript
apiTestsOutput.tests.forEach((test) => {
fs.writeFileSync(test.file, test.content, 'utf8');
console.log(`✅ Created (RED): ${test.file}`);
});
```
**Write E2E test files:**
```javascript
e2eTestsOutput.tests.forEach((test) => {
fs.writeFileSync(test.file, test.content, 'utf8');
console.log(`✅ Created (RED): ${test.file}`);
});
```
---
### 4. Aggregate Fixture Needs
**Collect all fixture needs from both subagents:**
```javascript
const allFixtureNeeds = [...apiTestsOutput.fixture_needs, ...e2eTestsOutput.fixture_needs];
// Remove duplicates
const uniqueFixtures = [...new Set(allFixtureNeeds)];
```
---
### 5. Generate Fixture Infrastructure
**Create fixtures needed by ATDD tests:**
(Similar to automate workflow, but may be simpler for ATDD since feature not implemented)
**Minimal fixtures for TDD red phase:**
```typescript
// tests/fixtures/test-data.ts
export const testUserData = {
email: 'test@example.com',
password: 'SecurePass123!',
};
```
Note: More complete fixtures will be needed when moving to green phase.
---
### 6. Generate ATDD Checklist
**Create ATDD checklist document:**
```markdown
# ATDD Checklist: [Story Name]
## TDD Red Phase (Current)
✅ Failing tests generated
- API Tests: {api_test_count} tests (all skipped)
- E2E Tests: {e2e_test_count} tests (all skipped)
## Acceptance Criteria Coverage
{list all acceptance criteria with test coverage}
## Next Steps (TDD Green Phase)
After implementing the feature:
1. Remove `test.skip()` from all test files
2. Run tests: `npm test`
3. Verify tests PASS (green phase)
4. If any tests fail:
- Either fix implementation (feature bug)
- Or fix test (test bug)
5. Commit passing tests
## Implementation Guidance
Feature endpoints to implement:
{list endpoints from API tests}
UI components to implement:
{list UI flows from E2E tests}
```
**Save checklist:**
```javascript
fs.writeFileSync(`{test_artifacts}/atdd-checklist-{story-id}.md`, checklistContent, 'utf8');
```
---
### 7. Calculate Summary Statistics
**Aggregate test counts:**
```javascript
const resolvedMode = subagentContext?.execution?.resolvedMode; // Provided by Step 4's orchestration context
const subagentExecutionLabel =
resolvedMode === 'sequential'
? 'SEQUENTIAL (API → E2E)'
: resolvedMode === 'agent-team'
? 'AGENT-TEAM (API + E2E)'
: resolvedMode === 'subagent'
? 'SUBAGENT (API + E2E)'
: 'PARALLEL (API + E2E)';
const performanceGainLabel =
resolvedMode === 'sequential'
? 'baseline (no parallel speedup)'
: resolvedMode === 'agent-team' || resolvedMode === 'subagent'
? '~50% faster than sequential'
: 'mode-dependent';
const summary = {
tdd_phase: 'RED',
total_tests: apiTestsOutput.test_count + e2eTestsOutput.test_count,
api_tests: apiTestsOutput.test_count,
e2e_tests: e2eTestsOutput.test_count,
all_tests_skipped: true,
expected_to_fail: true,
fixtures_created: uniqueFixtures.length,
acceptance_criteria_covered: [
...apiTestsOutput.tests.flatMap((t) => t.acceptance_criteria_covered),
...e2eTestsOutput.tests.flatMap((t) => t.acceptance_criteria_covered),
],
knowledge_fragments_used: [...apiTestsOutput.knowledge_fragments_used, ...e2eTestsOutput.knowledge_fragments_used],
subagent_execution: subagentExecutionLabel,
performance_gain: performanceGainLabel,
};
```
**Store summary for Step 5:**
```javascript
fs.writeFileSync('/tmp/tea-atdd-summary-{{timestamp}}.json', JSON.stringify(summary, null, 2), 'utf8');
```
---
## OUTPUT SUMMARY
Display to user:
```
✅ ATDD Test Generation Complete (TDD RED PHASE)
🔴 TDD Red Phase: Failing Tests Generated
📊 Summary:
- Total Tests: {total_tests} (all with test.skip())
- API Tests: {api_tests} (RED)
- E2E Tests: {e2e_tests} (RED)
- Fixtures Created: {fixtures_created}
- All tests will FAIL until feature implemented
✅ Acceptance Criteria Coverage:
{list all covered criteria}
🚀 Performance: {performance_gain}
📂 Generated Files:
- tests/api/[feature].spec.ts (with test.skip())
- tests/e2e/[feature].spec.ts (with test.skip())
- tests/fixtures/test-data.ts
- {test_artifacts}/atdd-checklist-{story-id}.md
📝 Next Steps:
1. Implement the feature
2. Remove test.skip() from tests
3. Run tests → verify PASS (green phase)
4. Commit passing tests
✅ Ready for validation (Step 5 - verify tests fail as expected)
```
---
## EXIT CONDITION
Proceed to Step 5 when:
- ✅ All test files written to disk (API + E2E)
- ✅ All tests verified to have test.skip()
- ✅ All fixtures created
- ✅ ATDD checklist generated
- ✅ Summary statistics calculated and saved
- ✅ Output displayed to user
---
### 8. Save Progress
**Save this step's accumulated work to `{outputFile}`.**
- **If `{outputFile}` does not exist** (first save), create it with YAML frontmatter:
```yaml
---
stepsCompleted: ['step-04c-aggregate']
lastStep: 'step-04c-aggregate'
lastSaved: '{date}'
---
```
Then write this step's output below the frontmatter.
- **If `{outputFile}` already exists**, update:
- Add `'step-04c-aggregate'` to `stepsCompleted` array (only if not already present)
- Set `lastStep: 'step-04c-aggregate'`
- Set `lastSaved: '{date}'`
- Append this step's output to the appropriate section.
Load next step: `{nextStepFile}`
---
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Both subagents succeeded
- All tests have test.skip() (TDD red phase compliant)
- All tests assert expected behavior (not placeholders)
- All test files written to disk
- ATDD checklist generated
### ❌ SYSTEM FAILURE:
- One or both subagents failed
- Tests missing test.skip() (would break CI)
- Tests have placeholder assertions
- Test files not written to disk
- ATDD checklist missing
**Master Rule:** TDD RED PHASE requires ALL tests to use test.skip() and assert expected behavior.

View File

@@ -0,0 +1,106 @@
---
name: 'step-05-validate-and-complete'
description: 'Validate ATDD outputs and summarize'
outputFile: '{test_artifacts}/atdd-checklist-{story_id}.md'
---
# Step 5: Validate & Complete
## STEP GOAL
Validate ATDD outputs and provide a completion summary.
## MANDATORY EXECUTION RULES
- 📖 Read the entire step file before acting
- ✅ Speak in `{communication_language}`
- ✅ Validate against the checklist
---
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
- 💾 Record outputs before proceeding
- 📖 Load the next step only when instructed
## CONTEXT BOUNDARIES:
- Available context: config, loaded artifacts, and knowledge fragments
- Focus: this step's goal only
- Limits: do not execute future steps
- Dependencies: prior steps' outputs (if any)
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
## 1. Validation
Use `checklist.md` to validate:
- Prerequisites satisfied
- Test files created correctly
- Checklist matches acceptance criteria
- Tests are designed to fail before implementation
- [ ] CLI sessions cleaned up (no orphaned browsers)
- [ ] Temp artifacts stored in `{test_artifacts}/` not random locations
Fix any gaps before completion.
---
## 2. Polish Output
Before finalizing, review the complete output document for quality:
1. **Remove duplication**: Progressive-append workflow may have created repeated sections — consolidate
2. **Verify consistency**: Ensure terminology, risk scores, and references are consistent throughout
3. **Check completeness**: All template sections should be populated or explicitly marked N/A
4. **Format cleanup**: Ensure markdown formatting is clean (tables aligned, headers consistent, no orphaned references)
---
## 3. Completion Summary
Report:
- Test files created
- Checklist output path
- Key risks or assumptions
- Next recommended workflow (e.g., implementation or `automate`)
---
## 4. Save Progress
**Save this step's accumulated work to `{outputFile}`.**
- **If `{outputFile}` does not exist** (first save), create it with YAML frontmatter:
```yaml
---
stepsCompleted: ['step-05-validate-and-complete']
lastStep: 'step-05-validate-and-complete'
lastSaved: '{date}'
---
```
Then write this step's output below the frontmatter.
- **If `{outputFile}` already exists**, update:
- Add `'step-05-validate-and-complete'` to `stepsCompleted` array (only if not already present)
- Set `lastStep: 'step-05-validate-and-complete'`
- Set `lastSaved: '{date}'`
- Append this step's output to the appropriate section.
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Step completed in full with required outputs
### ❌ SYSTEM FAILURE:
- Skipped sequence steps or missing outputs
**Master Rule:** Skipping steps is FORBIDDEN.

View File

@@ -0,0 +1,65 @@
---
name: 'step-01-assess'
description: 'Load an existing output for editing'
nextStepFile: './step-02-apply-edit.md'
---
# Step 1: Assess Edit Target
## STEP GOAL:
Identify which output should be edited and load it.
## MANDATORY EXECUTION RULES (READ FIRST):
### Universal Rules:
- 📖 Read the complete step file before taking any action
- ✅ Speak in `{communication_language}`
### Role Reinforcement:
- ✅ You are the Master Test Architect
### Step-Specific Rules:
- 🎯 Ask the user which output file to edit
- 🚫 Do not edit until target is confirmed
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
## CONTEXT BOUNDARIES:
- Available context: existing outputs
- Focus: select edit target
- Limits: no edits yet
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly.
### 1. Identify Target
Ask the user to provide the output file path or select from known outputs.
### 2. Load Target
Read the provided output file in full.
### 3. Confirm
Confirm the target and proceed to edit.
Load next step: `{nextStepFile}`
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Target identified and loaded
### ❌ SYSTEM FAILURE:
- Proceeding without a confirmed target

View File

@@ -0,0 +1,60 @@
---
name: 'step-02-apply-edit'
description: 'Apply edits to the selected output'
---
# Step 2: Apply Edits
## STEP GOAL:
Apply the requested edits to the selected output and confirm changes.
## MANDATORY EXECUTION RULES (READ FIRST):
### Universal Rules:
- 📖 Read the complete step file before taking any action
- ✅ Speak in `{communication_language}`
### Role Reinforcement:
- ✅ You are the Master Test Architect
### Step-Specific Rules:
- 🎯 Only apply edits explicitly requested by the user
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
## CONTEXT BOUNDARIES:
- Available context: selected output and user changes
- Focus: apply edits only
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly.
### 1. Confirm Requested Changes
Restate what will be changed and confirm.
### 2. Apply Changes
Update the output file accordingly.
### 3. Report
Summarize the edits applied.
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Changes applied and confirmed
### ❌ SYSTEM FAILURE:
- Unconfirmed edits or missing update

View File

@@ -0,0 +1,67 @@
---
name: 'step-01-validate'
description: 'Validate workflow outputs against checklist'
outputFile: '{test_artifacts}/atdd-validation-report.md'
validationChecklist: '../checklist.md'
---
# Step 1: Validate Outputs
## STEP GOAL:
Validate outputs using the workflow checklist and record findings.
## MANDATORY EXECUTION RULES (READ FIRST):
### Universal Rules:
- 📖 Read the complete step file before taking any action
- ✅ Speak in `{communication_language}`
### Role Reinforcement:
- ✅ You are the Master Test Architect
### Step-Specific Rules:
- 🎯 Validate against `{validationChecklist}`
- 🚫 Do not skip checks
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
- 💾 Write findings to `{outputFile}`
## CONTEXT BOUNDARIES:
- Available context: workflow outputs and checklist
- Focus: validation only
- Limits: do not modify outputs in this step
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly.
### 1. Load Checklist
Read `{validationChecklist}` and list all criteria.
### 2. Validate Outputs
Evaluate outputs against each checklist item.
### 3. Write Report
Write a validation report to `{outputFile}` with PASS/WARN/FAIL per section.
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Validation report written
- All checklist items evaluated
### ❌ SYSTEM FAILURE:
- Skipped checklist items
- No report produced

View File

@@ -0,0 +1,73 @@
---
validationDate: 2026-01-27
workflowName: testarch-atdd
workflowPath: {project-root}/src/workflows/testarch/atdd
validationStatus: COMPLETE
completionDate: 2026-01-27 10:03:10
---
# Validation Report: testarch-atdd
**Validation Started:** 2026-01-27 09:50:21
**Validator:** BMAD Workflow Validation System (Codex)
**Standards Version:** BMAD Workflow Standards
## File Structure & Size
- workflow.md present: YES
- instructions.md present: YES
- workflow.yaml present: YES
- step files found: 8
**Step File Sizes:**
- steps-c/step-01-preflight-and-context.md: 101 lines [GOOD]
- steps-c/step-02-generation-mode.md: 71 lines [GOOD]
- steps-c/step-03-test-strategy.md: 70 lines [GOOD]
- steps-c/step-04-generate-tests.md: 70 lines [GOOD]
- steps-c/step-05-validate-and-complete.md: 61 lines [GOOD]
- steps-e/step-01-assess.md: 51 lines [GOOD]
- steps-e/step-02-apply-edit.md: 46 lines [GOOD]
- steps-v/step-01-validate.md: 53 lines [GOOD]
- workflow-plan.md present: YES
## Frontmatter Validation
- No frontmatter violations found
## Critical Path Violations
- No {project-root} hardcoded paths detected in body
- No dead relative links detected
## Menu Handling Validation
- No menu structures detected (linear step flow) [N/A]
## Step Type Validation
- Last step steps-v/step-01-validate.md has no nextStepFile (final step OK)
- Step type validation assumes linear sequence (no branching/menu). Workflow-plan.md present for reference. [INFO]
## Output Format Validation
- Templates present: atdd-checklist-template.md
- Steps with outputFile in frontmatter:
- steps-c/step-04-generate-tests.md
- steps-v/step-01-validate.md
## Validation Design Check
- checklist.md present: YES
- Validation steps folder (steps-v) present: YES
## Instruction Style Check
- All steps include STEP GOAL, MANDATORY EXECUTION RULES, EXECUTION PROTOCOLS, CONTEXT BOUNDARIES, and SUCCESS/FAILURE metrics
## Summary
- Validation completed: 2026-01-27 10:03:10
- Critical issues: 0
- Warnings: 0 (informational notes only)
- Readiness: READY (manual review optional)

View File

@@ -0,0 +1,116 @@
---
validationDate: 2026-01-27
workflowName: testarch-atdd
workflowPath: {project-root}/src/workflows/testarch/atdd
validationStatus: COMPLETE
completionDate: 2026-01-27 10:24:01
---
# Validation Report: testarch-atdd
**Validation Started:** 2026-01-27 10:24:01
**Validator:** BMAD Workflow Validation System (Codex)
**Standards Version:** BMAD Workflow Standards
## File Structure & Size
- workflow.md present: YES
- instructions.md present: YES
- workflow.yaml present: YES
- step files found: 8
**Step File Sizes:**
- steps-c/step-01-preflight-and-context.md: 100 lines [GOOD]
- steps-c/step-02-generation-mode.md: 70 lines [GOOD]
- steps-c/step-03-test-strategy.md: 69 lines [GOOD]
- steps-c/step-04-generate-tests.md: 69 lines [GOOD]
- steps-c/step-05-validate-and-complete.md: 60 lines [GOOD]
- steps-e/step-01-assess.md: 50 lines [GOOD]
- steps-e/step-02-apply-edit.md: 45 lines [GOOD]
- steps-v/step-01-validate.md: 52 lines [GOOD]
- workflow-plan.md present: YES
## Frontmatter Validation
- No frontmatter violations found
## Critical Path Violations
### Config Variables (Exceptions)
Standard BMAD config variables treated as valid exceptions: bmb_creations_output_folder, communication_language, document_output_language, output_folder, planning_artifacts, project-root, project_name, test_artifacts, user_name
- No {project-root} hardcoded paths detected in body
- No dead relative links detected
- No module path assumptions detected
**Status:** ✅ PASS - No critical violations
## Menu Handling Validation
- No menu structures detected (linear step flow) [N/A]
## Step Type Validation
- steps-c/step-01-preflight-and-context.md: Init [PASS]
- steps-c/step-02-generation-mode.md: Middle [PASS]
- steps-c/step-03-test-strategy.md: Middle [PASS]
- steps-c/step-04-generate-tests.md: Middle [PASS]
- steps-c/step-05-validate-and-complete.md: Final [PASS]
- Step type validation assumes linear sequence (no branching/menu). Workflow-plan.md present for reference. [INFO]
## Output Format Validation
- Templates present: atdd-checklist-template.md
- Steps with outputFile in frontmatter:
- steps-c/step-04-generate-tests.md
- steps-v/step-01-validate.md
- checklist.md present: YES
## Validation Design Check
- Validation steps folder (steps-v) present: YES
- Validation step(s) present: step-01-validate.md
- Validation steps reference checklist data and auto-proceed
## Instruction Style Check
- Instruction style: Prescriptive (appropriate for TEA quality/compliance workflows)
- Steps emphasize mandatory sequence, explicit success/failure metrics, and risk-based guidance
## Collaborative Experience Check
- Overall facilitation quality: GOOD
- Steps use progressive prompts and clear role reinforcement; no laundry-list interrogation detected
- Flow progression is clear and aligned to workflow goals
## Subagent Optimization Opportunities
- No high-priority subagent optimizations identified; workflow already uses step-file architecture
- Pattern 1 (grep/regex): N/A for most steps
- Pattern 2 (per-file analysis): already aligned to validation structure
- Pattern 3 (data ops): minimal data file loads
- Pattern 4 (parallel): optional for validation only
## Cohesive Review
- Overall assessment: GOOD
- Flow is linear, goals are clear, and outputs map to TEA artifacts
- Voice and tone consistent with Test Architect persona
- Recommendation: READY (minor refinements optional)
## Plan Quality Validation
- Plan file present: workflow-plan.md
- Planned steps found: 8 (all implemented)
- Plan implementation status: Fully Implemented
## Summary
- Validation completed: 2026-01-27 10:24:01
- Critical issues: 0
- Warnings: 0 (informational notes only)
- Readiness: READY (manual review optional)

View File

@@ -0,0 +1,21 @@
# Workflow Plan: testarch-atdd
## Create Mode (steps-c)
- step-01-preflight-and-context.md
- step-02-generation-mode.md
- step-03-test-strategy.md
- step-04-generate-tests.md
- step-05-validate-and-complete.md
## Validate Mode (steps-v)
- step-01-validate.md
## Edit Mode (steps-e)
- step-01-assess.md
- step-02-apply-edit.md
## Outputs
- {test_artifacts}/atdd-checklist-{story_id}.md
- Failing acceptance tests under {project-root}/tests

View File

@@ -0,0 +1,41 @@
---
name: testarch-atdd
description: Generate failing acceptance tests using TDD cycle. Use when user says 'lets write acceptance tests' or 'I want to do ATDD'
web_bundle: true
---
# Acceptance Test-Driven Development (ATDD)
**Goal:** Generate failing acceptance tests before implementation using TDD red-green-refactor cycle
**Role:** You are the Master Test Architect.
---
## WORKFLOW ARCHITECTURE
This workflow uses **tri-modal step-file architecture**:
- **Create mode (steps-c/)**: primary execution flow
- **Validate mode (steps-v/)**: validation against checklist
- **Edit mode (steps-e/)**: revise existing outputs
---
## INITIALIZATION SEQUENCE
### 1. Mode Determination
"Welcome to the workflow. What would you like to do?"
- **[C] Create** — Run the workflow
- **[R] Resume** — Resume an interrupted workflow
- **[V] Validate** — Validate existing outputs
- **[E] Edit** — Edit existing outputs
### 2. Route to First Step
- **If C:** Load `steps-c/step-01-preflight-and-context.md`
- **If R:** Load `steps-c/step-01b-resume.md`
- **If V:** Load `steps-v/step-01-validate.md`
- **If E:** Load `steps-e/step-01-assess.md`

View File

@@ -0,0 +1,46 @@
# Test Architect workflow: atdd
name: testarch-atdd
# prettier-ignore
description: 'Generate failing acceptance tests using TDD cycle. Use when the user says "lets write acceptance tests" or "I want to do ATDD"'
# Critical variables from config
config_source: "{project-root}/_bmad/tea/config.yaml"
output_folder: "{config_source}:output_folder"
test_artifacts: "{config_source}:test_artifacts"
user_name: "{config_source}:user_name"
communication_language: "{config_source}:communication_language"
document_output_language: "{config_source}:document_output_language"
date: system-generated
# Workflow components
installed_path: "{project-root}/_bmad/tea/workflows/testarch/atdd"
instructions: "{installed_path}/instructions.md"
validation: "{installed_path}/checklist.md"
template: "{installed_path}/atdd-checklist-template.md"
# Variables and inputs
variables:
test_dir: "{project-root}/tests" # Root test directory
# Output configuration
default_output_file: "{test_artifacts}/atdd-checklist-{story_id}.md"
# Required tools
required_tools:
- read_file # Read story markdown, framework config
- write_file # Create test files, checklist, factory stubs
- create_directory # Create test directories
- list_files # Find existing fixtures and helpers
- search_repo # Search for similar test patterns
tags:
- qa
- atdd
- test-architect
- tdd
- red-green-refactor
execution_hints:
interactive: false # Minimize prompts
autonomous: true # Proceed without user input unless blocked
iterative: true

View File

@@ -0,0 +1,611 @@
# Automate Workflow Validation Checklist
Use this checklist to validate that the automate workflow has been executed correctly and all deliverables meet quality standards.
## Prerequisites
Before starting this workflow, verify:
- [ ] Framework scaffolding configured (playwright.config.ts or cypress.config.ts exists)
- [ ] Test directory structure exists (tests/ folder with subdirectories)
- [ ] Package.json has test framework dependencies installed
**Halt only if:** Framework scaffolding is completely missing (run `framework` workflow first)
**Note:** BMad artifacts (story, tech-spec, PRD) are OPTIONAL - workflow can run without them
**Note:** `automate` generates tests; it does not run `*atdd` or `*test-review`. If ATDD outputs exist, use them as input and avoid duplicate coverage.
---
## Step 1: Execution Mode Determination and Context Loading
### Mode Detection
- [ ] Execution mode correctly determined:
- [ ] BMad-Integrated Mode (story_file variable set) OR
- [ ] Standalone Mode (target_feature or target_files set) OR
- [ ] Auto-discover Mode (no targets specified)
### BMad Artifacts (If Available - OPTIONAL)
- [ ] Story markdown loaded (if `{story_file}` provided)
- [ ] Acceptance criteria extracted from story (if available)
- [ ] Tech-spec.md loaded (if `{use_tech_spec}` true and file exists)
- [ ] Test-design.md loaded (if `{use_test_design}` true and file exists)
- [ ] PRD.md loaded (if `{use_prd}` true and file exists)
- [ ] **Note**: Absence of BMad artifacts does NOT halt workflow
### Framework Configuration
- [ ] Test framework config loaded (playwright.config.ts or cypress.config.ts)
- [ ] Test directory structure identified from `{test_dir}`
- [ ] Existing test patterns reviewed
- [ ] Test runner capabilities noted (parallel execution, fixtures, etc.)
### Coverage Analysis
- [ ] Existing test files searched in `{test_dir}` (if `{analyze_coverage}` true)
- [ ] Tested features vs untested features identified
- [ ] Coverage gaps mapped (tests to source files)
- [ ] Existing fixture and factory patterns checked
### Knowledge Base Fragments Loaded
- [ ] `test-levels-framework.md` - Test level selection
- [ ] `test-priorities.md` - Priority classification (P0-P3)
- [ ] `fixture-architecture.md` - Fixture patterns with auto-cleanup
- [ ] `data-factories.md` - Factory patterns using faker
- [ ] `selective-testing.md` - Targeted test execution strategies
- [ ] `ci-burn-in.md` - Flaky test detection patterns
- [ ] `test-quality.md` - Test design principles
---
## Step 2: Automation Targets Identification
### Target Determination
**BMad-Integrated Mode (if story available):**
- [ ] Acceptance criteria mapped to test scenarios
- [ ] Features implemented in story identified
- [ ] Existing ATDD tests checked (if any)
- [ ] Expansion beyond ATDD planned (edge cases, negative paths)
**Standalone Mode (if no story):**
- [ ] Specific feature analyzed (if `{target_feature}` specified)
- [ ] Specific files analyzed (if `{target_files}` specified)
- [ ] Features auto-discovered (if `{auto_discover_features}` true)
- [ ] Features prioritized by:
- [ ] No test coverage (highest priority)
- [ ] Complex business logic
- [ ] External integrations (API, database, auth)
- [ ] Critical user paths (login, checkout, etc.)
### Test Level Selection
- [ ] Test level selection framework applied (from `test-levels-framework.md`)
- [ ] E2E tests identified: Critical user journeys, multi-system integration
- [ ] API tests identified: Business logic, service contracts, data transformations
- [ ] Component tests identified: UI behavior, interactions, state management
- [ ] Unit tests identified: Pure logic, edge cases, error handling
### Duplicate Coverage Avoidance
- [ ] Same behavior NOT tested at multiple levels unnecessarily
- [ ] E2E used for critical happy path only
- [ ] API tests used for business logic variations
- [ ] Component tests used for UI interaction edge cases
- [ ] Unit tests used for pure logic edge cases
### Priority Assignment
- [ ] Test priorities assigned using `test-priorities.md` framework
- [ ] P0 tests: Critical paths, security-critical, data integrity
- [ ] P1 tests: Important features, integration points, error handling
- [ ] P2 tests: Edge cases, less-critical variations, performance
- [ ] P3 tests: Nice-to-have, rarely-used features, exploratory
- [ ] Priority variables respected:
- [ ] `{include_p0}` = true (always include)
- [ ] `{include_p1}` = true (high priority)
- [ ] `{include_p2}` = true (medium priority)
- [ ] `{include_p3}` = false (low priority, skip by default)
### Coverage Plan Created
- [ ] Test coverage plan documented
- [ ] What will be tested at each level listed
- [ ] Priorities assigned to each test
- [ ] Coverage strategy clear (critical-paths, comprehensive, or selective)
---
## Step 3: Test Infrastructure Generated
### Fixture Architecture
- [ ] Existing fixtures checked in `tests/support/fixtures/`
- [ ] Fixture architecture created/enhanced (if `{generate_fixtures}` true)
- [ ] All fixtures use Playwright's `test.extend()` pattern
- [ ] All fixtures have auto-cleanup in teardown
- [ ] Common fixtures created/enhanced:
- [ ] authenticatedUser (with auto-delete)
- [ ] apiRequest (authenticated client)
- [ ] mockNetwork (external service mocking)
- [ ] testDatabase (with auto-cleanup)
### Data Factories
- [ ] Existing factories checked in `tests/support/factories/`
- [ ] Factory architecture created/enhanced (if `{generate_factories}` true)
- [ ] All factories use `@faker-js/faker` for random data (no hardcoded values)
- [ ] All factories support overrides for specific scenarios
- [ ] Common factories created/enhanced:
- [ ] User factory (email, password, name, role)
- [ ] Product factory (name, price, SKU)
- [ ] Order factory (items, total, status)
- [ ] Cleanup helpers provided (e.g., deleteUser(), deleteProduct())
### Helper Utilities
- [ ] Existing helpers checked in `tests/support/helpers/` (if `{update_helpers}` true)
- [ ] Common utilities created/enhanced:
- [ ] waitFor (polling for complex conditions)
- [ ] retry (retry helper for flaky operations)
- [ ] testData (test data generation)
- [ ] assertions (custom assertion helpers)
---
## Step 4: Test Files Generated
### Test File Structure
- [ ] Test files organized correctly:
- [ ] `tests/e2e/` for E2E tests
- [ ] `tests/api/` for API tests
- [ ] `tests/component/` for component tests
- [ ] `tests/unit/` for unit tests
- [ ] `tests/support/` for fixtures/factories/helpers
### E2E Tests (If Applicable)
- [ ] E2E test files created in `tests/e2e/`
- [ ] All tests follow Given-When-Then format
- [ ] All tests have priority tags ([P0], [P1], [P2], [P3]) in test name
- [ ] All tests use data-testid selectors (not CSS classes)
- [ ] One assertion per test (atomic design)
- [ ] No hard waits or sleeps (explicit waits only)
- [ ] Network-first pattern applied (route interception BEFORE navigation)
- [ ] Clear Given-When-Then comments in test code
### API Tests (If Applicable)
- [ ] API test files created in `tests/api/`
- [ ] All tests follow Given-When-Then format
- [ ] All tests have priority tags in test name
- [ ] API contracts validated (request/response structure)
- [ ] HTTP status codes verified
- [ ] Response body validation includes required fields
- [ ] Error cases tested (400, 401, 403, 404, 500)
- [ ] JWT token format validated (if auth tests)
### Consumer Contract Tests / CDC (If `use_pactjs_utils` Enabled)
**Provider Endpoint Comments:**
- [ ] Every Pact interaction has `// Provider endpoint:` comment
- [ ] Comment includes exact file path to provider route handler, OR uses the TODO form when provider is inaccessible
- [ ] Comment follows format: `// Provider endpoint: <path> -> <METHOD> <route>` or `// Provider endpoint: TODO — provider source not accessible, verify manually`
**Provider Source Scrutiny:**
- [ ] Provider route handlers and/or OpenAPI spec read before generating each interaction
- [ ] Status codes verified against provider source (e.g., 201 not assumed 200)
- [ ] Field names cross-referenced with provider type/DTO definitions
- [ ] Data types verified (string ID vs number ID, date formats)
- [ ] Enum/union values extracted from provider validation schemas
- [ ] Required request fields and headers checked against provider validation
- [ ] Nested response structures match provider's actual response construction
- [ ] Scrutiny evidence documented as block comment in each test file
**CDC Quality Gates:**
- [ ] Postel's Law enforced: exact values in `withRequest`, matchers in `willRespondWith`
- [ ] Response matchers (`like`, `eachLike`, `string`, `integer`) used only in `willRespondWith`
- [ ] Provider state names are consistent with provider's state handler naming
- [ ] DI pattern used for consumer function imports (actual consumer code, not raw `fetch()`)
- [ ] One logical endpoint per Pact interaction (no multi-endpoint interactions)
### Component Tests (If Applicable)
- [ ] Component test files created in `tests/component/`
- [ ] All tests follow Given-When-Then format
- [ ] All tests have priority tags in test name
- [ ] Component mounting works correctly
- [ ] Interaction testing covers user actions (click, hover, keyboard)
- [ ] State management validated
- [ ] Props and events tested
### Unit Tests (If Applicable)
- [ ] Unit test files created in `tests/unit/`
- [ ] All tests follow Given-When-Then format
- [ ] All tests have priority tags in test name
- [ ] Pure logic tested (no dependencies)
- [ ] Edge cases covered
- [ ] Error handling tested
### Quality Standards Enforced
- [ ] All tests use Given-When-Then format with clear comments
- [ ] All tests have descriptive names with priority tags
- [ ] No duplicate tests (same behavior tested multiple times)
- [ ] No flaky patterns (race conditions, timing issues)
- [ ] No test interdependencies (tests can run in any order)
- [ ] Tests are deterministic (same input always produces same result)
- [ ] All tests use data-testid selectors (E2E tests)
- [ ] No hard waits: `await page.waitForTimeout()` (forbidden)
- [ ] No conditional flow: `if (await element.isVisible())` (forbidden)
- [ ] No try-catch for test logic (only for cleanup)
- [ ] No hardcoded test data (use factories with faker)
- [ ] No page object classes (tests are direct and simple)
- [ ] No shared state between tests
### Network-First Pattern Applied
- [ ] Route interception set up BEFORE navigation (E2E tests with network requests)
- [ ] `page.route()` called before `page.goto()` to prevent race conditions
- [ ] Network-first pattern verified in all E2E tests that make API calls
---
## Step 5: Test Validation and Healing (NEW - Phase 2.5)
### Healing Configuration
- [ ] Healing configuration checked:
- [ ] `{auto_validate}` setting noted (default: true)
- [ ] `{auto_heal_failures}` setting noted (default: false)
- [ ] `{max_healing_iterations}` setting noted (default: 3)
- [ ] `{use_mcp_healing}` setting noted (default: true)
### Healing Knowledge Fragments Loaded (If Healing Enabled)
- [ ] `test-healing-patterns.md` loaded (common failure patterns and fixes)
- [ ] `selector-resilience.md` loaded (selector refactoring guide)
- [ ] `timing-debugging.md` loaded (race condition fixes)
### Test Execution and Validation
- [ ] Generated tests executed (if `{auto_validate}` true)
- [ ] Test results captured:
- [ ] Total tests run
- [ ] Passing tests count
- [ ] Failing tests count
- [ ] Error messages and stack traces captured
### Healing Loop (If Enabled and Tests Failed)
- [ ] Healing loop entered (if `{auto_heal_failures}` true AND tests failed)
- [ ] For each failing test:
- [ ] Failure pattern identified (selector, timing, data, network, hard wait)
- [ ] Appropriate healing strategy applied:
- [ ] Stale selector → Replaced with data-testid or ARIA role
- [ ] Race condition → Added network-first interception or state waits
- [ ] Dynamic data → Replaced hardcoded values with regex/dynamic generation
- [ ] Network error → Added route mocking
- [ ] Hard wait → Replaced with event-based wait
- [ ] Healed test re-run to validate fix
- [ ] Iteration count tracked (max 3 attempts)
### Unfixable Tests Handling
- [ ] Tests that couldn't be healed after 3 iterations marked with `test.fixme()` (if `{mark_unhealable_as_fixme}` true)
- [ ] Detailed comment added to test.fixme() tests:
- [ ] What failure occurred
- [ ] What healing was attempted (3 iterations)
- [ ] Why healing failed
- [ ] Manual investigation steps needed
- [ ] Original test logic preserved in comments
### Healing Report Generated
- [ ] Healing report generated (if healing attempted)
- [ ] Report includes:
- [ ] Auto-heal enabled status
- [ ] Healing mode (MCP-assisted or Pattern-based)
- [ ] Iterations allowed (max_healing_iterations)
- [ ] Validation results (total, passing, failing)
- [ ] Successfully healed tests (count, file:line, fix applied)
- [ ] Unable to heal tests (count, file:line, reason)
- [ ] Healing patterns applied (selector fixes, timing fixes, data fixes)
- [ ] Knowledge base references used
---
## Step 6: Documentation and Scripts Updated
### Test README Updated
- [ ] `tests/README.md` created or updated (if `{update_readme}` true)
- [ ] Test suite structure overview included
- [ ] Test execution instructions provided (all, specific files, by priority)
- [ ] Fixture usage examples provided
- [ ] Factory usage examples provided
- [ ] Priority tagging convention explained ([P0], [P1], [P2], [P3])
- [ ] How to write new tests documented
- [ ] Common patterns documented
- [ ] Anti-patterns documented (what to avoid)
### package.json Scripts Updated
- [ ] package.json scripts added/updated (if `{update_package_scripts}` true)
- [ ] `test:e2e` script for all E2E tests
- [ ] `test:e2e:p0` script for P0 tests only
- [ ] `test:e2e:p1` script for P0 + P1 tests
- [ ] `test:api` script for API tests
- [ ] `test:component` script for component tests
- [ ] `test:unit` script for unit tests (if applicable)
### Test Suite Executed
- [ ] Test suite run locally (if `{run_tests_after_generation}` true)
- [ ] Test results captured (passing/failing counts)
- [ ] No flaky patterns detected (tests are deterministic)
- [ ] Setup requirements documented (if any)
- [ ] Known issues documented (if any)
---
## Step 6: Automation Summary Generated
### Automation Summary Document
- [ ] Output file created at `{output_summary}`
- [ ] Document includes execution mode (BMad-Integrated, Standalone, Auto-discover)
- [ ] Feature analysis included (source files, coverage gaps) - Standalone mode
- [ ] Tests created listed (E2E, API, Component, Unit) with counts and paths
- [ ] Infrastructure created listed (fixtures, factories, helpers)
- [ ] Test execution instructions provided
- [ ] Coverage analysis included:
- [ ] Total test count
- [ ] Priority breakdown (P0, P1, P2, P3 counts)
- [ ] Test level breakdown (E2E, API, Component, Unit counts)
- [ ] Coverage percentage (if calculated)
- [ ] Coverage status (acceptance criteria covered, gaps identified)
- [ ] Definition of Done checklist included
- [ ] Next steps provided
- [ ] Recommendations included (if Standalone mode)
### Summary Provided to User
- [ ] Concise summary output provided
- [ ] Total tests created across test levels
- [ ] Priority breakdown (P0, P1, P2, P3 counts)
- [ ] Infrastructure counts (fixtures, factories, helpers)
- [ ] Test execution command provided
- [ ] Output file path provided
- [ ] Next steps listed
---
## Quality Checks
### Test Design Quality
- [ ] Tests are readable (clear Given-When-Then structure)
- [ ] Tests are maintainable (use factories/fixtures, not hardcoded data)
- [ ] Tests are isolated (no shared state between tests)
- [ ] Tests are deterministic (no race conditions or flaky patterns)
- [ ] Tests are atomic (one assertion per test)
- [ ] Tests are fast (no unnecessary waits or delays)
- [ ] Tests are lean (files under {max_file_lines} lines)
### Knowledge Base Integration
- [ ] Test level selection framework applied (from `test-levels-framework.md`)
- [ ] Priority classification applied (from `test-priorities.md`)
- [ ] Fixture architecture patterns applied (from `fixture-architecture.md`)
- [ ] Data factory patterns applied (from `data-factories.md`)
- [ ] Selective testing strategies considered (from `selective-testing.md`)
- [ ] Flaky test detection patterns considered (from `ci-burn-in.md`)
- [ ] Test quality principles applied (from `test-quality.md`)
### Code Quality
- [ ] All TypeScript types are correct and complete
- [ ] No linting errors in generated test files
- [ ] Consistent naming conventions followed
- [ ] Imports are organized and correct
- [ ] Code follows project style guide
- [ ] No console.log or debug statements in test code
---
## Integration Points
### With Framework Workflow
- [ ] Test framework configuration detected and used
- [ ] Directory structure matches framework setup
- [ ] Fixtures and helpers follow established patterns
- [ ] Naming conventions consistent with framework standards
### With BMad Workflows (If Available - OPTIONAL)
**With Story Workflow:**
- [ ] Story ID correctly referenced in output (if story available)
- [ ] Acceptance criteria from story reflected in tests (if story available)
- [ ] Technical constraints from story considered (if story available)
**With test-design Workflow:**
- [ ] P0 scenarios from test-design prioritized (if test-design available)
- [ ] Risk assessment from test-design considered (if test-design available)
- [ ] Coverage strategy aligned with test-design (if test-design available)
**With atdd Workflow:**
- [ ] ATDD artifacts provided or located (manual handoff; `atdd` not auto-run)
- [ ] Existing ATDD tests checked (if story had ATDD workflow run)
- [ ] Expansion beyond ATDD planned (edge cases, negative paths)
- [ ] No duplicate coverage with ATDD tests
### With CI Pipeline
- [ ] Tests can run in CI environment
- [ ] Tests are parallelizable (no shared state)
- [ ] Tests have appropriate timeouts
- [ ] Tests clean up their data (no CI environment pollution)
---
## Completion Criteria
All of the following must be true before marking this workflow as complete:
- [ ] **Execution mode determined** (BMad-Integrated, Standalone, or Auto-discover)
- [ ] **Framework configuration loaded** and validated
- [ ] **Coverage analysis completed** (gaps identified if analyze_coverage true)
- [ ] **Automation targets identified** (what needs testing)
- [ ] **Test levels selected** appropriately (E2E, API, Component, Unit)
- [ ] **Duplicate coverage avoided** (same behavior not tested at multiple levels)
- [ ] **Test priorities assigned** (P0, P1, P2, P3)
- [ ] **Fixture architecture created/enhanced** with auto-cleanup
- [ ] **Data factories created/enhanced** using faker (no hardcoded data)
- [ ] **Helper utilities created/enhanced** (if needed)
- [ ] **Test files generated** at appropriate levels (E2E, API, Component, Unit)
- [ ] **Given-When-Then format used** consistently across all tests
- [ ] **Priority tags added** to all test names ([P0], [P1], [P2], [P3])
- [ ] **data-testid selectors used** in E2E tests (not CSS classes)
- [ ] **Network-first pattern applied** (route interception before navigation)
- [ ] **Quality standards enforced** (no hard waits, no flaky patterns, self-cleaning, deterministic)
- [ ] **Test README updated** with execution instructions and patterns
- [ ] **package.json scripts updated** with test execution commands
- [ ] **Test suite run locally** (if run_tests_after_generation true)
- [ ] **Tests validated** (if auto_validate enabled)
- [ ] **Failures healed** (if auto_heal_failures enabled and tests failed)
- [ ] **Healing report generated** (if healing attempted)
- [ ] **Unfixable tests marked** with test.fixme() and detailed comments (if any)
- [ ] **Automation summary created** and saved to correct location
- [ ] **Output file formatted correctly**
- [ ] **Knowledge base references applied** and documented (including healing fragments if used)
- [ ] **No test quality issues** (flaky patterns, race conditions, hardcoded data, page objects)
- [ ] **Provider scrutiny completed or gracefully degraded** for all CDC interactions — each interaction either has scrutiny evidence or a TODO marker (if `use_pactjs_utils` enabled)
- [ ] **Provider endpoint comments present** on every Pact interaction (if `use_pactjs_utils` enabled)
---
## Common Issues and Resolutions
### Issue: BMad artifacts not found
**Problem:** Story, tech-spec, or PRD files not found when variables are set.
**Resolution:**
- **automate does NOT require BMad artifacts** - they are OPTIONAL enhancements
- If files not found, switch to Standalone Mode automatically
- Analyze source code directly without BMad context
- Continue workflow without halting
### Issue: Framework configuration not found
**Problem:** No playwright.config.ts or cypress.config.ts found.
**Resolution:**
- **HALT workflow** - framework is required
- Message: "Framework scaffolding required. Run `bmad tea *framework` first."
- User must run framework workflow before automate
### Issue: No automation targets identified
**Problem:** Neither story, target_feature, nor target_files specified, and auto-discover finds nothing.
**Resolution:**
- Check if source_dir variable is correct
- Verify source code exists in project
- Ask user to specify target_feature or target_files explicitly
- Provide examples: `target_feature: "src/auth/"` or `target_files: "src/auth/login.ts,src/auth/session.ts"`
### Issue: Duplicate coverage detected
**Problem:** Same behavior tested at multiple levels (E2E + API + Component).
**Resolution:**
- Review test level selection framework (test-levels-framework.md)
- Use E2E for critical happy path ONLY
- Use API for business logic variations
- Use Component for UI edge cases
- Remove redundant tests that duplicate coverage
### Issue: Tests have hardcoded data
**Problem:** Tests use hardcoded email addresses, passwords, or other data.
**Resolution:**
- Replace all hardcoded data with factory function calls
- Use faker for all random data generation
- Update data-factories to support all required test scenarios
- Example: `createUser({ email: faker.internet.email() })`
### Issue: Tests are flaky
**Problem:** Tests fail intermittently, pass on retry.
**Resolution:**
- Remove all hard waits (`page.waitForTimeout()`)
- Use explicit waits (`page.waitForSelector()`)
- Apply network-first pattern (route interception before navigation)
- Remove conditional flow (`if (await element.isVisible())`)
- Ensure tests are deterministic (no race conditions)
- Run burn-in loop (10 iterations) to detect flakiness
### Issue: Fixtures don't clean up data
**Problem:** Test data persists after test run, causing test pollution.
**Resolution:**
- Ensure all fixtures have cleanup in teardown phase
- Cleanup happens AFTER `await use(data)`
- Call deletion/cleanup functions (deleteUser, deleteProduct, etc.)
- Verify cleanup works by checking database/storage after test run
### Issue: Tests too slow
**Problem:** Tests take longer than 90 seconds (max_test_duration).
**Resolution:**
- Remove unnecessary waits and delays
- Use parallel execution where possible
- Mock external services (don't make real API calls)
- Use API tests instead of E2E for business logic
- Optimize test data creation (use in-memory database, etc.)
---
## Notes for TEA Agent
- **automate is flexible:** Can work with or without BMad artifacts (story, tech-spec, PRD are OPTIONAL)
- **Standalone mode is powerful:** Analyze any codebase and generate tests independently
- **Auto-discover mode:** Scan codebase for features needing tests when no targets specified
- **Framework is the ONLY hard requirement:** HALT if framework config missing, otherwise proceed
- **Avoid duplicate coverage:** E2E for critical paths only, API/Component for variations
- **Priority tagging enables selective execution:** P0 tests run on every commit, P1 on PR, P2 nightly
- **Network-first pattern prevents race conditions:** Route interception BEFORE navigation
- **No page objects:** Keep tests simple, direct, and maintainable
- **Use knowledge base:** Load relevant fragments (test-levels, test-priorities, fixture-architecture, data-factories, healing patterns) for guidance
- **Deterministic tests only:** No hard waits, no conditional flow, no flaky patterns allowed
- **Optional healing:** auto_heal_failures disabled by default (opt-in for automatic test healing)
- **Graceful degradation:** Healing works without Playwright MCP (pattern-based fallback)
- **Unfixable tests handled:** Mark with test.fixme() and detailed comments (not silently broken)

View File

@@ -0,0 +1,50 @@
<!-- Powered by BMAD-CORE™ -->
# Test Automation Expansion
**Workflow ID**: `_bmad/tea/testarch/automate`
**Version**: 5.0 (Step-File Architecture)
---
## Overview
Expands test automation coverage by generating prioritized tests at the appropriate level (E2E, API, Component, Unit) with supporting fixtures and helpers.
Modes:
- **BMad-Integrated**: Uses story/PRD/test-design artifacts when available
- **Standalone**: Analyzes existing codebase without BMad artifacts
---
## WORKFLOW ARCHITECTURE
This workflow uses **step-file architecture** for disciplined execution:
- **Micro-file Design**: Each step is self-contained
- **JIT Loading**: Only the current step file is in memory
- **Sequential Enforcement**: Execute steps in order without skipping
---
## INITIALIZATION SEQUENCE
### 1. Configuration Loading
From `workflow.yaml`, resolve:
- `config_source`, `test_artifacts`, `user_name`, `communication_language`, `document_output_language`, `date`
- `test_dir`, `source_dir`, `coverage_target`, `standalone_mode`
### 2. First Step
Load, read completely, and execute:
`{project-root}/_bmad/tea/workflows/testarch/automate/steps-c/step-01-preflight-and-context.md`
### 3. Resume Support
If the user selects **Resume** mode, load, read completely, and execute:
`{project-root}/_bmad/tea/workflows/testarch/automate/steps-c/step-01b-resume.md`
This checks the output document for progress tracking frontmatter and routes to the next incomplete step.

View File

@@ -0,0 +1,237 @@
---
name: 'step-01-preflight-and-context'
description: 'Determine mode, verify framework, and load context and knowledge'
outputFile: '{test_artifacts}/automation-summary.md'
nextStepFile: './step-02-identify-targets.md'
knowledgeIndex: '{project-root}/_bmad/tea/testarch/tea-index.csv'
---
# Step 1: Preflight & Context Loading
## STEP GOAL
Determine execution mode, verify framework readiness, and load the necessary artifacts and knowledge fragments.
## MANDATORY EXECUTION RULES
- 📖 Read the entire step file before acting
- ✅ Speak in `{communication_language}`
- 🚫 Halt if framework scaffolding is missing
---
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
- 💾 Record outputs before proceeding
- 📖 Load the next step only when instructed
## CONTEXT BOUNDARIES:
- Available context: config, loaded artifacts, and knowledge fragments
- Focus: this step's goal only
- Limits: do not execute future steps
- Dependencies: prior steps' outputs (if any)
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
## 1. Stack Detection & Verify Framework
**Read `config.test_stack_type`** from `{config_source}`.
**Auto-Detection Algorithm** (when `test_stack_type` is `"auto"` or not configured):
- Scan `{project-root}` for project manifests:
- **Frontend indicators**: `package.json` with react/vue/angular/next dependencies, `playwright.config.*`, `vite.config.*`, `webpack.config.*`
- **Backend indicators**: `pyproject.toml`, `pom.xml`/`build.gradle`, `go.mod`, `*.csproj`/`*.sln`, `Gemfile`, `Cargo.toml`
- **Both present** = `fullstack`; only frontend = `frontend`; only backend = `backend`
- Explicit `test_stack_type` config value overrides auto-detection
- **Backward compatibility**: if `test_stack_type` is not in config, treat as `"auto"` (preserves current frontend behavior for existing installs)
Store result as `{detected_stack}` = `frontend` | `backend` | `fullstack`
**Verify framework exists:**
**If {detected_stack} is `frontend` or `fullstack`:**
- `playwright.config.ts` or `cypress.config.ts`
- `package.json` includes test dependencies
**If {detected_stack} is `backend` or `fullstack`:**
- Relevant test config exists (e.g., `conftest.py`, `src/test/`, `*_test.go`, `.rspec`, test project `*.csproj`)
If missing: **HALT** with message "Run `framework` workflow first."
---
## 2. Determine Execution Mode
- **BMad-Integrated** if story/tech-spec/test-design artifacts are provided or found
- **Standalone** if only source code is available
- If unclear, ask the user which mode to use
---
## 3. Load Context
### BMad-Integrated (if available)
- Story with acceptance criteria
- PRD and/or tech spec
- Test-design document (if exists)
### Standalone
- Skip artifacts; proceed to codebase analysis
### Always Load
- Test framework config
- Existing test structure in `{test_dir}`
- Existing tests (for coverage gaps)
### Read TEA Config Flags
- From `{config_source}` read `tea_use_playwright_utils`
- From `{config_source}` read `tea_use_pactjs_utils`
- From `{config_source}` read `tea_pact_mcp`
- From `{config_source}` read `tea_browser_automation`
- From `{config_source}` read `test_stack_type`
---
### Tiered Knowledge Loading
Load fragments based on their `tier` classification in `tea-index.csv`:
1. **Core tier** (always load): Foundational fragments required for this workflow
2. **Extended tier** (load on-demand): Load when deeper analysis is needed or when the user's context requires it
3. **Specialized tier** (load only when relevant): Load only when the specific use case matches (e.g., contract-testing only for microservices, email-auth only for email flows)
> **Context Efficiency**: Loading only core fragments reduces context usage by 40-50% compared to loading all fragments.
### Playwright Utils Loading Profiles
**If `tea_use_playwright_utils` is enabled**, select the appropriate loading profile:
- **API-only profile** (when `{detected_stack}` is `backend` or no `page.goto`/`page.locator` found in test files):
Load: `overview`, `api-request`, `auth-session`, `recurse` (~1,800 lines)
- **Full UI+API profile** (when `{detected_stack}` is `frontend`/`fullstack` or browser tests detected):
Load: all Playwright Utils core fragments (~4,500 lines)
**Detection**: Scan `{test_dir}` for files containing `page.goto` or `page.locator`. If none found, use API-only profile.
### Pact.js Utils Loading
**If `tea_use_pactjs_utils` is enabled** (and `{detected_stack}` is `backend` or `fullstack`, or microservices indicators detected):
Load: `pactjs-utils-overview.md`, `pactjs-utils-consumer-helpers.md`, `pactjs-utils-provider-verifier.md`, `pactjs-utils-request-filter.md` (~800 lines)
**If `tea_use_pactjs_utils` is disabled** but contract testing is relevant (microservices architecture detected, existing Pact config found):
Load: `contract-testing.md` (~960 lines)
**Detection**: Scan `{project-root}` for Pact indicators: `pact/` directory, `@pact-foundation/pact` in `package.json`, `pactUrls` in test files, `PACT_BROKER` in env files.
### Pact MCP Loading
**If `tea_pact_mcp` is `"mcp"`:**
Load: `pact-mcp.md` (~150 lines) — enables agent to use SmartBear MCP tools for fetching provider states and generating pact tests during automation.
## 4. Load Knowledge Base Fragments
Use `{knowledgeIndex}` and load only what is required.
**Core (always load):**
- `test-levels-framework.md`
- `test-priorities-matrix.md`
- `data-factories.md`
- `selective-testing.md`
- `ci-burn-in.md`
- `test-quality.md`
**Playwright Utils (if enabled):**
- `overview.md`, `api-request.md`, `network-recorder.md`, `auth-session.md`, `intercept-network-call.md`, `recurse.md`, `log.md`, `file-utils.md`, `burn-in.md`, `network-error-monitor.md`, `fixtures-composition.md`
**Traditional Patterns (if Playwright Utils disabled):**
- `fixture-architecture.md`
- `network-first.md`
**Pact.js Utils (if enabled):**
- `pactjs-utils-overview.md`, `pactjs-utils-consumer-helpers.md`, `pactjs-utils-provider-verifier.md`, `pactjs-utils-request-filter.md`
**Contract Testing (if pactjs-utils disabled but relevant):**
- `contract-testing.md`
**Pact MCP (if tea_pact_mcp is "mcp"):**
- `pact-mcp.md`
**Healing (if auto-heal enabled):**
- `test-healing-patterns.md`
- `selector-resilience.md`
- `timing-debugging.md`
**Playwright CLI (if tea_browser_automation is "cli" or "auto"):**
- `playwright-cli.md`
**MCP Patterns (if tea_browser_automation is "mcp" or "auto"):**
- (existing MCP-related fragments, if any are added in future)
---
## 5. Confirm Inputs
Summarize loaded artifacts, framework, and knowledge fragments, then proceed.
---
## 6. Save Progress
**Save this step's accumulated work to `{outputFile}`.**
- **If `{outputFile}` does not exist** (first save), create it with YAML frontmatter:
```yaml
---
stepsCompleted: ['step-01-preflight-and-context']
lastStep: 'step-01-preflight-and-context'
lastSaved: '{date}'
---
```
Then write this step's output below the frontmatter.
- **If `{outputFile}` already exists**, update:
- Add `'step-01-preflight-and-context'` to `stepsCompleted` array (only if not already present)
- Set `lastStep: 'step-01-preflight-and-context'`
- Set `lastSaved: '{date}'`
- Append this step's output to the appropriate section.
**Update `inputDocuments`**: Set `inputDocuments` in the output template frontmatter to the list of artifact paths loaded in this step (e.g., knowledge fragments, test design documents, configuration files).
Load next step: `{nextStepFile}`
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Step completed in full with required outputs
### ❌ SYSTEM FAILURE:
- Skipped sequence steps or missing outputs
**Master Rule:** Skipping steps is FORBIDDEN.

View File

@@ -0,0 +1,94 @@
---
name: 'step-01b-resume'
description: 'Resume interrupted workflow from last completed step'
outputFile: '{test_artifacts}/automation-summary.md'
---
# Step 1b: Resume Workflow
## STEP GOAL
Resume an interrupted workflow by loading the existing output document, displaying progress, and routing to the next incomplete step.
## MANDATORY EXECUTION RULES
- 📖 Read the entire step file before acting
- ✅ Speak in `{communication_language}`
---
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
- 📖 Load the next step only when instructed
## CONTEXT BOUNDARIES:
- Available context: Output document with progress frontmatter
- Focus: Load progress and route to next step
- Limits: Do not re-execute completed steps
- Dependencies: Output document must exist from a previous run
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly.
### 1. Load Output Document
Read `{outputFile}` and parse YAML frontmatter for:
- `stepsCompleted` — array of completed step names
- `lastStep` — last completed step name
- `lastSaved` — timestamp of last save
**If `{outputFile}` does not exist**, display:
"⚠️ **No previous progress found.** There is no output document to resume from. Please use **[C] Create** to start a fresh workflow run."
**THEN:** Halt. Do not proceed.
---
### 2. Display Progress Dashboard
Display progress with ✅/⬜ indicators:
1. ✅/⬜ Preflight & Context (step-01-preflight-and-context)
2. ✅/⬜ Identify Targets (step-02-identify-targets)
3. ✅/⬜ Generate Tests + Aggregate (step-03c-aggregate)
4. ✅/⬜ Validate & Summarize (step-04-validate-and-summarize)
---
### 3. Route to Next Step
Based on `lastStep`, load the next incomplete step:
- `'step-01-preflight-and-context'` → load `./step-02-identify-targets.md`
- `'step-02-identify-targets'` → load `./step-03-generate-tests.md`
- `'step-03c-aggregate'` → load `./step-04-validate-and-summarize.md`
- `'step-04-validate-and-summarize'`**Workflow already complete.** Display: "✅ **All steps completed.** Use **[V] Validate** to review outputs or **[E] Edit** to make revisions." Then halt.
**If `lastStep` does not match any value above**, display: "⚠️ **Unknown progress state** (`lastStep`: {lastStep}). Please use **[C] Create** to start fresh." Then halt.
**Otherwise**, load the identified step file, read completely, and execute.
The existing content in `{outputFile}` provides context from previously completed steps.
---
## 🚨 SYSTEM SUCCESS/FAILURE METRICS
### ✅ SUCCESS:
- Output document loaded and parsed correctly
- Progress dashboard displayed accurately
- Routed to correct next step
### ❌ SYSTEM FAILURE:
- Not loading output document
- Incorrect progress display
- Routing to wrong step
**Master Rule:** Resume MUST route to the exact next incomplete step. Never re-execute completed steps.

View File

@@ -0,0 +1,169 @@
---
name: 'step-02-identify-targets'
description: 'Identify automation targets and create coverage plan'
outputFile: '{test_artifacts}/automation-summary.md'
nextStepFile: './step-03-generate-tests.md'
---
# Step 2: Identify Automation Targets
## STEP GOAL
Determine what needs to be tested and select appropriate test levels and priorities.
## MANDATORY EXECUTION RULES
- 📖 Read the entire step file before acting
- ✅ Speak in `{communication_language}`
- 🚫 Avoid duplicate coverage across test levels
---
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
- 💾 Record outputs before proceeding
- 📖 Load the next step only when instructed
## CONTEXT BOUNDARIES:
- Available context: config, loaded artifacts, and knowledge fragments
- Focus: this step's goal only
- Limits: do not execute future steps
- Dependencies: prior steps' outputs (if any)
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
## 1. Determine Targets
**BMad-Integrated:**
- Map acceptance criteria to test scenarios
- Check for existing ATDD outputs to avoid duplication
- Expand coverage with edge cases and negative paths
**Standalone:**
- If specific target feature/files are provided, focus there
- Otherwise auto-discover features in `{source_dir}`
- Prioritize critical paths, integrations, and untested logic
**If {detected_stack} is `frontend` or `fullstack`:**
**Browser Exploration (if `tea_browser_automation` is `cli` or `auto`):**
> **Fallback:** If CLI is not installed, fall back to MCP (if available) or skip browser exploration and rely on code/doc analysis.
Use CLI to explore the application and identify testable pages/flows:
1. `playwright-cli -s=tea-automate open <target_url>`
2. `playwright-cli -s=tea-automate snapshot` → capture page structure and element refs
3. Analyze snapshot output to identify testable elements and flows
4. `playwright-cli -s=tea-automate close`
> **Session Hygiene:** Always close sessions using `playwright-cli -s=tea-automate close`. Do NOT use `close-all` — it kills every session on the machine and breaks parallel execution.
**If {detected_stack} is `backend` or `fullstack`:**
**Source & API Analysis (no browser exploration):**
- Scan source code for route handlers, controllers, service classes, and public APIs
- Read OpenAPI/Swagger specs (`openapi.yaml`, `swagger.json`) if available
- Identify database models, migrations, and data access patterns
- Map service-to-service integrations and message queue consumers/producers
- Check for existing contract tests (Pact, etc.)
---
**If `use_pactjs_utils` is enabled — Provider Endpoint Mapping (all stacks):**
When consumer-driven contract tests will be generated, build a Provider Endpoint Map during target identification. This applies to all `{detected_stack}` values — frontend, backend, and fullstack consumers all need provider scrutiny.
1. **Locate provider source and/or OpenAPI spec**: Scan workspace for provider project (from config, monorepo structure, or adjacent repositories). Also check for OpenAPI/Swagger spec files (`openapi.yaml`, `openapi.json`, `swagger.json`) — these document the provider's contract explicitly and can supplement or replace handler code analysis.
2. **Map each consumer endpoint** to its provider counterpart:
- Provider file path (route handler)
- Route pattern (METHOD + path)
- Validation schema location (Joi, Zod, class-validator) or OpenAPI request schema
- Response type/DTO definition location or OpenAPI response schema
- OpenAPI spec path (if available, e.g., `server/openapi.yaml`)
3. **Output as "Provider Endpoint Map" table** in the coverage plan:
```markdown
| Consumer Endpoint | Provider File | Route | Validation Schema | Response Type | OpenAPI Spec |
| --------------------- | --------------------------------- | ------------------------- | ----------------------------------- | --------------- | ------------------------------------------------- |
| GET /api/v2/users/:id | server/src/routes/userHandlers.ts | GET /api/v2/users/:userId | server/src/validation/user.ts | UserResponseDto | server/openapi.yaml#/paths/~1api~1v2~1users~1{id} |
| POST /api/v2/users | server/src/routes/userHandlers.ts | POST /api/v2/users | server/src/validation/createUser.ts | UserResponseDto | server/openapi.yaml#/paths/~1api~1v2~1users |
```
4. **If provider source not accessible**: Mark entries with `TODO — provider source not accessible` and note in coverage plan that provider scrutiny will use graceful degradation (see `contract-testing.md` Provider Scrutiny Protocol)
---
## 2. Choose Test Levels
Use `test-levels-framework.md` to select:
- **E2E** for critical user journeys
- **API** for business logic and service contracts
- **Component** for UI behavior
- **Unit** for pure logic and edge cases
---
## 3. Assign Priorities
Use `test-priorities-matrix.md`:
- P0: Critical path + high risk
- P1: Important flows + medium/high risk
- P2: Secondary + edge cases
- P3: Optional/rare scenarios
---
## 4. Coverage Plan
Produce a concise coverage plan:
- Targets by test level
- Priority assignments
- Justification for coverage scope (critical-paths/comprehensive/selective)
---
## 5. Save Progress
**Save this step's accumulated work to `{outputFile}`.**
- **If `{outputFile}` does not exist** (first save), create it with YAML frontmatter:
```yaml
---
stepsCompleted: ['step-02-identify-targets']
lastStep: 'step-02-identify-targets'
lastSaved: '{date}'
---
```
Then write this step's output below the frontmatter.
- **If `{outputFile}` already exists**, update:
- Add `'step-02-identify-targets'` to `stepsCompleted` array (only if not already present)
- Set `lastStep: 'step-02-identify-targets'`
- Set `lastSaved: '{date}'`
- Append this step's output to the appropriate section.
Load next step: `{nextStepFile}`
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Step completed in full with required outputs
### ❌ SYSTEM FAILURE:
- Skipped sequence steps or missing outputs
**Master Rule:** Skipping steps is FORBIDDEN.

View File

@@ -0,0 +1,394 @@
---
name: 'step-03-generate-tests'
description: 'Orchestrate adaptive test generation (agent-team, subagent, or sequential)'
nextStepFile: './step-03c-aggregate.md'
---
# Step 3: Orchestrate Adaptive Test Generation
## STEP GOAL
Select execution mode deterministically, then generate tests using agent-team, subagent, or sequential execution while preserving the same output contract. Worker selection depends on `{detected_stack}`.
## MANDATORY EXECUTION RULES
- 📖 Read the entire step file before acting
- ✅ Speak in `{communication_language}`
- ✅ Resolve execution mode from config (`tea_execution_mode`, `tea_capability_probe`)
- ✅ Apply fallback rules deterministically when requested mode is unsupported
- ✅ Preserve output schema and temp file naming across all modes
- ❌ Do NOT skip capability checks when probing is enabled
- ❌ Do NOT change output paths or JSON schema by mode
---
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
- 💾 Wait for subagent outputs
- 📖 Load the next step only when instructed
## CONTEXT BOUNDARIES:
- Available context: config, coverage plan from Step 2, knowledge fragments
- Focus: orchestration only (mode selection + worker dispatch)
- Limits: do not generate tests directly (delegate to worker steps)
- Dependencies: Step 2 outputs (coverage plan, target features)
---
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
### 1. Prepare Execution Context
**Generate unique timestamp** for temp file naming:
```javascript
const timestamp = new Date().toISOString().replace(/[:.]/g, '-');
```
**Prepare input context for subagents:**
```javascript
const parseBooleanFlag = (value, defaultValue = true) => {
if (typeof value === 'string') {
const normalized = value.trim().toLowerCase();
if (['false', '0', 'off', 'no'].includes(normalized)) return false;
if (['true', '1', 'on', 'yes'].includes(normalized)) return true;
}
if (value === undefined || value === null) return defaultValue;
return Boolean(value);
};
const subagentContext = {
features: /* from Step 2 coverage plan */,
knowledge_fragments_loaded: /* list of fragments */,
config: {
test_framework: config.test_framework,
use_playwright_utils: config.tea_use_playwright_utils,
use_pactjs_utils: config.tea_use_pactjs_utils,
pact_mcp: config.tea_pact_mcp, // "mcp" | "none"
browser_automation: config.tea_browser_automation, // "auto" | "cli" | "mcp" | "none"
detected_stack: '{detected_stack}', // "frontend" | "backend" | "fullstack"
execution_mode: config.tea_execution_mode || 'auto', // "auto" | "subagent" | "agent-team" | "sequential"
capability_probe: parseBooleanFlag(config.tea_capability_probe, true), // supports booleans and "false"/"true" strings
provider_endpoint_map: /* from Step 2 coverage plan, if use_pactjs_utils enabled */,
},
timestamp: timestamp
};
```
---
### 2. Resolve Execution Mode with Capability Probe
```javascript
const normalizeUserExecutionMode = (mode) => {
if (typeof mode !== 'string') return null;
const normalized = mode.trim().toLowerCase().replace(/[-_]/g, ' ').replace(/\s+/g, ' ');
if (normalized === 'auto') return 'auto';
if (normalized === 'sequential') return 'sequential';
if (normalized === 'subagent' || normalized === 'sub agent' || normalized === 'subagents' || normalized === 'sub agents') {
return 'subagent';
}
if (normalized === 'agent team' || normalized === 'agent teams' || normalized === 'agentteam') {
return 'agent-team';
}
return null;
};
const normalizeConfigExecutionMode = (mode) => {
if (mode === 'subagent') return 'subagent';
if (mode === 'auto' || mode === 'sequential' || mode === 'subagent' || mode === 'agent-team') {
return mode;
}
return null;
};
// Explicit user instruction in the active run takes priority over config.
const explicitModeFromUser = normalizeUserExecutionMode(runtime.getExplicitExecutionModeHint?.() || null);
const requestedMode = explicitModeFromUser || normalizeConfigExecutionMode(subagentContext.config.execution_mode) || 'auto';
const probeEnabled = subagentContext.config.capability_probe;
const supports = {
subagent: false,
agentTeam: false,
};
if (probeEnabled) {
// Probe using runtime-native capability checks or a no-op launch test.
supports.subagent = runtime.canLaunchSubagents?.() === true;
supports.agentTeam = runtime.canLaunchAgentTeams?.() === true;
}
let resolvedMode = requestedMode;
if (requestedMode === 'auto') {
if (supports.agentTeam) resolvedMode = 'agent-team';
else if (supports.subagent) resolvedMode = 'subagent';
else resolvedMode = 'sequential';
} else if (probeEnabled && requestedMode === 'agent-team' && !supports.agentTeam) {
resolvedMode = supports.subagent ? 'subagent' : 'sequential';
} else if (probeEnabled && requestedMode === 'subagent' && !supports.subagent) {
resolvedMode = 'sequential';
}
subagentContext.execution = {
requestedMode,
resolvedMode,
probeEnabled,
supports,
};
```
Resolution precedence:
1. Explicit user request in this run (`agent team` => `agent-team`; `subagent` => `subagent`; `sequential`; `auto`)
2. `tea_execution_mode` from config
3. Runtime capability fallback (when probing enabled)
If probing is disabled, honor the requested mode strictly. If that mode cannot be executed at runtime, fail with explicit error instead of silent fallback.
Report selected mode before dispatch:
```
⚙️ Execution Mode Resolution:
- Requested: {requestedMode}
- Probe Enabled: {probeEnabled}
- Supports agent-team: {supports.agentTeam}
- Supports subagent: {supports.subagent}
- Resolved: {resolvedMode}
```
---
### 3. Subagent Dispatch Matrix
**Select subagents based on `{detected_stack}`:**
| `{detected_stack}` | Subagent A (API) | Subagent B (E2E) | Subagent B-backend |
| ------------------ | ---------------- | ---------------- | ------------------ |
| `frontend` | Launch | Launch | Skip |
| `backend` | Launch | Skip | Launch |
| `fullstack` | Launch | Launch | Launch |
### 3A. Runtime-Managed Parallelism
When `resolvedMode` is `agent-team` or `subagent`, let the runtime decide concurrency and scheduling. TEA does not impose an additional worker ceiling.
---
### Contract Test Generation Note
When `use_pactjs_utils` is enabled, the API test generation subagent (step-03a) also generates:
- **Consumer contract tests**: Using `createProviderState` for type-safe provider states
- **Provider verification tests**: Using `buildVerifierOptions` for one-call verifier setup
- **Message contract tests**: Using `buildMessageVerifierOptions` if async/Kafka patterns detected
- **Helper files**: Request filter setup with `createRequestFilter`, shared state constants
- **Provider scrutiny**: Subagent reads provider route handlers, types, and validation schemas before generating each interaction (see `contract-testing.md` Provider Scrutiny Protocol)
When `pact_mcp` is `"mcp"`, the subagent can use SmartBear MCP tools to fetch existing provider states and generate tests informed by broker data.
---
### 4. Dispatch Worker A: API Test Generation (always)
**Dispatch worker:**
- **Subagent File:** `./step-03a-subagent-api.md`
- **Output File:** `/tmp/tea-automate-api-tests-${timestamp}.json`
- **Context:** Pass `subagentContext`
- **Execution:**
- `agent-team` or `subagent`: launch non-blocking
- `sequential`: run blocking and wait before next dispatch
**System Action:**
```
🚀 Launching Subagent A: API Test Generation
📝 Output: /tmp/tea-automate-api-tests-${timestamp}.json
⚙️ Mode: ${resolvedMode}
⏳ Status: Running...
```
---
### 5. Dispatch Worker B: E2E Test Generation (frontend/fullstack only)
**If {detected_stack} is `frontend` or `fullstack`:**
**Dispatch worker:**
- **Subagent File:** `./step-03b-subagent-e2e.md`
- **Output File:** `/tmp/tea-automate-e2e-tests-${timestamp}.json`
- **Context:** Pass `subagentContext`
- **Execution:**
- `agent-team` or `subagent`: launch non-blocking
- `sequential`: run blocking and wait before next dispatch
**System Action:**
```
🚀 Launching Subagent B: E2E Test Generation
📝 Output: /tmp/tea-automate-e2e-tests-${timestamp}.json
⚙️ Mode: ${resolvedMode}
⏳ Status: Running...
```
**If {detected_stack} is `backend`:** Skip this subagent.
---
### 6. Dispatch Worker B-backend: Backend Test Generation (backend/fullstack only)
**If {detected_stack} is `backend` or `fullstack`:**
**Dispatch worker:**
- **Subagent File:** `./step-03b-subagent-backend.md`
- **Output File:** `/tmp/tea-automate-backend-tests-${timestamp}.json`
- **Context:** Pass `subagentContext`
- **Execution:**
- `agent-team` or `subagent`: launch non-blocking
- `sequential`: run blocking and wait before next dispatch
**System Action:**
```
🚀 Launching Subagent B-backend: Backend Test Generation
📝 Output: /tmp/tea-automate-backend-tests-${timestamp}.json
⚙️ Mode: ${resolvedMode}
⏳ Status: Running...
```
**If {detected_stack} is `frontend`:** Skip this subagent.
---
### 7. Wait for Expected Worker Completion
**If `resolvedMode` is `agent-team` or `subagent`:**
```
⏳ Waiting for subagents to complete...
├── Subagent A (API): Running... ⟳
├── Subagent B (E2E): Running... ⟳ [if frontend/fullstack]
└── Subagent B-backend: Running... ⟳ [if backend/fullstack]
[... time passes ...]
├── Subagent A (API): Complete ✅
├── Subagent B (E2E): Complete ✅ [if frontend/fullstack]
└── Subagent B-backend: Complete ✅ [if backend/fullstack]
✅ All subagents completed successfully!
```
**If `resolvedMode` is `sequential`:**
```
✅ Sequential mode: each worker already completed during dispatch.
```
**Verify outputs exist (based on `{detected_stack}`):**
```javascript
const apiOutputExists = fs.existsSync(`/tmp/tea-automate-api-tests-${timestamp}.json`);
// Check based on detected_stack
if (detected_stack === 'frontend' || detected_stack === 'fullstack') {
const e2eOutputExists = fs.existsSync(`/tmp/tea-automate-e2e-tests-${timestamp}.json`);
if (!e2eOutputExists) throw new Error('E2E subagent output missing!');
}
if (detected_stack === 'backend' || detected_stack === 'fullstack') {
const backendOutputExists = fs.existsSync(`/tmp/tea-automate-backend-tests-${timestamp}.json`);
if (!backendOutputExists) throw new Error('Backend subagent output missing!');
}
if (!apiOutputExists) throw new Error('API subagent output missing!');
```
---
### Subagent Output Schema Contract
The aggregate step expects both outputs to include `success`, but the payload shapes are intentionally different:
- `step-03b-subagent-e2e.md` output includes `success`, `subagent`, `tests`, `fixture_needs`, `knowledge_fragments_used`, `test_count`, and `summary`.
- `step-03b-subagent-backend.md` output includes `success`, `subagent`, `subagentType`, `testsGenerated`, `coverageSummary` (with `fixtureNeeds`), `status`, `knowledge_fragments_used`, and `summary`.
The aggregate step reads whichever output file(s) exist based on `{detected_stack}` and must use the matching schema per subagent type.
---
### 8. Execution Report
**Display performance metrics:**
```
🚀 Performance Report:
- Execution Mode: {resolvedMode}
- Stack Type: {detected_stack}
- API Test Generation: ~X minutes
- E2E Test Generation: ~Y minutes [if frontend/fullstack]
- Backend Test Generation: ~Z minutes [if backend/fullstack]
- Total Elapsed: ~mode-dependent
- Parallel Gain: ~40-70% faster when mode is subagent/agent-team
```
---
### 9. Proceed to Aggregation
**Load aggregation step:**
Load next step: `{nextStepFile}`
The aggregation step (3C) will:
- Read all subagent outputs (based on `{detected_stack}`)
- Write all test files to disk
- Generate shared fixtures and helpers
- Calculate summary statistics
---
## EXIT CONDITION
Proceed to Step 3C (Aggregation) when:
- ✅ Subagent A (API tests) completed successfully
- ✅ Subagent B (E2E tests) completed successfully [if frontend/fullstack]
- ✅ Subagent B-backend (Backend tests) completed successfully [if backend/fullstack]
- ✅ All expected output files exist and are valid JSON
- ✅ Execution metrics displayed
**Do NOT proceed if:**
- ❌ Any launched subagent failed
- ❌ Output files missing or corrupted
- ❌ Timeout occurred (parallel mode only)
---
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- All required subagents launched successfully (based on `{detected_stack}`)
- All required worker steps completed without errors
- Output files generated and valid
- Fallback behavior respected configuration and capability probe rules
### ❌ SYSTEM FAILURE:
- Failed to launch subagents
- One or more subagents failed
- Output files missing or invalid
- Unsupported requested mode with probing disabled
**Master Rule:** Deterministic mode selection + stable output contract. Use the best supported mode, then aggregate normally.

View File

@@ -0,0 +1,263 @@
---
name: 'step-03a-subagent-api'
description: 'Subagent: Generate API tests only'
subagent: true
outputFile: '/tmp/tea-automate-api-tests-{{timestamp}}.json'
---
# Subagent 3A: Generate API Tests
## SUBAGENT CONTEXT
This is an **isolated subagent** running in parallel with E2E test generation.
**What you have from parent workflow:**
- Target features/components identified in Step 2
- Knowledge fragments loaded: api-request, data-factories, api-testing-patterns
- Config: test framework, Playwright Utils enabled/disabled, Pact.js Utils enabled/disabled, Pact MCP mode
- Coverage plan: which API endpoints need testing
**Your task:** Generate API tests ONLY (not E2E, not fixtures, not other test types).
**If `use_pactjs_utils` is enabled:** Also generate consumer contract tests and provider verification tests alongside API tests. Use the loaded pactjs-utils fragments (`pactjs-utils-overview`, `pactjs-utils-consumer-helpers`, `pactjs-utils-provider-verifier`, `pactjs-utils-request-filter`) for patterns. If `pact_mcp` is `"mcp"`, use SmartBear MCP tools (Fetch Provider States, Generate Pact Tests) to inform test generation.
---
## MANDATORY EXECUTION RULES
- 📖 Read this entire subagent file before acting
- ✅ Generate API tests ONLY
- ✅ Output structured JSON to temp file
- ✅ Follow knowledge fragment patterns
- ❌ Do NOT generate E2E tests (that's subagent 3B)
- ❌ Do NOT run tests (that's step 4)
- ❌ Do NOT generate fixtures yet (that's step 3C aggregation)
---
## SUBAGENT TASK
### 1. Identify API Endpoints
From the coverage plan (Step 2 output), identify:
- Which API endpoints need test coverage
- Expected request/response formats
- Authentication requirements
- Error scenarios to test
### 2. Generate API Test Files
For each API endpoint, create test file in `tests/api/[feature].spec.ts`:
**Test Structure:**
```typescript
import { test, expect } from '@playwright/test';
// If Playwright Utils enabled:
// import { apiRequest } from '@playwright-utils/api';
test.describe('[Feature] API Tests', () => {
test('[P0] should handle successful [operation]', async ({ request }) => {
// Use apiRequest helper if Playwright Utils enabled
// Otherwise use standard request fixture
const response = await request.post('/api/endpoint', {
data: {
/* test data */
},
});
expect(response.status()).toBe(200);
expect(await response.json()).toMatchObject({
/* expected */
});
});
test('[P1] should handle [error scenario]', async ({ request }) => {
// Test error handling
});
});
```
**Requirements:**
- ✅ Use `apiRequest()` helper if Playwright Utils enabled (from api-request fragment)
- ✅ Use data factories for test data (from data-factories fragment)
- ✅ Follow API testing patterns (from api-testing-patterns fragment)
- ✅ Include priority tags [P0], [P1], [P2], [P3]
- ✅ Test both happy path and error scenarios
- ✅ Use proper TypeScript types
- ✅ Deterministic assertions (no timing dependencies)
**If Pact.js Utils enabled (from `subagentContext.config.use_pactjs_utils`):**
- ✅ Generate consumer contract tests in `pact/http/consumer/` using `createProviderState({ name, params })` pattern
- ✅ Generate provider verification tests in `pact/http/provider/` using `buildVerifierOptions({ provider, port, includeMainAndDeployed, stateHandlers })` pattern
- ✅ Generate request filter helpers in `pact/http/helpers/` using `createRequestFilter({ tokenGenerator: () => string })`
- ✅ Generate shared state constants in `pact/http/helpers/states.ts`
- ✅ If async/message patterns detected, generate message consumer tests in `pact/message/` using `buildMessageVerifierOptions`
-**Provider endpoint comment MANDATORY** on every Pact interaction: `// Provider endpoint: <path> -> <METHOD> <route>`
- ⚠️ **Postel's Law for matchers**: Use `like()`, `eachLike()`, `string()`, `integer()` matchers ONLY in `willRespondWith` (responses). Request bodies in `withRequest` MUST use exact values — never wrap request bodies in `like()`. The consumer controls what it sends, so contracts should be strict about request shape.
### 1.5 Provider Source Scrutiny (CDC Only)
**CRITICAL**: Before generating ANY Pact consumer interaction, perform provider source scrutiny per the **Seven-Point Scrutiny Checklist** defined in `contract-testing.md`. Do NOT generate response matchers from consumer-side types alone — this is the #1 cause of contract verification failures.
The seven points to verify for each interaction:
1. Response shape
2. Status codes
3. Field names
4. Enum values
5. Required fields
6. Data types
7. Nested structures
**Source priority**: Provider source code is most authoritative. When an OpenAPI/Swagger spec exists (`openapi.yaml`, `openapi.json`, `swagger.json`), use it as a complementary or alternative source — it documents the provider's contract explicitly and can be faster to parse than tracing through handler code. When both exist, cross-reference them; if they disagree, the source code wins. Document the discrepancy in the scrutiny evidence block (e.g., `OpenAPI shows 200 but handler returns 201; using handler behavior`) and flag it in the output JSON `summary` so it is discoverable by downstream consumers or audits.
**Scrutiny Sequence** (for each endpoint in the coverage plan):
1. **READ provider route handler and/or OpenAPI spec**: Find the handler file from `subagentContext.config.provider_endpoint_map` or by scanning the provider codebase. Also check for OpenAPI/Swagger spec files. Extract:
- Exact status codes returned (`res.status(201)` / OpenAPI `responses` keys)
- Response construction (`res.json({ data: ... })` / OpenAPI `schema`)
- Error handling paths (what status codes for what conditions)
2. **READ provider type/model/DTO definitions**: Find the response type referenced by the handler or OpenAPI `$ref` schemas. Extract:
- Exact field names (`transaction_id` not `transactionId`)
- Field types (`string` ID vs `number` ID / OpenAPI `type` + `format`)
- Optional vs required fields (OpenAPI `required` array)
- Nested object structures (OpenAPI `$ref`, `allOf`, `oneOf`)
3. **READ provider validation schemas**: Find Joi/Zod/class-validator schemas or OpenAPI request body `schema.required`. Extract:
- Required request fields and headers
- Enum/union type allowed values (`"active" | "inactive"` / OpenAPI `enum`)
- Request body constraints
4. **Cross-reference findings** against consumer expectations:
- Does the consumer expect the same field names the provider sends?
- Does the consumer expect the same status codes the provider returns?
- Does the consumer expect the same nesting the provider produces?
5. **Document scrutiny evidence** as a block comment in the generated test:
```typescript
/*
* Provider Scrutiny Evidence:
* - Handler: server/src/routes/userHandlers.ts:45
* - OpenAPI: server/openapi.yaml paths./api/v2/users/{userId}.get (if available)
* - Response type: UserResponseDto (server/src/types/user.ts:12)
* - Status: 201 for creation (line 52), 400 for validation error (line 48)
* - Fields: { id: number, name: string, email: string, role: "user" | "admin" }
* - Required request headers: Authorization (Bearer token)
*/
```
6. **Graceful degradation** when provider source is not accessible (follows the canonical four-step protocol from `contract-testing.md`):
1. **OpenAPI/Swagger spec available**: Use the spec as the source of truth for response shapes, status codes, and field names
2. **Pact Broker available** (when `pact_mcp` is `"mcp"` in `subagentContext.config`): Use SmartBear MCP tools to fetch existing provider states and verified interactions as reference
3. **Neither available**: Generate from consumer types but use the TODO form of the mandatory comment: `// Provider endpoint: TODO — provider source not accessible, verify manually`. Set `provider_scrutiny: "pending"` in output JSON
4. **Never silently guess**: Document all assumptions in the scrutiny evidence block
> ⚠️ **Anti-pattern**: Generating response matchers from consumer-side types alone. This produces contracts that reflect what the consumer _wishes_ the provider returns, not what it _actually_ returns. Always read provider source or OpenAPI spec first.
### 3. Track Fixture Needs
Identify fixtures needed for API tests:
- Authentication fixtures (auth tokens, API keys)
- Data factories (user data, product data, etc.)
- API client configurations
**Do NOT create fixtures yet** - just track what's needed for aggregation step.
---
## OUTPUT FORMAT
Write JSON to temp file: `/tmp/tea-automate-api-tests-{{timestamp}}.json`
```json
{
"success": true,
"subagent": "api-tests",
"tests": [
{
"file": "tests/api/auth.spec.ts",
"content": "[full TypeScript test file content]",
"description": "API tests for authentication endpoints",
"priority_coverage": {
"P0": 3,
"P1": 2,
"P2": 1,
"P3": 0
}
},
{
"file": "tests/api/checkout.spec.ts",
"content": "[full TypeScript test file content]",
"description": "API tests for checkout endpoints",
"priority_coverage": {
"P0": 2,
"P1": 3,
"P2": 1,
"P3": 0
}
}
],
"fixture_needs": ["authToken", "userDataFactory", "productDataFactory"],
"knowledge_fragments_used": ["api-request", "data-factories", "api-testing-patterns"],
"provider_scrutiny": "completed",
"provider_files_read": ["server/src/routes/authHandlers.ts", "server/src/routes/checkoutHandlers.ts", "server/src/types/auth.ts"],
"test_count": 12,
"summary": "Generated 12 API test cases covering 3 features"
}
```
**On Error:**
```json
{
"success": false,
"subagent": "api-tests",
"error": "Error message describing what went wrong",
"partial_output": {
/* any tests generated before error */
}
}
```
---
## EXIT CONDITION
Subagent completes when:
- ✅ All API endpoints have test files generated
- ✅ All tests follow knowledge fragment patterns
- ✅ JSON output written to temp file
- ✅ Fixture needs tracked
**Subagent terminates here.** Parent workflow will read output and proceed to aggregation.
---
## 🚨 SUBAGENT SUCCESS METRICS
### ✅ SUCCESS:
- All API tests generated following patterns
- JSON output valid and complete
- No E2E/component/unit tests included (out of scope)
- Every Pact interaction has `// Provider endpoint:` comment (if CDC enabled)
- Provider source scrutiny completed or gracefully degraded with TODO markers (if CDC enabled)
- Scrutiny evidence documented as block comments in test files (if CDC enabled)
### ❌ FAILURE:
- Generated tests other than API tests
- Did not follow knowledge fragment patterns
- Invalid or missing JSON output
- Ran tests (not subagent responsibility)
- Pact interactions missing provider endpoint comments (if CDC enabled)
- Response matchers generated from consumer-side types without provider scrutiny (if CDC enabled)

View File

@@ -0,0 +1,246 @@
---
name: 'step-03b-subagent-backend'
description: 'Subagent: Generate backend tests only (unit, integration, contract)'
subagent: true
outputFile: '/tmp/tea-automate-backend-tests-{{timestamp}}.json'
---
# Subagent 3B-backend: Generate Backend Tests
## SUBAGENT CONTEXT
This is an **isolated subagent** running in parallel with API test generation (and optionally E2E test generation for fullstack projects).
**What you have from parent workflow:**
- Target features/services identified in Step 2
- Knowledge fragments loaded: test-levels-framework, test-priorities-matrix, data-factories
- Config: test framework, detected stack type
- Coverage plan: which services/modules need backend testing
**Your task:** Generate backend tests ONLY (unit, integration, contract - not API endpoint tests, not E2E).
---
## MANDATORY EXECUTION RULES
- Read this entire subagent file before acting
- Generate backend tests ONLY (unit, integration, contract)
- Output structured JSON to temp file using the subagent output schema contract
- Follow knowledge fragment patterns
- Do NOT generate API endpoint tests (that's subagent 3A)
- Do NOT generate E2E tests (that's subagent 3B-E2E)
- Do NOT run tests (that's step 4)
- Do NOT generate fixtures yet (that's step 3C aggregation)
---
## SUBAGENT TASK
### 1. Identify Test Targets
From the coverage plan (Step 2 output), identify:
- Which services/modules need unit test coverage
- Which integrations need integration test coverage (database, message queues, external services)
- Which service contracts need contract test coverage (Pact, schema validation)
- Business logic functions requiring edge case coverage
### 2. Detect Framework & Language
From `config.test_framework` and project manifests, determine:
- **Python (pytest)**: Use `pytest` conventions, `conftest.py` fixtures, `@pytest.mark` decorators
- **Java/Kotlin (JUnit)**: Use JUnit 5 annotations (`@Test`, `@BeforeEach`, `@Nested`), Mockito for mocking
- **Go (go test)**: Use `*_test.go` files, `testing.T`, table-driven tests, `testify` assertions
- **C#/.NET (xUnit)**: Use `[Fact]`, `[Theory]`, `[InlineData]`, `Moq` for mocking
- **Ruby (RSpec)**: Use `describe`/`context`/`it` blocks, `let`/`before` helpers, `FactoryBot`
### 3. Generate Unit Tests
For each module/service, create test files following language-idiomatic patterns:
**Python (pytest) example:**
```python
import pytest
from unittest.mock import MagicMock, patch
from myapp.services.user_service import UserService
class TestUserService:
"""[P0] Unit tests for UserService"""
def test_create_user_with_valid_data(self, user_factory):
"""Should create user when data is valid"""
user_data = user_factory.build()
result = UserService.create(user_data)
assert result.email == user_data["email"]
def test_create_user_rejects_duplicate_email(self, user_factory):
"""[P1] Should reject duplicate email"""
user_data = user_factory.build(email="existing@test.com")
with pytest.raises(DuplicateEmailError):
UserService.create(user_data)
```
**Go (go test) example:**
```go
func TestUserService_Create(t *testing.T) {
tests := []struct {
name string
input CreateUserInput
wantErr bool
}{
{"valid user", validInput(), false},
{"duplicate email", duplicateInput(), true},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
svc := NewUserService(mockRepo)
_, err := svc.Create(tt.input)
if (err != nil) != tt.wantErr {
t.Errorf("Create() error = %v, wantErr %v", err, tt.wantErr)
}
})
}
}
```
**Requirements:**
- Follow the detected framework's idiomatic test patterns
- Include priority tags [P0], [P1], [P2], [P3] in test descriptions
- Use proper mocking for external dependencies (database, APIs, message queues)
- Test both happy path and error cases
- Use proper typing/type hints where applicable
- No hard-coded test data; use factories or builders
### 4. Generate Integration Tests
For service integrations, create integration test files:
- Database integration tests (with test database or in-memory alternatives)
- Message queue consumer/producer tests
- Cache integration tests
- External service integration tests (with mocked HTTP clients)
### 5. Generate Contract Tests (if applicable)
If the project uses microservices or has defined API contracts:
- Pact consumer/provider tests
- Schema validation tests (JSON Schema, Protobuf)
- OpenAPI spec compliance tests
### 6. Track Fixture Needs
Identify fixtures/helpers needed for backend tests:
- Database fixtures (seed data, cleanup)
- Factory functions (test data builders)
- Mock services (HTTP mocks, message queue mocks)
- Configuration fixtures (test environment config)
**Do NOT create fixtures yet** - just track what's needed for aggregation step.
---
## OUTPUT FORMAT
Write JSON to temp file: `/tmp/tea-automate-backend-tests-{{timestamp}}.json`
```json
{
"subagentType": "backend",
"testsGenerated": [
{
"file": "tests/unit/test_user_service.py",
"content": "[full test file content]",
"description": "Unit tests for UserService",
"priority_coverage": {
"P0": 3,
"P1": 2,
"P2": 1,
"P3": 0
}
},
{
"file": "tests/integration/test_user_repository.py",
"content": "[full test file content]",
"description": "Integration tests for user database operations",
"priority_coverage": {
"P0": 1,
"P1": 2,
"P2": 1,
"P3": 0
}
}
],
"coverageSummary": {
"totalTests": 15,
"testLevels": ["unit", "integration", "contract"],
"fixtureNeeds": ["databaseFixture", "userFactory", "mockHttpClient"]
},
"status": "complete",
"success": true,
"subagent": "backend-tests",
"knowledge_fragments_used": ["test-levels-framework", "test-priorities-matrix", "data-factories"],
"summary": "Generated 15 backend test cases (10 unit, 4 integration, 1 contract)"
}
```
**On Error:**
```json
{
"subagentType": "backend",
"testsGenerated": [],
"coverageSummary": {
"totalTests": 0,
"testLevels": [],
"fixtureNeeds": []
},
"status": "partial",
"success": false,
"subagent": "backend-tests",
"error": "Error message describing what went wrong",
"partial_output": {
/* any tests generated before error */
}
}
```
---
## EXIT CONDITION
Subagent completes when:
- All identified modules have backend test files generated
- All tests follow language-idiomatic patterns
- JSON output written to temp file using the subagent output schema contract
- Fixture needs tracked
**Subagent terminates here.** Parent workflow will read output and proceed to aggregation.
---
## SUBAGENT SUCCESS METRICS
### SUCCESS:
- All backend tests generated following idiomatic patterns
- JSON output valid and complete, matches subagent output schema contract
- No E2E or browser tests included (out of scope)
- Proper mocking used for external dependencies
- Priority tags assigned to all test cases
### FAILURE:
- Generated tests other than backend tests (unit/integration/contract)
- Did not follow language-idiomatic patterns
- Invalid or missing JSON output
- Output schema does not match the contract
- Ran tests (not subagent responsibility)
- Used real external services instead of mocks

View File

@@ -0,0 +1,213 @@
---
name: 'step-03b-subagent-e2e'
description: 'Subagent: Generate E2E tests only'
subagent: true
outputFile: '/tmp/tea-automate-e2e-tests-{{timestamp}}.json'
---
# Subagent 3B: Generate E2E Tests
## SUBAGENT CONTEXT
This is an **isolated subagent** running in parallel with API test generation.
**What you have from parent workflow:**
- Target features/user journeys identified in Step 2
- Knowledge fragments loaded: fixture-architecture, network-first, selector-resilience
- Config: test framework, Playwright Utils enabled/disabled
- Coverage plan: which user journeys need E2E testing
**Your task:** Generate E2E tests ONLY (not API, not fixtures, not other test types).
---
## MANDATORY EXECUTION RULES
- 📖 Read this entire subagent file before acting
- ✅ Generate E2E tests ONLY
- ✅ Output structured JSON to temp file
- ✅ Follow knowledge fragment patterns
- ❌ Do NOT generate API tests (that's subagent 3A)
- ❌ Do NOT run tests (that's step 4)
- ❌ Do NOT generate fixtures yet (that's step 3C aggregation)
---
## SUBAGENT TASK
### 1. Identify User Journeys
From the coverage plan (Step 2 output), identify:
- Which user journeys need E2E coverage
- Critical user paths (authentication, checkout, profile, etc.)
- UI interactions required
- Expected visual states
### 2. Browser Interaction (Selector Verification)
**Automation mode:** `config.tea_browser_automation`
If `auto` (fall back to MCP if CLI unavailable; if neither available, generate from best practices):
- Open the target page first, then verify selectors with a snapshot:
`playwright-cli -s=tea-automate-{{timestamp}} open <target_url>`
`playwright-cli -s=tea-automate-{{timestamp}} snapshot` → map refs to Playwright locators
- ref `{role: "button", name: "Submit"}``page.getByRole('button', { name: 'Submit' })`
- ref `{role: "textbox", name: "Email"}``page.getByRole('textbox', { name: 'Email' })`
- `playwright-cli -s=tea-automate-{{timestamp}} close` when done
If `cli` (CLI only — do NOT fall back to MCP; generate from best practices if CLI unavailable):
- Open the target page first, then verify selectors with a snapshot:
`playwright-cli -s=tea-automate-{{timestamp}} open <target_url>`
`playwright-cli -s=tea-automate-{{timestamp}} snapshot` → map refs to Playwright locators
- ref `{role: "button", name: "Submit"}``page.getByRole('button', { name: 'Submit' })`
- ref `{role: "textbox", name: "Email"}``page.getByRole('textbox', { name: 'Email' })`
- `playwright-cli -s=tea-automate-{{timestamp}} close` when done
> **Session Hygiene:** Always close sessions using `playwright-cli -s=tea-automate-{{timestamp}} close`. Do NOT use `close-all` — it kills every session on the machine and breaks parallel execution.
If `mcp`:
- Use MCP tools for selector verification (current behavior)
If `none`:
- Generate selectors from best practices without browser verification
### 3. Generate E2E Test Files
For each user journey, create test file in `tests/e2e/[feature].spec.ts`:
**Test Structure:**
```typescript
import { test, expect } from '@playwright/test';
test.describe('[Feature] E2E User Journey', () => {
test('[P0] should complete [user journey]', async ({ page }) => {
// Navigate to starting point
await page.goto('/feature');
// Interact with UI
await page.getByRole('button', { name: 'Submit' }).click();
// Assert expected state
await expect(page.getByText('Success')).toBeVisible();
});
test('[P1] should handle [edge case]', async ({ page }) => {
// Test edge case scenario
});
});
```
**Requirements:**
- ✅ Follow fixture architecture patterns (from fixture-architecture fragment)
- ✅ Use network-first patterns: intercept before navigate (from network-first fragment)
- ✅ Use resilient selectors: getByRole, getByText, getByLabel (from selector-resilience fragment)
- ✅ Include priority tags [P0], [P1], [P2], [P3]
- ✅ Test complete user journeys (not isolated clicks)
- ✅ Use proper TypeScript types
- ✅ Deterministic waits (no hard sleeps, use expect().toBeVisible())
### 4. Track Fixture Needs
Identify fixtures needed for E2E tests:
- Page object models (if complex)
- Authentication fixtures (logged-in user state)
- Network mocks/intercepts
- Test data fixtures
**Do NOT create fixtures yet** - just track what's needed for aggregation step.
---
## OUTPUT FORMAT
Write JSON to temp file: `/tmp/tea-automate-e2e-tests-{{timestamp}}.json`
```json
{
"success": true,
"subagent": "e2e-tests",
"tests": [
{
"file": "tests/e2e/authentication.spec.ts",
"content": "[full TypeScript test file content]",
"description": "E2E tests for user authentication journey",
"priority_coverage": {
"P0": 2,
"P1": 3,
"P2": 2,
"P3": 0
}
},
{
"file": "tests/e2e/checkout.spec.ts",
"content": "[full TypeScript test file content]",
"description": "E2E tests for checkout journey",
"priority_coverage": {
"P0": 3,
"P1": 2,
"P2": 1,
"P3": 0
}
}
],
"fixture_needs": ["authenticatedUserFixture", "paymentMockFixture", "checkoutDataFixture"],
"knowledge_fragments_used": ["fixture-architecture", "network-first", "selector-resilience"],
"test_count": 15,
"summary": "Generated 15 E2E test cases covering 5 user journeys"
}
```
**On Error:**
```json
{
"success": false,
"subagent": "e2e-tests",
"error": "Error message describing what went wrong",
"partial_output": {
/* any tests generated before error */
}
}
```
---
## EXIT CONDITION
Subagent completes when:
- ✅ All user journeys have E2E test files generated
- ✅ All tests follow knowledge fragment patterns
- ✅ JSON output written to temp file
- ✅ Fixture needs tracked
**Subagent terminates here.** Parent workflow will read output and proceed to aggregation.
---
## 🚨 SUBAGENT SUCCESS METRICS
### ✅ SUCCESS:
- All E2E tests generated following patterns
- JSON output valid and complete
- No API/component/unit tests included (out of scope)
- Resilient selectors used (getByRole, getByText)
- Network-first patterns applied (intercept before navigate)
### ❌ FAILURE:
- Generated tests other than E2E tests
- Did not follow knowledge fragment patterns
- Invalid or missing JSON output
- Ran tests (not subagent responsibility)
- Used brittle selectors (CSS classes, XPath)

View File

@@ -0,0 +1,393 @@
---
name: 'step-03c-aggregate'
description: 'Aggregate subagent outputs and complete test infrastructure'
outputFile: '{test_artifacts}/automation-summary.md'
nextStepFile: './step-04-validate-and-summarize.md'
---
# Step 3C: Aggregate Test Generation Results
## STEP GOAL
Read outputs from parallel subagents (API + E2E and/or Backend test generation based on `{detected_stack}`), aggregate results, and create supporting infrastructure (fixtures, helpers).
---
## MANDATORY EXECUTION RULES
- 📖 Read the entire step file before acting
- ✅ Speak in `{communication_language}`
- ✅ Read subagent outputs from temp files
- ✅ Generate shared fixtures based on fixture needs from both subagents
- ✅ Write all generated test files to disk
- ❌ Do NOT regenerate tests (use subagent outputs)
- ❌ Do NOT run tests yet (that's step 4)
---
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
- 💾 Record outputs before proceeding
- 📖 Load the next step only when instructed
## CONTEXT BOUNDARIES:
- Available context: config, subagent outputs from temp files
- Focus: aggregation and fixture generation only
- Limits: do not execute future steps
- Dependencies: Step 3A and 3B subagent outputs
---
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
### 1. Read Subagent Outputs
**Read API test subagent output (always):**
```javascript
const apiTestsPath = '/tmp/tea-automate-api-tests-{{timestamp}}.json';
const apiTestsOutput = JSON.parse(fs.readFileSync(apiTestsPath, 'utf8'));
```
**Read E2E test subagent output (if {detected_stack} is `frontend` or `fullstack`):**
```javascript
let e2eTestsOutput = null;
if (detected_stack === 'frontend' || detected_stack === 'fullstack') {
const e2eTestsPath = '/tmp/tea-automate-e2e-tests-{{timestamp}}.json';
e2eTestsOutput = JSON.parse(fs.readFileSync(e2eTestsPath, 'utf8'));
}
```
**Read Backend test subagent output (if {detected_stack} is `backend` or `fullstack`):**
```javascript
let backendTestsOutput = null;
if (detected_stack === 'backend' || detected_stack === 'fullstack') {
const backendTestsPath = '/tmp/tea-automate-backend-tests-{{timestamp}}.json';
backendTestsOutput = JSON.parse(fs.readFileSync(backendTestsPath, 'utf8'));
}
```
**Verify all launched subagents succeeded:**
- Check `apiTestsOutput.success === true`
- If E2E was launched: check `e2eTestsOutput.success === true`
- If Backend was launched: check `backendTestsOutput.success === true`
- If any failed, report error and stop (don't proceed)
---
### 2. Write All Test Files to Disk
**Write API test files:**
```javascript
apiTestsOutput.tests.forEach((test) => {
fs.writeFileSync(test.file, test.content, 'utf8');
console.log(`✅ Created: ${test.file}`);
});
```
**Write E2E test files (if {detected_stack} is `frontend` or `fullstack`):**
```javascript
if (e2eTestsOutput) {
e2eTestsOutput.tests.forEach((test) => {
fs.writeFileSync(test.file, test.content, 'utf8');
console.log(`✅ Created: ${test.file}`);
});
}
```
**Write Backend test files (if {detected_stack} is `backend` or `fullstack`):**
```javascript
if (backendTestsOutput) {
backendTestsOutput.testsGenerated.forEach((test) => {
fs.writeFileSync(test.file, test.content, 'utf8');
console.log(`✅ Created: ${test.file}`);
});
}
```
---
### 3. Aggregate Fixture Needs
**Collect all fixture needs from all launched subagents:**
```javascript
const allFixtureNeeds = [
...apiTestsOutput.fixture_needs,
...(e2eTestsOutput ? e2eTestsOutput.fixture_needs : []),
...(backendTestsOutput ? backendTestsOutput.coverageSummary?.fixtureNeeds || [] : []),
];
// Remove duplicates
const uniqueFixtures = [...new Set(allFixtureNeeds)];
```
**Categorize fixtures:**
- **Authentication fixtures:** authToken, authenticatedUserFixture, etc.
- **Data factories:** userDataFactory, productDataFactory, etc.
- **Network mocks:** paymentMockFixture, apiResponseMocks, etc.
- **Test helpers:** wait/retry/assertion helpers
---
### 4. Generate Fixture Infrastructure
**Create or update fixture files based on needs:**
**A) Authentication Fixtures** (`tests/fixtures/auth.ts`):
```typescript
import { test as base } from '@playwright/test';
export const test = base.extend({
authenticatedUser: async ({ page }, use) => {
// Login logic
await page.goto('/login');
await page.fill('[name="email"]', 'test@example.com');
await page.fill('[name="password"]', 'password');
await page.click('button[type="submit"]');
await page.waitForURL('/dashboard');
await use(page);
},
authToken: async ({ request }, use) => {
// Get auth token for API tests
const response = await request.post('/api/auth/login', {
data: { email: 'test@example.com', password: 'password' },
});
const { token } = await response.json();
await use(token);
},
});
```
**B) Data Factories** (`tests/fixtures/data-factories.ts`):
```typescript
import { faker } from '@faker-js/faker';
export const createUserData = (overrides = {}) => ({
name: faker.person.fullName(),
email: faker.internet.email(),
...overrides,
});
export const createProductData = (overrides = {}) => ({
name: faker.commerce.productName(),
price: faker.number.int({ min: 10, max: 1000 }),
...overrides,
});
```
**C) Network Mocks** (`tests/fixtures/network-mocks.ts`):
```typescript
import { Page } from '@playwright/test';
export const mockPaymentSuccess = async (page: Page) => {
await page.route('/api/payment/**', (route) => {
route.fulfill({
status: 200,
body: JSON.stringify({ success: true, transactionId: '12345' }),
});
});
};
```
**D) Helper Utilities** (`tests/fixtures/helpers.ts`):
```typescript
import { expect, Page } from '@playwright/test';
export const waitForApiResponse = async (page: Page, urlPattern: string) => {
return page.waitForResponse((response) => response.url().includes(urlPattern) && response.ok());
};
```
---
### 5. Calculate Summary Statistics
**Aggregate test counts (based on `{detected_stack}`):**
```javascript
const e2eCount = e2eTestsOutput ? e2eTestsOutput.test_count : 0;
const backendCount = backendTestsOutput ? (backendTestsOutput.coverageSummary?.totalTests ?? 0) : 0;
const resolvedMode = subagentContext?.execution?.resolvedMode;
const subagentExecutionLabel =
resolvedMode === 'sequential'
? 'SEQUENTIAL (API then dependent workers)'
: resolvedMode === 'agent-team'
? 'AGENT-TEAM (parallel worker squad)'
: resolvedMode === 'subagent'
? 'SUBAGENT (parallel subagents)'
: `PARALLEL (based on ${detected_stack})`;
const performanceGainLabel =
resolvedMode === 'sequential'
? 'baseline (no parallel speedup)'
: resolvedMode === 'agent-team' || resolvedMode === 'subagent'
? '~40-70% faster than sequential'
: 'mode-dependent';
const summary = {
detected_stack: '{detected_stack}',
total_tests: apiTestsOutput.test_count + e2eCount + backendCount,
api_tests: apiTestsOutput.test_count,
e2e_tests: e2eCount,
backend_tests: backendCount,
fixtures_created: uniqueFixtures.length,
api_test_files: apiTestsOutput.tests.length,
e2e_test_files: e2eTestsOutput ? e2eTestsOutput.tests.length : 0,
backend_test_files: backendTestsOutput ? backendTestsOutput.testsGenerated.length : 0,
priority_coverage: {
P0:
(apiTestsOutput.priority_coverage?.P0 ?? 0) +
(e2eTestsOutput?.priority_coverage?.P0 ?? 0) +
(backendTestsOutput?.testsGenerated?.reduce((sum, t) => sum + (t.priority_coverage?.P0 ?? 0), 0) ?? 0),
P1:
(apiTestsOutput.priority_coverage?.P1 ?? 0) +
(e2eTestsOutput?.priority_coverage?.P1 ?? 0) +
(backendTestsOutput?.testsGenerated?.reduce((sum, t) => sum + (t.priority_coverage?.P1 ?? 0), 0) ?? 0),
P2:
(apiTestsOutput.priority_coverage?.P2 ?? 0) +
(e2eTestsOutput?.priority_coverage?.P2 ?? 0) +
(backendTestsOutput?.testsGenerated?.reduce((sum, t) => sum + (t.priority_coverage?.P2 ?? 0), 0) ?? 0),
P3:
(apiTestsOutput.priority_coverage?.P3 ?? 0) +
(e2eTestsOutput?.priority_coverage?.P3 ?? 0) +
(backendTestsOutput?.testsGenerated?.reduce((sum, t) => sum + (t.priority_coverage?.P3 ?? 0), 0) ?? 0),
},
knowledge_fragments_used: [
...apiTestsOutput.knowledge_fragments_used,
...(e2eTestsOutput ? e2eTestsOutput.knowledge_fragments_used : []),
...(backendTestsOutput ? backendTestsOutput.knowledge_fragments_used || [] : []),
],
subagent_execution: subagentExecutionLabel,
performance_gain: performanceGainLabel,
};
```
**Store summary for Step 4:**
Save summary to temp file for validation step:
```javascript
fs.writeFileSync('/tmp/tea-automate-summary-{{timestamp}}.json', JSON.stringify(summary, null, 2), 'utf8');
```
---
### 6. Optional Cleanup
**Clean up subagent temp files** (optional - can keep for debugging):
```javascript
fs.unlinkSync(apiTestsPath);
if (e2eTestsOutput) fs.unlinkSync('/tmp/tea-automate-e2e-tests-{{timestamp}}.json');
if (backendTestsOutput) fs.unlinkSync('/tmp/tea-automate-backend-tests-{{timestamp}}.json');
console.log('✅ Subagent temp files cleaned up');
```
---
## OUTPUT SUMMARY
Display to user:
```
✅ Test Generation Complete ({subagent_execution})
📊 Summary:
- Stack Type: {detected_stack}
- Total Tests: {total_tests}
- API Tests: {api_tests} ({api_test_files} files)
- E2E Tests: {e2e_tests} ({e2e_test_files} files) [if frontend/fullstack]
- Backend Tests: {backend_tests} ({backend_test_files} files) [if backend/fullstack]
- Fixtures Created: {fixtures_created}
- Priority Coverage:
- P0 (Critical): {P0} tests
- P1 (High): {P1} tests
- P2 (Medium): {P2} tests
- P3 (Low): {P3} tests
🚀 Performance: {performance_gain}
📂 Generated Files:
- tests/api/[feature].spec.ts [always]
- tests/e2e/[feature].spec.ts [if frontend/fullstack]
- tests/unit/[feature].test.* [if backend/fullstack]
- tests/integration/[feature].test.* [if backend/fullstack]
- tests/fixtures/ or tests/support/ [shared infrastructure]
✅ Ready for validation (Step 4)
```
---
## EXIT CONDITION
Proceed to Step 4 when:
- ✅ All test files written to disk (API + E2E and/or Backend, based on `{detected_stack}`)
- ✅ All fixtures and helpers created
- ✅ Summary statistics calculated and saved
- ✅ Output displayed to user
---
### 7. Save Progress
**Save this step's accumulated work to `{outputFile}`.**
- **If `{outputFile}` does not exist** (first save), create it with YAML frontmatter:
```yaml
---
stepsCompleted: ['step-03c-aggregate']
lastStep: 'step-03c-aggregate'
lastSaved: '{date}'
---
```
Then write this step's output below the frontmatter.
- **If `{outputFile}` already exists**, update:
- Add `'step-03c-aggregate'` to `stepsCompleted` array (only if not already present)
- Set `lastStep: 'step-03c-aggregate'`
- Set `lastSaved: '{date}'`
- Append this step's output to the appropriate section.
Load next step: `{nextStepFile}`
---
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- All launched subagents succeeded (based on `{detected_stack}`)
- All test files written to disk
- Fixtures generated based on subagent needs
- Summary complete and accurate
### ❌ SYSTEM FAILURE:
- One or more subagents failed
- Test files not written to disk
- Fixtures missing or incomplete
- Summary missing or inaccurate
**Master Rule:** Do NOT proceed to Step 4 if aggregation incomplete.

View File

@@ -0,0 +1,106 @@
---
name: 'step-04-validate-and-summarize'
description: 'Validate outputs and produce automation summary'
outputFile: '{test_artifacts}/automation-summary.md'
---
# Step 4: Validate & Summarize
## STEP GOAL
Validate generated outputs and produce a concise automation summary.
## MANDATORY EXECUTION RULES
- 📖 Read the entire step file before acting
- ✅ Speak in `{communication_language}`
- ✅ Validate against the checklist before completion
---
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
- 💾 Record outputs before proceeding
- 📖 Load the next step only when instructed
## CONTEXT BOUNDARIES:
- Available context: config, loaded artifacts, and knowledge fragments
- Focus: this step's goal only
- Limits: do not execute future steps
- Dependencies: prior steps' outputs (if any)
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
## 1. Validate
Use `checklist.md` to validate:
- Framework readiness
- Coverage mapping
- Test quality and structure
- Fixtures, factories, helpers
- [ ] CLI sessions cleaned up (no orphaned browsers)
- [ ] Temp artifacts stored in `{test_artifacts}/` not random locations
Fix gaps before proceeding.
---
## 2. Polish Output
Before finalizing, review the complete output document for quality:
1. **Remove duplication**: Progressive-append workflow may have created repeated sections — consolidate
2. **Verify consistency**: Ensure terminology, risk scores, and references are consistent throughout
3. **Check completeness**: All template sections should be populated or explicitly marked N/A
4. **Format cleanup**: Ensure markdown formatting is clean (tables aligned, headers consistent, no orphaned references)
---
## 3. Summary Output
Write `{outputFile}` including:
- Coverage plan by test level and priority
- Files created/updated
- Key assumptions and risks
- Next recommended workflow (e.g., `test-review` or `trace`)
---
## 4. Save Progress
**Save this step's accumulated work to `{outputFile}`.**
- **If `{outputFile}` does not exist** (first save), create it with YAML frontmatter:
```yaml
---
stepsCompleted: ['step-04-validate-and-summarize']
lastStep: 'step-04-validate-and-summarize'
lastSaved: '{date}'
---
```
Then write this step's output below the frontmatter.
- **If `{outputFile}` already exists**, update:
- Add `'step-04-validate-and-summarize'` to `stepsCompleted` array (only if not already present)
- Set `lastStep: 'step-04-validate-and-summarize'`
- Set `lastSaved: '{date}'`
- Append this step's output to the appropriate section.
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Step completed in full with required outputs
### ❌ SYSTEM FAILURE:
- Skipped sequence steps or missing outputs
**Master Rule:** Skipping steps is FORBIDDEN.

View File

@@ -0,0 +1,65 @@
---
name: 'step-01-assess'
description: 'Load an existing output for editing'
nextStepFile: './step-02-apply-edit.md'
---
# Step 1: Assess Edit Target
## STEP GOAL:
Identify which output should be edited and load it.
## MANDATORY EXECUTION RULES (READ FIRST):
### Universal Rules:
- 📖 Read the complete step file before taking any action
- ✅ Speak in `{communication_language}`
### Role Reinforcement:
- ✅ You are the Master Test Architect
### Step-Specific Rules:
- 🎯 Ask the user which output file to edit
- 🚫 Do not edit until target is confirmed
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
## CONTEXT BOUNDARIES:
- Available context: existing outputs
- Focus: select edit target
- Limits: no edits yet
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly.
### 1. Identify Target
Ask the user to provide the output file path or select from known outputs.
### 2. Load Target
Read the provided output file in full.
### 3. Confirm
Confirm the target and proceed to edit.
Load next step: `{nextStepFile}`
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Target identified and loaded
### ❌ SYSTEM FAILURE:
- Proceeding without a confirmed target

View File

@@ -0,0 +1,60 @@
---
name: 'step-02-apply-edit'
description: 'Apply edits to the selected output'
---
# Step 2: Apply Edits
## STEP GOAL:
Apply the requested edits to the selected output and confirm changes.
## MANDATORY EXECUTION RULES (READ FIRST):
### Universal Rules:
- 📖 Read the complete step file before taking any action
- ✅ Speak in `{communication_language}`
### Role Reinforcement:
- ✅ You are the Master Test Architect
### Step-Specific Rules:
- 🎯 Only apply edits explicitly requested by the user
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
## CONTEXT BOUNDARIES:
- Available context: selected output and user changes
- Focus: apply edits only
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly.
### 1. Confirm Requested Changes
Restate what will be changed and confirm.
### 2. Apply Changes
Update the output file accordingly.
### 3. Report
Summarize the edits applied.
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Changes applied and confirmed
### ❌ SYSTEM FAILURE:
- Unconfirmed edits or missing update

View File

@@ -0,0 +1,67 @@
---
name: 'step-01-validate'
description: 'Validate workflow outputs against checklist'
outputFile: '{test_artifacts}/automate-validation-report.md'
validationChecklist: '../checklist.md'
---
# Step 1: Validate Outputs
## STEP GOAL:
Validate outputs using the workflow checklist and record findings.
## MANDATORY EXECUTION RULES (READ FIRST):
### Universal Rules:
- 📖 Read the complete step file before taking any action
- ✅ Speak in `{communication_language}`
### Role Reinforcement:
- ✅ You are the Master Test Architect
### Step-Specific Rules:
- 🎯 Validate against `{validationChecklist}`
- 🚫 Do not skip checks
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
- 💾 Write findings to `{outputFile}`
## CONTEXT BOUNDARIES:
- Available context: workflow outputs and checklist
- Focus: validation only
- Limits: do not modify outputs in this step
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly.
### 1. Load Checklist
Read `{validationChecklist}` and list all criteria.
### 2. Validate Outputs
Evaluate outputs against each checklist item.
### 3. Write Report
Write a validation report to `{outputFile}` with PASS/WARN/FAIL per section.
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Validation report written
- All checklist items evaluated
### ❌ SYSTEM FAILURE:
- Skipped checklist items
- No report produced

View File

@@ -0,0 +1,72 @@
---
validationDate: 2026-01-27
workflowName: testarch-automate
workflowPath: {project-root}/src/workflows/testarch/automate
validationStatus: COMPLETE
completionDate: 2026-01-27 10:03:10
---
# Validation Report: testarch-automate
**Validation Started:** 2026-01-27 09:50:21
**Validator:** BMAD Workflow Validation System (Codex)
**Standards Version:** BMAD Workflow Standards
## File Structure & Size
- workflow.md present: YES
- instructions.md present: YES
- workflow.yaml present: YES
- step files found: 7
**Step File Sizes:**
- steps-c/step-01-preflight-and-context.md: 113 lines [GOOD]
- steps-c/step-02-identify-targets.md: 85 lines [GOOD]
- steps-c/step-03-generate-tests.md: 76 lines [GOOD]
- steps-c/step-04-validate-and-summarize.md: 62 lines [GOOD]
- steps-e/step-01-assess.md: 51 lines [GOOD]
- steps-e/step-02-apply-edit.md: 46 lines [GOOD]
- steps-v/step-01-validate.md: 53 lines [GOOD]
- workflow-plan.md present: YES
## Frontmatter Validation
- No frontmatter violations found
## Critical Path Violations
- No {project-root} hardcoded paths detected in body
- No dead relative links detected
## Menu Handling Validation
- No menu structures detected (linear step flow) [N/A]
## Step Type Validation
- Last step steps-v/step-01-validate.md has no nextStepFile (final step OK)
- Step type validation assumes linear sequence (no branching/menu). Workflow-plan.md present for reference. [INFO]
## Output Format Validation
- No templates found in workflow root
- Steps with outputFile in frontmatter:
- steps-c/step-04-validate-and-summarize.md
- steps-v/step-01-validate.md
## Validation Design Check
- checklist.md present: YES
- Validation steps folder (steps-v) present: YES
## Instruction Style Check
- All steps include STEP GOAL, MANDATORY EXECUTION RULES, EXECUTION PROTOCOLS, CONTEXT BOUNDARIES, and SUCCESS/FAILURE metrics
## Summary
- Validation completed: 2026-01-27 10:03:10
- Critical issues: 0
- Warnings: 0 (informational notes only)
- Readiness: READY (manual review optional)

View File

@@ -0,0 +1,114 @@
---
validationDate: 2026-01-27
workflowName: testarch-automate
workflowPath: {project-root}/src/workflows/testarch/automate
validationStatus: COMPLETE
completionDate: 2026-01-27 10:24:01
---
# Validation Report: testarch-automate
**Validation Started:** 2026-01-27 10:24:01
**Validator:** BMAD Workflow Validation System (Codex)
**Standards Version:** BMAD Workflow Standards
## File Structure & Size
- workflow.md present: YES
- instructions.md present: YES
- workflow.yaml present: YES
- step files found: 7
**Step File Sizes:**
- steps-c/step-01-preflight-and-context.md: 112 lines [GOOD]
- steps-c/step-02-identify-targets.md: 84 lines [GOOD]
- steps-c/step-03-generate-tests.md: 75 lines [GOOD]
- steps-c/step-04-validate-and-summarize.md: 61 lines [GOOD]
- steps-e/step-01-assess.md: 50 lines [GOOD]
- steps-e/step-02-apply-edit.md: 45 lines [GOOD]
- steps-v/step-01-validate.md: 52 lines [GOOD]
- workflow-plan.md present: YES
## Frontmatter Validation
- No frontmatter violations found
## Critical Path Violations
### Config Variables (Exceptions)
Standard BMAD config variables treated as valid exceptions: bmb_creations_output_folder, communication_language, document_output_language, output_folder, planning_artifacts, project-root, project_name, test_artifacts, user_name
- No {project-root} hardcoded paths detected in body
- No dead relative links detected
- No module path assumptions detected
**Status:** ✅ PASS - No critical violations
## Menu Handling Validation
- No menu structures detected (linear step flow) [N/A]
## Step Type Validation
- steps-c/step-01-preflight-and-context.md: Init [PASS]
- steps-c/step-02-identify-targets.md: Middle [PASS]
- steps-c/step-03-generate-tests.md: Middle [PASS]
- steps-c/step-04-validate-and-summarize.md: Final [PASS]
- Step type validation assumes linear sequence (no branching/menu). Workflow-plan.md present for reference. [INFO]
## Output Format Validation
- Templates present: NONE
- Steps with outputFile in frontmatter:
- steps-c/step-04-validate-and-summarize.md
- steps-v/step-01-validate.md
- checklist.md present: YES
## Validation Design Check
- Validation steps folder (steps-v) present: YES
- Validation step(s) present: step-01-validate.md
- Validation steps reference checklist data and auto-proceed
## Instruction Style Check
- Instruction style: Prescriptive (appropriate for TEA quality/compliance workflows)
- Steps emphasize mandatory sequence, explicit success/failure metrics, and risk-based guidance
## Collaborative Experience Check
- Overall facilitation quality: GOOD
- Steps use progressive prompts and clear role reinforcement; no laundry-list interrogation detected
- Flow progression is clear and aligned to workflow goals
## Subagent Optimization Opportunities
- No high-priority subagent optimizations identified; workflow already uses step-file architecture
- Pattern 1 (grep/regex): N/A for most steps
- Pattern 2 (per-file analysis): already aligned to validation structure
- Pattern 3 (data ops): minimal data file loads
- Pattern 4 (parallel): optional for validation only
## Cohesive Review
- Overall assessment: GOOD
- Flow is linear, goals are clear, and outputs map to TEA artifacts
- Voice and tone consistent with Test Architect persona
- Recommendation: READY (minor refinements optional)
## Plan Quality Validation
- Plan file present: workflow-plan.md
- Planned steps found: 7 (all implemented)
- Plan implementation status: Fully Implemented
## Summary
- Validation completed: 2026-01-27 10:24:01
- Critical issues: 0
- Warnings: 0 (informational notes only)
- Readiness: READY (manual review optional)

View File

@@ -0,0 +1,20 @@
# Workflow Plan: testarch-automate
## Create Mode (steps-c)
- step-01-preflight-and-context.md
- step-02-identify-targets.md
- step-03-generate-tests.md
- step-04-validate-and-summarize.md
## Validate Mode (steps-v)
- step-01-validate.md
## Edit Mode (steps-e)
- step-01-assess.md
- step-02-apply-edit.md
## Outputs
- {test_artifacts}/automation-summary.md
- Test files under {project-root}/tests

View File

@@ -0,0 +1,41 @@
---
name: testarch-automate
description: Expand test automation coverage for codebase. Use when user says 'lets expand test coverage' or 'I want to automate tests'
web_bundle: true
---
# Test Automation Expansion
**Goal:** Expand test automation coverage after implementation or analyze existing codebase to generate comprehensive test suite
**Role:** You are the Master Test Architect.
---
## WORKFLOW ARCHITECTURE
This workflow uses **tri-modal step-file architecture**:
- **Create mode (steps-c/)**: primary execution flow
- **Validate mode (steps-v/)**: validation against checklist
- **Edit mode (steps-e/)**: revise existing outputs
---
## INITIALIZATION SEQUENCE
### 1. Mode Determination
"Welcome to the workflow. What would you like to do?"
- **[C] Create** — Run the workflow
- **[R] Resume** — Resume an interrupted workflow
- **[V] Validate** — Validate existing outputs
- **[E] Edit** — Edit existing outputs
### 2. Route to First Step
- **If C:** Load `steps-c/step-01-preflight-and-context.md`
- **If R:** Load `steps-c/step-01b-resume.md`
- **If V:** Load `steps-v/step-01-validate.md`
- **If E:** Load `steps-e/step-01-assess.md`

View File

@@ -0,0 +1,53 @@
# Test Architect workflow: automate
name: testarch-automate
# prettier-ignore
description: 'Expand test automation coverage for codebase. Use when the user says "lets expand test coverage" or "I want to automate tests"'
# Critical variables from config
config_source: "{project-root}/_bmad/tea/config.yaml"
output_folder: "{config_source}:output_folder"
test_artifacts: "{config_source}:test_artifacts"
user_name: "{config_source}:user_name"
communication_language: "{config_source}:communication_language"
document_output_language: "{config_source}:document_output_language"
date: system-generated
# Workflow components
installed_path: "{project-root}/_bmad/tea/workflows/testarch/automate"
instructions: "{installed_path}/instructions.md"
validation: "{installed_path}/checklist.md"
template: false
# Variables and inputs
variables:
# Execution mode and targeting
standalone_mode: true # Can work without BMad artifacts (true) or integrate with BMad (false)
coverage_target: "critical-paths" # critical-paths, comprehensive, selective
# Directory paths
test_dir: "{project-root}/tests" # Root test directory
source_dir: "{project-root}" # Source code directory (customize if needed, e.g., {project-root}/src or {project-root}/lib)
# Output configuration
default_output_file: "{test_artifacts}/automation-summary.md"
# Required tools
required_tools:
- read_file # Read source code, existing tests, BMad artifacts
- write_file # Create test files, fixtures, factories, summaries
- create_directory # Create test directories
- list_files # Discover features and existing tests
- search_repo # Find coverage gaps and patterns
- glob # Find test files and source files
tags:
- qa
- automation
- test-architect
- regression
- coverage
execution_hints:
interactive: false # Minimize prompts
autonomous: true # Proceed without user input unless blocked
iterative: true

View File

@@ -0,0 +1,155 @@
# Azure DevOps CI/CD Pipeline for Test Execution
# Generated by BMad TEA Agent - Test Architect Module
# Optimized for: Parallel Sharding, Burn-In Loop
# Stack: {test_stack_type} | Framework: {test_framework}
#
# Variables to customize per project:
# INSTALL_CMD - dependency install command (e.g., npm ci, pnpm install --frozen-lockfile)
# TEST_CMD - main test command (e.g., npm run test:e2e, npm test, npx vitest)
# LINT_CMD - lint command (e.g., npm run lint)
# BROWSER_INSTALL - browser install command (frontend/fullstack only; omit for backend)
# DEFAULT_NODE_VERSION - Node.js version (read from .nvmrc or default to 24)
trigger:
branches:
include:
- main
- develop
pr:
branches:
include:
- main
- develop
variables:
DEFAULT_NODE_VERSION: "24"
npm_config_cache: $(Pipeline.Workspace)/.npm
# Set TEST_STACK_TYPE to 'backend' to skip Playwright browser installs
TEST_STACK_TYPE: "" # Values: frontend, backend, fullstack (leave empty for auto)
stages:
# Lint stage - Code quality checks
- stage: Lint
displayName: "Lint"
jobs:
- job: LintJob
displayName: "Code Quality"
pool:
vmImage: "ubuntu-latest"
timeoutInMinutes: 5
steps:
- task: NodeTool@0
inputs:
versionSpec: $(DEFAULT_NODE_VERSION)
displayName: "Setup Node.js"
- task: Cache@2
inputs:
key: 'npm | "$(Agent.OS)" | package-lock.json'
restoreKeys: 'npm | "$(Agent.OS)"'
path: $(npm_config_cache)
displayName: "Cache npm"
- script: npm ci
displayName: "Install dependencies" # Replace with INSTALL_CMD
- script: npm run lint
displayName: "Run linter" # Replace with LINT_CMD
# Test stage - Parallel execution with sharding
- stage: Test
displayName: "Test"
dependsOn: Lint
jobs:
- job: TestShard
displayName: "Test Shard"
pool:
vmImage: "ubuntu-latest"
timeoutInMinutes: 30
strategy:
matrix:
Shard1:
SHARD_INDEX: 1
Shard2:
SHARD_INDEX: 2
Shard3:
SHARD_INDEX: 3
Shard4:
SHARD_INDEX: 4
steps:
- task: NodeTool@0
inputs:
versionSpec: $(DEFAULT_NODE_VERSION)
displayName: "Setup Node.js"
- task: Cache@2
inputs:
key: 'npm | "$(Agent.OS)" | package-lock.json'
restoreKeys: 'npm | "$(Agent.OS)"'
path: $(npm_config_cache)
displayName: "Cache npm"
- script: npm ci
displayName: "Install dependencies" # Replace with INSTALL_CMD
# Frontend/Fullstack only — skipped for backend-only stacks
- script: npx playwright install --with-deps chromium
condition: ne(variables['TEST_STACK_TYPE'], 'backend')
displayName: "Install Playwright browsers" # Replace with BROWSER_INSTALL
- script: npm run test:e2e -- --shard=$(SHARD_INDEX)/4
displayName: "Run tests (shard $(SHARD_INDEX)/4)" # Replace with TEST_CMD + shard args
- task: PublishTestResults@2
condition: always()
inputs:
testResultsFormat: "JUnit"
testResultsFiles: "test-results/**/*.xml"
mergeTestResults: true
displayName: "Publish test results"
- publish: test-results/
artifact: test-results-$(SHARD_INDEX)
condition: failed()
displayName: "Upload failure artifacts"
# Burn-in stage - Flaky test detection
# Note: Burn-in targets UI flakiness. For backend-only stacks, remove this stage entirely.
- stage: BurnIn
displayName: "Burn-In (Flaky Detection)"
dependsOn: Test
condition: and(succeeded(), or(eq(variables['Build.Reason'], 'PullRequest'), eq(variables['Build.CronSchedule.DisplayName'], 'Weekly burn-in')))
jobs:
- job: BurnInJob
displayName: "Burn-In Loop"
pool:
vmImage: "ubuntu-latest"
timeoutInMinutes: 60
steps:
- task: NodeTool@0
inputs:
versionSpec: $(DEFAULT_NODE_VERSION)
displayName: "Setup Node.js"
- script: npm ci
displayName: "Install dependencies" # Replace with INSTALL_CMD
# Frontend/Fullstack only — skipped for backend-only stacks
- script: npx playwright install --with-deps chromium
condition: ne(variables['TEST_STACK_TYPE'], 'backend')
displayName: "Install Playwright browsers" # Replace with BROWSER_INSTALL
- script: |
echo "Starting burn-in loop - detecting flaky tests"
for i in $(seq 1 10); do
echo "Burn-in iteration $i/10"
npm run test:e2e || exit 1
done
echo "Burn-in complete - no flaky tests detected"
displayName: "Run burn-in loop (10 iterations)" # Replace npm run test:e2e with TEST_CMD
- publish: test-results/
artifact: burn-in-failures
condition: failed()
displayName: "Upload burn-in failure artifacts"

View File

@@ -0,0 +1,289 @@
# CI/CD Pipeline Setup - Validation Checklist
## Prerequisites
- [ ] Git repository initialized (`.git/` exists)
- [ ] Git remote configured (`git remote -v` shows origin)
- [ ] Test framework configured (appropriate config for detected stack type)
- [ ] Local tests pass (test command succeeds)
- [ ] Team agrees on CI platform
- [ ] Access to CI platform settings (if updating)
### Multi-Stack Detection
- [ ] Test stack type detected or configured (`frontend`, `backend`, `fullstack`)
- [ ] Test framework detected or configured (Playwright, Cypress, Jest, Vitest, etc.)
- [ ] Stack-appropriate test commands identified
### Multi-Platform Detection
- [ ] CI platform detected or configured
- [ ] Supported platform: GitHub Actions, GitLab CI, Jenkins, Azure DevOps, Harness, or Circle CI
- [ ] Platform-specific template selected
Note: CI setup is typically a one-time task per repo and can be run any time after the test framework is configured.
## Process Steps
### Step 1: Preflight Checks
- [ ] Git repository validated
- [ ] Framework configuration detected
- [ ] Local test execution successful
- [ ] CI platform detected or selected
- [ ] Node version identified (.nvmrc or default)
- [ ] No blocking issues found
### Step 2: CI Pipeline Configuration
- [ ] CI configuration file created at platform-correct path
- GitHub Actions: `.github/workflows/test.yml`
- GitLab CI: `.gitlab-ci.yml`
- Jenkins: `Jenkinsfile`
- Azure DevOps: `azure-pipelines.yml`
- Harness: `.harness/pipeline.yaml`
- Circle CI: `.circleci/config.yml`
- [ ] File is syntactically valid (no YAML/Groovy errors)
- [ ] Correct framework commands configured for detected stack type
- [ ] Node version matches project
- [ ] Test directory paths correct
- [ ] Stack-conditional steps applied:
- [ ] Browser install included for frontend/fullstack stacks
- [ ] Browser install omitted for backend-only stacks
- [ ] Test commands match detected framework
### Step 3: Parallel Sharding
- [ ] Matrix strategy configured (4 shards default)
- [ ] Shard syntax correct for framework
- [ ] fail-fast set to false
- [ ] Shard count appropriate for test suite size
### Step 4: Burn-In Loop
- [ ] Burn-in job created (frontend/fullstack stacks) or intentionally skipped (backend-only)
- [ ] 10 iterations configured (when enabled)
- [ ] Proper exit on failure (`|| exit 1`)
- [ ] Runs on appropriate triggers (PR, cron)
- [ ] Failure artifacts uploaded
- [ ] Backend-only stacks: burn-in skipped by default (documented reason: targets UI flakiness)
### Step 5: Caching Configuration
- [ ] Dependency cache configured (npm/yarn)
- [ ] Cache key uses lockfile hash
- [ ] Browser cache configured (Playwright/Cypress)
- [ ] Restore-keys defined for fallback
- [ ] Cache paths correct for platform
### Step 6: Artifact Collection
- [ ] Artifacts upload on failure only
- [ ] Correct artifact paths (test-results/, traces/, etc.)
- [ ] Retention days set (30 default)
- [ ] Artifact names unique per shard
- [ ] No sensitive data in artifacts
### Step 7: Retry Logic
- [ ] Retry action/strategy configured
- [ ] Max attempts: 2-3
- [ ] Timeout appropriate (30 min)
- [ ] Retry only on transient errors
### Step 8: Helper Scripts
- [ ] `scripts/test-changed.sh` created
- [ ] `scripts/ci-local.sh` created
- [ ] `scripts/burn-in.sh` created (optional)
- [ ] Scripts are executable (`chmod +x`)
- [ ] Scripts use correct test commands
- [ ] Shebang present (`#!/bin/bash`)
### Step 9: Documentation
- [ ] `docs/ci.md` created with pipeline guide
- [ ] `docs/ci-secrets-checklist.md` created
- [ ] Required secrets documented
- [ ] Setup instructions clear
- [ ] Troubleshooting section included
- [ ] Badge URLs provided (optional)
## Output Validation
### Configuration Validation
- [ ] CI file loads without errors
- [ ] All paths resolve correctly
- [ ] No hardcoded values (use env vars)
- [ ] Triggers configured (push, pull_request, schedule)
- [ ] Platform-specific syntax correct
### Execution Validation
- [ ] First CI run triggered (push to remote)
- [ ] Pipeline starts without errors
- [ ] All jobs appear in CI dashboard
- [ ] Caching works (check logs for cache hit)
- [ ] Tests execute in parallel
- [ ] Artifacts collected on failure
### Performance Validation
- [ ] Lint stage: <2 minutes
- [ ] Test stage (per shard): <10 minutes
- [ ] Burn-in stage: <30 minutes
- [ ] Total pipeline: <45 minutes
- [ ] Cache reduces install time by 2-5 minutes
## Quality Checks
### Best Practices Compliance
- [ ] Burn-in loop follows production patterns
- [ ] Parallel sharding configured optimally
- [ ] Failure-only artifact collection
- [ ] Selective testing enabled (optional)
- [ ] Retry logic handles transient failures only
- [ ] No secrets in configuration files
### Knowledge Base Alignment
- [ ] Burn-in pattern matches `ci-burn-in.md`
- [ ] Selective testing matches `selective-testing.md`
- [ ] Artifact collection matches `visual-debugging.md`
- [ ] Test quality matches `test-quality.md`
### Security Checks
- [ ] No credentials in CI configuration
- [ ] Secrets use platform secret management
- [ ] Environment variables for sensitive data
- [ ] Artifact retention appropriate (not too long)
- [ ] No debug output exposing secrets
- [ ] **MUST**: No `${{ inputs.* }}` or user-controlled GitHub context (`github.event.pull_request.title`, `github.event.issue.body`, `github.event.comment.body`, `github.head_ref`) directly in `run:` blocks all passed through `env:` intermediaries and referenced as `"$ENV_VAR"`
## Integration Points
### Status File Integration
- [ ] CI setup logged in Quality & Testing Progress section
- [ ] Status updated with completion timestamp
- [ ] Platform and configuration noted
### Knowledge Base Integration
- [ ] Relevant knowledge fragments loaded
- [ ] Patterns applied from knowledge base
- [ ] Documentation references knowledge base
- [ ] Knowledge base references in README
### Workflow Dependencies
- [ ] `framework` workflow completed first
- [ ] Can proceed to `atdd` workflow after CI setup
- [ ] Can proceed to `automate` workflow
- [ ] CI integrates with `gate` workflow
## Completion Criteria
**All must be true:**
- [ ] All prerequisites met
- [ ] All process steps completed
- [ ] All output validations passed
- [ ] All quality checks passed
- [ ] All integration points verified
- [ ] First CI run successful
- [ ] Performance targets met
- [ ] Documentation complete
## Post-Workflow Actions
**User must complete:**
1. [ ] Commit CI configuration
2. [ ] Push to remote repository
3. [ ] Configure required secrets in CI platform
4. [ ] Open PR to trigger first CI run
5. [ ] Monitor and verify pipeline execution
6. [ ] Adjust parallelism if needed (based on actual run times)
7. [ ] Set up notifications (optional)
**Recommended next workflows:**
1. [ ] Run `atdd` workflow for test generation
2. [ ] Run `automate` workflow for coverage expansion
3. [ ] Run `gate` workflow for quality gates
## Rollback Procedure
If workflow fails:
1. [ ] Delete CI configuration file
2. [ ] Remove helper scripts directory
3. [ ] Remove documentation (docs/ci.md, etc.)
4. [ ] Clear CI platform secrets (if added)
5. [ ] Review error logs
6. [ ] Fix issues and retry workflow
## Notes
### Common Issues
**Issue**: CI file syntax errors
- **Solution**: Validate YAML syntax online or with linter
**Issue**: Tests fail in CI but pass locally
- **Solution**: Use `scripts/ci-local.sh` to mirror CI environment
**Issue**: Caching not working
- **Solution**: Check cache key formula, verify paths
**Issue**: Burn-in too slow
- **Solution**: Reduce iterations or run on cron only
### Platform-Specific
**GitHub Actions:**
- Secrets: Repository Settings Secrets and variables Actions
- Runners: Ubuntu latest recommended
- Concurrency limits: 20 jobs for free tier
**GitLab CI:**
- Variables: Project Settings CI/CD Variables
- Runners: Shared or project-specific
- Pipeline quota: 400 minutes/month free tier
**Jenkins:**
- Credentials: Manage Jenkins Manage Credentials
- Agents: Configure build agents with Node.js
- Plugins: Pipeline, JUnit, HTML Publisher recommended
**Azure DevOps:**
- Variables: Pipelines Library Variable groups
- Agent pools: Azure-hosted or self-hosted
- Parallel jobs: 1 free (Microsoft-hosted)
**Harness:**
- Connectors: Configure container registry and code repo connectors
- Delegates: Install Harness delegate in target infrastructure
- Steps: Use Run steps with appropriate container images
---
**Checklist Complete**: Sign off when all items validated.
**Completed by:** {name}
**Date:** {date}
**Platform:** {GitHub Actions, GitLab CI, Other}
**Notes:** {notes}

View File

@@ -0,0 +1,328 @@
# GitHub Actions CI/CD Pipeline for Test Execution
# Generated by BMad TEA Agent - Test Architect Module
# Optimized for: Parallel Sharding, Burn-In Loop
# Stack: {test_stack_type} | Framework: {test_framework}
#
# Variables to customize per project:
# INSTALL_CMD - dependency install command (e.g., npm ci, pnpm install --frozen-lockfile, yarn --frozen-lockfile)
# TEST_CMD - main test command (e.g., npm run test:e2e, npm test, npx vitest)
# LINT_CMD - lint command (e.g., npm run lint)
# BROWSER_INSTALL - browser install command (frontend/fullstack only; omit for backend)
# BROWSER_CACHE_PATH - browser cache path (frontend/fullstack only; omit for backend)
name: Test Pipeline
on:
push:
branches: [main, develop]
pull_request:
branches: [main, develop]
schedule:
# Weekly burn-in on Sundays at 2 AM UTC
- cron: "0 2 * * 0"
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
# Lint stage - Code quality checks
lint:
name: Lint
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- uses: actions/checkout@v4
- name: Determine Node version
id: node-version
run: |
if [ -f .nvmrc ]; then
echo "value=$(cat .nvmrc)" >> "$GITHUB_OUTPUT"
echo "Using Node from .nvmrc"
else
echo "value=24" >> "$GITHUB_OUTPUT"
echo "Using default Node 24 (current LTS)"
fi
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: ${{ steps.node-version.outputs.value }}
cache: "npm"
- name: Install dependencies
run: npm ci # Replace with INSTALL_CMD
- name: Run linter
run: npm run lint # Replace with LINT_CMD
# Test stage - Parallel execution with sharding
test:
name: Test (Shard ${{ matrix.shard }})
runs-on: ubuntu-latest
timeout-minutes: 30
needs: lint
strategy:
fail-fast: false
matrix:
shard: [1, 2, 3, 4]
steps:
- uses: actions/checkout@v4
- name: Determine Node version
id: node-version
run: |
if [ -f .nvmrc ]; then
echo "value=$(cat .nvmrc)" >> "$GITHUB_OUTPUT"
echo "Using Node from .nvmrc"
else
echo "value=22" >> "$GITHUB_OUTPUT"
echo "Using default Node 22 (current LTS)"
fi
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: ${{ steps.node-version.outputs.value }}
cache: "npm"
- name: Cache Playwright browsers
uses: actions/cache@v4
with:
path: ~/.cache/ms-playwright
key: ${{ runner.os }}-playwright-${{ hashFiles('**/package-lock.json') }}
restore-keys: |
${{ runner.os }}-playwright-
- name: Install dependencies
run: npm ci # Replace with INSTALL_CMD
# Frontend/Fullstack only — remove this step for backend-only stacks
- name: Install Playwright browsers
run: npx playwright install --with-deps chromium # Replace with BROWSER_INSTALL
- name: Run tests (shard ${{ matrix.shard }}/4)
run: npm run test:e2e -- --shard=${{ matrix.shard }}/4 # Replace with TEST_CMD + shard args
- name: Upload test results
if: failure()
uses: actions/upload-artifact@v4
with:
name: test-results-${{ matrix.shard }}
path: |
test-results/
playwright-report/
retention-days: 30
# Burn-in stage - Flaky test detection
burn-in:
name: Burn-In (Flaky Detection)
runs-on: ubuntu-latest
timeout-minutes: 60
needs: test
# Only run burn-in on PRs to main/develop or on schedule
if: github.event_name == 'pull_request' || github.event_name == 'schedule'
steps:
- uses: actions/checkout@v4
- name: Determine Node version
id: node-version
run: |
if [ -f .nvmrc ]; then
echo "value=$(cat .nvmrc)" >> "$GITHUB_OUTPUT"
echo "Using Node from .nvmrc"
else
echo "value=22" >> "$GITHUB_OUTPUT"
echo "Using default Node 22 (current LTS)"
fi
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: ${{ steps.node-version.outputs.value }}
cache: "npm"
# Frontend/Fullstack only — remove this step for backend-only stacks
- name: Cache Playwright browsers
uses: actions/cache@v4
with:
path: ~/.cache/ms-playwright # Replace with BROWSER_CACHE_PATH
key: ${{ runner.os }}-playwright-${{ hashFiles('**/package-lock.json') }}
- name: Install dependencies
run: npm ci # Replace with INSTALL_CMD
# Frontend/Fullstack only — remove this step for backend-only stacks
- name: Install Playwright browsers
run: npx playwright install --with-deps chromium # Replace with BROWSER_INSTALL
# Note: Burn-in targets UI flakiness. For backend-only stacks, remove this job entirely.
- name: Run burn-in loop (10 iterations)
run: |
echo "🔥 Starting burn-in loop - detecting flaky tests"
for i in {1..10}; do
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
echo "🔥 Burn-in iteration $i/10"
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
npm run test:e2e || exit 1 # Replace with TEST_CMD
done
echo "✅ Burn-in complete - no flaky tests detected"
- name: Upload burn-in failure artifacts
if: failure()
uses: actions/upload-artifact@v4
with:
name: burn-in-failures
path: |
test-results/
playwright-report/
retention-days: 30
# Report stage - Aggregate and publish results
report:
name: Test Report
runs-on: ubuntu-latest
needs: [test, burn-in]
if: always()
steps:
- name: Download all artifacts
uses: actions/download-artifact@v4
with:
path: artifacts
- name: Generate summary
run: |
echo "## Test Execution Summary" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
echo "- **Status**: ${{ needs.test.result }}" >> $GITHUB_STEP_SUMMARY
echo "- **Burn-in**: ${{ needs.burn-in.result }}" >> $GITHUB_STEP_SUMMARY
echo "- **Shards**: 4" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
if [ "${{ needs.burn-in.result }}" == "failure" ]; then
echo "⚠️ **Flaky tests detected** - Review burn-in artifacts" >> $GITHUB_STEP_SUMMARY
fi
# ============================================================================
# EXTENSION PATTERNS — Script Injection Prevention
# ============================================================================
# When extending this template into reusable workflows, manual dispatch
# workflows, or composite actions, NEVER use ${{ inputs.* }} directly in
# run: blocks. Always pass through env: intermediaries.
#
# KEY PRINCIPLE: Inputs must be DATA, not COMMANDS.
# Pass inputs through env: and interpolate as quoted arguments into fixed
# commands. NEVER accept command-shaped inputs (e.g., install-command,
# test-command) that get executed as shell code — even through env:.
#
# --- Reusable Workflow (workflow_call) ---
#
# on:
# workflow_call:
# inputs:
# test-grep:
# description: 'Test grep filter (data only — not a command)'
# type: string
# required: false
# default: ''
# base-ref:
# description: 'Base branch for diff'
# type: string
# required: false
# default: 'main'
# burn-in-count:
# description: 'Number of burn-in iterations'
# type: string
# required: false
# default: '10'
#
# jobs:
# test:
# runs-on: ubuntu-latest
# steps:
# - uses: actions/checkout@v4
# # Fixed command — not derived from inputs
# - name: Install dependencies
# run: npm ci
# # ✅ SAFE — input is DATA passed as an argument to a fixed command
# - name: Run tests
# env:
# TEST_GREP: ${{ inputs.test-grep }}
# run: |
# # Security: inputs passed through env: to prevent script injection
# if [ -n "$TEST_GREP" ]; then
# npx playwright test --grep "$TEST_GREP"
# else
# npx playwright test
# fi
#
# --- Manual Dispatch (workflow_dispatch) ---
#
# on:
# workflow_dispatch:
# inputs:
# test-grep:
# description: 'Test grep filter (data only — not a command)'
# type: string
# required: false
# environment:
# description: 'Target environment'
# type: choice
# options: [staging, production]
#
# jobs:
# run-tests:
# runs-on: ubuntu-latest
# steps:
# - uses: actions/checkout@v4
# # ✅ SAFE — input is DATA interpolated into a fixed command
# - name: Run selected tests
# env:
# TEST_GREP: ${{ inputs.test-grep }}
# run: |
# # Security: inputs passed through env: to prevent script injection
# npx playwright test --grep "$TEST_GREP"
#
# --- Composite Action (action.yml) ---
#
# inputs:
# test-grep:
# description: 'Test grep filter (data only — not a command)'
# required: false
# default: ''
# burn-in-count:
# description: 'Number of burn-in iterations'
# required: false
# default: '10'
#
# runs:
# using: composite
# steps:
# # ✅ SAFE — inputs are DATA arguments to fixed commands
# - name: Run burn-in
# shell: bash
# env:
# TEST_GREP: ${{ inputs.test-grep }}
# BURN_IN_COUNT: ${{ inputs.burn-in-count }}
# run: |
# # Security: inputs passed through env: to prevent script injection
# for i in $(seq 1 "$BURN_IN_COUNT"); do
# echo "Burn-in iteration $i/$BURN_IN_COUNT"
# npx playwright test --grep "$TEST_GREP" || exit 1
# done
#
# ❌ NEVER DO THIS:
# # Direct ${{ inputs.* }} in run: — GitHub expression injection
# - run: npx playwright test --grep "${{ inputs.test-grep }}"
#
# # Executing input-derived env var as a command — still command injection
# - env:
# CMD: ${{ inputs.test-command }}
# run: $CMD
# ============================================================================

View File

@@ -0,0 +1,158 @@
# GitLab CI/CD Pipeline for Test Execution
# Generated by BMad TEA Agent - Test Architect Module
# Optimized for: Parallel Sharding, Burn-In Loop
# Stack: {test_stack_type} | Framework: {test_framework}
#
# Variables to customize per project:
# INSTALL_CMD - dependency install command (e.g., npm ci, pnpm install --frozen-lockfile)
# TEST_CMD - main test command (e.g., npm run test:e2e, npm test, npx vitest)
# LINT_CMD - lint command (e.g., npm run lint)
# BROWSER_INSTALL - browser install command (frontend/fullstack only; omit for backend)
# BROWSER_CACHE_PATH - browser cache path (frontend/fullstack only; omit for backend)
stages:
- lint
- test
- burn-in
- report
variables:
# Disable git depth for accurate change detection
GIT_DEPTH: 0
# Use npm ci for faster, deterministic installs
npm_config_cache: "$CI_PROJECT_DIR/.npm"
# Playwright browser cache
PLAYWRIGHT_BROWSERS_PATH: "$CI_PROJECT_DIR/.cache/ms-playwright"
# Default Node version when .nvmrc is missing
DEFAULT_NODE_VERSION: "24"
# Caching configuration
cache:
key:
files:
- package-lock.json
paths:
- .npm/
- .cache/ms-playwright/
- node_modules/
# Lint stage - Code quality checks
lint:
stage: lint
image: node:$DEFAULT_NODE_VERSION
before_script:
- |
NODE_VERSION=$(cat .nvmrc 2>/dev/null || echo "$DEFAULT_NODE_VERSION")
echo "Using Node $NODE_VERSION"
npm install -g n
n "$NODE_VERSION"
node -v
- npm ci # Replace with INSTALL_CMD
script:
- npm run lint # Replace with LINT_CMD
timeout: 5 minutes
# Test stage - Parallel execution with sharding
.test-template: &test-template
stage: test
image: node:$DEFAULT_NODE_VERSION
needs:
- lint
before_script:
- |
NODE_VERSION=$(cat .nvmrc 2>/dev/null || echo "$DEFAULT_NODE_VERSION")
echo "Using Node $NODE_VERSION"
npm install -g n
n "$NODE_VERSION"
node -v
- npm ci # Replace with INSTALL_CMD
- npx playwright install --with-deps chromium # Replace with BROWSER_INSTALL; remove for backend-only
artifacts:
when: on_failure
paths:
- test-results/
- playwright-report/
expire_in: 30 days
timeout: 30 minutes
test:shard-1:
<<: *test-template
script:
- npm run test:e2e -- --shard=1/4 # Replace with TEST_CMD + shard args
test:shard-2:
<<: *test-template
script:
- npm run test:e2e -- --shard=2/4 # Replace with TEST_CMD + shard args
test:shard-3:
<<: *test-template
script:
- npm run test:e2e -- --shard=3/4 # Replace with TEST_CMD + shard args
test:shard-4:
<<: *test-template
script:
- npm run test:e2e -- --shard=4/4 # Replace with TEST_CMD + shard args
# Burn-in stage - Flaky test detection
burn-in:
stage: burn-in
image: node:$DEFAULT_NODE_VERSION
needs:
- test:shard-1
- test:shard-2
- test:shard-3
- test:shard-4
# Only run burn-in on merge requests to main/develop or on schedule
rules:
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
- if: '$CI_PIPELINE_SOURCE == "schedule"'
before_script:
- |
NODE_VERSION=$(cat .nvmrc 2>/dev/null || echo "$DEFAULT_NODE_VERSION")
echo "Using Node $NODE_VERSION"
npm install -g n
n "$NODE_VERSION"
node -v
- npm ci # Replace with INSTALL_CMD
- npx playwright install --with-deps chromium # Replace with BROWSER_INSTALL; remove for backend-only
# Note: Burn-in targets UI flakiness. For backend-only stacks, remove this job entirely.
script:
- |
echo "🔥 Starting burn-in loop - detecting flaky tests"
for i in {1..10}; do
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
echo "🔥 Burn-in iteration $i/10"
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
npm run test:e2e || exit 1 # Replace with TEST_CMD
done
echo "✅ Burn-in complete - no flaky tests detected"
artifacts:
when: on_failure
paths:
- test-results/
- playwright-report/
expire_in: 30 days
timeout: 60 minutes
# Report stage - Aggregate results
report:
stage: report
image: alpine:latest
needs:
- test:shard-1
- test:shard-2
- test:shard-3
- test:shard-4
- burn-in
when: always
script:
- |
echo "## Test Execution Summary"
echo ""
echo "- Pipeline: $CI_PIPELINE_ID"
echo "- Shards: 4"
echo "- Branch: $CI_COMMIT_REF_NAME"
echo ""
echo "View detailed results in job artifacts"

View File

@@ -0,0 +1,159 @@
# Harness CI Pipeline for Test Execution
# Generated by BMad TEA Agent - Test Architect Module
# Optimized for: Parallel Sharding, Burn-In Loop
# Stack: {test_stack_type} | Framework: {test_framework}
#
# Variables to customize per project:
# INSTALL_CMD - dependency install command (e.g., npm ci, pnpm install --frozen-lockfile)
# TEST_CMD - main test command (e.g., npm run test:e2e, npm test, npx vitest)
# LINT_CMD - lint command (e.g., npm run lint)
# BROWSER_INSTALL - browser install command (frontend/fullstack only; omit for backend)
pipeline:
name: Test Pipeline
identifier: test_pipeline
projectIdentifier: default
orgIdentifier: default
stages:
# Lint stage - Code quality checks
- stage:
name: Lint
identifier: lint
type: CI
spec:
cloneCodebase: true
infrastructure:
type: KubernetesDirect
spec:
connectorRef: account.harnessImage
namespace: default
execution:
steps:
- step:
type: Run
name: Install dependencies
identifier: install
spec:
connectorRef: account.harnessImage
image: node:24
shell: Sh
command: npm ci # Replace with INSTALL_CMD
- step:
type: Run
name: Run linter
identifier: lint
spec:
connectorRef: account.harnessImage
image: node:24
shell: Sh
command: npm run lint # Replace with LINT_CMD
# Test stage - Parallel execution with sharding
- stage:
name: Test
identifier: test
type: CI
spec:
cloneCodebase: true
infrastructure:
type: KubernetesDirect
spec:
connectorRef: account.harnessImage
namespace: default
execution:
steps:
- step:
type: Run
name: Install dependencies
identifier: install
spec:
connectorRef: account.harnessImage
image: node:24
shell: Sh
command: npm ci # Replace with INSTALL_CMD
# Frontend/Fullstack only — remove this step for backend-only stacks
- step:
type: Run
name: Install browsers
identifier: browsers
spec:
connectorRef: account.harnessImage
image: mcr.microsoft.com/playwright:v1.50.0-noble
shell: Sh
command: npx playwright install --with-deps chromium # Replace with BROWSER_INSTALL
- parallel:
- step:
type: Run
name: Test Shard 1
identifier: shard_1
spec:
connectorRef: account.harnessImage
image: mcr.microsoft.com/playwright:v1.50.0-noble
shell: Sh
command: npm run test:e2e -- --shard=1/4 # Replace with TEST_CMD + shard args
- step:
type: Run
name: Test Shard 2
identifier: shard_2
spec:
connectorRef: account.harnessImage
image: mcr.microsoft.com/playwright:v1.50.0-noble
shell: Sh
command: npm run test:e2e -- --shard=2/4 # Replace with TEST_CMD + shard args
- step:
type: Run
name: Test Shard 3
identifier: shard_3
spec:
connectorRef: account.harnessImage
image: mcr.microsoft.com/playwright:v1.50.0-noble
shell: Sh
command: npm run test:e2e -- --shard=3/4 # Replace with TEST_CMD + shard args
- step:
type: Run
name: Test Shard 4
identifier: shard_4
spec:
connectorRef: account.harnessImage
image: mcr.microsoft.com/playwright:v1.50.0-noble
shell: Sh
command: npm run test:e2e -- --shard=4/4 # Replace with TEST_CMD + shard args
# Burn-in stage - Flaky test detection
# Note: Burn-in targets UI flakiness. For backend-only stacks, remove this stage entirely.
- stage:
name: Burn-In
identifier: burn_in
type: CI
when:
condition: <+pipeline.triggerType> == "WEBHOOK" || <+pipeline.triggerType> == "SCHEDULER"
spec:
cloneCodebase: true
infrastructure:
type: KubernetesDirect
spec:
connectorRef: account.harnessImage
namespace: default
execution:
steps:
- step:
type: Run
name: Install and burn-in
identifier: burn_in_loop
spec:
connectorRef: account.harnessImage
image: mcr.microsoft.com/playwright:v1.50.0-noble
shell: Sh
command: |
npm ci
npx playwright install --with-deps chromium
echo "Starting burn-in loop - detecting flaky tests"
for i in $(seq 1 10); do
echo "Burn-in iteration $i/10"
npm run test:e2e || exit 1
done
echo "Burn-in complete - no flaky tests detected"
# Replace npm ci with INSTALL_CMD, npm run test:e2e with TEST_CMD

View File

@@ -0,0 +1,45 @@
<!-- Powered by BMAD-CORE™ -->
# CI/CD Pipeline Setup
**Workflow ID**: `_bmad/tea/testarch/ci`
**Version**: 5.0 (Step-File Architecture)
---
## Overview
Scaffold a production-ready CI/CD quality pipeline with test execution, burn-in loops for flaky detection, parallel sharding, artifact collection, and notifications.
---
## WORKFLOW ARCHITECTURE
This workflow uses **step-file architecture**:
- **Micro-file Design**: Each step is self-contained
- **JIT Loading**: Only the current step file is in memory
- **Sequential Enforcement**: Execute steps in order
---
## INITIALIZATION SEQUENCE
### 1. Configuration Loading
From `workflow.yaml`, resolve:
- `config_source`, `test_artifacts`, `user_name`, `communication_language`, `document_output_language`, `date`
- `ci_platform`, `test_dir`
### 2. First Step
Load, read completely, and execute:
`{project-root}/_bmad/tea/workflows/testarch/ci/steps-c/step-01-preflight.md`
### 3. Resume Support
If the user selects **Resume** mode, load, read completely, and execute:
`{project-root}/_bmad/tea/workflows/testarch/ci/steps-c/step-01b-resume.md`
This checks the output document for progress tracking frontmatter and routes to the next incomplete step.

View File

@@ -0,0 +1,129 @@
// Jenkinsfile CI/CD Pipeline for Test Execution
// Generated by BMad TEA Agent - Test Architect Module
// Optimized for: Parallel Sharding, Burn-In Loop
// Stack: {test_stack_type} | Framework: {test_framework}
//
// Variables to customize per project:
// INSTALL_CMD - dependency install command (e.g., npm ci, pnpm install --frozen-lockfile)
// TEST_CMD - main test command (e.g., npm run test:e2e, npm test, npx vitest)
// LINT_CMD - lint command (e.g., npm run lint)
// BROWSER_INSTALL - browser install command (frontend/fullstack only; omit for backend)
//
// Node.js version management — choose one:
// Option A (recommended): Configure NodeJS Plugin in Jenkins Global Tool Configuration,
// then add to pipeline: tools { nodejs 'NodeJS-24' }
// Option B: Use nvm (pre-installed on agent) — this template uses nvm as the default
// Option C: Use a Docker agent — agent { docker { image 'node:24' } }
pipeline {
agent any
environment {
CI = 'true'
}
options {
timeout(time: 45, unit: 'MINUTES')
disableConcurrentBuilds()
}
stages {
stage('Checkout') {
steps {
checkout scm
}
}
stage('Install') {
steps {
// Detect and apply Node.js version from .nvmrc (falls back to v24)
// If using NodeJS Plugin instead, remove this block and add: tools { nodejs 'NodeJS-24' }
sh '''
export NVM_DIR="$HOME/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && . "$NVM_DIR/nvm.sh"
NODE_VERSION=$(cat .nvmrc 2>/dev/null || echo "24")
nvm install "$NODE_VERSION" 2>/dev/null || true
nvm use "$NODE_VERSION" 2>/dev/null || true
node --version
npm ci
''' // Replace npm ci with INSTALL_CMD
// Stash installed dependencies so parallel shards can restore them
stash includes: 'node_modules/**', name: 'deps'
}
}
stage('Lint') {
steps {
sh 'npm run lint' // Replace with LINT_CMD
}
}
// Test stage - Parallel execution with sharding
// Each shard restores dependencies via unstash for workspace safety
stage('Test') {
parallel {
stage('Shard 1') {
steps {
unstash 'deps'
// Frontend/Fullstack only — remove browser install for backend-only stacks
sh 'npx playwright install --with-deps chromium' // Replace with BROWSER_INSTALL
sh 'npm run test:e2e -- --shard=1/4' // Replace with TEST_CMD + shard args
}
}
stage('Shard 2') {
steps {
unstash 'deps'
sh 'npx playwright install --with-deps chromium' // Replace with BROWSER_INSTALL
sh 'npm run test:e2e -- --shard=2/4' // Replace with TEST_CMD + shard args
}
}
stage('Shard 3') {
steps {
unstash 'deps'
sh 'npx playwright install --with-deps chromium' // Replace with BROWSER_INSTALL
sh 'npm run test:e2e -- --shard=3/4' // Replace with TEST_CMD + shard args
}
}
stage('Shard 4') {
steps {
unstash 'deps'
sh 'npx playwright install --with-deps chromium' // Replace with BROWSER_INSTALL
sh 'npm run test:e2e -- --shard=4/4' // Replace with TEST_CMD + shard args
}
}
}
}
// Burn-in stage - Flaky test detection
// Note: Burn-in targets UI flakiness. For backend-only stacks, remove this stage entirely.
stage('Burn-In') {
when {
anyOf {
changeRequest()
triggeredBy 'TimerTrigger'
}
}
steps {
sh '''
echo "Starting burn-in loop - detecting flaky tests"
for i in $(seq 1 10); do
echo "Burn-in iteration $i/10"
npm run test:e2e || exit 1
done
echo "Burn-in complete - no flaky tests detected"
''' // Replace npm run test:e2e with TEST_CMD
}
}
}
post {
always {
// Archive test results and reports
archiveArtifacts artifacts: 'test-results/**,playwright-report/**', allowEmptyArchive: true
junit testResults: 'test-results/**/*.xml', allowEmptyResults: true
}
failure {
echo 'Pipeline failed - check test results and artifacts'
}
}
}

View File

@@ -0,0 +1,158 @@
---
name: 'step-01-preflight'
description: 'Verify prerequisites and detect CI platform'
nextStepFile: './step-02-generate-pipeline.md'
outputFile: '{test_artifacts}/ci-pipeline-progress.md'
---
# Step 1: Preflight Checks
## STEP GOAL
Verify CI prerequisites and determine target CI platform.
## MANDATORY EXECUTION RULES
- 📖 Read the entire step file before acting
- ✅ Speak in `{communication_language}`
- 🚫 Halt if requirements fail
---
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
- 💾 Record outputs before proceeding
- 📖 Load the next step only when instructed
## CONTEXT BOUNDARIES:
- Available context: config, loaded artifacts, and knowledge fragments
- Focus: this step's goal only
- Limits: do not execute future steps
- Dependencies: prior steps' outputs (if any)
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
## 1. Verify Git Repository
- `.git/` exists
- Remote configured (if available)
If missing: **HALT** with "Git repository required for CI/CD setup."
---
## 2. Detect Test Stack Type
Determine the project's test stack type (`test_stack_type`) using the following algorithm:
1. If `test_stack_type` is explicitly set in config (not `"auto"`), use that value.
2. Otherwise, auto-detect by scanning project manifests:
- **Frontend indicators**: `playwright.config.*`, `cypress.config.*`, `vite.config.*`, `next.config.*`, `src/components/`, `src/pages/`, `src/app/`
- **Backend indicators**: `pyproject.toml`, `pom.xml`/`build.gradle`, `go.mod`, `*.csproj`/`*.sln`, `Gemfile`, `Cargo.toml`, `jest.config.*`, `vitest.config.*`, `src/routes/`, `src/controllers/`, `src/api/`, `Dockerfile`, `serverless.yml`
- **Both present** → `fullstack`
- **Only frontend** → `frontend`
- **Only backend** → `backend`
- **Cannot determine** → default to `fullstack` and note assumption
Record detected `test_stack_type` in step output.
---
## 3. Verify Test Framework
- Check for framework configuration based on detected stack:
- **Frontend/Fullstack**: `playwright.config.*` or `cypress.config.*` exists
- **Backend (Node.js)**: `jest.config.*` or `vitest.config.*` or test scripts in `package.json`
- **Backend (Python)**: `pyproject.toml` with `[tool.pytest]` or `pytest.ini` or `setup.cfg` with pytest config
- **Backend (Java/Kotlin)**: `pom.xml` with surefire/failsafe plugins or `build.gradle` with test task
- **Backend (Go)**: `*_test.go` files present (Go convention — no config file needed)
- **Backend (C#/.NET)**: `*.csproj` with xUnit/NUnit/MSTest references
- **Backend (Ruby)**: `Gemfile` with rspec or `.rspec` config file
- If `test_framework` is `"auto"`, detect from config files and project manifests found
- Verify test dependencies are installed (language-appropriate package manager)
If missing: **HALT** with "Run `framework` workflow first."
---
## 4. Ensure Tests Pass Locally
- Run the main test command based on detected stack and framework:
- **Node.js**: `npm test` or `npm run test:e2e`
- **Python**: `pytest` or `python -m pytest`
- **Java**: `mvn test` or `gradle test`
- **Go**: `go test ./...`
- **C#/.NET**: `dotnet test`
- **Ruby**: `bundle exec rspec`
- If failing: **HALT** and request fixes before CI setup
---
## 5. Detect CI Platform
- If `ci_platform` is explicitly set in config (not `"auto"`), use that value.
- Otherwise, scan for existing CI configuration files:
- `.github/workflows/*.yml``github-actions`
- `.gitlab-ci.yml``gitlab-ci`
- `Jenkinsfile``jenkins`
- `azure-pipelines.yml``azure-devops`
- `.harness/*.yaml``harness`
- `.circleci/config.yml``circle-ci`
- If found, ask whether to update or replace
- If not found, infer from git remote (github.com → `github-actions`, gitlab.com → `gitlab-ci`)
- If still unresolved, default to `github-actions`
Record detected `ci_platform` in step output.
---
## 6. Read Environment Context
- Read environment context based on detected stack:
- **Node.js**: Read `.nvmrc` if present (default to Node 24+ LTS if missing); read `package.json` for dependency caching strategy
- **Python**: Read `.python-version` or `pyproject.toml` for Python version; note `pip`/`poetry`/`pipenv` for caching
- **Java**: Read `pom.xml`/`build.gradle` for Java version; note Maven/Gradle for caching
- **Go**: Read `go.mod` for Go version; note Go module cache path
- **C#/.NET**: Read `*.csproj`/`global.json` for .NET SDK version; note NuGet cache
- **Ruby**: Read `.ruby-version` or `Gemfile` for Ruby version; note Bundler cache
---
### 7. Save Progress
**Save this step's accumulated work to `{outputFile}`.**
- **If `{outputFile}` does not exist** (first save), create it with YAML frontmatter:
```yaml
---
stepsCompleted: ['step-01-preflight']
lastStep: 'step-01-preflight'
lastSaved: '{date}'
---
```
Then write this step's output below the frontmatter.
- **If `{outputFile}` already exists**, update:
- Add `'step-01-preflight'` to `stepsCompleted` array (only if not already present)
- Set `lastStep: 'step-01-preflight'`
- Set `lastSaved: '{date}'`
- Append this step's output to the appropriate section of the document.
Load next step: `{nextStepFile}`
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Step completed in full with required outputs
### ❌ SYSTEM FAILURE:
- Skipped sequence steps or missing outputs
**Master Rule:** Skipping steps is FORBIDDEN.

View File

@@ -0,0 +1,110 @@
---
name: 'step-01b-resume'
description: 'Resume interrupted workflow from last completed step'
outputFile: '{test_artifacts}/ci-pipeline-progress.md'
---
# Step 1b: Resume Workflow
## STEP GOAL
Resume an interrupted workflow by loading the existing progress document, displaying progress, verifying previously created artifacts, and routing to the next incomplete step.
## MANDATORY EXECUTION RULES
- 📖 Read the entire step file before acting
- ✅ Speak in `{communication_language}`
---
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
- 📖 Load the next step only when instructed
## CONTEXT BOUNDARIES:
- Available context: Output document with progress frontmatter
- Focus: Load progress and route to next step
- Limits: Do not re-execute completed steps
- Dependencies: Output document must exist from a previous run
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
### 1. Load Output Document
Read `{outputFile}` and parse YAML frontmatter for:
- `stepsCompleted` — array of completed step names
- `lastStep` — last completed step name
- `lastSaved` — timestamp of last save
**If `{outputFile}` does not exist**, display:
"⚠️ **No previous progress found.** There is no output document to resume from. Please use **[C] Create** to start a fresh workflow run."
**THEN:** Halt. Do not proceed.
---
### 2. Verify Previously Created Artifacts
Since this is a file-creation workflow, verify that artifacts from completed steps still exist on disk:
- If `step-02-generate-pipeline` is in `stepsCompleted`, check that the pipeline config file exists (e.g., `.github/workflows/test.yml` or equivalent)
- If any expected artifact is missing, warn the user and suggest re-running from the step that creates it
---
### 3. Display Progress Dashboard
Display:
"📋 **Workflow Resume — CI/CD Pipeline Setup**
**Last saved:** {lastSaved}
**Steps completed:** {stepsCompleted.length} of 4
1. Preflight Checks (step-01-preflight) — {✅ if in stepsCompleted, ⬜ otherwise}
2. Generate Pipeline (step-02-generate-pipeline) — {✅ if in stepsCompleted, ⬜ otherwise}
3. Configure Quality Gates (step-03-configure-quality-gates) — {✅ if in stepsCompleted, ⬜ otherwise}
4. Validate & Summary (step-04-validate-and-summary) — {✅ if in stepsCompleted, ⬜ otherwise}"
---
### 4. Route to Next Step
Based on `lastStep`, load the next incomplete step:
- `'step-01-preflight'` → Load `./step-02-generate-pipeline.md`
- `'step-02-generate-pipeline'` → Load `./step-03-configure-quality-gates.md`
- `'step-03-configure-quality-gates'` → Load `./step-04-validate-and-summary.md`
- `'step-04-validate-and-summary'`**Workflow already complete.** Display: "✅ **All steps completed.** Use **[V] Validate** to review outputs or **[E] Edit** to make revisions." Then halt.
**If `lastStep` does not match any value above**, display: "⚠️ **Unknown progress state** (`lastStep`: {lastStep}). Please use **[C] Create** to start fresh." Then halt.
**Otherwise**, load the identified step file, read completely, and execute.
The existing content in `{outputFile}` provides context from previously completed steps. Use it as reference for remaining steps.
---
## 🚨 SYSTEM SUCCESS/FAILURE METRICS
### ✅ SUCCESS:
- Output document loaded and parsed correctly
- Previously created artifacts verified
- Progress dashboard displayed accurately
- Routed to correct next step
### ❌ SYSTEM FAILURE:
- Not loading output document
- Incorrect progress display
- Routing to wrong step
- Re-executing completed steps
**Master Rule:** Resume MUST route to the exact next incomplete step. Never re-execute completed steps.

View File

@@ -0,0 +1,279 @@
---
name: 'step-02-generate-pipeline'
description: 'Generate CI pipeline configuration with adaptive orchestration (agent-team, subagent, or sequential)'
nextStepFile: './step-03-configure-quality-gates.md'
outputFile: '{test_artifacts}/ci-pipeline-progress.md'
---
# Step 2: Generate CI Pipeline
## STEP GOAL
Create platform-specific CI configuration with test execution, sharding, burn-in, and artifacts.
## MANDATORY EXECUTION RULES
- 📖 Read the entire step file before acting
- ✅ Speak in `{communication_language}`
- ✅ Resolve execution mode from explicit user request first, then config
- ✅ Apply fallback rules deterministically when requested mode is unsupported
---
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
- 💾 Record outputs before proceeding
- 📖 Load the next step only when instructed
## CONTEXT BOUNDARIES:
- Available context: config, loaded artifacts, and knowledge fragments
- Focus: this step's goal only
- Limits: do not execute future steps
- Dependencies: prior steps' outputs (if any)
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
## 0. Resolve Execution Mode (User Override First)
```javascript
const orchestrationContext = {
config: {
execution_mode: config.tea_execution_mode || 'auto', // "auto" | "subagent" | "agent-team" | "sequential"
capability_probe: config.tea_capability_probe !== false, // true by default
},
timestamp: new Date().toISOString().replace(/[:.]/g, '-'),
};
const normalizeUserExecutionMode = (mode) => {
if (typeof mode !== 'string') return null;
const normalized = mode.trim().toLowerCase().replace(/[-_]/g, ' ').replace(/\s+/g, ' ');
if (normalized === 'auto') return 'auto';
if (normalized === 'sequential') return 'sequential';
if (normalized === 'subagent' || normalized === 'sub agent' || normalized === 'subagents' || normalized === 'sub agents') {
return 'subagent';
}
if (normalized === 'agent team' || normalized === 'agent teams' || normalized === 'agentteam') {
return 'agent-team';
}
return null;
};
const normalizeConfigExecutionMode = (mode) => {
if (mode === 'subagent') return 'subagent';
if (mode === 'auto' || mode === 'sequential' || mode === 'subagent' || mode === 'agent-team') {
return mode;
}
return null;
};
// Explicit user instruction in the active run takes priority over config.
const explicitModeFromUser = normalizeUserExecutionMode(runtime.getExplicitExecutionModeHint?.() || null);
const requestedMode = explicitModeFromUser || normalizeConfigExecutionMode(orchestrationContext.config.execution_mode) || 'auto';
const probeEnabled = orchestrationContext.config.capability_probe;
const supports = { subagent: false, agentTeam: false };
if (probeEnabled) {
supports.subagent = runtime.canLaunchSubagents?.() === true;
supports.agentTeam = runtime.canLaunchAgentTeams?.() === true;
}
let resolvedMode = requestedMode;
if (requestedMode === 'auto') {
if (supports.agentTeam) resolvedMode = 'agent-team';
else if (supports.subagent) resolvedMode = 'subagent';
else resolvedMode = 'sequential';
} else if (probeEnabled && requestedMode === 'agent-team' && !supports.agentTeam) {
resolvedMode = supports.subagent ? 'subagent' : 'sequential';
} else if (probeEnabled && requestedMode === 'subagent' && !supports.subagent) {
resolvedMode = 'sequential';
}
```
Resolution precedence:
1. Explicit user request in this run (`agent team` => `agent-team`; `subagent` => `subagent`; `sequential`; `auto`)
2. `tea_execution_mode` from config
3. Runtime capability fallback (when probing enabled)
## 1. Resolve Output Path and Select Template
Determine the pipeline output file path based on the detected `ci_platform`:
| CI Platform | Output Path | Template File |
| ---------------- | ------------------------------------------- | --------------------------------------------------- |
| `github-actions` | `{project-root}/.github/workflows/test.yml` | `{installed_path}/github-actions-template.yaml` |
| `gitlab-ci` | `{project-root}/.gitlab-ci.yml` | `{installed_path}/gitlab-ci-template.yaml` |
| `jenkins` | `{project-root}/Jenkinsfile` | `{installed_path}/jenkins-pipeline-template.groovy` |
| `azure-devops` | `{project-root}/azure-pipelines.yml` | `{installed_path}/azure-pipelines-template.yaml` |
| `harness` | `{project-root}/.harness/pipeline.yaml` | `{installed_path}/harness-pipeline-template.yaml` |
| `circle-ci` | `{project-root}/.circleci/config.yml` | _(no template; generate from first principles)_ |
Use templates from `{installed_path}` when available. Adapt the template to the project's `test_stack_type` and `test_framework`.
---
## Security: Script Injection Prevention
> **CRITICAL:** Treat `${{ inputs.* }}` and the entire `${{ github.event.* }}` namespace as unsafe by default. ALWAYS route them through `env:` intermediaries and reference as double-quoted `"$ENV_VAR"` in `run:` blocks. NEVER interpolate them directly.
When the generated pipeline is extended into reusable workflows (`on: workflow_call`), manual dispatch (`on: workflow_dispatch`), or composite actions, these values become user-controllable and can inject arbitrary shell commands.
**Two rules for generated `run:` blocks:**
1. **No direct interpolation** — pass unsafe contexts through `env:`, reference as `"$ENV_VAR"`
2. **Inputs must be DATA, not COMMANDS** — never accept command-shaped inputs (e.g., `inputs.install-command`) that get executed as shell code. Even through `env:`, running `$CMD` where CMD comes from an input is still command injection. Use fixed commands and pass inputs only as arguments.
```yaml
# ✅ SAFE — input is DATA interpolated into a fixed command
- name: Run tests
env:
TEST_GREP: ${{ inputs.test-grep }}
run: |
# Security: inputs passed through env: to prevent script injection
npx playwright test --grep "$TEST_GREP"
# ❌ NEVER — direct GitHub expression injection
- name: Run tests
run: |
npx playwright test --grep "${{ inputs.test-grep }}"
# ❌ NEVER — executing input-derived env var as a command
- name: Install
env:
CMD: ${{ inputs.install-command }}
run: $CMD
```
Include a `# Security: inputs passed through env: to prevent script injection` comment in generated YAML wherever this pattern is applied.
**Safe contexts** (do NOT need `env:` intermediaries): `${{ steps.*.outputs.* }}`, `${{ matrix.* }}`, `${{ runner.os }}`, `${{ github.sha }}`, `${{ github.ref }}`, `${{ secrets.* }}`, `${{ env.* }}`.
---
## 2. Pipeline Stages
Include stages:
- lint
- test (parallel shards)
- contract-test (if `tea_use_pactjs_utils` enabled)
- burn-in (flaky detection)
- report (aggregate + publish)
---
## 3. Test Execution
- Parallel sharding enabled
- CI retries configured
- Capture artifacts (HTML report, JUnit XML, traces/videos on failure)
- Cache dependencies (language-appropriate: node_modules, .venv, .m2, go module cache, NuGet, bundler)
Write the selected pipeline configuration to the resolved output path from step 1. Adjust test commands based on `test_stack_type` and `test_framework`:
- **Frontend/Fullstack**: Include browser install, E2E/component test commands, Playwright/Cypress artifacts
- **Backend (Node.js)**: Use `npm test` or framework-specific commands (`vitest`, `jest`), skip browser install
- **Backend (Python)**: Use `pytest` with coverage (`pytest --cov`), install via `pip install -r requirements.txt` or `poetry install`
- **Backend (Java/Kotlin)**: Use `mvn test` or `gradle test`, cache `.m2/repository` or `.gradle/caches`
- **Backend (Go)**: Use `go test ./...` with coverage (`-coverprofile`), cache Go modules
- **Backend (C#/.NET)**: Use `dotnet test` with coverage, restore NuGet packages
- **Backend (Ruby)**: Use `bundle exec rspec` with coverage, cache `vendor/bundle`
### Contract Testing Pipeline (if `tea_use_pactjs_utils` enabled)
When `tea_use_pactjs_utils` is enabled, add a `contract-test` stage after `test`:
**Required env block** (add to the generated pipeline):
```yaml
env:
PACT_BROKER_BASE_URL: ${{ secrets.PACT_BROKER_BASE_URL }}
PACT_BROKER_TOKEN: ${{ secrets.PACT_BROKER_TOKEN }}
GITHUB_SHA: ${{ github.sha }} # auto-set by GitHub Actions
GITHUB_BRANCH: ${{ github.head_ref || github.ref_name }} # NOT auto-set — must be defined explicitly
```
> **Note:** `GITHUB_SHA` is auto-set by GitHub Actions, but `GITHUB_BRANCH` is **not** — it must be derived from `github.head_ref` (for PRs) or `github.ref_name` (for pushes). The pactjs-utils library reads both from `process.env`.
1. **Consumer test + publish**: Run consumer contract tests, then publish pacts to broker
- `npm run test:pact:consumer`
- `npm run publish:pact`
- Only publish on PR and main branch pushes
2. **Provider verification**: Run provider verification against published pacts
- `npm run test:pact:provider:remote:contract`
- `buildVerifierOptions` auto-reads `PACT_BROKER_BASE_URL`, `PACT_BROKER_TOKEN`, `GITHUB_SHA`, `GITHUB_BRANCH`
- Verification results published to broker when `CI=true`
3. **Can-I-Deploy gate**: Block deployment if contracts are incompatible
- `npm run can:i:deploy:provider`
- Ensure the script adds `--retry-while-unknown 6 --retry-interval 10` for async verification
4. **Webhook job**: Add `repository_dispatch` trigger for `pact_changed` event
- Provider verification runs when consumers publish new pacts
- Ensures compatibility is checked on both consumer and provider changes
5. **Breaking change handling**: When `PACT_BREAKING_CHANGE=true` env var is set:
- Provider test passes `includeMainAndDeployed: false` to `buildVerifierOptions` — verifies only matching branch
- Coordinate with consumer team before removing the flag
6. **Record deployment**: After successful deployment, record version in broker
- `npm run record:provider:deployment --env=production`
Required CI secrets: `PACT_BROKER_BASE_URL`, `PACT_BROKER_TOKEN`
**If `tea_pact_mcp` is `"mcp"`:** Reference the SmartBear MCP `Can I Deploy` and `Matrix` tools for pipeline guidance in `pact-mcp.md`.
---
### 4. Save Progress
**Save this step's accumulated work to `{outputFile}`.**
- **If `{outputFile}` does not exist** (first save), create it with YAML frontmatter:
```yaml
---
stepsCompleted: ['step-02-generate-pipeline']
lastStep: 'step-02-generate-pipeline'
lastSaved: '{date}'
---
```
Then write this step's output below the frontmatter.
- **If `{outputFile}` already exists**, update:
- Add `'step-02-generate-pipeline'` to `stepsCompleted` array (only if not already present)
- Set `lastStep: 'step-02-generate-pipeline'`
- Set `lastSaved: '{date}'`
- Append this step's output to the appropriate section of the document.
### 5. Orchestration Notes for This Step
For this step, treat these work units as parallelizable when `resolvedMode` is `agent-team` or `subagent`:
- Worker A: resolve platform path/template and produce base pipeline skeleton (section 1)
- Worker B: construct stage definitions and test execution blocks (sections 2-3)
- Worker C: contract-testing block (only when `tea_use_pactjs_utils` is true)
If `resolvedMode` is `sequential`, execute sections 1→4 in order.
Load next step: `{nextStepFile}`
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Step completed in full with required outputs
### ❌ SYSTEM FAILURE:
- Skipped sequence steps or missing outputs
**Master Rule:** Skipping steps is FORBIDDEN.

View File

@@ -0,0 +1,135 @@
---
name: 'step-03-configure-quality-gates'
description: 'Configure burn-in, quality gates, and notifications'
nextStepFile: './step-04-validate-and-summary.md'
knowledgeIndex: '{project-root}/_bmad/tea/testarch/tea-index.csv'
outputFile: '{test_artifacts}/ci-pipeline-progress.md'
---
# Step 3: Quality Gates & Notifications
## STEP GOAL
Configure burn-in loops, quality thresholds, and notification hooks.
## MANDATORY EXECUTION RULES
- 📖 Read the entire step file before acting
- ✅ Speak in `{communication_language}`
---
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
- 💾 Record outputs before proceeding
- 📖 Load the next step only when instructed
## CONTEXT BOUNDARIES:
- Available context: config, loaded artifacts, and knowledge fragments
- Focus: this step's goal only
- Limits: do not execute future steps
- Dependencies: prior steps' outputs (if any)
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
## 1. Burn-In Configuration
Use `{knowledgeIndex}` to load `ci-burn-in.md` guidance:
- Run N-iteration burn-in for flaky detection
- Gate promotion based on burn-in stability
**Stack-conditional burn-in:**
- **Frontend or Fullstack** (`test_stack_type` is `frontend` or `fullstack`): Enable burn-in by default. Burn-in targets UI flakiness (race conditions, selector instability, timing issues).
- **Backend only** (`test_stack_type` is `backend`): Skip burn-in by default. Backend tests (unit, integration, API) are deterministic and rarely exhibit UI-related flakiness. If the user explicitly requests burn-in for backend, honor that override.
**Security: Script injection prevention for reusable burn-in workflows:**
When burn-in is extracted into a reusable workflow (`on: workflow_call`), all `${{ inputs.* }}` values MUST be passed through `env:` intermediaries and referenced as quoted `"$ENV_VAR"`. Never interpolate them directly.
**Inputs must be DATA, not COMMANDS.** Do not accept command-shaped inputs (e.g., `inputs.install-command`, `inputs.test-command`) that get executed as shell code — even through `env:`, running `$CMD` is still command injection. Use fixed commands (e.g., `npm ci`, `npx playwright test`) and pass inputs only as data arguments.
```yaml
# ✅ SAFE — fixed commands with data-only inputs
- name: Install dependencies
run: npm ci
- name: Run burn-in loop
env:
TEST_GREP: ${{ inputs.test-grep }}
BURN_IN_COUNT: ${{ inputs.burn-in-count }}
BASE_REF: ${{ inputs.base-ref }}
run: |
# Security: inputs passed through env: to prevent script injection
for i in $(seq 1 "$BURN_IN_COUNT"); do
echo "Burn-in iteration $i/$BURN_IN_COUNT"
npx playwright test --grep "$TEST_GREP" || exit 1
done
```
---
## 2. Quality Gates
Define:
- Minimum pass rates (P0 = 100%, P1 ≥ 95%)
- Fail CI on critical test failures
- Optional: require traceability or nfr-assess output before release
**Contract testing gate** (if `tea_use_pactjs_utils` is enabled):
- **can-i-deploy must pass** before any deployment to staging or production
- Block the deployment pipeline if contract verification fails
- Treat consumer pact publishing failures as CI failures (contracts must stay up-to-date)
- Provider verification must pass for all consumer pacts before merge
---
## 3. Notifications
Configure:
- Failure notifications (Slack/email)
- Artifact links
---
### 4. Save Progress
**Save this step's accumulated work to `{outputFile}`.**
- **If `{outputFile}` does not exist** (first save), create it with YAML frontmatter:
```yaml
---
stepsCompleted: ['step-03-configure-quality-gates']
lastStep: 'step-03-configure-quality-gates'
lastSaved: '{date}'
---
```
Then write this step's output below the frontmatter.
- **If `{outputFile}` already exists**, update:
- Add `'step-03-configure-quality-gates'` to `stepsCompleted` array (only if not already present)
- Set `lastStep: 'step-03-configure-quality-gates'`
- Set `lastSaved: '{date}'`
- Append this step's output to the appropriate section of the document.
Load next step: `{nextStepFile}`
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Step completed in full with required outputs
### ❌ SYSTEM FAILURE:
- Skipped sequence steps or missing outputs
**Master Rule:** Skipping steps is FORBIDDEN.

View File

@@ -0,0 +1,92 @@
---
name: 'step-04-validate-and-summary'
description: 'Validate pipeline and summarize'
outputFile: '{test_artifacts}/ci-pipeline-progress.md'
---
# Step 4: Validate & Summarize
## STEP GOAL
Validate CI configuration and report completion details.
## MANDATORY EXECUTION RULES
- 📖 Read the entire step file before acting
- ✅ Speak in `{communication_language}`
---
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
- 💾 Record outputs before proceeding
- 📖 Load the next step only when instructed
## CONTEXT BOUNDARIES:
- Available context: config, loaded artifacts, and knowledge fragments
- Focus: this step's goal only
- Limits: do not execute future steps
- Dependencies: prior steps' outputs (if any)
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
## 1. Validation
Validate against `checklist.md`:
- Config file created
- Stages and sharding configured
- Burn-in and artifacts enabled
- Secrets/variables documented
Fix gaps before completion.
---
## 2. Completion Summary
Report:
- CI platform and config path
- Key stages enabled
- Artifacts and notifications
- Next steps (set secrets, run pipeline)
---
### 3. Save Progress
**Save this step's accumulated work to `{outputFile}`.**
- **If `{outputFile}` does not exist** (first save), create it with YAML frontmatter:
```yaml
---
stepsCompleted: ['step-04-validate-and-summary']
lastStep: 'step-04-validate-and-summary'
lastSaved: '{date}'
---
```
Then write this step's output below the frontmatter.
- **If `{outputFile}` already exists**, update:
- Add `'step-04-validate-and-summary'` to `stepsCompleted` array (only if not already present)
- Set `lastStep: 'step-04-validate-and-summary'`
- Set `lastSaved: '{date}'`
- Append this step's output to the appropriate section of the document.
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Step completed in full with required outputs
### ❌ SYSTEM FAILURE:
- Skipped sequence steps or missing outputs
**Master Rule:** Skipping steps is FORBIDDEN.

View File

@@ -0,0 +1,65 @@
---
name: 'step-01-assess'
description: 'Load an existing output for editing'
nextStepFile: './step-02-apply-edit.md'
---
# Step 1: Assess Edit Target
## STEP GOAL:
Identify which output should be edited and load it.
## MANDATORY EXECUTION RULES (READ FIRST):
### Universal Rules:
- 📖 Read the complete step file before taking any action
- ✅ Speak in `{communication_language}`
### Role Reinforcement:
- ✅ You are the Master Test Architect
### Step-Specific Rules:
- 🎯 Ask the user which output file to edit
- 🚫 Do not edit until target is confirmed
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
## CONTEXT BOUNDARIES:
- Available context: existing outputs
- Focus: select edit target
- Limits: no edits yet
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly.
### 1. Identify Target
Ask the user to provide the output file path or select from known outputs.
### 2. Load Target
Read the provided output file in full.
### 3. Confirm
Confirm the target and proceed to edit.
Load next step: `{nextStepFile}`
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Target identified and loaded
### ❌ SYSTEM FAILURE:
- Proceeding without a confirmed target

View File

@@ -0,0 +1,60 @@
---
name: 'step-02-apply-edit'
description: 'Apply edits to the selected output'
---
# Step 2: Apply Edits
## STEP GOAL:
Apply the requested edits to the selected output and confirm changes.
## MANDATORY EXECUTION RULES (READ FIRST):
### Universal Rules:
- 📖 Read the complete step file before taking any action
- ✅ Speak in `{communication_language}`
### Role Reinforcement:
- ✅ You are the Master Test Architect
### Step-Specific Rules:
- 🎯 Only apply edits explicitly requested by the user
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
## CONTEXT BOUNDARIES:
- Available context: selected output and user changes
- Focus: apply edits only
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly.
### 1. Confirm Requested Changes
Restate what will be changed and confirm.
### 2. Apply Changes
Update the output file accordingly.
### 3. Report
Summarize the edits applied.
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Changes applied and confirmed
### ❌ SYSTEM FAILURE:
- Unconfirmed edits or missing update

View File

@@ -0,0 +1,81 @@
---
name: 'step-01-validate'
description: 'Validate workflow outputs against checklist'
outputFile: '{test_artifacts}/ci-validation-report.md'
validationChecklist: '../checklist.md'
---
# Step 1: Validate Outputs
## STEP GOAL:
Validate outputs using the workflow checklist and record findings.
## MANDATORY EXECUTION RULES (READ FIRST):
### Universal Rules:
- 📖 Read the complete step file before taking any action
- ✅ Speak in `{communication_language}`
### Role Reinforcement:
- ✅ You are the Master Test Architect
### Step-Specific Rules:
- 🎯 Validate against `{validationChecklist}`
- 🚫 Do not skip checks
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
- 💾 Write findings to `{outputFile}`
## CONTEXT BOUNDARIES:
- Available context: workflow outputs and checklist
- Focus: validation only
- Limits: do not modify outputs in this step
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly.
### 1. Load Checklist
Read `{validationChecklist}` and list all criteria.
### 2. Validate Outputs
Evaluate outputs against each checklist item.
### 2a. Script Injection Scan
Scan all generated YAML workflow files for unsafe interpolation patterns inside `run:` blocks.
**Unsafe patterns to flag (FAIL):**
- `${{ inputs.* }}` — all workflow inputs are user-controllable
- `${{ github.event.* }}` — treat the entire event namespace as unsafe by default (includes PR titles, issue bodies, comment bodies, label names, etc.)
- `${{ github.head_ref }}` — PR source branch name (user-controlled)
**Detection method:** For each `run:` block in generated YAML, check if any of the above expressions appears in the run script body. If found, flag as **FAIL** with the exact line and recommend converting to the safe `env:` intermediary pattern (pass through `env:`, reference as double-quoted `"$ENV_VAR"`).
**Safe patterns to ignore** (exempt from flagging): `${{ steps.*.outputs.* }}`, `${{ matrix.* }}`, `${{ runner.os }}`, `${{ github.sha }}`, `${{ github.ref }}`, `${{ secrets.* }}`, `${{ env.* }}` — these are safe from GitHub expression injection when used in `run:` blocks.
### 3. Write Report
Write a validation report to `{outputFile}` with PASS/WARN/FAIL per section.
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Validation report written
- All checklist items evaluated
### ❌ SYSTEM FAILURE:
- Skipped checklist items
- No report produced

View File

@@ -0,0 +1,72 @@
---
validationDate: 2026-01-27
workflowName: testarch-ci
workflowPath: {project-root}/src/workflows/testarch/ci
validationStatus: COMPLETE
completionDate: 2026-01-27 10:03:10
---
# Validation Report: testarch-ci
**Validation Started:** 2026-01-27 09:50:21
**Validator:** BMAD Workflow Validation System (Codex)
**Standards Version:** BMAD Workflow Standards
## File Structure & Size
- workflow.md present: YES
- instructions.md present: YES
- workflow.yaml present: YES
- step files found: 7
**Step File Sizes:**
- steps-c/step-01-preflight.md: 87 lines [GOOD]
- steps-c/step-02-generate-pipeline.md: 75 lines [GOOD]
- steps-c/step-03-configure-quality-gates.md: 67 lines [GOOD]
- steps-c/step-04-validate-and-summary.md: 60 lines [GOOD]
- steps-e/step-01-assess.md: 51 lines [GOOD]
- steps-e/step-02-apply-edit.md: 46 lines [GOOD]
- steps-v/step-01-validate.md: 53 lines [GOOD]
- workflow-plan.md present: YES
## Frontmatter Validation
- No frontmatter violations found
## Critical Path Violations
- No {project-root} hardcoded paths detected in body
- No dead relative links detected
## Menu Handling Validation
- No menu structures detected (linear step flow) [N/A]
## Step Type Validation
- Last step steps-v/step-01-validate.md has no nextStepFile (final step OK)
- Step type validation assumes linear sequence (no branching/menu). Workflow-plan.md present for reference. [INFO]
## Output Format Validation
- No templates found in workflow root
- Steps with outputFile in frontmatter:
- steps-c/step-02-generate-pipeline.md
- steps-v/step-01-validate.md
## Validation Design Check
- checklist.md present: YES
- Validation steps folder (steps-v) present: YES
## Instruction Style Check
- All steps include STEP GOAL, MANDATORY EXECUTION RULES, EXECUTION PROTOCOLS, CONTEXT BOUNDARIES, and SUCCESS/FAILURE metrics
## Summary
- Validation completed: 2026-01-27 10:03:10
- Critical issues: 0
- Warnings: 0 (informational notes only)
- Readiness: READY (manual review optional)

View File

@@ -0,0 +1,114 @@
---
validationDate: 2026-01-27
workflowName: testarch-ci
workflowPath: {project-root}/src/workflows/testarch/ci
validationStatus: COMPLETE
completionDate: 2026-01-27 10:24:01
---
# Validation Report: testarch-ci
**Validation Started:** 2026-01-27 10:24:01
**Validator:** BMAD Workflow Validation System (Codex)
**Standards Version:** BMAD Workflow Standards
## File Structure & Size
- workflow.md present: YES
- instructions.md present: YES
- workflow.yaml present: YES
- step files found: 7
**Step File Sizes:**
- steps-c/step-01-preflight.md: 86 lines [GOOD]
- steps-c/step-02-generate-pipeline.md: 74 lines [GOOD]
- steps-c/step-03-configure-quality-gates.md: 66 lines [GOOD]
- steps-c/step-04-validate-and-summary.md: 59 lines [GOOD]
- steps-e/step-01-assess.md: 50 lines [GOOD]
- steps-e/step-02-apply-edit.md: 45 lines [GOOD]
- steps-v/step-01-validate.md: 52 lines [GOOD]
- workflow-plan.md present: YES
## Frontmatter Validation
- No frontmatter violations found
## Critical Path Violations
### Config Variables (Exceptions)
Standard BMAD config variables treated as valid exceptions: bmb_creations_output_folder, communication_language, document_output_language, output_folder, planning_artifacts, project-root, project_name, test_artifacts, user_name
- No {project-root} hardcoded paths detected in body
- No dead relative links detected
- No module path assumptions detected
**Status:** ✅ PASS - No critical violations
## Menu Handling Validation
- No menu structures detected (linear step flow) [N/A]
## Step Type Validation
- steps-c/step-01-preflight.md: Init [PASS]
- steps-c/step-02-generate-pipeline.md: Middle [PASS]
- steps-c/step-03-configure-quality-gates.md: Middle [PASS]
- steps-c/step-04-validate-and-summary.md: Final [PASS]
- Step type validation assumes linear sequence (no branching/menu). Workflow-plan.md present for reference. [INFO]
## Output Format Validation
- Templates present: NONE
- Steps with outputFile in frontmatter:
- steps-c/step-02-generate-pipeline.md
- steps-v/step-01-validate.md
- checklist.md present: YES
## Validation Design Check
- Validation steps folder (steps-v) present: YES
- Validation step(s) present: step-01-validate.md
- Validation steps reference checklist data and auto-proceed
## Instruction Style Check
- Instruction style: Prescriptive (appropriate for TEA quality/compliance workflows)
- Steps emphasize mandatory sequence, explicit success/failure metrics, and risk-based guidance
## Collaborative Experience Check
- Overall facilitation quality: GOOD
- Steps use progressive prompts and clear role reinforcement; no laundry-list interrogation detected
- Flow progression is clear and aligned to workflow goals
## Subagent Optimization Opportunities
- No high-priority subagent optimizations identified; workflow already uses step-file architecture
- Pattern 1 (grep/regex): N/A for most steps
- Pattern 2 (per-file analysis): already aligned to validation structure
- Pattern 3 (data ops): minimal data file loads
- Pattern 4 (parallel): optional for validation only
## Cohesive Review
- Overall assessment: GOOD
- Flow is linear, goals are clear, and outputs map to TEA artifacts
- Voice and tone consistent with Test Architect persona
- Recommendation: READY (minor refinements optional)
## Plan Quality Validation
- Plan file present: workflow-plan.md
- Planned steps found: 7 (all implemented)
- Plan implementation status: Fully Implemented
## Summary
- Validation completed: 2026-01-27 10:24:01
- Critical issues: 0
- Warnings: 0 (informational notes only)
- Readiness: READY (manual review optional)

View File

@@ -0,0 +1,20 @@
# Workflow Plan: testarch-ci
## Create Mode (steps-c)
- step-01-preflight.md
- step-02-generate-pipeline.md
- step-03-configure-quality-gates.md
- step-04-validate-and-summary.md
## Validate Mode (steps-v)
- step-01-validate.md
## Edit Mode (steps-e)
- step-01-assess.md
- step-02-apply-edit.md
## Outputs
- CI config (e.g., {project-root}/.github/workflows/test.yml)
- Pipeline guidance and artifacts configuration

View File

@@ -0,0 +1,41 @@
---
name: testarch-ci
description: Scaffold CI/CD quality pipeline with test execution. Use when user says 'lets setup CI pipeline' or 'I want to create quality gates'
web_bundle: true
---
# CI/CD Pipeline Setup
**Goal:** Scaffold CI/CD quality pipeline with test execution, burn-in loops, and artifact collection
**Role:** You are the Master Test Architect.
---
## WORKFLOW ARCHITECTURE
This workflow uses **tri-modal step-file architecture**:
- **Create mode (steps-c/)**: primary execution flow
- **Validate mode (steps-v/)**: validation against checklist
- **Edit mode (steps-e/)**: revise existing outputs
---
## INITIALIZATION SEQUENCE
### 1. Mode Determination
"Welcome to the workflow. What would you like to do?"
- **[C] Create** — Run the workflow
- **[R] Resume** — Resume an interrupted workflow
- **[V] Validate** — Validate existing outputs
- **[E] Edit** — Edit existing outputs
### 2. Route to First Step
- **If C:** Load `steps-c/step-01-preflight.md`
- **If R:** Load `steps-c/step-01b-resume.md`
- **If V:** Load `steps-v/step-01-validate.md`
- **If E:** Load `steps-e/step-01-assess.md`

View File

@@ -0,0 +1,48 @@
# Test Architect workflow: ci
name: testarch-ci
# prettier-ignore
description: 'Scaffold CI/CD quality pipeline with test execution. Use when the user says "lets setup CI pipeline" or "I want to create quality gates"'
# Critical variables from config
config_source: "{project-root}/_bmad/tea/config.yaml"
output_folder: "{config_source}:output_folder"
test_artifacts: "{config_source}:test_artifacts"
user_name: "{config_source}:user_name"
communication_language: "{config_source}:communication_language"
document_output_language: "{config_source}:document_output_language"
date: system-generated
# Workflow components
installed_path: "{project-root}/_bmad/tea/workflows/testarch/ci"
instructions: "{installed_path}/instructions.md"
validation: "{installed_path}/checklist.md"
# Variables and inputs
variables:
ci_platform: "auto" # auto, github-actions, gitlab-ci, circle-ci, jenkins, azure-devops, harness - user can override
test_dir: "{project-root}/tests" # Root test directory
test_stack_type: "auto" # auto, frontend, backend, fullstack - detected or user override
test_framework: "auto" # auto, playwright, cypress, jest, vitest - detected or user override
# Output configuration (resolved dynamically based on ci_platform detection)
default_output_file: "{project-root}/.github/workflows/test.yml" # GitHub Actions default; overridden per platform
# Required tools
required_tools:
- read_file # Read .nvmrc, package.json, framework config
- write_file # Create CI config, scripts, documentation
- create_directory # Create .github/workflows/ or .gitlab-ci/ directories
- list_files # Detect existing CI configuration
- search_repo # Find test files for selective testing
tags:
- qa
- ci-cd
- test-architect
- pipeline
- automation
execution_hints:
interactive: false # Minimize prompts, auto-detect when possible
autonomous: true # Proceed without user input unless blocked
iterative: true

View File

@@ -0,0 +1,345 @@
# Test Framework Setup - Validation Checklist
This checklist ensures the framework workflow completes successfully and all deliverables meet quality standards.
---
## Prerequisites
Before starting the workflow:
- [ ] Project root contains a valid project manifest (`package.json`, `pyproject.toml`, `pom.xml`, `build.gradle`, `go.mod`, `*.csproj`, `Gemfile`, or `Cargo.toml`)
- [ ] No existing test framework detected that conflicts with the target setup
- [ ] Project type identifiable (React, Vue, Angular, Next.js, Node, Python, Java, Go, .NET, Ruby, Rust, etc.)
- [ ] Bundler identifiable (Vite, Webpack, Rollup, esbuild) or not applicable (backend projects)
- [ ] User has write permissions to create directories and files
---
## Process Steps
### Step 1: Preflight Checks
- [ ] Stack type detected (`frontend`, `backend`, or `fullstack`)
- [ ] Project manifest successfully read and parsed (`package.json`, `pyproject.toml`, `pom.xml`, `go.mod`, etc.)
- [ ] Project type extracted correctly
- [ ] Bundler identified (or marked as N/A for backend projects)
- [ ] No framework conflicts detected
- [ ] Architecture documents located (if available)
### Step 2: Framework Selection
- [ ] Framework auto-detection logic executed
- [ ] Framework choice justified (Playwright vs Cypress for frontend; pytest/JUnit/Go test/xUnit/RSpec for backend)
- [ ] Framework preference respected (if explicitly set via `config.test_framework`)
- [ ] User notified of framework selection and rationale
### Step 3: Directory Structure
- [ ] `tests/` root directory created
- [ ] `tests/e2e/` directory created (or user's preferred structure)
- [ ] `tests/support/` directory created (critical pattern)
- [ ] `tests/support/fixtures/` directory created
- [ ] `tests/support/fixtures/factories/` directory created
- [ ] `tests/support/helpers/` directory created
- [ ] `tests/support/page-objects/` directory created (if applicable)
- [ ] All directories have correct permissions
**Note**: Test organization is flexible (e2e/, api/, integration/). The **support/** folder is the key pattern.
### Step 4: Configuration Files
- [ ] Framework config file created (`playwright.config.ts` or `cypress.config.ts`)
- [ ] Config file uses TypeScript (if `use_typescript: true`)
- [ ] Timeouts configured correctly (action: 15s, navigation: 30s, test: 60s)
- [ ] Base URL configured with environment variable fallback
- [ ] Trace/screenshot/video set to retain-on-failure
- [ ] Multiple reporters configured (HTML + JUnit + console)
- [ ] Parallel execution enabled
- [ ] CI-specific settings configured (retries, workers)
- [ ] Config file is syntactically valid (no compilation errors)
### Step 5: Environment Configuration
- [ ] `.env.example` created in project root
- [ ] `TEST_ENV` variable defined
- [ ] `BASE_URL` variable defined with default
- [ ] `API_URL` variable defined (if applicable)
- [ ] Authentication variables defined (if applicable)
- [ ] Feature flag variables defined (if applicable)
- [ ] `.nvmrc` created with appropriate Node version
### Step 6: Fixture Architecture
- [ ] `tests/support/fixtures/index.ts` created
- [ ] Base fixture extended from Playwright/Cypress
- [ ] Type definitions for fixtures created
- [ ] mergeTests pattern implemented (if multiple fixtures)
- [ ] Auto-cleanup logic included in fixtures
- [ ] Fixture architecture follows knowledge base patterns
### Step 7: Data Factories
- [ ] At least one factory created (e.g., UserFactory)
- [ ] Factories use @faker-js/faker for realistic data
- [ ] Factories track created entities (for cleanup)
- [ ] Factories implement `cleanup()` method
- [ ] Factories integrate with fixtures
- [ ] Factories follow knowledge base patterns
### Step 8: Sample Tests
- [ ] Example test file created (`tests/e2e/example.spec.ts`)
- [ ] Test uses fixture architecture
- [ ] Test demonstrates data factory usage
- [ ] Test uses proper selector strategy (data-testid)
- [ ] Test follows Given-When-Then structure
- [ ] Test includes proper assertions
- [ ] Network interception demonstrated (if applicable)
### Step 9: Helper Utilities
- [ ] API helper created (if API testing needed)
- [ ] Network helper created (if network mocking needed)
- [ ] Auth helper created (if authentication needed)
- [ ] Helpers follow functional patterns
- [ ] Helpers have proper error handling
### Step 10: Documentation
- [ ] `tests/README.md` created
- [ ] Setup instructions included
- [ ] Running tests section included
- [ ] Architecture overview section included
- [ ] Best practices section included
- [ ] CI integration section included
- [ ] Knowledge base references included
- [ ] Troubleshooting section included
### Step 11: Build & Test Script Updates
- [ ] Minimal test script added to appropriate config (`package.json` for frontend, `Makefile`/`pyproject.toml`/`build.gradle` for backend)
- [ ] Test framework dependency added (if not already present)
- [ ] Type definitions added (if TypeScript)
- [ ] Users can extend with additional scripts as needed
---
## Output Validation
### Configuration Validation
- [ ] Config file loads without errors
- [ ] Config file passes linting (if linter configured)
- [ ] Config file uses correct syntax for chosen framework
- [ ] All paths in config resolve correctly
- [ ] Reporter output directories exist or are created on test run
### Test Execution Validation
- [ ] Sample test runs successfully
- [ ] Test execution produces expected output (pass/fail)
- [ ] Test artifacts generated correctly (traces, screenshots, videos)
- [ ] Test report generated successfully
- [ ] No console errors or warnings during test run
### Directory Structure Validation
- [ ] All required directories exist
- [ ] Directory structure matches framework conventions
- [ ] No duplicate or conflicting directories
- [ ] Directories accessible with correct permissions
### File Integrity Validation
- [ ] All generated files are syntactically correct
- [ ] No placeholder text left in files (e.g., "TODO", "FIXME")
- [ ] All imports resolve correctly
- [ ] No hardcoded credentials or secrets in files
- [ ] All file paths use correct separators for OS
---
## Quality Checks
### Code Quality
- [ ] Generated code follows project coding standards
- [ ] TypeScript types are complete and accurate (no `any` unless necessary)
- [ ] No unused imports or variables
- [ ] Consistent code formatting (matches project style)
- [ ] No linting errors in generated files
### Best Practices Compliance
- [ ] Fixture architecture follows pure function → fixture → mergeTests pattern
- [ ] Data factories implement auto-cleanup
- [ ] Network interception occurs before navigation
- [ ] Selectors use data-testid strategy
- [ ] Artifacts only captured on failure
- [ ] Tests follow Given-When-Then structure
- [ ] No hard-coded waits or sleeps
### Knowledge Base Alignment
- [ ] Fixture pattern matches `fixture-architecture.md`
- [ ] Data factories match `data-factories.md`
- [ ] Network handling matches `network-first.md`
- [ ] Config follows `playwright-config.md` or `test-config.md`
- [ ] Test quality matches `test-quality.md`
### Pact Consumer CDC Alignment (when `tea_use_pactjs_utils` enabled)
- [ ] `vitest.config.pact.ts` is minimal (no pool/coverage/setup copied from unit config)
- [ ] Script names match pactjs-utils (`test:pact:consumer`, `publish:pact`, `can:i:deploy:consumer`, `record:consumer:deployment`)
- [ ] Scripts source `env-setup.sh` inline in package.json
- [ ] Shell scripts use `pact-broker` not `npx pact-broker`
- [ ] Shell scripts use `PACTICIPANT` env var pattern (not hardcoded service names)
- [ ] `can-i-deploy.sh` has `--retry-while-unknown=10 --retry-interval=30`
- [ ] `record-deployment.sh` has branch guard (only records on main/master)
- [ ] `env-setup.sh` uses `set -eu`; broker scripts use `set -euo pipefail` — each with explanatory comment
- [ ] CI workflow named `contract-test-consumer.yml`
- [ ] CI has workflow-level env block (not per-step)
- [ ] CI has `detect-breaking-change` step before install
- [ ] CI step numbering skips (3) — webhook-triggered provider verification
- [ ] CI can-i-deploy has `PACT_BREAKING_CHANGE != 'true'` condition
- [ ] CI has NO upload-artifact step (broker is source of truth)
- [ ] `.github/actions/detect-breaking-change/action.yml` exists
- [ ] Consumer tests use `.pacttest.ts` extension
- [ ] Consumer tests use PactV4 `addInteraction()` builder (not PactV3 fluent API)
- [ ] Consumer tests call REAL consumer code (actual API client functions), NOT raw `fetch()`
- [ ] Consumer code exposes URL injection mechanism (`setApiUrl()`, env var, or constructor param)
- [ ] Local consumer-helpers shim present if `@seontechnologies/pactjs-utils` not installed
- [ ] `.gitignore` includes `/pacts/` and `pact-logs/`
### Security Checks
- [ ] No credentials in configuration files
- [ ] .env.example contains placeholders, not real values
- [ ] Sensitive test data handled securely
- [ ] API keys and tokens use environment variables
- [ ] No secrets committed to version control
---
## Integration Points
### Status File Integration
- [ ] Framework initialization logged in Quality & Testing Progress section
- [ ] Status file updated with completion timestamp
- [ ] Status file shows framework: Playwright or Cypress
### Knowledge Base Integration
- [ ] Relevant knowledge fragments identified from tea-index.csv
- [ ] Knowledge fragments successfully loaded
- [ ] Patterns from knowledge base applied correctly
- [ ] Knowledge base references included in documentation
### Workflow Dependencies
- [ ] Can proceed to `ci` workflow after completion
- [ ] Can proceed to `test-design` workflow after completion
- [ ] Can proceed to `atdd` workflow after completion
- [ ] Framework setup compatible with downstream workflows
---
## Completion Criteria
**All of the following must be true:**
- [ ] All prerequisite checks passed
- [ ] All process steps completed without errors
- [ ] All output validations passed
- [ ] All quality checks passed
- [ ] All integration points verified
- [ ] Sample test executes successfully
- [ ] User can run the appropriate test command without errors (`npm run test:e2e`, `pytest`, `go test ./...`, etc.)
- [ ] Documentation is complete and accurate
- [ ] No critical issues or blockers identified
---
## Post-Workflow Actions
**User must complete:**
1. [ ] Copy `.env.example` to `.env`
2. [ ] Fill in environment-specific values in `.env`
3. [ ] Run `npm install` to install test dependencies
4. [ ] Run `npm run test:e2e` to verify setup
5. [ ] Review `tests/README.md` for project-specific guidance
**Recommended next workflows:**
1. [ ] Run `ci` workflow to set up CI/CD pipeline
2. [ ] Run `test-design` workflow to plan test coverage
3. [ ] Run `atdd` workflow when ready to develop stories
---
## Rollback Procedure
If workflow fails and needs to be rolled back:
1. [ ] Delete `tests/` directory
2. [ ] Remove test scripts from package.json
3. [ ] Delete `.env.example` (if created)
4. [ ] Delete `.nvmrc` (if created)
5. [ ] Delete framework config file
6. [ ] Remove test dependencies from package.json (if added)
7. [ ] Run `npm install` to clean up node_modules
---
## Notes
### Common Issues
**Issue**: Config file has TypeScript errors
- **Solution**: Ensure `@playwright/test` or `cypress` types are installed
**Issue**: Sample test fails to run
- **Solution**: Check BASE_URL in .env, ensure app is running
**Issue**: Fixture cleanup not working
- **Solution**: Verify cleanup() is called in fixture teardown
**Issue**: Network interception not working
- **Solution**: Ensure route setup occurs before page.goto()
### Framework-Specific Considerations
**Playwright:**
- Requires Node.js 18+
- Browser binaries auto-installed on first run
- Trace viewer requires running `npx playwright show-trace`
**Cypress:**
- Requires Node.js 18+
- Cypress app opens on first run
- Component testing requires additional setup
### Version Compatibility
- [ ] Node.js version matches .nvmrc
- [ ] Framework version compatible with Node.js version
- [ ] TypeScript version compatible with framework
- [ ] All peer dependencies satisfied
---
**Checklist Complete**: Sign off when all items checked and validated.
**Completed by:** {name}
**Date:** {date}
**Framework:** { Playwright / Cypress or something else}
**Notes:** {notes}

View File

@@ -0,0 +1,45 @@
<!-- Powered by BMAD-CORE™ -->
# Test Framework Setup
**Workflow ID**: `_bmad/tea/testarch/framework`
**Version**: 5.0 (Step-File Architecture)
---
## Overview
Initialize a production-ready test framework (Playwright or Cypress) with fixtures, helpers, configuration, and best practices.
---
## WORKFLOW ARCHITECTURE
This workflow uses **step-file architecture**:
- **Micro-file Design**: Each step is self-contained
- **JIT Loading**: Only the current step file is in memory
- **Sequential Enforcement**: Execute steps in order without skipping
---
## INITIALIZATION SEQUENCE
### 1. Configuration Loading
From `workflow.yaml`, resolve:
- `config_source`, `test_artifacts`, `user_name`, `communication_language`, `document_output_language`, `date`
- `test_dir`, `use_typescript`, `framework_preference`, `project_size`
### 2. First Step
Load, read completely, and execute:
`{project-root}/_bmad/tea/workflows/testarch/framework/steps-c/step-01-preflight.md`
### 3. Resume Support
If the user selects **Resume** mode, load, read completely, and execute:
`{project-root}/_bmad/tea/workflows/testarch/framework/steps-c/step-01b-resume.md`
This checks the output document for progress tracking frontmatter and routes to the next incomplete step.

View File

@@ -0,0 +1,132 @@
---
name: 'step-01-preflight'
description: 'Verify prerequisites and gather project context'
nextStepFile: './step-02-select-framework.md'
outputFile: '{test_artifacts}/framework-setup-progress.md'
---
# Step 1: Preflight Checks
## STEP GOAL
Verify the project is ready for framework scaffolding and gather key context.
## MANDATORY EXECUTION RULES
- 📖 Read the entire step file before acting
- ✅ Speak in `{communication_language}`
- 🚫 Halt if preflight requirements fail
---
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
- 💾 Record outputs before proceeding
- 📖 Load the next step only when instructed
## CONTEXT BOUNDARIES:
- Available context: config, loaded artifacts, and knowledge fragments
- Focus: this step's goal only
- Limits: do not execute future steps
- Dependencies: prior steps' outputs (if any)
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
## 1. Stack Detection
**Read `config.test_stack_type`** from `{config_source}`.
**Auto-Detection Algorithm** (when `test_stack_type` is `"auto"` or not configured):
- Scan `{project-root}` for project manifests:
- **Frontend indicators**: `package.json` with react/vue/angular/next dependencies, `playwright.config.*`, `vite.config.*`, `webpack.config.*`
- **Backend indicators**: `pyproject.toml`, `pom.xml`/`build.gradle`, `go.mod`, `*.csproj`/`*.sln`, `Gemfile`, `Cargo.toml`
- **Both present** = `fullstack`; only frontend = `frontend`; only backend = `backend`
- Explicit `test_stack_type` config value overrides auto-detection
- **Backward compatibility**: if `test_stack_type` is not in config, treat as `"auto"` (preserves current frontend behavior for existing installs)
Store result as `{detected_stack}` = `frontend` | `backend` | `fullstack`
---
## 2. Validate Prerequisites
**If {detected_stack} is `frontend` or `fullstack`:**
- `package.json` exists in project root
- No existing E2E framework (`playwright.config.*`, `cypress.config.*`, `cypress.json`)
**If {detected_stack} is `backend` or `fullstack`:**
- At least one backend project manifest exists (`pyproject.toml`, `pom.xml`, `build.gradle`, `go.mod`, `*.csproj`, `Gemfile`, `Cargo.toml`)
- No existing test framework config that conflicts (e.g., `conftest.py` with full pytest suite, `src/test/` with JUnit suite)
- Architecture/stack context available (project type, bundler, dependencies)
If any fail, **HALT** and report the missing requirement.
---
## 3. Gather Project Context
**If {detected_stack} is `frontend` or `fullstack`:**
- Read `package.json` to identify framework, bundler, dependencies
**If {detected_stack} is `backend` or `fullstack`:**
- Read the relevant project manifest (`pyproject.toml`, `pom.xml`, `go.mod`, `*.csproj`, `Gemfile`, `Cargo.toml`) to identify language, framework, and dependencies
- Check for architecture docs (`architecture.md`, `tech-spec*.md`) if available
- Note auth requirements and APIs (if documented)
---
## 3. Confirm Findings
Summarize:
- Project type and bundler
- Whether a framework is already installed
- Any relevant context docs found
---
### 4. Save Progress
**Save this step's accumulated work to `{outputFile}`.**
- **If `{outputFile}` does not exist** (first save), create it with YAML frontmatter:
```yaml
---
stepsCompleted: ['step-01-preflight']
lastStep: 'step-01-preflight'
lastSaved: '{date}'
---
```
Then write this step's output below the frontmatter.
- **If `{outputFile}` already exists**, update:
- Add `'step-01-preflight'` to `stepsCompleted` array (only if not already present)
- Set `lastStep: 'step-01-preflight'`
- Set `lastSaved: '{date}'`
- Append this step's output to the appropriate section of the document.
Load next step: `{nextStepFile}`
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Step completed in full with required outputs
### ❌ SYSTEM FAILURE:
- Skipped sequence steps or missing outputs
**Master Rule:** Skipping steps is FORBIDDEN.

View File

@@ -0,0 +1,116 @@
---
name: 'step-01b-resume'
description: 'Resume interrupted workflow from last completed step'
outputFile: '{test_artifacts}/framework-setup-progress.md'
---
# Step 1b: Resume Workflow
## STEP GOAL
Resume an interrupted workflow by loading the existing progress document, verifying previously created artifacts still exist on disk, displaying progress, and routing to the next incomplete step.
## MANDATORY EXECUTION RULES
- 📖 Read the entire step file before acting
- ✅ Speak in `{communication_language}`
---
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
- 📖 Load the next step only when instructed
## CONTEXT BOUNDARIES:
- Available context: Output document with progress frontmatter
- Focus: Load progress and route to next step
- Limits: Do not re-execute completed steps
- Dependencies: Output document must exist from a previous run
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
### 1. Load Output Document
Read `{outputFile}` and parse YAML frontmatter for:
- `stepsCompleted` — array of completed step names
- `lastStep` — last completed step name
- `lastSaved` — timestamp of last save
**If `{outputFile}` does not exist**, display:
"⚠️ **No previous progress found.** There is no output document to resume from. Please use **[C] Create** to start a fresh workflow run."
**THEN:** Halt. Do not proceed.
---
### 2. Verify Previously Created Artifacts
Since this workflow creates code files, verify that artifacts from completed steps still exist on disk:
- If `step-01-preflight` completed: Confirm `package.json` still exists
- If `step-03-scaffold-framework` completed: Confirm directory structure and config files exist
- If `step-04-docs-and-scripts` completed: Confirm `{test_dir}/README.md` exists
If any expected artifacts are missing, warn the user and suggest re-running from the step that created them.
---
### 3. Display Progress Dashboard
Display:
"📋 **Workflow Resume — Test Framework Setup**
**Last saved:** {lastSaved}
**Steps completed:** {stepsCompleted.length} of 5
1. ✅/⬜ Preflight Checks (step-01-preflight)
2. ✅/⬜ Select Framework (step-02-select-framework)
3. ✅/⬜ Scaffold Framework (step-03-scaffold-framework)
4. ✅/⬜ Docs & Scripts (step-04-docs-and-scripts)
5. ✅/⬜ Validate & Summary (step-05-validate-and-summary)"
---
### 4. Route to Next Step
Based on `lastStep`, load the next incomplete step:
- `'step-01-preflight'``./step-02-select-framework.md`
- `'step-02-select-framework'``./step-03-scaffold-framework.md`
- `'step-03-scaffold-framework'``./step-04-docs-and-scripts.md`
- `'step-04-docs-and-scripts'``./step-05-validate-and-summary.md`
- `'step-05-validate-and-summary'`**Workflow already complete.** Display: "✅ **All steps completed.** Use **[V] Validate** to review outputs or **[E] Edit** to make revisions." Then halt.
**If `lastStep` does not match any value above**, display: "⚠️ **Unknown progress state** (`lastStep`: {lastStep}). Please use **[C] Create** to start fresh." Then halt.
**Otherwise**, load the identified step file, read completely, and execute.
The existing content in `{outputFile}` provides context from previously completed steps.
---
## 🚨 SYSTEM SUCCESS/FAILURE METRICS
### ✅ SUCCESS:
- Output document loaded and parsed correctly
- Previously created artifacts verified on disk
- Progress dashboard displayed accurately
- Routed to correct next step
### ❌ SYSTEM FAILURE:
- Not loading output document
- Not verifying existing artifacts
- Incorrect progress display
- Routing to wrong step
- Re-executing completed steps
**Master Rule:** Resume MUST route to the exact next incomplete step. Never re-execute completed steps.

View File

@@ -0,0 +1,117 @@
---
name: 'step-02-select-framework'
description: 'Select Playwright or Cypress and justify choice'
nextStepFile: './step-03-scaffold-framework.md'
outputFile: '{test_artifacts}/framework-setup-progress.md'
---
# Step 2: Framework Selection
## STEP GOAL
Choose the most appropriate framework and document the rationale.
## MANDATORY EXECUTION RULES
- 📖 Read the entire step file before acting
- ✅ Speak in `{communication_language}`
---
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
- 💾 Record outputs before proceeding
- 📖 Load the next step only when instructed
## CONTEXT BOUNDARIES:
- Available context: config, loaded artifacts, and knowledge fragments
- Focus: this step's goal only
- Limits: do not execute future steps
- Dependencies: prior steps' outputs (if any)
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
## 1. Selection Logic
Use `{detected_stack}` from Step 1 to guide framework selection.
**If {detected_stack} is `frontend` or `fullstack` (browser-based testing):**
Default to **Playwright** unless strong reasons suggest Cypress.
**Playwright recommended when:**
- Large or complex repo
- Multi-browser support needed
- Heavy API + UI integration
- CI speed/parallelism is important
**Cypress recommended when:**
- Small team prioritizes DX
- Component testing focus
- Simpler setup needed
**If {detected_stack} is `backend` (no browser-based testing):**
Select the framework matching the project language:
- **Python**: pytest (default), unittest
- **Java/Kotlin**: JUnit 5 (default), TestNG
- **Go**: Go test (built-in)
- **C#/.NET**: xUnit (default), NUnit, MSTest
- **Ruby**: RSpec (default), Minitest
- **Rust**: cargo test (built-in)
**If {detected_stack} is `fullstack`:**
Select both a browser-based framework (Playwright/Cypress) AND the appropriate backend framework for the detected language.
Respect `config.test_framework` if explicitly set (not `"auto"`).
---
## 2. Announce Decision
State the selected framework and reasoning.
---
### 3. Save Progress
**Save this step's accumulated work to `{outputFile}`.**
- **If `{outputFile}` does not exist** (first save), create it with YAML frontmatter:
```yaml
---
stepsCompleted: ['step-02-select-framework']
lastStep: 'step-02-select-framework'
lastSaved: '{date}'
---
```
Then write this step's output below the frontmatter.
- **If `{outputFile}` already exists**, update:
- Add `'step-02-select-framework'` to `stepsCompleted` array (only if not already present)
- Set `lastStep: 'step-02-select-framework'`
- Set `lastSaved: '{date}'`
- Append this step's output to the appropriate section of the document.
Load next step: `{nextStepFile}`
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Step completed in full with required outputs
### ❌ SYSTEM FAILURE:
- Skipped sequence steps or missing outputs
**Master Rule:** Skipping steps is FORBIDDEN.

View File

@@ -0,0 +1,323 @@
---
name: 'step-03-scaffold-framework'
description: 'Create framework scaffold with adaptive orchestration (agent-team, subagent, or sequential)'
nextStepFile: './step-04-docs-and-scripts.md'
knowledgeIndex: '{project-root}/_bmad/tea/testarch/tea-index.csv'
outputFile: '{test_artifacts}/framework-setup-progress.md'
---
# Step 3: Scaffold Framework
## STEP GOAL
Generate the test directory structure, configuration files, fixtures, factories, helpers, and sample tests using deterministic mode selection with runtime fallback.
## MANDATORY EXECUTION RULES
- 📖 Read the entire step file before acting
- ✅ Speak in `{communication_language}`
- ✅ Apply knowledge base patterns where required
- ✅ Resolve execution mode from explicit user request first, then config
- ✅ Apply fallback rules deterministically when requested mode is unsupported
---
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
- 💾 Record outputs before proceeding
- 📖 Load the next step only when instructed
## CONTEXT BOUNDARIES:
- Available context: config, loaded artifacts, and knowledge fragments
- Focus: this step's goal only
- Limits: do not execute future steps
- Dependencies: prior steps' outputs (if any)
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
## 0. Resolve Execution Mode (User Override First)
```javascript
const parseBooleanFlag = (value, defaultValue = true) => {
if (typeof value === 'string') {
const normalized = value.trim().toLowerCase();
if (['false', '0', 'off', 'no'].includes(normalized)) return false;
if (['true', '1', 'on', 'yes'].includes(normalized)) return true;
}
if (value === undefined || value === null) return defaultValue;
return Boolean(value);
};
const orchestrationContext = {
config: {
execution_mode: config.tea_execution_mode || 'auto', // "auto" | "subagent" | "agent-team" | "sequential"
capability_probe: parseBooleanFlag(config.tea_capability_probe, true), // supports booleans and "false"/"true" strings
},
timestamp: new Date().toISOString().replace(/[:.]/g, '-'),
};
const normalizeUserExecutionMode = (mode) => {
if (typeof mode !== 'string') return null;
const normalized = mode.trim().toLowerCase().replace(/[-_]/g, ' ').replace(/\s+/g, ' ');
if (normalized === 'auto') return 'auto';
if (normalized === 'sequential') return 'sequential';
if (normalized === 'subagent' || normalized === 'sub agent' || normalized === 'subagents' || normalized === 'sub agents') {
return 'subagent';
}
if (normalized === 'agent team' || normalized === 'agent teams' || normalized === 'agentteam') {
return 'agent-team';
}
return null;
};
const normalizeConfigExecutionMode = (mode) => {
if (mode === 'subagent') return 'subagent';
if (mode === 'auto' || mode === 'sequential' || mode === 'subagent' || mode === 'agent-team') {
return mode;
}
return null;
};
// Explicit user instruction in the active run takes priority over config.
const explicitModeFromUser = normalizeUserExecutionMode(runtime.getExplicitExecutionModeHint?.() || null);
const requestedMode = explicitModeFromUser || normalizeConfigExecutionMode(orchestrationContext.config.execution_mode) || 'auto';
const probeEnabled = orchestrationContext.config.capability_probe;
const supports = { subagent: false, agentTeam: false };
if (probeEnabled) {
supports.subagent = runtime.canLaunchSubagents?.() === true;
supports.agentTeam = runtime.canLaunchAgentTeams?.() === true;
}
let resolvedMode = requestedMode;
if (requestedMode === 'auto') {
if (supports.agentTeam) resolvedMode = 'agent-team';
else if (supports.subagent) resolvedMode = 'subagent';
else resolvedMode = 'sequential';
} else if (probeEnabled && requestedMode === 'agent-team' && !supports.agentTeam) {
resolvedMode = supports.subagent ? 'subagent' : 'sequential';
} else if (probeEnabled && requestedMode === 'subagent' && !supports.subagent) {
resolvedMode = 'sequential';
}
```
Resolution precedence:
1. Explicit user request in this run (`agent team` => `agent-team`; `subagent` => `subagent`; `sequential`; `auto`)
2. `tea_execution_mode` from config
3. Runtime capability fallback (when probing enabled)
## 1. Create Directory Structure
Use `{detected_stack}` from Step 1 to determine directory layout.
**If {detected_stack} is `frontend` or `fullstack`:**
- `{test_dir}/e2e/`
- `{test_dir}/support/fixtures/`
- `{test_dir}/support/helpers/`
- `{test_dir}/support/page-objects/` (optional)
**If {detected_stack} is `backend` or `fullstack`:**
Create the idiomatic test directory for the detected language:
- **Python (pytest)**: `tests/` with `conftest.py`, `tests/unit/`, `tests/integration/`, `tests/api/`
- **Java/Kotlin (JUnit)**: `src/test/java/` mirroring `src/main/java/` package structure, with `unit/`, `integration/`, `api/` sub-packages
- **Go**: `*_test.go` files alongside source files (Go convention), plus `testdata/` for fixtures
- **C#/.NET (xUnit)**: `tests/` project with `Unit/`, `Integration/`, `Api/` directories
- **Ruby (RSpec)**: `spec/` with `spec/unit/`, `spec/integration/`, `spec/api/`, `spec/support/`
- **Rust**: `tests/` for integration tests, inline `#[cfg(test)]` modules for unit tests
**If `config.tea_use_pactjs_utils` is enabled and runtime is Node.js/TypeScript** (i.e., `{detected_stack}` is `frontend` or `fullstack`, or `{detected_stack}` is `backend` with Node.js/TypeScript runtime):
Create Node.js/TypeScript contract testing directory structure per `pact-consumer-framework-setup.md`:
- `tests/contract/consumer/` — consumer contract test files (`.pacttest.ts` extension)
- `tests/contract/support/` — pact config, provider state factories, consumer helpers shim
- `scripts/` — shell scripts (`env-setup.sh`, `publish-pact.sh`, `can-i-deploy.sh`, `record-deployment.sh`)
- `.github/actions/detect-breaking-change/` — PR checkbox-driven breaking change detection
- `.github/workflows/contract-test-consumer.yml` — consumer CDC CI workflow
---
## 2. Generate Framework Config
**If {detected_stack} is `frontend` or `fullstack`:**
Create `playwright.config.ts` or `cypress.config.ts` with:
- **Timeouts**: action 15s, navigation 30s, test 60s
- **Base URL**: env fallback (`BASE_URL`)
- **Artifacts**: retain-on-failure (trace/screenshot/video)
- **Reporters**: HTML + JUnit + console
- **Parallelism**: enabled (CI tuned)
Use TypeScript if `use_typescript: true`.
**If {detected_stack} is `backend` or `fullstack`:**
Create the idiomatic test config for the detected framework:
- **pytest**: `pyproject.toml` `[tool.pytest.ini_options]` or `pytest.ini` with markers, test paths, coverage settings
- **JUnit**: `build.gradle`/`pom.xml` test configuration with JUnit 5 dependencies, Surefire/Failsafe plugins
- **Go test**: no config file needed (Go convention); optionally create `Makefile` test targets
- **xUnit**: `.csproj` test project with xUnit and coverlet dependencies
- **RSpec**: `.rspec` config file with `spec_helper.rb` and `rails_helper.rb` (if Rails)
---
## 3. Environment Setup
Create `.env.example` with `TEST_ENV`, `BASE_URL`, `API_URL`.
**Stack-conditional environment files:**
**If {detected_stack} is `frontend` or `fullstack` (Node.js):**
- `.nvmrc` using current LTS Node (prefer Node 24+)
**If {detected_stack} is `backend`:**
Create the idiomatic version file for the detected language:
- **Python**: `.python-version` with current stable Python (prefer 3.12+)
- **Java**: `.java-version` or `JAVA_HOME` documentation in `.env.example`
- **Go**: Go version is already in `go.mod` (no additional file needed)
- **C#/.NET**: `global.json` with SDK version if not already present
- **Ruby**: `.ruby-version` with current stable Ruby
---
## 4. Fixtures & Factories
Read `{config_source}` and use `{knowledgeIndex}` to load fragments based on `config.tea_use_playwright_utils`:
**If Playwright Utils enabled:**
- `overview.md`, `fixtures-composition.md`, `auth-session.md`, `api-request.md`, `burn-in.md`, `network-error-monitor.md`, `data-factories.md`
- Recommend installing `@seontechnologies/playwright-utils`
**If disabled:**
- `fixture-architecture.md`, `data-factories.md`, `network-first.md`, `playwright-config.md`, `test-quality.md`
**If Pact.js Utils enabled** (`config.tea_use_pactjs_utils`):
- `pact-consumer-framework-setup.md` (CRITICAL: load this for directory structure, scripts, CI workflow, and PactV4 patterns)
- `pactjs-utils-overview.md`, `pactjs-utils-consumer-helpers.md`, `pactjs-utils-provider-verifier.md`, `pactjs-utils-request-filter.md`, `contract-testing.md`
- Recommend installing `@seontechnologies/pactjs-utils` and `@pact-foundation/pact`
**If Pact.js Utils disabled but contract testing relevant:**
- `contract-testing.md`
**If Pact MCP enabled** (`config.tea_pact_mcp` is `"mcp"`):
- `pact-mcp.md`
Implement:
- Fixture index with `mergeTests`
- Auto-cleanup hooks
- Faker-based data factories with overrides
---
## 5. Sample Tests & Helpers
**If {detected_stack} is `frontend` or `fullstack`:**
Create example tests in `{test_dir}/e2e/` demonstrating:
- Given/When/Then format
- data-testid selector strategy
- Factory usage
- Network interception pattern (if applicable)
**If {detected_stack} is `backend` or `fullstack`:**
Create example tests in the idiomatic location for the detected language:
- **Python**: `tests/test_example.py` with pytest fixtures, parametrize, and factory usage
- **Java**: `src/test/java/.../ExampleTest.java` with JUnit 5 annotations, `@BeforeEach` setup
- **Go**: `example_test.go` alongside source with table-driven tests and `testify` assertions
- **C#/.NET**: `tests/ExampleTests.cs` with xUnit `[Fact]`/`[Theory]` and fixture injection
- **Ruby**: `spec/example_spec.rb` with RSpec `describe`/`context`/`it` and factory_bot
Create helpers for:
- API clients (if needed)
- Network utilities (frontend/fullstack only)
- Auth helpers
- Test data factories (language-idiomatic patterns)
**If `config.tea_use_pactjs_utils` is enabled and runtime is Node.js/TypeScript** (i.e., `{detected_stack}` is `frontend` or `fullstack`, or `{detected_stack}` is `backend` with Node.js/TypeScript runtime):
Create Node.js/TypeScript contract test samples per `pact-consumer-framework-setup.md`:
- **Consumer test**: Example using PactV4 `addInteraction()` builder + `createProviderState` + real consumer code with URL injection (`.pacttest.ts` extension)
- **Support files**: Pact config factory (`pact-config.ts`), provider state factories (`provider-states.ts`), local consumer-helpers shim (`consumer-helpers.ts`)
- **Vitest config**: Minimal `vitest.config.pact.ts` (do NOT copy settings from unit config)
- **Shell scripts**: `env-setup.sh`, `publish-pact.sh`, `can-i-deploy.sh`, `record-deployment.sh` in `scripts/`
- **CI workflow**: `contract-test-consumer.yml` with detect-breaking-change action
- **package.json scripts**: `test:pact:consumer`, `publish:pact`, `can:i:deploy:consumer`, `record:consumer:deployment`
- **.gitignore**: Add `/pacts/` and `pact-logs/`
---
### 6. Orchestration Notes for This Step
For this step, treat these work units as parallelizable when `resolvedMode` is `agent-team` or `subagent`:
- Worker A: directory + framework config + env setup (sections 1-3)
- Worker B: fixtures + factories (section 4)
- Worker C: sample tests + helpers (section 5)
In parallel-capable modes, runtime decides worker scheduling and concurrency.
If `resolvedMode` is `sequential`, execute sections 1→5 in order.
Regardless of mode, outputs must be identical in structure and quality.
### 7. Save Progress
**Save this step's accumulated work to `{outputFile}`.**
- **If `{outputFile}` does not exist** (first save), create it with YAML frontmatter:
```yaml
---
stepsCompleted: ['step-03-scaffold-framework']
lastStep: 'step-03-scaffold-framework'
lastSaved: '{date}'
---
```
Then write this step's output below the frontmatter.
- **If `{outputFile}` already exists**, update:
- Add `'step-03-scaffold-framework'` to `stepsCompleted` array (only if not already present)
- Set `lastStep: 'step-03-scaffold-framework'`
- Set `lastSaved: '{date}'`
- Append this step's output to the appropriate section of the document.
Load next step: `{nextStepFile}`
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Step completed in full with required outputs
### ❌ SYSTEM FAILURE:
- Skipped sequence steps or missing outputs
**Master Rule:** Skipping steps is FORBIDDEN.

View File

@@ -0,0 +1,105 @@
---
name: 'step-04-docs-and-scripts'
description: 'Document setup and add package.json scripts'
nextStepFile: './step-05-validate-and-summary.md'
outputFile: '{test_dir}/README.md'
progressFile: '{test_artifacts}/framework-setup-progress.md'
---
# Step 4: Documentation & Scripts
## STEP GOAL
Create test documentation and add build/test scripts appropriate for `{detected_stack}`.
## MANDATORY EXECUTION RULES
- 📖 Read the entire step file before acting
- ✅ Speak in `{communication_language}`
---
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
- 💾 Record outputs before proceeding
- 📖 Load the next step only when instructed
## CONTEXT BOUNDARIES:
- Available context: config, loaded artifacts, and knowledge fragments
- Focus: this step's goal only
- Limits: do not execute future steps
- Dependencies: prior steps' outputs (if any)
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
## 1. tests/README.md
Create `{outputFile}` and include:
- Setup instructions
- Running tests (local/headed/debug)
- Architecture overview (fixtures, factories, helpers)
- Best practices (selectors, isolation, cleanup)
- CI integration notes
- Knowledge base references
---
## 2. Build & Test Scripts
**If {detected_stack} is `frontend` or `fullstack`:**
Add to `package.json` at minimum:
- `test:e2e`: framework execution command (e.g., `npx playwright test`)
**If {detected_stack} is `backend` or `fullstack`:**
Add the idiomatic test commands for the detected framework:
- **Python (pytest)**: Add to `pyproject.toml` scripts or `Makefile`: `pytest`, `pytest --cov`, `pytest -m integration`
- **Java (JUnit)**: Add to `build.gradle`/`pom.xml`: `./gradlew test`, `mvn test`, `mvn verify` (integration)
- **Go**: Add to `Makefile`: `go test ./...`, `go test -race ./...`, `go test -cover ./...`
- **C#/.NET**: Add to CI scripts or `Makefile`: `dotnet test`, `dotnet test --collect:"XPlat Code Coverage"`
- **Ruby (RSpec)**: Add to `Gemfile` binstubs or `Makefile`: `bundle exec rspec`, `bundle exec rspec spec/integration`
---
### 3. Save Progress
**Save this step's accumulated work to `{progressFile}`.**
- **If `{progressFile}` does not exist** (first save), create it with YAML frontmatter:
```yaml
---
stepsCompleted: ['step-04-docs-and-scripts']
lastStep: 'step-04-docs-and-scripts'
lastSaved: '{date}'
---
```
Then write this step's output below the frontmatter.
- **If `{progressFile}` already exists**, update:
- Add `'step-04-docs-and-scripts'` to `stepsCompleted` array (only if not already present)
- Set `lastStep: 'step-04-docs-and-scripts'`
- Set `lastSaved: '{date}'`
- Append this step's output to the appropriate section of the document.
Load next step: `{nextStepFile}`
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Step completed in full with required outputs
### ❌ SYSTEM FAILURE:
- Skipped sequence steps or missing outputs
**Master Rule:** Skipping steps is FORBIDDEN.

View File

@@ -0,0 +1,93 @@
---
name: 'step-05-validate-and-summary'
description: 'Validate against checklist and summarize'
outputFile: '{test_artifacts}/framework-setup-progress.md'
---
# Step 5: Validate & Summarize
## STEP GOAL
Validate framework setup and provide a completion summary.
## MANDATORY EXECUTION RULES
- 📖 Read the entire step file before acting
- ✅ Speak in `{communication_language}`
---
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
- 💾 Record outputs before proceeding
- 📖 Load the next step only when instructed
## CONTEXT BOUNDARIES:
- Available context: config, loaded artifacts, and knowledge fragments
- Focus: this step's goal only
- Limits: do not execute future steps
- Dependencies: prior steps' outputs (if any)
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
## 1. Validation
Validate against `checklist.md`:
- Preflight success
- Directory structure created
- Config correctness
- Fixtures/factories created
- Docs and scripts present
Fix any gaps before completion.
---
## 2. Completion Summary
Report:
- Framework selected
- Artifacts created
- Next steps (install deps, run tests)
- Knowledge fragments applied
---
### 3. Save Progress
**Save this step's accumulated work to `{outputFile}`.**
- **If `{outputFile}` does not exist** (first save), create it with YAML frontmatter:
```yaml
---
stepsCompleted: ['step-05-validate-and-summary']
lastStep: 'step-05-validate-and-summary'
lastSaved: '{date}'
---
```
Then write this step's output below the frontmatter.
- **If `{outputFile}` already exists**, update:
- Add `'step-05-validate-and-summary'` to `stepsCompleted` array (only if not already present)
- Set `lastStep: 'step-05-validate-and-summary'`
- Set `lastSaved: '{date}'`
- Append this step's output to the appropriate section of the document.
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Step completed in full with required outputs
### ❌ SYSTEM FAILURE:
- Skipped sequence steps or missing outputs
**Master Rule:** Skipping steps is FORBIDDEN.

View File

@@ -0,0 +1,65 @@
---
name: 'step-01-assess'
description: 'Load an existing output for editing'
nextStepFile: './step-02-apply-edit.md'
---
# Step 1: Assess Edit Target
## STEP GOAL:
Identify which output should be edited and load it.
## MANDATORY EXECUTION RULES (READ FIRST):
### Universal Rules:
- 📖 Read the complete step file before taking any action
- ✅ Speak in `{communication_language}`
### Role Reinforcement:
- ✅ You are the Master Test Architect
### Step-Specific Rules:
- 🎯 Ask the user which output file to edit
- 🚫 Do not edit until target is confirmed
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
## CONTEXT BOUNDARIES:
- Available context: existing outputs
- Focus: select edit target
- Limits: no edits yet
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly.
### 1. Identify Target
Ask the user to provide the output file path or select from known outputs.
### 2. Load Target
Read the provided output file in full.
### 3. Confirm
Confirm the target and proceed to edit.
Load next step: `{nextStepFile}`
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Target identified and loaded
### ❌ SYSTEM FAILURE:
- Proceeding without a confirmed target

View File

@@ -0,0 +1,60 @@
---
name: 'step-02-apply-edit'
description: 'Apply edits to the selected output'
---
# Step 2: Apply Edits
## STEP GOAL:
Apply the requested edits to the selected output and confirm changes.
## MANDATORY EXECUTION RULES (READ FIRST):
### Universal Rules:
- 📖 Read the complete step file before taking any action
- ✅ Speak in `{communication_language}`
### Role Reinforcement:
- ✅ You are the Master Test Architect
### Step-Specific Rules:
- 🎯 Only apply edits explicitly requested by the user
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
## CONTEXT BOUNDARIES:
- Available context: selected output and user changes
- Focus: apply edits only
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly.
### 1. Confirm Requested Changes
Restate what will be changed and confirm.
### 2. Apply Changes
Update the output file accordingly.
### 3. Report
Summarize the edits applied.
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Changes applied and confirmed
### ❌ SYSTEM FAILURE:
- Unconfirmed edits or missing update

View File

@@ -0,0 +1,67 @@
---
name: 'step-01-validate'
description: 'Validate workflow outputs against checklist'
outputFile: '{test_artifacts}/framework-validation-report.md'
validationChecklist: '../checklist.md'
---
# Step 1: Validate Outputs
## STEP GOAL:
Validate outputs using the workflow checklist and record findings.
## MANDATORY EXECUTION RULES (READ FIRST):
### Universal Rules:
- 📖 Read the complete step file before taking any action
- ✅ Speak in `{communication_language}`
### Role Reinforcement:
- ✅ You are the Master Test Architect
### Step-Specific Rules:
- 🎯 Validate against `{validationChecklist}`
- 🚫 Do not skip checks
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
- 💾 Write findings to `{outputFile}`
## CONTEXT BOUNDARIES:
- Available context: workflow outputs and checklist
- Focus: validation only
- Limits: do not modify outputs in this step
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly.
### 1. Load Checklist
Read `{validationChecklist}` and list all criteria.
### 2. Validate Outputs
Evaluate outputs against each checklist item.
### 3. Write Report
Write a validation report to `{outputFile}` with PASS/WARN/FAIL per section.
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Validation report written
- All checklist items evaluated
### ❌ SYSTEM FAILURE:
- Skipped checklist items
- No report produced

View File

@@ -0,0 +1,73 @@
---
validationDate: 2026-01-27
workflowName: testarch-framework
workflowPath: {project-root}/src/workflows/testarch/framework
validationStatus: COMPLETE
completionDate: 2026-01-27 10:03:10
---
# Validation Report: testarch-framework
**Validation Started:** 2026-01-27 09:50:21
**Validator:** BMAD Workflow Validation System (Codex)
**Standards Version:** BMAD Workflow Standards
## File Structure & Size
- workflow.md present: YES
- instructions.md present: YES
- workflow.yaml present: YES
- step files found: 8
**Step File Sizes:**
- steps-c/step-01-preflight.md: 69 lines [GOOD]
- steps-c/step-02-select-framework.md: 66 lines [GOOD]
- steps-c/step-03-scaffold-framework.md: 107 lines [GOOD]
- steps-c/step-04-docs-and-scripts.md: 63 lines [GOOD]
- steps-c/step-05-validate-and-summary.md: 61 lines [GOOD]
- steps-e/step-01-assess.md: 51 lines [GOOD]
- steps-e/step-02-apply-edit.md: 46 lines [GOOD]
- steps-v/step-01-validate.md: 53 lines [GOOD]
- workflow-plan.md present: YES
## Frontmatter Validation
- No frontmatter violations found
## Critical Path Violations
- No {project-root} hardcoded paths detected in body
- No dead relative links detected
## Menu Handling Validation
- No menu structures detected (linear step flow) [N/A]
## Step Type Validation
- Last step steps-v/step-01-validate.md has no nextStepFile (final step OK)
- Step type validation assumes linear sequence (no branching/menu). Workflow-plan.md present for reference. [INFO]
## Output Format Validation
- No templates found in workflow root
- Steps with outputFile in frontmatter:
- steps-c/step-04-docs-and-scripts.md
- steps-v/step-01-validate.md
## Validation Design Check
- checklist.md present: YES
- Validation steps folder (steps-v) present: YES
## Instruction Style Check
- All steps include STEP GOAL, MANDATORY EXECUTION RULES, EXECUTION PROTOCOLS, CONTEXT BOUNDARIES, and SUCCESS/FAILURE metrics
## Summary
- Validation completed: 2026-01-27 10:03:10
- Critical issues: 0
- Warnings: 0 (informational notes only)
- Readiness: READY (manual review optional)

View File

@@ -0,0 +1,116 @@
---
validationDate: 2026-01-27
workflowName: testarch-framework
workflowPath: {project-root}/src/workflows/testarch/framework
validationStatus: COMPLETE
completionDate: 2026-01-27 10:24:01
---
# Validation Report: testarch-framework
**Validation Started:** 2026-01-27 10:24:01
**Validator:** BMAD Workflow Validation System (Codex)
**Standards Version:** BMAD Workflow Standards
## File Structure & Size
- workflow.md present: YES
- instructions.md present: YES
- workflow.yaml present: YES
- step files found: 8
**Step File Sizes:**
- steps-c/step-01-preflight.md: 68 lines [GOOD]
- steps-c/step-02-select-framework.md: 65 lines [GOOD]
- steps-c/step-03-scaffold-framework.md: 106 lines [GOOD]
- steps-c/step-04-docs-and-scripts.md: 62 lines [GOOD]
- steps-c/step-05-validate-and-summary.md: 60 lines [GOOD]
- steps-e/step-01-assess.md: 50 lines [GOOD]
- steps-e/step-02-apply-edit.md: 45 lines [GOOD]
- steps-v/step-01-validate.md: 52 lines [GOOD]
- workflow-plan.md present: YES
## Frontmatter Validation
- No frontmatter violations found
## Critical Path Violations
### Config Variables (Exceptions)
Standard BMAD config variables treated as valid exceptions: bmb_creations_output_folder, communication_language, document_output_language, output_folder, planning_artifacts, project-root, project_name, test_artifacts, user_name
- No {project-root} hardcoded paths detected in body
- No dead relative links detected
- No module path assumptions detected
**Status:** ✅ PASS - No critical violations
## Menu Handling Validation
- No menu structures detected (linear step flow) [N/A]
## Step Type Validation
- steps-c/step-01-preflight.md: Init [PASS]
- steps-c/step-02-select-framework.md: Middle [PASS]
- steps-c/step-03-scaffold-framework.md: Middle [PASS]
- steps-c/step-04-docs-and-scripts.md: Middle [PASS]
- steps-c/step-05-validate-and-summary.md: Final [PASS]
- Step type validation assumes linear sequence (no branching/menu). Workflow-plan.md present for reference. [INFO]
## Output Format Validation
- Templates present: NONE
- Steps with outputFile in frontmatter:
- steps-c/step-04-docs-and-scripts.md
- steps-v/step-01-validate.md
- checklist.md present: YES
## Validation Design Check
- Validation steps folder (steps-v) present: YES
- Validation step(s) present: step-01-validate.md
- Validation steps reference checklist data and auto-proceed
## Instruction Style Check
- Instruction style: Prescriptive (appropriate for TEA quality/compliance workflows)
- Steps emphasize mandatory sequence, explicit success/failure metrics, and risk-based guidance
## Collaborative Experience Check
- Overall facilitation quality: GOOD
- Steps use progressive prompts and clear role reinforcement; no laundry-list interrogation detected
- Flow progression is clear and aligned to workflow goals
## Subagent Optimization Opportunities
- No high-priority subagent optimizations identified; workflow already uses step-file architecture
- Pattern 1 (grep/regex): N/A for most steps
- Pattern 2 (per-file analysis): already aligned to validation structure
- Pattern 3 (data ops): minimal data file loads
- Pattern 4 (parallel): optional for validation only
## Cohesive Review
- Overall assessment: GOOD
- Flow is linear, goals are clear, and outputs map to TEA artifacts
- Voice and tone consistent with Test Architect persona
- Recommendation: READY (minor refinements optional)
## Plan Quality Validation
- Plan file present: workflow-plan.md
- Planned steps found: 8 (all implemented)
- Plan implementation status: Fully Implemented
## Summary
- Validation completed: 2026-01-27 10:24:01
- Critical issues: 0
- Warnings: 0 (informational notes only)
- Readiness: READY (manual review optional)

View File

@@ -0,0 +1,22 @@
# Workflow Plan: testarch-framework
## Create Mode (steps-c)
- step-01-preflight.md
- step-02-select-framework.md
- step-03-scaffold-framework.md
- step-04-docs-and-scripts.md
- step-05-validate-and-summary.md
## Validate Mode (steps-v)
- step-01-validate.md
## Edit Mode (steps-e)
- step-01-assess.md
- step-02-apply-edit.md
## Outputs
- Framework config (playwright.config.ts or cypress.config.ts)
- {project-root}/tests/README.md
- Test support scaffolding under {project-root}/tests

View File

@@ -0,0 +1,41 @@
---
name: testarch-framework
description: Initialize test framework with Playwright or Cypress. Use when user says 'lets setup test framework' or 'I want to initialize testing framework'
web_bundle: true
---
# Test Framework Setup
**Goal:** Initialize production-ready test framework architecture (Playwright or Cypress) with fixtures, helpers, and configuration
**Role:** You are the Master Test Architect.
---
## WORKFLOW ARCHITECTURE
This workflow uses **tri-modal step-file architecture**:
- **Create mode (steps-c/)**: primary execution flow
- **Validate mode (steps-v/)**: validation against checklist
- **Edit mode (steps-e/)**: revise existing outputs
---
## INITIALIZATION SEQUENCE
### 1. Mode Determination
"Welcome to the workflow. What would you like to do?"
- **[C] Create** — Run the workflow
- **[R] Resume** — Resume an interrupted workflow
- **[V] Validate** — Validate existing outputs
- **[E] Edit** — Edit existing outputs
### 2. Route to First Step
- **If C:** Load `steps-c/step-01-preflight.md`
- **If R:** Load `steps-c/step-01b-resume.md`
- **If V:** Load `steps-v/step-01-validate.md`
- **If E:** Load `steps-e/step-01-assess.md`

View File

@@ -0,0 +1,48 @@
# Test Architect workflow: framework
name: testarch-framework
# prettier-ignore
description: 'Initialize test framework with Playwright or Cypress. Use when the user says "lets setup test framework" or "I want to initialize testing framework"'
# Critical variables from config
config_source: "{project-root}/_bmad/tea/config.yaml"
output_folder: "{config_source}:output_folder"
test_artifacts: "{config_source}:test_artifacts"
user_name: "{config_source}:user_name"
communication_language: "{config_source}:communication_language"
document_output_language: "{config_source}:document_output_language"
date: system-generated
# Workflow components
installed_path: "{project-root}/_bmad/tea/workflows/testarch/framework"
instructions: "{installed_path}/instructions.md"
validation: "{installed_path}/checklist.md"
# Variables and inputs
variables:
test_dir: "{project-root}/tests" # Root test directory
use_typescript: true # Prefer TypeScript configuration
framework_preference: "auto" # auto, playwright, cypress - user can override auto-detection
project_size: "auto" # auto, small, large - influences framework recommendation
# Output configuration
default_output_file: "{test_dir}/README.md" # Main deliverable is test setup README
# Required tools
required_tools:
- read_file # Read package.json, existing configs
- write_file # Create config files, helpers, fixtures, tests
- create_directory # Create test directory structure
- list_files # Check for existing framework
- search_repo # Find architecture docs
tags:
- qa
- setup
- test-architect
- framework
- initialization
execution_hints:
interactive: false # Minimize prompts; auto-detect when possible
autonomous: true # Proceed without user input unless blocked
iterative: true

View File

@@ -0,0 +1,407 @@
# Non-Functional Requirements Assessment - Validation Checklist
**Workflow:** `testarch-nfr`
**Purpose:** Ensure comprehensive and evidence-based NFR assessment with actionable recommendations
---
Note: `nfr-assess` evaluates existing evidence; it does not run tests or CI workflows.
## Prerequisites Validation
- [ ] Implementation is deployed and accessible for evaluation
- [ ] Evidence sources are available (test results, metrics, logs, CI results)
- [ ] NFR categories are determined (performance, security, reliability, maintainability, custom)
- [ ] Evidence directories exist and are accessible (`test_results_dir`, `metrics_dir`, `logs_dir`)
- [ ] Knowledge base is loaded (nfr-criteria, ci-burn-in, test-quality)
---
## Context Loading
- [ ] Tech-spec.md loaded successfully (if available)
- [ ] PRD.md loaded (if available)
- [ ] Story file loaded (if applicable)
- [ ] Relevant knowledge fragments loaded from `tea-index.csv`:
- [ ] `nfr-criteria.md`
- [ ] `ci-burn-in.md`
- [ ] `test-quality.md`
- [ ] `playwright-config.md` (if using Playwright)
---
## NFR Categories and Thresholds
### Performance
- [ ] Response time threshold defined or marked as UNKNOWN
- [ ] Throughput threshold defined or marked as UNKNOWN
- [ ] Resource usage thresholds defined or marked as UNKNOWN
- [ ] Scalability requirements defined or marked as UNKNOWN
### Security
- [ ] Authentication requirements defined or marked as UNKNOWN
- [ ] Authorization requirements defined or marked as UNKNOWN
- [ ] Data protection requirements defined or marked as UNKNOWN
- [ ] Vulnerability management thresholds defined or marked as UNKNOWN
- [ ] Compliance requirements identified (GDPR, HIPAA, PCI-DSS, etc.)
### Reliability
- [ ] Availability (uptime) threshold defined or marked as UNKNOWN
- [ ] Error rate threshold defined or marked as UNKNOWN
- [ ] MTTR (Mean Time To Recovery) threshold defined or marked as UNKNOWN
- [ ] Fault tolerance requirements defined or marked as UNKNOWN
- [ ] Disaster recovery requirements defined (RTO, RPO) or marked as UNKNOWN
### Maintainability
- [ ] Test coverage threshold defined or marked as UNKNOWN
- [ ] Code quality threshold defined or marked as UNKNOWN
- [ ] Technical debt threshold defined or marked as UNKNOWN
- [ ] Documentation completeness threshold defined or marked as UNKNOWN
### Custom NFR Categories (if applicable)
- [ ] Custom NFR category 1: Thresholds defined or marked as UNKNOWN
- [ ] Custom NFR category 2: Thresholds defined or marked as UNKNOWN
- [ ] Custom NFR category 3: Thresholds defined or marked as UNKNOWN
---
## Evidence Gathering
### Performance Evidence
- [ ] Load test results collected (JMeter, k6, Gatling, etc.)
- [ ] Application metrics collected (response times, throughput, resource usage)
- [ ] APM data collected (New Relic, Datadog, Dynatrace, etc.)
- [ ] Lighthouse reports collected (if web app)
- [ ] Playwright performance traces collected (if applicable)
### Security Evidence
- [ ] SAST results collected (SonarQube, Checkmarx, Veracode, etc.)
- [ ] DAST results collected (OWASP ZAP, Burp Suite, etc.)
- [ ] Dependency scanning results collected (Snyk, Dependabot, npm audit)
- [ ] Penetration test reports collected (if available)
- [ ] Security audit logs collected
- [ ] Compliance audit results collected (if applicable)
### Reliability Evidence
- [ ] Uptime monitoring data collected (Pingdom, UptimeRobot, StatusCake)
- [ ] Error logs collected
- [ ] Error rate metrics collected
- [ ] CI burn-in results collected (stability over time)
- [ ] Chaos engineering test results collected (if available)
- [ ] Failover/recovery test results collected (if available)
- [ ] Incident reports and postmortems collected (if applicable)
### Maintainability Evidence
- [ ] Code coverage reports collected (Istanbul, NYC, c8, JaCoCo)
- [ ] Static analysis results collected (ESLint, SonarQube, CodeClimate)
- [ ] Technical debt metrics collected
- [ ] Documentation audit results collected
- [ ] Test review report collected (from test-review workflow, if available)
- [ ] Git metrics collected (code churn, commit frequency, etc.)
---
## NFR Assessment with Deterministic Rules
### Performance Assessment
- [ ] Response time assessed against threshold
- [ ] Throughput assessed against threshold
- [ ] Resource usage assessed against threshold
- [ ] Scalability assessed against requirements
- [ ] Status classified (PASS/CONCERNS/FAIL) with justification
- [ ] Evidence source documented (file path, metric name)
### Security Assessment
- [ ] Authentication strength assessed against requirements
- [ ] Authorization controls assessed against requirements
- [ ] Data protection assessed against requirements
- [ ] Vulnerability management assessed against thresholds
- [ ] Compliance assessed against requirements
- [ ] Status classified (PASS/CONCERNS/FAIL) with justification
- [ ] Evidence source documented (file path, scan result)
### Reliability Assessment
- [ ] Availability (uptime) assessed against threshold
- [ ] Error rate assessed against threshold
- [ ] MTTR assessed against threshold
- [ ] Fault tolerance assessed against requirements
- [ ] Disaster recovery assessed against requirements (RTO, RPO)
- [ ] CI burn-in assessed (stability over time)
- [ ] Status classified (PASS/CONCERNS/FAIL) with justification
- [ ] Evidence source documented (file path, monitoring data)
### Maintainability Assessment
- [ ] Test coverage assessed against threshold
- [ ] Code quality assessed against threshold
- [ ] Technical debt assessed against threshold
- [ ] Documentation completeness assessed against threshold
- [ ] Test quality assessed (from test-review, if available)
- [ ] Status classified (PASS/CONCERNS/FAIL) with justification
- [ ] Evidence source documented (file path, coverage report)
### Custom NFR Assessment (if applicable)
- [ ] Custom NFR 1 assessed against threshold with justification
- [ ] Custom NFR 2 assessed against threshold with justification
- [ ] Custom NFR 3 assessed against threshold with justification
---
## Status Classification Validation
### PASS Criteria Verified
- [ ] Evidence exists for PASS status
- [ ] Evidence meets or exceeds threshold
- [ ] No concerns flagged in evidence
- [ ] Quality is acceptable
### CONCERNS Criteria Verified
- [ ] Threshold is UNKNOWN (documented) OR
- [ ] Evidence is MISSING or INCOMPLETE (documented) OR
- [ ] Evidence is close to threshold (within 10%, documented) OR
- [ ] Evidence shows intermittent issues (documented)
### FAIL Criteria Verified
- [ ] Evidence exists BUT does not meet threshold (documented) OR
- [ ] Critical evidence is MISSING (documented) OR
- [ ] Evidence shows consistent failures (documented) OR
- [ ] Quality is unacceptable (documented)
### No Threshold Guessing
- [ ] All thresholds are either defined or marked as UNKNOWN
- [ ] No thresholds were guessed or inferred
- [ ] All UNKNOWN thresholds result in CONCERNS status
---
## Quick Wins and Recommended Actions
### Quick Wins Identified
- [ ] Low-effort, high-impact improvements identified for CONCERNS/FAIL
- [ ] Configuration changes (no code changes) identified
- [ ] Optimization opportunities identified (caching, indexing, compression)
- [ ] Monitoring additions identified (detect issues before failures)
### Recommended Actions
- [ ] Specific remediation steps provided (not generic advice)
- [ ] Priority assigned (CRITICAL, HIGH, MEDIUM, LOW)
- [ ] Estimated effort provided (hours, days)
- [ ] Owner suggestions provided (dev, ops, security)
### Monitoring Hooks
- [ ] Performance monitoring suggested (APM, synthetic monitoring)
- [ ] Error tracking suggested (Sentry, Rollbar, error logs)
- [ ] Security monitoring suggested (intrusion detection, audit logs)
- [ ] Alerting thresholds suggested (notify before breach)
### Fail-Fast Mechanisms
- [ ] Circuit breakers suggested for reliability
- [ ] Rate limiting suggested for performance
- [ ] Validation gates suggested for security
- [ ] Smoke tests suggested for maintainability
---
## Deliverables Generated
### NFR Assessment Report
- [ ] File created at `{test_artifacts}/nfr-assessment.md`
- [ ] Template from `nfr-report-template.md` used
- [ ] Executive summary included (overall status, critical issues)
- [ ] Assessment by category included (performance, security, reliability, maintainability)
- [ ] Evidence for each NFR documented
- [ ] Status classifications documented (PASS/CONCERNS/FAIL)
- [ ] Findings summary included (PASS count, CONCERNS count, FAIL count)
- [ ] Quick wins section included
- [ ] Recommended actions section included
- [ ] Evidence gaps checklist included
### Gate YAML Snippet (if enabled)
- [ ] YAML snippet generated
- [ ] Date included
- [ ] Categories status included (performance, security, reliability, maintainability)
- [ ] Overall status included (PASS/CONCERNS/FAIL)
- [ ] Issue counts included (critical, high, medium, concerns)
- [ ] Blockers flag included (true/false)
- [ ] Recommendations included
### Evidence Checklist (if enabled)
- [ ] All NFRs with MISSING or INCOMPLETE evidence listed
- [ ] Owners assigned for evidence collection
- [ ] Suggested evidence sources provided
- [ ] Deadlines set for evidence collection
### Updated Story File (if enabled and requested)
- [ ] "NFR Assessment" section added to story markdown
- [ ] Link to NFR assessment report included
- [ ] Overall status and critical issues included
- [ ] Gate status included
---
## Quality Assurance
### Accuracy Checks
- [ ] All NFR categories assessed (none skipped)
- [ ] All thresholds documented (defined or UNKNOWN)
- [ ] All evidence sources documented (file paths, metric names)
- [ ] Status classifications are deterministic and consistent
- [ ] No false positives (status correctly assigned)
- [ ] No false negatives (all issues identified)
### Completeness Checks
- [ ] All NFR categories covered (performance, security, reliability, maintainability, custom)
- [ ] All evidence sources checked (test results, metrics, logs, CI results)
- [ ] All status types used appropriately (PASS, CONCERNS, FAIL)
- [ ] All NFRs with CONCERNS/FAIL have recommendations
- [ ] All evidence gaps have owners and deadlines
### Actionability Checks
- [ ] Recommendations are specific (not generic)
- [ ] Remediation steps are clear and actionable
- [ ] Priorities are assigned (CRITICAL, HIGH, MEDIUM, LOW)
- [ ] Effort estimates are provided (hours, days)
- [ ] Owners are suggested (dev, ops, security)
---
## Integration with BMad Artifacts
### With tech-spec.md
- [ ] Tech spec loaded for NFR requirements and thresholds
- [ ] Performance targets extracted
- [ ] Security requirements extracted
- [ ] Reliability SLAs extracted
- [ ] Architectural decisions considered
### With test-design.md
- [ ] Test design loaded for NFR test plan
- [ ] Test priorities referenced (P0/P1/P2/P3)
- [ ] Assessment aligned with planned NFR validation
### With PRD.md
- [ ] PRD loaded for product-level NFR context
- [ ] User experience goals considered
- [ ] Unstated requirements checked
- [ ] Product-level SLAs referenced
---
## Quality Gates Validation
### Release Blocker (FAIL)
- [ ] Critical NFR status checked (security, reliability)
- [ ] Performance failures assessed for user impact
- [ ] Release blocker flagged if critical NFR has FAIL status
### PR Blocker (HIGH CONCERNS)
- [ ] High-priority NFR status checked
- [ ] Multiple CONCERNS assessed
- [ ] PR blocker flagged if HIGH priority issues exist
### Warning (CONCERNS)
- [ ] Any NFR with CONCERNS status flagged
- [ ] Missing or incomplete evidence documented
- [ ] Warning issued to address before next release
### Pass (PASS)
- [ ] All NFRs have PASS status
- [ ] No blockers or concerns exist
- [ ] Ready for release confirmed
---
## Non-Prescriptive Validation
- [ ] NFR categories adapted to team needs
- [ ] Thresholds appropriate for project context
- [ ] Assessment criteria customized as needed
- [ ] Teams can extend with custom NFR categories
- [ ] Integration with external tools supported (New Relic, Datadog, SonarQube, JIRA)
---
## Documentation and Communication
- [ ] NFR assessment report is readable and well-formatted
- [ ] Tables render correctly in markdown
- [ ] Code blocks have proper syntax highlighting
- [ ] Links are valid and accessible
- [ ] Recommendations are clear and prioritized
- [ ] Overall status is prominent and unambiguous
- [ ] Executive summary provides quick understanding
---
## Final Validation
- [ ] All prerequisites met
- [ ] All NFR categories assessed with evidence (or gaps documented)
- [ ] No thresholds were guessed (all defined or UNKNOWN)
- [ ] Status classifications are deterministic and justified
- [ ] Quick wins identified for all CONCERNS/FAIL
- [ ] Recommended actions are specific and actionable
- [ ] Evidence gaps documented with owners and deadlines
- [ ] NFR assessment report generated and saved
- [ ] Gate YAML snippet generated (if enabled)
- [ ] Evidence checklist generated (if enabled)
- [ ] Workflow completed successfully
---
## Sign-Off
**NFR Assessment Status:**
- [ ] ✅ PASS - All NFRs meet requirements, ready for release
- [ ] ⚠️ CONCERNS - Some NFRs have concerns, address before next release
- [ ] ❌ FAIL - Critical NFRs not met, BLOCKER for release
**Next Actions:**
- If PASS ✅: Proceed to `*gate` workflow or release
- If CONCERNS ⚠️: Address HIGH/CRITICAL issues, re-run `*nfr-assess`
- If FAIL ❌: Resolve FAIL status NFRs, re-run `*nfr-assess`
**Critical Issues:** {COUNT}
**High Priority Issues:** {COUNT}
**Concerns:** {COUNT}
---
<!-- Powered by BMAD-CORE™ -->

View File

@@ -0,0 +1,43 @@
# Non-Functional Requirements Assessment
**Workflow:** `testarch-nfr`
**Version:** 5.0 (Step-File Architecture)
---
## Overview
Assess non-functional requirements (performance, security, reliability, maintainability) with evidence-based validation and deterministic PASS/CONCERNS/FAIL outcomes.
---
## WORKFLOW ARCHITECTURE
This workflow uses **step-file architecture**:
- **Micro-file Design**: Each step is self-contained
- **JIT Loading**: Only the current step file is in memory
- **Sequential Enforcement**: Execute steps in order
---
## INITIALIZATION SEQUENCE
### 1. Configuration Loading
From `workflow.yaml`, resolve:
- `config_source`, `test_artifacts`, `user_name`, `communication_language`, `document_output_language`, `date`
- `custom_nfr_categories`
### 2. First Step
Load, read completely, and execute:
`{project-root}/_bmad/tea/workflows/testarch/nfr-assess/steps-c/step-01-load-context.md`
### 3. Resume Support
If the user selects **Resume** mode, load, read completely, and execute:
`{project-root}/_bmad/tea/workflows/testarch/nfr-assess/steps-c/step-01b-resume.md`
This checks the output document for progress tracking frontmatter and routes to the next incomplete step.

View File

@@ -0,0 +1,470 @@
---
stepsCompleted: []
lastStep: ''
lastSaved: ''
workflowType: 'testarch-nfr-assess'
inputDocuments: []
---
# NFR Assessment - {FEATURE_NAME}
**Date:** {DATE}
**Story:** {STORY_ID} (if applicable)
**Overall Status:** {OVERALL_STATUS} {STATUS_ICON}
---
Note: This assessment summarizes existing evidence; it does not run tests or CI workflows.
## Executive Summary
**Assessment:** {PASS_COUNT} PASS, {CONCERNS_COUNT} CONCERNS, {FAIL_COUNT} FAIL
**Blockers:** {BLOCKER_COUNT} {BLOCKER_DESCRIPTION}
**High Priority Issues:** {HIGH_PRIORITY_COUNT} {HIGH_PRIORITY_DESCRIPTION}
**Recommendation:** {OVERALL_RECOMMENDATION}
---
## Performance Assessment
### Response Time (p95)
- **Status:** {STATUS} {STATUS_ICON}
- **Threshold:** {THRESHOLD_VALUE}
- **Actual:** {ACTUAL_VALUE}
- **Evidence:** {EVIDENCE_SOURCE}
- **Findings:** {FINDINGS_DESCRIPTION}
### Throughput
- **Status:** {STATUS} {STATUS_ICON}
- **Threshold:** {THRESHOLD_VALUE}
- **Actual:** {ACTUAL_VALUE}
- **Evidence:** {EVIDENCE_SOURCE}
- **Findings:** {FINDINGS_DESCRIPTION}
### Resource Usage
- **CPU Usage**
- **Status:** {STATUS} {STATUS_ICON}
- **Threshold:** {THRESHOLD_VALUE}
- **Actual:** {ACTUAL_VALUE}
- **Evidence:** {EVIDENCE_SOURCE}
- **Memory Usage**
- **Status:** {STATUS} {STATUS_ICON}
- **Threshold:** {THRESHOLD_VALUE}
- **Actual:** {ACTUAL_VALUE}
- **Evidence:** {EVIDENCE_SOURCE}
### Scalability
- **Status:** {STATUS} {STATUS_ICON}
- **Threshold:** {THRESHOLD_DESCRIPTION}
- **Actual:** {ACTUAL_DESCRIPTION}
- **Evidence:** {EVIDENCE_SOURCE}
- **Findings:** {FINDINGS_DESCRIPTION}
---
## Security Assessment
### Authentication Strength
- **Status:** {STATUS} {STATUS_ICON}
- **Threshold:** {THRESHOLD_DESCRIPTION}
- **Actual:** {ACTUAL_DESCRIPTION}
- **Evidence:** {EVIDENCE_SOURCE}
- **Findings:** {FINDINGS_DESCRIPTION}
- **Recommendation:** {RECOMMENDATION} (if CONCERNS or FAIL)
### Authorization Controls
- **Status:** {STATUS} {STATUS_ICON}
- **Threshold:** {THRESHOLD_DESCRIPTION}
- **Actual:** {ACTUAL_DESCRIPTION}
- **Evidence:** {EVIDENCE_SOURCE}
- **Findings:** {FINDINGS_DESCRIPTION}
### Data Protection
- **Status:** {STATUS} {STATUS_ICON}
- **Threshold:** {THRESHOLD_DESCRIPTION}
- **Actual:** {ACTUAL_DESCRIPTION}
- **Evidence:** {EVIDENCE_SOURCE}
- **Findings:** {FINDINGS_DESCRIPTION}
### Vulnerability Management
- **Status:** {STATUS} {STATUS_ICON}
- **Threshold:** {THRESHOLD_DESCRIPTION} (e.g., "0 critical, <3 high vulnerabilities")
- **Actual:** {ACTUAL_DESCRIPTION} (e.g., "0 critical, 1 high, 5 medium vulnerabilities")
- **Evidence:** {EVIDENCE_SOURCE} (e.g., "Snyk scan results - scan-2025-10-14.json")
- **Findings:** {FINDINGS_DESCRIPTION}
### Compliance (if applicable)
- **Status:** {STATUS} {STATUS_ICON}
- **Standards:** {COMPLIANCE_STANDARDS} (e.g., "GDPR, HIPAA, PCI-DSS")
- **Actual:** {ACTUAL_COMPLIANCE_STATUS}
- **Evidence:** {EVIDENCE_SOURCE}
- **Findings:** {FINDINGS_DESCRIPTION}
---
## Reliability Assessment
### Availability (Uptime)
- **Status:** {STATUS} {STATUS_ICON}
- **Threshold:** {THRESHOLD_VALUE} (e.g., "99.9%")
- **Actual:** {ACTUAL_VALUE} (e.g., "99.95%")
- **Evidence:** {EVIDENCE_SOURCE} (e.g., "Uptime monitoring - uptime-report-2025-10-14.csv")
- **Findings:** {FINDINGS_DESCRIPTION}
### Error Rate
- **Status:** {STATUS} {STATUS_ICON}
- **Threshold:** {THRESHOLD_VALUE} (e.g., "<0.1%")
- **Actual:** {ACTUAL_VALUE} (e.g., "0.05%")
- **Evidence:** {EVIDENCE_SOURCE} (e.g., "Error logs - logs/errors-2025-10.log")
- **Findings:** {FINDINGS_DESCRIPTION}
### MTTR (Mean Time To Recovery)
- **Status:** {STATUS} {STATUS_ICON}
- **Threshold:** {THRESHOLD_VALUE} (e.g., "<15 minutes")
- **Actual:** {ACTUAL_VALUE} (e.g., "12 minutes")
- **Evidence:** {EVIDENCE_SOURCE} (e.g., "Incident reports - incidents/")
- **Findings:** {FINDINGS_DESCRIPTION}
### Fault Tolerance
- **Status:** {STATUS} {STATUS_ICON}
- **Threshold:** {THRESHOLD_DESCRIPTION}
- **Actual:** {ACTUAL_DESCRIPTION}
- **Evidence:** {EVIDENCE_SOURCE}
- **Findings:** {FINDINGS_DESCRIPTION}
### CI Burn-In (Stability)
- **Status:** {STATUS} {STATUS_ICON}
- **Threshold:** {THRESHOLD_VALUE} (e.g., "100 consecutive successful runs")
- **Actual:** {ACTUAL_VALUE} (e.g., "150 consecutive successful runs")
- **Evidence:** {EVIDENCE_SOURCE} (e.g., "CI burn-in results - ci-burn-in-2025-10-14.log")
- **Findings:** {FINDINGS_DESCRIPTION}
### Disaster Recovery (if applicable)
- **RTO (Recovery Time Objective)**
- **Status:** {STATUS} {STATUS_ICON}
- **Threshold:** {THRESHOLD_VALUE}
- **Actual:** {ACTUAL_VALUE}
- **Evidence:** {EVIDENCE_SOURCE}
- **RPO (Recovery Point Objective)**
- **Status:** {STATUS} {STATUS_ICON}
- **Threshold:** {THRESHOLD_VALUE}
- **Actual:** {ACTUAL_VALUE}
- **Evidence:** {EVIDENCE_SOURCE}
---
## Maintainability Assessment
### Test Coverage
- **Status:** {STATUS} {STATUS_ICON}
- **Threshold:** {THRESHOLD_VALUE} (e.g., ">=80%")
- **Actual:** {ACTUAL_VALUE} (e.g., "87%")
- **Evidence:** {EVIDENCE_SOURCE} (e.g., "Coverage report - coverage/lcov-report/index.html")
- **Findings:** {FINDINGS_DESCRIPTION}
### Code Quality
- **Status:** {STATUS} {STATUS_ICON}
- **Threshold:** {THRESHOLD_VALUE} (e.g., ">=85/100")
- **Actual:** {ACTUAL_VALUE} (e.g., "92/100")
- **Evidence:** {EVIDENCE_SOURCE} (e.g., "SonarQube analysis - sonarqube-report-2025-10-14.pdf")
- **Findings:** {FINDINGS_DESCRIPTION}
### Technical Debt
- **Status:** {STATUS} {STATUS_ICON}
- **Threshold:** {THRESHOLD_VALUE} (e.g., "<5% debt ratio")
- **Actual:** {ACTUAL_VALUE} (e.g., "3.2% debt ratio")
- **Evidence:** {EVIDENCE_SOURCE} (e.g., "CodeClimate analysis - codeclimate-2025-10-14.json")
- **Findings:** {FINDINGS_DESCRIPTION}
### Documentation Completeness
- **Status:** {STATUS} {STATUS_ICON}
- **Threshold:** {THRESHOLD_VALUE} (e.g., ">=90%")
- **Actual:** {ACTUAL_VALUE} (e.g., "95%")
- **Evidence:** {EVIDENCE_SOURCE} (e.g., "Documentation audit - docs-audit-2025-10-14.md")
- **Findings:** {FINDINGS_DESCRIPTION}
### Test Quality (from test-review, if available)
- **Status:** {STATUS} {STATUS_ICON}
- **Threshold:** {THRESHOLD_DESCRIPTION}
- **Actual:** {ACTUAL_DESCRIPTION}
- **Evidence:** {EVIDENCE_SOURCE} (e.g., "Test review report - test-review-2025-10-14.md")
- **Findings:** {FINDINGS_DESCRIPTION}
---
## Custom NFR Assessments (if applicable)
### {CUSTOM_NFR_NAME_1}
- **Status:** {STATUS} {STATUS_ICON}
- **Threshold:** {THRESHOLD_DESCRIPTION}
- **Actual:** {ACTUAL_DESCRIPTION}
- **Evidence:** {EVIDENCE_SOURCE}
- **Findings:** {FINDINGS_DESCRIPTION}
### {CUSTOM_NFR_NAME_2}
- **Status:** {STATUS} {STATUS_ICON}
- **Threshold:** {THRESHOLD_DESCRIPTION}
- **Actual:** {ACTUAL_DESCRIPTION}
- **Evidence:** {EVIDENCE_SOURCE}
- **Findings:** {FINDINGS_DESCRIPTION}
---
## Quick Wins
{QUICK_WIN_COUNT} quick wins identified for immediate implementation:
1. **{QUICK_WIN_TITLE_1}** ({NFR_CATEGORY}) - {PRIORITY} - {ESTIMATED_EFFORT}
- {QUICK_WIN_DESCRIPTION}
- No code changes needed / Minimal code changes
2. **{QUICK_WIN_TITLE_2}** ({NFR_CATEGORY}) - {PRIORITY} - {ESTIMATED_EFFORT}
- {QUICK_WIN_DESCRIPTION}
---
## Recommended Actions
### Immediate (Before Release) - CRITICAL/HIGH Priority
1. **{ACTION_TITLE_1}** - {PRIORITY} - {ESTIMATED_EFFORT} - {OWNER}
- {ACTION_DESCRIPTION}
- {SPECIFIC_STEPS}
- {VALIDATION_CRITERIA}
2. **{ACTION_TITLE_2}** - {PRIORITY} - {ESTIMATED_EFFORT} - {OWNER}
- {ACTION_DESCRIPTION}
- {SPECIFIC_STEPS}
- {VALIDATION_CRITERIA}
### Short-term (Next Milestone) - MEDIUM Priority
1. **{ACTION_TITLE_3}** - {PRIORITY} - {ESTIMATED_EFFORT} - {OWNER}
- {ACTION_DESCRIPTION}
2. **{ACTION_TITLE_4}** - {PRIORITY} - {ESTIMATED_EFFORT} - {OWNER}
- {ACTION_DESCRIPTION}
### Long-term (Backlog) - LOW Priority
1. **{ACTION_TITLE_5}** - {PRIORITY} - {ESTIMATED_EFFORT} - {OWNER}
- {ACTION_DESCRIPTION}
---
## Monitoring Hooks
{MONITORING_HOOK_COUNT} monitoring hooks recommended to detect issues before failures:
### Performance Monitoring
- [ ] {MONITORING_TOOL_1} - {MONITORING_DESCRIPTION}
- **Owner:** {OWNER}
- **Deadline:** {DEADLINE}
- [ ] {MONITORING_TOOL_2} - {MONITORING_DESCRIPTION}
- **Owner:** {OWNER}
- **Deadline:** {DEADLINE}
### Security Monitoring
- [ ] {MONITORING_TOOL_3} - {MONITORING_DESCRIPTION}
- **Owner:** {OWNER}
- **Deadline:** {DEADLINE}
### Reliability Monitoring
- [ ] {MONITORING_TOOL_4} - {MONITORING_DESCRIPTION}
- **Owner:** {OWNER}
- **Deadline:** {DEADLINE}
### Alerting Thresholds
- [ ] {ALERT_DESCRIPTION} - Notify when {THRESHOLD_CONDITION}
- **Owner:** {OWNER}
- **Deadline:** {DEADLINE}
---
## Fail-Fast Mechanisms
{FAIL_FAST_COUNT} fail-fast mechanisms recommended to prevent failures:
### Circuit Breakers (Reliability)
- [ ] {CIRCUIT_BREAKER_DESCRIPTION}
- **Owner:** {OWNER}
- **Estimated Effort:** {EFFORT}
### Rate Limiting (Performance)
- [ ] {RATE_LIMITING_DESCRIPTION}
- **Owner:** {OWNER}
- **Estimated Effort:** {EFFORT}
### Validation Gates (Security)
- [ ] {VALIDATION_GATE_DESCRIPTION}
- **Owner:** {OWNER}
- **Estimated Effort:** {EFFORT}
### Smoke Tests (Maintainability)
- [ ] {SMOKE_TEST_DESCRIPTION}
- **Owner:** {OWNER}
- **Estimated Effort:** {EFFORT}
---
## Evidence Gaps
{EVIDENCE_GAP_COUNT} evidence gaps identified - action required:
- [ ] **{NFR_NAME_1}** ({NFR_CATEGORY})
- **Owner:** {OWNER}
- **Deadline:** {DEADLINE}
- **Suggested Evidence:** {SUGGESTED_EVIDENCE_SOURCE}
- **Impact:** {IMPACT_DESCRIPTION}
- [ ] **{NFR_NAME_2}** ({NFR_CATEGORY})
- **Owner:** {OWNER}
- **Deadline:** {DEADLINE}
- **Suggested Evidence:** {SUGGESTED_EVIDENCE_SOURCE}
- **Impact:** {IMPACT_DESCRIPTION}
---
## Findings Summary
**Based on ADR Quality Readiness Checklist (8 categories, 29 criteria)**
| Category | Criteria Met | PASS | CONCERNS | FAIL | Overall Status |
| ------------------------------------------------ | ------------------ | ---------------- | -------------------- | ---------------- | ----------------------------------- |
| 1. Testability & Automation | {T_MET}/4 | {T_PASS} | {T_CONCERNS} | {T_FAIL} | {T_STATUS} {T_ICON} |
| 2. Test Data Strategy | {TD_MET}/3 | {TD_PASS} | {TD_CONCERNS} | {TD_FAIL} | {TD_STATUS} {TD_ICON} |
| 3. Scalability & Availability | {SA_MET}/4 | {SA_PASS} | {SA_CONCERNS} | {SA_FAIL} | {SA_STATUS} {SA_ICON} |
| 4. Disaster Recovery | {DR_MET}/3 | {DR_PASS} | {DR_CONCERNS} | {DR_FAIL} | {DR_STATUS} {DR_ICON} |
| 5. Security | {SEC_MET}/4 | {SEC_PASS} | {SEC_CONCERNS} | {SEC_FAIL} | {SEC_STATUS} {SEC_ICON} |
| 6. Monitorability, Debuggability & Manageability | {MON_MET}/4 | {MON_PASS} | {MON_CONCERNS} | {MON_FAIL} | {MON_STATUS} {MON_ICON} |
| 7. QoS & QoE | {QOS_MET}/4 | {QOS_PASS} | {QOS_CONCERNS} | {QOS_FAIL} | {QOS_STATUS} {QOS_ICON} |
| 8. Deployability | {DEP_MET}/3 | {DEP_PASS} | {DEP_CONCERNS} | {DEP_FAIL} | {DEP_STATUS} {DEP_ICON} |
| **Total** | **{TOTAL_MET}/29** | **{TOTAL_PASS}** | **{TOTAL_CONCERNS}** | **{TOTAL_FAIL}** | **{OVERALL_STATUS} {OVERALL_ICON}** |
**Criteria Met Scoring:**
- ≥26/29 (90%+) = Strong foundation
- 20-25/29 (69-86%) = Room for improvement
- <20/29 (<69%) = Significant gaps
---
## Gate YAML Snippet
```yaml
nfr_assessment:
date: '{DATE}'
story_id: '{STORY_ID}'
feature_name: '{FEATURE_NAME}'
adr_checklist_score: '{TOTAL_MET}/29' # ADR Quality Readiness Checklist
categories:
testability_automation: '{T_STATUS}'
test_data_strategy: '{TD_STATUS}'
scalability_availability: '{SA_STATUS}'
disaster_recovery: '{DR_STATUS}'
security: '{SEC_STATUS}'
monitorability: '{MON_STATUS}'
qos_qoe: '{QOS_STATUS}'
deployability: '{DEP_STATUS}'
overall_status: '{OVERALL_STATUS}'
critical_issues: { CRITICAL_COUNT }
high_priority_issues: { HIGH_COUNT }
medium_priority_issues: { MEDIUM_COUNT }
concerns: { CONCERNS_COUNT }
blockers: { BLOCKER_BOOLEAN } # true/false
quick_wins: { QUICK_WIN_COUNT }
evidence_gaps: { EVIDENCE_GAP_COUNT }
recommendations:
- '{RECOMMENDATION_1}'
- '{RECOMMENDATION_2}'
- '{RECOMMENDATION_3}'
```
---
## Related Artifacts
- **Story File:** {STORY_FILE_PATH} (if applicable)
- **Tech Spec:** {TECH_SPEC_PATH} (if available)
- **PRD:** {PRD_PATH} (if available)
- **Test Design:** {TEST_DESIGN_PATH} (if available)
- **Evidence Sources:**
- Test Results: {TEST_RESULTS_DIR}
- Metrics: {METRICS_DIR}
- Logs: {LOGS_DIR}
- CI Results: {CI_RESULTS_PATH}
---
## Recommendations Summary
**Release Blocker:** {RELEASE_BLOCKER_SUMMARY}
**High Priority:** {HIGH_PRIORITY_SUMMARY}
**Medium Priority:** {MEDIUM_PRIORITY_SUMMARY}
**Next Steps:** {NEXT_STEPS_DESCRIPTION}
---
## Sign-Off
**NFR Assessment:**
- Overall Status: {OVERALL_STATUS} {OVERALL_ICON}
- Critical Issues: {CRITICAL_COUNT}
- High Priority Issues: {HIGH_COUNT}
- Concerns: {CONCERNS_COUNT}
- Evidence Gaps: {EVIDENCE_GAP_COUNT}
**Gate Status:** {GATE_STATUS} {GATE_ICON}
**Next Actions:**
- If PASS : Proceed to `*gate` workflow or release
- If CONCERNS : Address HIGH/CRITICAL issues, re-run `*nfr-assess`
- If FAIL : Resolve FAIL status NFRs, re-run `*nfr-assess`
**Generated:** {DATE}
**Workflow:** testarch-nfr v4.0
---
<!-- Powered by BMAD-CORE -->

View File

@@ -0,0 +1,138 @@
---
name: 'step-01-load-context'
description: 'Load NFR requirements, evidence sources, and knowledge base'
nextStepFile: './step-02-define-thresholds.md'
knowledgeIndex: '{project-root}/_bmad/tea/testarch/tea-index.csv'
outputFile: '{test_artifacts}/nfr-assessment.md'
---
# Step 1: Load Context & Knowledge Base
## STEP GOAL
Gather NFR requirements, evidence sources, and knowledge fragments needed for assessment.
## MANDATORY EXECUTION RULES
- 📖 Read the entire step file before acting
- ✅ Speak in `{communication_language}`
- 🚫 Halt if implementation or evidence is unavailable
---
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
- 💾 Record outputs before proceeding
- 📖 Load the next step only when instructed
## CONTEXT BOUNDARIES:
- Available context: config, loaded artifacts, and knowledge fragments
- Focus: this step's goal only
- Limits: do not execute future steps
- Dependencies: prior steps' outputs (if any)
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
## 1. Prerequisites
- Implementation accessible for evaluation
- Evidence sources available (test results, metrics, logs)
If missing: **HALT** and request the missing inputs.
---
## 2. Load Configuration
From `{config_source}`:
- Read `tea_browser_automation`
---
### Tiered Knowledge Loading
Load fragments based on their `tier` classification in `tea-index.csv`:
1. **Core tier** (always load): Foundational fragments required for this workflow
2. **Extended tier** (load on-demand): Load when deeper analysis is needed or when the user's context requires it
3. **Specialized tier** (load only when relevant): Load only when the specific use case matches (e.g., contract-testing only for microservices, email-auth only for email flows)
> **Context Efficiency**: Loading only core fragments reduces context usage by 40-50% compared to loading all fragments.
## 3. Load Knowledge Base Fragments
From `{knowledgeIndex}` load:
- `adr-quality-readiness-checklist.md`
- `ci-burn-in.md`
- `test-quality.md`
- `playwright-config.md`
- `error-handling.md`
**Playwright CLI (if `tea_browser_automation` is "cli" or "auto"):**
- `playwright-cli.md`
**MCP Patterns (if `tea_browser_automation` is "mcp" or "auto"):**
- (existing MCP-related fragments, if any are added in future)
---
## 4. Load Artifacts
If available, read:
- `tech-spec.md` (primary NFRs)
- `PRD.md` (product-level NFRs)
- `story` or `test-design` docs (feature-level NFRs)
---
## 5. Confirm Inputs
Summarize loaded NFR sources and evidence availability.
---
## 6. Save Progress
**Save this step's accumulated work to `{outputFile}`.**
- **If `{outputFile}` does not exist** (first save), create it using the workflow template (if available) with YAML frontmatter:
```yaml
---
stepsCompleted: ['step-01-load-context']
lastStep: 'step-01-load-context'
lastSaved: '{date}'
---
```
Then write this step's output below the frontmatter.
- **If `{outputFile}` already exists**, update:
- Add `'step-01-load-context'` to `stepsCompleted` array (only if not already present)
- Set `lastStep: 'step-01-load-context'`
- Set `lastSaved: '{date}'`
- Append this step's output to the appropriate section of the document.
**Update `inputDocuments`**: Set `inputDocuments` in the output template frontmatter to the list of artifact paths loaded in this step (e.g., knowledge fragments, test design documents, configuration files).
Load next step: `{nextStepFile}`
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Step completed in full with required outputs
### ❌ SYSTEM FAILURE:
- Skipped sequence steps or missing outputs
**Master Rule:** Skipping steps is FORBIDDEN.

View File

@@ -0,0 +1,106 @@
---
name: 'step-01b-resume'
description: 'Resume interrupted workflow from last completed step'
outputFile: '{test_artifacts}/nfr-assessment.md'
---
# Step 1b: Resume Workflow
## STEP GOAL
Resume an interrupted workflow by loading the existing output document, displaying progress, and routing to the next incomplete step.
## MANDATORY EXECUTION RULES
- Read the entire step file before acting
- Speak in `{communication_language}`
---
## EXECUTION PROTOCOLS:
- Follow the MANDATORY SEQUENCE exactly
- Load the next step only when instructed
## CONTEXT BOUNDARIES:
- Available context: Output document with progress frontmatter
- Focus: Load progress and route to next step
- Limits: Do not re-execute completed steps
- Dependencies: Output document must exist from a previous run
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly.
### 1. Load Output Document
Read `{outputFile}` and parse YAML frontmatter for:
- `stepsCompleted` -- array of completed step names
- `lastStep` -- last completed step name
- `lastSaved` -- timestamp of last save
**If `{outputFile}` does not exist**, display:
"No previous progress found. There is no output document to resume from. Please use **[C] Create** to start a fresh workflow run."
**THEN:** Halt. Do not proceed.
---
### 2. Display Progress Dashboard
Display progress with checkmark/empty indicators:
```
NFR Assessment - Resume Progress:
1. Load Context (step-01-load-context) [completed/pending]
2. Define Thresholds (step-02-define-thresholds) [completed/pending]
3. Gather Evidence (step-03-gather-evidence) [completed/pending]
4. Evaluate & Aggregate (step-04e-aggregate-nfr) [completed/pending]
5. Generate Report (step-05-generate-report) [completed/pending]
Last saved: {lastSaved}
```
---
### 3. Route to Next Step
Based on `lastStep`, load the next incomplete step:
| lastStep | Next Step File |
| --------------------------- | --------------------------------- |
| `step-01-load-context` | `./step-02-define-thresholds.md` |
| `step-02-define-thresholds` | `./step-03-gather-evidence.md` |
| `step-03-gather-evidence` | `./step-04-evaluate-and-score.md` |
| `step-04e-aggregate-nfr` | `./step-05-generate-report.md` |
| `step-05-generate-report` | **Workflow already complete.** |
**If `lastStep` is the final step** (`step-05-generate-report`), display: "All steps completed. Use **[C] Create** to start fresh, **[V] Validate** to review outputs, or **[E] Edit** to make revisions." Then halt.
**If `lastStep` does not match any value above**, display: "Unknown progress state (`lastStep`: {lastStep}). Please use **[C] Create** to start fresh." Then halt.
**Otherwise**, load the identified step file, read completely, and execute.
The existing content in `{outputFile}` provides context from previously completed steps.
---
## SYSTEM SUCCESS/FAILURE METRICS
### SUCCESS:
- Output document loaded and parsed correctly
- Progress dashboard displayed accurately
- Routed to correct next step
### FAILURE:
- Not loading output document
- Incorrect progress display
- Routing to wrong step
**Master Rule:** Resume MUST route to the exact next incomplete step. Never re-execute completed steps.

View File

@@ -0,0 +1,107 @@
---
name: 'step-02-define-thresholds'
description: 'Identify NFR categories and thresholds'
nextStepFile: './step-03-gather-evidence.md'
outputFile: '{test_artifacts}/nfr-assessment.md'
---
# Step 2: Define NFR Categories & Thresholds
## STEP GOAL
Establish the NFR categories to assess and the thresholds used for validation.
## MANDATORY EXECUTION RULES
- 📖 Read the entire step file before acting
- ✅ Speak in `{communication_language}`
- 🚫 Never guess thresholds
---
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
- 💾 Record outputs before proceeding
- 📖 Load the next step only when instructed
## CONTEXT BOUNDARIES:
- Available context: config, loaded artifacts, and knowledge fragments
- Focus: this step's goal only
- Limits: do not execute future steps
- Dependencies: prior steps' outputs (if any)
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
## 1. Select Categories
Use the ADR Quality Readiness Checklist (8 categories):
1. Testability & Automation
2. Test Data Strategy
3. Scalability & Availability
4. Disaster Recovery
5. Security
6. Monitorability/Debuggability/Manageability
7. QoS/QoE
8. Deployability
Add any `custom_nfr_categories` if provided.
---
## 2. Define Thresholds
For each category, extract thresholds from:
- tech-spec (primary)
- PRD (secondary)
- story or test-design (feature-specific)
If a threshold is unknown, mark it **UNKNOWN** and plan to report **CONCERNS**.
---
## 3. Confirm NFR Matrix
List each NFR category with its threshold or UNKNOWN status.
---
## 4. Save Progress
**Save this step's accumulated work to `{outputFile}`.**
- **If `{outputFile}` does not exist** (first save), create it using the workflow template (if available) with YAML frontmatter:
```yaml
---
stepsCompleted: ['step-02-define-thresholds']
lastStep: 'step-02-define-thresholds'
lastSaved: '{date}'
---
```
Then write this step's output below the frontmatter.
- **If `{outputFile}` already exists**, update:
- Add `'step-02-define-thresholds'` to `stepsCompleted` array (only if not already present)
- Set `lastStep: 'step-02-define-thresholds'`
- Set `lastSaved: '{date}'`
- Append this step's output to the appropriate section of the document.
Load next step: `{nextStepFile}`
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Step completed in full with required outputs
### ❌ SYSTEM FAILURE:
- Skipped sequence steps or missing outputs
**Master Rule:** Skipping steps is FORBIDDEN.

View File

@@ -0,0 +1,108 @@
---
name: 'step-03-gather-evidence'
description: 'Collect evidence for each NFR category'
nextStepFile: './step-04-evaluate-and-score.md'
outputFile: '{test_artifacts}/nfr-assessment.md'
---
# Step 3: Gather Evidence
## STEP GOAL
Collect measurable evidence to evaluate each NFR category.
## MANDATORY EXECUTION RULES
- 📖 Read the entire step file before acting
- ✅ Speak in `{communication_language}`
---
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
- 💾 Record outputs before proceeding
- 📖 Load the next step only when instructed
## CONTEXT BOUNDARIES:
- Available context: config, loaded artifacts, and knowledge fragments
- Focus: this step's goal only
- Limits: do not execute future steps
- Dependencies: prior steps' outputs (if any)
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
## 1. Evidence Sources
Collect evidence for:
- **Performance**: load tests, metrics, response time data
- **Security**: scans, auth tests, vuln reports
- **Reliability**: error rates, burn-in runs, failover tests
- **Maintainability**: test quality, code health signals
- **Other categories**: logs, monitoring, DR drills, deployability checks
---
## 2. Browser-Based Evidence Collection (if `tea_browser_automation` is `cli` or `auto`)
> **Fallback:** If CLI is not installed, fall back to MCP (if available) or skip browser-based evidence collection.
For performance and security categories, CLI can gather live evidence:
**Performance evidence (page load, response times):**
1. `playwright-cli -s=tea-nfr open <target_url>`
2. `playwright-cli -s=tea-nfr network` → capture response times and payload sizes
3. `playwright-cli -s=tea-nfr screenshot --filename={test_artifacts}/nfr/perf-<page>.png`
4. `playwright-cli -s=tea-nfr close`
> **Session Hygiene:** Always close sessions using `playwright-cli -s=tea-nfr close`. Do NOT use `close-all` — it kills every session on the machine and breaks parallel execution.
Store artifacts under `{test_artifacts}/nfr/`
---
## 3. Evidence Gaps
If evidence is missing for a category, mark that category as **CONCERNS**.
---
## 4. Save Progress
**Save this step's accumulated work to `{outputFile}`.**
- **If `{outputFile}` does not exist** (first save), create it using the workflow template (if available) with YAML frontmatter:
```yaml
---
stepsCompleted: ['step-03-gather-evidence']
lastStep: 'step-03-gather-evidence'
lastSaved: '{date}'
---
```
Then write this step's output below the frontmatter.
- **If `{outputFile}` already exists**, update:
- Add `'step-03-gather-evidence'` to `stepsCompleted` array (only if not already present)
- Set `lastStep: 'step-03-gather-evidence'`
- Set `lastSaved: '{date}'`
- Append this step's output to the appropriate section of the document.
Load next step: `{nextStepFile}`
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Step completed in full with required outputs
### ❌ SYSTEM FAILURE:
- Skipped sequence steps or missing outputs
**Master Rule:** Skipping steps is FORBIDDEN.

View File

@@ -0,0 +1,254 @@
---
name: 'step-04-evaluate-and-score'
description: 'Orchestrate adaptive NFR domain assessments (agent-team, subagent, or sequential)'
nextStepFile: './step-04e-aggregate-nfr.md'
---
# Step 4: Orchestrate Adaptive NFR Assessment
## STEP GOAL
Select execution mode deterministically, then assess NFR domains using agent-team, subagent, or sequential execution while preserving output contracts.
## MANDATORY EXECUTION RULES
- 📖 Read the entire step file before acting
- ✅ Speak in `{communication_language}`
- ✅ Resolve execution mode from config (`tea_execution_mode`, `tea_capability_probe`)
- ✅ Apply fallback rules deterministically when requested mode is unsupported
- ✅ Wait for required worker steps to complete
- ❌ Do NOT skip capability checks when probing is enabled
---
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
- 💾 Wait for subagent outputs
- 📖 Load the next step only when instructed
---
## MANDATORY SEQUENCE
### 1. Prepare Execution Context
**Generate unique timestamp:**
```javascript
const timestamp = new Date().toISOString().replace(/[:.]/g, '-');
```
**Prepare context:**
```javascript
const parseBooleanFlag = (value, defaultValue = true) => {
if (typeof value === 'string') {
const normalized = value.trim().toLowerCase();
if (['false', '0', 'off', 'no'].includes(normalized)) return false;
if (['true', '1', 'on', 'yes'].includes(normalized)) return true;
}
if (value === undefined || value === null) return defaultValue;
return Boolean(value);
};
const subagentContext = {
system_context: /* from Step 1 */,
nfr_thresholds: /* from Step 2 */,
evidence_gathered: /* from Step 3 */,
config: {
execution_mode: config.tea_execution_mode || 'auto', // "auto" | "subagent" | "agent-team" | "sequential"
capability_probe: parseBooleanFlag(config.tea_capability_probe, true), // supports booleans and "false"/"true" strings
},
timestamp: timestamp
};
```
---
### 2. Resolve Execution Mode with Capability Probe
```javascript
const normalizeUserExecutionMode = (mode) => {
if (typeof mode !== 'string') return null;
const normalized = mode.trim().toLowerCase().replace(/[-_]/g, ' ').replace(/\s+/g, ' ');
if (normalized === 'auto') return 'auto';
if (normalized === 'sequential') return 'sequential';
if (normalized === 'subagent' || normalized === 'sub agent' || normalized === 'subagents' || normalized === 'sub agents') {
return 'subagent';
}
if (normalized === 'agent team' || normalized === 'agent teams' || normalized === 'agentteam') {
return 'agent-team';
}
return null;
};
const normalizeConfigExecutionMode = (mode) => {
if (mode === 'subagent') return 'subagent';
if (mode === 'auto' || mode === 'sequential' || mode === 'subagent' || mode === 'agent-team') {
return mode;
}
return null;
};
// Explicit user instruction in the active run takes priority over config.
const explicitModeFromUser = normalizeUserExecutionMode(runtime.getExplicitExecutionModeHint?.() || null);
const requestedMode = explicitModeFromUser || normalizeConfigExecutionMode(subagentContext.config.execution_mode) || 'auto';
const probeEnabled = subagentContext.config.capability_probe;
const supports = {
subagent: false,
agentTeam: false,
};
if (probeEnabled) {
supports.subagent = runtime.canLaunchSubagents?.() === true;
supports.agentTeam = runtime.canLaunchAgentTeams?.() === true;
}
let resolvedMode = requestedMode;
if (requestedMode === 'auto') {
if (supports.agentTeam) resolvedMode = 'agent-team';
else if (supports.subagent) resolvedMode = 'subagent';
else resolvedMode = 'sequential';
} else if (probeEnabled && requestedMode === 'agent-team' && !supports.agentTeam) {
resolvedMode = supports.subagent ? 'subagent' : 'sequential';
} else if (probeEnabled && requestedMode === 'subagent' && !supports.subagent) {
resolvedMode = 'sequential';
}
subagentContext.execution = {
requestedMode,
resolvedMode,
probeEnabled,
supports,
};
```
Resolution precedence:
1. Explicit user request in this run (`agent team` => `agent-team`; `subagent` => `subagent`; `sequential`; `auto`)
2. `tea_execution_mode` from config
3. Runtime capability fallback (when probing enabled)
If probing is disabled, honor the requested mode strictly. If that mode cannot be executed at runtime, fail with explicit error instead of silent fallback.
---
### 3. Dispatch 4 NFR Workers
**Subagent A: Security Assessment**
- File: `./step-04a-subagent-security.md`
- Output: `/tmp/tea-nfr-security-${timestamp}.json`
- Execution:
- `agent-team` or `subagent`: launch non-blocking
- `sequential`: run blocking and wait
- Status: Running... ⟳
**Subagent B: Performance Assessment**
- File: `./step-04b-subagent-performance.md`
- Output: `/tmp/tea-nfr-performance-${timestamp}.json`
- Status: Running... ⟳
**Subagent C: Reliability Assessment**
- File: `./step-04c-subagent-reliability.md`
- Output: `/tmp/tea-nfr-reliability-${timestamp}.json`
- Status: Running... ⟳
**Subagent D: Scalability Assessment**
- File: `./step-04d-subagent-scalability.md`
- Output: `/tmp/tea-nfr-scalability-${timestamp}.json`
- Status: Running... ⟳
In `agent-team` and `subagent` modes, runtime decides worker scheduling and concurrency.
---
### 4. Wait for Expected Worker Completion
**If `resolvedMode` is `agent-team` or `subagent`:**
```
⏳ Waiting for 4 NFR subagents to complete...
├── Subagent A (Security): Running... ⟳
├── Subagent B (Performance): Running... ⟳
├── Subagent C (Reliability): Running... ⟳
└── Subagent D (Scalability): Running... ⟳
[... time passes ...]
✅ All 4 NFR subagents completed!
```
**If `resolvedMode` is `sequential`:**
```
✅ Sequential mode: each worker already completed during dispatch.
```
---
### 5. Verify All Outputs Exist
```javascript
const outputs = ['security', 'performance', 'reliability', 'scalability'].map((domain) => `/tmp/tea-nfr-${domain}-${timestamp}.json`);
outputs.forEach((output) => {
if (!fs.existsSync(output)) {
throw new Error(`Subagent output missing: ${output}`);
}
});
```
---
### 6. Execution Report
```
🚀 Performance Report:
- Execution Mode: {resolvedMode}
- Total Elapsed: ~mode-dependent
- Parallel Gain: ~67% faster when mode is subagent/agent-team
```
---
### 7. Proceed to Aggregation
Load next step: `{nextStepFile}`
The aggregation step will:
- Read all 4 NFR domain outputs
- Calculate overall risk level
- Aggregate compliance status
- Identify cross-domain risks
- Generate executive summary
---
## EXIT CONDITION
Proceed when all 4 required worker steps completed and outputs exist.
---
## 🚨 SYSTEM SUCCESS METRICS
### ✅ SUCCESS:
- All required worker steps completed
- Fallback behavior respected configuration and capability probe rules
### ❌ FAILURE:
- One or more subagents failed
- Unsupported requested mode with probing disabled

View File

@@ -0,0 +1,138 @@
---
name: 'step-04a-subagent-security'
description: 'Subagent: Security NFR assessment'
subagent: true
outputFile: '/tmp/tea-nfr-security-{{timestamp}}.json'
---
# Subagent 4A: Security NFR Assessment
## SUBAGENT CONTEXT
This is an **isolated subagent** running in parallel with other NFR domain assessments.
**Your task:** Assess SECURITY NFR domain only.
---
## MANDATORY EXECUTION RULES
- ✅ Assess SECURITY only (not performance, reliability, scalability)
- ✅ Output structured JSON to temp file
- ❌ Do NOT assess other NFR domains
---
## SUBAGENT TASK
### 1. Security Assessment Categories
**Assess the following security dimensions:**
**A) Authentication & Authorization:**
- OAuth2/JWT implementation
- Session management
- Multi-factor authentication
- Role-based access control (RBAC)
**B) Data Protection:**
- Encryption at rest
- Encryption in transit (HTTPS/TLS)
- Sensitive data handling (PII, passwords)
- Database encryption
**C) Input Validation:**
- SQL injection prevention
- XSS prevention
- CSRF protection
- Input sanitization
**D) API Security:**
- Rate limiting
- API authentication
- CORS configuration
- Security headers
**E) Secrets Management:**
- Environment variables for secrets
- No hardcoded credentials
- Secret rotation policies
- Key management systems
### 2. Risk Assessment
For each category, determine status:
- **PASS**: Properly implemented
- **CONCERN**: Partially implemented or weak
- **FAIL**: Not implemented or critical vulnerability
- **N/A**: Not applicable to this system
### 3. Compliance Check
**Common compliance standards:**
- SOC2
- GDPR
- HIPAA
- PCI-DSS
- ISO 27001
---
## OUTPUT FORMAT
```json
{
"domain": "security",
"risk_level": "MEDIUM",
"findings": [
{
"category": "Authentication",
"status": "PASS",
"description": "OAuth2 with JWT tokens implemented",
"evidence": ["src/auth/oauth.ts", "JWT refresh token rotation"],
"recommendations": []
},
{
"category": "Data Encryption",
"status": "CONCERN",
"description": "Database encryption at rest not enabled",
"evidence": ["Database config shows no encryption"],
"recommendations": ["Enable database encryption at rest", "Use AWS RDS encryption or equivalent", "Implement key rotation policy"]
},
{
"category": "Input Validation",
"status": "FAIL",
"description": "SQL injection vulnerability in search endpoint",
"evidence": ["src/api/search.ts:42 - direct SQL concatenation"],
"recommendations": ["URGENT: Use parameterized queries", "Add input sanitization library", "Implement WAF rules"]
}
],
"compliance": {
"SOC2": "PARTIAL",
"GDPR": "PASS",
"HIPAA": "N/A",
"PCI-DSS": "FAIL"
},
"priority_actions": [
"Fix SQL injection vulnerability (URGENT)",
"Enable database encryption within 30 days",
"Implement rate limiting for all APIs"
],
"summary": "Security posture is MEDIUM risk with 1 critical vulnerability requiring immediate attention"
}
```
---
## EXIT CONDITION
Subagent completes when JSON output written to temp file.
**Subagent terminates here.**

View File

@@ -0,0 +1,84 @@
---
name: 'step-04b-subagent-performance'
description: 'Subagent: Performance NFR assessment'
subagent: true
outputFile: '/tmp/tea-nfr-performance-{{timestamp}}.json'
---
# Subagent 4B: Performance NFR Assessment
## SUBAGENT CONTEXT
This is an **isolated subagent** running in parallel with other NFR domain assessments.
**Your task:** Assess PERFORMANCE NFR domain only.
---
## SUBAGENT TASK
### 1. Performance Assessment Categories
**A) Response Times:**
- API response times (<200ms target)
- Page load times (<2s target)
- Time to interactive (<3s target)
**B) Throughput:**
- Requests per second capacity
- Concurrent user support
- Database query performance
**C) Resource Usage:**
- Memory consumption
- CPU utilization
- Database connection pooling
**D) Optimization:**
- Caching strategies
- CDN usage
- Code splitting/lazy loading
- Database indexing
---
## OUTPUT FORMAT
```json
{
"domain": "performance",
"risk_level": "LOW",
"findings": [
{
"category": "Response Times",
"status": "PASS",
"description": "API endpoints respond in <150ms (P95)",
"evidence": ["Load testing results show 140ms P95"],
"recommendations": []
},
{
"category": "Caching",
"status": "CONCERN",
"description": "No CDN for static assets",
"evidence": ["Static files served from origin"],
"recommendations": ["Implement CDN (CloudFront/Cloudflare)", "Cache static assets for 1 year"]
}
],
"compliance": {
"SLA_99.9": "PASS",
"SLA_99.99": "CONCERN"
},
"priority_actions": ["Implement CDN for static assets", "Add database query caching for frequent reads"],
"summary": "Performance is acceptable with minor optimization opportunities"
}
```
---
## EXIT CONDITION
Subagent completes when JSON output written to temp file.

View File

@@ -0,0 +1,85 @@
---
name: 'step-04c-subagent-reliability'
description: 'Subagent: Reliability NFR assessment'
subagent: true
outputFile: '/tmp/tea-nfr-reliability-{{timestamp}}.json'
---
# Subagent 4C: Reliability NFR Assessment
## SUBAGENT CONTEXT
This is an **isolated subagent** running in parallel with other NFR domain assessments.
**Your task:** Assess RELIABILITY NFR domain only.
---
## SUBAGENT TASK
### 1. Reliability Assessment Categories
**A) Error Handling:**
- Try-catch blocks for critical operations
- Graceful degradation
- Circuit breakers
- Retry mechanisms
**B) Monitoring & Observability:**
- Logging implementation
- Error tracking (Sentry/Datadog)
- Health check endpoints
- Alerting systems
**C) Fault Tolerance:**
- Database failover
- Service redundancy
- Backup strategies
- Disaster recovery plan
**D) Uptime & Availability:**
- SLA targets
- Historical uptime
- Incident response
---
## OUTPUT FORMAT
```json
{
"domain": "reliability",
"risk_level": "LOW",
"findings": [
{
"category": "Error Handling",
"status": "PASS",
"description": "Comprehensive error handling with circuit breakers",
"evidence": ["Circuit breaker pattern in src/services/", "Retry logic implemented"],
"recommendations": []
},
{
"category": "Monitoring",
"status": "CONCERN",
"description": "No APM (Application Performance Monitoring) tool",
"evidence": ["Logging present but no distributed tracing"],
"recommendations": ["Implement APM (Datadog/New Relic)", "Add distributed tracing"]
}
],
"compliance": {
"SLA_99.9": "PASS"
},
"priority_actions": ["Implement APM for better observability"],
"summary": "Reliability is good with minor monitoring gaps"
}
```
---
## EXIT CONDITION
Subagent completes when JSON output written to temp file.

View File

@@ -0,0 +1,88 @@
---
name: 'step-04d-subagent-scalability'
description: 'Subagent: Scalability NFR assessment'
subagent: true
outputFile: '/tmp/tea-nfr-scalability-{{timestamp}}.json'
---
# Subagent 4D: Scalability NFR Assessment
## SUBAGENT CONTEXT
This is an **isolated subagent** running in parallel with other NFR domain assessments.
**Your task:** Assess SCALABILITY NFR domain only.
---
## SUBAGENT TASK
### 1. Scalability Assessment Categories
**A) Horizontal Scaling:**
- Stateless architecture
- Load balancer configuration
- Container orchestration (K8s)
- Auto-scaling policies
**B) Vertical Scaling:**
- Resource allocation
- Database size limits
- Memory management
- CPU optimization
**C) Data Scaling:**
- Database partitioning/sharding
- Read replicas
- Caching layers
- Data archival strategy
**D) Traffic Handling:**
- CDN for static assets
- Rate limiting
- Queue systems for async work
- WebSocket scaling
---
## OUTPUT FORMAT
```json
{
"domain": "scalability",
"risk_level": "MEDIUM",
"findings": [
{
"category": "Horizontal Scaling",
"status": "PASS",
"description": "Stateless architecture with container orchestration",
"evidence": ["Docker + Kubernetes setup", "Auto-scaling configured"],
"recommendations": []
},
{
"category": "Data Scaling",
"status": "CONCERN",
"description": "No database sharding strategy for large data growth",
"evidence": ["Single database instance", "No partitioning"],
"recommendations": ["Plan database sharding strategy", "Implement read replicas", "Consider database clustering"]
}
],
"compliance": {
"1M_users": "PASS",
"10M_users": "CONCERN",
"100M_users": "FAIL"
},
"priority_actions": ["Design database sharding strategy for future growth", "Implement read replicas for read-heavy workloads"],
"summary": "Scalability is good up to 1M users, concerns for 10M+ users"
}
```
---
## EXIT CONDITION
Subagent completes when JSON output written to temp file.

View File

@@ -0,0 +1,264 @@
---
name: 'step-04e-aggregate-nfr'
description: 'Aggregate NFR domain assessments into executive summary'
nextStepFile: './step-05-generate-report.md'
outputFile: '{test_artifacts}/nfr-assessment.md'
---
# Step 4E: Aggregate NFR Assessment Results
## STEP GOAL
Read outputs from 4 parallel NFR subagents, calculate overall risk level, aggregate compliance status, and identify cross-domain risks.
---
## MANDATORY EXECUTION RULES
- 📖 Read the entire step file before acting
- ✅ Speak in `{communication_language}`
- ✅ Read all 4 subagent outputs
- ✅ Calculate overall risk level
- ❌ Do NOT re-assess NFRs (use subagent outputs)
---
## MANDATORY SEQUENCE
### 1. Read All Subagent Outputs
```javascript
const domains = ['security', 'performance', 'reliability', 'scalability'];
const assessments = {};
domains.forEach((domain) => {
const outputPath = `/tmp/tea-nfr-${domain}-{{timestamp}}.json`;
assessments[domain] = JSON.parse(fs.readFileSync(outputPath, 'utf8'));
});
```
---
### 2. Calculate Overall Risk Level
**Risk hierarchy:** HIGH > MEDIUM > LOW > NONE
```javascript
const riskLevels = { HIGH: 3, MEDIUM: 2, LOW: 1, NONE: 0 };
const domainRisks = domains.map((d) => assessments[d].risk_level);
const maxRiskValue = Math.max(...domainRisks.map((r) => riskLevels[r]));
const overallRisk = Object.keys(riskLevels).find((k) => riskLevels[k] === maxRiskValue);
```
**Risk assessment:**
- If ANY domain is HIGH → overall is HIGH
- If ANY domain is MEDIUM (and none HIGH) → overall is MEDIUM
- If ALL domains are LOW/NONE → overall is LOW
---
### 3. Aggregate Compliance Status
```javascript
const allCompliance = {};
domains.forEach((domain) => {
const compliance = assessments[domain].compliance;
Object.entries(compliance).forEach(([standard, status]) => {
if (!allCompliance[standard]) {
allCompliance[standard] = [];
}
allCompliance[standard].push({ domain, status });
});
});
// Determine overall compliance per standard
const complianceSummary = {};
Object.entries(allCompliance).forEach(([standard, statuses]) => {
const hasFail = statuses.some((s) => s.status === 'FAIL');
const hasPartial = statuses.some((s) => s.status === 'PARTIAL' || s.status === 'CONCERN');
complianceSummary[standard] = hasFail ? 'FAIL' : hasPartial ? 'PARTIAL' : 'PASS';
});
```
---
### 4. Identify Cross-Domain Risks
**Look for risks that span multiple domains:**
```javascript
const crossDomainRisks = [];
// Example: Performance + Scalability issue
const perfConcerns = assessments.performance.findings.filter((f) => f.status !== 'PASS');
const scaleConcerns = assessments.scalability.findings.filter((f) => f.status !== 'PASS');
if (perfConcerns.length > 0 && scaleConcerns.length > 0) {
crossDomainRisks.push({
domains: ['performance', 'scalability'],
description: 'Performance issues may worsen under scale',
impact: 'HIGH',
});
}
// Example: Security + Reliability issue
const securityFails = assessments.security.findings.filter((f) => f.status === 'FAIL');
const reliabilityConcerns = assessments.reliability.findings.filter((f) => f.status !== 'PASS');
if (securityFails.length > 0 && reliabilityConcerns.length > 0) {
crossDomainRisks.push({
domains: ['security', 'reliability'],
description: 'Security vulnerabilities may cause reliability incidents',
impact: 'CRITICAL',
});
}
```
---
### 5. Aggregate Priority Actions
```javascript
const allPriorityActions = domains.flatMap((domain) =>
assessments[domain].priority_actions.map((action) => ({
domain,
action,
urgency: assessments[domain].risk_level === 'HIGH' ? 'URGENT' : 'NORMAL',
})),
);
// Sort by urgency
const prioritizedActions = allPriorityActions.sort((a, b) => (a.urgency === 'URGENT' ? -1 : 1));
```
---
### 6. Generate Executive Summary
```javascript
const resolvedMode = subagentContext?.execution?.resolvedMode ?? 'unknown';
const subagentExecutionLabel =
resolvedMode === 'sequential'
? 'SEQUENTIAL (4 NFR domains)'
: resolvedMode === 'agent-team'
? 'AGENT-TEAM (4 NFR domains)'
: resolvedMode === 'subagent'
? 'SUBAGENT (4 NFR domains)'
: 'MODE-DEPENDENT (4 NFR domains)';
const performanceGainLabel =
resolvedMode === 'sequential'
? 'baseline (no parallel speedup)'
: resolvedMode === 'agent-team' || resolvedMode === 'subagent'
? '~67% faster than sequential'
: 'mode-dependent';
const executiveSummary = {
overall_risk: overallRisk,
assessment_date: new Date().toISOString(),
domain_assessments: assessments,
compliance_summary: complianceSummary,
cross_domain_risks: crossDomainRisks,
priority_actions: prioritizedActions,
risk_breakdown: {
security: assessments.security.risk_level,
performance: assessments.performance.risk_level,
reliability: assessments.reliability.risk_level,
scalability: assessments.scalability.risk_level,
},
subagent_execution: subagentExecutionLabel,
performance_gain: performanceGainLabel,
};
// Save for Step 5 (report generation)
fs.writeFileSync('/tmp/tea-nfr-summary-{{timestamp}}.json', JSON.stringify(executiveSummary, null, 2), 'utf8');
```
---
### 7. Display Summary to User
```
✅ NFR Assessment Complete ({subagentExecutionLabel})
🎯 Overall Risk Level: {overallRisk}
📊 Domain Risk Breakdown:
- Security: {security_risk}
- Performance: {performance_risk}
- Reliability: {reliability_risk}
- Scalability: {scalability_risk}
✅ Compliance Summary:
{list standards with PASS/PARTIAL/FAIL}
⚠️ Cross-Domain Risks: {cross_domain_risk_count}
🎯 Priority Actions: {priority_action_count}
🚀 Performance: {performanceGainLabel}
✅ Ready for report generation (Step 5)
```
---
---
### 8. Save Progress
**Save this step's accumulated work to `{outputFile}`.**
- **If `{outputFile}` does not exist** (first save), create it using the workflow template (if available) with YAML frontmatter:
```yaml
---
stepsCompleted: ['step-04e-aggregate-nfr']
lastStep: 'step-04e-aggregate-nfr'
lastSaved: '{date}'
---
```
Then write this step's output below the frontmatter.
- **If `{outputFile}` already exists**, update:
- Add `'step-04e-aggregate-nfr'` to `stepsCompleted` array (only if not already present)
- Set `lastStep: 'step-04e-aggregate-nfr'`
- Set `lastSaved: '{date}'`
- Append this step's output to the appropriate section of the document.
---
## EXIT CONDITION
Proceed to Step 5 when:
- ✅ All subagent outputs read
- ✅ Overall risk calculated
- ✅ Compliance aggregated
- ✅ Summary saved
- ✅ Progress saved to output document
Load next step: `{nextStepFile}`
---
## 🚨 SYSTEM SUCCESS METRICS
### ✅ SUCCESS:
- All 4 NFR domains aggregated correctly
- Overall risk level determined
- Executive summary complete
### ❌ FAILURE:
- Failed to read subagent outputs
- Risk calculation incorrect

View File

@@ -0,0 +1,108 @@
---
name: 'step-05-generate-report'
description: 'Create NFR report and validation summary'
outputFile: '{test_artifacts}/nfr-assessment.md'
---
# Step 5: Generate Report & Validate
## STEP GOAL
Produce the NFR assessment report and validate completeness.
## MANDATORY EXECUTION RULES
- 📖 Read the entire step file before acting
- ✅ Speak in `{communication_language}`
---
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
- 💾 Record outputs before proceeding
- 📖 Load the next step only when instructed
## CONTEXT BOUNDARIES:
- Available context: config, loaded artifacts, and knowledge fragments
- Focus: this step's goal only
- Limits: do not execute future steps
- Dependencies: prior steps' outputs (if any)
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
## 1. Report Generation
Use `nfr-report-template.md` to produce `{outputFile}` containing:
- Category results (PASS/CONCERNS/FAIL)
- Evidence summary
- Remediation actions
- Gate-ready YAML snippet (if applicable)
---
## 2. Polish Output
Before finalizing, review the complete output document for quality:
1. **Remove duplication**: Progressive-append workflow may have created repeated sections — consolidate
2. **Verify consistency**: Ensure terminology, risk scores, and references are consistent throughout
3. **Check completeness**: All template sections should be populated or explicitly marked N/A
4. **Format cleanup**: Ensure markdown formatting is clean (tables aligned, headers consistent, no orphaned references)
---
## 3. Validation
Validate against `checklist.md` and fix gaps.
- [ ] CLI sessions cleaned up (no orphaned browsers)
---
## 4. Save Progress
**Save this step's accumulated work to `{outputFile}`.**
- **If `{outputFile}` does not exist** (first save), create it using the workflow template (if available) with YAML frontmatter:
```yaml
---
stepsCompleted: ['step-05-generate-report']
lastStep: 'step-05-generate-report'
lastSaved: '{date}'
---
```
Then write this step's output below the frontmatter.
- **If `{outputFile}` already exists**, update:
- Add `'step-05-generate-report'` to `stepsCompleted` array (only if not already present)
- Set `lastStep: 'step-05-generate-report'`
- Set `lastSaved: '{date}'`
- Append this step's output to the appropriate section of the document.
---
## 5. Completion Summary
Report:
- Overall NFR status
- Critical blockers or waivers needed
- Next recommended workflow (`trace` or release gate)
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Step completed in full with required outputs
### ❌ SYSTEM FAILURE:
- Skipped sequence steps or missing outputs
**Master Rule:** Skipping steps is FORBIDDEN.

View File

@@ -0,0 +1,65 @@
---
name: 'step-01-assess'
description: 'Load an existing output for editing'
nextStepFile: './step-02-apply-edit.md'
---
# Step 1: Assess Edit Target
## STEP GOAL:
Identify which output should be edited and load it.
## MANDATORY EXECUTION RULES (READ FIRST):
### Universal Rules:
- 📖 Read the complete step file before taking any action
- ✅ Speak in `{communication_language}`
### Role Reinforcement:
- ✅ You are the Master Test Architect
### Step-Specific Rules:
- 🎯 Ask the user which output file to edit
- 🚫 Do not edit until target is confirmed
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
## CONTEXT BOUNDARIES:
- Available context: existing outputs
- Focus: select edit target
- Limits: no edits yet
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly.
### 1. Identify Target
Ask the user to provide the output file path or select from known outputs.
### 2. Load Target
Read the provided output file in full.
### 3. Confirm
Confirm the target and proceed to edit.
Load next step: `{nextStepFile}`
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Target identified and loaded
### ❌ SYSTEM FAILURE:
- Proceeding without a confirmed target

View File

@@ -0,0 +1,60 @@
---
name: 'step-02-apply-edit'
description: 'Apply edits to the selected output'
---
# Step 2: Apply Edits
## STEP GOAL:
Apply the requested edits to the selected output and confirm changes.
## MANDATORY EXECUTION RULES (READ FIRST):
### Universal Rules:
- 📖 Read the complete step file before taking any action
- ✅ Speak in `{communication_language}`
### Role Reinforcement:
- ✅ You are the Master Test Architect
### Step-Specific Rules:
- 🎯 Only apply edits explicitly requested by the user
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
## CONTEXT BOUNDARIES:
- Available context: selected output and user changes
- Focus: apply edits only
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly.
### 1. Confirm Requested Changes
Restate what will be changed and confirm.
### 2. Apply Changes
Update the output file accordingly.
### 3. Report
Summarize the edits applied.
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Changes applied and confirmed
### ❌ SYSTEM FAILURE:
- Unconfirmed edits or missing update

View File

@@ -0,0 +1,67 @@
---
name: 'step-01-validate'
description: 'Validate workflow outputs against checklist'
outputFile: '{test_artifacts}/nfr-assess-validation-report.md'
validationChecklist: '../checklist.md'
---
# Step 1: Validate Outputs
## STEP GOAL:
Validate outputs using the workflow checklist and record findings.
## MANDATORY EXECUTION RULES (READ FIRST):
### Universal Rules:
- 📖 Read the complete step file before taking any action
- ✅ Speak in `{communication_language}`
### Role Reinforcement:
- ✅ You are the Master Test Architect
### Step-Specific Rules:
- 🎯 Validate against `{validationChecklist}`
- 🚫 Do not skip checks
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
- 💾 Write findings to `{outputFile}`
## CONTEXT BOUNDARIES:
- Available context: workflow outputs and checklist
- Focus: validation only
- Limits: do not modify outputs in this step
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly.
### 1. Load Checklist
Read `{validationChecklist}` and list all criteria.
### 2. Validate Outputs
Evaluate outputs against each checklist item.
### 3. Write Report
Write a validation report to `{outputFile}` with PASS/WARN/FAIL per section.
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Validation report written
- All checklist items evaluated
### ❌ SYSTEM FAILURE:
- Skipped checklist items
- No report produced

View File

@@ -0,0 +1,73 @@
---
validationDate: 2026-01-27
workflowName: testarch-nfr
workflowPath: {project-root}/src/workflows/testarch/nfr-assess
validationStatus: COMPLETE
completionDate: 2026-01-27 10:03:10
---
# Validation Report: testarch-nfr
**Validation Started:** 2026-01-27 09:50:21
**Validator:** BMAD Workflow Validation System (Codex)
**Standards Version:** BMAD Workflow Standards
## File Structure & Size
- workflow.md present: YES
- instructions.md present: YES
- workflow.yaml present: YES
- step files found: 8
**Step File Sizes:**
- steps-c/step-01-load-context.md: 78 lines [GOOD]
- steps-c/step-02-define-thresholds.md: 75 lines [GOOD]
- steps-c/step-03-gather-evidence.md: 58 lines [GOOD]
- steps-c/step-04-evaluate-and-score.md: 61 lines [GOOD]
- steps-c/step-05-generate-report.md: 64 lines [GOOD]
- steps-e/step-01-assess.md: 51 lines [GOOD]
- steps-e/step-02-apply-edit.md: 46 lines [GOOD]
- steps-v/step-01-validate.md: 53 lines [GOOD]
- workflow-plan.md present: YES
## Frontmatter Validation
- No frontmatter violations found
## Critical Path Violations
- No {project-root} hardcoded paths detected in body
- No dead relative links detected
## Menu Handling Validation
- No menu structures detected (linear step flow) [N/A]
## Step Type Validation
- Last step steps-v/step-01-validate.md has no nextStepFile (final step OK)
- Step type validation assumes linear sequence (no branching/menu). Workflow-plan.md present for reference. [INFO]
## Output Format Validation
- Templates present: nfr-report-template.md
- Steps with outputFile in frontmatter:
- steps-c/step-05-generate-report.md
- steps-v/step-01-validate.md
## Validation Design Check
- checklist.md present: YES
- Validation steps folder (steps-v) present: YES
## Instruction Style Check
- All steps include STEP GOAL, MANDATORY EXECUTION RULES, EXECUTION PROTOCOLS, CONTEXT BOUNDARIES, and SUCCESS/FAILURE metrics
## Summary
- Validation completed: 2026-01-27 10:03:10
- Critical issues: 0
- Warnings: 0 (informational notes only)
- Readiness: READY (manual review optional)

View File

@@ -0,0 +1,116 @@
---
validationDate: 2026-01-27
workflowName: testarch-nfr
workflowPath: {project-root}/src/workflows/testarch/nfr-assess
validationStatus: COMPLETE
completionDate: 2026-01-27 10:24:01
---
# Validation Report: testarch-nfr
**Validation Started:** 2026-01-27 10:24:01
**Validator:** BMAD Workflow Validation System (Codex)
**Standards Version:** BMAD Workflow Standards
## File Structure & Size
- workflow.md present: YES
- instructions.md present: YES
- workflow.yaml present: YES
- step files found: 8
**Step File Sizes:**
- steps-c/step-01-load-context.md: 77 lines [GOOD]
- steps-c/step-02-define-thresholds.md: 74 lines [GOOD]
- steps-c/step-03-gather-evidence.md: 57 lines [GOOD]
- steps-c/step-04-evaluate-and-score.md: 60 lines [GOOD]
- steps-c/step-05-generate-report.md: 63 lines [GOOD]
- steps-e/step-01-assess.md: 50 lines [GOOD]
- steps-e/step-02-apply-edit.md: 45 lines [GOOD]
- steps-v/step-01-validate.md: 52 lines [GOOD]
- workflow-plan.md present: YES
## Frontmatter Validation
- No frontmatter violations found
## Critical Path Violations
### Config Variables (Exceptions)
Standard BMAD config variables treated as valid exceptions: bmb_creations_output_folder, communication_language, document_output_language, output_folder, planning_artifacts, project-root, project_name, test_artifacts, user_name
- No {project-root} hardcoded paths detected in body
- No dead relative links detected
- No module path assumptions detected
**Status:** ✅ PASS - No critical violations
## Menu Handling Validation
- No menu structures detected (linear step flow) [N/A]
## Step Type Validation
- steps-c/step-01-load-context.md: Init [PASS]
- steps-c/step-02-define-thresholds.md: Middle [PASS]
- steps-c/step-03-gather-evidence.md: Middle [PASS]
- steps-c/step-04-evaluate-and-score.md: Middle [PASS]
- steps-c/step-05-generate-report.md: Final [PASS]
- Step type validation assumes linear sequence (no branching/menu). Workflow-plan.md present for reference. [INFO]
## Output Format Validation
- Templates present: nfr-report-template.md
- Steps with outputFile in frontmatter:
- steps-c/step-05-generate-report.md
- steps-v/step-01-validate.md
- checklist.md present: YES
## Validation Design Check
- Validation steps folder (steps-v) present: YES
- Validation step(s) present: step-01-validate.md
- Validation steps reference checklist data and auto-proceed
## Instruction Style Check
- Instruction style: Prescriptive (appropriate for TEA quality/compliance workflows)
- Steps emphasize mandatory sequence, explicit success/failure metrics, and risk-based guidance
## Collaborative Experience Check
- Overall facilitation quality: GOOD
- Steps use progressive prompts and clear role reinforcement; no laundry-list interrogation detected
- Flow progression is clear and aligned to workflow goals
## Subagent Optimization Opportunities
- No high-priority subagent optimizations identified; workflow already uses step-file architecture
- Pattern 1 (grep/regex): N/A for most steps
- Pattern 2 (per-file analysis): already aligned to validation structure
- Pattern 3 (data ops): minimal data file loads
- Pattern 4 (parallel): optional for validation only
## Cohesive Review
- Overall assessment: GOOD
- Flow is linear, goals are clear, and outputs map to TEA artifacts
- Voice and tone consistent with Test Architect persona
- Recommendation: READY (minor refinements optional)
## Plan Quality Validation
- Plan file present: workflow-plan.md
- Planned steps found: 8 (all implemented)
- Plan implementation status: Fully Implemented
## Summary
- Validation completed: 2026-01-27 10:24:01
- Critical issues: 0
- Warnings: 0 (informational notes only)
- Readiness: READY (manual review optional)

View File

@@ -0,0 +1,19 @@
# Workflow Plan: testarch-nfr
## Create Mode (steps-c)
- step-01-load-context.md
- step-02-define-thresholds.md
- step-03-gather-evidence.md
- step-04-evaluate-and-score.md
- step-05-generate-report.md
## Validate Mode (steps-v)
- step-01-validate.md
## Edit Mode (steps-e)
- step-01-assess.md
- step-02-apply-edit.md
## Outputs
- {test_artifacts}/nfr-assessment.md

View File

@@ -0,0 +1,41 @@
---
name: testarch-nfr
description: Assess NFRs like performance security and reliability. Use when user says 'lets assess NFRs' or 'I want to evaluate non-functional requirements'
web_bundle: true
---
# Non-Functional Requirements Assessment
**Goal:** Assess non-functional requirements (performance, security, reliability, maintainability) before release with evidence-based validation
**Role:** You are the Master Test Architect.
---
## WORKFLOW ARCHITECTURE
This workflow uses **tri-modal step-file architecture**:
- **Create mode (steps-c/)**: primary execution flow
- **Validate mode (steps-v/)**: validation against checklist
- **Edit mode (steps-e/)**: revise existing outputs
---
## INITIALIZATION SEQUENCE
### 1. Mode Determination
"Welcome to the workflow. What would you like to do?"
- **[C] Create** — Run the workflow
- **[R] Resume** — Resume an interrupted workflow
- **[V] Validate** — Validate existing outputs
- **[E] Edit** — Edit existing outputs
### 2. Route to First Step
- **If C:** Load `steps-c/step-01-load-context.md`
- **If R:** Load `steps-c/step-01b-resume.md`
- **If V:** Load `steps-v/step-01-validate.md`
- **If E:** Load `steps-e/step-01-assess.md`

View File

@@ -0,0 +1,48 @@
# Test Architect workflow: nfr-assess
name: testarch-nfr
# prettier-ignore
description: 'Assess NFRs like performance security and reliability. Use when the user says "lets assess NFRs" or "I want to evaluate non-functional requirements"'
# Critical variables from config
config_source: "{project-root}/_bmad/tea/config.yaml"
output_folder: "{config_source}:output_folder"
test_artifacts: "{config_source}:test_artifacts"
user_name: "{config_source}:user_name"
communication_language: "{config_source}:communication_language"
document_output_language: "{config_source}:document_output_language"
date: system-generated
# Workflow components
installed_path: "{project-root}/_bmad/tea/workflows/testarch/nfr-assess"
instructions: "{installed_path}/instructions.md"
validation: "{installed_path}/checklist.md"
template: "{installed_path}/nfr-report-template.md"
# Variables and inputs
variables:
# NFR category assessment (defaults to all categories)
custom_nfr_categories: "" # Optional additional categories beyond standard (security, performance, reliability, maintainability)
# Output configuration
default_output_file: "{test_artifacts}/nfr-assessment.md"
# Required tools
required_tools:
- read_file # Read story, test results, metrics, logs, BMad artifacts
- write_file # Create NFR assessment, gate YAML, evidence checklist
- list_files # Discover test results, metrics, logs
- search_repo # Find NFR-related tests and evidence
- glob # Find result files matching patterns
tags:
- qa
- nfr
- test-architect
- performance
- security
- reliability
execution_hints:
interactive: false # Minimize prompts
autonomous: true # Proceed without user input unless blocked
iterative: true

View File

@@ -0,0 +1,197 @@
# Teach Me Testing - Quality Checklist
## Workflow Quality Standards
Use this checklist to validate the teaching workflow meets quality standards.
---
## Foundation Quality
- [ ] **workflow.md** exists with proper frontmatter
- [ ] Tri-modal routing logic present (Create/Edit/Validate)
- [ ] Configuration loading references correct module (TEA)
- [ ] First step path correct (`./steps-c/step-01-init.md`)
- [ ] Folder structure complete (steps-c/, steps-e/, steps-v/, data/, templates/)
---
## Template Quality
- [ ] **progress-template.yaml** has complete schema
- [ ] All 7 sessions defined with proper structure
- [ ] Session status tracking fields present (not-started/in-progress/completed)
- [ ] stepsCompleted array for continuation tracking
- [ ] **session-notes-template.md** has all required sections
- [ ] **certificate-template.md** includes all 7 sessions
---
## Step File Quality (CREATE mode)
### Initialization Steps
- [ ] **step-01-init.md** checks for existing progress file
- [ ] Continuation detection logic works correctly
- [ ] **step-01b-continue.md** loads progress and routes to session menu
- [ ] Progress dashboard displays completion status
### Assessment Step
- [ ] **step-02-assess.md** gathers role, experience, goals
- [ ] Validation for role (QA/Dev/Lead/VP)
- [ ] Validation for experience (beginner/intermediate/experienced)
- [ ] Assessment data written to progress file
### Session Menu Hub
- [ ] **step-03-session-menu.md** displays all 7 sessions
- [ ] Completion indicators shown (✓ completed, 🔄 in-progress, ⬜ not-started)
- [ ] Branching logic routes to selected session (1-7)
- [ ] Exit logic (X) routes to completion if all done, otherwise saves and exits
### Session Steps (1-7)
- [ ] Each session loads relevant TEA docs just-in-time
- [ ] Teaching content presented (mostly autonomous)
- [ ] Quiz validation with ≥70% threshold
- [ ] Session notes artifact generated
- [ ] Progress file updated (status, score, artifact path)
- [ ] Returns to session menu hub after completion
### Completion Step
- [ ] **step-05-completion.md** verifies all 7 sessions complete
- [ ] Certificate generated with accurate data
- [ ] Final progress file update (certificate_generated: true)
- [ ] Congratulations message shown
---
## Data File Quality
- [ ] **curriculum.yaml** defines all 7 sessions
- [ ] **role-paths.yaml** maps role customizations
- [ ] **session-content-map.yaml** references TEA docs/fragments/URLs correctly
- [ ] **quiz-questions.yaml** has questions for all sessions
- [ ] **tea-resources-index.yaml** has complete documentation index
---
## Content Quality
### TEA Documentation Integration
- [ ] Local file paths correct (`/docs/*.md`, `/src/testarch/knowledge/*.md`)
- [ ] Online URLs correct (<https://bmad-code-org.github.io/...>)
- [ ] GitHub fragment links correct
- [ ] Triple reference system (local + online + GitHub) implemented
### Role-Based Content
- [ ] QA examples present (practical testing focus)
- [ ] Dev examples present (integration/TDD focus)
- [ ] Lead examples present (architecture/patterns focus)
- [ ] VP examples present (strategy/metrics focus)
### Quiz Quality
- [ ] Questions test understanding, not memorization
- [ ] 3-5 questions per session
- [ ] Mix of difficulty levels
- [ ] Clear correct answers with explanations
---
## Error Handling
- [ ] Corrupted progress file detection
- [ ] Backup and recovery options
- [ ] Missing TEA docs fallback (Web-Browsing)
- [ ] Quiz failure recovery (review or continue)
- [ ] Session interruption handling (auto-save)
---
## User Experience
- [ ] Clear navigation instructions
- [ ] Progress visibility (completion percentage, next recommended)
- [ ] Auto-save after each session
- [ ] Resume capability works seamlessly
- [ ] Exit options clear at all decision points
---
## State Management
- [ ] stepsCompleted array updated correctly
- [ ] Session tracking accurate (status, dates, scores)
- [ ] Completion percentage calculated correctly
- [ ] Next recommended session logic works
- [ ] lastStep and lastContinued timestamps updated
---
## Validation Mode
- [ ] **step-v-01-validate.md** checks all quality standards
- [ ] Generates validation report
- [ ] Identifies issues clearly
- [ ] Provides remediation suggestions
---
## Edit Mode
- [ ] **step-e-01-assess-workflow.md** identifies what to edit
- [ ] **step-e-02-apply-edits.md** applies modifications safely
- [ ] Preserves workflow integrity during edits
---
## Documentation
- [ ] **instructions.md** clear and complete
- [ ] **checklist.md** (this file) comprehensive
- [ ] README (if present) accurate
- [ ] Inline comments in complex logic
---
## Performance
- [ ] Just-in-time loading (not loading all docs upfront)
- [ ] Session steps complete in reasonable time (<5 min)
- [ ] Quiz validation fast (<1 min)
- [ ] Progress file writes efficient
---
## Security
- [ ] No hardcoded credentials
- [ ] File paths use variables
- [ ] Progress files private to user
- [ ] No sensitive data in session notes
---
## Completion Criteria
**Workflow is ready for deployment when:**
- All checkboxes above are checked
- All step files exist and follow standards
- All templates present and correct
- Data files complete and accurate
- Error handling robust
- User experience smooth
- Documentation complete
---
**Validation Date:** **\*\***\_\_\_**\*\***
**Validated By:** **\*\***\_\_\_**\*\***
**Issues Found:** **\*\***\_\_\_**\*\***
**Status:** Ready for Production | Needs Revisions

View File

@@ -0,0 +1,129 @@
# TEA Academy Curriculum Structure
# Defines the 7-session learning path with objectives and content mappings
sessions:
- id: session-01-quickstart
name: "Quick Start"
duration: "30 min"
difficulty: beginner
objective: "Get immediate value by seeing TEA in action"
description: "TEA Lite intro, run automate workflow, understand engagement models"
recommended_for:
- beginner
- intermediate
- experienced
prerequisites: []
- id: session-02-concepts
name: "Core Concepts"
duration: "45 min"
difficulty: beginner
objective: "Understand WHY behind TEA principles"
description: "Risk-based testing, DoD, testing as engineering philosophy"
recommended_for:
- beginner
- intermediate
prerequisites: []
- id: session-03-architecture
name: "Architecture & Patterns"
duration: "60 min"
difficulty: intermediate
objective: "Understand TEA patterns and architecture"
description: "Fixtures, network-first patterns, data factories, step-file architecture"
recommended_for:
- intermediate
- experienced
prerequisites:
- session-02-concepts
- id: session-04-test-design
name: "Test Design"
duration: "60 min"
difficulty: intermediate
objective: "Learn risk assessment and coverage planning"
description: "Test Design workflow, risk/testability assessment, coverage planning"
recommended_for:
- intermediate
- experienced
prerequisites:
- session-02-concepts
- id: session-05-atdd-automate
name: "ATDD & Automate"
duration: "60 min"
difficulty: intermediate
objective: "Generate tests with TDD red-green approach"
description: "ATDD workflow (red phase), Automate workflow, component TDD, API testing"
recommended_for:
- intermediate
- experienced
prerequisites:
- session-02-concepts
- id: session-06-quality-trace
name: "Quality & Trace"
duration: "45 min"
difficulty: intermediate
objective: "Audit quality and ensure traceability"
description: "Test Review (5 dimensions), Trace workflow, quality metrics"
recommended_for:
- intermediate
- experienced
prerequisites:
- session-02-concepts
- id: session-07-advanced
name: "Advanced Patterns"
duration: "ongoing"
difficulty: advanced
objective: "Deep-dive into specific knowledge fragments"
description: "Menu-driven exploration of 35 knowledge fragments organized by category"
recommended_for:
- experienced
prerequisites: []
# Learning Paths by Experience Level
learning_paths:
beginner:
recommended_sequence:
- session-01-quickstart
- session-02-concepts
- session-03-architecture
- session-04-test-design
- session-05-atdd-automate
- session-06-quality-trace
- session-07-advanced
skip_optional: []
intermediate:
recommended_sequence:
- session-01-quickstart
- session-02-concepts
- session-03-architecture
- session-04-test-design
- session-05-atdd-automate
- session-06-quality-trace
- session-07-advanced
skip_optional:
- session-01-quickstart # Can skip if already familiar
certificate_eligible_if_skipped: false
experienced:
recommended_sequence:
- session-02-concepts
- session-03-architecture
- session-04-test-design
- session-05-atdd-automate
- session-06-quality-trace
- session-07-advanced
skip_optional:
- session-01-quickstart
certificate_eligible_if_skipped: false
# Completion Requirements
completion:
minimum_sessions: 7 # All sessions required for certificate
passing_score: 70 # Minimum quiz score to pass session
average_score_threshold: 70 # Minimum average for certificate
certificate_note: "Certificate eligibility requires completion.minimum_sessions. If intermediate.skip_optional or experienced.skip_optional sessions are skipped, certificate eligibility is forfeited."

Some files were not shown because too many files have changed in this diff Show More