219c1c3dfd
- generate-docs.py: extract_description_from_frontmatter() now handles multi-line YAML block scalars (|, >, indented continuation) — fixes 14 pages that had 56-65 char truncated descriptions - mkdocs.yml: add epic-design and research-summarizer to nav (orphan pages) - Regenerated 251 pages, rebuilt sitemap (278 URLs) - SEO audit: 0 broken links, 17→3 short descriptions, 278/278 pages have "Claude Code Skills" in <title> Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
122 lines
3.5 KiB
Markdown
122 lines
3.5 KiB
Markdown
---
|
|
title: "Fix Failing or Flaky Tests — Agent Skill & Codex Plugin"
|
|
description: "Fix failing or flaky Playwright tests. Use when user says 'fix test', 'flaky test', 'test failing', 'debug test', 'test broken', 'test passes. Agent skill for Claude Code, Codex CLI, Gemini CLI, OpenClaw."
|
|
---
|
|
|
|
# Fix Failing or Flaky Tests
|
|
|
|
<div class="page-meta" markdown>
|
|
<span class="meta-badge">:material-code-braces: Engineering - Core</span>
|
|
<span class="meta-badge">:material-identifier: `fix`</span>
|
|
<span class="meta-badge">:material-github: <a href="https://github.com/alirezarezvani/claude-skills/tree/main/engineering-team/playwright-pro/skills/fix/SKILL.md">Source</a></span>
|
|
</div>
|
|
|
|
<div class="install-banner" markdown>
|
|
<span class="install-label">Install:</span> <code>claude /plugin install engineering-skills</code>
|
|
</div>
|
|
|
|
|
|
Diagnose and fix a Playwright test that fails or passes intermittently using a systematic taxonomy.
|
|
|
|
## Input
|
|
|
|
`$ARGUMENTS` contains:
|
|
- A test file path: `e2e/login.spec.ts`
|
|
- A test name: ""should redirect after login"`
|
|
- A description: `"the checkout test fails in CI but passes locally"`
|
|
|
|
## Steps
|
|
|
|
### 1. Reproduce the Failure
|
|
|
|
Run the test to capture the error:
|
|
|
|
```bash
|
|
npx playwright test <file> --reporter=list
|
|
```
|
|
|
|
If the test passes, it's likely flaky. Run burn-in:
|
|
|
|
```bash
|
|
npx playwright test <file> --repeat-each=10 --reporter=list
|
|
```
|
|
|
|
If it still passes, try with parallel workers:
|
|
|
|
```bash
|
|
npx playwright test --fully-parallel --workers=4 --repeat-each=5
|
|
```
|
|
|
|
### 2. Capture Trace
|
|
|
|
Run with full tracing:
|
|
|
|
```bash
|
|
npx playwright test <file> --trace=on --retries=0
|
|
```
|
|
|
|
Read the trace output. Use `/debug` to analyze trace files if available.
|
|
|
|
### 3. Categorize the Failure
|
|
|
|
Load `flaky-taxonomy.md` from this skill directory.
|
|
|
|
Every failing test falls into one of four categories:
|
|
|
|
| Category | Symptom | Diagnosis |
|
|
|---|---|---|
|
|
| **Timing/Async** | Fails intermittently everywhere | `--repeat-each=20` reproduces locally |
|
|
| **Test Isolation** | Fails in suite, passes alone | `--workers=1 --grep "test name"` passes |
|
|
| **Environment** | Fails in CI, passes locally | Compare CI vs local screenshots/traces |
|
|
| **Infrastructure** | Random, no pattern | Error references browser internals |
|
|
|
|
### 4. Apply Targeted Fix
|
|
|
|
**Timing/Async:**
|
|
- Replace `waitForTimeout()` with web-first assertions
|
|
- Add `await` to missing Playwright calls
|
|
- Wait for specific network responses before asserting
|
|
- Use `toBeVisible()` before interacting with elements
|
|
|
|
**Test Isolation:**
|
|
- Remove shared mutable state between tests
|
|
- Create test data per-test via API or fixtures
|
|
- Use unique identifiers (timestamps, random strings) for test data
|
|
- Check for database state leaks
|
|
|
|
**Environment:**
|
|
- Match viewport sizes between local and CI
|
|
- Account for font rendering differences in screenshots
|
|
- Use `docker` locally to match CI environment
|
|
- Check for timezone-dependent assertions
|
|
|
|
**Infrastructure:**
|
|
- Increase timeout for slow CI runners
|
|
- Add retries in CI config (`retries: 2`)
|
|
- Check for browser OOM (reduce parallel workers)
|
|
- Ensure browser dependencies are installed
|
|
|
|
### 5. Verify the Fix
|
|
|
|
Run the test 10 times to confirm stability:
|
|
|
|
```bash
|
|
npx playwright test <file> --repeat-each=10 --reporter=list
|
|
```
|
|
|
|
All 10 must pass. If any fail, go back to step 3.
|
|
|
|
### 6. Prevent Recurrence
|
|
|
|
Suggest:
|
|
- Add to CI with `retries: 2` if not already
|
|
- Enable `trace: 'on-first-retry'` in config
|
|
- Add the fix pattern to project's test conventions doc
|
|
|
|
## Output
|
|
|
|
- Root cause category and specific issue
|
|
- The fix applied (with diff)
|
|
- Verification result (10/10 passes)
|
|
- Prevention recommendation
|