670930c69d
- Add CSS components: .page-meta badges, .domain-header, .install-banner - Fix invisible tab navigation (explicit color for light/dark modes) - Rewrite generate-docs.py with design system templates - Domain indexes: centered headers with icons, install banners, grid cards - Skill pages: pill badges (domain, skill ID, source), install commands - Agent/command pages: type badges with domain icons - Regenerate all 210 pages (180 skills + 15 agents + 15 commands) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
122 lines
3.4 KiB
Markdown
122 lines
3.4 KiB
Markdown
---
|
|
title: "Fix Failing or Flaky Tests"
|
|
description: "Fix Failing or Flaky Tests - Claude Code skill from the Engineering - Core domain."
|
|
---
|
|
|
|
# Fix Failing or Flaky Tests
|
|
|
|
<div class="page-meta" markdown>
|
|
<span class="meta-badge">:material-code-braces: Engineering - Core</span>
|
|
<span class="meta-badge">:material-identifier: `fix`</span>
|
|
<span class="meta-badge">:material-github: <a href="https://github.com/alirezarezvani/claude-skills/tree/main/engineering-team/playwright-pro/skills/fix/SKILL.md">Source</a></span>
|
|
</div>
|
|
|
|
<div class="install-banner" markdown>
|
|
<span class="install-label">Install:</span> <code>claude /plugin install engineering-skills</code>
|
|
</div>
|
|
|
|
|
|
Diagnose and fix a Playwright test that fails or passes intermittently using a systematic taxonomy.
|
|
|
|
## Input
|
|
|
|
`$ARGUMENTS` contains:
|
|
- A test file path: `e2e/login.spec.ts`
|
|
- A test name: ""should redirect after login"`
|
|
- A description: `"the checkout test fails in CI but passes locally"`
|
|
|
|
## Steps
|
|
|
|
### 1. Reproduce the Failure
|
|
|
|
Run the test to capture the error:
|
|
|
|
```bash
|
|
npx playwright test <file> --reporter=list
|
|
```
|
|
|
|
If the test passes, it's likely flaky. Run burn-in:
|
|
|
|
```bash
|
|
npx playwright test <file> --repeat-each=10 --reporter=list
|
|
```
|
|
|
|
If it still passes, try with parallel workers:
|
|
|
|
```bash
|
|
npx playwright test --fully-parallel --workers=4 --repeat-each=5
|
|
```
|
|
|
|
### 2. Capture Trace
|
|
|
|
Run with full tracing:
|
|
|
|
```bash
|
|
npx playwright test <file> --trace=on --retries=0
|
|
```
|
|
|
|
Read the trace output. Use `/debug` to analyze trace files if available.
|
|
|
|
### 3. Categorize the Failure
|
|
|
|
Load `flaky-taxonomy.md` from this skill directory.
|
|
|
|
Every failing test falls into one of four categories:
|
|
|
|
| Category | Symptom | Diagnosis |
|
|
|---|---|---|
|
|
| **Timing/Async** | Fails intermittently everywhere | `--repeat-each=20` reproduces locally |
|
|
| **Test Isolation** | Fails in suite, passes alone | `--workers=1 --grep "test name"` passes |
|
|
| **Environment** | Fails in CI, passes locally | Compare CI vs local screenshots/traces |
|
|
| **Infrastructure** | Random, no pattern | Error references browser internals |
|
|
|
|
### 4. Apply Targeted Fix
|
|
|
|
**Timing/Async:**
|
|
- Replace `waitForTimeout()` with web-first assertions
|
|
- Add `await` to missing Playwright calls
|
|
- Wait for specific network responses before asserting
|
|
- Use `toBeVisible()` before interacting with elements
|
|
|
|
**Test Isolation:**
|
|
- Remove shared mutable state between tests
|
|
- Create test data per-test via API or fixtures
|
|
- Use unique identifiers (timestamps, random strings) for test data
|
|
- Check for database state leaks
|
|
|
|
**Environment:**
|
|
- Match viewport sizes between local and CI
|
|
- Account for font rendering differences in screenshots
|
|
- Use `docker` locally to match CI environment
|
|
- Check for timezone-dependent assertions
|
|
|
|
**Infrastructure:**
|
|
- Increase timeout for slow CI runners
|
|
- Add retries in CI config (`retries: 2`)
|
|
- Check for browser OOM (reduce parallel workers)
|
|
- Ensure browser dependencies are installed
|
|
|
|
### 5. Verify the Fix
|
|
|
|
Run the test 10 times to confirm stability:
|
|
|
|
```bash
|
|
npx playwright test <file> --repeat-each=10 --reporter=list
|
|
```
|
|
|
|
All 10 must pass. If any fail, go back to step 3.
|
|
|
|
### 6. Prevent Recurrence
|
|
|
|
Suggest:
|
|
- Add to CI with `retries: 2` if not already
|
|
- Enable `trace: 'on-first-retry'` in config
|
|
- Add the fix pattern to project's test conventions doc
|
|
|
|
## Output
|
|
|
|
- Root cause category and specific issue
|
|
- The fix applied (with diff)
|
|
- Verification result (10/10 passes)
|
|
- Prevention recommendation
|