2f57ef8948
- AgentHub: 13 files updated with non-engineering examples (content drafts, research, strategy) — engineering stays primary, cross-domain secondary - AgentHub: 7 slash commands, 5 Python scripts, 3 references, 1 agent, dry_run.py validation (57 checks) - Marketplace: agenthub entry added with cross-domain keywords, engineering POWERFUL updated (25→30), product (12→13), counts synced across all configs - SEO: generate-docs.py now produces keyword-rich <title> tags and meta descriptions using SKILL.md frontmatter — "Claude Code Skills" in site_name propagates to all 276 HTML pages - SEO: per-domain title suffixes (Agent Skill for Codex & OpenClaw, etc.), slug-as-title cleanup, domain label stripping from titles - Broken links: 141→0 warnings — new rewrite_skill_internal_links() converts references/, scripts/, assets/ links to GitHub source URLs; skills/index.md phantom slugs fixed (6 marketing, 7 RA/QM) - Counts synced: 204 skills, 266 tools, 382 refs, 16 agents, 17 commands, 21 plugins — consistent across CLAUDE.md, README.md, docs/index.md, marketplace.json, getting-started.md, mkdocs.yml - Platform sync: Codex 163 skills, Gemini 246 items, OpenClaw compatible Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
136 lines
3.3 KiB
Markdown
136 lines
3.3 KiB
Markdown
---
|
|
title: "Smart Test Reporting — Agent Skill & Codex Plugin"
|
|
description: ">-. Agent skill for Claude Code, Codex CLI, Gemini CLI, OpenClaw."
|
|
---
|
|
|
|
# Smart Test Reporting
|
|
|
|
<div class="page-meta" markdown>
|
|
<span class="meta-badge">:material-code-braces: Engineering - Core</span>
|
|
<span class="meta-badge">:material-identifier: `report`</span>
|
|
<span class="meta-badge">:material-github: <a href="https://github.com/alirezarezvani/claude-skills/tree/main/engineering-team/playwright-pro/skills/report/SKILL.md">Source</a></span>
|
|
</div>
|
|
|
|
<div class="install-banner" markdown>
|
|
<span class="install-label">Install:</span> <code>claude /plugin install engineering-skills</code>
|
|
</div>
|
|
|
|
|
|
Generate test reports that plug into the user's existing workflow. Zero new tools.
|
|
|
|
## Steps
|
|
|
|
### 1. Run Tests (If Not Already Run)
|
|
|
|
Check if recent test results exist:
|
|
|
|
```bash
|
|
ls -la test-results/ playwright-report/ 2>/dev/null
|
|
```
|
|
|
|
If no recent results, run tests:
|
|
|
|
```bash
|
|
npx playwright test --reporter=json,html,list 2>&1 | tee test-output.log
|
|
```
|
|
|
|
### 2. Parse Results
|
|
|
|
Read the JSON report:
|
|
|
|
```bash
|
|
npx playwright test --reporter=json 2> /dev/null
|
|
```
|
|
|
|
Extract:
|
|
- Total tests, passed, failed, skipped, flaky
|
|
- Duration per test and total
|
|
- Failed test names with error messages
|
|
- Flaky tests (passed on retry)
|
|
|
|
### 3. Detect Report Destination
|
|
|
|
Check what's configured and route automatically:
|
|
|
|
| Check | If found | Action |
|
|
|---|---|---|
|
|
| `TESTRAIL_URL` env var | TestRail configured | Push results via `/pw:testrail push` |
|
|
| `SLACK_WEBHOOK_URL` env var | Slack configured | Post summary to Slack |
|
|
| `.github/workflows/` | GitHub Actions | Results go to PR comment via artifacts |
|
|
| `playwright-report/` | HTML reporter | Open or serve the report |
|
|
| None of the above | Default | Generate markdown report |
|
|
|
|
### 4. Generate Report
|
|
|
|
#### Markdown Report (Always Generated)
|
|
|
|
```markdown
|
|
# Test Results — {{date}}
|
|
|
|
## Summary
|
|
- ✅ Passed: {{passed}}
|
|
- ❌ Failed: {{failed}}
|
|
- ⏭️ Skipped: {{skipped}}
|
|
- 🔄 Flaky: {{flaky}}
|
|
- ⏱️ Duration: {{duration}}
|
|
|
|
## Failed Tests
|
|
| Test | Error | File |
|
|
|---|---|---|
|
|
| {{name}} | {{error}} | {{file}}:{{line}} |
|
|
|
|
## Flaky Tests
|
|
| Test | Retries | File |
|
|
|---|---|---|
|
|
| {{name}} | {{retries}} | {{file}} |
|
|
|
|
## By Project
|
|
| Browser | Passed | Failed | Duration |
|
|
|---|---|---|---|
|
|
| Chromium | X | Y | Zs |
|
|
| Firefox | X | Y | Zs |
|
|
| WebKit | X | Y | Zs |
|
|
```
|
|
|
|
Save to `test-reports/{{date}}-report.md`.
|
|
|
|
#### Slack Summary (If Webhook Configured)
|
|
|
|
```bash
|
|
curl -X POST "$SLACK_WEBHOOK_URL" \
|
|
-H 'Content-Type: application/json' \
|
|
-d '{
|
|
"text": "🧪 Test Results: ✅ {{passed}} | ❌ {{failed}} | ⏱️ {{duration}}\n{{failed_details}}"
|
|
}'
|
|
```
|
|
|
|
#### TestRail Push (If Configured)
|
|
|
|
Invoke `/pw:testrail push` with the JSON results.
|
|
|
|
#### HTML Report
|
|
|
|
```bash
|
|
npx playwright show-report
|
|
```
|
|
|
|
Or if in CI:
|
|
```bash
|
|
echo "HTML report available at: playwright-report/index.html"
|
|
```
|
|
|
|
### 5. Trend Analysis (If Historical Data Exists)
|
|
|
|
If previous reports exist in `test-reports/`:
|
|
- Compare pass rate over time
|
|
- Identify tests that became flaky recently
|
|
- Highlight new failures vs. recurring failures
|
|
|
|
## Output
|
|
|
|
- Summary with pass/fail/skip/flaky counts
|
|
- Failed test details with error messages
|
|
- Report destination confirmation
|
|
- Trend comparison (if historical data available)
|
|
- Next action recommendation (fix failures or celebrate green)
|