feat: add fact-checker skill v1.0.0
Add comprehensive fact-checking skill that verifies claims using web search and official sources, then proposes corrections with user confirmation. Features: - 5-step workflow: identify → search → compare → report → apply - Supports AI model specs, technical docs, statistics, general facts - Source evaluation framework prioritizing official documentation - Auto-correction with mandatory user approval gates - Temporal context to prevent information decay Real-world usage: - Successfully updated AI model specs (Claude, GPT, Gemini) - Corrected outdated version numbers and context windows - Added temporal markers for time-sensitive information Marketplace updates: - Bumped version to 1.19.0 - Added fact-checker to plugins list - Updated metadata description 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
This commit is contained in:
daymade
4
fact-checker/.security-scan-passed
Normal file
4
fact-checker/.security-scan-passed
Normal file
@@ -0,0 +1,4 @@
|
||||
Security scan passed
|
||||
Scanned at: 2026-01-05T23:29:12.688630
|
||||
Tool: gitleaks + pattern-based validation
|
||||
Content hash: f0ee8b5411aeb2d3058383ae536393e4e71e0fe0c240d0798a50aa10399870d4
|
||||
188
fact-checker/README.md
Normal file
188
fact-checker/README.md
Normal file
@@ -0,0 +1,188 @@
|
||||
# Fact Checker
|
||||
|
||||
Verify factual claims in documents using web search and official sources, then apply corrections with user confirmation.
|
||||
|
||||
## Features
|
||||
|
||||
- ✅ Comprehensive fact verification across multiple domains
|
||||
- 🔍 Searches authoritative sources (official docs, API specs, academic papers)
|
||||
- 📊 Generates detailed correction reports with sources
|
||||
- 🤖 Auto-applies corrections after user approval
|
||||
- 🕐 Adds temporal context to prevent information decay
|
||||
|
||||
## Supported Claim Types
|
||||
|
||||
- **AI Model Specifications**: Context windows, pricing, features, benchmarks
|
||||
- **Technical Documentation**: API capabilities, version numbers, library features
|
||||
- **Statistical Data**: Metrics, benchmark scores, performance data
|
||||
- **General Facts**: Any verifiable factual statement
|
||||
|
||||
## Usage Examples
|
||||
|
||||
### Example 1: Update Outdated AI Model Info
|
||||
|
||||
```
|
||||
User: Fact-check the AI model specifications in section 2.1
|
||||
```
|
||||
|
||||
**What happens:**
|
||||
1. Identifies claims: "Claude 3.5 Sonnet: 200K tokens", "GPT-4o: 128K tokens"
|
||||
2. Searches official documentation for current models
|
||||
3. Finds: Claude Sonnet 4.5, GPT-5.2 with updated specs
|
||||
4. Generates correction report with sources
|
||||
5. Applies fixes after user confirms
|
||||
|
||||
### Example 2: Verify Technical Claims
|
||||
|
||||
```
|
||||
User: Check if these library versions are still current
|
||||
```
|
||||
|
||||
**What happens:**
|
||||
1. Extracts version numbers from document
|
||||
2. Checks package registries (npm, PyPI, etc.)
|
||||
3. Identifies outdated versions
|
||||
4. Suggests updates with changelog references
|
||||
|
||||
### Example 3: Validate Statistics
|
||||
|
||||
```
|
||||
User: Verify the benchmark scores in this section
|
||||
```
|
||||
|
||||
**What happens:**
|
||||
1. Identifies numerical claims and metrics
|
||||
2. Searches official benchmark publications
|
||||
3. Compares document values vs. source data
|
||||
4. Flags discrepancies with authoritative links
|
||||
|
||||
## Workflow
|
||||
|
||||
The skill follows a 5-step process:
|
||||
|
||||
```
|
||||
Fact-checking Progress:
|
||||
- [ ] Step 1: Identify factual claims
|
||||
- [ ] Step 2: Search authoritative sources
|
||||
- [ ] Step 3: Compare claims against sources
|
||||
- [ ] Step 4: Generate correction report
|
||||
- [ ] Step 5: Apply corrections with user approval
|
||||
```
|
||||
|
||||
## Source Evaluation
|
||||
|
||||
**Preferred sources (in order):**
|
||||
1. Official product pages and documentation
|
||||
2. API documentation and developer guides
|
||||
3. Official blog announcements
|
||||
4. GitHub releases (for open source)
|
||||
|
||||
**Use with caution:**
|
||||
- Third-party aggregators (verify against official sources)
|
||||
- Blog posts and articles (cross-reference)
|
||||
|
||||
**Avoid:**
|
||||
- Outdated documentation
|
||||
- Unofficial wikis without citations
|
||||
- Speculation and rumors
|
||||
|
||||
## Real-World Example
|
||||
|
||||
**Before:**
|
||||
```markdown
|
||||
AI 大模型的"上下文窗口"不断升级：
|
||||
- Claude 3.5 Sonnet: 200K tokens（约 15 万汉字）
|
||||
- GPT-4o: 128K tokens（约 10 万汉字）
|
||||
- Gemini 1.5 Pro: 2M tokens（约 150 万汉字）
|
||||
```
|
||||
|
||||
**After fact-checking:**
|
||||
```markdown
|
||||
AI 大模型的"上下文窗口"不断升级（截至 2026 年 1 月）：
|
||||
- Claude Sonnet 4.5: 200K tokens（约 15 万汉字）
|
||||
- GPT-5.2: 400K tokens（约 30 万汉字）
|
||||
- Gemini 3 Pro: 1M tokens（约 75 万汉字）
|
||||
```
|
||||
|
||||
**Changes made:**
|
||||
- ✅ Updated Claude 3.5 Sonnet → Claude Sonnet 4.5
|
||||
- ✅ Corrected GPT-4o (128K) → GPT-5.2 (400K)
|
||||
- ✅ Fixed Gemini 1.5 Pro (2M) → Gemini 3 Pro (1M)
|
||||
- ✅ Added temporal marker "截至 2026 年 1 月"
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
# Via CCPM (recommended)
|
||||
ccpm install @daymade-skills/fact-checker
|
||||
|
||||
# Manual installation
|
||||
Download fact-checker.zip and install through Claude Code
|
||||
```
|
||||
|
||||
## Trigger Keywords
|
||||
|
||||
The skill activates when you mention:
|
||||
- "fact-check this document"
|
||||
- "verify these claims"
|
||||
- "check if this is accurate"
|
||||
- "update outdated information"
|
||||
- "validate the data"
|
||||
|
||||
## Configuration
|
||||
|
||||
No configuration required. The skill works out of the box.
|
||||
|
||||
## Limitations
|
||||
|
||||
**Cannot verify:**
|
||||
- Subjective opinions or judgments
|
||||
- Future predictions or specifications
|
||||
- Claims requiring paywalled sources
|
||||
- Disputed facts without authoritative consensus
|
||||
|
||||
**For such cases**, the skill will:
|
||||
- Note the limitation in the report
|
||||
- Suggest qualification language
|
||||
- Recommend user research or expert consultation
|
||||
|
||||
## Best Practices
|
||||
|
||||
### For Authors
|
||||
|
||||
1. **Run regularly**: Fact-check documents periodically to catch outdated info
|
||||
2. **Include dates**: Add temporal markers like "as of [date]" to claims
|
||||
3. **Cite sources**: Keep original source links for future verification
|
||||
4. **Review reports**: Always review the correction report before applying changes
|
||||
|
||||
### For Fact-Checking
|
||||
|
||||
1. **Be specific**: Target specific sections rather than entire books
|
||||
2. **Verify critical claims first**: Prioritize high-impact information
|
||||
3. **Cross-reference**: For important claims, verify across multiple sources
|
||||
4. **Update regularly**: Technical specs change frequently - recheck periodically
|
||||
|
||||
## Development
|
||||
|
||||
Created with skill-creator v1.2.2 following Anthropic's best practices.
|
||||
|
||||
**Testing:**
|
||||
- Verified on Claude Sonnet 4.5, Opus 4.5, and Haiku 4
|
||||
- Tested with real-world documentation updates
|
||||
- Validated correction workflow with user approval gates
|
||||
|
||||
## Version History
|
||||
|
||||
### 1.0.0 (2026-01-05)
|
||||
- Initial release
|
||||
- Support for AI models, technical docs, statistics
|
||||
- Auto-correction with user approval
|
||||
- Comprehensive source evaluation framework
|
||||
|
||||
## License
|
||||
|
||||
MIT License - See repository for details
|
||||
|
||||
## Contributing
|
||||
|
||||
Issues and pull requests welcome at [daymade/claude-code-skills](https://github.com/daymade/claude-code-skills)
|
||||
283
fact-checker/SKILL.md
Normal file
283
fact-checker/SKILL.md
Normal file
@@ -0,0 +1,283 @@
|
||||
---
|
||||
name: fact-checker
|
||||
description: Verifies factual claims in documents using web search and official sources, then proposes corrections with user confirmation. Use when the user asks to fact-check, verify information, validate claims, check accuracy, or update outdated information in documents. Supports AI model specs, technical documentation, statistics, and general factual statements.
|
||||
---
|
||||
|
||||
# Fact Checker
|
||||
|
||||
Verify factual claims in documents and propose corrections backed by authoritative sources.
|
||||
|
||||
## When to use
|
||||
|
||||
Trigger when users request:
|
||||
- "Fact-check this document"
|
||||
- "Verify these AI model specifications"
|
||||
- "Check if this information is still accurate"
|
||||
- "Update outdated data in this file"
|
||||
- "Validate the claims in this section"
|
||||
|
||||
## Workflow
|
||||
|
||||
Copy this checklist to track progress:
|
||||
|
||||
```
|
||||
Fact-checking Progress:
|
||||
- [ ] Step 1: Identify factual claims
|
||||
- [ ] Step 2: Search authoritative sources
|
||||
- [ ] Step 3: Compare claims against sources
|
||||
- [ ] Step 4: Generate correction report
|
||||
- [ ] Step 5: Apply corrections with user approval
|
||||
```
|
||||
|
||||
### Step 1: Identify factual claims
|
||||
|
||||
Scan the document for verifiable statements:
|
||||
|
||||
**Target claim types:**
|
||||
- Technical specifications (context windows, pricing, features)
|
||||
- Version numbers and release dates
|
||||
- Statistical data and metrics
|
||||
- API capabilities and limitations
|
||||
- Benchmark scores and performance data
|
||||
|
||||
**Skip subjective content:**
|
||||
- Opinions and recommendations
|
||||
- Explanatory prose
|
||||
- Tutorial instructions
|
||||
- Architectural discussions
|
||||
|
||||
### Step 2: Search authoritative sources
|
||||
|
||||
For each claim, search official sources:
|
||||
|
||||
**AI models:**
|
||||
- Official announcement pages (anthropic.com/news, openai.com/index, blog.google)
|
||||
- API documentation (platform.claude.com/docs, platform.openai.com/docs)
|
||||
- Developer guides and release notes
|
||||
|
||||
**Technical libraries:**
|
||||
- Official documentation sites
|
||||
- GitHub repositories (releases, README)
|
||||
- Package registries (npm, PyPI, crates.io)
|
||||
|
||||
**General claims:**
|
||||
- Academic papers and research
|
||||
- Government statistics
|
||||
- Industry standards bodies
|
||||
|
||||
**Search strategy:**
|
||||
- Use model names + specification (e.g., "Claude Opus 4.5 context window")
|
||||
- Include current year for recent information
|
||||
- Verify from multiple sources when possible
|
||||
|
||||
### Step 3: Compare claims against sources
|
||||
|
||||
Create a comparison table:
|
||||
|
||||
| Claim in Document | Source Information | Status | Authoritative Source |
|
||||
|-------------------|-------------------|--------|---------------------|
|
||||
| Claude 3.5 Sonnet: 200K tokens | Claude Sonnet 4.5: 200K tokens | ❌ Outdated model name | platform.claude.com/docs |
|
||||
| GPT-4o: 128K tokens | GPT-5.2: 400K tokens | ❌ Incorrect version & spec | openai.com/index/gpt-5-2 |
|
||||
|
||||
**Status codes:**
|
||||
- ✅ Accurate - claim matches sources
|
||||
- ❌ Incorrect - claim contradicts sources
|
||||
- ⚠️ Outdated - claim was true but superseded
|
||||
- ❓ Unverifiable - no authoritative source found
|
||||
|
||||
### Step 4: Generate correction report
|
||||
|
||||
Present findings in structured format:
|
||||
|
||||
```markdown
|
||||
## Fact-Check Report
|
||||
|
||||
### Summary
|
||||
- Total claims checked: X
|
||||
- Accurate: Y
|
||||
- Issues found: Z
|
||||
|
||||
### Issues Requiring Correction
|
||||
|
||||
#### Issue 1: Outdated AI Model Reference
|
||||
**Location:** Line 77-80 in docs/file.md
|
||||
**Current claim:** "Claude 3.5 Sonnet: 200K tokens"
|
||||
**Correction:** "Claude Sonnet 4.5: 200K tokens"
|
||||
**Source:** https://platform.claude.com/docs/en/build-with-claude/context-windows
|
||||
**Rationale:** Claude 3.5 Sonnet has been superseded by Claude Sonnet 4.5 (released Sept 2025)
|
||||
|
||||
#### Issue 2: Incorrect Context Window
|
||||
**Location:** Line 79 in docs/file.md
|
||||
**Current claim:** "GPT-4o: 128K tokens"
|
||||
**Correction:** "GPT-5.2: 400K tokens"
|
||||
**Source:** https://openai.com/index/introducing-gpt-5-2/
|
||||
**Rationale:** 128K was output limit; context window is 400K. Model also updated to GPT-5.2
|
||||
```
|
||||
|
||||
### Step 5: Apply corrections with user approval
|
||||
|
||||
**Before making changes:**
|
||||
|
||||
1. Show the correction report to the user
|
||||
2. Wait for explicit approval: "Should I apply these corrections?"
|
||||
3. Only proceed after confirmation
|
||||
|
||||
**When applying corrections:**
|
||||
|
||||
```python
|
||||
# Use Edit tool to update document
|
||||
# Example:
|
||||
Edit(
|
||||
file_path="docs/03-写作规范/AI辅助写书方法论.md",
|
||||
old_string="- Claude 3.5 Sonnet: 200K tokens（约 15 万汉字）",
|
||||
new_string="- Claude Sonnet 4.5: 200K tokens（约 15 万汉字）"
|
||||
)
|
||||
```
|
||||
|
||||
**After corrections:**
|
||||
|
||||
1. Verify all edits were applied successfully
|
||||
2. Note the correction summary (e.g., "Updated 4 claims in section 2.1")
|
||||
3. Remind user to commit changes
|
||||
|
||||
## Search best practices
|
||||
|
||||
### Query construction
|
||||
|
||||
**Good queries** (specific, current):
|
||||
- "Claude Opus 4.5 context window 2026"
|
||||
- "GPT-5.2 official release announcement"
|
||||
- "Gemini 3 Pro token limit specifications"
|
||||
|
||||
**Poor queries** (vague, generic):
|
||||
- "Claude context"
|
||||
- "AI models"
|
||||
- "Latest version"
|
||||
|
||||
### Source evaluation
|
||||
|
||||
**Prefer official sources:**
|
||||
1. Product official pages (highest authority)
|
||||
2. API documentation
|
||||
3. Official blog announcements
|
||||
4. GitHub releases (for open source)
|
||||
|
||||
**Use with caution:**
|
||||
- Third-party aggregators (llm-stats.com, etc.) - verify against official sources
|
||||
- Blog posts and articles - cross-reference claims
|
||||
- Social media - only for announcements, verify elsewhere
|
||||
|
||||
**Avoid:**
|
||||
- Outdated documentation
|
||||
- Unofficial wikis without citations
|
||||
- Speculation and rumors
|
||||
|
||||
### Handling ambiguity
|
||||
|
||||
When sources conflict:
|
||||
|
||||
1. Prioritize most recent official documentation
|
||||
2. Note the discrepancy in the report
|
||||
3. Present both sources to the user
|
||||
4. Recommend contacting vendor if critical
|
||||
|
||||
When no source found:
|
||||
|
||||
1. Mark as ❓ Unverifiable
|
||||
2. Suggest alternative phrasing: "According to [Source] as of [Date]..."
|
||||
3. Recommend adding qualification: "approximately", "reported as"
|
||||
|
||||
## Special considerations
|
||||
|
||||
### Time-sensitive information
|
||||
|
||||
Always include temporal context:
|
||||
|
||||
**Good corrections:**
|
||||
- "截至 2026 年 1 月" (As of January 2026)
|
||||
- "Claude Sonnet 4.5 (released September 2025)"
|
||||
|
||||
**Poor corrections:**
|
||||
- "Latest version" (becomes outdated)
|
||||
- "Current model" (ambiguous timeframe)
|
||||
|
||||
### Numerical precision
|
||||
|
||||
Match precision to source:
|
||||
|
||||
**Source says:** "approximately 1 million tokens"
|
||||
**Write:** "1M tokens (approximately)"
|
||||
|
||||
**Source says:** "200,000 token context window"
|
||||
**Write:** "200K tokens" (exact)
|
||||
|
||||
### Citation format
|
||||
|
||||
Include citations in corrections:
|
||||
|
||||
```markdown
|
||||
> **注**：具体上下文窗口以模型官方文档为准，本书写作时使用 Claude Sonnet 4.5 为主要工具。
|
||||
```
|
||||
|
||||
Link to sources when possible.
|
||||
|
||||
## Examples
|
||||
|
||||
### Example 1: Technical specification update
|
||||
|
||||
**User request:** "Fact-check the AI model context windows in section 2.1"
|
||||
|
||||
**Process:**
|
||||
1. Identify claims: Claude 3.5 Sonnet (200K), GPT-4o (128K), Gemini 1.5 Pro (2M)
|
||||
2. Search official docs for current models
|
||||
3. Find: Claude Sonnet 4.5, GPT-5.2, Gemini 3 Pro
|
||||
4. Generate report showing discrepancies
|
||||
5. Apply corrections after approval
|
||||
|
||||
### Example 2: Statistical data verification
|
||||
|
||||
**User request:** "Verify the benchmark scores in chapter 5"
|
||||
|
||||
**Process:**
|
||||
1. Extract numerical claims
|
||||
2. Search for official benchmark publications
|
||||
3. Compare reported vs. source values
|
||||
4. Flag any discrepancies with source links
|
||||
5. Update with verified figures
|
||||
|
||||
### Example 3: Version number validation
|
||||
|
||||
**User request:** "Check if these library versions are still current"
|
||||
|
||||
**Process:**
|
||||
1. List all version numbers mentioned
|
||||
2. Check package registries (npm, PyPI, etc.)
|
||||
3. Identify outdated versions
|
||||
4. Suggest updates with changelog references
|
||||
5. Update after user confirms
|
||||
|
||||
## Quality checklist
|
||||
|
||||
Before completing fact-check:
|
||||
|
||||
- [ ] All factual claims identified and categorized
|
||||
- [ ] Each claim verified against official sources
|
||||
- [ ] Sources are authoritative and current
|
||||
- [ ] Correction report is clear and actionable
|
||||
- [ ] Temporal context included where relevant
|
||||
- [ ] User approval obtained before changes
|
||||
- [ ] All edits verified successful
|
||||
- [ ] Summary provided to user
|
||||
|
||||
## Limitations
|
||||
|
||||
**This skill cannot:**
|
||||
- Verify subjective opinions or judgments
|
||||
- Access paywalled or restricted sources
|
||||
- Determine "truth" in disputed claims
|
||||
- Predict future specifications or features
|
||||
|
||||
**For such cases:**
|
||||
- Note the limitation in the report
|
||||
- Suggest qualification language
|
||||
- Recommend user research or expert consultation
|
||||
Reference in New Issue
Block a user