5d8c7e39f6
Completes the unified scraping system implementation: **Phase 7: Unified Skill Builder** - cli/unified_skill_builder.py: Generates final skill structure - Inline conflict warnings (⚠️) in API reference - Side-by-side docs vs code comparison - Severity-based conflict grouping - Separate conflicts.md report **Phase 8: MCP Integration** - skill_seeker_mcp/server.py: Auto-detects unified vs legacy configs - Routes to unified_scraper.py or doc_scraper.py automatically - Supports merge_mode parameter override - Maintains full backward compatibility **Phase 9: Example Unified Configs** - configs/react_unified.json: React docs + GitHub - configs/django_unified.json: Django docs + GitHub - configs/fastapi_unified.json: FastAPI docs + GitHub - configs/fastapi_unified_test.json: Test config with limited pages **Phase 10: Comprehensive Tests** - cli/test_unified_simple.py: Integration tests (all passing) - Tests unified config validation - Tests backward compatibility - Tests mixed source types - Tests error handling **Phase 11: Documentation** - docs/UNIFIED_SCRAPING.md: Complete guide (1000+ lines) - Examples, best practices, troubleshooting - Architecture diagrams and data flow - Command reference **Additional:** - demo_conflicts.py: Interactive conflict detection demo - TEST_RESULTS.md: Complete test results and findings - cli/unified_scraper.py: Fixed doc_scraper integration (subprocess) **Features:** ✅ Multi-source scraping (docs + GitHub + PDF) ✅ Conflict detection (4 types, 3 severity levels) ✅ Rule-based merging (fast, deterministic) ✅ Claude-enhanced merging (AI-powered) ✅ Transparent conflict reporting ✅ MCP auto-detection ✅ Backward compatibility **Test Results:** - 6/6 integration tests passed - 4 unified configs validated - 3 legacy configs backward compatible - 5 conflicts detected in test data - All documentation complete 🤖 Generated with Claude Code
50 lines
1.5 KiB
JSON
50 lines
1.5 KiB
JSON
{
|
|
"name": "django",
|
|
"description": "Complete Django framework knowledge combining official documentation and Django codebase. Use when building Django applications, understanding ORM internals, or debugging Django issues.",
|
|
"merge_mode": "rule-based",
|
|
"sources": [
|
|
{
|
|
"type": "documentation",
|
|
"base_url": "https://docs.djangoproject.com/en/stable/",
|
|
"extract_api": true,
|
|
"selectors": {
|
|
"main_content": "article",
|
|
"title": "h1",
|
|
"code_blocks": "pre"
|
|
},
|
|
"url_patterns": {
|
|
"include": [],
|
|
"exclude": ["/search/", "/genindex/"]
|
|
},
|
|
"categories": {
|
|
"getting_started": ["intro", "tutorial", "install"],
|
|
"models": ["models", "orm", "queries", "database"],
|
|
"views": ["views", "urls", "templates"],
|
|
"forms": ["forms", "modelforms"],
|
|
"admin": ["admin"],
|
|
"api": ["ref/"],
|
|
"topics": ["topics/"],
|
|
"security": ["security", "csrf", "authentication"]
|
|
},
|
|
"rate_limit": 0.5,
|
|
"max_pages": 300
|
|
},
|
|
{
|
|
"type": "github",
|
|
"repo": "django/django",
|
|
"include_issues": true,
|
|
"max_issues": 100,
|
|
"include_changelog": true,
|
|
"include_releases": true,
|
|
"include_code": true,
|
|
"code_analysis_depth": "surface",
|
|
"file_patterns": [
|
|
"django/db/**/*.py",
|
|
"django/views/**/*.py",
|
|
"django/forms/**/*.py",
|
|
"django/contrib/admin/**/*.py"
|
|
]
|
|
}
|
|
]
|
|
}
|