yusyus
f03f4cf569
feat: Phase 6 - Unified scraper orchestrator
Created main orchestrator that coordinates entire workflow:
Architecture:
- UnifiedScraper class orchestrates all phases
- Routes to appropriate scraper based on source type
- Supports any combination of sources
4-Phase Workflow:
1. Scrape all sources (docs, GitHub, PDF)
2. Detect conflicts (if multiple API sources)
3. Merge intelligently (rule-based or Claude-enhanced)
4. Build unified skill (placeholder for Phase 7)
Features:
✅ Validates unified config on startup
✅ Backward compatible with legacy configs
✅ Source-specific routing (documentation/github/pdf)
✅ Automatic conflict detection when needed
✅ Merge mode selection (rule-based/claude-enhanced)
✅ Creates organized output structure
✅ Comprehensive logging for each phase
✅ Error handling and graceful failures
CLI Usage:
- python3 cli/unified_scraper.py --config configs/godot_unified.json
- python3 cli/unified_scraper.py -c configs/react_unified.json -m claude-enhanced
Output Structure:
- output/{name}/ - Final skill directory
- output/{name}_unified_data/ - Intermediate data files
* documentation_data.json
* github_data.json
* conflicts.json
* merged_data.json
Next: Phase 7 - Skill builder to generate final SKILL.md
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-26 15:32:23 +03:00
..
2025-10-26 13:16:54 +03:00
2025-10-26 15:09:38 +03:00
2025-10-26 15:09:38 +03:00
2025-10-26 15:17:27 +03:00
2025-10-26 13:05:39 +03:00
2025-10-26 13:05:39 +03:00
2025-10-26 13:05:39 +03:00
2025-10-26 13:05:39 +03:00
2025-10-26 13:05:39 +03:00
2025-10-19 20:48:03 +03:00
2025-10-26 15:09:38 +03:00
2025-10-24 18:27:17 +04:00
2025-10-24 18:27:17 +04:00
2025-10-24 18:26:10 +04:00
2025-10-26 15:17:27 +03:00
2025-10-19 20:48:03 +03:00
2025-10-22 21:45:51 +03:00
2025-10-26 13:05:39 +03:00
2025-10-26 00:51:18 +03:00
2025-10-19 15:19:53 +03:00
2025-10-19 20:48:03 +03:00
2025-10-26 15:32:23 +03:00
2025-10-19 22:17:23 +03:00
2025-10-26 13:05:39 +03:00