Commit Graph

2 Commits

Author SHA1 Message Date
yusyus
9c1a133c51 Add page count estimator for fast config validation
- Add estimate_pages.py script (~270 lines)
- Fast estimation without downloading content (HEAD requests only)
- Shows estimated total pages and recommended max_pages
- Validates URL patterns work correctly
- Estimates scraping time based on rate_limit
- Update CLAUDE.md with estimator workflow and commands
- Update README.md features section with estimation benefits
- Usage: python3 estimate_pages.py configs/react.json
- Time: 1-2 minutes vs 20-40 minutes for full scrape

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-19 02:44:50 +03:00
yusyus
f8c75a3b2d Add comprehensive CLAUDE.md for Claude Code integration
- Add root-level CLAUDE.md with complete guidance for Claude Code
- Include Python 3.7+ requirement
- Add first-time user workflow with all commands
- Include CSS selector testing with BeautifulSoup examples
- Add output quality verification commands
- Document force re-scrape instructions
- Fix package_skill.py path (remove hardcoded /mnt/skills reference)
- Add complete config file structure with real examples
- Include testing section for selector validation
- Add performance metrics table
- Document all key code locations with line numbers
- Organize by: quick start → architecture → workflows → troubleshooting
- Preserve existing docs/CLAUDE.md as detailed technical reference

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-19 01:43:02 +03:00