- Add docs/VIDEO_GUIDE.md (483 lines) — comprehensive guide covering
Quick Start, CLI reference, visual pipeline, AI enhancement, output
structure, time clipping, and troubleshooting
- Update README.md video section with new CLI examples (enhance,
clipping, vision OCR, re-build from JSON) and link to full guide
- Sync README.zh-CN.md with all video feature additions:
- Quick Start section: video commands
- Core Features: new video extraction feature list
- Installation table: video/video-full packages + GPU note
- Usage Examples: full video extraction subsection
- Documentation links: VIDEO_GUIDE.md reference
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Skip OCR on WEBCAM/OTHER frames (eliminates ~64 junk results per video)
- Add _clean_ocr_line() to strip line numbers, IDE decorations, collapse markers
- Add _fix_intra_line_duplication() for multi-engine OCR overlap artifacts
- Add _is_likely_code() filter to prevent UI junk in reference code fences
- Add language detection to get_text_groups() via LanguageDetector
- Apply OCR cleaning in _assemble_structured_text() pipeline
- Add two-pass AI enhancement: Pass 1 cleans reference Code Timeline
using transcript context, Pass 2 generates SKILL.md from cleaned refs
- Update video-tutorial.yaml prompts for pre-cleaned references
- Add 17 new tests (197 total video tests), 2540 tests passing
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add missing video pipeline feature (the main 15K+ line addition),
15 video bug fixes, and restructure [Unreleased] section with
proper hierarchy: Video Pipeline Core → Video --setup → Word support.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- UP024: Replace `(OSError, IOError)` with `OSError` in video_setup.py
- E402: Use existing `os` import instead of `import os as _os` in video_visual.py
- SIM103: Inline condition in `_detect_gpu()` return statement
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Auto-detects NVIDIA (CUDA), AMD (ROCm), or CPU-only GPU and installs the
correct PyTorch variant + easyocr + all visual extraction dependencies.
Removes easyocr from video-full pip extras to avoid pulling ~2GB of wrong
CUDA packages on non-NVIDIA systems.
New files:
- video_setup.py (835 lines): GPU detection, PyTorch install, ROCm config,
venv checks, system dep validation, module selection, verification
- test_video_setup.py (60 tests): Full coverage of detection, install, verify
Updated docs: CHANGELOG, AGENTS.md, CLAUDE.md, README.md, CLI_REFERENCE,
FAQ, TROUBLESHOOTING, installation guide, video dependency plan
All 2523 tests passing (15 skipped).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Fix extract_visual_data returning 2-tuple instead of 3 (ValueError crash)
- Move pytesseract from core deps to [video-full] optional group
- Add 30-min timeout + user feedback to video enhancement subprocess
- Add scrape_video_impl to MCP server fallback import block
- Detect auto-generated YouTube captions via is_generated property
- Forward --vision-ocr and --video-playlist through create command
- Fix filename collision for non-ASCII video titles (fallback to video_id)
- Make _vision_used a proper dataclass field on FrameSubSection
- Expose 6 visual params in MCP scrape_video tool
- Add install instructions on missing video deps in unified scraper
- Update MCP docstring tool counts (25→33, 7 categories)
- Add video and word commands to main.py docstring
- Document video-full exclusion from [all] deps in pyproject.toml
- Update parser registry test count (22→23 for video parser)
All 2437 tests passing, 0 failures.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sync with latest development changes including ruff formatting,
bug fixes, and pinecone adaptor additions.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
DocToSkillConverter has self.skill_dir (string), not self.output_dir.
The --chunk-for-rag flag on scrape command crashed with AttributeError.
Changed to Path(converter.skill_dir).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Bare `pip3` can point to a different Python installation than `python3`.
On the reporter's macOS, python3 was 3.14 but pip3 was linked to 3.9,
causing "no matching distribution" since skill-seekers requires >=3.10.
Using `python3 -m pip` guarantees the same interpreter that passed the
version check is the one performing the install.
Closes#301
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add FALLBACK_MAIN_SELECTORS constant and _find_main_content() helper to
eliminate 3 duplicated fallback loops in doc_scraper.py
- Move link extraction before early return in extract_content() so links
are always discovered from the full page, not just main content
- Fix single-threaded dry-run to extract links from soup (full page)
instead of main element only — fixes reactflow.dev finding only 1 page
- Add link extraction to async dry-run path (was completely missing)
- Remove main_content from get_configuration() defaults so fallback logic
kicks in instead of a broad CSS comma selector matching body
- Smart create --config routing: peek at JSON to determine unified
(sources array → unified_scraper) vs simple (base_url → doc_scraper)
- Update docs/user-guide/02-scraping.md and docs/reference/CONFIG_FORMAT.md
to use unified config format (legacy format rejected since v2.11.0)
- Fix test_auto_fetch_enabled and test_mcp_validate_legacy_config
Closes#300
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Stage 1 quality improvements from the Arbitrary Limits & Dead Code audit:
Reference file truncation removed:
- codebase_scraper.py: remove code[:500] truncation at 5 locations — reference
files now contain complete code blocks for copy-paste usability
- unified_skill_builder.py: remove issues[:20], releases[:10], body[:500],
and code_snippet[:300] caps in reference files — full content preserved
Enhancement summarizer rewrite:
- enhance_skill_local.py: replace arbitrary [:5] code block cap with
character-budget approach using target_ratio * content_chars
- Fix intro boundary bug: track code block state so intro never ends
inside a code block, which was desynchronizing the parser
- Remove dead _target_lines variable (assigned but never used)
- Heading chunks now also respect the character budget
Hardcoded language fixes:
- unified_skill_builder.py: test examples use ex["language"] instead of
always "python" for syntax highlighting
- how_to_guide_builder.py: add language field to HowToGuide dataclass,
set from workflow at creation, used in AI enhancement prompt
Test fixes:
- test_enhance_skill_local.py: rename test to test_code_blocks_not_arbitrarily_capped,
fix assertion to count actual blocks (```count // 2), use target_ratio=0.9
Documentation:
- Add Stage 1 plan, implementation summary, review, and corrected docs
- Update CHANGELOG.md with all changes
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Fix#299: rename --chunk-size/--chunk-overlap to --streaming-chunk-size/
--streaming-overlap in arguments/package.py to avoid collision with the
RAG --chunk-size flag from arguments/common.py
- Phase 1a: make package_skill.py import args via add_package_arguments()
instead of a 105-line inline duplicate argparse block; fixes the root
cause of _reconstruct_argv() passing unrecognised flag names
- Phase 1b: centralise setup_logging() into utils.py and remove 4
duplicate module-level logging.basicConfig() calls from doc_scraper.py,
github_scraper.py, codebase_scraper.py, and unified_scraper.py
- Fix test_package_structure.py / test_cli_paths.py version strings
(3.1.1 → 3.1.2)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Fix 1 — gemini.py: replace deprecated gemini-2.0-flash-exp (404 errors)
with gemini-2.5-flash (stable, GA, Google's recommended replacement).
Closes#290.
Fix 2 — enhance dispatcher: implement the documented auto-detection that
was missing from the code. skill-seekers enhance now correctly routes:
- ANTHROPIC_API_KEY set → Claude API mode (enhance_skill.py)
- GOOGLE_API_KEY set → Gemini API mode
- OPENAI_API_KEY set → OpenAI API mode
- No API keys → LOCAL mode (Claude Code Max, free)
Use --mode LOCAL to force local mode even when an API key is present.
9 new tests cover _detect_api_target() priority logic and main()
routing (API delegation, --mode LOCAL override, no-key fallback).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
All scrapers (scrape, github, analyze, pdf) now share a common argument
contract via add_all_standard_arguments() in arguments/common.py.
Universal flags (--dry-run, --verbose, --quiet, --name, --description,
workflow args) work consistently across all source types.
Previously, `create <url> --dry-run`, `create owner/repo --dry-run`,
and `create ./path --dry-run` would crash because sub-scrapers didn't
accept those flags. Also fixes main.py _handle_analyze_command() not
forwarding --dry-run, --preset, --quiet, --name, --description to
codebase_scraper.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The create command crashed with 'Namespace' object has no attribute
'max_pages' because it accessed args.max_pages directly instead of
using getattr() like all other source-specific attributes in the
same method.
Closes#293
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- README, CONTRIBUTING, QUALITY_GUIDELINES, AGENTS.md all aligned with
production best practices (accurate counts, no max_pages, unified format)
- validate-config.py: fix two bugs (unified config categories lookup,
max_pages warning logic)
- Delete old submit-config.md (duplicate of submit-config.yml with
outdated content)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
