- Remove unused imports (F401): os/Path/json/threading in tests; os in estimate_pages;
Path in install_skill; pytest in test_unified_scraper_orchestration
- Fix F821 undefined 'args' in unified_scraper._scrape_local() by storing
self._cli_args = args in run() and reading via getattr in _scrape_local()
- Fix ARG001/ARG005 unused lambda/function arguments with _ prefix or # noqa:ARG001
where parameter names must be preserved for keyword-argument compatibility
- Fix C408 unnecessary dict() calls → dict literals in test_enhance_command
- Fix F841 unused variable 'stub' in test_enhance_command
- Fix SIM117 nested with statements → single with in test_unified_scraper_orchestration
- Fix SIM105 try/except/pass → contextlib.suppress in test_unified_scraper_orchestration
- Rewrite TestScrapeLocal to test fixed behavior (not the NameError bug)
All 2267 tests pass, 11 skipped.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- pyproject.toml: version 3.0.0 → 3.1.0
- src/skill_seekers/_version.py: update hardcoded fallback to 3.1.0
- CHANGELOG.md: comprehensive [3.1.0] release notes covering all
features and fixes since v3.0.0 (unified create command, workflow
presets, RST parser, smart enhance dispatcher, CLI flag parity,
60 new workflow YAMLs, test suite improvements)
- Deprecation messages: update "removed in v3.0.0" → "v4.0.0" across
analyze_presets.py, codebase_scraper.py, mcp/server.py
- tests/test_cli_paths.py: update version assertion to 3.1.0
- tests/test_package_structure.py: update __version__ assertions to 3.1.0
- tests/test_preset_system.py: update deprecation message version to v4.0.0
All 2267 tests passing.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Fixes several categories of test failures to achieve a clean test suite:
**Python 3.14 / chromadb compatibility**
- chroma.py: broaden except clause to catch pydantic ConfigError on Python 3.14
- test_adaptors_e2e.py, test_integration_adaptors.py: skip on (ImportError, Exception)
**sys.modules corruption (test isolation)**
- test_swift_detection.py: save/restore all skill_seekers.cli modules AND parent
package attributes in test_empty_swift_patterns_handled_gracefully; prevents
@patch decorators in downstream test files from targeting stale module objects
**Removed unnecessary @unittest.skip decorators**
- test_claude_adaptor.py, test_gemini_adaptor.py, test_openai_adaptor.py: remove
skip from tests that already had pass-body or were compatible once deps installed
**Fixed openai import guard for installed package**
- test_openai_adaptor.py: use patch.dict(sys.modules, {"openai": None}) for
test_upload_missing_library since openai is now a transitive dep
**langchain import path update**
- test_rag_chunker.py: fix from langchain.schema → langchain_core.documents
**config_extractor tomllib fallback**
- config_extractor.py: use stdlib tomllib (Python 3.11+) as fallback when
tomli/toml packages are not installed
**Remove redundant sys.path.insert() calls**
- codebase_scraper.py, doc_scraper.py, enhance_skill.py, enhance_skill_local.py,
estimate_pages.py, install_skill.py: remove legacy path manipulation no longer
needed with pip install -e . (src/ layout)
**Test fixes: removed @requires_github from fully-mocked tests**
- test_unified_analyzer.py: 5 tests that mock GitHubThreeStreamFetcher don't
need a real token; remove decorator so they always run
**macOS-specific test improvements**
- test_terminal_detection.py: use @patch(sys.platform, "darwin") instead of
runtime skipTest() so tests run on all platforms
**Dependency updates**
- pyproject.toml, uv.lock: add langchain and llama-index as core dependencies
**New workflow presets and tests**
- src/skill_seekers/workflows/: add 60 new domain-specific workflow YAML presets
- tests/test_mcp_workflow_tools.py: tests for MCP workflow tool implementations
- tests/test_unified_scraper_orchestration.py: tests for UnifiedScraper methods
Result: 2115 passed, 158 skipped (external services/long-running), 0 failures
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Fixes issues #289 and #286 (agent switching and Docker/root failures).
enhance_command.py (new smart dispatcher):
- Routes skill-seekers enhance to API mode (Gemini/OpenAI/Claude API)
when an API key is available, or LOCAL mode (Claude Code CLI) otherwise
- Decision priority: --target flag > config default_agent > auto-detect
from env vars (ANTHROPIC_API_KEY → claude, GOOGLE_API_KEY → gemini,
OPENAI_API_KEY → openai) > LOCAL fallback
- Blocks LOCAL mode when running as root (Docker/VPS) with clear error
message + API mode instructions
- Supports --dry-run, --target, --api-key as first-class flags
arguments/enhance.py:
- Added --target, --api-key, --dry-run, --interactive-enhancement to
ENHANCE_ARGUMENTS (shared by unified CLI parser and standalone entry point)
enhance_skill_local.py:
- Error output no longer truncated at 200 chars (shows up to 20 lines)
- Detects root/permission errors in stderr and prints actionable hint
config_manager.py:
- Added default_agent field to DEFAULT_CONFIG ai_enhancement section
- Added get_default_agent() and set_default_agent() methods
main.py:
- enhance command routed to enhance_command (was enhance_skill_local)
- _handle_analyze_command uses smart dispatcher for post-analysis enhancement
pyproject.toml:
- skill-seekers-enhance entry point updated to enhance_command:main
Tests: 1977 passed, 0 failed (28 new tests in test_enhance_command.py)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Flag/option synchronization fixes:
- analyze: add --dry-run, --api-key, and all workflow flags (--enhance-workflow,
--enhance-stage, --var, --workflow-dry-run) via WORKFLOW_ARGUMENTS merge
- pdf: add --api-key to PDF_ARGUMENTS; replace 5 hardcoded add_argument() calls
in pdf_scraper.py:main() with add_pdf_arguments() to activate all defined args
- unified: add --api-key and --enhance-level (global override) to UNIFIED_ARGUMENTS
and standalone parser; wire enhance_level CLI override into run() per-source loop
- codebase_scraper: fix --enhance-workflow to use action="append" (was type=str),
enabling multiple workflow chaining instead of silently dropping all but last
ConfigManager test isolation fix:
- __init__ now reads self.CONFIG_DIR/CONFIG_FILE/PROGRESS_DIR class variables
instead of calling _get_config_dir()/_get_progress_dir() directly, enabling
monkeypatching in tests (fixes pre-existing test_add_and_retrieve_github_profile)
Workflow JSON config support in unified_scraper:
- Phase 5 now reads workflows/workflow_stages/workflow_vars from top-level JSON
config and merges them with CLI args (CLI-first ordering); supports running
workflows even when unified scraper is called without CLI args (args=None)
Tests: 1,949 passed, 0 failed (added 18 new tests across 3 test files)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Previously _route_config only forwarded --dry-run, silently dropping
all enhancement workflows, --merge-mode, and --skip-codebase-analysis.
Changes:
- arguments/create.py: add CONFIG_ARGUMENTS dict with merge_mode and
skip_codebase_analysis; wire into get_source_specific_arguments(),
get_compatible_arguments(), and add_create_arguments(mode='config')
- create_command.py: fix _route_config to forward --fresh, --merge-mode,
--skip-codebase-analysis, and all 4 workflow flags; add --help-config
handler (skill-seekers create --help-config) matching other help modes
- parsers/create_parser.py: add --help-config flag for unified CLI parity
- tests/test_create_arguments.py: import CONFIG_ARGUMENTS; update config
source tests to assert correct content instead of empty dict
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
unified_scraper.py was the only scraper missing --enhance-workflow,
--enhance-stage, --var, and --workflow-dry-run support. All other
scrapers (doc_scraper, github_scraper, pdf_scraper, codebase_scraper)
already called run_workflows() after building the skill.
Changes:
- arguments/unified.py: add 4 workflow args to UNIFIED_ARGUMENTS so
the unified CLI subparser picks them up automatically
- unified_scraper.py main(): register the same 4 workflow args in the
standalone parser
- unified_scraper.py run(): accept optional `args` parameter and call
run_workflows() after build_skill(), passing unified context
(name + description) consistent with doc_scraper pattern
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Square brackets in URL paths (e.g. /api/[v1]/users from API reference docs)
are technically invalid unencoded per RFC 3986. httpx interprets them as IPv6
address literals and raises "Invalid IPv6 URL", crashing the llms-full.md
parse step.
Fix _clean_url() in LlmsTxtParser to percent-encode [ and ] in the path and
query components (-> %5B / %5D) using urlparse/urlunparse so only the path is
touched, not the host. Anchor-stripping logic is preserved and runs first.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add _get_config_dir() and _get_progress_dir() helpers that return
%APPDATA%/skill-seekers and %LOCALAPPDATA%/skill-seekers/progress on
Windows instead of Unix-only ~/.config and ~/.local/share paths
- Recompute paths at instance creation in __init__ so they are always
evaluated at runtime, not at class definition time
- Guard all chmod() calls with sys.platform != "win32" — chmod with
Unix stat flags is a no-op on Windows which caused config to appear
saved but be unreadable/unfindable on subsequent runs
- Fix should_show_welcome() and mark_welcome_shown() to use instance
config_dir instead of stale class-level WELCOME_FLAG constant
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add --local-repo-path to UNIVERSAL_ARGUMENTS in create.py so it is
registered in the actual parser (not just help display)
- Add --local-repo-path to GITHUB_ARGUMENTS in arguments/github.py for
the standalone github subcommand
- Forward --local-repo-path through create_command._route_github() to
github_scraper
- Add local_repo_path to the config dict built from CLI args in
github_scraper.main()
- Add early validation in GitHubScraper.__init__(): warn and reset to
None if path does not exist, triggering a real GitHub API fallback
instead of silently operating with an empty file tree (fixes#281)
- Update test_create_arguments.py count/names assertions (17 -> 18)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Fixes ruff format --check CI failure. 22 files reformatted to satisfy
the ruff formatter's style requirements. No logic changes, only
whitespace/formatting adjustments.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
The unified MarkdownParser returns all headings (h1-h6) and all paragraphs
without length filtering. Apply the documented behaviour at the call site:
- Exclude h1 from the headings list (return h2-h6 only)
- Filter out paragraphs shorter than 20 characters from content
Fixes test_extract_headings_h2_to_h6 and test_extract_content_paragraphs.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Add YAML-based enhancement workflow presets shipped inside the package
(default, minimal, security-focus, architecture-comprehensive, api-documentation)
- Add `skill-seekers workflows` subcommand: list, show, copy, add, remove, validate
- copy/add/remove all accept multiple names/files in one invocation with partial-failure behaviour
- `add --name` override restricted to single-file operations
- Add 5 MCP tools: list_workflows, get_workflow, create_workflow, update_workflow, delete_workflow
- Fix: create command _add_common_args() now correctly forwards each --enhance-workflow
as a separate flag instead of passing the whole list as a single argument
- Update README: reposition as "data layer for AI systems" with AI Skills front and centre
- Update CHANGELOG, QUICK_REFERENCE, CLAUDE.md with workflow preset details
- 1,880+ tests passing
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- enhancement_workflow.py: WorkflowEngine class for multi-stage AI
enhancement workflows with preset support (security-focus,
architecture-comprehensive, api-documentation, minimal, default)
- unified_enhancer.py: unified enhancement orchestrator integrating
workflow execution with traditional enhance-level based enhancement
- create_command.py: wire workflow args into the unified create command
- AGENTS.md: update agent capability documentation
- configs/godot_unified.json: add unified Godot documentation config
- ENHANCEMENT_WORKFLOW_SYSTEM.md: documentation for the workflow system
- WORKFLOW_ENHANCEMENT_SEQUENTIAL_EXECUTION.md: docs explaining
sequential execution of workflows followed by AI enhancement
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Change --enhance-workflow from type:str to action:append in all argument
files (workflow, create, scrape, github, pdf) so the flag can be given
multiple times to chain workflows in sequence
- Add workflow_runner.py: shared utility used by all 4 scrapers
- collect_workflow_vars(): merges extra context then user --var flags
(user flags take precedence over scraper metadata)
- run_workflows(): executes named workflows in order, then any inline
--enhance-stage workflow; handles dry-run/preview mode
- Remove duplicate ~115-130 line workflow blocks from doc_scraper,
github_scraper, pdf_scraper, and codebase_scraper; replace with
single run_workflows() call each
- Remove mutual exclusivity between workflows and AI enhancement:
workflows now run first, then traditional enhancement continues
independently (--enhance-level 0 to disable)
- Add tests/test_workflow_runner.py: 21 tests covering no-flags, single
workflow, multiple/chained workflows, inline stages, mixed mode,
variable precedence, and dry-run
- Fix test_markdown_parsing: accept "text" or "unknown" for unlabelled
code blocks (unified MarkdownParser returns "text" by default)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Fixes critical bug where RST/Markdown files in documentation
directories were not being parsed with the unified parser system.
Issue:
- Documentation files were found and categorized
- But were only copied, not parsed with unified RstParser/MarkdownParser
- Result: 0 tables, 0 cross-references extracted from 1,579 RST files
Fix:
- Updated extract_project_documentation() to use RstParser for .rst files
- Updated extract_project_documentation() to use MarkdownParser for .md files
- Extract rich structured data: tables, cross-refs, directives, quality scores
- Save extraction summary with parser version
Results (Godot documentation test):
- Enhanced files: 1,579/1,579 (100%)
- Tables extracted: 1,426 (was 0)
- Cross-references: 42,715 (was 0)
- Code blocks: 770 (with quality scoring)
Impact:
- Documentation extraction now benefits from unified parser system
- Complete parity with web documentation scraping (doc_scraper.py)
- RST API docs fully parsed (classes, methods, properties, signals)
- All content gets quality scoring
Files Changed:
- src/skill_seekers/cli/codebase_scraper.py (~100 lines)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Implements _scrape_local() method to handle local directories in unified configs.
Changes:
1. Added elif case for "local" type in scrape_all_sources()
2. Implemented _scrape_local() method (~130 lines)
- Calls analyze_codebase() from codebase_scraper
- Maps config fields to analysis parameters
- Handles all C3.x features (patterns, tests, guides, config, architecture, docs)
- Supports Godot signal flow analysis (automatic)
3. Added "local" to scraped_data and _source_counters initialization
Features supported:
- Local documentation files (RST, Markdown, etc.)
- Local source code analysis (9 languages)
- All C3.x features: patterns (C3.1), test examples (C3.2), how-to guides (C3.3), config patterns (C3.4), architecture (C3.7), docs (C3.9), signal flow (C3.10)
- AI enhancement levels (0-3)
- Analysis depth control (surface, deep, full)
Result:
✅ No more "Unknown source type: local" warnings
✅ Godot unified config works properly
✅ All 18 unified tests pass
✅ Local + documentation + GitHub sources can be combined
Example usage:
skill-seekers create configs/godot_unified.json
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Changed 'pathspec.PathSpec' | None to Optional['pathspec.PathSpec']
- Fixes TypeError in Python 3.10/3.11 where | operator doesn't work with string literals
- Adds Optional to typing imports
- Use underscore prefix for help flag destinations (_help_web, etc.)
- Handle help flags in main.py argv reconstruction
- Ensures progressive disclosure works through unified CLI
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Problem:
- Inconsistent preset names across commands caused confusion:
- analyze: quick, standard, **comprehensive**
- scrape: quick, standard, **deep**
- github: quick, standard, **full**
- Users had to remember different names for the same concept
Solution:
Standardized all preset systems to use consistent naming:
- quick, standard, comprehensive (everywhere)
Changes:
- scrape_presets.py: Renamed "deep" → "comprehensive"
- github_presets.py: Renamed "full" → "comprehensive"
- Updated docstrings to reflect new names
- All preset dictionaries now use identical keys
Result:
✅ Consistent preset names across all commands
✅ Users only need to remember 3 preset names
✅ Help text already shows "comprehensive" everywhere
✅ All 46 tests passing
✅ Better UX and less confusion
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Problem:
- Same argument names in different commands with different meanings
- --chunk-size: 512 tokens (scrape/create) vs 4000 chars (package)
- --chunk-overlap: 50 tokens (scrape/create) vs 200 chars (package)
- Users expect consistent behavior, this was confusing
Solution:
Renamed package.py streaming arguments to be more specific:
- --chunk-size → --streaming-chunk-size (4000 chars)
- --chunk-overlap → --streaming-overlap (200 chars)
Result:
✅ Clear distinction: streaming args vs RAG args
✅ No naming conflicts across commands
✅ --chunk-size now consistently means "RAG tokens" everywhere
✅ All 9 package tests passing
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Changes:
- Added RAG_ARGUMENTS dict to common.py with 3 flags:
- --chunk-for-rag (enable semantic chunking)
- --chunk-size (default: 512 tokens)
- --chunk-overlap (default: 50 tokens)
- Removed duplicate RAG arguments from create.py and scrape.py
- Used .update() pattern to merge RAG_ARGUMENTS into UNIVERSAL_ARGUMENTS and SCRAPE_ARGUMENTS
- Added helper functions: add_rag_arguments(), get_rag_argument_names()
- Updated tests to reflect new argument count (15 → 13 universal arguments)
- Fixed test expectations for boolean_args (removed 'enhance', 'enhance_local')
Result:
- Single source of truth for RAG arguments in common.py
- DRY principle maintained across all commands
- All 88 key tests passing
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Fixed 7 ruff linting errors:
- SIM102: Simplified nested if statements in rag_chunker.py
- SIM113: Use enumerate() in streaming_ingest.py
- ARG001: Prefix unused signal handler args with underscore
- SIM105: Replace try-except-pass with contextlib.suppress (3 instances)
Fixed 7 MCP server test failures:
- Updated generate_config_tool to output unified format (not legacy)
- Updated test_validate_valid_config to use unified format
- Renamed test_submit_config_accepts_legacy_format to
test_submit_config_rejects_legacy_format (tests rejection, not acceptance)
- Updated all submit_config tests to use unified format:
- test_submit_config_requires_token
- test_submit_config_from_file_path
- test_submit_config_detects_category
- test_submit_config_validates_name_format
- test_submit_config_validates_url_format
Added v3.0.0 release planning documents:
- RELEASE_EXECUTIVE_SUMMARY_v3.0.0.md (one-page overview)
- RELEASE_PLAN_v3.0.0.md (complete 4-week campaign)
- RELEASE_CONTENT_CHECKLIST_v3.0.0.md (content creation guide)
All tests should now pass. Ready for v3.0.0 release.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
BREAKING CHANGE: Legacy config format no longer supported
Changes:
- ConfigValidator now only accepts unified format with 'sources' array
- Removed _validate_legacy() method
- Removed convert_legacy_to_unified() and all conversion helpers
- Simplified get_sources_by_type() and has_multiple_sources()
- Updated __main__ to remove legacy format checks
- Converted claude-code.json to unified format
- Deleted blender.json (duplicate of blender-unified.json)
- Clear error message when legacy format detected
Error message shows:
- Legacy format was removed in v2.11.0
- Example of old vs new format
- Migration guide link
Code reduction: -86 lines
All 65 tests passing
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Updated chroma.py: Parallel arrays pattern with chunking support
- Updated llama_index.py: Node format with chunking support
- Updated haystack.py: Document format with chunking support
- Updated faiss_helpers.py: Parallel arrays pattern with chunking support
- Updated weaviate.py: Object/properties format with chunking support
- Updated qdrant.py: Points/payload format with chunking support
All adaptors now use base._maybe_chunk_content() for consistent chunking behavior:
- Auto-chunks large documents (>512 tokens by default)
- Preserves code blocks during chunking
- Adds chunk metadata (chunk_index, total_chunks, is_chunked, chunk_id)
- Configurable via enable_chunking, chunk_max_tokens, preserve_code_blocks
Test results: 174/174 tests passing (6 skipped E2E tests)
- All 10 chunking integration tests pass
- All 66 RAG adaptor tests pass
- All platform-specific tests pass
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Refactored all RAG adaptors (LangChain, LlamaIndex, Haystack, Weaviate, Chroma,
FAISS, Qdrant) to use existing helper methods from base.py, removing ~215 lines
of duplicate code (26% reduction).
Key improvements:
- All adaptors now use _format_output_path() for consistent path handling
- All adaptors now use _iterate_references() for reference file iteration
- Added _generate_deterministic_id() helper with 3 formats (hex, uuid, uuid5)
- 5 adaptors refactored to use unified ID generation
- Removed 6 unused imports (hashlib, uuid)
Benefits:
- DRY principles enforced across all RAG adaptors
- Single source of truth for common logic
- Easier maintenance and testing
- Consistent behavior across platforms
All 159 adaptor tests passing. Zero regressions.
Phase 1 of optional enhancements (Phases 2-5 pending).
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
