feat: agent-agnostic refactor, smart SPA discovery, marketplace pipeline (#336)
* feat: fix unified scraper pipeline gaps, add multi-agent support, and Unity skill configs Fix multiple bugs in the unified scraper pipeline discovered while creating Unity skills (Spine, Addressables, DOTween): - Fix doc scraper KeyError by passing base_url in temp config - Fix scraped_data list-vs-dict bug in detect_conflicts() and merge_sources() - Add Phase 6 auto-enhancement from config "enhancement" block (LOCAL + API mode) - Add "browser": true config support for JavaScript SPA documentation sites - Add Phase 3 skip message for better UX - Add subprocess timeout (3600s) for doc scraper - Fix SkillEnhancer missing skill_dir argument in API mode - Fix browser renderer defaults (60s timeout, domcontentloaded wait condition) - Fix C3.x JSON filename mismatch (design_patterns.json → all_patterns.json) - Fix workflow builtin target handling when no pattern data available - Make AI enhancement timeout configurable via SKILL_SEEKER_ENHANCE_TIMEOUT env var (300s default) - Add C#, Go, Rust, Swift, Ruby, PHP, GDScript to GitHub scraper extension map - Add multi-agent LOCAL mode support across all 17 scrapers (--agent flag) - Add Kimi/Moonshot platform support (API keys, agent presets, config wizard) - Add unity-game-dev.yaml workflow (7 stages covering Unity-specific patterns) - Add 3 Unity skill configs (Spine, Addressables, DOTween) - Add comprehensive Claude bias audit report Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: create AgentClient abstraction, remove hardcoded Claude from 5 enhancers (#334) Phase 1 of the full agent-agnostic refactor. Creates a centralized AgentClient that all enhancers use instead of hardcoded subprocess calls and model names. New file: - agent_client.py: Unified AI client supporting API mode (Anthropic, Moonshot, Google, OpenAI) and LOCAL mode (Claude Code, Kimi, Codex, Copilot, OpenCode, custom agents). Provides detect_api_key(), get_model(), detect_default_target(). Refactored (removed all hardcoded ["claude", ...] subprocess calls): - ai_enhancer.py: -140 lines, delegates to AgentClient - config_enhancer.py: -150 lines, removed _run_claude_cli() - guide_enhancer.py: -120 lines, removed _check_claude_cli(), _call_claude_*() - unified_enhancer.py: -100 lines, removed _check_claude_cli(), _call_claude_*() - codebase_scraper.py: collapsed 3 functions into 1 using AgentClient Fixed: - utils.py: has_api_key()/get_api_key() now check all providers - enhance_skill.py, video_scraper.py, video_visual.py: model names configurable via ANTHROPIC_MODEL env var - enhancement_workflow.py: uses call() with _call_claude() fallback Net: -153 lines of code while adding full multi-agent support. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: Phase 2 agent-agnostic refactor — defaults, help text, merge mode, MCP (#334) Phase 2 of the full agent-agnostic refactor: Default targets: - Changed default="claude" to auto-detect from API keys in 5 argument files and 3 CLI scripts (install_skill, upload_skill, enhance_skill) - Added AgentClient.detect_default_target() for runtime resolution - MCP server functions now use "auto" default with runtime detection Help text (16+ argument files): - Replaced "ANTHROPIC_API_KEY" / "Claude Code" with agent-neutral wording - Now mentions all API keys (ANTHROPIC, MOONSHOT, etc.) and "AI coding agent" Log messages: - main.py, enhance_command.py: "Claude Code CLI" → dynamic agent name - enhance_command.py docstring: "Claude Code" → "AI coding agent" Merge mode rename: - Added "ai-enhanced" as the preferred merge mode name - "claude-enhanced" kept as backward-compatible alias - Renamed ClaudeEnhancedMerger → AIEnhancedMerger (with alias) - Updated choices, validators, and descriptions MCP server descriptions: - server_fastmcp.py: "Claude AI skills" → "LLM skills" in tool descriptions - packaging_tools.py: Updated defaults and dry-run messages Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: Phase 3 agent-agnostic refactor — docstrings, MCP descriptions, README (#334) Phase 3 of the full agent-agnostic refactor: Module docstrings (17+ scraper files): - "Claude Skill Converter" → "AI Skill Converter" - "Build Claude skill" → "Build AI/LLM skill" - "Asking Claude" → "Asking AI" - Updated doc_scraper, github_scraper, pdf_scraper, word_scraper, epub_scraper, video_scraper, enhance_skill, enhance_skill_local, unified_scraper, and others MCP server_legacy.py (30+ fixes): - All tool descriptions: "Claude skill" → "LLM skill" - "Upload to Claude" → "Upload skill" - "enhance with Claude Code" → "enhance with AI agent" - Kept claude.ai/skills URLs (platform-specific, correct) MCP README.md: - Added multi-agent support note at top - "Claude AI skills" → "LLM skills" throughout - Updated examples to show multi-platform usage - Kept Claude Code in supported agents list (accurate) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: Phase 3 continued — remaining docstring and comment fixes (#334) Additional agent-neutral text fixes in 8 files missed from the initial Phase 3 commit: - config_extractor.py, config_manager.py, constants.py: comments - enhance_command.py: docstring and print messages - guide_enhancer.py: class/module docstrings - parsers/enhance_parser.py, install_parser.py: help text - signal_flow_analyzer.py: docstring Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * workflow added * fix: address code review issues in AgentClient and Phase 6 (#334) Fixes found during commit review: 1. AgentClient._call_local: Only append "Write your response to:" when caller explicitly passes output_file (was always appending) 2. Codex agent: Added uses_stdin flag to preset, pipe prompt via stdin instead of DEVNULL (codex reads from stdin with "-" arg) 3. Provider detection: Added _detect_provider_from_key() to detect provider from API key prefix (sk-ant- → anthropic, AIza → google) instead of always assuming anthropic 4. Phase 6 API mode: Replaced direct SkillEnhancer/ANTHROPIC_API_KEY with AgentClient for multi-provider support (Moonshot, Google, OpenAI) 5. config_enhancer: Removed output_file path from prompt — AgentClient manages temp files and output detection Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: make claude adaptor model name configurable via ANTHROPIC_MODEL env var Missed in the Phase 1 refactor — adaptors/claude.py:381 had a hardcoded model name without the os.environ.get() wrapper that all other files use. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add copilot stdin support, custom agent, and kimi aliases (#334) Additional agent improvements from Kimi review: - Added uses_stdin: True to copilot agent preset (reads from stdin like codex) - Added custom agent support via SKILL_SEEKER_AGENT_CMD env var in _call_local() - Added kimi_code/kimi-code aliases in normalize_agent_name() - Added "kimi" to --target choices in enhance arguments - Updated help text with MOONSHOT_API_KEY across argument files Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: Kimi CLI integration — add uses_stdin and output parsing (#334) Kimi CLI's --print mode requires stdin piping and outputs structured protocol messages (TurnBegin, TextPart, etc.) instead of plain text. Fixes: - Added uses_stdin: True to kimi preset (was not piping prompt) - Added parse_output: "kimi" flag to preset - Added _parse_kimi_output() to extract text from TextPart lines - Kimi now returns clean text instead of raw protocol dump Tested: kimi returns '{"status": "ok"}' correctly via AgentClient. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: Kimi CLI in enhance_skill_local — remove wrong skip-permissions, use absolute path Two bugs in enhance_skill_local.py AGENT_PRESETS for Kimi: 1. supports_skip_permissions was True — Kimi doesn't support --dangerously-skip-permissions, only Claude does. Fixed to False. 2. {skill_dir} was resolved as relative path — Kimi CLI requires absolute paths for --work-dir. Fixed with .resolve(). Tested: `skill-seekers enhance output/test-e2e/ --agent kimi` now works end-to-end (107s, 9233 bytes output). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: remove invalid --enhance-level flag from enhance subprocess calls doc_scraper.py and video_scraper.py were passing --enhance-level to skill-seekers-enhance, which doesn't accept that flag. This caused enhancement to fail silently after scraping completed. Fixes: - Removed --enhance-level from enhance subprocess calls - Added --agent passthrough in doc_scraper.py - Fixed log messages to show correct command Tested: `skill-seekers create <url> --enhance-level 1` now chains scrape → enhance successfully. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add --agent and --agent-cmd to create command UNIVERSAL_ARGUMENTS The --agent flag was defined in common.py but not imported into the create command's UNIVERSAL_ARGUMENTS, so it wasn't available when using `skill-seekers create <source> --agent kimi`. Now all 17 source types support the --agent flag via the create command. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: update docs data_file path after moving to cache directory The scraped_data["documentation"] stored the original output/ path for data_file, but the directory was moved to .skillseeker-cache/ afterward. Phase 2 conflict detection then failed with FileNotFoundError trying to read the old path. Now updates data_file to point to the cache location after the move. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: multi-language code signature extraction in GitHub scraper The GitHub scraper only analyzed files matching the primary language (by bytes). For multi-language repos like spine-runtimes (C++ primary but C# is the target), this meant 0 C# files were analyzed. Fix: Analyze top 3 languages with known extension mappings instead of just the primary. Also support "language" field in config source to explicitly target specific languages (e.g., "language": "C#"). Updated Unity configs to specify language: "C#" for focused analysis. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: per-file language detection + remove artificial analysis limits Rewrites GitHub scraper's _extract_signatures_and_tests() to detect language per-file from extension instead of only analyzing the primary language. This fixes multi-language repos like spine-runtimes (C++ primary) where C# files were never analyzed. Changes: - Build reverse ext→language map, detect language per-file - Analyze ALL files with known extensions (not just primary language) - Config "language" field works as optional filter, not a workaround - Store per-file language + languages_analyzed in output - Remove 50-file API mode limit (rate limiting already handles this) - Remove 100-file default config extraction limit (now unlimited by default) - Fix unified scraper default max_pages from 100 to 500 (matches constants.py) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: remove remaining 100-file limit in config_extractor.extract_from_directory The find_config_files default was changed to unlimited but extract_from_directory and CLI --max-files still defaulted to 100. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: replace interactive terminal merge with automated AgentClient call AIEnhancedMerger._launch_claude_merge() used to open a terminal window, run a bash script, and poll for a file — requiring manual interaction. Now uses AgentClient.call() to send the merge prompt directly and parse the JSON response. Fully automated, no terminal needed, works with any configured AI agent (Claude, Kimi, etc.). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add marketplace pipeline for publishing skills to Claude Code plugin repos Connect the three-repo pipeline: configs repo → Skill Seekers engine → plugin marketplace repos. Enables automated publishing of generated skills directly into Claude Code plugin repositories with proper plugin.json and marketplace.json structure. New components: - MarketplaceManager: Registry for plugin marketplace repos at ~/.skill-seekers/marketplaces.json with per-repo git tokens, branch config, and default author metadata - MarketplacePublisher: Clones marketplace repo, creates plugin directory structure (skills/, .claude-plugin/plugin.json), updates marketplace.json, commits and pushes. Includes skill_name validation to prevent path traversal, and cleanup of partial state on git failures - 4 MCP tools: add_marketplace, list_marketplaces, remove_marketplace, publish_to_marketplace — registered in FastMCP server - Phase 6 in install workflow: automatic marketplace publishing after packaging, triggered by --marketplace CLI arg or marketplace_targets config field CLI additions: - --marketplace NAME: publish to registered marketplace after packaging - --marketplace-category CAT: plugin category (default: development) - --create-branch: create feature branch instead of committing to main Security: - Skill name regex validation (^[a-zA-Z0-9][a-zA-Z0-9._-]*$) prevents path traversal attacks via malicious SKILL.md frontmatter - has_api_key variable scoping fix in install workflow summary - try/finally cleanup of partial plugin directories on publish failure Config schema: - Optional marketplace_targets field in config JSON for multi-marketplace auto-publishing: [{"marketplace": "spyke", "category": "development"}] - Backward compatible — ignored by older versions Tests: 58 tests (36 manager + 22 publisher including 2 integration tests using file:// git protocol for full publish success path) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: thread agent selection through entire enhancement pipeline Propagates the --agent and --agent-cmd CLI parameters through all enhancement components so users can use any supported coding agent (kimi, claude, copilot, codex, opencode) consistently across the full pipeline, not just in top-level enhancement. Agent parameter threading: - AIEnhancer: accepts agent param, passes to AgentClient - ConfigEnhancer: accepts agent param, passes to AgentClient - WorkflowEngine: accepts agent param, passes to sub-enhancers (PatternEnhancer, TestExampleEnhancer, AIEnhancer) - ArchitecturalPatternDetector: accepts agent param for AI enhancement - analyze_codebase(): accepts agent/agent_cmd, forwards to ConfigEnhancer, ArchitecturalPatternDetector, and doc processing - UnifiedScraper: reads agent from CLI args, forwards to doc scraper subprocess, C3.x analysis, and LOCAL enhancement - CreateCommand: forwards --agent and --agent-cmd to subprocess argv - workflow_runner: passes agent to WorkflowEngine for inline/named workflows Timeout improvements: - Default enhancement timeout increased from 300s (5min) to 2700s (45min) to accommodate large skill generation with local agents - New get_default_timeout() in agent_client.py with env var override (SKILL_SEEKER_ENHANCE_TIMEOUT) supporting 'unlimited' value - Config enhancement block supports "timeout": "unlimited" field - Removed hardcoded timeout=300 and timeout=600 calls in config_enhancer and merge_sources, now using centralized default CLI additions (unified_scraper): - --agent AGENT: select local coding agent for enhancement - --agent-cmd CMD: override agent command template (advanced) Config: unity-dotween.json updated with agent=kimi, timeout=unlimited, removed unused file_patterns Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add claude-code unified config for Claude Code CLI skill generation Unified config combining official Claude Code documentation and source code analysis. Covers internals, architecture, tools, commands, IDE integrations, MCP, plugins, skills, and development workflows. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: add multi-agent support verification report and test artifacts - AGENT_SUPPORT_VERIFICATION.md: verification report confirming agent parameter threading works across all enhancement components - END_TO_END_EXAMPLES.md: complete workflows for all 17 source types with both Claude and Kimi agents - test_agents.sh: shell script for real-world testing of agent support across major CLI commands with both agents - test_realworld.md: real-world test scenarios for manual QA Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: add .env to .gitignore to prevent secret exposure The .env file containing API keys (ANTHROPIC_API_KEY, GITHUB_TOKEN, etc.) was not in .gitignore, causing it to appear as untracked and risking accidental commit. Added .env, .env.local, and .env.*.local patterns. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: URL filtering uses base directory instead of full page URL (#331) is_valid_url() checked url.startswith(self.base_url) where base_url could be a full page path like ".../manual/index.html". Sibling pages like ".../manual/LoadingAssets.html" failed the check because they don't start with ".../index.html". Now strips the filename to get the directory prefix: "https://example.com/docs/index.html" → "https://example.com/docs/" This fixes SPA sites like Unity's DocFX docs where browser mode renders the page but sibling links were filtered out. Closes #331 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: pass language config through to GitHub scraper in unified flow The unified scraper built github_config from source fields but didn't include the "language" field. The GitHub scraper's per-file detection read self.config.get("language", "") which was always empty, so it fell back to analyzing all languages instead of the focused C# filter. For DOTween (C# only repo), this caused 0 files analyzed because without the language filter, it analyzed top 3 languages but the file tree matching failed silently. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: centralize all enhancement timeouts to 45min default with unlimited support All enhancement/AI timeouts now use get_default_timeout() from agent_client.py instead of scattered hardcoded values (120s, 300s, 600s). Default: 2700s (45 minutes) Override: SKILL_SEEKER_ENHANCE_TIMEOUT env var Unlimited: Set to "unlimited", "none", or "0" Updated: agent_client.py, enhance_skill_local.py, arguments/enhance.py, enhance_command.py, unified_enhancer.py, unified_scraper.py Not changed (different purposes): - Browser page load timeout (60s) - API HTTP request timeout (120s) - Doc scraper subprocess timeout (3600s) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add browser_wait_until and browser_extra_wait config for SPA docs DocFX sites (Unity docs) render navigation via JavaScript after initial page load. With domcontentloaded, only 1 link was found. With networkidle + 5s extra wait, 95 content pages are discovered. New config options for documentation sources: - browser_wait_until: "networkidle" | "load" | "domcontentloaded" - browser_extra_wait: milliseconds to wait after page load for lazy nav Updated Addressables config to use networkidle + 5000ms extra wait. Pass browser settings through unified scraper to doc scraper config. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: three-layer smart discovery engine for SPA documentation sites Replaces the browser_wait_until/browser_extra_wait config hacks with a proper discovery engine that runs before the BFS crawl loop: Layer 1: sitemap.xml — checks domain root for sitemap, parses <loc> tags Layer 2: llms.txt — existing mechanism (unchanged) Layer 3: SPA nav — renders index page with networkidle via Playwright, extracts all links from the fully-rendered DOM sidebar/TOC The BFS crawl then uses domcontentloaded (fast) since all pages are already discovered. No config hacks needed — browser mode automatically triggers SPA discovery when only 1 page is found. Tested: Unity Addressables DocFX site now discovers 95 pages (was 1). Removed browser_wait_until/browser_extra_wait from Addressables config. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: replace manual arg forwarding with dynamic routing in create command The create command manually hardcoded ~60% of scraper flags in _route_*() methods, causing ~40 flags to be silently dropped. Every new flag required edits in 2 places (arguments/create.py + create_command.py), guaranteed to drift. Replaced with _build_argv() — a dynamic forwarder that iterates vars(self.args) and forwards all explicitly-set arguments automatically, using the same pattern as main.py::_reconstruct_argv(). This eliminates the root cause of all flag gaps. Changes in create_command.py (-380 lines, +175 lines = net -205): - Added _build_argv() dynamic arg forwarder with dest→flag translation map for mismatched names (async_mode→--async, video_playlist→--playlist, skip_config→--skip-config-patterns, workflow_var→--var) - Added _call_module() helper (dedup sys.argv swap pattern) - Simplified all _route_*() methods from 50-70 lines to 5-10 lines each - Deleted _add_common_args() entirely (subsumed by _build_argv) - _route_generic() now forwards ALL args, not just universal ones New flags accessible via create command: - --from-json: build skill from pre-extracted JSON (all source types) - --skip-api-reference: skip API reference generation (local codebase) - --skip-dependency-graph: skip dependency analysis (local codebase) - --skip-config-patterns: skip config pattern extraction (local codebase) - --no-comments: skip comment extraction (local codebase) - --depth: analysis depth control (local codebase, deprecated) - --setup: auto-detect GPU/install video deps (video) Bug fix in unified_scraper.py: - Fixed C3.x pattern data loss: unified_scraper read patterns/detected_patterns.json but codebase_scraper writes patterns/all_patterns.json. Changed both read locations (line 828 for local sources, line 1597 for GitHub C3.x) to use the correct filename. This was causing 100% loss of design pattern data (e.g., 905 patterns detected but 0 included in final skill). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address 5 code review issues in marketplace and package pipeline Fixes found by automated code review of the marketplace feature and package command: 1. --marketplace flag silently ignored in package_skill.py CLI Added MarketplacePublisher invocation after successful packaging when --marketplace is provided. Previously the flag was parsed but never acted on. 2. Missing 7 platform choices in --target (package.py) Added minimax, opencode, deepseek, qwen, openrouter, together, fireworks to the argparse choices list. These platforms have registered adaptors but were rejected by the argument parser. 3. is_update always True for new marketplace registrations Two separate datetime.now() calls produced different microsecond timestamps, making added_at != updated_at always. Fixed by assigning a single timestamp to both fields. 4. Shallow clone (depth=1) caused push failures for marketplace repos MarketplacePublisher now does full clones instead of using GitConfigRepo's shallow clone (which is designed for read-only config fetching). Full clone is required for commit+push workflow. 5. Partial plugin dir not cleaned on force=True failure Removed the `and not force` guard from cleanup logic — if an operation fails midway, the partial directory should be cleaned regardless of whether force was set. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address dynamic routing edge cases in create_command Fixes from code review of the _build_argv() refactor: 1. Non-None defaults forwarded unconditionally — added enhance_level=2, doc_version="", video_languages="en", whisper_model="base", platform="slack", visual_interval=0.7, visual_min_gap=0.5, visual_similarity=3.0 to the defaults dict so they're only forwarded when the user explicitly overrides them. This fixes video sources incorrectly getting --enhance-level 2 (video default is 0). 2. video_url dest not translated — added "video_url": "--url" to _DEST_TO_FLAG so create correctly forwards --video-url as --url to video_scraper.py. 3. Video positional args double-forwarded — added video_url, video_playlist, video_file to _SKIP_ARGS since _route_video() already handles them via positional args from source detection. 4. Removed dead workflow_var entry from _DEST_TO_FLAG — the create parser uses key "var" not "workflow_var", so the translation was never triggered. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: resolve 15 broken tests and --from-json crash bug in create command Fixes found by Kimi code review of the dynamic routing refactor: 1. 3 test_create_arguments.py failures — UNIVERSAL_ARGUMENTS count changed from 19 to 21 (added agent, agent_cmd). Updated expected count and name set. Moved from_json out of UNIVERSAL to ADVANCED_ARGUMENTS since not all scrapers support it. 2. 12 test_create_integration_basic.py failures — tests called _add_common_args() which was deleted in the refactor. Rewrote _collect_argv() to use _build_argv() via CreateCommand with SourceDetector. Updated _make_args defaults to match new parameter set. 3. --from-json crash bug — was in UNIVERSAL_ARGUMENTS so create accepted it for all source types, but web/github/local scrapers don't support it. Forwarding it caused argparse "unrecognized arguments" errors. Moved to ADVANCED_ARGUMENTS with documentation listing which source types support it. 4. Additional _is_explicitly_set defaults — added enhance_level=2, doc_version="", video_languages="en", whisper_model="base", platform="slack", visual_interval/min_gap/similarity defaults to prevent unconditional forwarding of parser defaults. 5. Video arg handling — added video_url to _DEST_TO_FLAG translation map, added video_url/video_playlist/video_file to _SKIP_ARGS (handled as positionals by _route_video). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: C3.x analysis data loss — read from references/ after _generate_references() cleanup Root cause: _generate_references() in codebase_scraper.py copies analysis directories (patterns/, test_examples/, config_patterns/, architecture/, dependencies/, api_reference/) into references/ then DELETES the originals to avoid duplication (Issue #279). But unified_scraper.py reads from the original paths after analyze_codebase() returns — by which time the originals are gone. This caused 100% data loss for all 6 C3.x data types (design patterns, test examples, config patterns, architecture, dependencies, API reference) in the unified scraper pipeline. The data was correctly detected (e.g., 905 patterns in 510 files) but never made it into the final skill. Fix: Added _load_json_fallback() method that checks references/{subdir}/ first (where _generate_references moves the data), falling back to the original path. Applied to both GitHub C3.x analysis (line ~1599) and local source analysis (line ~828). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: add allowlist to _build_argv for config route to unified_scraper _build_argv() was forwarding all CLI args (--name, --doc-version, etc.) to unified_scraper which doesn't accept them. Added allowlist parameter to _build_argv() — when provided, ONLY args in the allowlist are forwarded. The config route now uses _UNIFIED_SCRAPER_ARGS allowlist with the exact set of flags unified_scraper accepts. This is a targeted patch — the proper fix is the ExecutionContext singleton refactor planned separately. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: add force=True to marketplace publish from package CLI The package command's --marketplace flag didn't pass force=True to MarketplacePublisher, so re-publishing an existing skill would fail with "already exists" error instead of overwriting. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add push_config tool for publishing configs to registered source repos New ConfigPublisher class that validates configs, places them in the correct category directory, commits, and pushes to registered source repositories. Follows the MarketplacePublisher pattern. Features: - Auto-detect category from config name/description - Validate via ConfigValidator + repo's validate-config.py - Support feature branch or direct push - Force overwrite existing configs - MCP tool: push_config(config_path, source_name, category) Usage: push_config(config_path="configs/unity-spine.json", source_name="spyke") Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: security hardening, error handling, tests, and cleanup Security: - Remove command injection via cloned repo script execution (config_publisher) - Replace git add -A with targeted staging (marketplace_publisher) - Clear auth tokens from cached .git/config after clone - Use defusedxml for sitemap XML parsing (XXE protection) - Add path traversal validation for config names Error handling: - AgentClient: specific exception handling for rate limit, auth, connection errors - AgentClient: log subprocess stderr on non-zero exit, raise on explicit API mode failure - config_publisher: only catch ValueError for validation warnings Logic bugs: - Fix _build_argv silently dropping --enhance-level 2 (matched default) - Fix URL filtering over-broadening (strip to parent instead of adding /) - Log warning when _call_module returns None exit code Tests (134 new): - test_agent_client.py: 71 tests for normalize, detect, init, timeout, model - test_config_publisher.py: 23 tests for detect_category, publish, errors - test_create_integration_basic.py: 20 tests for _build_argv routing - Fix 11 pre-existing failures (guide_enhancer, doctor, install_skill, marketplace) Cleanup: - Remove 5 dev artifact files (-1405 lines) - Rename _launch_claude_merge to _launch_ai_merge All 3194 tests pass, 39 expected skips. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: pin ruff==0.15.8 in CI and reformat packaging_tools.py Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: add missing pytest install to vector DB adaptor test jobs Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: reformat 7 files for ruff 0.15.8 and fix vector DB test path Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: remove test-week2-integration job referencing missing script Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: update e2e test to accept dynamic platform name in upload phase Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: YusufKaraaslanSpyke <yusuf@spykegames.com> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
yusyus
GitHub
@@ -1,7 +1,7 @@
|
||||
"""Skill Seekers CLI tools package.
|
||||
|
||||
This package provides command-line tools for converting documentation
|
||||
websites into Claude AI skills.
|
||||
websites into AI skills.
|
||||
|
||||
Main modules:
|
||||
- doc_scraper: Main documentation scraping and skill building tool
|
||||
@@ -13,7 +13,7 @@ Main modules:
|
||||
- enhance_skill_local: AI-powered skill enhancement (local)
|
||||
- estimate_pages: Estimate page count before scraping
|
||||
- package_skill: Package skills into .zip files
|
||||
- upload_skill: Upload skills to Claude
|
||||
- upload_skill: Upload skills to target platform
|
||||
- utils: Shared utility functions
|
||||
"""
|
||||
|
||||
|
||||
@@ -378,7 +378,7 @@ version: {metadata.version}
|
||||
client = anthropic.Anthropic(**client_kwargs)
|
||||
|
||||
message = client.messages.create(
|
||||
model="claude-sonnet-4-20250514",
|
||||
model=os.environ.get("ANTHROPIC_MODEL", "claude-sonnet-4-20250514"),
|
||||
max_tokens=4096,
|
||||
temperature=0.3,
|
||||
messages=[{"role": "user", "content": prompt}],
|
||||
|
||||
553
src/skill_seekers/cli/agent_client.py
Normal file
553
src/skill_seekers/cli/agent_client.py
Normal file
@@ -0,0 +1,553 @@
|
||||
"""
|
||||
Unified AI Client for Skill Seekers
|
||||
|
||||
Centralizes all AI invocations (API and LOCAL mode) so that every enhancer
|
||||
uses a single abstraction instead of hardcoding subprocess calls or model names.
|
||||
|
||||
Supports:
|
||||
- API mode: Anthropic, Moonshot/Kimi, Google Gemini, OpenAI (via adaptor pattern)
|
||||
- LOCAL mode: Claude Code, Kimi Code, Codex, Copilot, OpenCode, custom agents
|
||||
|
||||
Usage:
|
||||
from skill_seekers.cli.agent_client import AgentClient
|
||||
|
||||
client = AgentClient(mode="auto")
|
||||
response = client.call("Analyze this code and return JSON")
|
||||
|
||||
# Or with explicit agent
|
||||
client = AgentClient(mode="local", agent="kimi")
|
||||
response = client.call(prompt, timeout=300)
|
||||
|
||||
# Static helpers
|
||||
key, provider = AgentClient.detect_api_key()
|
||||
model = AgentClient.get_model()
|
||||
"""
|
||||
|
||||
import logging
|
||||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Agent presets for LOCAL mode — reused from enhance_skill_local.py pattern
|
||||
AGENT_PRESETS = {
|
||||
"claude": {
|
||||
"display_name": "Claude Code",
|
||||
"command": ["claude", "--dangerously-skip-permissions", "{prompt_file}"],
|
||||
"version_check": ["claude", "--version"],
|
||||
},
|
||||
"codex": {
|
||||
"display_name": "OpenAI Codex CLI",
|
||||
"command": ["codex", "exec", "--full-auto", "--skip-git-repo-check", "-"],
|
||||
"version_check": ["codex", "--version"],
|
||||
"uses_stdin": True,
|
||||
},
|
||||
"copilot": {
|
||||
"display_name": "GitHub Copilot CLI",
|
||||
"command": ["gh", "copilot", "chat", "-"],
|
||||
"version_check": ["gh", "copilot", "--version"],
|
||||
"uses_stdin": True,
|
||||
},
|
||||
"opencode": {
|
||||
"display_name": "OpenCode CLI",
|
||||
"command": ["opencode"],
|
||||
"version_check": ["opencode", "--version"],
|
||||
},
|
||||
"kimi": {
|
||||
"display_name": "Kimi Code CLI",
|
||||
"command": [
|
||||
"kimi",
|
||||
"--print",
|
||||
"--input-format",
|
||||
"text",
|
||||
"--work-dir",
|
||||
"{cwd}",
|
||||
],
|
||||
"version_check": ["kimi", "--version"],
|
||||
"uses_stdin": True,
|
||||
"parse_output": "kimi", # Needs special output parsing
|
||||
},
|
||||
}
|
||||
|
||||
# Default models per API provider
|
||||
DEFAULT_MODELS = {
|
||||
"anthropic": "claude-sonnet-4-20250514",
|
||||
"moonshot": "moonshot-v1-auto",
|
||||
"google": "gemini-2.0-flash",
|
||||
"openai": "gpt-4o",
|
||||
}
|
||||
|
||||
# API key env var → provider mapping
|
||||
API_KEY_MAP = {
|
||||
"ANTHROPIC_API_KEY": "anthropic",
|
||||
"ANTHROPIC_AUTH_TOKEN": "anthropic",
|
||||
"MOONSHOT_API_KEY": "moonshot",
|
||||
"GOOGLE_API_KEY": "google",
|
||||
"OPENAI_API_KEY": "openai",
|
||||
}
|
||||
|
||||
DEFAULT_ENHANCE_TIMEOUT = 2700 # 45 minutes
|
||||
UNLIMITED_TIMEOUT = 86400 # 24 hours (subprocess requires a finite number)
|
||||
|
||||
|
||||
def get_default_timeout() -> int:
|
||||
"""Return default enhancement timeout in seconds.
|
||||
|
||||
Priority:
|
||||
1. SKILL_SEEKER_ENHANCE_TIMEOUT environment variable
|
||||
2. DEFAULT_ENHANCE_TIMEOUT (45 minutes)
|
||||
|
||||
Supports 'unlimited' or 0/negative values which map to UNLIMITED_TIMEOUT (24h).
|
||||
"""
|
||||
env_val = os.environ.get("SKILL_SEEKER_ENHANCE_TIMEOUT", "").strip().lower()
|
||||
if env_val in ("unlimited", "none", "0"):
|
||||
return UNLIMITED_TIMEOUT
|
||||
try:
|
||||
timeout = int(env_val)
|
||||
if timeout <= 0:
|
||||
return UNLIMITED_TIMEOUT
|
||||
return timeout
|
||||
except ValueError:
|
||||
return DEFAULT_ENHANCE_TIMEOUT
|
||||
|
||||
|
||||
# Provider → target platform mapping (for --target defaults)
|
||||
PROVIDER_TARGET_MAP = {
|
||||
"anthropic": "claude",
|
||||
"moonshot": "kimi",
|
||||
"google": "gemini",
|
||||
"openai": "openai",
|
||||
}
|
||||
|
||||
|
||||
def normalize_agent_name(agent_name: str) -> str:
|
||||
"""Normalize agent name to canonical form."""
|
||||
if not agent_name:
|
||||
return "claude"
|
||||
normalized = agent_name.strip().lower()
|
||||
aliases = {
|
||||
"claude-code": "claude",
|
||||
"claude_code": "claude",
|
||||
"codex-cli": "codex",
|
||||
"copilot-cli": "copilot",
|
||||
"open-code": "opencode",
|
||||
"open_code": "opencode",
|
||||
"kimi-cli": "kimi",
|
||||
"kimi_code": "kimi",
|
||||
"kimi-code": "kimi",
|
||||
}
|
||||
return aliases.get(normalized, normalized)
|
||||
|
||||
|
||||
class AgentClient:
|
||||
"""
|
||||
Unified AI client that routes to API or LOCAL agent based on configuration.
|
||||
|
||||
All enhancers should use this instead of direct subprocess calls or API imports.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
mode: str = "auto",
|
||||
agent: str | None = None,
|
||||
api_key: str | None = None,
|
||||
):
|
||||
"""
|
||||
Initialize the agent client.
|
||||
|
||||
Args:
|
||||
mode: "auto" (detect from env), "api" (force API), "local" (force CLI agent)
|
||||
agent: LOCAL mode agent name ("claude", "kimi", "codex", "copilot", "opencode", "custom")
|
||||
Resolved from: arg → env SKILL_SEEKER_AGENT → "claude"
|
||||
api_key: API key override. If None, auto-detected from env vars.
|
||||
"""
|
||||
# Resolve agent name
|
||||
env_agent = os.environ.get("SKILL_SEEKER_AGENT", "").strip()
|
||||
self.agent = normalize_agent_name(agent or env_agent or "claude")
|
||||
self.agent_display = AGENT_PRESETS.get(self.agent, {}).get("display_name", self.agent)
|
||||
|
||||
# Detect API key and provider
|
||||
if api_key:
|
||||
self.api_key = api_key
|
||||
# Detect provider from key prefix or env vars
|
||||
self.provider = self._detect_provider_from_key(api_key)
|
||||
else:
|
||||
self.api_key, self.provider = self.detect_api_key()
|
||||
|
||||
# Determine mode (keep original for error handling decisions)
|
||||
self._requested_mode = mode
|
||||
self.mode = mode
|
||||
if mode == "auto":
|
||||
if self.api_key:
|
||||
self.mode = "api"
|
||||
else:
|
||||
self.mode = "local"
|
||||
|
||||
# Initialize API client if needed
|
||||
self.client = None
|
||||
if self.mode == "api" and self.api_key:
|
||||
self.client = self._init_api_client()
|
||||
|
||||
@staticmethod
|
||||
def _detect_provider_from_key(api_key: str) -> str:
|
||||
"""Detect provider from API key prefix or fall back to env var check."""
|
||||
if api_key.startswith("sk-ant-"):
|
||||
return "anthropic"
|
||||
if api_key.startswith("sk-"):
|
||||
# Could be OpenAI or Moonshot — check env vars
|
||||
if os.environ.get("MOONSHOT_API_KEY", "").strip() == api_key:
|
||||
return "moonshot"
|
||||
return "openai"
|
||||
if api_key.startswith("AIza"):
|
||||
return "google"
|
||||
# Default: check which env var matches
|
||||
for env_var, provider in API_KEY_MAP.items():
|
||||
if os.environ.get(env_var, "").strip() == api_key:
|
||||
return provider
|
||||
return "anthropic" # Safe fallback
|
||||
|
||||
def _init_api_client(self):
|
||||
"""Initialize the API client based on detected provider."""
|
||||
try:
|
||||
if self.provider == "anthropic":
|
||||
import anthropic
|
||||
|
||||
kwargs = {"api_key": self.api_key}
|
||||
base_url = os.environ.get("ANTHROPIC_BASE_URL")
|
||||
if base_url:
|
||||
kwargs["base_url"] = base_url
|
||||
return anthropic.Anthropic(**kwargs)
|
||||
elif self.provider == "moonshot":
|
||||
import anthropic
|
||||
|
||||
return anthropic.Anthropic(
|
||||
api_key=self.api_key,
|
||||
base_url="https://api.moonshot.cn/v1",
|
||||
)
|
||||
elif self.provider == "openai":
|
||||
from openai import OpenAI
|
||||
|
||||
return OpenAI(api_key=self.api_key)
|
||||
elif self.provider == "google":
|
||||
import google.generativeai as genai
|
||||
|
||||
genai.configure(api_key=self.api_key)
|
||||
return genai
|
||||
except ImportError as e:
|
||||
logger.info(f"{self.provider} SDK not installed, falling back to LOCAL mode: {e}")
|
||||
self.mode = "local"
|
||||
except Exception as e:
|
||||
if self._requested_mode == "api":
|
||||
raise RuntimeError(f"Failed to initialize {self.provider} API client: {e}") from e
|
||||
logger.error(f"Failed to initialize {self.provider} API client: {e}")
|
||||
self.mode = "local"
|
||||
return None
|
||||
|
||||
def call(
|
||||
self,
|
||||
prompt: str,
|
||||
max_tokens: int = 4096,
|
||||
timeout: int | None = None,
|
||||
output_file: str | Path | None = None,
|
||||
cwd: str | Path | None = None,
|
||||
) -> str | None:
|
||||
"""
|
||||
Call the AI agent (API or LOCAL mode).
|
||||
|
||||
Args:
|
||||
prompt: The prompt to send
|
||||
max_tokens: Max response tokens (API mode only)
|
||||
timeout: Timeout in seconds (default from SKILL_SEEKER_ENHANCE_TIMEOUT or 2700 = 45m)
|
||||
output_file: Path for agent to write output (LOCAL mode, some agents)
|
||||
cwd: Working directory for LOCAL mode subprocess
|
||||
|
||||
Returns:
|
||||
Response text, or None on failure
|
||||
"""
|
||||
if timeout is None:
|
||||
timeout = get_default_timeout()
|
||||
|
||||
if self.mode == "api":
|
||||
return self._call_api(prompt, max_tokens)
|
||||
elif self.mode == "local":
|
||||
return self._call_local(prompt, timeout, output_file, cwd)
|
||||
return None
|
||||
|
||||
def _call_api(self, prompt: str, max_tokens: int = 4096) -> str | None:
|
||||
"""Call via API using the detected provider."""
|
||||
if not self.client:
|
||||
return None
|
||||
|
||||
model = self.get_model(self.provider)
|
||||
|
||||
try:
|
||||
if self.provider in ("anthropic", "moonshot"):
|
||||
response = self.client.messages.create(
|
||||
model=model,
|
||||
max_tokens=max_tokens,
|
||||
messages=[{"role": "user", "content": prompt}],
|
||||
timeout=120,
|
||||
)
|
||||
return response.content[0].text
|
||||
|
||||
elif self.provider == "openai":
|
||||
response = self.client.chat.completions.create(
|
||||
model=model,
|
||||
max_tokens=max_tokens,
|
||||
messages=[{"role": "user", "content": prompt}],
|
||||
timeout=120,
|
||||
)
|
||||
return response.choices[0].message.content
|
||||
|
||||
elif self.provider == "google":
|
||||
gmodel = self.client.GenerativeModel(model)
|
||||
response = gmodel.generate_content(prompt)
|
||||
return response.text
|
||||
|
||||
except Exception as e:
|
||||
error_type = type(e).__name__
|
||||
error_module = type(e).__module__ or ""
|
||||
|
||||
# Rate limit errors
|
||||
if "rate" in error_type.lower() or "ratelimit" in error_type.lower():
|
||||
logger.error(
|
||||
f"{self.provider} API rate limited: {e}. "
|
||||
"Retry after waiting or reduce request frequency."
|
||||
)
|
||||
return None
|
||||
|
||||
# Auth / permission errors
|
||||
if "auth" in error_type.lower() or "permission" in error_type.lower():
|
||||
logger.error(
|
||||
f"{self.provider} API authentication failed: {e}. "
|
||||
"Check your API key is valid and has sufficient permissions."
|
||||
)
|
||||
return None
|
||||
|
||||
# Timeout / connection errors
|
||||
if (
|
||||
any(
|
||||
kw in error_type.lower()
|
||||
for kw in ("timeout", "connect", "connection", "network")
|
||||
)
|
||||
or "httpx" in error_module.lower()
|
||||
):
|
||||
logger.error(
|
||||
f"{self.provider} API connection error: {e}. "
|
||||
"Check your network connectivity and try again."
|
||||
)
|
||||
return None
|
||||
|
||||
# All other errors
|
||||
logger.error(f"{self.provider} API call failed: {e}")
|
||||
return None
|
||||
|
||||
def _call_local(
|
||||
self,
|
||||
prompt: str,
|
||||
timeout: int | None = None,
|
||||
output_file: str | Path | None = None,
|
||||
cwd: str | Path | None = None,
|
||||
) -> str | None:
|
||||
"""Call via LOCAL CLI agent using agent presets."""
|
||||
if timeout is None:
|
||||
timeout = get_default_timeout()
|
||||
# Handle custom agent from env var
|
||||
if self.agent == "custom":
|
||||
custom_cmd = os.environ.get("SKILL_SEEKER_AGENT_CMD", "").strip()
|
||||
if not custom_cmd:
|
||||
logger.warning("⚠️ Custom agent selected but SKILL_SEEKER_AGENT_CMD not set")
|
||||
return None
|
||||
preset = {
|
||||
"display_name": "Custom Agent",
|
||||
"command": custom_cmd.split(),
|
||||
"version_check": custom_cmd.split()[:1] + ["--version"],
|
||||
}
|
||||
else:
|
||||
preset = AGENT_PRESETS.get(self.agent)
|
||||
if not preset:
|
||||
logger.warning(f"⚠️ Unknown agent: {self.agent}")
|
||||
return None
|
||||
|
||||
try:
|
||||
with tempfile.TemporaryDirectory(prefix="agent_client_") as temp_dir:
|
||||
temp_path = Path(temp_dir)
|
||||
prompt_file = temp_path / "prompt.md"
|
||||
resp_file = Path(output_file) if output_file else (temp_path / "response.json")
|
||||
|
||||
# Only append output file instruction when caller explicitly requests it
|
||||
full_prompt = prompt
|
||||
if output_file:
|
||||
full_prompt += f"\n\nWrite your response to: {resp_file}\n"
|
||||
|
||||
prompt_file.write_text(full_prompt)
|
||||
|
||||
# Build command from preset
|
||||
cmd = []
|
||||
for part in preset["command"]:
|
||||
part = part.replace("{prompt_file}", str(prompt_file))
|
||||
part = part.replace("{cwd}", str(cwd or temp_path))
|
||||
part = part.replace("{skill_dir}", str(cwd or temp_path))
|
||||
cmd.append(part)
|
||||
|
||||
# Execute — pipe stdin for agents that read from it (e.g., codex)
|
||||
stdin_input = full_prompt if preset.get("uses_stdin") else None
|
||||
result = subprocess.run(
|
||||
cmd,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=timeout,
|
||||
cwd=str(cwd or temp_path),
|
||||
input=stdin_input,
|
||||
)
|
||||
|
||||
if result.returncode != 0:
|
||||
logger.error(f"{self.agent_display} returned error code {result.returncode}")
|
||||
if result.stderr and result.stderr.strip():
|
||||
logger.error(f"{self.agent_display} stderr: {result.stderr.strip()}")
|
||||
|
||||
# Try to read output file first
|
||||
resp_path = Path(resp_file)
|
||||
if resp_path.exists():
|
||||
return resp_path.read_text(encoding="utf-8")
|
||||
|
||||
# Try any JSON file in temp dir
|
||||
for json_file in temp_path.glob("*.json"):
|
||||
if json_file.name != "prompt.json":
|
||||
return json_file.read_text(encoding="utf-8")
|
||||
|
||||
# Fall back to stdout (with agent-specific parsing)
|
||||
if result.stdout and result.stdout.strip():
|
||||
stdout = result.stdout.strip()
|
||||
parser = preset.get("parse_output")
|
||||
if parser == "kimi":
|
||||
stdout = self._parse_kimi_output(stdout)
|
||||
return stdout
|
||||
|
||||
logger.warning(f"⚠️ No output from {self.agent_display}")
|
||||
return None
|
||||
|
||||
except subprocess.TimeoutExpired:
|
||||
logger.warning(f"⚠️ {self.agent_display} timeout ({timeout}s)")
|
||||
return None
|
||||
except FileNotFoundError:
|
||||
logger.warning(
|
||||
f"⚠️ {self.agent_display} CLI not found. "
|
||||
f"Install it or set SKILL_SEEKER_AGENT to a different agent."
|
||||
)
|
||||
return None
|
||||
except Exception as e:
|
||||
logger.error(f"{self.agent_display} error: {e}")
|
||||
return None
|
||||
|
||||
@staticmethod
|
||||
def _parse_kimi_output(raw_output: str) -> str:
|
||||
"""Parse Kimi CLI --print mode output to extract text content.
|
||||
|
||||
Kimi's --print mode outputs structured lines like:
|
||||
TurnBegin(...)
|
||||
StepBegin(...)
|
||||
TextPart(type='text', text='actual content')
|
||||
ThinkPart(type='think', think='...')
|
||||
|
||||
This extracts the text= values from TextPart lines.
|
||||
"""
|
||||
import re
|
||||
|
||||
text_parts = re.findall(r"TextPart\(type='text', text='(.+?)'\)", raw_output)
|
||||
if text_parts:
|
||||
return "\n".join(text_parts)
|
||||
# Fallback: return raw if no TextPart found
|
||||
return raw_output
|
||||
|
||||
def is_available(self) -> bool:
|
||||
"""Check if the configured agent/API is available."""
|
||||
if self.mode == "api":
|
||||
return self.client is not None
|
||||
|
||||
# LOCAL mode: check if CLI exists
|
||||
preset = AGENT_PRESETS.get(self.agent)
|
||||
if not preset:
|
||||
return False
|
||||
|
||||
version_cmd = preset.get("version_check")
|
||||
if not version_cmd:
|
||||
return shutil.which(preset["command"][0]) is not None
|
||||
|
||||
try:
|
||||
result = subprocess.run(
|
||||
version_cmd,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=5,
|
||||
)
|
||||
return result.returncode == 0
|
||||
except (FileNotFoundError, subprocess.TimeoutExpired):
|
||||
return False
|
||||
|
||||
@staticmethod
|
||||
def detect_api_key() -> tuple[str | None, str | None]:
|
||||
"""
|
||||
Detect API key from environment variables.
|
||||
|
||||
Returns:
|
||||
(api_key, provider) tuple. Provider is "anthropic", "moonshot", "google", or "openai".
|
||||
Returns (None, None) if no key found.
|
||||
"""
|
||||
for env_var, provider in API_KEY_MAP.items():
|
||||
key = os.environ.get(env_var, "").strip()
|
||||
if key:
|
||||
return key, provider
|
||||
return None, None
|
||||
|
||||
@staticmethod
|
||||
def get_model(provider: str = "anthropic") -> str:
|
||||
"""
|
||||
Get the model name for a provider.
|
||||
|
||||
Checks SKILL_SEEKER_MODEL env var first, then provider-specific env vars,
|
||||
then falls back to defaults.
|
||||
"""
|
||||
# Global override
|
||||
global_model = os.environ.get("SKILL_SEEKER_MODEL", "").strip()
|
||||
if global_model:
|
||||
return global_model
|
||||
|
||||
# Provider-specific env vars
|
||||
provider_env_map = {
|
||||
"anthropic": "ANTHROPIC_MODEL",
|
||||
"moonshot": "MOONSHOT_MODEL",
|
||||
"google": "GOOGLE_MODEL",
|
||||
"openai": "OPENAI_MODEL",
|
||||
}
|
||||
env_var = provider_env_map.get(provider)
|
||||
if env_var:
|
||||
model = os.environ.get(env_var, "").strip()
|
||||
if model:
|
||||
return model
|
||||
|
||||
return DEFAULT_MODELS.get(provider, "claude-sonnet-4-20250514")
|
||||
|
||||
@staticmethod
|
||||
def detect_default_target() -> str:
|
||||
"""
|
||||
Auto-detect the default --target platform from available API keys.
|
||||
|
||||
Returns platform name: "claude", "kimi", "gemini", "openai", or "markdown" (fallback).
|
||||
"""
|
||||
_, provider = AgentClient.detect_api_key()
|
||||
if provider:
|
||||
return PROVIDER_TARGET_MAP.get(provider, "markdown")
|
||||
return "markdown"
|
||||
|
||||
def log_mode(self) -> None:
|
||||
"""Log the current mode and agent for UX."""
|
||||
if self.mode == "api":
|
||||
logger.info(f"✅ AI enhancement enabled (using {self.provider} API)")
|
||||
elif self.mode == "local":
|
||||
logger.info(f"✅ AI enhancement enabled (using LOCAL mode - {self.agent_display})")
|
||||
else:
|
||||
logger.info("ℹ️ AI enhancement disabled")
|
||||
@@ -13,23 +13,17 @@ Features:
|
||||
- Identifies best practices
|
||||
|
||||
Modes:
|
||||
- API mode: Uses Claude API (requires ANTHROPIC_API_KEY)
|
||||
- LOCAL mode: Uses Claude Code CLI (no API key needed, uses your Claude Max plan)
|
||||
- API mode: Uses AI API (Anthropic, Moonshot/Kimi, Google, OpenAI)
|
||||
- LOCAL mode: Uses AI coding agent CLI (Claude Code, Kimi, Codex, Copilot, OpenCode)
|
||||
- AUTO mode: Tries API first, falls back to LOCAL
|
||||
|
||||
Credits:
|
||||
- Uses Claude AI (Anthropic) for analysis
|
||||
- Graceful degradation if API unavailable
|
||||
Uses AgentClient for all AI invocations — fully agent-agnostic.
|
||||
"""
|
||||
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import subprocess
|
||||
import tempfile
|
||||
from concurrent.futures import ThreadPoolExecutor, as_completed
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -55,24 +49,25 @@ class AIAnalysis:
|
||||
|
||||
|
||||
class AIEnhancer:
|
||||
"""Base class for AI enhancement"""
|
||||
"""Base class for AI enhancement — delegates to AgentClient for all AI calls."""
|
||||
|
||||
def __init__(self, api_key: str | None = None, enabled: bool = True, mode: str = "auto"):
|
||||
def __init__(
|
||||
self,
|
||||
api_key: str | None = None,
|
||||
enabled: bool = True,
|
||||
mode: str = "auto",
|
||||
agent: str | None = None,
|
||||
):
|
||||
"""
|
||||
Initialize AI enhancer.
|
||||
|
||||
Args:
|
||||
api_key: Anthropic API key (uses ANTHROPIC_API_KEY env if None)
|
||||
api_key: API key (auto-detected from env if None)
|
||||
enabled: Enable AI enhancement (default: True)
|
||||
mode: Enhancement mode - "auto" (default), "api", or "local"
|
||||
- "auto": Use API if key available, otherwise fall back to LOCAL
|
||||
- "api": Force API mode (fails if no key)
|
||||
- "local": Use Claude Code CLI (no API key needed)
|
||||
agent: Local CLI agent name (e.g., "kimi", "claude")
|
||||
"""
|
||||
self.enabled = enabled
|
||||
self.mode = mode
|
||||
self.api_key = api_key or os.environ.get("ANTHROPIC_API_KEY")
|
||||
self.client = None
|
||||
|
||||
# Get settings from config (with defaults)
|
||||
if CONFIG_AVAILABLE:
|
||||
@@ -80,161 +75,35 @@ class AIEnhancer:
|
||||
self.local_batch_size = config.get_local_batch_size()
|
||||
self.local_parallel_workers = config.get_local_parallel_workers()
|
||||
else:
|
||||
self.local_batch_size = 20 # Default
|
||||
self.local_parallel_workers = 3 # Default
|
||||
self.local_batch_size = 20
|
||||
self.local_parallel_workers = 3
|
||||
|
||||
# Determine actual mode
|
||||
if mode == "auto":
|
||||
if self.api_key:
|
||||
self.mode = "api"
|
||||
# Initialize AgentClient
|
||||
from skill_seekers.cli.agent_client import AgentClient
|
||||
|
||||
self._agent = AgentClient(mode=mode, api_key=api_key, agent=agent)
|
||||
self.mode = self._agent.mode
|
||||
self.client = self._agent.client # For backward compatibility
|
||||
|
||||
if self.enabled:
|
||||
if self._agent.is_available():
|
||||
self._agent.log_mode()
|
||||
else:
|
||||
# Fall back to LOCAL mode (Claude Code CLI)
|
||||
self.mode = "local"
|
||||
logger.info("ℹ️ No API key found, using LOCAL mode (Claude Code CLI)")
|
||||
|
||||
if self.mode == "api" and self.enabled:
|
||||
try:
|
||||
import anthropic
|
||||
|
||||
# Support custom base_url for GLM-4.7 and other Claude-compatible APIs
|
||||
client_kwargs = {"api_key": self.api_key}
|
||||
base_url = os.environ.get("ANTHROPIC_BASE_URL")
|
||||
if base_url:
|
||||
client_kwargs["base_url"] = base_url
|
||||
logger.info(f"✅ Using custom API base URL: {base_url}")
|
||||
self.client = anthropic.Anthropic(**client_kwargs)
|
||||
logger.info("✅ AI enhancement enabled (using Claude API)")
|
||||
except ImportError:
|
||||
logger.warning("⚠️ anthropic package not installed, falling back to LOCAL mode")
|
||||
self.mode = "local"
|
||||
except Exception as e:
|
||||
logger.warning(
|
||||
f"⚠️ Failed to initialize API client: {e}, falling back to LOCAL mode"
|
||||
f"⚠️ {self._agent.agent_display} not available. AI enhancement disabled."
|
||||
)
|
||||
self.mode = "local"
|
||||
|
||||
if self.mode == "local" and self.enabled:
|
||||
# Verify Claude CLI is available
|
||||
if self._check_claude_cli():
|
||||
logger.info("✅ AI enhancement enabled (using LOCAL mode - Claude Code CLI)")
|
||||
else:
|
||||
logger.warning("⚠️ Claude Code CLI not found. AI enhancement disabled.")
|
||||
logger.warning(" Install with: npm install -g @anthropic-ai/claude-code")
|
||||
self.enabled = False
|
||||
|
||||
def _check_claude_cli(self) -> bool:
|
||||
"""Check if Claude Code CLI is available"""
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["claude", "--version"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=5,
|
||||
)
|
||||
return result.returncode == 0
|
||||
except (FileNotFoundError, subprocess.TimeoutExpired):
|
||||
return False
|
||||
|
||||
def _call_claude(self, prompt: str, max_tokens: int = 1000) -> str | None:
|
||||
"""Call Claude (API or LOCAL mode) with error handling"""
|
||||
if self.mode == "api":
|
||||
return self._call_claude_api(prompt, max_tokens)
|
||||
elif self.mode == "local":
|
||||
return self._call_claude_local(prompt)
|
||||
return None
|
||||
"""Call AI agent (API or LOCAL mode) with error handling.
|
||||
|
||||
def _call_claude_api(self, prompt: str, max_tokens: int = 1000) -> str | None:
|
||||
"""Call Claude API"""
|
||||
if not self.client:
|
||||
return None
|
||||
Named _call_claude for backward compatibility — delegates to AgentClient.
|
||||
"""
|
||||
return self._agent.call(prompt, max_tokens=max_tokens)
|
||||
|
||||
try:
|
||||
response = self.client.messages.create(
|
||||
model="claude-sonnet-4-20250514",
|
||||
max_tokens=max_tokens,
|
||||
messages=[{"role": "user", "content": prompt}],
|
||||
)
|
||||
return response.content[0].text
|
||||
except Exception as e:
|
||||
logger.warning(f"⚠️ AI API call failed: {e}")
|
||||
return None
|
||||
|
||||
def _call_claude_local(self, prompt: str) -> str | None:
|
||||
"""Call Claude using LOCAL mode (Claude Code CLI)"""
|
||||
try:
|
||||
# Create a temporary directory for this enhancement
|
||||
with tempfile.TemporaryDirectory(prefix="ai_enhance_") as temp_dir:
|
||||
temp_path = Path(temp_dir)
|
||||
|
||||
# Create prompt file
|
||||
prompt_file = temp_path / "prompt.md"
|
||||
output_file = temp_path / "response.json"
|
||||
|
||||
# Write prompt with instructions to output JSON
|
||||
full_prompt = f"""# AI Analysis Task
|
||||
|
||||
IMPORTANT: You MUST write your response as valid JSON to this file:
|
||||
{output_file}
|
||||
|
||||
## Task
|
||||
|
||||
{prompt}
|
||||
|
||||
## Instructions
|
||||
|
||||
1. Analyze the input carefully
|
||||
2. Generate the JSON response as specified
|
||||
3. Use the Write tool to save the JSON to: {output_file}
|
||||
4. The JSON must be valid and parseable
|
||||
|
||||
DO NOT include any explanation - just write the JSON file.
|
||||
"""
|
||||
prompt_file.write_text(full_prompt)
|
||||
|
||||
# Run Claude CLI
|
||||
result = subprocess.run(
|
||||
["claude", "--dangerously-skip-permissions", str(prompt_file)],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=120, # 2 minute timeout per call
|
||||
cwd=str(temp_path),
|
||||
)
|
||||
|
||||
if result.returncode != 0:
|
||||
logger.warning(f"⚠️ Claude CLI returned error: {result.returncode}")
|
||||
return None
|
||||
|
||||
# Read output file
|
||||
if output_file.exists():
|
||||
response_text = output_file.read_text()
|
||||
# Try to extract JSON from response
|
||||
try:
|
||||
# Validate it's valid JSON
|
||||
json.loads(response_text)
|
||||
return response_text
|
||||
except json.JSONDecodeError:
|
||||
# Try to find JSON in the response
|
||||
import re
|
||||
|
||||
json_match = re.search(r"\[[\s\S]*\]|\{[\s\S]*\}", response_text)
|
||||
if json_match:
|
||||
return json_match.group()
|
||||
logger.warning("⚠️ Could not parse JSON from LOCAL response")
|
||||
return None
|
||||
else:
|
||||
# Look for any JSON file created
|
||||
for json_file in temp_path.glob("*.json"):
|
||||
if json_file.name != "prompt.json":
|
||||
return json_file.read_text()
|
||||
logger.warning("⚠️ No output file from LOCAL mode")
|
||||
return None
|
||||
|
||||
except subprocess.TimeoutExpired:
|
||||
logger.warning("⚠️ Claude CLI timeout (2 minutes)")
|
||||
return None
|
||||
except Exception as e:
|
||||
logger.warning(f"⚠️ LOCAL mode error: {e}")
|
||||
return None
|
||||
def call(self, prompt: str, max_tokens: int = 1000) -> str | None:
|
||||
"""Call AI agent — preferred method name over _call_claude."""
|
||||
return self._agent.call(prompt, max_tokens=max_tokens)
|
||||
|
||||
|
||||
class PatternEnhancer(AIEnhancer):
|
||||
|
||||
@@ -139,12 +139,13 @@ class ArchitecturalPatternDetector:
|
||||
"Laravel": ["laravel", "illuminate", "artisan", "app/Http/Controllers", "app/Models"],
|
||||
}
|
||||
|
||||
def __init__(self, enhance_with_ai: bool = True):
|
||||
def __init__(self, enhance_with_ai: bool = True, agent: str | None = None):
|
||||
"""
|
||||
Initialize detector.
|
||||
|
||||
Args:
|
||||
enhance_with_ai: Enable AI enhancement for detected patterns (C3.6)
|
||||
agent: Local CLI agent name (e.g., "kimi", "claude")
|
||||
"""
|
||||
self.enhance_with_ai = enhance_with_ai
|
||||
self.ai_enhancer = None
|
||||
@@ -153,7 +154,7 @@ class ArchitecturalPatternDetector:
|
||||
try:
|
||||
from skill_seekers.cli.ai_enhancer import AIEnhancer
|
||||
|
||||
self.ai_enhancer = AIEnhancer()
|
||||
self.ai_enhancer = AIEnhancer(agent=agent)
|
||||
except Exception as e:
|
||||
logger.warning(f"⚠️ Failed to initialize AI enhancer: {e}")
|
||||
self.enhance_with_ai = False
|
||||
|
||||
@@ -57,8 +57,8 @@ def add_asciidoc_arguments(parser: argparse.ArgumentParser) -> None:
|
||||
"AI enhancement level (auto-detects API vs LOCAL mode): "
|
||||
"0=disabled (default for AsciiDoc), 1=SKILL.md only, "
|
||||
"2=+architecture/config, 3=full enhancement. "
|
||||
"Mode selection: uses API if ANTHROPIC_API_KEY is set, "
|
||||
"otherwise LOCAL (Claude Code)"
|
||||
"Mode selection: uses API if API key is set (ANTHROPIC_API_KEY, MOONSHOT_API_KEY, etc.), "
|
||||
"otherwise LOCAL (AI coding agent)"
|
||||
)
|
||||
|
||||
# AsciiDoc-specific args
|
||||
|
||||
@@ -91,8 +91,8 @@ def add_chat_arguments(parser: argparse.ArgumentParser) -> None:
|
||||
"AI enhancement level (auto-detects API vs LOCAL mode): "
|
||||
"0=disabled (default for Chat), 1=SKILL.md only, "
|
||||
"2=+architecture/config, 3=full enhancement. "
|
||||
"Mode selection: uses API if ANTHROPIC_API_KEY is set, "
|
||||
"otherwise LOCAL (Claude Code)"
|
||||
"Mode selection: uses API if API key is set (ANTHROPIC_API_KEY, MOONSHOT_API_KEY, etc.), "
|
||||
"otherwise LOCAL (AI coding agent)"
|
||||
)
|
||||
|
||||
# Chat-specific args
|
||||
|
||||
@@ -55,7 +55,7 @@ COMMON_ARGUMENTS: dict[str, dict[str, Any]] = {
|
||||
"help": (
|
||||
"AI enhancement level (auto-detects API vs LOCAL mode): "
|
||||
"0=disabled, 1=SKILL.md only, 2=+architecture/config (default), 3=full enhancement. "
|
||||
"Mode selection: uses API if ANTHROPIC_API_KEY is set, otherwise LOCAL (Claude Code)"
|
||||
"Mode selection: uses API if API key is set (ANTHROPIC_API_KEY, MOONSHOT_API_KEY, etc.), otherwise LOCAL (AI coding agent)"
|
||||
),
|
||||
"metavar": "LEVEL",
|
||||
},
|
||||
@@ -64,10 +64,27 @@ COMMON_ARGUMENTS: dict[str, dict[str, Any]] = {
|
||||
"flags": ("--api-key",),
|
||||
"kwargs": {
|
||||
"type": str,
|
||||
"help": "Anthropic API key for --enhance (or set ANTHROPIC_API_KEY env var)",
|
||||
"help": "API key for enhancement (ANTHROPIC_API_KEY, GOOGLE_API_KEY, OPENAI_API_KEY, MOONSHOT_API_KEY)",
|
||||
"metavar": "KEY",
|
||||
},
|
||||
},
|
||||
"agent": {
|
||||
"flags": ("--agent",),
|
||||
"kwargs": {
|
||||
"type": str,
|
||||
"choices": ["claude", "codex", "copilot", "opencode", "kimi", "custom"],
|
||||
"help": "Local coding agent for enhancement (default: AI agent from SKILL_SEEKER_AGENT env var)",
|
||||
"metavar": "AGENT",
|
||||
},
|
||||
},
|
||||
"agent_cmd": {
|
||||
"flags": ("--agent-cmd",),
|
||||
"kwargs": {
|
||||
"type": str,
|
||||
"help": "Override agent command template (advanced)",
|
||||
"metavar": "CMD",
|
||||
},
|
||||
},
|
||||
"doc_version": {
|
||||
"flags": ("--doc-version",),
|
||||
"kwargs": {
|
||||
|
||||
@@ -98,8 +98,8 @@ def add_confluence_arguments(parser: argparse.ArgumentParser) -> None:
|
||||
"AI enhancement level (auto-detects API vs LOCAL mode): "
|
||||
"0=disabled (default for Confluence), 1=SKILL.md only, "
|
||||
"2=+architecture/config, 3=full enhancement. "
|
||||
"Mode selection: uses API if ANTHROPIC_API_KEY is set, "
|
||||
"otherwise LOCAL (Claude Code)"
|
||||
"Mode selection: uses API if API key is set (ANTHROPIC_API_KEY, MOONSHOT_API_KEY, etc.), "
|
||||
"otherwise LOCAL (AI coding agent)"
|
||||
)
|
||||
|
||||
# Confluence-specific args
|
||||
|
||||
@@ -57,7 +57,7 @@ UNIVERSAL_ARGUMENTS: dict[str, dict[str, Any]] = {
|
||||
"help": (
|
||||
"AI enhancement level (auto-detects API vs LOCAL mode): "
|
||||
"0=disabled, 1=SKILL.md only, 2=+architecture/config (default), 3=full enhancement. "
|
||||
"Mode selection: uses API if ANTHROPIC_API_KEY is set, otherwise LOCAL (Claude Code)"
|
||||
"Mode selection: uses API if API key is set (ANTHROPIC_API_KEY, MOONSHOT_API_KEY, etc.), otherwise LOCAL (AI coding agent)"
|
||||
),
|
||||
"metavar": "LEVEL",
|
||||
},
|
||||
@@ -66,7 +66,7 @@ UNIVERSAL_ARGUMENTS: dict[str, dict[str, Any]] = {
|
||||
"flags": ("--api-key",),
|
||||
"kwargs": {
|
||||
"type": str,
|
||||
"help": "Anthropic API key (or set ANTHROPIC_API_KEY env var)",
|
||||
"help": "API key for enhancement (ANTHROPIC_API_KEY, GOOGLE_API_KEY, OPENAI_API_KEY, MOONSHOT_API_KEY)",
|
||||
"metavar": "KEY",
|
||||
},
|
||||
},
|
||||
@@ -162,11 +162,30 @@ UNIVERSAL_ARGUMENTS: dict[str, dict[str, Any]] = {
|
||||
"metavar": "VERSION",
|
||||
},
|
||||
},
|
||||
# Agent selection for enhancement (added in v3.4.0 - multi-agent support)
|
||||
"agent": {
|
||||
"flags": ("--agent",),
|
||||
"kwargs": {
|
||||
"type": str,
|
||||
"choices": ["claude", "codex", "copilot", "opencode", "kimi", "custom"],
|
||||
"help": "Local coding agent for enhancement (default: AI agent from SKILL_SEEKER_AGENT env var)",
|
||||
"metavar": "AGENT",
|
||||
},
|
||||
},
|
||||
"agent_cmd": {
|
||||
"flags": ("--agent-cmd",),
|
||||
"kwargs": {
|
||||
"type": str,
|
||||
"help": "Override agent command template (advanced)",
|
||||
"metavar": "CMD",
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
# Merge RAG arguments from common.py into universal arguments
|
||||
UNIVERSAL_ARGUMENTS.update(RAG_ARGUMENTS)
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# TIER 2: SOURCE-SPECIFIC ARGUMENTS
|
||||
# =============================================================================
|
||||
@@ -376,6 +395,43 @@ LOCAL_ARGUMENTS: dict[str, dict[str, Any]] = {
|
||||
"help": "Skip documentation extraction",
|
||||
},
|
||||
},
|
||||
"skip_api_reference": {
|
||||
"flags": ("--skip-api-reference",),
|
||||
"kwargs": {
|
||||
"action": "store_true",
|
||||
"help": "Skip API reference generation",
|
||||
},
|
||||
},
|
||||
"skip_dependency_graph": {
|
||||
"flags": ("--skip-dependency-graph",),
|
||||
"kwargs": {
|
||||
"action": "store_true",
|
||||
"help": "Skip dependency graph analysis",
|
||||
},
|
||||
},
|
||||
"skip_config_patterns": {
|
||||
"flags": ("--skip-config-patterns",),
|
||||
"kwargs": {
|
||||
"action": "store_true",
|
||||
"help": "Skip configuration pattern extraction",
|
||||
},
|
||||
},
|
||||
"no_comments": {
|
||||
"flags": ("--no-comments",),
|
||||
"kwargs": {
|
||||
"action": "store_true",
|
||||
"help": "Skip comment extraction from source code",
|
||||
},
|
||||
},
|
||||
"depth": {
|
||||
"flags": ("--depth",),
|
||||
"kwargs": {
|
||||
"type": str,
|
||||
"choices": ["surface", "deep", "full"],
|
||||
"help": "Analysis depth (deprecated, use --preset instead)",
|
||||
"metavar": "LEVEL",
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
# PDF specific (from pdf.py)
|
||||
@@ -431,6 +487,13 @@ EPUB_ARGUMENTS: dict[str, dict[str, Any]] = {
|
||||
|
||||
# Video specific (from video.py)
|
||||
VIDEO_ARGUMENTS: dict[str, dict[str, Any]] = {
|
||||
"setup": {
|
||||
"flags": ("--setup",),
|
||||
"kwargs": {
|
||||
"action": "store_true",
|
||||
"help": "Auto-detect GPU and install video dependencies",
|
||||
},
|
||||
},
|
||||
"video_url": {
|
||||
"flags": ("--video-url",),
|
||||
"kwargs": {
|
||||
@@ -535,13 +598,14 @@ VIDEO_ARGUMENTS: dict[str, dict[str, Any]] = {
|
||||
}
|
||||
|
||||
# Multi-source config specific (from unified_scraper.py)
|
||||
# Note: --fresh is in WEB_ARGUMENTS, shared with config sources via dynamic forwarding
|
||||
CONFIG_ARGUMENTS: dict[str, dict[str, Any]] = {
|
||||
"merge_mode": {
|
||||
"flags": ("--merge-mode",),
|
||||
"kwargs": {
|
||||
"type": str,
|
||||
"choices": ["rule-based", "claude-enhanced"],
|
||||
"help": "Override merge mode from config (rule-based or claude-enhanced)",
|
||||
"choices": ["rule-based", "ai-enhanced", "claude-enhanced"],
|
||||
"help": "Override merge mode from config (rule-based or ai-enhanced). 'claude-enhanced' accepted as alias.",
|
||||
"metavar": "MODE",
|
||||
},
|
||||
},
|
||||
@@ -677,6 +741,14 @@ CHAT_ARGUMENTS: dict[str, dict[str, Any]] = {
|
||||
# Hidden from default help, shown only with --help-advanced
|
||||
|
||||
ADVANCED_ARGUMENTS: dict[str, dict[str, Any]] = {
|
||||
"from_json": {
|
||||
"flags": ("--from-json",),
|
||||
"kwargs": {
|
||||
"type": str,
|
||||
"help": "Build skill from pre-extracted JSON data (skip scraping). Supported by: PDF, Video, Jupyter, HTML, OpenAPI, AsciiDoc, PPTX, RSS, Manpage, Confluence, Notion, Chat.",
|
||||
"metavar": "PATH",
|
||||
},
|
||||
},
|
||||
"no_rate_limit": {
|
||||
"flags": ("--no-rate-limit",),
|
||||
"kwargs": {
|
||||
|
||||
@@ -22,12 +22,12 @@ ENHANCE_ARGUMENTS: dict[str, dict[str, Any]] = {
|
||||
"flags": ("--target",),
|
||||
"kwargs": {
|
||||
"type": str,
|
||||
"choices": ["claude", "gemini", "openai"],
|
||||
"choices": ["claude", "gemini", "openai", "kimi"],
|
||||
"help": (
|
||||
"AI platform for enhancement (uses API mode). "
|
||||
"Auto-detected from env vars if not specified: "
|
||||
"ANTHROPIC_API_KEY->claude, GOOGLE_API_KEY->gemini, OPENAI_API_KEY->openai. "
|
||||
"Falls back to LOCAL mode (Claude Code CLI) when no API keys are found."
|
||||
"ANTHROPIC_API_KEY->claude, GOOGLE_API_KEY->gemini, OPENAI_API_KEY->openai, MOONSHOT_API_KEY->kimi. "
|
||||
"Falls back to LOCAL mode (AI coding agent) when no API keys are found."
|
||||
),
|
||||
"metavar": "PLATFORM",
|
||||
},
|
||||
@@ -38,7 +38,7 @@ ENHANCE_ARGUMENTS: dict[str, dict[str, Any]] = {
|
||||
"type": str,
|
||||
"help": (
|
||||
"API key for the target platform "
|
||||
"(or set ANTHROPIC_API_KEY / GOOGLE_API_KEY / OPENAI_API_KEY)"
|
||||
"(or set ANTHROPIC_API_KEY / GOOGLE_API_KEY / OPENAI_API_KEY / MOONSHOT_API_KEY)"
|
||||
),
|
||||
"metavar": "KEY",
|
||||
},
|
||||
@@ -55,8 +55,8 @@ ENHANCE_ARGUMENTS: dict[str, dict[str, Any]] = {
|
||||
"flags": ("--agent",),
|
||||
"kwargs": {
|
||||
"type": str,
|
||||
"choices": ["claude", "codex", "copilot", "opencode", "custom"],
|
||||
"help": "Local coding agent to use (default: claude or SKILL_SEEKER_AGENT)",
|
||||
"choices": ["claude", "codex", "copilot", "opencode", "kimi", "custom"],
|
||||
"help": "Local coding agent to use (default: AI agent from SKILL_SEEKER_AGENT env var)",
|
||||
"metavar": "AGENT",
|
||||
},
|
||||
},
|
||||
@@ -101,8 +101,11 @@ ENHANCE_ARGUMENTS: dict[str, dict[str, Any]] = {
|
||||
"flags": ("--timeout",),
|
||||
"kwargs": {
|
||||
"type": int,
|
||||
"default": 600,
|
||||
"help": "Timeout in seconds (default: 600)",
|
||||
"default": None, # Resolved at runtime via get_default_timeout()
|
||||
"help": (
|
||||
"Timeout in seconds "
|
||||
"(default: 45 minutes, set SKILL_SEEKER_ENHANCE_TIMEOUT to override)"
|
||||
),
|
||||
"metavar": "SECONDS",
|
||||
},
|
||||
},
|
||||
|
||||
@@ -56,7 +56,7 @@ def add_epub_arguments(parser: argparse.ArgumentParser) -> None:
|
||||
action.help = (
|
||||
"AI enhancement level (auto-detects API vs LOCAL mode): "
|
||||
"0=disabled (default for EPUB), 1=SKILL.md only, 2=+architecture/config, 3=full enhancement. "
|
||||
"Mode selection: uses API if ANTHROPIC_API_KEY is set, otherwise LOCAL (Claude Code)"
|
||||
"Mode selection: uses API if API key is set (ANTHROPIC_API_KEY, MOONSHOT_API_KEY, etc.), otherwise LOCAL (AI coding agent)"
|
||||
)
|
||||
|
||||
# EPUB-specific args
|
||||
|
||||
@@ -57,8 +57,8 @@ def add_html_arguments(parser: argparse.ArgumentParser) -> None:
|
||||
"AI enhancement level (auto-detects API vs LOCAL mode): "
|
||||
"0=disabled (default for HTML), 1=SKILL.md only, "
|
||||
"2=+architecture/config, 3=full enhancement. "
|
||||
"Mode selection: uses API if ANTHROPIC_API_KEY is set, "
|
||||
"otherwise LOCAL (Claude Code)"
|
||||
"Mode selection: uses API if API key is set (ANTHROPIC_API_KEY, MOONSHOT_API_KEY, etc.), "
|
||||
"otherwise LOCAL (AI coding agent)"
|
||||
)
|
||||
|
||||
# HTML-specific args
|
||||
|
||||
@@ -57,8 +57,8 @@ def add_jupyter_arguments(parser: argparse.ArgumentParser) -> None:
|
||||
"AI enhancement level (auto-detects API vs LOCAL mode): "
|
||||
"0=disabled (default for Jupyter), 1=SKILL.md only, "
|
||||
"2=+architecture/config, 3=full enhancement. "
|
||||
"Mode selection: uses API if ANTHROPIC_API_KEY is set, "
|
||||
"otherwise LOCAL (Claude Code)"
|
||||
"Mode selection: uses API if API key is set (ANTHROPIC_API_KEY, MOONSHOT_API_KEY, etc.), "
|
||||
"otherwise LOCAL (AI coding agent)"
|
||||
)
|
||||
|
||||
# Jupyter-specific args
|
||||
|
||||
@@ -73,8 +73,8 @@ def add_manpage_arguments(parser: argparse.ArgumentParser) -> None:
|
||||
"AI enhancement level (auto-detects API vs LOCAL mode): "
|
||||
"0=disabled (default for ManPage), 1=SKILL.md only, "
|
||||
"2=+architecture/config, 3=full enhancement. "
|
||||
"Mode selection: uses API if ANTHROPIC_API_KEY is set, "
|
||||
"otherwise LOCAL (Claude Code)"
|
||||
"Mode selection: uses API if API key is set (ANTHROPIC_API_KEY, MOONSHOT_API_KEY, etc.), "
|
||||
"otherwise LOCAL (AI coding agent)"
|
||||
)
|
||||
|
||||
# ManPage-specific args
|
||||
|
||||
@@ -90,8 +90,8 @@ def add_notion_arguments(parser: argparse.ArgumentParser) -> None:
|
||||
"AI enhancement level (auto-detects API vs LOCAL mode): "
|
||||
"0=disabled (default for Notion), 1=SKILL.md only, "
|
||||
"2=+architecture/config, 3=full enhancement. "
|
||||
"Mode selection: uses API if ANTHROPIC_API_KEY is set, "
|
||||
"otherwise LOCAL (Claude Code)"
|
||||
"Mode selection: uses API if API key is set (ANTHROPIC_API_KEY, MOONSHOT_API_KEY, etc.), "
|
||||
"otherwise LOCAL (AI coding agent)"
|
||||
)
|
||||
|
||||
# Notion-specific args
|
||||
|
||||
@@ -65,8 +65,8 @@ def add_openapi_arguments(parser: argparse.ArgumentParser) -> None:
|
||||
"AI enhancement level (auto-detects API vs LOCAL mode): "
|
||||
"0=disabled (default for OpenAPI), 1=SKILL.md only, "
|
||||
"2=+architecture/config, 3=full enhancement. "
|
||||
"Mode selection: uses API if ANTHROPIC_API_KEY is set, "
|
||||
"otherwise LOCAL (Claude Code)"
|
||||
"Mode selection: uses API if API key is set (ANTHROPIC_API_KEY, MOONSHOT_API_KEY, etc.), "
|
||||
"otherwise LOCAL (AI coding agent)"
|
||||
)
|
||||
|
||||
# OpenAPI-specific args
|
||||
|
||||
@@ -43,6 +43,14 @@ PACKAGE_ARGUMENTS: dict[str, dict[str, Any]] = {
|
||||
"claude",
|
||||
"gemini",
|
||||
"openai",
|
||||
"kimi",
|
||||
"minimax",
|
||||
"opencode",
|
||||
"deepseek",
|
||||
"qwen",
|
||||
"openrouter",
|
||||
"together",
|
||||
"fireworks",
|
||||
"markdown",
|
||||
"langchain",
|
||||
"llama-index",
|
||||
@@ -53,8 +61,8 @@ PACKAGE_ARGUMENTS: dict[str, dict[str, Any]] = {
|
||||
"qdrant",
|
||||
"pinecone",
|
||||
],
|
||||
"default": "claude",
|
||||
"help": "Target LLM platform (default: claude)",
|
||||
"default": None,
|
||||
"help": "Target LLM platform (auto-detected from API keys, or 'markdown' if none set)",
|
||||
"metavar": "PLATFORM",
|
||||
},
|
||||
},
|
||||
@@ -133,6 +141,32 @@ PACKAGE_ARGUMENTS: dict[str, dict[str, Any]] = {
|
||||
"help": "Allow code block splitting (default: code blocks preserved)",
|
||||
},
|
||||
},
|
||||
# Marketplace options
|
||||
"marketplace": {
|
||||
"flags": ("--marketplace",),
|
||||
"kwargs": {
|
||||
"type": str,
|
||||
"default": None,
|
||||
"help": "Publish to registered marketplace after packaging (use add_marketplace to register)",
|
||||
"metavar": "NAME",
|
||||
},
|
||||
},
|
||||
"marketplace_category": {
|
||||
"flags": ("--marketplace-category",),
|
||||
"kwargs": {
|
||||
"type": str,
|
||||
"default": "development",
|
||||
"help": "Plugin category in marketplace (default: development)",
|
||||
"metavar": "CAT",
|
||||
},
|
||||
},
|
||||
"create_branch": {
|
||||
"flags": ("--create-branch",),
|
||||
"kwargs": {
|
||||
"action": "store_true",
|
||||
"help": "Create a feature branch in marketplace repo instead of committing to main",
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
|
||||
@@ -64,7 +64,7 @@ def add_pdf_arguments(parser: argparse.ArgumentParser) -> None:
|
||||
action.help = (
|
||||
"AI enhancement level (auto-detects API vs LOCAL mode): "
|
||||
"0=disabled (default for PDF), 1=SKILL.md only, 2=+architecture/config, 3=full enhancement. "
|
||||
"Mode selection: uses API if ANTHROPIC_API_KEY is set, otherwise LOCAL (Claude Code)"
|
||||
"Mode selection: uses API if API key is set (ANTHROPIC_API_KEY, MOONSHOT_API_KEY, etc.), otherwise LOCAL (AI coding agent)"
|
||||
)
|
||||
|
||||
# PDF-specific args
|
||||
|
||||
@@ -57,8 +57,8 @@ def add_pptx_arguments(parser: argparse.ArgumentParser) -> None:
|
||||
"AI enhancement level (auto-detects API vs LOCAL mode): "
|
||||
"0=disabled (default for PPTX), 1=SKILL.md only, "
|
||||
"2=+architecture/config, 3=full enhancement. "
|
||||
"Mode selection: uses API if ANTHROPIC_API_KEY is set, "
|
||||
"otherwise LOCAL (Claude Code)"
|
||||
"Mode selection: uses API if API key is set (ANTHROPIC_API_KEY, MOONSHOT_API_KEY, etc.), "
|
||||
"otherwise LOCAL (AI coding agent)"
|
||||
)
|
||||
|
||||
# PPTX-specific args
|
||||
|
||||
@@ -90,8 +90,8 @@ def add_rss_arguments(parser: argparse.ArgumentParser) -> None:
|
||||
"AI enhancement level (auto-detects API vs LOCAL mode): "
|
||||
"0=disabled (default for RSS), 1=SKILL.md only, "
|
||||
"2=+architecture/config, 3=full enhancement. "
|
||||
"Mode selection: uses API if ANTHROPIC_API_KEY is set, "
|
||||
"otherwise LOCAL (Claude Code)"
|
||||
"Mode selection: uses API if API key is set (ANTHROPIC_API_KEY, MOONSHOT_API_KEY, etc.), "
|
||||
"otherwise LOCAL (AI coding agent)"
|
||||
)
|
||||
|
||||
# RSS-specific args
|
||||
|
||||
@@ -22,7 +22,7 @@ UNIFIED_ARGUMENTS: dict[str, dict[str, Any]] = {
|
||||
"flags": ("--merge-mode",),
|
||||
"kwargs": {
|
||||
"type": str,
|
||||
"help": "Merge mode (rule-based, claude-enhanced)",
|
||||
"help": "Merge mode (rule-based, ai-enhanced). 'claude-enhanced' is accepted as alias.",
|
||||
"metavar": "MODE",
|
||||
},
|
||||
},
|
||||
@@ -72,12 +72,30 @@ UNIFIED_ARGUMENTS: dict[str, dict[str, Any]] = {
|
||||
"help": "Preview workflow stages without executing (requires --enhance-workflow)",
|
||||
},
|
||||
},
|
||||
# Agent selection for LOCAL mode enhancement
|
||||
"agent": {
|
||||
"flags": ("--agent",),
|
||||
"kwargs": {
|
||||
"type": str,
|
||||
"choices": ["claude", "codex", "copilot", "opencode", "kimi", "custom"],
|
||||
"help": "Local coding agent for enhancement (default: AI agent from SKILL_SEEKER_AGENT env var)",
|
||||
"metavar": "AGENT",
|
||||
},
|
||||
},
|
||||
"agent_cmd": {
|
||||
"flags": ("--agent-cmd",),
|
||||
"kwargs": {
|
||||
"type": str,
|
||||
"help": "Override agent command template (use {prompt_file} or stdin)",
|
||||
"metavar": "CMD",
|
||||
},
|
||||
},
|
||||
# API key and enhance-level (parity with scrape/github/analyze/pdf)
|
||||
"api_key": {
|
||||
"flags": ("--api-key",),
|
||||
"kwargs": {
|
||||
"type": str,
|
||||
"help": "Anthropic API key (or set ANTHROPIC_API_KEY env var)",
|
||||
"help": "API key for enhancement (ANTHROPIC_API_KEY, GOOGLE_API_KEY, OPENAI_API_KEY, or MOONSHOT_API_KEY)",
|
||||
"metavar": "KEY",
|
||||
},
|
||||
},
|
||||
|
||||
@@ -22,9 +22,9 @@ UPLOAD_ARGUMENTS: dict[str, dict[str, Any]] = {
|
||||
"flags": ("--target",),
|
||||
"kwargs": {
|
||||
"type": str,
|
||||
"choices": ["claude", "gemini", "openai", "chroma", "weaviate"],
|
||||
"default": "claude",
|
||||
"help": "Target platform (default: claude)",
|
||||
"choices": ["claude", "gemini", "openai", "kimi", "chroma", "weaviate"],
|
||||
"default": None,
|
||||
"help": "Target platform (auto-detected from API keys, or 'claude' if none set)",
|
||||
"metavar": "PLATFORM",
|
||||
},
|
||||
},
|
||||
|
||||
@@ -156,7 +156,7 @@ def add_video_arguments(parser: argparse.ArgumentParser) -> None:
|
||||
action.help = (
|
||||
"AI enhancement level (auto-detects API vs LOCAL mode): "
|
||||
"0=disabled (default for video), 1=SKILL.md only, 2=+architecture/config, 3=full enhancement. "
|
||||
"Mode selection: uses API if ANTHROPIC_API_KEY is set, otherwise LOCAL (Claude Code)"
|
||||
"Mode selection: uses API if API key is set (ANTHROPIC_API_KEY, MOONSHOT_API_KEY, etc.), otherwise LOCAL (AI coding agent)"
|
||||
)
|
||||
|
||||
# Video-specific args
|
||||
|
||||
@@ -56,7 +56,7 @@ def add_word_arguments(parser: argparse.ArgumentParser) -> None:
|
||||
action.help = (
|
||||
"AI enhancement level (auto-detects API vs LOCAL mode): "
|
||||
"0=disabled (default for Word), 1=SKILL.md only, 2=+architecture/config, 3=full enhancement. "
|
||||
"Mode selection: uses API if ANTHROPIC_API_KEY is set, otherwise LOCAL (Claude Code)"
|
||||
"Mode selection: uses API if API key is set (ANTHROPIC_API_KEY, MOONSHOT_API_KEY, etc.), otherwise LOCAL (AI coding agent)"
|
||||
)
|
||||
|
||||
# Word-specific args
|
||||
|
||||
@@ -1059,13 +1059,17 @@ def main() -> int:
|
||||
print("❌ API enhancement not available. Falling back to LOCAL mode...")
|
||||
from skill_seekers.cli.enhance_skill_local import LocalSkillEnhancer
|
||||
|
||||
enhancer = LocalSkillEnhancer(Path(skill_dir))
|
||||
agent = getattr(args, "agent", None) if args else None
|
||||
agent_cmd = getattr(args, "agent_cmd", None) if args else None
|
||||
enhancer = LocalSkillEnhancer(Path(skill_dir), agent=agent, agent_cmd=agent_cmd)
|
||||
enhancer.run(headless=True)
|
||||
print("✅ Local enhancement complete!")
|
||||
else:
|
||||
from skill_seekers.cli.enhance_skill_local import LocalSkillEnhancer
|
||||
|
||||
enhancer = LocalSkillEnhancer(Path(skill_dir))
|
||||
agent = getattr(args, "agent", None) if args else None
|
||||
agent_cmd = getattr(args, "agent_cmd", None) if args else None
|
||||
enhancer = LocalSkillEnhancer(Path(skill_dir), agent=agent, agent_cmd=agent_cmd)
|
||||
enhancer.run(headless=True)
|
||||
print("✅ Local enhancement complete!")
|
||||
|
||||
|
||||
@@ -64,12 +64,21 @@ class BrowserRenderer:
|
||||
html = renderer.render_page(url)
|
||||
"""
|
||||
|
||||
def __init__(self, timeout: int = 30000, wait_until: str = "networkidle"):
|
||||
def __init__(
|
||||
self,
|
||||
timeout: int = 60000,
|
||||
wait_until: str = "domcontentloaded",
|
||||
extra_wait: int = 0,
|
||||
):
|
||||
"""Initialize renderer.
|
||||
|
||||
Args:
|
||||
timeout: Page load timeout in milliseconds (default: 30s)
|
||||
timeout: Page load timeout in milliseconds (default: 60s)
|
||||
wait_until: Playwright wait condition — "networkidle", "load", "domcontentloaded"
|
||||
Default changed to "domcontentloaded" for better compatibility
|
||||
with heavy sites (Unity docs, DocFX, etc.) that never reach networkidle.
|
||||
extra_wait: Additional milliseconds to wait after page load for lazy-loaded
|
||||
navigation/content (e.g., 5000 for DocFX sidebar). Default: 0.
|
||||
"""
|
||||
if not _check_playwright_available():
|
||||
raise ImportError(
|
||||
@@ -80,6 +89,7 @@ class BrowserRenderer:
|
||||
|
||||
self._timeout = timeout
|
||||
self._wait_until = wait_until
|
||||
self._extra_wait = extra_wait
|
||||
self._playwright = None
|
||||
self._browser = None
|
||||
self._context = None
|
||||
@@ -127,6 +137,8 @@ class BrowserRenderer:
|
||||
page = self._context.new_page()
|
||||
try:
|
||||
page.goto(url, wait_until=self._wait_until, timeout=self._timeout)
|
||||
if self._extra_wait > 0:
|
||||
page.wait_for_timeout(self._extra_wait)
|
||||
html = page.content()
|
||||
return html
|
||||
finally:
|
||||
|
||||
@@ -1885,7 +1885,9 @@ Examples:
|
||||
LocalSkillEnhancer,
|
||||
)
|
||||
|
||||
enhancer = LocalSkillEnhancer(Path(skill_dir))
|
||||
agent = getattr(args, "agent", None) if args else None
|
||||
agent_cmd = getattr(args, "agent_cmd", None) if args else None
|
||||
enhancer = LocalSkillEnhancer(Path(skill_dir), agent=agent, agent_cmd=agent_cmd)
|
||||
enhancer.run(headless=True)
|
||||
print("✅ Local enhancement complete!")
|
||||
else:
|
||||
@@ -1893,7 +1895,9 @@ Examples:
|
||||
LocalSkillEnhancer,
|
||||
)
|
||||
|
||||
enhancer = LocalSkillEnhancer(Path(skill_dir))
|
||||
agent = getattr(args, "agent", None) if args else None
|
||||
agent_cmd = getattr(args, "agent_cmd", None) if args else None
|
||||
enhancer = LocalSkillEnhancer(Path(skill_dir), agent=agent, agent_cmd=agent_cmd)
|
||||
enhancer.run(headless=True)
|
||||
print("✅ Local enhancement complete!")
|
||||
|
||||
|
||||
@@ -650,6 +650,7 @@ def process_markdown_docs(
|
||||
gitignore_spec: pathspec.PathSpec | None = None,
|
||||
enhance_with_ai: bool = False,
|
||||
ai_mode: str = "none",
|
||||
agent: str | None = None,
|
||||
) -> dict[str, Any]:
|
||||
"""
|
||||
Process all markdown documentation files in a directory.
|
||||
@@ -822,7 +823,7 @@ def process_markdown_docs(
|
||||
if enhance_with_ai and ai_mode != "none" and processed_docs:
|
||||
logger.info("🤖 Enhancing documentation analysis with AI...")
|
||||
try:
|
||||
processed_docs = _enhance_docs_with_ai(processed_docs, ai_mode)
|
||||
processed_docs = _enhance_docs_with_ai(processed_docs, ai_mode, agent=agent)
|
||||
logger.info("✅ AI documentation enhancement complete")
|
||||
except Exception as e:
|
||||
logger.warning(f"⚠️ AI enhancement failed: {e}")
|
||||
@@ -898,55 +899,46 @@ def process_markdown_docs(
|
||||
return index_data
|
||||
|
||||
|
||||
def _enhance_docs_with_ai(docs: list[dict], ai_mode: str) -> list[dict]:
|
||||
def _enhance_docs_with_ai(docs: list[dict], ai_mode: str, agent: str | None = None) -> list[dict]:
|
||||
"""
|
||||
Enhance documentation analysis with AI.
|
||||
Enhance documentation analysis with AI via AgentClient.
|
||||
|
||||
Args:
|
||||
docs: List of processed document dictionaries
|
||||
ai_mode: AI mode ('api' or 'local')
|
||||
ai_mode: AI mode ('api', 'local', or 'auto')
|
||||
agent: Local CLI agent name (e.g., "kimi", "claude")
|
||||
|
||||
Returns:
|
||||
Enhanced document list
|
||||
"""
|
||||
# Try API mode first
|
||||
if ai_mode in ("api", "auto"):
|
||||
api_key = os.environ.get("ANTHROPIC_API_KEY")
|
||||
if api_key:
|
||||
return _enhance_docs_api(docs, api_key)
|
||||
from skill_seekers.cli.agent_client import AgentClient
|
||||
|
||||
# Fall back to LOCAL mode
|
||||
if ai_mode in ("local", "auto"):
|
||||
return _enhance_docs_local(docs)
|
||||
client = AgentClient(mode=ai_mode, agent=agent)
|
||||
|
||||
return docs
|
||||
if not client.is_available():
|
||||
logger.warning("⚠️ No AI agent available for documentation enhancement")
|
||||
return docs
|
||||
|
||||
# Batch documents for efficiency
|
||||
batch_size = 10
|
||||
docs_with_summary = [d for d in docs if d.get("summary")]
|
||||
if not docs_with_summary:
|
||||
return docs
|
||||
|
||||
def _enhance_docs_api(docs: list[dict], api_key: str) -> list[dict]:
|
||||
"""Enhance docs using Claude API."""
|
||||
try:
|
||||
import anthropic
|
||||
for i in range(0, len(docs_with_summary), batch_size):
|
||||
batch = docs_with_summary[i : i + batch_size]
|
||||
|
||||
client = anthropic.Anthropic(api_key=api_key)
|
||||
docs_text = "\n\n".join(
|
||||
[
|
||||
f"## {d.get('title', d['filename'])}\nCategory: {d['category']}\nSummary: {d.get('summary', 'N/A')}"
|
||||
for d in batch
|
||||
]
|
||||
)
|
||||
|
||||
# Batch documents for efficiency
|
||||
batch_size = 10
|
||||
for i in range(0, len(docs), batch_size):
|
||||
batch = docs[i : i + batch_size]
|
||||
if not docs_text:
|
||||
continue
|
||||
|
||||
# Create prompt for batch
|
||||
docs_text = "\n\n".join(
|
||||
[
|
||||
f"## {d.get('title', d['filename'])}\nCategory: {d['category']}\nSummary: {d.get('summary', 'N/A')}"
|
||||
for d in batch
|
||||
if d.get("summary")
|
||||
]
|
||||
)
|
||||
|
||||
if not docs_text:
|
||||
continue
|
||||
|
||||
prompt = f"""Analyze these documentation files and provide:
|
||||
prompt = f"""Analyze these documentation files and provide:
|
||||
1. A brief description of what each document covers
|
||||
2. Key topics/concepts mentioned
|
||||
3. How they relate to each other
|
||||
@@ -957,15 +949,10 @@ Documents:
|
||||
Return JSON with format:
|
||||
{{"enhancements": [{{"filename": "...", "description": "...", "key_topics": [...], "related_to": [...]}}]}}"""
|
||||
|
||||
response = client.messages.create(
|
||||
model="claude-sonnet-4-20250514",
|
||||
max_tokens=2000,
|
||||
messages=[{"role": "user", "content": prompt}],
|
||||
)
|
||||
|
||||
# Parse response and merge enhancements
|
||||
try:
|
||||
json_match = re.search(r"\{.*\}", response.content[0].text, re.DOTALL)
|
||||
try:
|
||||
response = client.call(prompt, max_tokens=2000)
|
||||
if response:
|
||||
json_match = re.search(r"\{.*\}", response, re.DOTALL)
|
||||
if json_match:
|
||||
enhancements = json.loads(json_match.group())
|
||||
for enh in enhancements.get("enhancements", []):
|
||||
@@ -974,72 +961,8 @@ Return JSON with format:
|
||||
doc["ai_description"] = enh.get("description")
|
||||
doc["ai_topics"] = enh.get("key_topics", [])
|
||||
doc["ai_related"] = enh.get("related_to", [])
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
except Exception as e:
|
||||
logger.warning(f"API enhancement failed: {e}")
|
||||
|
||||
return docs
|
||||
|
||||
|
||||
def _enhance_docs_local(docs: list[dict]) -> list[dict]:
|
||||
"""Enhance docs using Claude Code CLI (LOCAL mode)."""
|
||||
import subprocess
|
||||
import tempfile
|
||||
|
||||
# Prepare batch of docs for enhancement
|
||||
docs_with_summary = [d for d in docs if d.get("summary")]
|
||||
if not docs_with_summary:
|
||||
return docs
|
||||
|
||||
docs_text = "\n\n".join(
|
||||
[
|
||||
f"## {d.get('title', d['filename'])}\nCategory: {d['category']}\nPath: {d['path']}\nSummary: {d.get('summary', 'N/A')}"
|
||||
for d in docs_with_summary[:20] # Limit to 20 docs
|
||||
]
|
||||
)
|
||||
|
||||
prompt = f"""Analyze these documentation files from a codebase and provide insights.
|
||||
|
||||
For each document, provide:
|
||||
1. A brief description of what it covers
|
||||
2. Key topics/concepts
|
||||
3. Related documents
|
||||
|
||||
Documents:
|
||||
{docs_text}
|
||||
|
||||
Output JSON only:
|
||||
{{"enhancements": [{{"filename": "...", "description": "...", "key_topics": ["..."], "related_to": ["..."]}}]}}"""
|
||||
|
||||
try:
|
||||
with tempfile.NamedTemporaryFile(mode="w", suffix=".txt", delete=False) as f:
|
||||
f.write(prompt)
|
||||
prompt_file = f.name
|
||||
|
||||
result = subprocess.run(
|
||||
["claude", "--dangerously-skip-permissions", "-p", prompt],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=120,
|
||||
)
|
||||
|
||||
os.unlink(prompt_file)
|
||||
|
||||
if result.returncode == 0 and result.stdout:
|
||||
json_match = re.search(r"\{.*\}", result.stdout, re.DOTALL)
|
||||
if json_match:
|
||||
enhancements = json.loads(json_match.group())
|
||||
for enh in enhancements.get("enhancements", []):
|
||||
for doc in docs:
|
||||
if doc["filename"] == enh.get("filename"):
|
||||
doc["ai_description"] = enh.get("description")
|
||||
doc["ai_topics"] = enh.get("key_topics", [])
|
||||
doc["ai_related"] = enh.get("related_to", [])
|
||||
|
||||
except Exception as e:
|
||||
logger.warning(f"LOCAL enhancement failed: {e}")
|
||||
except Exception as e:
|
||||
logger.warning(f"AI enhancement batch failed: {e}")
|
||||
|
||||
return docs
|
||||
|
||||
@@ -1062,6 +985,8 @@ def analyze_codebase(
|
||||
skill_name: str | None = None,
|
||||
skill_description: str | None = None,
|
||||
doc_version: str = "",
|
||||
agent: str | None = None,
|
||||
agent_cmd: str | None = None,
|
||||
) -> dict[str, Any]:
|
||||
"""
|
||||
Analyze local codebase and extract code knowledge.
|
||||
@@ -1467,7 +1392,7 @@ def analyze_codebase(
|
||||
from skill_seekers.cli.config_enhancer import ConfigEnhancer
|
||||
|
||||
logger.info(f"🤖 Enhancing config analysis with AI (mode: {ai_mode})...")
|
||||
enhancer = ConfigEnhancer(mode=ai_mode)
|
||||
enhancer = ConfigEnhancer(mode=ai_mode, agent=agent)
|
||||
result_dict = enhancer.enhance_config_result(result_dict)
|
||||
logger.info("✅ AI enhancement complete")
|
||||
except Exception as e:
|
||||
@@ -1514,7 +1439,7 @@ def analyze_codebase(
|
||||
logger.info("Analyzing architectural patterns...")
|
||||
from skill_seekers.cli.architectural_pattern_detector import ArchitecturalPatternDetector
|
||||
|
||||
arch_detector = ArchitecturalPatternDetector(enhance_with_ai=enhance_architecture)
|
||||
arch_detector = ArchitecturalPatternDetector(enhance_with_ai=enhance_architecture, agent=agent)
|
||||
arch_report = arch_detector.analyze(directory, results["files"])
|
||||
|
||||
# Save architecture analysis if we have patterns OR frameworks (fixes #239)
|
||||
@@ -1579,6 +1504,7 @@ def analyze_codebase(
|
||||
gitignore_spec=gitignore_spec,
|
||||
enhance_with_ai=enhance_docs_ai,
|
||||
ai_mode=ai_mode,
|
||||
agent=agent,
|
||||
)
|
||||
|
||||
if docs_data and docs_data.get("total_files", 0) > 0:
|
||||
@@ -1831,8 +1757,8 @@ def _get_language_stats(files: list[dict]) -> dict[str, int]:
|
||||
|
||||
|
||||
def _format_patterns_section(output_dir: Path) -> str:
|
||||
"""Format design patterns section from patterns/detected_patterns.json."""
|
||||
patterns_file = output_dir / "patterns" / "detected_patterns.json"
|
||||
"""Format design patterns section from patterns/all_patterns.json."""
|
||||
patterns_file = output_dir / "patterns" / "all_patterns.json"
|
||||
if not patterns_file.exists():
|
||||
return ""
|
||||
|
||||
@@ -2315,8 +2241,8 @@ Examples:
|
||||
help=(
|
||||
"AI enhancement mode for how-to guides: "
|
||||
"auto (auto-detect: API if ANTHROPIC_API_KEY set, else LOCAL), "
|
||||
"api (Claude API, requires ANTHROPIC_API_KEY), "
|
||||
"local (Claude Code Max, FREE, no API key), "
|
||||
"api (Anthropic API, requires ANTHROPIC_API_KEY), "
|
||||
"local (coding agent CLI, FREE, no API key), "
|
||||
"none (disable AI enhancement). "
|
||||
"💡 TIP: Use --enhance flag instead for simpler UX!"
|
||||
),
|
||||
|
||||
@@ -313,7 +313,7 @@ def api_keys_menu():
|
||||
print("╚═══════════════════════════════════════════════════╝\n")
|
||||
|
||||
print("Current status:")
|
||||
for provider in ["anthropic", "google", "openai"]:
|
||||
for provider in ["anthropic", "google", "openai", "moonshot"]:
|
||||
key = config.get_api_key(provider)
|
||||
status = "✅ Set" if key else "❌ Not set"
|
||||
source = ""
|
||||
@@ -324,6 +324,7 @@ def api_keys_menu():
|
||||
"anthropic": "ANTHROPIC_API_KEY",
|
||||
"google": "GOOGLE_API_KEY",
|
||||
"openai": "OPENAI_API_KEY",
|
||||
"moonshot": "MOONSHOT_API_KEY",
|
||||
}[provider]
|
||||
source = " (from environment)" if os.getenv(env_var) else " (from config)"
|
||||
print(f" • {provider.capitalize()}: {status}{source}")
|
||||
@@ -332,14 +333,16 @@ def api_keys_menu():
|
||||
print(" 1. Set Anthropic (Claude) API Key")
|
||||
print(" 2. Set Google (Gemini) API Key")
|
||||
print(" 3. Set OpenAI (ChatGPT) API Key")
|
||||
print(" 4. Set Moonshot (Kimi) API Key")
|
||||
print(" 0. Back to Main Menu\n")
|
||||
|
||||
choice = input("Select an option [0-3]: ").strip()
|
||||
choice = input("Select an option [0-4]: ").strip()
|
||||
|
||||
provider_map = {
|
||||
"1": ("anthropic", "https://console.anthropic.com/settings/keys"),
|
||||
"2": ("google", "https://makersuite.google.com/app/apikey"),
|
||||
"3": ("openai", "https://platform.openai.com/api-keys"),
|
||||
"4": ("moonshot", "https://platform.moonshot.cn/"),
|
||||
}
|
||||
|
||||
if choice in provider_map:
|
||||
|
||||
@@ -14,26 +14,15 @@ Similar to GuideEnhancer (C3.3) but for configuration files.
|
||||
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
|
||||
from skill_seekers.cli.agent_client import AgentClient
|
||||
|
||||
# Configure logging
|
||||
logging.basicConfig(level=logging.INFO, format="%(levelname)s - %(message)s")
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Optional anthropic import
|
||||
ANTHROPIC_AVAILABLE = False
|
||||
try:
|
||||
import anthropic
|
||||
|
||||
ANTHROPIC_AVAILABLE = True
|
||||
except ImportError:
|
||||
pass
|
||||
|
||||
|
||||
@dataclass
|
||||
class ConfigEnhancement:
|
||||
@@ -62,51 +51,22 @@ class ConfigEnhancer:
|
||||
AI enhancement for configuration extraction results.
|
||||
|
||||
Supports dual-mode operation:
|
||||
- API mode: Uses Claude API (requires ANTHROPIC_API_KEY)
|
||||
- LOCAL mode: Uses Claude Code CLI (no API key needed)
|
||||
- API mode: Uses Anthropic API (requires ANTHROPIC_API_KEY)
|
||||
- LOCAL mode: Uses a coding agent CLI (no API key needed)
|
||||
- AUTO mode: Automatically detects best available mode
|
||||
"""
|
||||
|
||||
def __init__(self, mode: str = "auto"):
|
||||
def __init__(self, mode: str = "auto", agent: str | None = None):
|
||||
"""
|
||||
Initialize ConfigEnhancer.
|
||||
|
||||
Args:
|
||||
mode: Enhancement mode - "api", "local", or "auto" (default)
|
||||
agent: Local CLI agent name (e.g., "kimi", "claude")
|
||||
"""
|
||||
self.mode = self._detect_mode(mode)
|
||||
self.api_key = os.environ.get("ANTHROPIC_API_KEY")
|
||||
self.client = None
|
||||
|
||||
if self.mode == "api" and ANTHROPIC_AVAILABLE and self.api_key:
|
||||
# Support custom base_url for GLM-4.7 and other Claude-compatible APIs
|
||||
client_kwargs = {"api_key": self.api_key}
|
||||
base_url = os.environ.get("ANTHROPIC_BASE_URL")
|
||||
if base_url:
|
||||
client_kwargs["base_url"] = base_url
|
||||
logger.info(f"✅ Using custom API base URL: {base_url}")
|
||||
self.client = anthropic.Anthropic(**client_kwargs)
|
||||
|
||||
def _detect_mode(self, requested_mode: str) -> str:
|
||||
"""
|
||||
Detect best enhancement mode.
|
||||
|
||||
Args:
|
||||
requested_mode: User-requested mode
|
||||
|
||||
Returns:
|
||||
Actual mode to use
|
||||
"""
|
||||
if requested_mode in ["api", "local"]:
|
||||
return requested_mode
|
||||
|
||||
# Auto-detect
|
||||
if os.environ.get("ANTHROPIC_API_KEY") and ANTHROPIC_AVAILABLE:
|
||||
logger.info("🤖 AI enhancement: API mode (Claude API detected)")
|
||||
return "api"
|
||||
else:
|
||||
logger.info("🤖 AI enhancement: LOCAL mode (using Claude Code CLI)")
|
||||
return "local"
|
||||
self._agent = AgentClient(mode=mode, agent=agent)
|
||||
self.mode = self._agent.mode
|
||||
self._agent.log_mode()
|
||||
|
||||
def enhance_config_result(self, result: dict) -> dict:
|
||||
"""
|
||||
@@ -126,29 +86,29 @@ class ConfigEnhancer:
|
||||
return self._enhance_via_local(result)
|
||||
|
||||
# =========================================================================
|
||||
# API MODE - Direct Claude API calls
|
||||
# API MODE - Direct AI API calls
|
||||
# =========================================================================
|
||||
|
||||
def _enhance_via_api(self, result: dict) -> dict:
|
||||
"""Enhance configs using Claude API"""
|
||||
if not self.client:
|
||||
logger.error("❌ API mode requested but no API key available")
|
||||
"""Enhance configs using AI API"""
|
||||
if not self._agent.is_available():
|
||||
logger.error("❌ API mode requested but no API client available")
|
||||
return result
|
||||
|
||||
try:
|
||||
# Create enhancement prompt
|
||||
prompt = self._create_enhancement_prompt(result)
|
||||
|
||||
# Call Claude API
|
||||
logger.info("📡 Calling Claude API for config analysis...")
|
||||
response = self.client.messages.create(
|
||||
model="claude-sonnet-4-20250514",
|
||||
max_tokens=8000,
|
||||
messages=[{"role": "user", "content": prompt}],
|
||||
)
|
||||
# Call AI agent for config analysis
|
||||
logger.info("📡 Calling AI agent for config analysis...")
|
||||
response_text = self._agent.call(prompt, max_tokens=8000)
|
||||
|
||||
if not response_text:
|
||||
logger.error("❌ AI agent returned no response")
|
||||
return result
|
||||
|
||||
# Parse response
|
||||
enhanced_result = self._parse_api_response(response.content[0].text, result)
|
||||
enhanced_result = self._parse_api_response(response_text, result)
|
||||
logger.info("✅ API enhancement complete")
|
||||
return enhanced_result
|
||||
|
||||
@@ -157,7 +117,7 @@ class ConfigEnhancer:
|
||||
return result
|
||||
|
||||
def _create_enhancement_prompt(self, result: dict) -> str:
|
||||
"""Create prompt for Claude API"""
|
||||
"""Create prompt for AI API"""
|
||||
config_files = result.get("config_files", [])
|
||||
|
||||
# Summarize configs for prompt
|
||||
@@ -217,7 +177,7 @@ Focus on actionable insights that help developers understand and improve their c
|
||||
return prompt
|
||||
|
||||
def _parse_api_response(self, response_text: str, original_result: dict) -> dict:
|
||||
"""Parse Claude API response and merge with original result"""
|
||||
"""Parse AI API response and merge with original result"""
|
||||
try:
|
||||
# Extract JSON from response
|
||||
import re
|
||||
@@ -248,56 +208,54 @@ Focus on actionable insights that help developers understand and improve their c
|
||||
return original_result
|
||||
|
||||
# =========================================================================
|
||||
# LOCAL MODE - Claude Code CLI
|
||||
# LOCAL MODE - Coding Agent CLI
|
||||
# =========================================================================
|
||||
|
||||
def _enhance_via_local(self, result: dict) -> dict:
|
||||
"""Enhance configs using Claude Code CLI"""
|
||||
"""Enhance configs using LOCAL CLI agent"""
|
||||
try:
|
||||
# Create a temporary directory for this enhancement session
|
||||
with tempfile.TemporaryDirectory(prefix="config_enhance_") as temp_dir:
|
||||
temp_path = Path(temp_dir)
|
||||
logger.info("🖥️ Launching LOCAL agent for config analysis...")
|
||||
logger.info("⏱️ This will take 30-60 seconds...")
|
||||
|
||||
# Define output file path (absolute path that Claude will write to)
|
||||
output_file = temp_path / "config_enhancement.json"
|
||||
# Build the prompt — AgentClient handles output file management
|
||||
prompt_content = self._create_local_prompt(result)
|
||||
|
||||
# Create prompt file with the output path embedded
|
||||
prompt_file = temp_path / "enhance_prompt.md"
|
||||
prompt_content = self._create_local_prompt(result, output_file)
|
||||
prompt_file.write_text(prompt_content)
|
||||
# Call via AgentClient which handles temp dirs, file polling, etc.
|
||||
response_text = self._agent.call(prompt_content)
|
||||
|
||||
logger.info("🖥️ Launching Claude Code CLI for config analysis...")
|
||||
logger.info("⏱️ This will take 30-60 seconds...")
|
||||
if response_text:
|
||||
try:
|
||||
import re
|
||||
|
||||
# Run Claude Code CLI
|
||||
result_data = self._run_claude_cli(prompt_file, output_file, temp_path)
|
||||
json_match = re.search(r"\{.*\}", response_text, re.DOTALL)
|
||||
if json_match:
|
||||
data = json.loads(json_match.group())
|
||||
if "file_enhancements" in data or "overall_insights" in data:
|
||||
result["ai_enhancements"] = data
|
||||
logger.info("✅ LOCAL enhancement complete")
|
||||
return result
|
||||
except json.JSONDecodeError as e:
|
||||
logger.error(f"❌ Failed to parse LOCAL response as JSON: {e}")
|
||||
|
||||
if result_data:
|
||||
# Merge LOCAL enhancements
|
||||
result["ai_enhancements"] = result_data
|
||||
logger.info("✅ LOCAL enhancement complete")
|
||||
return result
|
||||
else:
|
||||
logger.warning("⚠️ LOCAL enhancement produced no results")
|
||||
return result
|
||||
logger.warning("⚠️ LOCAL enhancement produced no results")
|
||||
return result
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"❌ LOCAL enhancement failed: {e}")
|
||||
return result
|
||||
|
||||
def _create_local_prompt(self, result: dict, output_file: Path) -> str:
|
||||
"""Create prompt file for Claude Code CLI
|
||||
def _create_local_prompt(self, result: dict) -> str:
|
||||
"""Create prompt for AI agent.
|
||||
|
||||
Args:
|
||||
result: Config extraction result dict
|
||||
output_file: Absolute path where Claude should write the JSON output
|
||||
|
||||
Returns:
|
||||
Prompt content string
|
||||
"""
|
||||
config_files = result.get("config_files", [])
|
||||
|
||||
# Format config data for Claude (limit to 15 files for reasonable prompt size)
|
||||
# Format config data for AI agent (limit to 15 files for reasonable prompt size)
|
||||
config_data = []
|
||||
for cf in config_files[:15]:
|
||||
# Support both "type" (from config_extractor) and "config_type" (legacy)
|
||||
@@ -318,9 +276,6 @@ Focus on actionable insights that help developers understand and improve their c
|
||||
|
||||
prompt = f"""# Configuration Analysis Task
|
||||
|
||||
IMPORTANT: You MUST write the output to this EXACT file path:
|
||||
{output_file}
|
||||
|
||||
## Configuration Files ({len(config_files)} total, showing first 15)
|
||||
|
||||
{chr(10).join(config_data)}
|
||||
@@ -354,7 +309,7 @@ The JSON must have this EXACT structure:
|
||||
|
||||
## Instructions
|
||||
|
||||
1. Use the Write tool to create the JSON file at: {output_file}
|
||||
1. Return the JSON response directly
|
||||
2. Include an enhancement entry for each config file shown above
|
||||
3. Focus on actionable insights:
|
||||
- Explain what each config does in 1-2 sentences
|
||||
@@ -366,97 +321,6 @@ DO NOT explain your work - just write the JSON file directly.
|
||||
"""
|
||||
return prompt
|
||||
|
||||
def _run_claude_cli(
|
||||
self, prompt_file: Path, output_file: Path, working_dir: Path
|
||||
) -> dict | None:
|
||||
"""Run Claude Code CLI and wait for completion
|
||||
|
||||
Args:
|
||||
prompt_file: Path to the prompt markdown file
|
||||
output_file: Expected path where Claude will write the JSON output
|
||||
working_dir: Working directory to run Claude from
|
||||
|
||||
Returns:
|
||||
Parsed JSON dict if successful, None otherwise
|
||||
"""
|
||||
import time
|
||||
|
||||
try:
|
||||
start_time = time.time()
|
||||
|
||||
# Run claude command with --dangerously-skip-permissions to bypass all prompts
|
||||
# This allows Claude to write files without asking for confirmation
|
||||
logger.info(f" Running: claude --dangerously-skip-permissions {prompt_file.name}")
|
||||
logger.info(f" Output expected at: {output_file}")
|
||||
|
||||
result = subprocess.run(
|
||||
["claude", "--dangerously-skip-permissions", str(prompt_file)],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=300, # 5 minute timeout
|
||||
cwd=str(working_dir),
|
||||
)
|
||||
|
||||
elapsed = time.time() - start_time
|
||||
logger.info(f" Claude finished in {elapsed:.1f} seconds")
|
||||
|
||||
if result.returncode != 0:
|
||||
logger.error(f"❌ Claude CLI failed (exit code {result.returncode})")
|
||||
if result.stderr:
|
||||
logger.error(f" Error: {result.stderr[:200]}")
|
||||
return None
|
||||
|
||||
# Check if the expected output file was created
|
||||
if output_file.exists():
|
||||
try:
|
||||
with open(output_file) as f:
|
||||
data = json.load(f)
|
||||
if "file_enhancements" in data or "overall_insights" in data:
|
||||
logger.info(f"✅ Found enhancement data in {output_file.name}")
|
||||
return data
|
||||
else:
|
||||
logger.warning("⚠️ Output file exists but missing expected keys")
|
||||
except json.JSONDecodeError as e:
|
||||
logger.error(f"❌ Failed to parse output JSON: {e}")
|
||||
return None
|
||||
|
||||
# Fallback: Look for any JSON files created in the working directory
|
||||
logger.info(" Looking for JSON files in working directory...")
|
||||
current_time = time.time()
|
||||
potential_files = []
|
||||
|
||||
for json_file in working_dir.glob("*.json"):
|
||||
# Check if created recently (within last 2 minutes)
|
||||
if current_time - json_file.stat().st_mtime < 120:
|
||||
potential_files.append(json_file)
|
||||
|
||||
# Try to load the most recent JSON file with expected structure
|
||||
for json_file in sorted(potential_files, key=lambda f: f.stat().st_mtime, reverse=True):
|
||||
try:
|
||||
with open(json_file) as f:
|
||||
data = json.load(f)
|
||||
if "file_enhancements" in data or "overall_insights" in data:
|
||||
logger.info(f"✅ Found enhancement data in {json_file.name}")
|
||||
return data
|
||||
except Exception:
|
||||
continue
|
||||
|
||||
logger.warning("⚠️ Could not find enhancement output file")
|
||||
logger.info(f" Expected file: {output_file}")
|
||||
logger.info(f" Files in dir: {list(working_dir.glob('*'))}")
|
||||
return None
|
||||
|
||||
except subprocess.TimeoutExpired:
|
||||
logger.error("❌ Claude CLI timeout (5 minutes)")
|
||||
return None
|
||||
except FileNotFoundError:
|
||||
logger.error("❌ 'claude' command not found. Is Claude Code CLI installed?")
|
||||
logger.error(" Install with: npm install -g @anthropic-ai/claude-code")
|
||||
return None
|
||||
except Exception as e:
|
||||
logger.error(f"❌ Error running Claude CLI: {e}")
|
||||
return None
|
||||
|
||||
|
||||
def main():
|
||||
"""Command-line interface for config enhancement"""
|
||||
|
||||
@@ -271,13 +271,13 @@ class ConfigFileDetector:
|
||||
".tmp",
|
||||
}
|
||||
|
||||
def find_config_files(self, directory: Path, max_files: int = 100) -> list[ConfigFile]:
|
||||
def find_config_files(self, directory: Path, max_files: int = 0) -> list[ConfigFile]:
|
||||
"""
|
||||
Find all configuration files in directory.
|
||||
|
||||
Args:
|
||||
directory: Root directory to search
|
||||
max_files: Maximum number of config files to find
|
||||
max_files: Maximum number of config files to find (0 = unlimited)
|
||||
|
||||
Returns:
|
||||
List of ConfigFile objects
|
||||
@@ -286,7 +286,7 @@ class ConfigFileDetector:
|
||||
found_count = 0
|
||||
|
||||
for file_path in self._walk_directory(directory):
|
||||
if found_count >= max_files:
|
||||
if max_files > 0 and found_count >= max_files:
|
||||
logger.info(f"Reached max_files limit ({max_files})")
|
||||
break
|
||||
|
||||
@@ -760,15 +760,13 @@ class ConfigExtractor:
|
||||
self.parser = ConfigParser()
|
||||
self.pattern_detector = ConfigPatternDetector()
|
||||
|
||||
def extract_from_directory(
|
||||
self, directory: Path, max_files: int = 100
|
||||
) -> ConfigExtractionResult:
|
||||
def extract_from_directory(self, directory: Path, max_files: int = 0) -> ConfigExtractionResult:
|
||||
"""
|
||||
Extract configuration patterns from directory.
|
||||
|
||||
Args:
|
||||
directory: Root directory to analyze
|
||||
max_files: Maximum config files to process
|
||||
max_files: Maximum config files to process (0 = unlimited)
|
||||
|
||||
Returns:
|
||||
ConfigExtractionResult with all findings
|
||||
@@ -857,7 +855,7 @@ def main():
|
||||
parser.add_argument("directory", type=Path, help="Directory to analyze")
|
||||
parser.add_argument("--output", "-o", type=Path, help="Output JSON file")
|
||||
parser.add_argument(
|
||||
"--max-files", type=int, default=100, help="Maximum config files to process"
|
||||
"--max-files", type=int, default=0, help="Maximum config files to process (0 = unlimited)"
|
||||
)
|
||||
parser.add_argument(
|
||||
"--enhance",
|
||||
@@ -867,13 +865,13 @@ def main():
|
||||
parser.add_argument(
|
||||
"--enhance-local",
|
||||
action="store_true",
|
||||
help="Enhance with AI analysis (LOCAL mode, uses Claude Code CLI)",
|
||||
help="Enhance with AI analysis (LOCAL mode, uses coding agent CLI)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--ai-mode",
|
||||
choices=["auto", "api", "local", "none"],
|
||||
default="none",
|
||||
help="AI enhancement mode: auto (detect), api (Claude API), local (Claude Code CLI), none (disable)",
|
||||
help="AI enhancement mode: auto (detect), api (Anthropic API), local (coding agent CLI), none (disable)",
|
||||
)
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
@@ -47,12 +47,12 @@ class ConfigManager:
|
||||
"show_countdown": True,
|
||||
},
|
||||
"resume": {"auto_save_interval_seconds": 60, "keep_progress_days": 7},
|
||||
"api_keys": {"anthropic": None, "google": None, "openai": None},
|
||||
"api_keys": {"anthropic": None, "google": None, "openai": None, "moonshot": None},
|
||||
"ai_enhancement": {
|
||||
"default_enhance_level": 1, # Default AI enhancement level (0-3)
|
||||
"default_agent": None, # "claude", "gemini", "openai", or None (auto-detect)
|
||||
"local_batch_size": 20, # Patterns per Claude CLI call (default was 5)
|
||||
"local_parallel_workers": 3, # Concurrent Claude CLI calls
|
||||
"default_agent": None, # "claude", "gemini", "openai", "kimi", or None (auto-detect)
|
||||
"local_batch_size": 20, # Patterns per CLI agent call (default was 5)
|
||||
"local_parallel_workers": 3, # Concurrent CLI agent calls
|
||||
},
|
||||
"first_run": {"completed": False, "version": "2.7.0"},
|
||||
}
|
||||
|
||||
@@ -61,7 +61,11 @@ class ConfigValidator:
|
||||
}
|
||||
|
||||
# Valid merge modes
|
||||
VALID_MERGE_MODES = {"rule-based", "claude-enhanced"}
|
||||
VALID_MERGE_MODES = {
|
||||
"rule-based",
|
||||
"ai-enhanced",
|
||||
"claude-enhanced",
|
||||
} # claude-enhanced kept as alias
|
||||
|
||||
# Valid code analysis depth levels
|
||||
VALID_DEPTH_LEVELS = {"surface", "deep", "full"}
|
||||
@@ -161,6 +165,21 @@ class ConfigValidator:
|
||||
f"Invalid merge_mode: '{merge_mode}'. Must be one of {self.VALID_MERGE_MODES}"
|
||||
)
|
||||
|
||||
# Validate marketplace_targets (optional)
|
||||
marketplace_targets = self.config.get("marketplace_targets")
|
||||
if marketplace_targets is not None:
|
||||
if not isinstance(marketplace_targets, list):
|
||||
raise ValueError("'marketplace_targets' must be an array")
|
||||
for i, mt in enumerate(marketplace_targets):
|
||||
if not isinstance(mt, dict):
|
||||
raise ValueError(f"marketplace_targets[{i}]: must be an object")
|
||||
if "marketplace" not in mt:
|
||||
raise ValueError(
|
||||
f"marketplace_targets[{i}]: missing required field 'marketplace'"
|
||||
)
|
||||
if not isinstance(mt["marketplace"], str):
|
||||
raise ValueError(f"marketplace_targets[{i}]: 'marketplace' must be a string")
|
||||
|
||||
# Validate each source
|
||||
for i, source in enumerate(sources):
|
||||
self._validate_source(source, i)
|
||||
|
||||
@@ -1962,7 +1962,7 @@ def main() -> int:
|
||||
"0=disabled (default for Confluence), 1=SKILL.md only, "
|
||||
"2=+architecture/config, 3=full enhancement. "
|
||||
"Mode selection: uses API if ANTHROPIC_API_KEY is set, "
|
||||
"otherwise LOCAL (Claude Code)"
|
||||
"otherwise LOCAL (Claude Code, Kimi, etc.)"
|
||||
)
|
||||
|
||||
# Confluence-specific arguments
|
||||
@@ -2137,7 +2137,9 @@ def main() -> int:
|
||||
LocalSkillEnhancer,
|
||||
)
|
||||
|
||||
enhancer = LocalSkillEnhancer(Path(skill_dir))
|
||||
agent = getattr(args, "agent", None) if args else None
|
||||
agent_cmd = getattr(args, "agent_cmd", None) if args else None
|
||||
enhancer = LocalSkillEnhancer(Path(skill_dir), agent=agent, agent_cmd=agent_cmd)
|
||||
enhancer.run(headless=True)
|
||||
print(" Local enhancement complete!")
|
||||
else:
|
||||
@@ -2145,7 +2147,9 @@ def main() -> int:
|
||||
LocalSkillEnhancer,
|
||||
)
|
||||
|
||||
enhancer = LocalSkillEnhancer(Path(skill_dir))
|
||||
agent = getattr(args, "agent", None) if args else None
|
||||
agent_cmd = getattr(args, "agent_cmd", None) if args else None
|
||||
enhancer = LocalSkillEnhancer(Path(skill_dir), agent=agent, agent_cmd=agent_cmd)
|
||||
enhancer.run(headless=True)
|
||||
print(" Local enhancement complete!")
|
||||
|
||||
|
||||
@@ -28,7 +28,7 @@ CONTENT_MATCH_POINTS = 1 # points for content keyword match
|
||||
API_CONTENT_LIMIT = 100000 # max characters for API enhancement
|
||||
API_PREVIEW_LIMIT = 40000 # max characters for preview
|
||||
|
||||
# Local enhancement limits (uses Claude Code Max)
|
||||
# Local enhancement limits (uses coding agent CLI)
|
||||
LOCAL_CONTENT_LIMIT = 50000 # max characters for local enhancement
|
||||
LOCAL_PREVIEW_LIMIT = 20000 # max characters for preview
|
||||
|
||||
|
||||
@@ -104,12 +104,20 @@ class CreateCommand:
|
||||
if arg_value is None:
|
||||
return False
|
||||
|
||||
# Check against common defaults
|
||||
# Check against common defaults — args with these values were NOT
|
||||
# explicitly set by the user and should not be forwarded.
|
||||
defaults = {
|
||||
"max_issues": 100,
|
||||
"chunk_tokens": DEFAULT_CHUNK_TOKENS,
|
||||
"chunk_overlap_tokens": DEFAULT_CHUNK_OVERLAP_TOKENS,
|
||||
"output": None,
|
||||
"doc_version": "",
|
||||
"video_languages": "en",
|
||||
"whisper_model": "base",
|
||||
"platform": "slack",
|
||||
"visual_interval": 0.7,
|
||||
"visual_min_gap": 0.5,
|
||||
"visual_similarity": 3.0,
|
||||
}
|
||||
|
||||
if arg_name in defaults:
|
||||
@@ -164,436 +172,227 @@ class CreateCommand:
|
||||
logger.error(f"Unknown source type: {self.source_info.type}")
|
||||
return 1
|
||||
|
||||
# ── Dynamic argument forwarding ──────────────────────────────────────
|
||||
#
|
||||
# Instead of manually checking each flag in every _route_*() method,
|
||||
# _build_argv() dynamically iterates vars(self.args) and forwards all
|
||||
# explicitly-set arguments. This is the same pattern used by
|
||||
# main.py::_reconstruct_argv() and eliminates ~40 missing-flag gaps.
|
||||
|
||||
# Dest names that differ from their CLI flag (dest → flag)
|
||||
_DEST_TO_FLAG = {
|
||||
"async_mode": "--async",
|
||||
"video_url": "--url",
|
||||
"video_playlist": "--playlist",
|
||||
"video_languages": "--languages",
|
||||
"skip_config": "--skip-config-patterns",
|
||||
}
|
||||
|
||||
# Internal args that should never be forwarded to sub-scrapers.
|
||||
# video_url/video_playlist/video_file are handled as positionals by _route_video().
|
||||
# config is forwarded manually only by routes that need it (web, github).
|
||||
_SKIP_ARGS = frozenset(
|
||||
{
|
||||
"source",
|
||||
"func",
|
||||
"subcommand",
|
||||
"command",
|
||||
"config",
|
||||
"video_url",
|
||||
"video_playlist",
|
||||
"video_file",
|
||||
}
|
||||
)
|
||||
|
||||
def _build_argv(
|
||||
self,
|
||||
module_name: str,
|
||||
positional_args: list[str],
|
||||
allowlist: frozenset[str] | None = None,
|
||||
) -> list[str]:
|
||||
"""Build argv dynamically by forwarding all explicitly-set arguments.
|
||||
|
||||
Uses the same pattern as main.py::_reconstruct_argv().
|
||||
Replaces manual per-flag checking in _route_*() and _add_common_args().
|
||||
|
||||
Args:
|
||||
module_name: Scraper module name (e.g., "doc_scraper")
|
||||
positional_args: Positional arguments to prepend (e.g., [url] or ["--repo", repo])
|
||||
allowlist: If provided, ONLY forward args in this set (overrides _SKIP_ARGS).
|
||||
Used for targets with strict arg sets like unified_scraper.
|
||||
|
||||
Returns:
|
||||
Complete argv list for the scraper
|
||||
"""
|
||||
argv = [module_name] + positional_args
|
||||
|
||||
# Auto-add suggested name if user didn't provide one (skip for allowlisted targets)
|
||||
if not allowlist and not self.args.name and self.source_info:
|
||||
argv.extend(["--name", self.source_info.suggested_name])
|
||||
|
||||
for key, value in vars(self.args).items():
|
||||
# If allowlist provided, only forward args in the allowlist
|
||||
if allowlist is not None:
|
||||
if key not in allowlist:
|
||||
continue
|
||||
elif key in self._SKIP_ARGS or key.startswith("_help_"):
|
||||
continue
|
||||
if not self._is_explicitly_set(key, value):
|
||||
continue
|
||||
|
||||
# Use translation map for mismatched dest→flag names, else derive from key
|
||||
if key in self._DEST_TO_FLAG:
|
||||
arg_flag = self._DEST_TO_FLAG[key]
|
||||
else:
|
||||
arg_flag = f"--{key.replace('_', '-')}"
|
||||
|
||||
if isinstance(value, bool):
|
||||
if value:
|
||||
argv.append(arg_flag)
|
||||
elif isinstance(value, list):
|
||||
for item in value:
|
||||
argv.extend([arg_flag, str(item)])
|
||||
elif value is not None:
|
||||
argv.extend([arg_flag, str(value)])
|
||||
|
||||
return argv
|
||||
|
||||
def _call_module(self, module, argv: list[str]) -> int:
|
||||
"""Call a scraper module with the given argv.
|
||||
|
||||
Swaps sys.argv, calls module.main(), restores sys.argv.
|
||||
"""
|
||||
logger.debug(f"Calling {argv[0]} with argv: {argv}")
|
||||
original_argv = sys.argv
|
||||
try:
|
||||
sys.argv = argv
|
||||
result = module.main()
|
||||
if result is None:
|
||||
logger.warning(f"Module returned None exit code, treating as success")
|
||||
return 0
|
||||
return result
|
||||
finally:
|
||||
sys.argv = original_argv
|
||||
|
||||
def _route_web(self) -> int:
|
||||
"""Route to web documentation scraper (doc_scraper.py)."""
|
||||
from skill_seekers.cli import doc_scraper
|
||||
|
||||
# Reconstruct argv for doc_scraper
|
||||
argv = ["doc_scraper"]
|
||||
url = self.source_info.parsed.get("url", self.source_info.raw_source)
|
||||
argv = self._build_argv("doc_scraper", [url])
|
||||
|
||||
# Add URL
|
||||
url = self.source_info.parsed["url"]
|
||||
argv.append(url)
|
||||
|
||||
# Add universal arguments
|
||||
self._add_common_args(argv)
|
||||
|
||||
# Config file (web-specific — loads selectors, categories, etc.)
|
||||
# Forward config if set (not in _build_argv since it's in SKIP_ARGS
|
||||
# to avoid double-forwarding for config-type sources)
|
||||
if self.args.config:
|
||||
argv.extend(["--config", self.args.config])
|
||||
|
||||
# RAG arguments (web scraper only)
|
||||
if getattr(self.args, "chunk_for_rag", False):
|
||||
argv.append("--chunk-for-rag")
|
||||
if (
|
||||
getattr(self.args, "chunk_tokens", None)
|
||||
and self.args.chunk_tokens != DEFAULT_CHUNK_TOKENS
|
||||
):
|
||||
argv.extend(["--chunk-tokens", str(self.args.chunk_tokens)])
|
||||
if (
|
||||
getattr(self.args, "chunk_overlap_tokens", None)
|
||||
and self.args.chunk_overlap_tokens != DEFAULT_CHUNK_OVERLAP_TOKENS
|
||||
):
|
||||
argv.extend(["--chunk-overlap-tokens", str(self.args.chunk_overlap_tokens)])
|
||||
|
||||
# Advanced web-specific arguments
|
||||
if getattr(self.args, "no_preserve_code_blocks", False):
|
||||
argv.append("--no-preserve-code-blocks")
|
||||
if getattr(self.args, "no_preserve_paragraphs", False):
|
||||
argv.append("--no-preserve-paragraphs")
|
||||
if getattr(self.args, "interactive_enhancement", False):
|
||||
argv.append("--interactive-enhancement")
|
||||
|
||||
# Web-specific arguments
|
||||
if getattr(self.args, "max_pages", None):
|
||||
argv.extend(["--max-pages", str(self.args.max_pages)])
|
||||
if getattr(self.args, "skip_scrape", False):
|
||||
argv.append("--skip-scrape")
|
||||
if getattr(self.args, "resume", False):
|
||||
argv.append("--resume")
|
||||
if getattr(self.args, "fresh", False):
|
||||
argv.append("--fresh")
|
||||
if getattr(self.args, "rate_limit", None):
|
||||
argv.extend(["--rate-limit", str(self.args.rate_limit)])
|
||||
if getattr(self.args, "workers", None):
|
||||
argv.extend(["--workers", str(self.args.workers)])
|
||||
if getattr(self.args, "async_mode", False):
|
||||
argv.append("--async")
|
||||
if getattr(self.args, "no_rate_limit", False):
|
||||
argv.append("--no-rate-limit")
|
||||
if getattr(self.args, "browser", False):
|
||||
argv.append("--browser")
|
||||
|
||||
# Call doc_scraper with modified argv
|
||||
logger.debug(f"Calling doc_scraper with argv: {argv}")
|
||||
original_argv = sys.argv
|
||||
try:
|
||||
sys.argv = argv
|
||||
return doc_scraper.main()
|
||||
finally:
|
||||
sys.argv = original_argv
|
||||
return self._call_module(doc_scraper, argv)
|
||||
|
||||
def _route_github(self) -> int:
|
||||
"""Route to GitHub repository scraper (github_scraper.py)."""
|
||||
from skill_seekers.cli import github_scraper
|
||||
|
||||
# Reconstruct argv for github_scraper
|
||||
argv = ["github_scraper"]
|
||||
repo = self.source_info.parsed.get("repo", self.source_info.raw_source)
|
||||
argv = self._build_argv("github_scraper", ["--repo", repo])
|
||||
|
||||
# Add repo
|
||||
repo = self.source_info.parsed["repo"]
|
||||
argv.extend(["--repo", repo])
|
||||
|
||||
# Add universal arguments
|
||||
self._add_common_args(argv)
|
||||
|
||||
# Config file (github-specific)
|
||||
if self.args.config:
|
||||
argv.extend(["--config", self.args.config])
|
||||
|
||||
# Add GitHub-specific arguments
|
||||
if getattr(self.args, "token", None):
|
||||
argv.extend(["--token", self.args.token])
|
||||
if getattr(self.args, "profile", None):
|
||||
argv.extend(["--profile", self.args.profile])
|
||||
if getattr(self.args, "non_interactive", False):
|
||||
argv.append("--non-interactive")
|
||||
if getattr(self.args, "no_issues", False):
|
||||
argv.append("--no-issues")
|
||||
if getattr(self.args, "no_changelog", False):
|
||||
argv.append("--no-changelog")
|
||||
if getattr(self.args, "no_releases", False):
|
||||
argv.append("--no-releases")
|
||||
if getattr(self.args, "max_issues", None) and self.args.max_issues != 100:
|
||||
argv.extend(["--max-issues", str(self.args.max_issues)])
|
||||
if getattr(self.args, "scrape_only", False):
|
||||
argv.append("--scrape-only")
|
||||
if getattr(self.args, "local_repo_path", None):
|
||||
argv.extend(["--local-repo-path", self.args.local_repo_path])
|
||||
|
||||
# Call github_scraper with modified argv
|
||||
logger.debug(f"Calling github_scraper with argv: {argv}")
|
||||
original_argv = sys.argv
|
||||
try:
|
||||
sys.argv = argv
|
||||
return github_scraper.main()
|
||||
finally:
|
||||
sys.argv = original_argv
|
||||
return self._call_module(github_scraper, argv)
|
||||
|
||||
def _route_local(self) -> int:
|
||||
"""Route to local codebase analyzer (codebase_scraper.py)."""
|
||||
from skill_seekers.cli import codebase_scraper
|
||||
|
||||
# Reconstruct argv for codebase_scraper
|
||||
argv = ["codebase_scraper"]
|
||||
|
||||
# Add directory
|
||||
directory = self.source_info.parsed["directory"]
|
||||
argv.extend(["--directory", directory])
|
||||
|
||||
# Add universal arguments
|
||||
self._add_common_args(argv)
|
||||
|
||||
# Preset (local codebase scraper has preset support)
|
||||
if getattr(self.args, "preset", None):
|
||||
argv.extend(["--preset", self.args.preset])
|
||||
|
||||
# Add local-specific arguments
|
||||
if getattr(self.args, "languages", None):
|
||||
argv.extend(["--languages", self.args.languages])
|
||||
if getattr(self.args, "file_patterns", None):
|
||||
argv.extend(["--file-patterns", self.args.file_patterns])
|
||||
if getattr(self.args, "skip_patterns", False):
|
||||
argv.append("--skip-patterns")
|
||||
if getattr(self.args, "skip_test_examples", False):
|
||||
argv.append("--skip-test-examples")
|
||||
if getattr(self.args, "skip_how_to_guides", False):
|
||||
argv.append("--skip-how-to-guides")
|
||||
if getattr(self.args, "skip_config", False):
|
||||
argv.append("--skip-config")
|
||||
if getattr(self.args, "skip_docs", False):
|
||||
argv.append("--skip-docs")
|
||||
|
||||
# Call codebase_scraper with modified argv
|
||||
logger.debug(f"Calling codebase_scraper with argv: {argv}")
|
||||
original_argv = sys.argv
|
||||
try:
|
||||
sys.argv = argv
|
||||
return codebase_scraper.main()
|
||||
finally:
|
||||
sys.argv = original_argv
|
||||
directory = self.source_info.parsed.get("directory", self.source_info.raw_source)
|
||||
argv = self._build_argv("codebase_scraper", ["--directory", directory])
|
||||
return self._call_module(codebase_scraper, argv)
|
||||
|
||||
def _route_pdf(self) -> int:
|
||||
"""Route to PDF scraper (pdf_scraper.py)."""
|
||||
from skill_seekers.cli import pdf_scraper
|
||||
|
||||
# Reconstruct argv for pdf_scraper
|
||||
argv = ["pdf_scraper"]
|
||||
|
||||
# Add PDF file
|
||||
file_path = self.source_info.parsed["file_path"]
|
||||
argv.extend(["--pdf", file_path])
|
||||
|
||||
# Add universal arguments
|
||||
self._add_common_args(argv)
|
||||
|
||||
# Add PDF-specific arguments
|
||||
if getattr(self.args, "ocr", False):
|
||||
argv.append("--ocr")
|
||||
if getattr(self.args, "pages", None):
|
||||
argv.extend(["--pages", self.args.pages])
|
||||
|
||||
# Call pdf_scraper with modified argv
|
||||
logger.debug(f"Calling pdf_scraper with argv: {argv}")
|
||||
original_argv = sys.argv
|
||||
try:
|
||||
sys.argv = argv
|
||||
return pdf_scraper.main()
|
||||
finally:
|
||||
sys.argv = original_argv
|
||||
file_path = self.source_info.parsed.get("file_path", self.source_info.raw_source)
|
||||
argv = self._build_argv("pdf_scraper", ["--pdf", file_path])
|
||||
return self._call_module(pdf_scraper, argv)
|
||||
|
||||
def _route_word(self) -> int:
|
||||
"""Route to Word document scraper (word_scraper.py)."""
|
||||
from skill_seekers.cli import word_scraper
|
||||
|
||||
# Reconstruct argv for word_scraper
|
||||
argv = ["word_scraper"]
|
||||
|
||||
# Add DOCX file
|
||||
file_path = self.source_info.parsed["file_path"]
|
||||
argv.extend(["--docx", file_path])
|
||||
|
||||
# Add universal arguments
|
||||
self._add_common_args(argv)
|
||||
|
||||
# Call word_scraper with modified argv
|
||||
logger.debug(f"Calling word_scraper with argv: {argv}")
|
||||
original_argv = sys.argv
|
||||
try:
|
||||
sys.argv = argv
|
||||
return word_scraper.main()
|
||||
finally:
|
||||
sys.argv = original_argv
|
||||
file_path = self.source_info.parsed.get("file_path", self.source_info.raw_source)
|
||||
argv = self._build_argv("word_scraper", ["--docx", file_path])
|
||||
return self._call_module(word_scraper, argv)
|
||||
|
||||
def _route_epub(self) -> int:
|
||||
"""Route to EPUB scraper (epub_scraper.py)."""
|
||||
from skill_seekers.cli import epub_scraper
|
||||
|
||||
# Reconstruct argv for epub_scraper
|
||||
argv = ["epub_scraper"]
|
||||
|
||||
# Add EPUB file
|
||||
file_path = self.source_info.parsed["file_path"]
|
||||
argv.extend(["--epub", file_path])
|
||||
|
||||
# Add universal arguments
|
||||
self._add_common_args(argv)
|
||||
|
||||
# Call epub_scraper with modified argv
|
||||
logger.debug(f"Calling epub_scraper with argv: {argv}")
|
||||
original_argv = sys.argv
|
||||
try:
|
||||
sys.argv = argv
|
||||
return epub_scraper.main()
|
||||
finally:
|
||||
sys.argv = original_argv
|
||||
file_path = self.source_info.parsed.get("file_path", self.source_info.raw_source)
|
||||
argv = self._build_argv("epub_scraper", ["--epub", file_path])
|
||||
return self._call_module(epub_scraper, argv)
|
||||
|
||||
def _route_video(self) -> int:
|
||||
"""Route to video scraper (video_scraper.py)."""
|
||||
from skill_seekers.cli import video_scraper
|
||||
|
||||
# Reconstruct argv for video_scraper
|
||||
argv = ["video_scraper"]
|
||||
|
||||
# Add video source (URL or file)
|
||||
parsed = self.source_info.parsed
|
||||
video_playlist = getattr(self.args, "video_playlist", None)
|
||||
if parsed.get("source_kind") == "file":
|
||||
argv.extend(["--video-file", parsed["file_path"]])
|
||||
elif video_playlist:
|
||||
# Explicit --video-playlist flag takes precedence
|
||||
argv.extend(["--playlist", video_playlist])
|
||||
positional = ["--video-file", parsed["file_path"]]
|
||||
elif parsed.get("url"):
|
||||
url = parsed["url"]
|
||||
# Detect playlist vs single video
|
||||
if "playlist" in url.lower():
|
||||
argv.extend(["--playlist", url])
|
||||
else:
|
||||
argv.extend(["--url", url])
|
||||
flag = "--playlist" if "playlist" in url.lower() else "--url"
|
||||
positional = [flag, url]
|
||||
else:
|
||||
positional = []
|
||||
|
||||
# Add universal arguments
|
||||
self._add_common_args(argv)
|
||||
argv = self._build_argv("video_scraper", positional)
|
||||
return self._call_module(video_scraper, argv)
|
||||
|
||||
# Add video-specific arguments
|
||||
video_langs = getattr(self.args, "video_languages", None) or getattr(
|
||||
self.args, "languages", None
|
||||
)
|
||||
if video_langs:
|
||||
argv.extend(["--languages", video_langs])
|
||||
if getattr(self.args, "visual", False):
|
||||
argv.append("--visual")
|
||||
if getattr(self.args, "vision_ocr", False):
|
||||
argv.append("--vision-ocr")
|
||||
if getattr(self.args, "whisper_model", None) and self.args.whisper_model != "base":
|
||||
argv.extend(["--whisper-model", self.args.whisper_model])
|
||||
vi = getattr(self.args, "visual_interval", None)
|
||||
if vi is not None and vi != 0.7:
|
||||
argv.extend(["--visual-interval", str(vi)])
|
||||
vmg = getattr(self.args, "visual_min_gap", None)
|
||||
if vmg is not None and vmg != 0.5:
|
||||
argv.extend(["--visual-min-gap", str(vmg)])
|
||||
vs = getattr(self.args, "visual_similarity", None)
|
||||
if vs is not None and vs != 3.0:
|
||||
argv.extend(["--visual-similarity", str(vs)])
|
||||
st = getattr(self.args, "start_time", None)
|
||||
if st is not None:
|
||||
argv.extend(["--start-time", str(st)])
|
||||
et = getattr(self.args, "end_time", None)
|
||||
if et is not None:
|
||||
argv.extend(["--end-time", str(et)])
|
||||
|
||||
# Call video_scraper with modified argv
|
||||
logger.debug(f"Calling video_scraper with argv: {argv}")
|
||||
original_argv = sys.argv
|
||||
try:
|
||||
sys.argv = argv
|
||||
return video_scraper.main()
|
||||
finally:
|
||||
sys.argv = original_argv
|
||||
# Args accepted by unified_scraper (allowlist for config route)
|
||||
_UNIFIED_SCRAPER_ARGS = frozenset(
|
||||
{
|
||||
"merge_mode",
|
||||
"skip_codebase_analysis",
|
||||
"fresh",
|
||||
"dry_run",
|
||||
"enhance_workflow",
|
||||
"enhance_stage",
|
||||
"var",
|
||||
"workflow_dry_run",
|
||||
"api_key",
|
||||
"enhance_level",
|
||||
"agent",
|
||||
"agent_cmd",
|
||||
}
|
||||
)
|
||||
|
||||
def _route_config(self) -> int:
|
||||
"""Route to unified scraper for config files (unified_scraper.py)."""
|
||||
from skill_seekers.cli import unified_scraper
|
||||
|
||||
# Reconstruct argv for unified_scraper
|
||||
argv = ["unified_scraper"]
|
||||
|
||||
# Add config file
|
||||
config_path = self.source_info.parsed["config_path"]
|
||||
argv.extend(["--config", config_path])
|
||||
|
||||
# Behavioral flags supported by unified_scraper
|
||||
# Note: name/output/enhance-level come from the JSON config file, not CLI
|
||||
if self.args.dry_run:
|
||||
argv.append("--dry-run")
|
||||
if getattr(self.args, "fresh", False):
|
||||
argv.append("--fresh")
|
||||
|
||||
# Config-specific flags (--merge-mode, --skip-codebase-analysis)
|
||||
if getattr(self.args, "merge_mode", None):
|
||||
argv.extend(["--merge-mode", self.args.merge_mode])
|
||||
if getattr(self.args, "skip_codebase_analysis", False):
|
||||
argv.append("--skip-codebase-analysis")
|
||||
|
||||
# Enhancement workflow flags (unified_scraper now supports these)
|
||||
if getattr(self.args, "enhance_workflow", None):
|
||||
for wf in self.args.enhance_workflow:
|
||||
argv.extend(["--enhance-workflow", wf])
|
||||
if getattr(self.args, "enhance_stage", None):
|
||||
for stage in self.args.enhance_stage:
|
||||
argv.extend(["--enhance-stage", stage])
|
||||
if getattr(self.args, "var", None):
|
||||
for var in self.args.var:
|
||||
argv.extend(["--var", var])
|
||||
if getattr(self.args, "workflow_dry_run", False):
|
||||
argv.append("--workflow-dry-run")
|
||||
|
||||
# Call unified_scraper with modified argv
|
||||
logger.debug(f"Calling unified_scraper with argv: {argv}")
|
||||
original_argv = sys.argv
|
||||
try:
|
||||
sys.argv = argv
|
||||
return unified_scraper.main()
|
||||
finally:
|
||||
sys.argv = original_argv
|
||||
argv = self._build_argv(
|
||||
"unified_scraper",
|
||||
["--config", config_path],
|
||||
allowlist=self._UNIFIED_SCRAPER_ARGS,
|
||||
)
|
||||
return self._call_module(unified_scraper, argv)
|
||||
|
||||
def _route_generic(self, module_name: str, file_flag: str) -> int:
|
||||
"""Generic routing for new source types.
|
||||
|
||||
Most new source types (jupyter, html, openapi, asciidoc, pptx, rss,
|
||||
manpage, confluence, notion, chat) follow the same pattern:
|
||||
import module, build argv with --flag <file_path>, add common args, call main().
|
||||
|
||||
Args:
|
||||
module_name: Python module name under skill_seekers.cli (e.g., "jupyter_scraper")
|
||||
file_flag: CLI flag for the source file (e.g., "--notebook")
|
||||
|
||||
Returns:
|
||||
Exit code from scraper
|
||||
All new source types (jupyter, html, openapi, asciidoc, pptx, rss,
|
||||
manpage, confluence, notion, chat) use dynamic argument forwarding.
|
||||
"""
|
||||
import importlib
|
||||
|
||||
module = importlib.import_module(f"skill_seekers.cli.{module_name}")
|
||||
|
||||
argv = [module_name]
|
||||
|
||||
file_path = self.source_info.parsed.get("file_path", "")
|
||||
if file_path:
|
||||
argv.extend([file_flag, file_path])
|
||||
|
||||
self._add_common_args(argv)
|
||||
|
||||
logger.debug(f"Calling {module_name} with argv: {argv}")
|
||||
original_argv = sys.argv
|
||||
try:
|
||||
sys.argv = argv
|
||||
return module.main()
|
||||
finally:
|
||||
sys.argv = original_argv
|
||||
|
||||
def _add_common_args(self, argv: list[str]) -> None:
|
||||
"""Add truly universal arguments to argv list.
|
||||
|
||||
These flags are accepted by ALL scrapers (doc, github, codebase, pdf)
|
||||
because each scraper calls ``add_all_standard_arguments(parser)``
|
||||
which registers: name, description, output, enhance-level, api-key,
|
||||
dry-run, verbose, quiet, and workflow args.
|
||||
|
||||
Route-specific flags (preset, config, RAG, preserve, etc.) are
|
||||
forwarded only by the _route_*() method that needs them.
|
||||
"""
|
||||
# Identity arguments
|
||||
if self.args.name:
|
||||
argv.extend(["--name", self.args.name])
|
||||
elif hasattr(self, "source_info") and self.source_info:
|
||||
# Use suggested name from source detection
|
||||
argv.extend(["--name", self.source_info.suggested_name])
|
||||
|
||||
if self.args.description:
|
||||
argv.extend(["--description", self.args.description])
|
||||
if self.args.output:
|
||||
argv.extend(["--output", self.args.output])
|
||||
|
||||
# Enhancement arguments (consolidated to --enhance-level only)
|
||||
if self.args.enhance_level > 0:
|
||||
argv.extend(["--enhance-level", str(self.args.enhance_level)])
|
||||
if self.args.api_key:
|
||||
argv.extend(["--api-key", self.args.api_key])
|
||||
|
||||
# Behavior arguments
|
||||
if self.args.dry_run:
|
||||
argv.append("--dry-run")
|
||||
if self.args.verbose:
|
||||
argv.append("--verbose")
|
||||
if self.args.quiet:
|
||||
argv.append("--quiet")
|
||||
|
||||
# Documentation version metadata
|
||||
if getattr(self.args, "doc_version", ""):
|
||||
argv.extend(["--doc-version", self.args.doc_version])
|
||||
|
||||
# Enhancement Workflow arguments
|
||||
if getattr(self.args, "enhance_workflow", None):
|
||||
for wf in self.args.enhance_workflow:
|
||||
argv.extend(["--enhance-workflow", wf])
|
||||
if getattr(self.args, "enhance_stage", None):
|
||||
for stage in self.args.enhance_stage:
|
||||
argv.extend(["--enhance-stage", stage])
|
||||
if getattr(self.args, "var", None):
|
||||
for var in self.args.var:
|
||||
argv.extend(["--var", var])
|
||||
if getattr(self.args, "workflow_dry_run", False):
|
||||
argv.append("--workflow-dry-run")
|
||||
positional = [file_flag, file_path] if file_path else []
|
||||
argv = self._build_argv(module_name, positional)
|
||||
return self._call_module(module, argv)
|
||||
|
||||
|
||||
def main() -> int:
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Documentation to Claude Skill Converter
|
||||
Single tool to scrape any documentation and create high-quality Claude skills.
|
||||
Documentation to AI Skill Converter
|
||||
Single tool to scrape any documentation and create high-quality AI skills.
|
||||
|
||||
Usage:
|
||||
skill-seekers scrape --interactive
|
||||
@@ -187,6 +187,8 @@ class DocToSkillConverter:
|
||||
# Browser rendering mode (for JavaScript SPA sites)
|
||||
self.browser_mode = config.get("browser", False)
|
||||
self._browser_renderer = None
|
||||
self._browser_wait_until = config.get("browser_wait_until", "domcontentloaded")
|
||||
self._browser_extra_wait = config.get("browser_extra_wait", 0) # ms
|
||||
|
||||
# Parallel scraping config
|
||||
self.workers = config.get("workers", 1)
|
||||
@@ -250,7 +252,10 @@ class DocToSkillConverter:
|
||||
Returns:
|
||||
bool: True if URL matches include patterns and doesn't match exclude patterns
|
||||
"""
|
||||
if not url.startswith(self.base_url):
|
||||
# Use directory part of base_url for prefix check so sibling pages match.
|
||||
# e.g., base_url "https://example.com/docs/index.html" → prefix "https://example.com/docs/"
|
||||
base_dir = self.base_url if self.base_url.endswith("/") else self.base_url + "/"
|
||||
if not url.startswith(base_dir):
|
||||
return False
|
||||
|
||||
if self._include_patterns and not any(pattern in url for pattern in self._include_patterns):
|
||||
@@ -730,8 +735,14 @@ class DocToSkillConverter:
|
||||
if self._browser_renderer is None:
|
||||
from skill_seekers.cli.browser_renderer import BrowserRenderer
|
||||
|
||||
self._browser_renderer = BrowserRenderer()
|
||||
logger.info("Launched headless browser for JavaScript rendering")
|
||||
self._browser_renderer = BrowserRenderer(
|
||||
wait_until=self._browser_wait_until,
|
||||
extra_wait=self._browser_extra_wait,
|
||||
)
|
||||
logger.info(
|
||||
f"Launched headless browser for JavaScript rendering "
|
||||
f"(wait_until={self._browser_wait_until})"
|
||||
)
|
||||
return self._browser_renderer.render_page(url)
|
||||
|
||||
def scrape_page(self, url: str) -> None:
|
||||
@@ -1092,6 +1103,126 @@ class DocToSkillConverter:
|
||||
|
||||
return True
|
||||
|
||||
def _try_sitemap(self) -> list[str]:
|
||||
"""Layer 1: Try to discover pages via sitemap.xml.
|
||||
|
||||
Checks common sitemap locations at the domain root.
|
||||
Parses XML for <loc> tags, filters by is_valid_url().
|
||||
|
||||
Returns:
|
||||
List of discovered valid URLs (empty if no sitemap found).
|
||||
"""
|
||||
try:
|
||||
import defusedxml.ElementTree as ET
|
||||
except ImportError:
|
||||
import xml.etree.ElementTree as ET
|
||||
|
||||
from urllib.parse import urlparse
|
||||
|
||||
parsed = urlparse(self.base_url)
|
||||
domain = f"{parsed.scheme}://{parsed.netloc}"
|
||||
|
||||
sitemap_urls_to_try = [
|
||||
f"{domain}/sitemap.xml",
|
||||
f"{domain}/sitemap_index.xml",
|
||||
]
|
||||
|
||||
discovered = []
|
||||
|
||||
for sitemap_url in sitemap_urls_to_try:
|
||||
try:
|
||||
response = requests.get(
|
||||
sitemap_url, timeout=10, headers={"User-Agent": "SkillSeekers/3.4"}
|
||||
)
|
||||
if response.status_code != 200:
|
||||
continue
|
||||
|
||||
if "xml" not in response.headers.get("content-type", ""):
|
||||
continue
|
||||
|
||||
root = ET.fromstring(response.text)
|
||||
ns = {"sm": "http://www.sitemaps.org/schemas/sitemap/0.9"}
|
||||
|
||||
# Handle sitemap index (nested sitemaps)
|
||||
for sitemap in root.findall(".//sm:sitemap/sm:loc", ns):
|
||||
try:
|
||||
sub_resp = requests.get(
|
||||
sitemap.text.strip(),
|
||||
timeout=10,
|
||||
headers={"User-Agent": "SkillSeekers/3.4"},
|
||||
)
|
||||
if sub_resp.status_code == 200:
|
||||
sub_root = ET.fromstring(sub_resp.text)
|
||||
for loc in sub_root.findall(".//sm:url/sm:loc", ns):
|
||||
url = loc.text.strip().split("#")[0]
|
||||
if self.is_valid_url(url):
|
||||
discovered.append(url)
|
||||
except Exception:
|
||||
continue
|
||||
|
||||
# Handle direct sitemap
|
||||
for loc in root.findall(".//sm:url/sm:loc", ns):
|
||||
url = loc.text.strip().split("#")[0]
|
||||
if self.is_valid_url(url):
|
||||
discovered.append(url)
|
||||
|
||||
if discovered:
|
||||
logger.info(f"📋 Found sitemap at {sitemap_url} ({len(discovered)} valid URLs)")
|
||||
return list(set(discovered))
|
||||
|
||||
except Exception as e:
|
||||
logger.debug(f"Sitemap check failed for {sitemap_url}: {e}")
|
||||
continue
|
||||
|
||||
return []
|
||||
|
||||
def _discover_spa_nav(self) -> list[str]:
|
||||
"""Layer 3: Render index page with networkidle to discover SPA navigation.
|
||||
|
||||
Used when browser mode is on and sitemap/llms.txt didn't find pages.
|
||||
Renders the first page with networkidle (slower but discovers full nav),
|
||||
then normal crawl uses domcontentloaded (fast).
|
||||
|
||||
Returns:
|
||||
List of discovered valid URLs from the rendered navigation.
|
||||
"""
|
||||
if not self.browser_mode:
|
||||
return []
|
||||
|
||||
logger.info("🌐 Rendering index page with networkidle to discover SPA navigation...")
|
||||
|
||||
try:
|
||||
from skill_seekers.cli.browser_renderer import BrowserRenderer
|
||||
|
||||
# Use a separate renderer with networkidle for discovery only
|
||||
discovery_renderer = BrowserRenderer(
|
||||
timeout=60000,
|
||||
wait_until="networkidle",
|
||||
extra_wait=3000, # 3s extra for lazy-loaded nav
|
||||
)
|
||||
html = discovery_renderer.render_page(self.base_url)
|
||||
discovery_renderer.close()
|
||||
|
||||
# Parse rendered DOM for all links
|
||||
soup = BeautifulSoup(html, "html.parser")
|
||||
discovered = []
|
||||
seen = set()
|
||||
|
||||
for link in soup.find_all("a", href=True):
|
||||
href = urljoin(self.base_url, link["href"]).split("#")[0]
|
||||
if href not in seen and self.is_valid_url(href):
|
||||
seen.add(href)
|
||||
discovered.append(href)
|
||||
|
||||
if discovered:
|
||||
logger.info(f"🌐 Discovered {len(discovered)} pages from rendered SPA navigation")
|
||||
|
||||
return discovered
|
||||
|
||||
except Exception as e:
|
||||
logger.warning(f"⚠️ SPA navigation discovery failed: {e}")
|
||||
return []
|
||||
|
||||
def scrape_all(self) -> None:
|
||||
"""Scrape all pages (supports llms.txt and HTML scraping)
|
||||
|
||||
@@ -1102,16 +1233,35 @@ class DocToSkillConverter:
|
||||
asyncio.run(self.scrape_all_async())
|
||||
return
|
||||
|
||||
# Try llms.txt first (unless dry-run or explicitly disabled)
|
||||
if not self.dry_run and not self.skip_llms_txt:
|
||||
llms_result = self._try_llms_txt()
|
||||
if llms_result:
|
||||
logger.info(
|
||||
"\n✅ Used llms.txt (%s) - skipping HTML scraping",
|
||||
self.llms_txt_variant,
|
||||
)
|
||||
self.save_summary()
|
||||
return
|
||||
# === Three-Layer Discovery Engine ===
|
||||
# Discovers pages before the BFS crawl loop starts.
|
||||
# Layer 1: sitemap.xml — instant, no rendering needed
|
||||
# Layer 2: llms.txt — existing mechanism
|
||||
# Layer 3: SPA nav — renders index with networkidle to find JS-rendered links
|
||||
|
||||
if not self.dry_run:
|
||||
# Layer 1: Try sitemap.xml
|
||||
sitemap_urls = self._try_sitemap()
|
||||
if sitemap_urls:
|
||||
for url in sitemap_urls:
|
||||
self._enqueue_url(url)
|
||||
|
||||
# Layer 2: Try llms.txt (unless explicitly disabled)
|
||||
if not sitemap_urls and not self.skip_llms_txt:
|
||||
llms_result = self._try_llms_txt()
|
||||
if llms_result:
|
||||
logger.info(
|
||||
"\n✅ Used llms.txt (%s) - skipping HTML scraping",
|
||||
self.llms_txt_variant,
|
||||
)
|
||||
self.save_summary()
|
||||
return
|
||||
|
||||
# Layer 3: SPA nav discovery (browser mode only, when other layers found few pages)
|
||||
if self.browser_mode and len(self.pending_urls) <= 1:
|
||||
spa_urls = self._discover_spa_nav()
|
||||
for url in spa_urls:
|
||||
self._enqueue_url(url)
|
||||
|
||||
# HTML scraping (sync/thread-based logic)
|
||||
logger.info("\n" + "=" * 60)
|
||||
@@ -2158,7 +2308,7 @@ def setup_argument_parser() -> argparse.ArgumentParser:
|
||||
configs/react.json
|
||||
"""
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Convert documentation websites to Claude skills",
|
||||
description="Convert documentation websites to AI skills",
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter,
|
||||
)
|
||||
|
||||
@@ -2444,11 +2594,11 @@ def execute_scraping_and_building(
|
||||
|
||||
|
||||
def execute_enhancement(config: dict[str, Any], args: argparse.Namespace, converter=None) -> None:
|
||||
"""Execute optional SKILL.md enhancement with Claude.
|
||||
"""Execute optional SKILL.md enhancement with AI.
|
||||
|
||||
Supports two enhancement modes:
|
||||
1. API-based enhancement (requires ANTHROPIC_API_KEY)
|
||||
2. Local enhancement using Claude Code (no API key needed)
|
||||
2. Local enhancement using a coding agent CLI (no API key needed)
|
||||
|
||||
Prints appropriate messages and suggestions based on whether
|
||||
enhancement was requested and whether it succeeded.
|
||||
@@ -2490,10 +2640,11 @@ def execute_enhancement(config: dict[str, Any], args: argparse.Namespace, conver
|
||||
|
||||
try:
|
||||
enhance_cmd = ["skill-seekers-enhance", f"output/{config['name']}/"]
|
||||
enhance_cmd.extend(["--enhance-level", str(args.enhance_level)])
|
||||
|
||||
if args.api_key:
|
||||
enhance_cmd.extend(["--api-key", args.api_key])
|
||||
if getattr(args, "agent", None):
|
||||
enhance_cmd.extend(["--agent", args.agent])
|
||||
if getattr(args, "interactive_enhancement", False):
|
||||
enhance_cmd.append("--interactive-enhancement")
|
||||
|
||||
@@ -2505,9 +2656,8 @@ def execute_enhancement(config: dict[str, Any], args: argparse.Namespace, conver
|
||||
except FileNotFoundError:
|
||||
logger.warning("\n⚠ skill-seekers-enhance command not found. Run manually:")
|
||||
logger.info(
|
||||
" skill-seekers-enhance output/%s/ --enhance-level %d",
|
||||
" skill-seekers enhance output/%s/",
|
||||
config["name"],
|
||||
args.enhance_level,
|
||||
)
|
||||
|
||||
# Print packaging instructions
|
||||
@@ -2516,8 +2666,8 @@ def execute_enhancement(config: dict[str, Any], args: argparse.Namespace, conver
|
||||
|
||||
# Suggest enhancement if not done
|
||||
if getattr(args, "enhance_level", 0) == 0:
|
||||
logger.info("\n💡 Optional: Enhance SKILL.md with Claude:")
|
||||
logger.info(" skill-seekers-enhance output/%s/ --enhance-level 2", config["name"])
|
||||
logger.info("\n💡 Optional: Enhance SKILL.md with AI:")
|
||||
logger.info(" skill-seekers enhance output/%s/", config["name"])
|
||||
logger.info(" or re-run with: --enhance-level 2 (auto-detects API vs LOCAL mode)")
|
||||
logger.info(
|
||||
" API-based: skill-seekers-enhance-api output/%s/",
|
||||
|
||||
@@ -60,7 +60,13 @@ OPTIONAL_DEPS = {
|
||||
}
|
||||
|
||||
# ── API keys to check ────────────────────────────────────────────────────────
|
||||
API_KEYS = ["ANTHROPIC_API_KEY", "GITHUB_TOKEN", "GOOGLE_API_KEY", "OPENAI_API_KEY"]
|
||||
API_KEYS = [
|
||||
"ANTHROPIC_API_KEY",
|
||||
"GITHUB_TOKEN",
|
||||
"GOOGLE_API_KEY",
|
||||
"OPENAI_API_KEY",
|
||||
"MOONSHOT_API_KEY",
|
||||
]
|
||||
|
||||
|
||||
def _try_import(module_name: str) -> tuple[bool, str]:
|
||||
|
||||
@@ -4,25 +4,28 @@ Smart Enhancement Dispatcher
|
||||
|
||||
Routes `skill-seekers enhance` to the correct backend:
|
||||
|
||||
API mode — when an API key is available (Claude/Gemini/OpenAI).
|
||||
API mode — when an API key is available (Anthropic/Gemini/OpenAI).
|
||||
Calls enhance_skill.py which uses platform adaptors.
|
||||
|
||||
LOCAL mode — when no API key is found.
|
||||
Calls LocalSkillEnhancer from enhance_skill_local.py.
|
||||
Supports: Claude Code, OpenAI Codex, GitHub Copilot, OpenCode, Kimi, and other agents.
|
||||
|
||||
Decision priority:
|
||||
1. Explicit --target flag → API mode with that platform.
|
||||
2. Config ai_enhancement.default_agent + matching env key → API mode.
|
||||
3. Auto-detect from env vars: ANTHROPIC_API_KEY → claude,
|
||||
GOOGLE_API_KEY → gemini, OPENAI_API_KEY → openai.
|
||||
4. No API keys → LOCAL mode (Claude Code CLI).
|
||||
5. LOCAL mode + running as root → clear error (Claude Code refuses root).
|
||||
4. No API keys → LOCAL mode (AI coding agent).
|
||||
5. LOCAL mode + running as root → clear error (AI coding agent refuses root).
|
||||
"""
|
||||
|
||||
import os
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
from skill_seekers.cli.agent_client import get_default_timeout
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Helpers
|
||||
@@ -43,6 +46,7 @@ def _get_api_keys() -> dict[str, str | None]:
|
||||
"claude": (os.environ.get("ANTHROPIC_API_KEY") or os.environ.get("ANTHROPIC_AUTH_TOKEN")),
|
||||
"gemini": os.environ.get("GOOGLE_API_KEY"),
|
||||
"openai": os.environ.get("OPENAI_API_KEY"),
|
||||
"kimi": os.environ.get("MOONSHOT_API_KEY"),
|
||||
}
|
||||
|
||||
|
||||
@@ -73,11 +77,11 @@ def _pick_mode(args) -> tuple[str, str | None]:
|
||||
|
||||
# 2. Config default_agent preference (if a matching key is available).
|
||||
config_agent = _get_config_default_agent()
|
||||
if config_agent in ("claude", "gemini", "openai") and api_keys.get(config_agent):
|
||||
if config_agent in ("claude", "gemini", "openai", "kimi") and api_keys.get(config_agent):
|
||||
return "api", config_agent
|
||||
|
||||
# 3. Auto-detect from environment variables.
|
||||
# Priority: Claude > Gemini > OpenAI (Claude is Anthropic's native platform).
|
||||
# Priority: Anthropic > Gemini > OpenAI.
|
||||
if api_keys["claude"]:
|
||||
return "api", "claude"
|
||||
if api_keys["gemini"]:
|
||||
@@ -156,7 +160,7 @@ def _run_local_mode(args) -> int:
|
||||
headless = not interactive
|
||||
success = enhancer.run(
|
||||
headless=headless,
|
||||
timeout=getattr(args, "timeout", 600),
|
||||
timeout=getattr(args, "timeout", None) or get_default_timeout(),
|
||||
background=getattr(args, "background", False),
|
||||
daemon=getattr(args, "daemon", False),
|
||||
)
|
||||
@@ -176,15 +180,15 @@ def main() -> int:
|
||||
parser = argparse.ArgumentParser(
|
||||
description=(
|
||||
"Enhance SKILL.md using AI. "
|
||||
"Automatically selects API mode (Gemini/OpenAI/Claude API) when an API key "
|
||||
"is available, or falls back to LOCAL mode (Claude Code CLI)."
|
||||
"Automatically selects API mode (Anthropic/Gemini/OpenAI API) when an API key "
|
||||
"is available, or falls back to LOCAL mode (AI coding agent)."
|
||||
),
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter,
|
||||
epilog="""
|
||||
Mode selection (automatic — no flags required):
|
||||
API mode : Set ANTHROPIC_API_KEY, GOOGLE_API_KEY, or OPENAI_API_KEY.
|
||||
Or use --target to force a platform.
|
||||
LOCAL mode: Falls back when no API keys are found. Requires Claude Code CLI.
|
||||
LOCAL mode: Falls back when no API keys are found. Requires AI coding agent.
|
||||
Does NOT work as root (Docker/VPS) — use API mode instead.
|
||||
|
||||
Examples:
|
||||
@@ -194,7 +198,7 @@ Examples:
|
||||
# Force Gemini API
|
||||
skill-seekers enhance output/react/ --target gemini
|
||||
|
||||
# Force Claude API with explicit key
|
||||
# Force Anthropic API with explicit key
|
||||
skill-seekers enhance output/react/ --target claude --api-key sk-ant-...
|
||||
|
||||
# LOCAL mode options
|
||||
@@ -244,10 +248,10 @@ Examples:
|
||||
if _is_root():
|
||||
print("❌ Cannot run LOCAL enhancement as root.")
|
||||
print()
|
||||
print(" Claude Code CLI refuses to execute as root (Docker/VPS security policy).")
|
||||
print(" AI coding agent refuses to execute as root (Docker/VPS security policy).")
|
||||
print(" Use API mode instead by setting one of these environment variables:")
|
||||
print()
|
||||
print(" export ANTHROPIC_API_KEY=sk-ant-... # Claude")
|
||||
print(" export ANTHROPIC_API_KEY=sk-ant-... # Anthropic")
|
||||
print(" export GOOGLE_API_KEY=AIza... # Gemini")
|
||||
print(" export OPENAI_API_KEY=sk-proj-... # OpenAI")
|
||||
print()
|
||||
@@ -255,7 +259,8 @@ Examples:
|
||||
print(f" skill-seekers enhance {args.skill_directory}")
|
||||
return 1
|
||||
|
||||
print("🤖 Enhancement mode: LOCAL (Claude Code CLI)")
|
||||
agent_name = os.environ.get("SKILL_SEEKER_AGENT", "claude").strip() or "claude"
|
||||
print(f"🤖 Enhancement mode: LOCAL ({agent_name})")
|
||||
return _run_local_mode(args)
|
||||
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@ SKILL.md Enhancement Script
|
||||
Uses platform AI APIs to improve SKILL.md by analyzing reference documentation.
|
||||
|
||||
Usage:
|
||||
# Claude (default)
|
||||
# Anthropic (default)
|
||||
skill-seekers enhance output/react/
|
||||
skill-seekers enhance output/react/ --api-key sk-ant-...
|
||||
|
||||
@@ -63,17 +63,17 @@ class SkillEnhancer:
|
||||
return self.skill_md_path.read_text(encoding="utf-8")
|
||||
|
||||
def enhance_skill_md(self, references, current_skill_md):
|
||||
"""Use Claude to enhance SKILL.md"""
|
||||
"""Use AI to enhance SKILL.md"""
|
||||
|
||||
# Build prompt
|
||||
prompt = self._build_enhancement_prompt(references, current_skill_md)
|
||||
|
||||
print("\n🤖 Asking Claude to enhance SKILL.md...")
|
||||
print("\n🤖 Asking AI to enhance SKILL.md...")
|
||||
print(f" Input: {len(prompt):,} characters")
|
||||
|
||||
try:
|
||||
message = self.client.messages.create(
|
||||
model="claude-sonnet-4-20250514",
|
||||
model=os.environ.get("ANTHROPIC_MODEL", "claude-sonnet-4-20250514"),
|
||||
max_tokens=4096,
|
||||
temperature=0.3,
|
||||
messages=[{"role": "user", "content": prompt}],
|
||||
@@ -94,7 +94,7 @@ class SkillEnhancer:
|
||||
return enhanced_content
|
||||
|
||||
except Exception as e:
|
||||
print(f"❌ Error calling Claude API: {e}")
|
||||
print(f"❌ Error calling AI API: {e}")
|
||||
return None
|
||||
|
||||
def _is_video_source(self, references):
|
||||
@@ -102,7 +102,7 @@ class SkillEnhancer:
|
||||
return any(meta["source"] == "video_tutorial" for meta in references.values())
|
||||
|
||||
def _build_enhancement_prompt(self, references, current_skill_md):
|
||||
"""Build the prompt for Claude with multi-source awareness"""
|
||||
"""Build the prompt for AI with multi-source awareness"""
|
||||
|
||||
# Dispatch to video-specific prompt if video source detected
|
||||
if self._is_video_source(references):
|
||||
@@ -119,7 +119,7 @@ class SkillEnhancer:
|
||||
# Analyze conflicts if present
|
||||
has_conflicts = any("conflicts" in meta["path"] for meta in references.values())
|
||||
|
||||
prompt = f"""You are enhancing a Claude skill's SKILL.md file. This skill is about: {skill_name}
|
||||
prompt = f"""You are enhancing an LLM skill's SKILL.md file. This skill is about: {skill_name}
|
||||
|
||||
I've scraped documentation from multiple sources and organized it into reference files. Your job is to create an EXCELLENT SKILL.md that synthesizes knowledge from these sources.
|
||||
|
||||
@@ -275,7 +275,7 @@ IMPORTANT:
|
||||
- Prioritize SHORT, clear examples (5-20 lines max)
|
||||
- Make it actionable and practical
|
||||
- Don't be too verbose - be concise but useful
|
||||
- Maintain the markdown structure for Claude skills
|
||||
- Maintain the markdown structure for LLM skills
|
||||
- Keep code examples properly formatted with language tags
|
||||
|
||||
OUTPUT:
|
||||
@@ -294,7 +294,7 @@ Return ONLY the complete SKILL.md content, starting with the frontmatter (---).
|
||||
"""
|
||||
skill_name = self.skill_dir.name
|
||||
|
||||
prompt = f"""You are enhancing a Claude skill built from VIDEO TUTORIAL extraction. This skill is about: {skill_name}
|
||||
prompt = f"""You are enhancing an LLM skill built from VIDEO TUTORIAL extraction. This skill is about: {skill_name}
|
||||
|
||||
The raw data was extracted from video tutorials using:
|
||||
1. **Transcript** (speech-to-text) — HIGH quality, this is the primary signal
|
||||
@@ -471,7 +471,7 @@ Return ONLY the complete SKILL.md content, starting with the frontmatter (---).
|
||||
else:
|
||||
print(" ℹ No existing SKILL.md, will create new one")
|
||||
|
||||
# Enhance with Claude
|
||||
# Enhance with AI
|
||||
enhanced = self.enhance_skill_md(references, current_skill_md)
|
||||
|
||||
if not enhanced:
|
||||
@@ -502,7 +502,7 @@ def main():
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter,
|
||||
epilog="""
|
||||
Examples:
|
||||
# Claude (default)
|
||||
# Anthropic (default)
|
||||
export ANTHROPIC_API_KEY=sk-ant-...
|
||||
skill-seekers enhance output/react/
|
||||
|
||||
@@ -530,9 +530,9 @@ Examples:
|
||||
)
|
||||
parser.add_argument(
|
||||
"--target",
|
||||
choices=["claude", "gemini", "openai"],
|
||||
default="claude",
|
||||
help="Target LLM platform (default: claude)",
|
||||
choices=["claude", "gemini", "openai", "kimi"],
|
||||
default=None,
|
||||
help="Target LLM platform (auto-detected from API keys, or 'claude' if none set)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--dry-run", action="store_true", help="Show what would be done without calling API"
|
||||
@@ -540,6 +540,12 @@ Examples:
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
# Auto-detect target platform if not specified
|
||||
if args.target is None:
|
||||
from skill_seekers.cli.agent_client import AgentClient
|
||||
|
||||
args.target = AgentClient.detect_default_target()
|
||||
|
||||
# Validate skill directory
|
||||
skill_dir = Path(args.skill_dir)
|
||||
if not skill_dir.exists():
|
||||
@@ -578,7 +584,7 @@ Examples:
|
||||
if not adaptor.supports_enhancement():
|
||||
print(f"❌ Error: {adaptor.PLATFORM_NAME} does not support AI enhancement")
|
||||
print("\nSupported platforms for enhancement:")
|
||||
print(" - Claude AI (Anthropic)")
|
||||
print(" - Anthropic (Claude AI)")
|
||||
print(" - Google Gemini")
|
||||
print(" - OpenAI ChatGPT")
|
||||
sys.exit(1)
|
||||
|
||||
@@ -132,6 +132,12 @@ AGENT_PRESETS = {
|
||||
"command": ["opencode"],
|
||||
"supports_skip_permissions": False,
|
||||
},
|
||||
"kimi": {
|
||||
"display_name": "Kimi Code CLI",
|
||||
"command": ["kimi", "--print", "--input-format", "text", "--work-dir", "{skill_dir}"],
|
||||
"supports_skip_permissions": False,
|
||||
"uses_stdin": True,
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
@@ -146,6 +152,7 @@ def _normalize_agent_name(agent_name: str) -> str:
|
||||
"copilot-cli": "copilot",
|
||||
"open-code": "opencode",
|
||||
"open_code": "opencode",
|
||||
"kimi-cli": "kimi",
|
||||
}
|
||||
return aliases.get(normalized, normalized)
|
||||
|
||||
@@ -238,6 +245,8 @@ class LocalSkillEnhancer:
|
||||
if "{prompt_file}" in arg:
|
||||
cmd_parts[idx] = arg.replace("{prompt_file}", prompt_file)
|
||||
uses_prompt_file = True
|
||||
if "{skill_dir}" in arg:
|
||||
cmd_parts[idx] = arg.replace("{skill_dir}", str(self.skill_dir.resolve()))
|
||||
|
||||
return cmd_parts, uses_prompt_file
|
||||
|
||||
@@ -652,7 +661,7 @@ After writing, the file SKILL.md should:
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
def run(self, headless=True, timeout=600, background=False, daemon=False):
|
||||
def run(self, headless=True, timeout=2700, background=False, daemon=False):
|
||||
"""Main enhancement workflow with automatic smart summarization for large skills.
|
||||
|
||||
Automatically detects large skills (>30K chars) and applies smart summarization
|
||||
@@ -666,7 +675,7 @@ After writing, the file SKILL.md should:
|
||||
|
||||
Args:
|
||||
headless: If True, run local agent directly without opening terminal (default: True)
|
||||
timeout: Maximum time to wait for enhancement in seconds (default: 600 = 10 minutes)
|
||||
timeout: Maximum time to wait for enhancement in seconds (default: 2700 = 45 minutes)
|
||||
background: If True, run in background and return immediately (default: False)
|
||||
daemon: If True, run as persistent daemon with monitoring (default: False)
|
||||
|
||||
@@ -711,7 +720,7 @@ After writing, the file SKILL.md should:
|
||||
print("⚠️ LARGE SKILL DETECTED")
|
||||
print(f" 📊 Reference content: {total_size:,} characters")
|
||||
if self.agent == "claude":
|
||||
print(" 💡 Claude CLI limit: ~30,000-40,000 characters")
|
||||
print(" 💡 CLI agent limit: ~30,000-40,000 characters")
|
||||
else:
|
||||
print(" 💡 Local CLI agents often have input limits; summarizing to be safe")
|
||||
print()
|
||||
@@ -739,7 +748,7 @@ After writing, the file SKILL.md should:
|
||||
if use_summarization:
|
||||
print(f" ✓ Prompt created and optimized ({len(prompt):,} characters)")
|
||||
if self.agent == "claude":
|
||||
print(" ✓ Ready for Claude CLI (within safe limits)")
|
||||
print(" ✓ Ready for CLI agent (within safe limits)")
|
||||
else:
|
||||
print(" ✓ Ready for local CLI (within safe limits)")
|
||||
print()
|
||||
@@ -905,6 +914,45 @@ rm {prompt_file}
|
||||
print("❌ SKILL.md not found after enhancement")
|
||||
return False
|
||||
else:
|
||||
# Exit code 75 = EX_TEMPFAIL (retryable temporary failure from Kimi CLI)
|
||||
if result.returncode == 75:
|
||||
print(f"⚠️ {self.agent_display} returned temporary failure (exit code: 75)")
|
||||
print(
|
||||
" This usually means a transient API issue (timeout, rate limit, or empty response)."
|
||||
)
|
||||
print(" Retrying once in 5 seconds...")
|
||||
print()
|
||||
time.sleep(5)
|
||||
result_retry, error = self._run_agent_command(
|
||||
prompt_file, timeout, include_permissions_flag=True
|
||||
)
|
||||
if error:
|
||||
print(f"❌ {error}")
|
||||
with contextlib.suppress(Exception):
|
||||
os.unlink(prompt_file)
|
||||
return False
|
||||
if result_retry.returncode == 0:
|
||||
elapsed = time.time() - start_time
|
||||
if self.skill_md_path.exists():
|
||||
new_mtime = self.skill_md_path.stat().st_mtime
|
||||
new_size = self.skill_md_path.stat().st_size
|
||||
if new_mtime > initial_mtime and new_size > initial_size:
|
||||
print(f"✅ Enhancement complete on retry! ({elapsed:.1f} seconds)")
|
||||
print(f" SKILL.md updated: {new_size:,} bytes")
|
||||
print()
|
||||
with contextlib.suppress(Exception):
|
||||
os.unlink(prompt_file)
|
||||
return True
|
||||
print(f"❌ Retry also failed (exit code: {result_retry.returncode})")
|
||||
if result_retry.stderr:
|
||||
stderr_lines = result_retry.stderr.strip().split("\n")
|
||||
for line in stderr_lines[:10]:
|
||||
print(f" | {line}")
|
||||
print(" Try again later or use API mode:")
|
||||
print(" export ANTHROPIC_API_KEY=sk-ant-...")
|
||||
print(f" skill-seekers enhance {self.skill_dir} --target claude")
|
||||
return False
|
||||
|
||||
print(f"❌ {self.agent_display} returned error (exit code: {result.returncode})")
|
||||
if result.stderr:
|
||||
stderr_lines = result.stderr.strip().split("\n")
|
||||
@@ -919,7 +967,7 @@ rm {prompt_file}
|
||||
):
|
||||
print()
|
||||
print(" ⚠️ This looks like a root/permission error.")
|
||||
print(" Claude Code CLI refuses to run as root (security policy).")
|
||||
print(" The CLI agent refuses to run as root (security policy).")
|
||||
print(" Use API mode instead:")
|
||||
print(" export ANTHROPIC_API_KEY=sk-ant-...")
|
||||
print(f" skill-seekers enhance {self.skill_dir} --target claude")
|
||||
@@ -1289,10 +1337,10 @@ def main():
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter,
|
||||
epilog="""
|
||||
Auto-detection (no flags needed):
|
||||
If ANTHROPIC_API_KEY is set → Claude API mode
|
||||
If ANTHROPIC_API_KEY is set → Anthropic API mode
|
||||
If GOOGLE_API_KEY is set → Gemini API mode
|
||||
If OPENAI_API_KEY is set → OpenAI API mode
|
||||
Otherwise → LOCAL mode (Claude Code Max, free)
|
||||
Otherwise → LOCAL mode (coding agent CLI, free)
|
||||
|
||||
Examples:
|
||||
# Auto-detect mode based on env vars (recommended)
|
||||
@@ -1373,11 +1421,16 @@ Force Mode (LOCAL only, Default ON):
|
||||
help="Disable force mode: enable confirmation prompts (default: force mode ON)",
|
||||
)
|
||||
|
||||
from skill_seekers.cli.agent_client import get_default_timeout
|
||||
|
||||
parser.add_argument(
|
||||
"--timeout",
|
||||
type=int,
|
||||
default=600,
|
||||
help="Timeout in seconds for headless mode (default: 600 = 10 minutes)",
|
||||
default=get_default_timeout(),
|
||||
help=(
|
||||
"Timeout in seconds for headless mode "
|
||||
"(default: 45 minutes, set SKILL_SEEKER_ENHANCE_TIMEOUT to override)"
|
||||
),
|
||||
)
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
@@ -84,12 +84,13 @@ class WorkflowEngine:
|
||||
- Target specific parts of the analysis
|
||||
"""
|
||||
|
||||
def __init__(self, workflow: EnhancementWorkflow | str | Path):
|
||||
def __init__(self, workflow: EnhancementWorkflow | str | Path, agent: str | None = None):
|
||||
"""
|
||||
Initialize workflow engine.
|
||||
|
||||
Args:
|
||||
workflow: EnhancementWorkflow object or path to YAML file
|
||||
agent: Local CLI agent name (e.g., "kimi", "claude")
|
||||
"""
|
||||
if isinstance(workflow, (str, Path)):
|
||||
self.workflow = self._load_workflow(workflow)
|
||||
@@ -98,6 +99,7 @@ class WorkflowEngine:
|
||||
|
||||
self.history: list[dict[str, Any]] = []
|
||||
self.enhancer = None # Lazy load UnifiedEnhancer
|
||||
self.agent = agent
|
||||
|
||||
def _load_workflow(self, workflow_ref: str | Path) -> EnhancementWorkflow:
|
||||
"""Load workflow from YAML file using 3-level search order.
|
||||
@@ -350,15 +352,23 @@ class WorkflowEngine:
|
||||
current = context["current_results"]
|
||||
|
||||
# Determine what to enhance based on target
|
||||
if stage.target == "patterns" and "patterns" in current:
|
||||
enhancer = PatternEnhancer()
|
||||
enhanced_patterns = enhancer.enhance_patterns(current["patterns"])
|
||||
return {"patterns": enhanced_patterns}
|
||||
if stage.target == "patterns":
|
||||
if "patterns" in current:
|
||||
enhancer = PatternEnhancer(agent=self.agent)
|
||||
enhanced_patterns = enhancer.enhance_patterns(current["patterns"])
|
||||
return {"patterns": enhanced_patterns}
|
||||
else:
|
||||
logger.info(f" ℹ️ No {stage.target} data available, skipping builtin stage")
|
||||
return {}
|
||||
|
||||
elif stage.target == "examples" and "examples" in current:
|
||||
enhancer = TestExampleEnhancer()
|
||||
enhanced_examples = enhancer.enhance_examples(current["examples"])
|
||||
return {"examples": enhanced_examples}
|
||||
elif stage.target == "examples":
|
||||
if "examples" in current:
|
||||
enhancer = TestExampleEnhancer(agent=self.agent)
|
||||
enhanced_examples = enhancer.enhance_examples(current["examples"])
|
||||
return {"examples": enhanced_examples}
|
||||
else:
|
||||
logger.info(f" ℹ️ No {stage.target} data available, skipping builtin stage")
|
||||
return {}
|
||||
|
||||
else:
|
||||
logger.warning(f"Unknown builtin target: {stage.target}")
|
||||
@@ -374,7 +384,7 @@ class WorkflowEngine:
|
||||
if not self.enhancer:
|
||||
from skill_seekers.cli.ai_enhancer import AIEnhancer
|
||||
|
||||
self.enhancer = AIEnhancer()
|
||||
self.enhancer = AIEnhancer(agent=self.agent)
|
||||
|
||||
# Format prompt with context
|
||||
try:
|
||||
@@ -385,7 +395,11 @@ class WorkflowEngine:
|
||||
|
||||
# Call AI with custom prompt
|
||||
logger.info(f" 🤖 Running custom AI prompt...")
|
||||
response = self.enhancer._call_claude(formatted_prompt, max_tokens=3000)
|
||||
# Use call() (agent-agnostic) with _call_claude() as fallback for older enhancers
|
||||
if hasattr(self.enhancer, "call"):
|
||||
response = self.enhancer.call(formatted_prompt, max_tokens=3000)
|
||||
else:
|
||||
response = self.enhancer._call_claude(formatted_prompt, max_tokens=3000)
|
||||
|
||||
if not response:
|
||||
logger.warning(f" ⚠️ No response from AI")
|
||||
|
||||
@@ -69,7 +69,7 @@ def infer_description_from_epub(metadata: dict | None = None, name: str = "") ->
|
||||
|
||||
|
||||
class EpubToSkillConverter:
|
||||
"""Convert EPUB e-book to Claude skill."""
|
||||
"""Convert EPUB e-book to AI skill."""
|
||||
|
||||
def __init__(self, config):
|
||||
self.config = config
|
||||
@@ -1179,13 +1179,17 @@ def main():
|
||||
print("❌ API enhancement not available. Falling back to LOCAL mode...")
|
||||
from skill_seekers.cli.enhance_skill_local import LocalSkillEnhancer
|
||||
|
||||
enhancer = LocalSkillEnhancer(Path(skill_dir))
|
||||
agent = getattr(args, "agent", None) if args else None
|
||||
agent_cmd = getattr(args, "agent_cmd", None) if args else None
|
||||
enhancer = LocalSkillEnhancer(Path(skill_dir), agent=agent, agent_cmd=agent_cmd)
|
||||
enhancer.run(headless=True)
|
||||
print("✅ Local enhancement complete!")
|
||||
else:
|
||||
from skill_seekers.cli.enhance_skill_local import LocalSkillEnhancer
|
||||
|
||||
enhancer = LocalSkillEnhancer(Path(skill_dir))
|
||||
agent = getattr(args, "agent", None) if args else None
|
||||
agent_cmd = getattr(args, "agent_cmd", None) if args else None
|
||||
enhancer = LocalSkillEnhancer(Path(skill_dir), agent=agent, agent_cmd=agent_cmd)
|
||||
enhancer.run(headless=True)
|
||||
print("✅ Local enhancement complete!")
|
||||
|
||||
|
||||
@@ -1202,7 +1202,7 @@ Examples:
|
||||
print(f" skill-seekers scrape --config {config_path}")
|
||||
print("3. Package router skill:")
|
||||
print(f" skill-seekers package output/{generator.router_name}/")
|
||||
print("4. Upload router + all sub-skills to Claude")
|
||||
print("4. Upload router + all sub-skills to target platform")
|
||||
print("")
|
||||
|
||||
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
GitHub Repository to Claude Skill Converter (Tasks C1.1-C1.12)
|
||||
GitHub Repository to AI Skill Converter (Tasks C1.1-C1.12)
|
||||
|
||||
Converts GitHub repositories into Claude AI skills by extracting:
|
||||
Converts GitHub repositories into AI skills by extracting:
|
||||
- README and documentation
|
||||
- Code structure and signatures
|
||||
- GitHub Issues, Changelog, and Releases
|
||||
@@ -714,17 +714,7 @@ class GitHubScraper:
|
||||
|
||||
logger.info(f"Extracting code signatures ({self.code_analysis_depth} analysis)...")
|
||||
|
||||
# Get primary language for the repository
|
||||
languages = self.extracted_data.get("languages", {})
|
||||
if not languages:
|
||||
logger.warning("No languages detected - skipping code analysis")
|
||||
return
|
||||
|
||||
# Determine primary language
|
||||
primary_language = max(languages.items(), key=lambda x: x[1]["bytes"])[0]
|
||||
logger.info(f"Primary language: {primary_language}")
|
||||
|
||||
# Determine file extensions to analyze
|
||||
# Build reverse extension → language map for per-file detection
|
||||
extension_map = {
|
||||
"Python": [".py"],
|
||||
"JavaScript": [".js", ".jsx"],
|
||||
@@ -733,72 +723,86 @@ class GitHubScraper:
|
||||
"Java": [".java"],
|
||||
"C": [".c", ".h"],
|
||||
"C++": [".cpp", ".hpp", ".cc", ".hh", ".cxx"],
|
||||
"C#": [".cs"],
|
||||
"Go": [".go"],
|
||||
"Rust": [".rs"],
|
||||
"Swift": [".swift"],
|
||||
"Ruby": [".rb"],
|
||||
"PHP": [".php"],
|
||||
"GDScript": [".gd"],
|
||||
}
|
||||
ext_to_lang = {}
|
||||
for lang, exts in extension_map.items():
|
||||
for ext in exts:
|
||||
ext_to_lang[ext] = lang
|
||||
|
||||
extensions = extension_map.get(primary_language, [])
|
||||
if not extensions:
|
||||
logger.warning(f"No file extensions mapped for {primary_language}")
|
||||
return
|
||||
# Optional: filter to specific languages from config
|
||||
target_languages = None
|
||||
config_language = self.config.get("language", "")
|
||||
if config_language:
|
||||
target_languages = {lang.strip() for lang in config_language.split(",")}
|
||||
logger.info(f"Language filter from config: {', '.join(sorted(target_languages))}")
|
||||
|
||||
# Analyze files matching patterns and extensions
|
||||
# Analyze ALL files, detecting language per-file from extension
|
||||
analyzed_files = []
|
||||
file_tree = self.extracted_data.get("file_tree", [])
|
||||
languages_found = set()
|
||||
|
||||
for file_info in file_tree:
|
||||
file_path = file_info["path"]
|
||||
|
||||
# Check if file matches extension
|
||||
if not any(file_path.endswith(ext) for ext in extensions):
|
||||
# Detect language from file extension
|
||||
ext = os.path.splitext(file_path)[1].lower()
|
||||
language = ext_to_lang.get(ext)
|
||||
if not language:
|
||||
continue
|
||||
|
||||
# Check if file matches patterns (if specified)
|
||||
# Apply language filter if config specifies target languages
|
||||
if target_languages and language not in target_languages:
|
||||
continue
|
||||
|
||||
# Check if file matches patterns (if specified in config)
|
||||
if self.file_patterns and not any(
|
||||
fnmatch.fnmatch(file_path, pattern) for pattern in self.file_patterns
|
||||
):
|
||||
continue
|
||||
|
||||
# Analyze this file
|
||||
# Analyze this file with the correct language
|
||||
try:
|
||||
# Read file content based on mode
|
||||
if self.local_repo_path:
|
||||
# Local mode - read from filesystem
|
||||
full_path = os.path.join(self.local_repo_path, file_path)
|
||||
with open(full_path, encoding="utf-8") as f:
|
||||
content = f.read()
|
||||
else:
|
||||
# GitHub API mode - fetch from API
|
||||
file_content = self.repo.get_contents(file_path)
|
||||
content = file_content.decoded_content.decode("utf-8")
|
||||
|
||||
analysis_result = self.code_analyzer.analyze_file(
|
||||
file_path, content, primary_language
|
||||
)
|
||||
analysis_result = self.code_analyzer.analyze_file(file_path, content, language)
|
||||
|
||||
if analysis_result and (
|
||||
analysis_result.get("classes") or analysis_result.get("functions")
|
||||
):
|
||||
analyzed_files.append(
|
||||
{"file": file_path, "language": primary_language, **analysis_result}
|
||||
)
|
||||
|
||||
logger.debug(
|
||||
f"Analyzed {file_path}: "
|
||||
f"{len(analysis_result.get('classes', []))} classes, "
|
||||
f"{len(analysis_result.get('functions', []))} functions"
|
||||
{"file": file_path, "language": language, **analysis_result}
|
||||
)
|
||||
languages_found.add(language)
|
||||
|
||||
except Exception as e:
|
||||
logger.debug(f"Could not analyze {file_path}: {e}")
|
||||
continue
|
||||
|
||||
# Limit number of files analyzed to avoid rate limits (GitHub API mode only)
|
||||
if not self.local_repo_path and len(analyzed_files) >= 50:
|
||||
logger.info("Reached analysis limit (50 files, GitHub API mode)")
|
||||
break
|
||||
# Determine primary language for backward compat in output
|
||||
repo_languages = self.extracted_data.get("languages", {})
|
||||
primary_language = (
|
||||
max(repo_languages.items(), key=lambda x: x[1]["bytes"])[0]
|
||||
if repo_languages
|
||||
else "Unknown"
|
||||
)
|
||||
|
||||
self.extracted_data["code_analysis"] = {
|
||||
"depth": self.code_analysis_depth,
|
||||
"language": primary_language,
|
||||
"languages_analyzed": sorted(languages_found),
|
||||
"files_analyzed": len(analyzed_files),
|
||||
"files": analyzed_files,
|
||||
}
|
||||
@@ -807,8 +811,9 @@ class GitHubScraper:
|
||||
total_classes = sum(len(f.get("classes", [])) for f in analyzed_files)
|
||||
total_functions = sum(len(f.get("functions", [])) for f in analyzed_files)
|
||||
|
||||
lang_summary = ", ".join(sorted(languages_found)) if languages_found else "none"
|
||||
logger.info(
|
||||
f"Code analysis complete: {len(analyzed_files)} files, {total_classes} classes, {total_functions} functions"
|
||||
f"Code analysis complete: {len(analyzed_files)} files, {total_classes} classes, {total_functions} functions ({lang_summary})"
|
||||
)
|
||||
|
||||
def _extract_issues(self):
|
||||
@@ -913,7 +918,7 @@ class GitHubScraper:
|
||||
|
||||
class GitHubToSkillConverter:
|
||||
"""
|
||||
Convert extracted GitHub data to Claude skill format (C1.10).
|
||||
Convert extracted GitHub data to AI skill format (C1.10).
|
||||
"""
|
||||
|
||||
def __init__(self, config: dict[str, Any]):
|
||||
@@ -1387,7 +1392,7 @@ def setup_argument_parser() -> argparse.ArgumentParser:
|
||||
argparse.ArgumentParser: Configured argument parser
|
||||
"""
|
||||
parser = argparse.ArgumentParser(
|
||||
description="GitHub Repository to Claude Skill Converter",
|
||||
description="GitHub Repository to AI Skill Converter",
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter,
|
||||
epilog="""
|
||||
Examples:
|
||||
@@ -1518,23 +1523,29 @@ def main():
|
||||
from pathlib import Path
|
||||
from skill_seekers.cli.enhance_skill_local import LocalSkillEnhancer
|
||||
|
||||
enhancer = LocalSkillEnhancer(Path(skill_dir))
|
||||
agent = getattr(args, "agent", None) if args else None
|
||||
agent_cmd = getattr(args, "agent_cmd", None) if args else None
|
||||
enhancer = LocalSkillEnhancer(Path(skill_dir), agent=agent, agent_cmd=agent_cmd)
|
||||
enhancer.run(headless=True)
|
||||
logger.info("✅ Local enhancement complete!")
|
||||
agent_name = agent or "claude"
|
||||
logger.info(f"✅ Local enhancement complete! (via {agent_name})")
|
||||
else:
|
||||
# LOCAL enhancement (no API key)
|
||||
from pathlib import Path
|
||||
from skill_seekers.cli.enhance_skill_local import LocalSkillEnhancer
|
||||
|
||||
enhancer = LocalSkillEnhancer(Path(skill_dir))
|
||||
agent = getattr(args, "agent", None) if args else None
|
||||
agent_cmd = getattr(args, "agent_cmd", None) if args else None
|
||||
enhancer = LocalSkillEnhancer(Path(skill_dir), agent=agent, agent_cmd=agent_cmd)
|
||||
enhancer.run(headless=True)
|
||||
logger.info("✅ Local enhancement complete!")
|
||||
agent_name = agent or "claude"
|
||||
logger.info(f"✅ Local enhancement complete! (via {agent_name})")
|
||||
|
||||
logger.info(f"\n✅ Success! Skill created at: {skill_dir}/")
|
||||
|
||||
# Only suggest enhancement if neither workflow nor traditional enhancement was done
|
||||
if not workflow_executed and getattr(args, "enhance_level", 0) == 0:
|
||||
logger.info("\n💡 Optional: Enhance SKILL.md with Claude:")
|
||||
logger.info("\n💡 Optional: Enhance SKILL.md with AI:")
|
||||
logger.info(f" skill-seekers enhance {skill_dir}/ --enhance-level 2")
|
||||
logger.info(" (auto-detects API vs LOCAL mode based on ANTHROPIC_API_KEY)")
|
||||
logger.info("\n💡 Or use a workflow:")
|
||||
|
||||
@@ -2,8 +2,8 @@
|
||||
AI Enhancement for How-To Guides (C3.3)
|
||||
|
||||
This module provides comprehensive AI enhancement for how-to guides with dual-mode support:
|
||||
- API mode: Uses Claude API (requires ANTHROPIC_API_KEY)
|
||||
- LOCAL mode: Uses Claude Code CLI (no API key needed)
|
||||
- API mode: Uses Anthropic API (requires ANTHROPIC_API_KEY)
|
||||
- LOCAL mode: Uses a coding agent CLI (no API key needed)
|
||||
|
||||
Provides 5 automatic enhancements:
|
||||
1. Step Descriptions - Natural language explanations (not just syntax)
|
||||
@@ -15,11 +15,7 @@ Provides 5 automatic enhancements:
|
||||
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import subprocess
|
||||
import tempfile
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
# Avoid circular imports by using TYPE_CHECKING
|
||||
@@ -47,14 +43,8 @@ else:
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Conditional import for Anthropic API
|
||||
try:
|
||||
import anthropic
|
||||
|
||||
ANTHROPIC_AVAILABLE = True
|
||||
except ImportError:
|
||||
ANTHROPIC_AVAILABLE = False
|
||||
logger.debug("Anthropic library not available - API mode will be unavailable")
|
||||
# ANTHROPIC_AVAILABLE kept for backward compatibility — AgentClient handles detection
|
||||
ANTHROPIC_AVAILABLE = True # Detection delegated to AgentClient
|
||||
|
||||
|
||||
@dataclass
|
||||
@@ -71,8 +61,8 @@ class GuideEnhancer:
|
||||
AI enhancement for how-to guides with dual-mode support.
|
||||
|
||||
Modes:
|
||||
- api: Uses Claude API (requires ANTHROPIC_API_KEY)
|
||||
- local: Uses Claude Code CLI (no API key needed)
|
||||
- api: Uses Anthropic API (requires ANTHROPIC_API_KEY)
|
||||
- local: Uses a coding agent CLI (no API key needed)
|
||||
- auto: Automatically detect best mode
|
||||
"""
|
||||
|
||||
@@ -83,77 +73,17 @@ class GuideEnhancer:
|
||||
Args:
|
||||
mode: Enhancement mode - "api", "local", or "auto"
|
||||
"""
|
||||
self.mode = self._detect_mode(mode)
|
||||
self.api_key = os.environ.get("ANTHROPIC_API_KEY")
|
||||
self.client = None
|
||||
from skill_seekers.cli.agent_client import AgentClient
|
||||
|
||||
if self.mode == "api":
|
||||
if ANTHROPIC_AVAILABLE and self.api_key:
|
||||
# Support custom base_url for GLM-4.7 and other Claude-compatible APIs
|
||||
client_kwargs = {"api_key": self.api_key}
|
||||
base_url = os.environ.get("ANTHROPIC_BASE_URL")
|
||||
if base_url:
|
||||
client_kwargs["base_url"] = base_url
|
||||
logger.info(f"✅ Using custom API base URL: {base_url}")
|
||||
self.client = anthropic.Anthropic(**client_kwargs)
|
||||
logger.info("✨ GuideEnhancer initialized in API mode")
|
||||
else:
|
||||
logger.warning(
|
||||
"⚠️ API mode requested but anthropic library not available or no API key"
|
||||
)
|
||||
self.mode = "none"
|
||||
elif self.mode == "local":
|
||||
# Check if claude CLI is available
|
||||
if not self._check_claude_cli():
|
||||
logger.warning("⚠️ Claude CLI not found - falling back to API mode")
|
||||
self.mode = "api"
|
||||
if ANTHROPIC_AVAILABLE and self.api_key:
|
||||
# Support custom base_url for GLM-4.7 and other Claude-compatible APIs
|
||||
client_kwargs = {"api_key": self.api_key}
|
||||
base_url = os.environ.get("ANTHROPIC_BASE_URL")
|
||||
if base_url:
|
||||
client_kwargs["base_url"] = base_url
|
||||
logger.info(f"✅ Using custom API base URL: {base_url}")
|
||||
self.client = anthropic.Anthropic(**client_kwargs)
|
||||
else:
|
||||
logger.warning("⚠️ API fallback also unavailable")
|
||||
self.mode = "none"
|
||||
else:
|
||||
logger.info("✨ GuideEnhancer initialized in LOCAL mode")
|
||||
self._agent = AgentClient(mode=mode)
|
||||
self.mode = self._agent.mode
|
||||
|
||||
if self._agent.is_available():
|
||||
self._agent.log_mode()
|
||||
else:
|
||||
logger.warning("⚠️ No AI enhancement available (no API key or Claude CLI)")
|
||||
logger.warning("⚠️ No AI enhancement available")
|
||||
self.mode = "none"
|
||||
|
||||
def _detect_mode(self, requested_mode: str) -> str:
|
||||
"""
|
||||
Detect the best enhancement mode.
|
||||
|
||||
Args:
|
||||
requested_mode: User-requested mode
|
||||
|
||||
Returns:
|
||||
Detected mode: "api", "local", or "none"
|
||||
"""
|
||||
if requested_mode == "auto":
|
||||
# Prefer API if key available, else LOCAL
|
||||
if os.environ.get("ANTHROPIC_API_KEY") and ANTHROPIC_AVAILABLE:
|
||||
return "api"
|
||||
elif self._check_claude_cli():
|
||||
return "local"
|
||||
else:
|
||||
return "none"
|
||||
return requested_mode
|
||||
|
||||
def _check_claude_cli(self) -> bool:
|
||||
"""Check if Claude Code CLI is available."""
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["claude", "--version"], capture_output=True, text=True, timeout=5
|
||||
)
|
||||
return result.returncode == 0
|
||||
except (FileNotFoundError, subprocess.TimeoutExpired):
|
||||
return False
|
||||
|
||||
def enhance_guide(self, guide_data: dict) -> dict:
|
||||
"""
|
||||
Apply all 5 enhancements to a guide.
|
||||
@@ -332,7 +262,7 @@ class GuideEnhancer:
|
||||
|
||||
def _call_ai(self, prompt: str, max_tokens: int = 4000) -> str | None:
|
||||
"""
|
||||
Call AI with the given prompt.
|
||||
Call AI with the given prompt via AgentClient.
|
||||
|
||||
Args:
|
||||
prompt: Prompt text
|
||||
@@ -341,73 +271,7 @@ class GuideEnhancer:
|
||||
Returns:
|
||||
AI response text or None if failed
|
||||
"""
|
||||
if self.mode == "api":
|
||||
return self._call_claude_api(prompt, max_tokens)
|
||||
elif self.mode == "local":
|
||||
return self._call_claude_local(prompt)
|
||||
return None
|
||||
|
||||
def _call_claude_api(self, prompt: str, max_tokens: int = 4000) -> str | None:
|
||||
"""
|
||||
Call Claude API.
|
||||
|
||||
Args:
|
||||
prompt: Prompt text
|
||||
max_tokens: Maximum tokens in response
|
||||
|
||||
Returns:
|
||||
API response text or None if failed
|
||||
"""
|
||||
if not self.client:
|
||||
return None
|
||||
|
||||
try:
|
||||
response = self.client.messages.create(
|
||||
model="claude-sonnet-4-20250514",
|
||||
max_tokens=max_tokens,
|
||||
messages=[{"role": "user", "content": prompt}],
|
||||
)
|
||||
return response.content[0].text
|
||||
except Exception as e:
|
||||
logger.warning(f"⚠️ Claude API call failed: {e}")
|
||||
return None
|
||||
|
||||
def _call_claude_local(self, prompt: str) -> str | None:
|
||||
"""
|
||||
Call Claude Code CLI.
|
||||
|
||||
Args:
|
||||
prompt: Prompt text
|
||||
|
||||
Returns:
|
||||
CLI response text or None if failed
|
||||
"""
|
||||
try:
|
||||
# Create temporary prompt file
|
||||
with tempfile.NamedTemporaryFile(mode="w", suffix=".md", delete=False) as f:
|
||||
f.write(prompt)
|
||||
prompt_file = f.name
|
||||
|
||||
# Run claude CLI
|
||||
result = subprocess.run(
|
||||
["claude", prompt_file],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=300, # 5 min timeout
|
||||
)
|
||||
|
||||
# Clean up prompt file
|
||||
Path(prompt_file).unlink(missing_ok=True)
|
||||
|
||||
if result.returncode == 0:
|
||||
return result.stdout
|
||||
else:
|
||||
logger.warning(f"⚠️ Claude CLI failed: {result.stderr}")
|
||||
return None
|
||||
|
||||
except (subprocess.TimeoutExpired, Exception) as e:
|
||||
logger.warning(f"⚠️ Claude CLI execution failed: {e}")
|
||||
return None
|
||||
return self._agent.call(prompt, max_tokens=max_tokens)
|
||||
|
||||
# === Prompt Creation Methods ===
|
||||
|
||||
@@ -422,7 +286,7 @@ class GuideEnhancer:
|
||||
Enhanced guide data
|
||||
"""
|
||||
prompt = self._create_enhancement_prompt(guide_data)
|
||||
response = self._call_claude_api(prompt)
|
||||
response = self._call_ai(prompt)
|
||||
|
||||
if not response:
|
||||
return guide_data
|
||||
|
||||
@@ -1785,7 +1785,7 @@ def main() -> int:
|
||||
"0=disabled (default for HTML), 1=SKILL.md only, "
|
||||
"2=+architecture/config, 3=full enhancement. "
|
||||
"Mode selection: uses API if ANTHROPIC_API_KEY is set, "
|
||||
"otherwise LOCAL (Claude Code)"
|
||||
"otherwise LOCAL (Claude Code, Kimi, etc.)"
|
||||
)
|
||||
|
||||
# HTML-specific args
|
||||
@@ -1907,7 +1907,9 @@ def main() -> int:
|
||||
LocalSkillEnhancer,
|
||||
)
|
||||
|
||||
enhancer = LocalSkillEnhancer(Path(skill_dir))
|
||||
agent = getattr(args, "agent", None) if args else None
|
||||
agent_cmd = getattr(args, "agent_cmd", None) if args else None
|
||||
enhancer = LocalSkillEnhancer(Path(skill_dir), agent=agent, agent_cmd=agent_cmd)
|
||||
enhancer.run(headless=True)
|
||||
print("✅ Local enhancement complete!")
|
||||
else:
|
||||
@@ -1915,7 +1917,9 @@ def main() -> int:
|
||||
LocalSkillEnhancer,
|
||||
)
|
||||
|
||||
enhancer = LocalSkillEnhancer(Path(skill_dir))
|
||||
agent = getattr(args, "agent", None) if args else None
|
||||
agent_cmd = getattr(args, "agent_cmd", None) if args else None
|
||||
enhancer = LocalSkillEnhancer(Path(skill_dir), agent=agent, agent_cmd=agent_cmd)
|
||||
enhancer.run(headless=True)
|
||||
print("✅ Local enhancement complete!")
|
||||
|
||||
|
||||
@@ -301,6 +301,8 @@ def install_to_agent(
|
||||
msg += "Restart Cursor to load the new skill."
|
||||
elif agent_name.lower() in ["vscode", "copilot"]:
|
||||
msg += "Restart VS Code to load the new skill."
|
||||
elif agent_name.lower() == "kimi-code":
|
||||
msg += "Restart Kimi Code to load the new skill."
|
||||
else:
|
||||
msg += f"Restart {agent_name.capitalize()} to load the new skill."
|
||||
|
||||
|
||||
@@ -74,7 +74,7 @@ Examples:
|
||||
# Preview workflow (dry run)
|
||||
skill-seekers install --config react --dry-run
|
||||
|
||||
# Install for Gemini instead of Claude
|
||||
# Install for Gemini instead of default platform
|
||||
skill-seekers install --config react --target gemini
|
||||
|
||||
# Install for OpenAI ChatGPT
|
||||
@@ -107,7 +107,9 @@ Phases:
|
||||
help="Output directory for skill files (default: output/)",
|
||||
)
|
||||
|
||||
parser.add_argument("--no-upload", action="store_true", help="Skip automatic upload to Claude")
|
||||
parser.add_argument(
|
||||
"--no-upload", action="store_true", help="Skip automatic upload to target platform"
|
||||
)
|
||||
|
||||
parser.add_argument(
|
||||
"--unlimited",
|
||||
@@ -119,13 +121,19 @@ Phases:
|
||||
|
||||
parser.add_argument(
|
||||
"--target",
|
||||
choices=["claude", "gemini", "openai", "markdown"],
|
||||
default="claude",
|
||||
help="Target LLM platform (default: claude)",
|
||||
choices=["claude", "gemini", "openai", "kimi", "markdown"],
|
||||
default=None,
|
||||
help="Target LLM platform (auto-detected from API keys, or 'claude' if none set)",
|
||||
)
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
# Auto-detect target platform if not specified
|
||||
if args.target is None:
|
||||
from skill_seekers.cli.agent_client import AgentClient
|
||||
|
||||
args.target = AgentClient.detect_default_target()
|
||||
|
||||
# Determine if config is a name or path
|
||||
config_arg = args.config
|
||||
if config_arg.endswith(".json") or "/" in config_arg or "\\" in config_arg:
|
||||
|
||||
@@ -1182,13 +1182,17 @@ def main() -> int:
|
||||
print("❌ API enhancement not available. Falling back to LOCAL mode...")
|
||||
from skill_seekers.cli.enhance_skill_local import LocalSkillEnhancer
|
||||
|
||||
enhancer = LocalSkillEnhancer(Path(skill_dir))
|
||||
agent = getattr(args, "agent", None) if args else None
|
||||
agent_cmd = getattr(args, "agent_cmd", None) if args else None
|
||||
enhancer = LocalSkillEnhancer(Path(skill_dir), agent=agent, agent_cmd=agent_cmd)
|
||||
enhancer.run(headless=True)
|
||||
print("✅ Local enhancement complete!")
|
||||
else:
|
||||
from skill_seekers.cli.enhance_skill_local import LocalSkillEnhancer
|
||||
|
||||
enhancer = LocalSkillEnhancer(Path(skill_dir))
|
||||
agent = getattr(args, "agent", None) if args else None
|
||||
agent_cmd = getattr(args, "agent_cmd", None) if args else None
|
||||
enhancer = LocalSkillEnhancer(Path(skill_dir), agent=agent, agent_cmd=agent_cmd)
|
||||
enhancer.run(headless=True)
|
||||
print("✅ Local enhancement complete!")
|
||||
|
||||
|
||||
@@ -30,7 +30,7 @@ Commands:
|
||||
enhance AI-powered enhancement (auto: API or LOCAL mode)
|
||||
enhance-status Check enhancement status (for background/daemon modes)
|
||||
package Package skill into .zip file
|
||||
upload Upload skill to Claude
|
||||
upload Upload skill to target platform
|
||||
estimate Estimate page count before scraping
|
||||
extract-test-examples Extract usage examples from test files
|
||||
install-agent Install skill to AI agent directories
|
||||
@@ -47,6 +47,7 @@ Examples:
|
||||
|
||||
import argparse
|
||||
import importlib
|
||||
import os
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
@@ -101,7 +102,7 @@ def create_parser() -> argparse.ArgumentParser:
|
||||
|
||||
parser = argparse.ArgumentParser(
|
||||
prog="skill-seekers",
|
||||
description="Convert documentation, GitHub repos, and PDFs into Claude AI skills",
|
||||
description="Convert documentation, GitHub repos, and PDFs into AI skills",
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter,
|
||||
epilog="""
|
||||
Examples:
|
||||
@@ -391,7 +392,7 @@ def _handle_analyze_command(args: argparse.Namespace) -> int:
|
||||
background=False,
|
||||
daemon=False,
|
||||
no_force=False,
|
||||
timeout=600,
|
||||
timeout=2700,
|
||||
)
|
||||
_mode, _target = _pick_mode(_fake_args)
|
||||
|
||||
@@ -403,7 +404,10 @@ def _handle_analyze_command(args: argparse.Namespace) -> int:
|
||||
print(" Set ANTHROPIC_API_KEY / GOOGLE_API_KEY to enable API mode")
|
||||
success = False
|
||||
else:
|
||||
print("\n🤖 Enhancement mode: LOCAL (Claude Code CLI)")
|
||||
agent_name = (
|
||||
os.environ.get("SKILL_SEEKER_AGENT", "claude").strip() or "claude"
|
||||
)
|
||||
print(f"\n🤖 Enhancement mode: LOCAL ({agent_name})")
|
||||
success = _run_local_mode(_fake_args) == 0
|
||||
|
||||
if success:
|
||||
|
||||
@@ -1332,7 +1332,7 @@ def main() -> int:
|
||||
"0=disabled (default for man), 1=SKILL.md only, "
|
||||
"2=+architecture/config, 3=full enhancement. "
|
||||
"Mode selection: uses API if ANTHROPIC_API_KEY is set, "
|
||||
"otherwise LOCAL (Claude Code)"
|
||||
"otherwise LOCAL (Claude Code, Kimi, etc.)"
|
||||
)
|
||||
|
||||
# Man-specific arguments
|
||||
@@ -1486,13 +1486,17 @@ def main() -> int:
|
||||
print("❌ API enhancement not available. Falling back to LOCAL mode...")
|
||||
from skill_seekers.cli.enhance_skill_local import LocalSkillEnhancer
|
||||
|
||||
enhancer = LocalSkillEnhancer(Path(skill_dir))
|
||||
agent = getattr(args, "agent", None) if args else None
|
||||
agent_cmd = getattr(args, "agent_cmd", None) if args else None
|
||||
enhancer = LocalSkillEnhancer(Path(skill_dir), agent=agent, agent_cmd=agent_cmd)
|
||||
enhancer.run(headless=True)
|
||||
print("✅ Local enhancement complete!")
|
||||
else:
|
||||
from skill_seekers.cli.enhance_skill_local import LocalSkillEnhancer
|
||||
|
||||
enhancer = LocalSkillEnhancer(Path(skill_dir))
|
||||
agent = getattr(args, "agent", None) if args else None
|
||||
agent_cmd = getattr(args, "agent_cmd", None) if args else None
|
||||
enhancer = LocalSkillEnhancer(Path(skill_dir), agent=agent, agent_cmd=agent_cmd)
|
||||
enhancer.run(headless=True)
|
||||
print("✅ Local enhancement complete!")
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@ Source Merger for Multi-Source Skills
|
||||
|
||||
Merges documentation and code data intelligently with GitHub insights:
|
||||
- Rule-based merge: Fast, deterministic rules
|
||||
- Claude-enhanced merge: AI-powered reconciliation
|
||||
- AI-enhanced merge: AI-powered reconciliation
|
||||
|
||||
Handles conflicts and creates unified API reference with GitHub metadata.
|
||||
|
||||
@@ -18,7 +18,6 @@ Multi-layer architecture (Phase 3):
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import subprocess
|
||||
import tempfile
|
||||
from typing import Any, Optional
|
||||
|
||||
@@ -440,11 +439,11 @@ class RuleBasedMerger:
|
||||
return signature
|
||||
|
||||
|
||||
class ClaudeEnhancedMerger:
|
||||
class AIEnhancedMerger:
|
||||
"""
|
||||
Claude-enhanced API merger using local Claude Code with GitHub insights.
|
||||
AI-enhanced API merger using local AI coding agent with GitHub insights.
|
||||
|
||||
Opens Claude Code in a new terminal to intelligently reconcile conflicts.
|
||||
Uses the configured AI agent to intelligently reconcile conflicts.
|
||||
Uses the same approach as enhance_skill_local.py.
|
||||
|
||||
Multi-layer architecture (Phase 3):
|
||||
@@ -462,7 +461,7 @@ class ClaudeEnhancedMerger:
|
||||
github_streams: Optional["ThreeStreamData"] = None,
|
||||
):
|
||||
"""
|
||||
Initialize Claude-enhanced merger with GitHub streams support.
|
||||
Initialize AI-enhanced merger with GitHub streams support.
|
||||
|
||||
Args:
|
||||
docs_data: Documentation scraper data (Layer 2: HTML docs)
|
||||
@@ -480,31 +479,31 @@ class ClaudeEnhancedMerger:
|
||||
|
||||
def merge_all(self) -> dict[str, Any]:
|
||||
"""
|
||||
Merge all APIs using Claude enhancement.
|
||||
Merge all APIs using AI enhancement.
|
||||
|
||||
Returns:
|
||||
Dict containing merged API data
|
||||
"""
|
||||
logger.info("Starting Claude-enhanced merge...")
|
||||
logger.info("Starting AI-enhanced merge...")
|
||||
|
||||
# Create temporary workspace
|
||||
workspace_dir = self._create_workspace()
|
||||
|
||||
# Launch Claude Code for enhancement
|
||||
logger.info("Launching Claude Code for intelligent merging...")
|
||||
logger.info("Claude will analyze conflicts and create reconciled API reference")
|
||||
# Launch AI agent for enhancement
|
||||
logger.info("Launching AI agent for intelligent merging...")
|
||||
logger.info("AI will analyze conflicts and create reconciled API reference")
|
||||
|
||||
try:
|
||||
self._launch_claude_merge(workspace_dir)
|
||||
self._launch_ai_merge(workspace_dir)
|
||||
|
||||
# Read enhanced results
|
||||
merged_data = self._read_merged_results(workspace_dir)
|
||||
|
||||
logger.info("Claude-enhanced merge complete")
|
||||
logger.info("AI-enhanced merge complete")
|
||||
return merged_data
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Claude enhancement failed: {e}")
|
||||
logger.error(f"AI enhancement failed: {e}")
|
||||
logger.info("Falling back to rule-based merge")
|
||||
return self.rule_merger.merge_all()
|
||||
|
||||
@@ -518,13 +517,13 @@ class ClaudeEnhancedMerger:
|
||||
workspace = tempfile.mkdtemp(prefix="skill_merge_")
|
||||
logger.info(f"Created merge workspace: {workspace}")
|
||||
|
||||
# Write context files for Claude
|
||||
# Write context files for AI agent
|
||||
self._write_context_files(workspace)
|
||||
|
||||
return workspace
|
||||
|
||||
def _write_context_files(self, workspace: str):
|
||||
"""Write context files for Claude to analyze."""
|
||||
"""Write context files for AI agent to analyze."""
|
||||
|
||||
# 1. Write conflicts summary
|
||||
conflicts_file = os.path.join(workspace, "conflicts.json")
|
||||
@@ -553,7 +552,7 @@ class ClaudeEnhancedMerger:
|
||||
with open(code_apis_file, "w") as f:
|
||||
json.dump(detector.code_apis, f, indent=2)
|
||||
|
||||
# 4. Write merge instructions for Claude
|
||||
# 4. Write merge instructions for AI agent
|
||||
instructions = """# API Merge Task
|
||||
|
||||
You are merging API documentation from two sources:
|
||||
@@ -625,75 +624,79 @@ Take your time to analyze each conflict carefully. The goal is to create the mos
|
||||
counts[value] = counts.get(value, 0) + 1
|
||||
return counts
|
||||
|
||||
def _launch_claude_merge(self, workspace: str):
|
||||
def _launch_ai_merge(self, workspace: str):
|
||||
"""
|
||||
Launch Claude Code to perform merge.
|
||||
|
||||
Similar to enhance_skill_local.py approach.
|
||||
Run AI-enhanced merge via AgentClient (automated, no terminal).
|
||||
"""
|
||||
# Create a script that Claude will execute
|
||||
script_path = os.path.join(workspace, "merge_script.sh")
|
||||
from skill_seekers.cli.agent_client import AgentClient
|
||||
|
||||
script_content = f"""#!/bin/bash
|
||||
# Automatic merge script for Claude Code
|
||||
|
||||
cd "{workspace}"
|
||||
|
||||
echo "📊 Analyzing conflicts..."
|
||||
cat conflicts.json | head -20
|
||||
|
||||
echo ""
|
||||
echo "📖 Documentation APIs: $(cat docs_apis.json | grep -c '\"name\"')"
|
||||
echo "💻 Code APIs: $(cat code_apis.json | grep -c '\"name\"')"
|
||||
echo ""
|
||||
echo "Please review the conflicts and create merged_apis.json"
|
||||
echo "Follow the instructions in MERGE_INSTRUCTIONS.md"
|
||||
echo ""
|
||||
echo "When done, save merged_apis.json and close this terminal."
|
||||
|
||||
# Wait for user to complete merge
|
||||
read -p "Press Enter when merge is complete..."
|
||||
"""
|
||||
|
||||
with open(script_path, "w") as f:
|
||||
f.write(script_content)
|
||||
|
||||
os.chmod(script_path, 0o755)
|
||||
|
||||
# Open new terminal with Claude Code
|
||||
# Try different terminal emulators
|
||||
terminals = [
|
||||
["x-terminal-emulator", "-e"],
|
||||
["gnome-terminal", "--"],
|
||||
["xterm", "-e"],
|
||||
["konsole", "-e"],
|
||||
]
|
||||
|
||||
for terminal_cmd in terminals:
|
||||
try:
|
||||
cmd = terminal_cmd + ["bash", script_path]
|
||||
subprocess.Popen(cmd)
|
||||
logger.info(f"Opened terminal with {terminal_cmd[0]}")
|
||||
break
|
||||
except FileNotFoundError:
|
||||
continue
|
||||
|
||||
# Wait for merge to complete
|
||||
# Read context files to build prompt
|
||||
conflicts_file = os.path.join(workspace, "conflicts.json")
|
||||
docs_apis_file = os.path.join(workspace, "docs_apis.json")
|
||||
code_apis_file = os.path.join(workspace, "code_apis.json")
|
||||
instructions_file = os.path.join(workspace, "MERGE_INSTRUCTIONS.md")
|
||||
merged_file = os.path.join(workspace, "merged_apis.json")
|
||||
logger.info(f"Waiting for merged results at: {merged_file}")
|
||||
logger.info("Close the terminal when done to continue...")
|
||||
|
||||
# Poll for file existence
|
||||
import time
|
||||
with open(instructions_file) as f:
|
||||
instructions = f.read()
|
||||
|
||||
timeout = 3600 # 1 hour max
|
||||
elapsed = 0
|
||||
while not os.path.exists(merged_file) and elapsed < timeout:
|
||||
time.sleep(5)
|
||||
elapsed += 5
|
||||
with open(conflicts_file) as f:
|
||||
conflicts_data = f.read()
|
||||
|
||||
if not os.path.exists(merged_file):
|
||||
raise TimeoutError("Claude merge timed out after 1 hour")
|
||||
# Limit conflict data to avoid token overflow
|
||||
if len(conflicts_data) > 30000:
|
||||
conflicts_data = conflicts_data[:30000] + "\n... (truncated)"
|
||||
|
||||
with open(docs_apis_file) as f:
|
||||
docs_apis = f.read()
|
||||
if len(docs_apis) > 15000:
|
||||
docs_apis = docs_apis[:15000] + "\n... (truncated)"
|
||||
|
||||
with open(code_apis_file) as f:
|
||||
code_apis = f.read()
|
||||
if len(code_apis) > 15000:
|
||||
code_apis = code_apis[:15000] + "\n... (truncated)"
|
||||
|
||||
prompt = f"""{instructions}
|
||||
|
||||
## Conflicts Data:
|
||||
{conflicts_data}
|
||||
|
||||
## Documentation APIs:
|
||||
{docs_apis}
|
||||
|
||||
## Code APIs:
|
||||
{code_apis}
|
||||
|
||||
Write the merged_apis.json output as valid JSON following the format in the instructions above.
|
||||
Return ONLY the JSON, no explanation."""
|
||||
|
||||
client = AgentClient(mode="auto")
|
||||
logger.info(f"Running AI merge via {client.agent_display}...")
|
||||
|
||||
response = client.call(prompt, max_tokens=8192)
|
||||
|
||||
if response:
|
||||
# Try to extract JSON from response
|
||||
import re
|
||||
|
||||
json_match = re.search(r"\{[\s\S]*\}", response)
|
||||
if json_match:
|
||||
try:
|
||||
merged = json.loads(json_match.group())
|
||||
with open(merged_file, "w") as f:
|
||||
json.dump(merged, f, indent=2)
|
||||
logger.info("✅ AI merge complete — merged_apis.json written")
|
||||
return
|
||||
except json.JSONDecodeError:
|
||||
logger.warning("⚠️ Could not parse JSON from AI response")
|
||||
|
||||
# Fallback: write raw response
|
||||
with open(merged_file, "w") as f:
|
||||
f.write(response)
|
||||
logger.info("✅ AI merge complete (raw response saved)")
|
||||
else:
|
||||
raise RuntimeError("AI agent returned no response for merge")
|
||||
|
||||
def _read_merged_results(self, workspace: str) -> dict[str, Any]:
|
||||
"""Read merged results from workspace."""
|
||||
@@ -804,3 +807,7 @@ if __name__ == "__main__":
|
||||
print(f" Code only: {summary.get('code_only', 0)}")
|
||||
print(f" Conflicts: {summary.get('conflict', 0)}")
|
||||
print(f"\n📄 Saved to: {args.output}")
|
||||
|
||||
|
||||
# Backward compatibility alias
|
||||
ClaudeEnhancedMerger = AIEnhancedMerger
|
||||
|
||||
@@ -1002,11 +1002,17 @@ def main() -> int:
|
||||
except ImportError:
|
||||
from skill_seekers.cli.enhance_skill_local import LocalSkillEnhancer
|
||||
|
||||
LocalSkillEnhancer(Path(skill_dir)).run(headless=True)
|
||||
agent = getattr(args, "agent", None) if args else None
|
||||
agent_cmd = getattr(args, "agent_cmd", None) if args else None
|
||||
enhancer = LocalSkillEnhancer(Path(skill_dir), agent=agent, agent_cmd=agent_cmd)
|
||||
enhancer.run(headless=True)
|
||||
else:
|
||||
from skill_seekers.cli.enhance_skill_local import LocalSkillEnhancer
|
||||
|
||||
LocalSkillEnhancer(Path(skill_dir)).run(headless=True)
|
||||
agent = getattr(args, "agent", None) if args else None
|
||||
agent_cmd = getattr(args, "agent_cmd", None) if args else None
|
||||
enhancer = LocalSkillEnhancer(Path(skill_dir), agent=agent, agent_cmd=agent_cmd)
|
||||
enhancer.run(headless=True)
|
||||
except RuntimeError as e:
|
||||
print(f"\n Error: {e}", file=sys.stderr)
|
||||
sys.exit(1) # noqa: E702
|
||||
|
||||
@@ -1817,7 +1817,7 @@ Examples:
|
||||
"0=disabled (default for OpenAPI), 1=SKILL.md only, "
|
||||
"2=+architecture/config, 3=full enhancement. "
|
||||
"Mode selection: uses API if ANTHROPIC_API_KEY is set, "
|
||||
"otherwise LOCAL (Claude Code)"
|
||||
"otherwise LOCAL (Claude Code, Kimi, etc.)"
|
||||
)
|
||||
|
||||
# OpenAPI-specific arguments
|
||||
@@ -1932,13 +1932,17 @@ Examples:
|
||||
print(" API enhancement not available. Falling back to LOCAL mode...")
|
||||
from skill_seekers.cli.enhance_skill_local import LocalSkillEnhancer
|
||||
|
||||
enhancer = LocalSkillEnhancer(Path(skill_dir))
|
||||
agent = getattr(args, "agent", None) if args else None
|
||||
agent_cmd = getattr(args, "agent_cmd", None) if args else None
|
||||
enhancer = LocalSkillEnhancer(Path(skill_dir), agent=agent, agent_cmd=agent_cmd)
|
||||
enhancer.run(headless=True)
|
||||
print(" Local enhancement complete!")
|
||||
else:
|
||||
from skill_seekers.cli.enhance_skill_local import LocalSkillEnhancer
|
||||
|
||||
enhancer = LocalSkillEnhancer(Path(skill_dir))
|
||||
agent = getattr(args, "agent", None) if args else None
|
||||
agent_cmd = getattr(args, "agent_cmd", None) if args else None
|
||||
enhancer = LocalSkillEnhancer(Path(skill_dir), agent=agent, agent_cmd=agent_cmd)
|
||||
enhancer.run(headless=True)
|
||||
print(" Local enhancement complete!")
|
||||
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Simple Skill Packager
|
||||
Packages a skill directory into a .zip file for Claude.
|
||||
Packages a skill directory into a .zip file for LLM platforms.
|
||||
|
||||
Usage:
|
||||
skill-seekers package output/steam-inventory/
|
||||
@@ -197,7 +197,7 @@ def main():
|
||||
from skill_seekers.cli.arguments.package import add_package_arguments
|
||||
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Package a skill directory into a .zip file for Claude",
|
||||
description="Package a skill directory into a .zip file for LLM platforms",
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter,
|
||||
epilog="""
|
||||
Examples:
|
||||
@@ -210,7 +210,7 @@ Examples:
|
||||
# Skip quality checks (faster, but not recommended)
|
||||
skill-seekers package output/react/ --skip-quality-check
|
||||
|
||||
# Package and auto-upload to Claude
|
||||
# Package and auto-upload to target platform
|
||||
skill-seekers package output/react/ --upload
|
||||
|
||||
# Get help
|
||||
@@ -295,6 +295,31 @@ Examples:
|
||||
print(f"\n❌ Upload error: {e}")
|
||||
sys.exit(1)
|
||||
|
||||
# Publish to marketplace if requested
|
||||
marketplace_name = getattr(args, "marketplace", None)
|
||||
if marketplace_name:
|
||||
try:
|
||||
from skill_seekers.mcp.marketplace_publisher import MarketplacePublisher
|
||||
|
||||
publisher = MarketplacePublisher()
|
||||
pub_result = publisher.publish(
|
||||
skill_dir=args.skill_directory,
|
||||
marketplace_name=marketplace_name,
|
||||
category=getattr(args, "marketplace_category", "development"),
|
||||
create_branch=getattr(args, "create_branch", False),
|
||||
force=True,
|
||||
)
|
||||
if pub_result["success"]:
|
||||
print(f"\n✅ {pub_result['message']}")
|
||||
print(f" Plugin: {pub_result['plugin_path']}")
|
||||
print(f" Branch: {pub_result['branch']}")
|
||||
print(f" Commit: {pub_result['commit_sha']}")
|
||||
else:
|
||||
print(f"\n⚠️ Marketplace publish failed: {pub_result['message']}")
|
||||
except Exception as e:
|
||||
print(f"\n⚠️ Marketplace publish failed: {e}")
|
||||
print(" Packaging was successful — publish manually later.")
|
||||
|
||||
sys.exit(0)
|
||||
|
||||
|
||||
|
||||
@@ -23,8 +23,8 @@ class EnhanceParser(SubcommandParser):
|
||||
def description(self) -> str:
|
||||
return (
|
||||
"Enhance SKILL.md using AI. "
|
||||
"Automatically uses API mode (Gemini/OpenAI/Claude) when an API key is "
|
||||
"available, or falls back to LOCAL mode (Claude Code CLI)."
|
||||
"Automatically uses API mode (Anthropic/Gemini/OpenAI) when an API key is "
|
||||
"available, or falls back to LOCAL mode (coding agent CLI)."
|
||||
)
|
||||
|
||||
def add_arguments(self, parser):
|
||||
|
||||
@@ -24,7 +24,7 @@ class InstallAgentParser(SubcommandParser):
|
||||
parser.add_argument(
|
||||
"--agent",
|
||||
required=True,
|
||||
help="Agent name (claude, cursor, vscode, amp, goose, opencode, all)",
|
||||
help="Agent name (claude, cursor, vscode, amp, goose, opencode, kimi-code, all)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--force", action="store_true", help="Overwrite existing installation without asking"
|
||||
|
||||
@@ -29,7 +29,7 @@ class InstallParser(SubcommandParser):
|
||||
"--destination", default="output", help="Output directory (default: output/)"
|
||||
)
|
||||
parser.add_argument(
|
||||
"--no-upload", action="store_true", help="Skip automatic upload to Claude"
|
||||
"--no-upload", action="store_true", help="Skip automatic upload to target platform"
|
||||
)
|
||||
parser.add_argument(
|
||||
"--unlimited", action="store_true", help="Remove page limits during scraping"
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
PDF Documentation to Claude Skill Converter (Task B1.6)
|
||||
PDF Documentation to AI Skill Converter (Task B1.6)
|
||||
|
||||
Converts PDF documentation into Claude AI skills.
|
||||
Converts PDF documentation into AI skills.
|
||||
Uses pdf_extractor_poc.py for extraction, builds skill structure.
|
||||
|
||||
Usage:
|
||||
@@ -63,7 +63,7 @@ def infer_description_from_pdf(pdf_metadata: dict = None, name: str = "") -> str
|
||||
|
||||
|
||||
class PDFToSkillConverter:
|
||||
"""Convert PDF documentation to Claude skill"""
|
||||
"""Convert PDF documentation to AI skill"""
|
||||
|
||||
def __init__(self, config):
|
||||
self.config = config
|
||||
@@ -637,7 +637,7 @@ def main():
|
||||
from .arguments.pdf import add_pdf_arguments
|
||||
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Convert PDF documentation to Claude skill",
|
||||
description="Convert PDF documentation to AI skill",
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter,
|
||||
)
|
||||
|
||||
@@ -749,16 +749,22 @@ def main():
|
||||
from pathlib import Path
|
||||
from skill_seekers.cli.enhance_skill_local import LocalSkillEnhancer
|
||||
|
||||
enhancer = LocalSkillEnhancer(Path(skill_dir))
|
||||
agent = getattr(args, "agent", None) if args else None
|
||||
agent_cmd = getattr(args, "agent_cmd", None) if args else None
|
||||
enhancer = LocalSkillEnhancer(Path(skill_dir), agent=agent, agent_cmd=agent_cmd)
|
||||
enhancer.run(headless=True)
|
||||
print("✅ Local enhancement complete!")
|
||||
agent_name = agent or "claude"
|
||||
print(f"✅ Local enhancement complete! (via {agent_name})")
|
||||
else:
|
||||
from pathlib import Path
|
||||
from skill_seekers.cli.enhance_skill_local import LocalSkillEnhancer
|
||||
|
||||
enhancer = LocalSkillEnhancer(Path(skill_dir))
|
||||
agent = getattr(args, "agent", None) if args else None
|
||||
agent_cmd = getattr(args, "agent_cmd", None) if args else None
|
||||
enhancer = LocalSkillEnhancer(Path(skill_dir), agent=agent, agent_cmd=agent_cmd)
|
||||
enhancer.run(headless=True)
|
||||
print("✅ Local enhancement complete!")
|
||||
agent_name = agent or "claude"
|
||||
print(f"✅ Local enhancement complete! (via {agent_name})")
|
||||
|
||||
except RuntimeError as e:
|
||||
print(f"\n❌ Error: {e}", file=sys.stderr)
|
||||
|
||||
@@ -1788,13 +1788,17 @@ def main() -> int:
|
||||
print("❌ API enhancement not available. Falling back to LOCAL mode...")
|
||||
from skill_seekers.cli.enhance_skill_local import LocalSkillEnhancer
|
||||
|
||||
enhancer = LocalSkillEnhancer(Path(skill_dir))
|
||||
agent = getattr(args, "agent", None) if args else None
|
||||
agent_cmd = getattr(args, "agent_cmd", None) if args else None
|
||||
enhancer = LocalSkillEnhancer(Path(skill_dir), agent=agent, agent_cmd=agent_cmd)
|
||||
enhancer.run(headless=True)
|
||||
print("✅ Local enhancement complete!")
|
||||
else:
|
||||
from skill_seekers.cli.enhance_skill_local import LocalSkillEnhancer
|
||||
|
||||
enhancer = LocalSkillEnhancer(Path(skill_dir))
|
||||
agent = getattr(args, "agent", None) if args else None
|
||||
agent_cmd = getattr(args, "agent_cmd", None) if args else None
|
||||
enhancer = LocalSkillEnhancer(Path(skill_dir), agent=agent, agent_cmd=agent_cmd)
|
||||
enhancer.run(headless=True)
|
||||
print("✅ Local enhancement complete!")
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Quality Checker for Claude Skills
|
||||
Quality Checker for AI Skills
|
||||
Validates skill quality, checks links, and generates quality reports.
|
||||
|
||||
Usage:
|
||||
@@ -350,7 +350,7 @@ class SkillQualityChecker:
|
||||
else:
|
||||
self.report.add_info(
|
||||
"completeness",
|
||||
"Consider adding prerequisites section - helps Claude verify conditions first",
|
||||
"Consider adding prerequisites section - helps the LLM verify conditions first",
|
||||
"SKILL.md",
|
||||
)
|
||||
|
||||
|
||||
@@ -900,7 +900,7 @@ def main() -> int:
|
||||
"0=disabled (default for RSS), 1=SKILL.md only, "
|
||||
"2=+architecture/config, 3=full enhancement. "
|
||||
"Mode selection: uses API if ANTHROPIC_API_KEY is set, "
|
||||
"otherwise LOCAL (Claude Code)"
|
||||
"otherwise LOCAL (Claude Code, Kimi, etc.)"
|
||||
)
|
||||
|
||||
# RSS-specific arguments
|
||||
@@ -1060,13 +1060,17 @@ def main() -> int:
|
||||
print("❌ API enhancement not available. Falling back to LOCAL mode...")
|
||||
from skill_seekers.cli.enhance_skill_local import LocalSkillEnhancer
|
||||
|
||||
enhancer = LocalSkillEnhancer(Path(skill_dir))
|
||||
agent = getattr(args, "agent", None) if args else None
|
||||
agent_cmd = getattr(args, "agent_cmd", None) if args else None
|
||||
enhancer = LocalSkillEnhancer(Path(skill_dir), agent=agent, agent_cmd=agent_cmd)
|
||||
enhancer.run(headless=True)
|
||||
print("✅ Local enhancement complete!")
|
||||
else:
|
||||
from skill_seekers.cli.enhance_skill_local import LocalSkillEnhancer
|
||||
|
||||
enhancer = LocalSkillEnhancer(Path(skill_dir))
|
||||
agent = getattr(args, "agent", None) if args else None
|
||||
agent_cmd = getattr(args, "agent_cmd", None) if args else None
|
||||
enhancer = LocalSkillEnhancer(Path(skill_dir), agent=agent, agent_cmd=agent_cmd)
|
||||
enhancer.run(headless=True)
|
||||
print("✅ Local enhancement complete!")
|
||||
|
||||
|
||||
@@ -268,7 +268,7 @@ class SignalFlowAnalyzer:
|
||||
|
||||
Args:
|
||||
output_dir: Directory to save guides
|
||||
ai_mode: "LOCAL" (Claude Code) or "API" (Anthropic API)
|
||||
ai_mode: "LOCAL" (coding agent CLI) or "API" (Anthropic API)
|
||||
|
||||
Returns:
|
||||
Path to generated guide file
|
||||
|
||||
@@ -319,7 +319,7 @@ class UnifiedCodebaseAnalyzer:
|
||||
c3x_data = {}
|
||||
|
||||
# C3.1: Design Patterns
|
||||
patterns_file = output_dir / "patterns" / "design_patterns.json"
|
||||
patterns_file = output_dir / "patterns" / "all_patterns.json"
|
||||
if patterns_file.exists():
|
||||
with open(patterns_file) as f:
|
||||
patterns_data = json.load(f)
|
||||
|
||||
@@ -19,12 +19,8 @@ Benefits:
|
||||
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import subprocess
|
||||
import tempfile
|
||||
from concurrent.futures import ThreadPoolExecutor, as_completed
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Literal
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
@@ -89,58 +85,24 @@ class UnifiedEnhancer:
|
||||
self.config.batch_size = cfg.get_local_batch_size()
|
||||
self.config.parallel_workers = cfg.get_local_parallel_workers()
|
||||
|
||||
# Determine actual mode
|
||||
self.api_key = self.config.api_key or os.environ.get("ANTHROPIC_API_KEY")
|
||||
# Initialize AgentClient for AI calls
|
||||
from skill_seekers.cli.agent_client import AgentClient
|
||||
|
||||
if self.config.mode == "auto":
|
||||
if self.api_key:
|
||||
self.config.mode = "api"
|
||||
self._agent = AgentClient(
|
||||
mode=self.config.mode,
|
||||
api_key=self.config.api_key,
|
||||
)
|
||||
|
||||
# Sync resolved mode back to config
|
||||
self.config.mode = self._agent.mode
|
||||
|
||||
if self.config.enabled:
|
||||
if self._agent.is_available():
|
||||
self._agent.log_mode()
|
||||
else:
|
||||
self.config.mode = "local"
|
||||
logger.info("ℹ️ No API key found, using LOCAL mode (Claude Code CLI)")
|
||||
|
||||
# Initialize API client if needed
|
||||
self.client = None
|
||||
if self.config.mode == "api" and self.config.enabled:
|
||||
try:
|
||||
import anthropic
|
||||
|
||||
client_kwargs = {"api_key": self.api_key}
|
||||
base_url = os.environ.get("ANTHROPIC_BASE_URL")
|
||||
if base_url:
|
||||
client_kwargs["base_url"] = base_url
|
||||
logger.info(f"✅ Using custom API base URL: {base_url}")
|
||||
self.client = anthropic.Anthropic(**client_kwargs)
|
||||
logger.info("✅ AI enhancement enabled (using Claude API)")
|
||||
except ImportError:
|
||||
logger.warning("⚠️ anthropic package not installed, falling back to LOCAL mode")
|
||||
self.config.mode = "local"
|
||||
except Exception as e:
|
||||
logger.warning(
|
||||
f"⚠️ Failed to initialize API client: {e}, falling back to LOCAL mode"
|
||||
)
|
||||
self.config.mode = "local"
|
||||
|
||||
if self.config.mode == "local" and self.config.enabled:
|
||||
if self._check_claude_cli():
|
||||
logger.info("✅ AI enhancement enabled (using LOCAL mode - Claude Code CLI)")
|
||||
else:
|
||||
logger.warning("⚠️ Claude Code CLI not found. AI enhancement disabled.")
|
||||
logger.warning("⚠️ AI agent not available. AI enhancement disabled.")
|
||||
self.config.enabled = False
|
||||
|
||||
def _check_claude_cli(self) -> bool:
|
||||
"""Check if Claude Code CLI is available."""
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["claude", "--version"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=5,
|
||||
)
|
||||
return result.returncode == 0
|
||||
except (FileNotFoundError, subprocess.TimeoutExpired):
|
||||
return False
|
||||
|
||||
def enhance(
|
||||
self,
|
||||
items: list[dict],
|
||||
@@ -265,88 +227,11 @@ class UnifiedEnhancer:
|
||||
return items
|
||||
|
||||
def _call_claude(self, prompt: str, max_tokens: int = 1000) -> str | None:
|
||||
"""Call Claude (API or LOCAL mode) with error handling."""
|
||||
if self.config.mode == "api":
|
||||
return self._call_claude_api(prompt, max_tokens)
|
||||
elif self.config.mode == "local":
|
||||
return self._call_claude_local(prompt)
|
||||
return None
|
||||
"""Call AI agent (API or LOCAL mode) via AgentClient."""
|
||||
from skill_seekers.cli.agent_client import get_default_timeout
|
||||
|
||||
def _call_claude_api(self, prompt: str, max_tokens: int = 1000) -> str | None:
|
||||
"""Call Claude API."""
|
||||
if not self.client:
|
||||
return None
|
||||
|
||||
try:
|
||||
response = self.client.messages.create(
|
||||
model="claude-sonnet-4-20250514",
|
||||
max_tokens=max_tokens,
|
||||
messages=[{"role": "user", "content": prompt}],
|
||||
)
|
||||
return response.content[0].text
|
||||
except Exception as e:
|
||||
logger.warning(f"⚠️ API call failed: {e}")
|
||||
return None
|
||||
|
||||
def _call_claude_local(self, prompt: str) -> str | None:
|
||||
"""Call Claude Code CLI in LOCAL mode."""
|
||||
try:
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
temp_path = Path(temp_dir)
|
||||
|
||||
# Write prompt to file
|
||||
prompt_file = temp_path / "prompt.txt"
|
||||
prompt_file.write_text(prompt)
|
||||
|
||||
# Output file
|
||||
output_file = temp_path / "response.json"
|
||||
|
||||
# Call Claude CLI
|
||||
result = subprocess.run(
|
||||
[
|
||||
"claude",
|
||||
str(prompt_file),
|
||||
"--output",
|
||||
str(output_file),
|
||||
"--model",
|
||||
"sonnet",
|
||||
],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=120,
|
||||
cwd=str(temp_path),
|
||||
)
|
||||
|
||||
if result.returncode != 0:
|
||||
logger.warning(f"⚠️ Claude CLI returned error: {result.returncode}")
|
||||
return None
|
||||
|
||||
# Read output
|
||||
if output_file.exists():
|
||||
response_text = output_file.read_text()
|
||||
try:
|
||||
json.loads(response_text)
|
||||
return response_text
|
||||
except json.JSONDecodeError:
|
||||
# Try to extract JSON
|
||||
import re
|
||||
|
||||
json_match = re.search(r"\[[\s\S]*\]|\{[\s\S]*\}", response_text)
|
||||
if json_match:
|
||||
return json_match.group()
|
||||
return None
|
||||
else:
|
||||
for json_file in temp_path.glob("*.json"):
|
||||
if json_file.name != "prompt.json":
|
||||
return json_file.read_text()
|
||||
return None
|
||||
|
||||
except subprocess.TimeoutExpired:
|
||||
logger.warning("⚠️ Claude CLI timeout (2 minutes)")
|
||||
return None
|
||||
except Exception as e:
|
||||
logger.warning(f"⚠️ LOCAL mode error: {e}")
|
||||
return None
|
||||
timeout = get_default_timeout() if self._agent.mode == "local" else None
|
||||
return self._agent.call(prompt, max_tokens=max_tokens, timeout=timeout)
|
||||
|
||||
def _get_default_prompt(self, enhancement_type: str) -> str:
|
||||
"""Get default prompt for enhancement type."""
|
||||
|
||||
@@ -9,7 +9,7 @@ This is the main entry point for unified config workflow.
|
||||
|
||||
Usage:
|
||||
skill-seekers unified --config configs/godot_unified.json
|
||||
skill-seekers unified --config configs/react_unified.json --merge-mode claude-enhanced
|
||||
skill-seekers unified --config configs/react_unified.json --merge-mode ai-enhanced
|
||||
"""
|
||||
|
||||
import argparse
|
||||
@@ -24,9 +24,10 @@ from typing import Any
|
||||
|
||||
# Import validators and scrapers
|
||||
try:
|
||||
from skill_seekers.cli.agent_client import get_default_timeout
|
||||
from skill_seekers.cli.config_validator import validate_config
|
||||
from skill_seekers.cli.conflict_detector import ConflictDetector
|
||||
from skill_seekers.cli.merge_sources import ClaudeEnhancedMerger, RuleBasedMerger
|
||||
from skill_seekers.cli.merge_sources import AIEnhancedMerger, RuleBasedMerger
|
||||
from skill_seekers.cli.unified_skill_builder import UnifiedSkillBuilder
|
||||
from skill_seekers.cli.utils import setup_logging
|
||||
except ImportError as e:
|
||||
@@ -45,7 +46,7 @@ class UnifiedScraper:
|
||||
1. Load and validate unified config
|
||||
2. Scrape all sources (docs, GitHub, PDF)
|
||||
3. Detect conflicts between sources
|
||||
4. Merge intelligently (rule-based or Claude-enhanced)
|
||||
4. Merge intelligently (rule-based or AI-enhanced)
|
||||
5. Build unified skill
|
||||
"""
|
||||
|
||||
@@ -64,8 +65,9 @@ class UnifiedScraper:
|
||||
self.validator = validate_config(config_path)
|
||||
self.config = self.validator.config
|
||||
|
||||
# Determine merge mode
|
||||
self.merge_mode = merge_mode or self.config.get("merge_mode", "rule-based")
|
||||
# Determine merge mode (normalize claude-enhanced → ai-enhanced for backward compat)
|
||||
raw_mode = merge_mode or self.config.get("merge_mode", "rule-based")
|
||||
self.merge_mode = "ai-enhanced" if raw_mode == "claude-enhanced" else raw_mode
|
||||
logger.info(f"Merge mode: {self.merge_mode}")
|
||||
|
||||
# Storage for scraped data - use lists to support multiple sources of same type
|
||||
@@ -225,7 +227,7 @@ class UnifiedScraper:
|
||||
"url_patterns": source.get("url_patterns", {}),
|
||||
"categories": source.get("categories", {}),
|
||||
"rate_limit": source.get("rate_limit", 0.5),
|
||||
"max_pages": source.get("max_pages", 100),
|
||||
"max_pages": source.get("max_pages", 500),
|
||||
}
|
||||
|
||||
# Pass through llms.txt settings (so unified configs behave the same as doc_scraper configs)
|
||||
@@ -239,9 +241,21 @@ class UnifiedScraper:
|
||||
if "start_urls" in source:
|
||||
doc_source["start_urls"] = source["start_urls"]
|
||||
|
||||
# Pass through browser rendering settings
|
||||
if source.get("browser"):
|
||||
doc_source["browser"] = True
|
||||
if "browser_wait_until" in source:
|
||||
doc_source["browser_wait_until"] = source["browser_wait_until"]
|
||||
if "browser_extra_wait" in source:
|
||||
doc_source["browser_extra_wait"] = source["browser_extra_wait"]
|
||||
|
||||
doc_config = {
|
||||
"name": f"{self.name}_docs",
|
||||
"description": f"Documentation for {self.name}",
|
||||
"base_url": source["base_url"],
|
||||
"browser": source.get("browser", False),
|
||||
"browser_wait_until": source.get("browser_wait_until", "domcontentloaded"),
|
||||
"browser_extra_wait": source.get("browser_extra_wait", 0),
|
||||
"sources": [doc_source],
|
||||
}
|
||||
|
||||
@@ -256,7 +270,29 @@ class UnifiedScraper:
|
||||
doc_scraper_path = Path(__file__).parent / "doc_scraper.py"
|
||||
cmd = [sys.executable, str(doc_scraper_path), "--config", temp_config_path, "--fresh"]
|
||||
|
||||
result = subprocess.run(cmd, capture_output=True, text=True, stdin=subprocess.DEVNULL)
|
||||
# Forward agent-related CLI args so doc scraper enhancement respects
|
||||
# the user's chosen agent instead of defaulting to claude.
|
||||
cli_args = getattr(self, "_cli_args", None)
|
||||
if cli_args is not None:
|
||||
if getattr(cli_args, "agent", None):
|
||||
cmd.extend(["--agent", cli_args.agent])
|
||||
if getattr(cli_args, "agent_cmd", None):
|
||||
cmd.extend(["--agent-cmd", cli_args.agent_cmd])
|
||||
if getattr(cli_args, "api_key", None):
|
||||
cmd.extend(["--api-key", cli_args.api_key])
|
||||
|
||||
# Support "browser": true in source config for JavaScript SPA sites
|
||||
if source.get("browser", False):
|
||||
cmd.append("--browser")
|
||||
logger.info(" 🌐 Browser mode enabled (JavaScript rendering via Playwright)")
|
||||
|
||||
try:
|
||||
result = subprocess.run(
|
||||
cmd, capture_output=True, text=True, stdin=subprocess.DEVNULL, timeout=3600
|
||||
)
|
||||
except subprocess.TimeoutExpired:
|
||||
logger.error("Documentation scraping timed out after 60 minutes")
|
||||
return
|
||||
|
||||
if result.returncode != 0:
|
||||
logger.error(f"Documentation scraping failed with return code {result.returncode}")
|
||||
@@ -318,6 +354,11 @@ class UnifiedScraper:
|
||||
shutil.move(docs_data_dir, cache_data_dir)
|
||||
logger.info(f"📦 Moved docs data to cache: {cache_data_dir}")
|
||||
|
||||
# Update data_file path to point to cache location
|
||||
if self.scraped_data["documentation"]:
|
||||
cached_data_file = os.path.join(cache_data_dir, "summary.json")
|
||||
self.scraped_data["documentation"][-1]["data_file"] = cached_data_file
|
||||
|
||||
def _clone_github_repo(self, repo_name: str, idx: int = 0) -> str | None:
|
||||
"""
|
||||
Clone GitHub repository to cache directory for C3.x analysis.
|
||||
@@ -353,7 +394,7 @@ class UnifiedScraper:
|
||||
["git", "clone", repo_url, clone_path],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=600, # 10 minute timeout for full clone
|
||||
timeout=get_default_timeout(), # default 45 min, configurable via SKILL_SEEKER_ENHANCE_TIMEOUT
|
||||
)
|
||||
|
||||
if result.returncode == 0:
|
||||
@@ -422,6 +463,7 @@ class UnifiedScraper:
|
||||
"include_code": source.get("include_code", True),
|
||||
"code_analysis_depth": source.get("code_analysis_depth", "surface"),
|
||||
"file_patterns": source.get("file_patterns", []),
|
||||
"language": source.get("language", ""),
|
||||
"local_repo_path": local_repo_path, # Use cloned path if available
|
||||
}
|
||||
|
||||
@@ -776,25 +818,38 @@ class UnifiedScraper:
|
||||
enhance_level=enhance_level,
|
||||
)
|
||||
|
||||
# Load analysis outputs into memory
|
||||
# Load analysis outputs into memory.
|
||||
# _generate_references() moves dirs into references/ and deletes originals.
|
||||
refs = temp_output / "references"
|
||||
local_data = {
|
||||
"source_id": f"{self.name}_local_{idx}_{path_id}",
|
||||
"path": local_path,
|
||||
"name": source_name,
|
||||
"description": source.get("description", f"Local analysis of {path_id}"),
|
||||
"weight": source.get("weight", 1.0),
|
||||
"patterns": self._load_json(temp_output / "patterns" / "detected_patterns.json"),
|
||||
"test_examples": self._load_json(
|
||||
temp_output / "test_examples" / "test_examples.json"
|
||||
"patterns": self._load_json_fallback(
|
||||
refs / "patterns" / "all_patterns.json",
|
||||
temp_output / "patterns" / "all_patterns.json",
|
||||
),
|
||||
"how_to_guides": self._load_guide_collection(temp_output / "tutorials"),
|
||||
"config_patterns": self._load_json(
|
||||
temp_output / "config_patterns" / "config_patterns.json"
|
||||
"test_examples": self._load_json_fallback(
|
||||
refs / "test_examples" / "test_examples.json",
|
||||
temp_output / "test_examples" / "test_examples.json",
|
||||
),
|
||||
"architecture": self._load_json(temp_output / "ARCHITECTURE.json"),
|
||||
"api_reference": self._load_api_reference(temp_output / "api_reference"),
|
||||
"dependency_graph": self._load_json(
|
||||
temp_output / "dependencies" / "dependency_graph.json"
|
||||
"how_to_guides": self._load_guide_collection(refs / "tutorials")
|
||||
or self._load_guide_collection(temp_output / "tutorials"),
|
||||
"config_patterns": self._load_json_fallback(
|
||||
refs / "config_patterns" / "config_patterns.json",
|
||||
temp_output / "config_patterns" / "config_patterns.json",
|
||||
),
|
||||
"architecture": self._load_json_fallback(
|
||||
refs / "architecture" / "architectural_patterns.json",
|
||||
temp_output / "architecture" / "architectural_patterns.json",
|
||||
),
|
||||
"api_reference": self._load_api_reference(refs / "api_reference")
|
||||
or self._load_api_reference(temp_output / "api_reference"),
|
||||
"dependency_graph": self._load_json_fallback(
|
||||
refs / "dependencies" / "dependency_graph.json",
|
||||
temp_output / "dependencies" / "dependency_graph.json",
|
||||
),
|
||||
}
|
||||
|
||||
@@ -1425,6 +1480,12 @@ class UnifiedScraper:
|
||||
|
||||
logger.info(f"✅ Chat: {len(chat_data.get('messages', []))} messages extracted")
|
||||
|
||||
def _load_json_fallback(self, primary: Path, fallback: Path) -> dict:
|
||||
"""Load JSON from primary path, falling back to secondary if not found."""
|
||||
if primary.exists():
|
||||
return self._load_json(primary)
|
||||
return self._load_json(fallback)
|
||||
|
||||
def _load_json(self, file_path: Path) -> dict:
|
||||
"""
|
||||
Load JSON file safely.
|
||||
@@ -1525,6 +1586,11 @@ class UnifiedScraper:
|
||||
logger.info(f" Analyzing codebase: {local_repo_path}")
|
||||
|
||||
try:
|
||||
# Resolve agent from CLI args for C3.x analysis
|
||||
cli_args = getattr(self, "_cli_args", None)
|
||||
agent = getattr(cli_args, "agent", None) if cli_args else None
|
||||
agent_cmd = getattr(cli_args, "agent_cmd", None) if cli_args else None
|
||||
|
||||
# Run full C3.x analysis
|
||||
_results = analyze_codebase(
|
||||
directory=Path(local_repo_path),
|
||||
@@ -1541,25 +1607,39 @@ class UnifiedScraper:
|
||||
extract_config_patterns=True, # C3.4: Config patterns
|
||||
extract_docs=True,
|
||||
enhance_level=0 if source.get("ai_mode", "auto") == "none" else 2,
|
||||
agent=agent,
|
||||
agent_cmd=agent_cmd,
|
||||
)
|
||||
|
||||
# Load C3.x outputs into memory
|
||||
# Load C3.x outputs into memory.
|
||||
# _generate_references() inside analyze_codebase() moves analysis dirs
|
||||
# into references/ and deletes the originals, so we check both locations.
|
||||
refs = temp_output / "references"
|
||||
c3_data = {
|
||||
"patterns": self._load_json(temp_output / "patterns" / "detected_patterns.json"),
|
||||
"test_examples": self._load_json(
|
||||
temp_output / "test_examples" / "test_examples.json"
|
||||
"patterns": self._load_json_fallback(
|
||||
refs / "patterns" / "all_patterns.json",
|
||||
temp_output / "patterns" / "all_patterns.json",
|
||||
),
|
||||
"how_to_guides": self._load_guide_collection(temp_output / "tutorials"),
|
||||
"config_patterns": self._load_json(
|
||||
temp_output / "config_patterns" / "config_patterns.json"
|
||||
"test_examples": self._load_json_fallback(
|
||||
refs / "test_examples" / "test_examples.json",
|
||||
temp_output / "test_examples" / "test_examples.json",
|
||||
),
|
||||
"architecture": self._load_json(
|
||||
temp_output / "architecture" / "architectural_patterns.json"
|
||||
"how_to_guides": self._load_guide_collection(refs / "tutorials")
|
||||
or self._load_guide_collection(temp_output / "tutorials"),
|
||||
"config_patterns": self._load_json_fallback(
|
||||
refs / "config_patterns" / "config_patterns.json",
|
||||
temp_output / "config_patterns" / "config_patterns.json",
|
||||
),
|
||||
"architecture": self._load_json_fallback(
|
||||
refs / "architecture" / "architectural_patterns.json",
|
||||
temp_output / "architecture" / "architectural_patterns.json",
|
||||
),
|
||||
"api_reference": self._load_api_reference(refs / "api_reference")
|
||||
or self._load_api_reference(temp_output / "api_reference"),
|
||||
"dependency_graph": self._load_json_fallback(
|
||||
refs / "dependencies" / "dependency_graph.json",
|
||||
temp_output / "dependencies" / "dependency_graph.json",
|
||||
),
|
||||
"api_reference": self._load_api_reference(temp_output / "api_reference"), # C2.5
|
||||
"dependency_graph": self._load_json(
|
||||
temp_output / "dependencies" / "dependency_graph.json"
|
||||
), # C2.6
|
||||
}
|
||||
|
||||
# Log summary
|
||||
@@ -1607,16 +1687,23 @@ class UnifiedScraper:
|
||||
|
||||
if not self.validator.needs_api_merge():
|
||||
logger.info("No API merge needed (only one API source)")
|
||||
logger.info("\n" + "=" * 60)
|
||||
logger.info("PHASE 3: Merging sources (skipped - no conflicts detected)")
|
||||
logger.info("=" * 60)
|
||||
return []
|
||||
|
||||
# Get documentation and GitHub data
|
||||
docs_data = self.scraped_data.get("documentation", {})
|
||||
github_data = self.scraped_data.get("github", {})
|
||||
# Get documentation and GitHub data (scraped_data stores lists of sources)
|
||||
docs_list = self.scraped_data.get("documentation", [])
|
||||
github_list = self.scraped_data.get("github", [])
|
||||
|
||||
if not docs_data or not github_data:
|
||||
if not docs_list or not github_list:
|
||||
logger.warning("Missing documentation or GitHub data for conflict detection")
|
||||
return []
|
||||
|
||||
# Use the first source from each list
|
||||
docs_data = docs_list[0]
|
||||
github_data = github_list[0]
|
||||
|
||||
# Load data files
|
||||
with open(docs_data["data_file"], encoding="utf-8") as f:
|
||||
docs_json = json.load(f)
|
||||
@@ -1663,9 +1750,16 @@ class UnifiedScraper:
|
||||
logger.info("No conflicts to merge")
|
||||
return None
|
||||
|
||||
# Get data files
|
||||
docs_data = self.scraped_data.get("documentation", {})
|
||||
github_data = self.scraped_data.get("github", {})
|
||||
# Get data files (scraped_data stores lists of sources)
|
||||
docs_list = self.scraped_data.get("documentation", [])
|
||||
github_list = self.scraped_data.get("github", [])
|
||||
|
||||
if not docs_list or not github_list:
|
||||
logger.warning("Missing documentation or GitHub data for merging")
|
||||
return None
|
||||
|
||||
docs_data = docs_list[0]
|
||||
github_data = github_list[0]
|
||||
|
||||
# Load data
|
||||
with open(docs_data["data_file"], encoding="utf-8") as f:
|
||||
@@ -1675,8 +1769,8 @@ class UnifiedScraper:
|
||||
github_json = json.load(f)
|
||||
|
||||
# Choose merger
|
||||
if self.merge_mode == "claude-enhanced":
|
||||
merger = ClaudeEnhancedMerger(docs_json, github_json, conflicts)
|
||||
if self.merge_mode in ("ai-enhanced", "claude-enhanced"):
|
||||
merger = AIEnhancedMerger(docs_json, github_json, conflicts)
|
||||
else:
|
||||
merger = RuleBasedMerger(docs_json, github_json, conflicts)
|
||||
|
||||
@@ -1797,6 +1891,124 @@ class UnifiedScraper:
|
||||
}
|
||||
run_workflows(effective_args, context=unified_context)
|
||||
|
||||
# Phase 6: AI Enhancement of SKILL.md
|
||||
# Triggered by config "enhancement" block or CLI --enhance-level
|
||||
enhancement_config = self.config.get("enhancement", {})
|
||||
enhancement_enabled = enhancement_config.get("enabled", False)
|
||||
enhancement_level = enhancement_config.get("level", 0)
|
||||
enhancement_mode = enhancement_config.get("mode", "AUTO").upper()
|
||||
|
||||
# CLI --enhance-level overrides config
|
||||
cli_enhance_level = getattr(args, "enhance_level", None) if args is not None else None
|
||||
if cli_enhance_level is not None:
|
||||
enhancement_enabled = cli_enhance_level > 0
|
||||
enhancement_level = cli_enhance_level
|
||||
|
||||
if enhancement_enabled and enhancement_level > 0:
|
||||
logger.info("\n" + "=" * 60)
|
||||
logger.info(
|
||||
f"PHASE 6: Enhancing SKILL.md (level {enhancement_level}, mode {enhancement_mode})"
|
||||
)
|
||||
logger.info("=" * 60)
|
||||
|
||||
skill_md_path = os.path.join(self.output_dir, "SKILL.md")
|
||||
if not os.path.exists(skill_md_path):
|
||||
logger.warning("⚠️ SKILL.md not found, skipping enhancement")
|
||||
elif enhancement_mode == "LOCAL":
|
||||
try:
|
||||
from skill_seekers.cli.enhance_skill_local import LocalSkillEnhancer
|
||||
|
||||
# Get agent from CLI args, config enhancement block, or env var
|
||||
agent = None
|
||||
agent_cmd = None
|
||||
if args is not None:
|
||||
agent = getattr(args, "agent", None)
|
||||
agent_cmd = getattr(args, "agent_cmd", None)
|
||||
if not agent:
|
||||
agent = enhancement_config.get("agent", None)
|
||||
if not agent:
|
||||
agent = os.environ.get("SKILL_SEEKER_AGENT", "").strip() or None
|
||||
|
||||
# Read timeout from config enhancement block
|
||||
timeout_val = enhancement_config.get("timeout")
|
||||
if timeout_val is not None:
|
||||
if isinstance(timeout_val, str) and timeout_val.lower() in (
|
||||
"unlimited",
|
||||
"none",
|
||||
):
|
||||
timeout_val = 86400 # 24 hours
|
||||
else:
|
||||
try:
|
||||
timeout_val = int(timeout_val)
|
||||
if timeout_val <= 0:
|
||||
timeout_val = 86400
|
||||
except (ValueError, TypeError):
|
||||
timeout_val = 2700
|
||||
else:
|
||||
timeout_val = 2700
|
||||
|
||||
enhancer = LocalSkillEnhancer(
|
||||
self.output_dir, force=True, agent=agent, agent_cmd=agent_cmd
|
||||
)
|
||||
success = enhancer.run(headless=True, timeout=timeout_val)
|
||||
agent_name = agent or "claude"
|
||||
if success:
|
||||
logger.info(f"✅ SKILL.md enhanced (LOCAL mode via {agent_name})")
|
||||
else:
|
||||
logger.warning(
|
||||
f"⚠️ SKILL.md enhancement returned false (LOCAL mode via {agent_name}). "
|
||||
"Check logs above for the exact error."
|
||||
)
|
||||
except Exception as e:
|
||||
logger.warning(f"⚠️ LOCAL enhancement failed: {e}")
|
||||
logger.info(
|
||||
" Try manually: skill-seekers enhance "
|
||||
+ self.output_dir
|
||||
+ " --agent kimi"
|
||||
)
|
||||
else:
|
||||
# API mode — use AgentClient for multi-provider support
|
||||
try:
|
||||
from skill_seekers.cli.agent_client import AgentClient
|
||||
|
||||
client = AgentClient(mode="api")
|
||||
if client.client:
|
||||
# Read references and current SKILL.md
|
||||
references = ""
|
||||
refs_dir = Path(self.output_dir) / "references"
|
||||
if refs_dir.exists():
|
||||
for md_file in sorted(refs_dir.rglob("*.md")):
|
||||
content = md_file.read_text(encoding="utf-8", errors="ignore")
|
||||
references += f"\n\n## {md_file.name}\n\n{content}"
|
||||
current_skill = Path(skill_md_path).read_text(encoding="utf-8")
|
||||
|
||||
# Build enhancement prompt
|
||||
prompt = (
|
||||
f"Enhance this SKILL.md using the reference documentation.\n\n"
|
||||
f"CURRENT SKILL.MD:\n{current_skill}\n\n"
|
||||
f"REFERENCES:\n{references}\n\n"
|
||||
f"Return ONLY the complete enhanced SKILL.md content, "
|
||||
f"starting with the frontmatter (---)."
|
||||
)
|
||||
enhanced = client.call(prompt, max_tokens=8192)
|
||||
if enhanced:
|
||||
shutil.copy2(skill_md_path, skill_md_path + ".backup")
|
||||
Path(skill_md_path).write_text(enhanced, encoding="utf-8")
|
||||
logger.info(
|
||||
f"✅ SKILL.md enhanced (API mode via {client.provider})"
|
||||
)
|
||||
else:
|
||||
logger.warning("⚠️ API enhancement returned empty result")
|
||||
else:
|
||||
logger.warning("⚠️ No API key found, skipping API enhancement")
|
||||
logger.info(' Set an API key or use "mode": "LOCAL" in config')
|
||||
except Exception as e:
|
||||
logger.warning(f"⚠️ API enhancement failed: {e}")
|
||||
else:
|
||||
logger.info("\n" + "=" * 60)
|
||||
logger.info("PHASE 6: Enhancement (skipped - not enabled in config)")
|
||||
logger.info("=" * 60)
|
||||
|
||||
logger.info("\n" + "✅ " * 20)
|
||||
logger.info("Unified scraping complete!")
|
||||
logger.info("✅ " * 20 + "\n")
|
||||
@@ -1826,7 +2038,7 @@ Examples:
|
||||
skill-seekers unified --config configs/godot_unified.json
|
||||
|
||||
# Override merge mode
|
||||
skill-seekers unified --config configs/react_unified.json --merge-mode claude-enhanced
|
||||
skill-seekers unified --config configs/react_unified.json --merge-mode ai-enhanced
|
||||
|
||||
# Backward compatible with legacy configs
|
||||
skill-seekers unified --config configs/react.json
|
||||
@@ -1837,8 +2049,8 @@ Examples:
|
||||
parser.add_argument(
|
||||
"--merge-mode",
|
||||
"-m",
|
||||
choices=["rule-based", "claude-enhanced"],
|
||||
help="Override config merge mode",
|
||||
choices=["rule-based", "ai-enhanced", "claude-enhanced"],
|
||||
help="Override config merge mode (ai-enhanced or rule-based). 'claude-enhanced' accepted as alias.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--skip-codebase-analysis",
|
||||
@@ -1901,6 +2113,19 @@ Examples:
|
||||
"Overrides per-source enhance_level in config."
|
||||
),
|
||||
)
|
||||
parser.add_argument(
|
||||
"--agent",
|
||||
type=str,
|
||||
choices=["claude", "codex", "copilot", "opencode", "kimi", "custom"],
|
||||
metavar="AGENT",
|
||||
help="Local coding agent for enhancement (default: AI agent from SKILL_SEEKER_AGENT env var)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--agent-cmd",
|
||||
type=str,
|
||||
metavar="CMD",
|
||||
help="Override agent command template (advanced)",
|
||||
)
|
||||
|
||||
args = parser.parse_args()
|
||||
setup_logging()
|
||||
|
||||
@@ -4,7 +4,7 @@ Automatic Skill Uploader
|
||||
Uploads a skill package to LLM platforms (Claude, Gemini, OpenAI, etc.)
|
||||
|
||||
Usage:
|
||||
# Claude (default)
|
||||
# Anthropic (default)
|
||||
export ANTHROPIC_API_KEY=sk-ant-...
|
||||
skill-seekers upload output/react.zip
|
||||
|
||||
@@ -117,7 +117,7 @@ def main():
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter,
|
||||
epilog="""
|
||||
Setup:
|
||||
Claude:
|
||||
Anthropic (Claude):
|
||||
export ANTHROPIC_API_KEY=sk-ant-...
|
||||
|
||||
Gemini:
|
||||
@@ -135,7 +135,7 @@ Setup:
|
||||
docker run -p 8080:8080 semitechnologies/weaviate:latest
|
||||
|
||||
Examples:
|
||||
# Upload to Claude (default)
|
||||
# Upload to default platform
|
||||
skill-seekers upload output/react.zip
|
||||
|
||||
# Upload to Gemini
|
||||
@@ -162,9 +162,9 @@ Examples:
|
||||
|
||||
parser.add_argument(
|
||||
"--target",
|
||||
choices=["claude", "gemini", "openai", "chroma", "weaviate"],
|
||||
default="claude",
|
||||
help="Target platform (default: claude)",
|
||||
choices=["claude", "gemini", "openai", "kimi", "chroma", "weaviate"],
|
||||
default=None,
|
||||
help="Target platform (auto-detected from API keys, or 'claude' if none set)",
|
||||
)
|
||||
|
||||
parser.add_argument("--api-key", help="Platform API key (or set environment variable)")
|
||||
@@ -209,6 +209,12 @@ Examples:
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
# Auto-detect target platform if not specified
|
||||
if args.target is None:
|
||||
from skill_seekers.cli.agent_client import AgentClient
|
||||
|
||||
args.target = AgentClient.detect_default_target()
|
||||
|
||||
# Build kwargs for vector DB upload
|
||||
upload_kwargs = {}
|
||||
|
||||
|
||||
@@ -76,32 +76,48 @@ def open_folder(folder_path: str | Path) -> bool:
|
||||
|
||||
def has_api_key() -> bool:
|
||||
"""
|
||||
Check if ANTHROPIC_API_KEY is set in environment
|
||||
Check if any AI API key is set in environment.
|
||||
|
||||
Checks: ANTHROPIC_API_KEY, MOONSHOT_API_KEY, GOOGLE_API_KEY, OPENAI_API_KEY
|
||||
|
||||
Returns:
|
||||
bool: True if API key is set, False otherwise
|
||||
bool: True if any API key is set, False otherwise
|
||||
"""
|
||||
api_key = os.environ.get("ANTHROPIC_API_KEY", "").strip()
|
||||
return len(api_key) > 0
|
||||
for env_var in ("ANTHROPIC_API_KEY", "MOONSHOT_API_KEY", "GOOGLE_API_KEY", "OPENAI_API_KEY"):
|
||||
if os.environ.get(env_var, "").strip():
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def get_api_key() -> str | None:
|
||||
"""
|
||||
Get ANTHROPIC_API_KEY from environment
|
||||
Get the first available AI API key from environment.
|
||||
|
||||
Checks: ANTHROPIC_API_KEY, ANTHROPIC_AUTH_TOKEN, MOONSHOT_API_KEY,
|
||||
GOOGLE_API_KEY, OPENAI_API_KEY
|
||||
|
||||
Returns:
|
||||
str: API key or None if not set
|
||||
"""
|
||||
api_key = os.environ.get("ANTHROPIC_API_KEY", "").strip()
|
||||
return api_key if api_key else None
|
||||
for env_var in (
|
||||
"ANTHROPIC_API_KEY",
|
||||
"ANTHROPIC_AUTH_TOKEN",
|
||||
"MOONSHOT_API_KEY",
|
||||
"GOOGLE_API_KEY",
|
||||
"OPENAI_API_KEY",
|
||||
):
|
||||
key = os.environ.get(env_var, "").strip()
|
||||
if key:
|
||||
return key
|
||||
return None
|
||||
|
||||
|
||||
def get_upload_url() -> str:
|
||||
"""
|
||||
Get the Claude skills upload URL
|
||||
Get the skills upload URL
|
||||
|
||||
Returns:
|
||||
str: Claude skills upload URL
|
||||
str: Skills upload URL
|
||||
"""
|
||||
return "https://claude.ai/skills"
|
||||
|
||||
@@ -120,7 +136,7 @@ def print_upload_instructions(zip_path: str | Path) -> None:
|
||||
print("║ NEXT STEP ║")
|
||||
print("╚══════════════════════════════════════════════════════════╝")
|
||||
print()
|
||||
print(f"📤 Upload to Claude: {get_upload_url()}")
|
||||
print(f"📤 Upload to platform: {get_upload_url()}")
|
||||
print()
|
||||
print(f"1. Go to {get_upload_url()}")
|
||||
print('2. Click "Upload Skill"')
|
||||
|
||||
@@ -1,9 +1,9 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Video to Claude Skill Converter
|
||||
Video to AI Skill Converter
|
||||
|
||||
Extracts transcripts, metadata, and visual content from videos
|
||||
and converts them into Claude AI skills.
|
||||
and converts them into AI skills.
|
||||
|
||||
Supports YouTube videos/playlists, Vimeo, and local video files.
|
||||
|
||||
@@ -264,7 +264,7 @@ def _is_likely_code(text: str) -> bool:
|
||||
def _ai_clean_reference(ref_path: str, content: str, api_key: str | None = None) -> None:
|
||||
"""Use AI to clean Code Timeline section in a reference file.
|
||||
|
||||
Sends the reference file content to Claude with a focused prompt
|
||||
Sends the reference file content to the AI with a focused prompt
|
||||
to reconstruct the Code Timeline from noisy OCR + transcript context.
|
||||
"""
|
||||
try:
|
||||
@@ -300,7 +300,7 @@ def _ai_clean_reference(ref_path: str, content: str, api_key: str | None = None)
|
||||
try:
|
||||
client = anthropic.Anthropic(**client_kwargs)
|
||||
response = client.messages.create(
|
||||
model="claude-sonnet-4-20250514",
|
||||
model=os.environ.get("ANTHROPIC_MODEL", "claude-sonnet-4-20250514"),
|
||||
max_tokens=8000,
|
||||
messages=[{"role": "user", "content": prompt}],
|
||||
)
|
||||
@@ -319,7 +319,7 @@ def _ai_clean_reference(ref_path: str, content: str, api_key: str | None = None)
|
||||
|
||||
|
||||
class VideoToSkillConverter:
|
||||
"""Convert video content to Claude skill."""
|
||||
"""Convert video content to AI skill."""
|
||||
|
||||
def __init__(self, config: dict):
|
||||
"""Initialize converter.
|
||||
@@ -865,9 +865,12 @@ class VideoToSkillConverter:
|
||||
"""First-pass: AI-clean reference files before SKILL.md enhancement.
|
||||
|
||||
When enhance_level >= 2 and an API key is available, sends each
|
||||
reference file to Claude to reconstruct noisy Code Timeline
|
||||
reference file to the AI to reconstruct noisy Code Timeline
|
||||
sections using transcript context.
|
||||
"""
|
||||
# Note: Middle-layer AI cleaning currently only supports Anthropic API
|
||||
# For other agents (kimi, etc.), this step is skipped and enhancement
|
||||
# happens at the SKILL.md level instead of per-reference-file
|
||||
has_api_key = bool(
|
||||
os.environ.get("ANTHROPIC_API_KEY")
|
||||
or os.environ.get("ANTHROPIC_AUTH_TOKEN")
|
||||
@@ -1203,9 +1206,12 @@ def _run_video_enhancement(skill_dir: str, enhance_level: int, args) -> None:
|
||||
os.environ.get("ANTHROPIC_API_KEY")
|
||||
or os.environ.get("ANTHROPIC_AUTH_TOKEN")
|
||||
or getattr(args, "api_key", None)
|
||||
or os.environ.get("MOONSHOT_API_KEY")
|
||||
)
|
||||
|
||||
if not has_api_key:
|
||||
agent = getattr(args, "agent", None)
|
||||
|
||||
if not has_api_key and not agent:
|
||||
logger.info("\n💡 Enhance your video skill with AI:")
|
||||
logger.info(f" export ANTHROPIC_API_KEY=sk-ant-...")
|
||||
logger.info(f" skill-seekers enhance {skill_dir} --enhance-level {enhance_level}")
|
||||
@@ -1215,10 +1221,11 @@ def _run_video_enhancement(skill_dir: str, enhance_level: int, args) -> None:
|
||||
|
||||
try:
|
||||
enhance_cmd = ["skill-seekers-enhance", skill_dir]
|
||||
enhance_cmd.extend(["--enhance-level", str(enhance_level)])
|
||||
api_key = getattr(args, "api_key", None)
|
||||
if api_key:
|
||||
enhance_cmd.extend(["--api-key", api_key])
|
||||
if agent:
|
||||
enhance_cmd.extend(["--agent", agent])
|
||||
|
||||
logger.info(
|
||||
"Starting video skill enhancement (this may take 10+ minutes "
|
||||
|
||||
@@ -590,7 +590,7 @@ def _ocr_with_claude_vision(frame_path: str, frame_type: FrameType) -> tuple[str
|
||||
|
||||
client = anthropic.Anthropic(api_key=api_key)
|
||||
response = client.messages.create(
|
||||
model="claude-haiku-4-5-20251001",
|
||||
model=os.environ.get("ANTHROPIC_MODEL", "claude-haiku-4-5-20251001"),
|
||||
max_tokens=4096,
|
||||
messages=[
|
||||
{
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Word Document (.docx) to Claude Skill Converter (Task B2)
|
||||
Word Document (.docx) to AI Skill Converter (Task B2)
|
||||
|
||||
Converts Word documents into Claude AI skills.
|
||||
Converts Word documents into AI skills.
|
||||
Uses mammoth for HTML conversion and python-docx for metadata/tables.
|
||||
|
||||
Usage:
|
||||
@@ -73,7 +73,7 @@ def infer_description_from_word(metadata: dict = None, name: str = "") -> str:
|
||||
|
||||
|
||||
class WordToSkillConverter:
|
||||
"""Convert Word document (.docx) to Claude skill."""
|
||||
"""Convert Word document (.docx) to AI skill."""
|
||||
|
||||
def __init__(self, config):
|
||||
self.config = config
|
||||
@@ -924,7 +924,7 @@ def main():
|
||||
from .arguments.word import add_word_arguments
|
||||
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Convert Word document (.docx) to Claude skill",
|
||||
description="Convert Word document (.docx) to AI skill",
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter,
|
||||
)
|
||||
|
||||
@@ -1031,14 +1031,18 @@ def main():
|
||||
from pathlib import Path
|
||||
from skill_seekers.cli.enhance_skill_local import LocalSkillEnhancer
|
||||
|
||||
enhancer = LocalSkillEnhancer(Path(skill_dir))
|
||||
agent = getattr(args, "agent", None) if args else None
|
||||
agent_cmd = getattr(args, "agent_cmd", None) if args else None
|
||||
enhancer = LocalSkillEnhancer(Path(skill_dir), agent=agent, agent_cmd=agent_cmd)
|
||||
enhancer.run(headless=True)
|
||||
print("✅ Local enhancement complete!")
|
||||
else:
|
||||
from pathlib import Path
|
||||
from skill_seekers.cli.enhance_skill_local import LocalSkillEnhancer
|
||||
|
||||
enhancer = LocalSkillEnhancer(Path(skill_dir))
|
||||
agent = getattr(args, "agent", None) if args else None
|
||||
agent_cmd = getattr(args, "agent_cmd", None) if args else None
|
||||
enhancer = LocalSkillEnhancer(Path(skill_dir), agent=agent, agent_cmd=agent_cmd)
|
||||
enhancer.run(headless=True)
|
||||
print("✅ Local enhancement complete!")
|
||||
|
||||
|
||||
@@ -43,6 +43,7 @@ def _build_inline_engine(args: argparse.Namespace):
|
||||
"""Build a WorkflowEngine from --enhance-stage flags."""
|
||||
from skill_seekers.cli.enhancement_workflow import WorkflowEngine
|
||||
|
||||
agent = getattr(args, "agent", None)
|
||||
stages = []
|
||||
for i, spec in enumerate(args.enhance_stage, 1):
|
||||
if ":" in spec:
|
||||
@@ -63,7 +64,7 @@ def _build_inline_engine(args: argparse.Namespace):
|
||||
"description": "Custom inline workflow from --enhance-stage arguments",
|
||||
"stages": stages,
|
||||
}
|
||||
return WorkflowEngine(workflow_data=inline_def)
|
||||
return WorkflowEngine(workflow_data=inline_def, agent=agent)
|
||||
|
||||
|
||||
def run_workflows(
|
||||
@@ -107,6 +108,7 @@ def run_workflows(
|
||||
logger.info(f" {k} = {v}")
|
||||
|
||||
executed: list[str] = []
|
||||
agent = getattr(args, "agent", None)
|
||||
|
||||
# ── Named workflows ────────────────────────────────────────────────────
|
||||
total = len(named_workflows) + (1 if inline_stages else 0)
|
||||
@@ -118,7 +120,7 @@ def run_workflows(
|
||||
logger.info(header)
|
||||
|
||||
try:
|
||||
engine = WorkflowEngine(workflow_name)
|
||||
engine = WorkflowEngine(workflow_name, agent=agent)
|
||||
except Exception as exc:
|
||||
logger.error(f"❌ Failed to load workflow '{workflow_name}': {exc}")
|
||||
logger.info(" Skipping this workflow and continuing...")
|
||||
|
||||
@@ -1,10 +1,13 @@
|
||||
# Skill Seeker MCP Server
|
||||
|
||||
Model Context Protocol (MCP) server for Skill Seeker - enables Claude Code to generate documentation skills directly.
|
||||
> Works with **Claude Code**, **Cursor**, **Windsurf**, **VS Code + Cline**, and **IntelliJ IDEA**.
|
||||
> Supports API mode (Anthropic, Moonshot/Kimi, Google Gemini, OpenAI) and LOCAL mode (any AI coding agent).
|
||||
|
||||
Model Context Protocol (MCP) server for Skill Seeker - enables AI coding agents to generate documentation skills directly.
|
||||
|
||||
## What is This?
|
||||
|
||||
This MCP server allows Claude Code to use Skill Seeker's tools directly through natural language commands. Instead of running CLI commands manually, you can ask Claude Code to:
|
||||
This MCP server allows your AI coding agent to use Skill Seeker's tools directly through natural language commands. Instead of running CLI commands manually, you can ask your agent to:
|
||||
|
||||
- Generate config files for any documentation site
|
||||
- Estimate page counts before scraping
|
||||
@@ -36,12 +39,12 @@ pip3 install -e ".[mcp]"
|
||||
# - Install dependencies
|
||||
# - Test the server
|
||||
# - Generate configuration
|
||||
# - Guide you through Claude Code setup
|
||||
# - Guide you through agent setup
|
||||
```
|
||||
|
||||
### 3. Manual Setup
|
||||
|
||||
Add to `~/.claude.json`:
|
||||
**For Claude Code** - add to `~/.claude.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
@@ -62,13 +65,15 @@ Add to `~/.claude.json`:
|
||||
|
||||
**Replace `/path/to/Skill_Seekers`** with your actual repository path!
|
||||
|
||||
### 4. Restart Claude Code
|
||||
**For Cursor/Windsurf** - use HTTP transport mode. See your editor's MCP documentation for configuration details.
|
||||
|
||||
Quit and reopen Claude Code (don't just close the window).
|
||||
### 4. Restart Your Agent
|
||||
|
||||
Quit and reopen your AI coding agent (don't just close the window).
|
||||
|
||||
### 5. Test
|
||||
|
||||
In Claude Code, type:
|
||||
In your AI coding agent, type:
|
||||
```
|
||||
List all available configs
|
||||
```
|
||||
@@ -107,7 +112,7 @@ Estimate pages for configs/react.json
|
||||
```
|
||||
|
||||
### 3. `scrape_docs`
|
||||
Scrape documentation and build Claude skill.
|
||||
Scrape documentation and build LLM skill.
|
||||
|
||||
**Parameters:**
|
||||
- `config_path` (required): Path to config file
|
||||
@@ -125,16 +130,17 @@ Package skill directory into platform-specific format. Automatically uploads if
|
||||
|
||||
**Parameters:**
|
||||
- `skill_dir` (required): Path to skill directory (e.g., "output/react/")
|
||||
- `target` (optional): Target platform - "claude", "gemini", "openai", "markdown" (default: "claude")
|
||||
- `target` (optional): Target platform - "claude", "gemini", "openai", "markdown", and more (default: auto-detected from environment)
|
||||
- `auto_upload` (optional): Try to upload automatically if API key is available (default: true)
|
||||
|
||||
**Platform-specific outputs:**
|
||||
- Claude/OpenAI/Markdown: `.zip` file
|
||||
- Claude/OpenAI/Markdown/Kimi/DeepSeek/Qwen: `.zip` file
|
||||
- Gemini: `.tar.gz` file
|
||||
|
||||
**Examples:**
|
||||
```
|
||||
Package skill for Claude (default): output/react/
|
||||
Package skill (auto-detected platform): output/react/
|
||||
Package skill for Claude: output/react/ with target claude
|
||||
Package skill for Gemini: output/react/ with target gemini
|
||||
Package skill for OpenAI: output/react/ with target openai
|
||||
Package skill for Markdown: output/react/ with target markdown
|
||||
@@ -145,7 +151,7 @@ Upload skill package to target LLM platform (requires platform-specific API key)
|
||||
|
||||
**Parameters:**
|
||||
- `skill_zip` (required): Path to skill package (`.zip` or `.tar.gz`)
|
||||
- `target` (optional): Target platform - "claude", "gemini", "openai" (default: "claude")
|
||||
- `target` (optional): Target platform - "claude", "gemini", "openai" (default: auto-detected from environment)
|
||||
|
||||
**Examples:**
|
||||
```
|
||||
@@ -161,8 +167,8 @@ Enhance SKILL.md with AI using target platform's model. Transforms basic templat
|
||||
|
||||
**Parameters:**
|
||||
- `skill_dir` (required): Path to skill directory (e.g., "output/react/")
|
||||
- `target` (optional): Target platform - "claude", "gemini", "openai" (default: "claude")
|
||||
- `mode` (optional): "local" (Claude Code Max, no API key) or "api" (requires API key) (default: "local")
|
||||
- `target` (optional): Target platform - "claude", "gemini", "openai" (default: auto-detected from environment)
|
||||
- `mode` (optional): "local" (AI coding agent, no API key) or "api" (requires API key) (default: "local")
|
||||
- `api_key` (optional): Platform API key (uses env var if not provided)
|
||||
|
||||
**What it does:**
|
||||
@@ -173,12 +179,12 @@ Enhance SKILL.md with AI using target platform's model. Transforms basic templat
|
||||
|
||||
**Examples:**
|
||||
```
|
||||
Enhance with Claude locally (no API key): output/react/
|
||||
Enhance locally (no API key): output/react/
|
||||
Enhance with Gemini API: output/react/ with target gemini and mode api
|
||||
Enhance with OpenAI API: output/react/ with target openai and mode api
|
||||
```
|
||||
|
||||
**Note:** Local mode uses Claude Code Max (requires Claude Code but no API key). API mode requires platform-specific API key.
|
||||
**Note:** Local mode uses your AI coding agent (no API key needed). API mode requires a platform-specific API key.
|
||||
|
||||
### 7. `list_configs`
|
||||
List all available preset configurations.
|
||||
@@ -240,7 +246,7 @@ Generate router for configs/godot-*.json
|
||||
- Users can ask questions naturally, router directs to appropriate sub-skill
|
||||
|
||||
### 11. `scrape_pdf`
|
||||
Scrape PDF documentation and build Claude skill. Extracts text, code blocks, images, and tables from PDF files with advanced features.
|
||||
Scrape PDF documentation and build LLM skill. Extracts text, code blocks, images, and tables from PDF files with advanced features.
|
||||
|
||||
**Parameters:**
|
||||
- `config_path` (optional): Path to PDF config JSON file (e.g., "configs/manual_pdf.json")
|
||||
@@ -293,20 +299,20 @@ Fast parallel processing: --pdf docs/large.pdf --parallel --workers 8
|
||||
```
|
||||
User: Generate config for Svelte at https://svelte.dev/docs
|
||||
|
||||
Claude: ✅ Config created: configs/svelte.json
|
||||
Agent: ✅ Config created: configs/svelte.json
|
||||
|
||||
User: Estimate pages for configs/svelte.json
|
||||
|
||||
Claude: 📊 Estimated pages: 150
|
||||
Agent: 📊 Estimated pages: 150
|
||||
|
||||
User: Scrape docs using configs/svelte.json
|
||||
|
||||
Claude: ✅ Skill created at output/svelte/
|
||||
Agent: ✅ Skill created at output/svelte/
|
||||
|
||||
User: Package skill at output/svelte/
|
||||
|
||||
Claude: ✅ Created: output/svelte.zip
|
||||
Ready to upload to Claude!
|
||||
Agent: ✅ Created: output/svelte.zip
|
||||
Ready to upload!
|
||||
```
|
||||
|
||||
### Use Existing Preset
|
||||
@@ -314,15 +320,15 @@ Claude: ✅ Created: output/svelte.zip
|
||||
```
|
||||
User: List all available configs
|
||||
|
||||
Claude: [Shows all configs: godot, react, vue, django, fastapi, etc.]
|
||||
Agent: [Shows all configs: godot, react, vue, django, fastapi, etc.]
|
||||
|
||||
User: Scrape docs using configs/react.json
|
||||
|
||||
Claude: ✅ Skill created at output/react/
|
||||
Agent: ✅ Skill created at output/react/
|
||||
|
||||
User: Package skill at output/react/
|
||||
|
||||
Claude: ✅ Created: output/react.zip
|
||||
Agent: ✅ Created: output/react.zip
|
||||
```
|
||||
|
||||
### Validate Before Scraping
|
||||
@@ -330,7 +336,7 @@ Claude: ✅ Created: output/react.zip
|
||||
```
|
||||
User: Validate configs/godot.json
|
||||
|
||||
Claude: ✅ Config is valid!
|
||||
Agent: ✅ Config is valid!
|
||||
Name: godot
|
||||
Base URL: https://docs.godotengine.org/en/stable/
|
||||
Max pages: 500
|
||||
@@ -338,7 +344,7 @@ Claude: ✅ Config is valid!
|
||||
|
||||
User: Scrape docs using configs/godot.json
|
||||
|
||||
Claude: [Starts scraping...]
|
||||
Agent: [Starts scraping...]
|
||||
```
|
||||
|
||||
### PDF Documentation - NEW
|
||||
@@ -346,17 +352,17 @@ Claude: [Starts scraping...]
|
||||
```
|
||||
User: Scrape PDF at docs/api-manual.pdf and create skill named api-docs
|
||||
|
||||
Claude: 📄 Scraping PDF documentation...
|
||||
✅ Extracted 120 pages
|
||||
✅ Found 45 code blocks (Python, JavaScript, C++)
|
||||
✅ Extracted 12 images
|
||||
✅ Created skill at output/api-docs/
|
||||
📦 Package with: python3 cli/package_skill.py output/api-docs/
|
||||
Agent: 📄 Scraping PDF documentation...
|
||||
✅ Extracted 120 pages
|
||||
✅ Found 45 code blocks (Python, JavaScript, C++)
|
||||
✅ Extracted 12 images
|
||||
✅ Created skill at output/api-docs/
|
||||
📦 Package with: python3 cli/package_skill.py output/api-docs/
|
||||
|
||||
User: Package skill at output/api-docs/
|
||||
|
||||
Claude: ✅ Created: output/api-docs.zip
|
||||
Ready to upload to Claude!
|
||||
Agent: ✅ Created: output/api-docs.zip
|
||||
Ready to upload!
|
||||
```
|
||||
|
||||
### Large Documentation (40K Pages)
|
||||
@@ -364,13 +370,13 @@ Claude: ✅ Created: output/api-docs.zip
|
||||
```
|
||||
User: Estimate pages for configs/godot.json
|
||||
|
||||
Claude: 📊 Estimated pages: 40,000
|
||||
⚠️ Large documentation detected!
|
||||
💡 Recommend splitting into multiple skills
|
||||
Agent: 📊 Estimated pages: 40,000
|
||||
⚠️ Large documentation detected!
|
||||
💡 Recommend splitting into multiple skills
|
||||
|
||||
User: Split configs/godot.json using router strategy
|
||||
|
||||
Claude: ✅ Split complete!
|
||||
Agent: ✅ Split complete!
|
||||
Created 5 sub-skills:
|
||||
- godot-scripting.json (5,000 pages)
|
||||
- godot-2d.json (8,000 pages)
|
||||
@@ -380,12 +386,12 @@ Claude: ✅ Split complete!
|
||||
|
||||
User: Scrape all godot sub-skills in parallel
|
||||
|
||||
Claude: [Starts scraping all 5 configs in parallel...]
|
||||
✅ All skills created in 4-8 hours instead of 20-40!
|
||||
Agent: [Starts scraping all 5 configs in parallel...]
|
||||
✅ All skills created in 4-8 hours instead of 20-40!
|
||||
|
||||
User: Generate router for configs/godot-*.json
|
||||
|
||||
Claude: ✅ Router skill created at output/godot/
|
||||
Agent: ✅ Router skill created at output/godot/
|
||||
Routing logic:
|
||||
- "scripting", "gdscript" → godot-scripting
|
||||
- "2d", "sprites", "tilemap" → godot-2d
|
||||
@@ -395,16 +401,16 @@ Claude: ✅ Router skill created at output/godot/
|
||||
|
||||
User: Package all godot skills
|
||||
|
||||
Claude: ✅ 6 skills packaged:
|
||||
- godot.zip (router)
|
||||
- godot-scripting.zip
|
||||
- godot-2d.zip
|
||||
- godot-3d.zip
|
||||
- godot-physics.zip
|
||||
- godot-shaders.zip
|
||||
Agent: ✅ 6 skills packaged:
|
||||
- godot.zip (router)
|
||||
- godot-scripting.zip
|
||||
- godot-2d.zip
|
||||
- godot-3d.zip
|
||||
- godot-physics.zip
|
||||
- godot-shaders.zip
|
||||
|
||||
Upload all to Claude!
|
||||
Users just ask questions naturally - router handles routing!
|
||||
Upload all to your LLM platform!
|
||||
Users just ask questions naturally - router handles routing!
|
||||
```
|
||||
|
||||
## Architecture
|
||||
@@ -420,11 +426,11 @@ mcp/
|
||||
|
||||
### How It Works
|
||||
|
||||
1. **Claude Code** sends MCP requests to the server
|
||||
1. **AI coding agent** (Claude Code, Cursor, Windsurf, etc.) sends MCP requests to the server
|
||||
2. **Server** routes requests to appropriate tool functions
|
||||
3. **Tools** call CLI scripts (`doc_scraper.py`, `estimate_pages.py`, etc.)
|
||||
4. **CLI scripts** perform actual work (scraping, packaging, etc.)
|
||||
5. **Results** returned to Claude Code via MCP protocol
|
||||
5. **Results** returned to the agent via MCP protocol
|
||||
|
||||
### Tool Implementation
|
||||
|
||||
@@ -482,12 +488,12 @@ python3 -m pytest tests/test_mcp_server.py -v
|
||||
### MCP Server Not Loading
|
||||
|
||||
**Symptoms:**
|
||||
- Tools don't appear in Claude Code
|
||||
- Tools don't appear in your AI coding agent
|
||||
- No response to skill-seeker commands
|
||||
|
||||
**Solutions:**
|
||||
|
||||
1. Check configuration:
|
||||
1. Check configuration (for Claude Code):
|
||||
```bash
|
||||
cat ~/.config/claude-code/mcp.json
|
||||
```
|
||||
@@ -503,11 +509,12 @@ python3 -m pytest tests/test_mcp_server.py -v
|
||||
pip3 install -r mcp/requirements.txt
|
||||
```
|
||||
|
||||
4. Completely restart Claude Code (quit and reopen)
|
||||
4. Completely restart your AI coding agent (quit and reopen)
|
||||
|
||||
5. Check Claude Code logs:
|
||||
- macOS: `~/Library/Logs/Claude Code/`
|
||||
- Linux: `~/.config/claude-code/logs/`
|
||||
5. Check agent logs:
|
||||
- Claude Code (macOS): `~/Library/Logs/Claude Code/`
|
||||
- Claude Code (Linux): `~/.config/claude-code/logs/`
|
||||
- Cursor/Windsurf: Check your editor's output panel for MCP errors
|
||||
|
||||
### "ModuleNotFoundError: No module named 'mcp'"
|
||||
|
||||
@@ -551,7 +558,7 @@ pip install requests beautifulsoup4
|
||||
which python3 # Copy this path
|
||||
```
|
||||
|
||||
Configure Claude Code to use venv Python:
|
||||
Configure your AI coding agent to use venv Python (example for Claude Code):
|
||||
|
||||
```json
|
||||
{
|
||||
|
||||
230
src/skill_seekers/mcp/config_publisher.py
Normal file
230
src/skill_seekers/mcp/config_publisher.py
Normal file
@@ -0,0 +1,230 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Config Publisher
|
||||
Pushes validated config files to registered config source repositories.
|
||||
|
||||
Follows the same pattern as MarketplacePublisher but for configs.
|
||||
"""
|
||||
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import shutil
|
||||
from pathlib import Path
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Category keywords for auto-detection
|
||||
CATEGORY_KEYWORDS = {
|
||||
"game-engines": ["unity", "godot", "unreal", "gamemaker", "spine", "dotween", "addressable"],
|
||||
"web-frameworks": [
|
||||
"react",
|
||||
"vue",
|
||||
"angular",
|
||||
"next",
|
||||
"nuxt",
|
||||
"svelte",
|
||||
"django",
|
||||
"flask",
|
||||
"fastapi",
|
||||
"express",
|
||||
],
|
||||
"ai-ml": ["tensorflow", "pytorch", "langchain", "llama", "openai", "anthropic", "huggingface"],
|
||||
"databases": ["postgres", "mysql", "mongo", "redis", "sqlite", "prisma", "drizzle"],
|
||||
"devops": ["docker", "kubernetes", "terraform", "ansible", "jenkins", "github-actions"],
|
||||
"cloud": ["aws", "gcp", "azure", "vercel", "netlify", "cloudflare"],
|
||||
"mobile": ["flutter", "react-native", "swift", "kotlin", "expo"],
|
||||
"testing": ["jest", "pytest", "cypress", "playwright", "vitest"],
|
||||
"build-tools": ["webpack", "vite", "esbuild", "turbo", "nx"],
|
||||
"css-frameworks": ["tailwind", "bootstrap", "material-ui", "chakra"],
|
||||
"security": ["oauth", "jwt", "auth0", "keycloak"],
|
||||
"development-tools": ["git", "vscode", "neovim", "cursor"],
|
||||
"messaging": ["kafka", "rabbitmq", "nats", "redis-streams"],
|
||||
}
|
||||
|
||||
|
||||
def detect_category(config: dict) -> str:
|
||||
"""Auto-detect category from config name and description.
|
||||
|
||||
Args:
|
||||
config: Parsed config dictionary
|
||||
|
||||
Returns:
|
||||
Category string (e.g., "game-engines") or "custom" if undetected
|
||||
"""
|
||||
name = config.get("name", "").lower()
|
||||
description = config.get("description", "").lower()
|
||||
text = f"{name} {description}"
|
||||
|
||||
best_category = "custom"
|
||||
best_score = 0
|
||||
|
||||
for category, keywords in CATEGORY_KEYWORDS.items():
|
||||
score = sum(1 for kw in keywords if kw in text)
|
||||
if score > best_score:
|
||||
best_score = score
|
||||
best_category = category
|
||||
|
||||
return best_category
|
||||
|
||||
|
||||
class ConfigPublisher:
|
||||
"""Pushes validated configs to registered config source repositories."""
|
||||
|
||||
def __init__(self, cache_dir: str | None = None):
|
||||
"""Initialize publisher.
|
||||
|
||||
Args:
|
||||
cache_dir: Base cache directory. Defaults to ~/.skill-seekers/cache/
|
||||
"""
|
||||
from skill_seekers.mcp.git_repo import GitConfigRepo
|
||||
|
||||
self.git_repo = GitConfigRepo(cache_dir)
|
||||
|
||||
def publish(
|
||||
self,
|
||||
config_path: str | Path,
|
||||
source_name: str,
|
||||
category: str = "auto",
|
||||
create_branch: bool = False,
|
||||
force: bool = False,
|
||||
) -> dict:
|
||||
"""Publish a config to a registered config source repository.
|
||||
|
||||
Args:
|
||||
config_path: Path to config JSON file
|
||||
source_name: Registered source name (e.g., "spyke")
|
||||
category: Category directory (e.g., "game-engines") or "auto" to detect
|
||||
create_branch: Create feature branch instead of committing to main
|
||||
force: Overwrite existing config if it exists
|
||||
|
||||
Returns:
|
||||
Dict with success, config_path, commit_sha, branch, message
|
||||
"""
|
||||
import git
|
||||
|
||||
config_path = Path(config_path)
|
||||
|
||||
# 1. Validate config file exists
|
||||
if not config_path.exists():
|
||||
raise FileNotFoundError(f"Config file not found: {config_path}")
|
||||
|
||||
# 2. Load and validate config
|
||||
with open(config_path, encoding="utf-8") as f:
|
||||
config = json.load(f)
|
||||
|
||||
config_name = config.get("name")
|
||||
if not config_name:
|
||||
raise ValueError("Config JSON must have a 'name' field")
|
||||
|
||||
# Validate config_name to prevent path traversal
|
||||
if "/" in config_name or "\\" in config_name or ".." in config_name:
|
||||
raise ValueError(
|
||||
f"Invalid config name '{config_name}'. "
|
||||
"Path separators (/, \\) and traversal sequences (..) are not allowed."
|
||||
)
|
||||
|
||||
try:
|
||||
from skill_seekers.cli.config_validator import validate_config
|
||||
|
||||
validate_config(str(config_path))
|
||||
logger.info(f"✅ Config validated: {config_name}")
|
||||
except ValueError as e:
|
||||
logger.warning(f"⚠️ Config validation warning: {e}")
|
||||
# Continue — validation warnings shouldn't block push
|
||||
|
||||
# 3. Resolve source from registry
|
||||
from skill_seekers.mcp.source_manager import SourceManager
|
||||
|
||||
manager = SourceManager()
|
||||
source = manager.get_source(source_name)
|
||||
if not source:
|
||||
available = [s["name"] for s in manager.list_sources()]
|
||||
raise ValueError(f"Source '{source_name}' not found. Available sources: {available}")
|
||||
|
||||
git_url = source["git_url"]
|
||||
branch = source.get("branch", "main")
|
||||
token_env = source.get("token_env")
|
||||
|
||||
# 4. Get token
|
||||
token = os.environ.get(token_env) if token_env else None
|
||||
if not token:
|
||||
raise RuntimeError(
|
||||
f"Token not found. Set {token_env} environment variable for source '{source_name}'"
|
||||
)
|
||||
|
||||
# 5. Clone/pull source repo (full clone for push support)
|
||||
cache_name = f"source_{source_name}"
|
||||
repo_path = self.git_repo.cache_dir / cache_name
|
||||
clone_url = self.git_repo.inject_token(git_url, token) if token else git_url
|
||||
|
||||
try:
|
||||
if repo_path.exists() and (repo_path / ".git").exists():
|
||||
repo_obj = git.Repo(repo_path)
|
||||
repo_obj.remotes.origin.pull(branch)
|
||||
logger.info(f"📥 Pulled latest from {source_name}/{branch}")
|
||||
else:
|
||||
repo_obj = git.Repo.clone_from(clone_url, repo_path, branch=branch)
|
||||
logger.info(f"📥 Cloned {source_name} repo")
|
||||
# Clear token from cached .git/config by resetting to non-token URL
|
||||
repo_obj.remotes.origin.set_url(git_url)
|
||||
except git.GitCommandError as e:
|
||||
raise RuntimeError(f"Failed to clone/pull source repo: {e}") from e
|
||||
|
||||
# 6. Auto-detect category
|
||||
if category == "auto":
|
||||
category = detect_category(config)
|
||||
logger.info(f"📂 Auto-detected category: {category}")
|
||||
|
||||
# 7. Check if config already exists
|
||||
target_dir = repo_path / "configs" / category
|
||||
target_file = target_dir / f"{config_name}.json"
|
||||
|
||||
if target_file.exists() and not force:
|
||||
raise ValueError(
|
||||
f"Config '{config_name}' already exists in {source_name}/configs/{category}/. "
|
||||
"Use force=True to overwrite."
|
||||
)
|
||||
|
||||
# 8. Copy config to target directory
|
||||
target_dir.mkdir(parents=True, exist_ok=True)
|
||||
shutil.copy2(config_path, target_file)
|
||||
logger.info(f"📄 Placed config at configs/{category}/{config_name}.json")
|
||||
|
||||
# 9. Git commit and push
|
||||
repo = git.Repo(repo_path)
|
||||
|
||||
target_branch = branch
|
||||
if create_branch:
|
||||
target_branch = f"config/{config_name}"
|
||||
repo.git.checkout("-b", target_branch)
|
||||
|
||||
repo.index.add([str(target_file.relative_to(repo_path))])
|
||||
|
||||
action = "update" if target_file.exists() and force else "add"
|
||||
commit_msg = f"feat: {action} {config_name} config in {category}"
|
||||
commit = repo.index.commit(commit_msg)
|
||||
|
||||
# Push
|
||||
try:
|
||||
repo.remotes.origin.push(target_branch)
|
||||
logger.info(f"🚀 Pushed to {source_name}/{target_branch}")
|
||||
except git.GitCommandError as e:
|
||||
raise RuntimeError(
|
||||
f"Failed to push to {source_name}. Check permissions for {token_env}. Error: {e}"
|
||||
) from e
|
||||
|
||||
# Switch back to main if we created a branch
|
||||
if create_branch:
|
||||
repo.git.checkout(branch)
|
||||
|
||||
return {
|
||||
"success": True,
|
||||
"config_name": config_name,
|
||||
"config_path": f"configs/{category}/{config_name}.json",
|
||||
"source": source_name,
|
||||
"category": category,
|
||||
"commit_sha": str(commit.hexsha)[:8],
|
||||
"branch": target_branch,
|
||||
"message": commit_msg,
|
||||
}
|
||||
196
src/skill_seekers/mcp/marketplace_manager.py
Normal file
196
src/skill_seekers/mcp/marketplace_manager.py
Normal file
@@ -0,0 +1,196 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Marketplace Manager
|
||||
Manages registry of plugin marketplace repositories for skill publishing.
|
||||
"""
|
||||
|
||||
import json
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
class MarketplaceManager:
|
||||
"""Manages marketplace registry at ~/.skill-seekers/marketplaces.json"""
|
||||
|
||||
def __init__(self, config_dir: str | None = None):
|
||||
"""
|
||||
Initialize marketplace manager.
|
||||
|
||||
Args:
|
||||
config_dir: Base config directory. Defaults to ~/.skill-seekers/
|
||||
"""
|
||||
if config_dir:
|
||||
self.config_dir = Path(config_dir)
|
||||
else:
|
||||
self.config_dir = Path.home() / ".skill-seekers"
|
||||
|
||||
self.config_dir.mkdir(parents=True, exist_ok=True)
|
||||
self.registry_file = self.config_dir / "marketplaces.json"
|
||||
|
||||
if not self.registry_file.exists():
|
||||
self._write_registry({"version": "1.0", "marketplaces": []})
|
||||
|
||||
def add_marketplace(
|
||||
self,
|
||||
name: str,
|
||||
git_url: str,
|
||||
token_env: str | None = None,
|
||||
branch: str = "main",
|
||||
author: dict | None = None,
|
||||
enabled: bool = True,
|
||||
) -> dict:
|
||||
"""
|
||||
Add or update a marketplace repository.
|
||||
|
||||
Args:
|
||||
name: Marketplace identifier (lowercase, alphanumeric + hyphens/underscores)
|
||||
git_url: Git repository URL
|
||||
token_env: Environment variable name for auth token
|
||||
branch: Git branch to use (default: main)
|
||||
author: Default author for plugin.json ({"name": str, "email": str})
|
||||
enabled: Whether marketplace is enabled (default: True)
|
||||
|
||||
Returns:
|
||||
Marketplace dictionary
|
||||
|
||||
Raises:
|
||||
ValueError: If name is invalid or git_url is empty
|
||||
"""
|
||||
if not name or not name.replace("-", "").replace("_", "").isalnum():
|
||||
raise ValueError(
|
||||
f"Invalid marketplace name '{name}'. "
|
||||
"Must be alphanumeric with optional hyphens/underscores."
|
||||
)
|
||||
|
||||
if not git_url or not git_url.strip():
|
||||
raise ValueError("git_url cannot be empty")
|
||||
|
||||
if token_env is None:
|
||||
token_env = self._default_token_env(git_url)
|
||||
|
||||
now = datetime.now(timezone.utc).isoformat()
|
||||
marketplace = {
|
||||
"name": name.lower(),
|
||||
"git_url": git_url.strip(),
|
||||
"token_env": token_env,
|
||||
"branch": branch,
|
||||
"author": author or {"name": "", "email": ""},
|
||||
"enabled": enabled,
|
||||
"added_at": now,
|
||||
"updated_at": now,
|
||||
}
|
||||
|
||||
registry = self._read_registry()
|
||||
|
||||
existing_index = None
|
||||
for i, existing in enumerate(registry["marketplaces"]):
|
||||
if existing["name"] == marketplace["name"]:
|
||||
existing_index = i
|
||||
marketplace["added_at"] = existing.get("added_at", marketplace["added_at"])
|
||||
break
|
||||
|
||||
if existing_index is not None:
|
||||
registry["marketplaces"][existing_index] = marketplace
|
||||
else:
|
||||
registry["marketplaces"].append(marketplace)
|
||||
|
||||
self._write_registry(registry)
|
||||
return marketplace
|
||||
|
||||
def get_marketplace(self, name: str) -> dict:
|
||||
"""
|
||||
Get marketplace by name.
|
||||
|
||||
Raises:
|
||||
KeyError: If marketplace not found
|
||||
"""
|
||||
registry = self._read_registry()
|
||||
name_lower = name.lower()
|
||||
for marketplace in registry["marketplaces"]:
|
||||
if marketplace["name"] == name_lower:
|
||||
return marketplace
|
||||
|
||||
available = [m["name"] for m in registry["marketplaces"]]
|
||||
raise KeyError(
|
||||
f"Marketplace '{name}' not found. "
|
||||
f"Available marketplaces: {', '.join(available) if available else 'none'}"
|
||||
)
|
||||
|
||||
def list_marketplaces(self, enabled_only: bool = False) -> list[dict]:
|
||||
"""List all marketplaces."""
|
||||
registry = self._read_registry()
|
||||
if enabled_only:
|
||||
return [m for m in registry["marketplaces"] if m.get("enabled", True)]
|
||||
return registry["marketplaces"]
|
||||
|
||||
def remove_marketplace(self, name: str) -> bool:
|
||||
"""Remove marketplace by name. Returns True if removed."""
|
||||
registry = self._read_registry()
|
||||
name_lower = name.lower()
|
||||
for i, marketplace in enumerate(registry["marketplaces"]):
|
||||
if marketplace["name"] == name_lower:
|
||||
del registry["marketplaces"][i]
|
||||
self._write_registry(registry)
|
||||
return True
|
||||
return False
|
||||
|
||||
def update_marketplace(self, name: str, **kwargs) -> dict:
|
||||
"""
|
||||
Update specific fields of an existing marketplace.
|
||||
|
||||
Raises:
|
||||
KeyError: If marketplace not found
|
||||
"""
|
||||
marketplace = self.get_marketplace(name)
|
||||
allowed_fields = {"git_url", "token_env", "branch", "author", "enabled"}
|
||||
for field, value in kwargs.items():
|
||||
if field in allowed_fields:
|
||||
marketplace[field] = value
|
||||
|
||||
marketplace["updated_at"] = datetime.now(timezone.utc).isoformat()
|
||||
|
||||
registry = self._read_registry()
|
||||
for i, m in enumerate(registry["marketplaces"]):
|
||||
if m["name"] == marketplace["name"]:
|
||||
registry["marketplaces"][i] = marketplace
|
||||
break
|
||||
|
||||
self._write_registry(registry)
|
||||
return marketplace
|
||||
|
||||
def _read_registry(self) -> dict:
|
||||
"""Read registry from file."""
|
||||
try:
|
||||
with open(self.registry_file, encoding="utf-8") as f:
|
||||
return json.load(f)
|
||||
except json.JSONDecodeError as e:
|
||||
raise ValueError(f"Corrupted registry file: {e}") from e
|
||||
|
||||
def _write_registry(self, registry: dict) -> None:
|
||||
"""Write registry to file atomically."""
|
||||
if "version" not in registry or "marketplaces" not in registry:
|
||||
raise ValueError("Invalid registry schema")
|
||||
|
||||
temp_file = self.registry_file.with_suffix(".tmp")
|
||||
try:
|
||||
with open(temp_file, "w", encoding="utf-8") as f:
|
||||
json.dump(registry, f, indent=2, ensure_ascii=False)
|
||||
temp_file.replace(self.registry_file)
|
||||
except Exception as e:
|
||||
if temp_file.exists():
|
||||
temp_file.unlink()
|
||||
raise e
|
||||
|
||||
@staticmethod
|
||||
def _default_token_env(git_url: str) -> str:
|
||||
"""Get default token environment variable name from git URL."""
|
||||
url_lower = git_url.lower()
|
||||
if "github" in url_lower:
|
||||
return "GITHUB_TOKEN"
|
||||
elif "gitlab" in url_lower:
|
||||
return "GITLAB_TOKEN"
|
||||
elif "bitbucket" in url_lower:
|
||||
return "BITBUCKET_TOKEN"
|
||||
elif "gitea" in url_lower:
|
||||
return "GITEA_TOKEN"
|
||||
return "GIT_TOKEN"
|
||||
291
src/skill_seekers/mcp/marketplace_publisher.py
Normal file
291
src/skill_seekers/mcp/marketplace_publisher.py
Normal file
@@ -0,0 +1,291 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Marketplace Publisher
|
||||
Publishes packaged skills to Claude Code plugin marketplace repositories.
|
||||
"""
|
||||
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import re
|
||||
import shutil
|
||||
from pathlib import Path
|
||||
|
||||
import yaml
|
||||
|
||||
from skill_seekers.mcp.marketplace_manager import MarketplaceManager
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class MarketplacePublisher:
|
||||
"""Publishes skills to registered plugin marketplace repositories."""
|
||||
|
||||
def __init__(self, cache_dir: str | None = None):
|
||||
"""
|
||||
Initialize publisher.
|
||||
|
||||
Args:
|
||||
cache_dir: Base cache directory. Defaults to ~/.skill-seekers/cache/
|
||||
"""
|
||||
from skill_seekers.mcp.git_repo import GitConfigRepo
|
||||
|
||||
self.git_repo = GitConfigRepo(cache_dir)
|
||||
|
||||
def publish(
|
||||
self,
|
||||
skill_dir: str | Path,
|
||||
marketplace_name: str,
|
||||
category: str = "development",
|
||||
skill_name: str | None = None,
|
||||
description: str | None = None,
|
||||
create_branch: bool = False,
|
||||
force: bool = False,
|
||||
) -> dict:
|
||||
"""
|
||||
Publish a skill to a plugin marketplace repository.
|
||||
|
||||
Args:
|
||||
skill_dir: Path to skill directory (contains SKILL.md + references/)
|
||||
marketplace_name: Registered marketplace name
|
||||
category: Plugin category in marketplace (default: "development")
|
||||
skill_name: Override skill name (defaults to SKILL.md frontmatter or dir name)
|
||||
description: Override description (defaults to SKILL.md frontmatter)
|
||||
create_branch: Create feature branch instead of committing to main
|
||||
force: Overwrite existing plugin if it exists
|
||||
|
||||
Returns:
|
||||
Dict with success, plugin_path, commit_sha, branch, message
|
||||
"""
|
||||
import git
|
||||
|
||||
skill_dir = Path(skill_dir)
|
||||
|
||||
# 1. Validate skill directory
|
||||
skill_md_path = skill_dir / "SKILL.md"
|
||||
if not skill_md_path.exists():
|
||||
raise FileNotFoundError(f"SKILL.md not found in {skill_dir}")
|
||||
|
||||
# 2. Read frontmatter for metadata
|
||||
frontmatter = self._read_frontmatter(skill_md_path)
|
||||
if not skill_name:
|
||||
skill_name = frontmatter.get("name") or skill_dir.name
|
||||
if not description:
|
||||
description = frontmatter.get("description", f"Skill for {skill_name}")
|
||||
|
||||
# 2b. Validate skill_name to prevent path traversal
|
||||
skill_name = self._validate_skill_name(skill_name)
|
||||
|
||||
# 3. Resolve marketplace
|
||||
manager = MarketplaceManager()
|
||||
marketplace = manager.get_marketplace(marketplace_name)
|
||||
git_url = marketplace["git_url"]
|
||||
branch = marketplace["branch"]
|
||||
token_env = marketplace["token_env"]
|
||||
author = marketplace.get("author", {"name": "", "email": ""})
|
||||
|
||||
# 4. Get token
|
||||
token = os.environ.get(token_env) if token_env else None
|
||||
if not token:
|
||||
raise RuntimeError(
|
||||
f"Token not found. Set {token_env} environment variable "
|
||||
f"for marketplace '{marketplace_name}'"
|
||||
)
|
||||
|
||||
# 5. Clone/pull marketplace repo (full clone, not shallow — needed for push)
|
||||
cache_name = f"marketplace_{marketplace_name}"
|
||||
repo_path = self.git_repo.cache_dir / cache_name
|
||||
clone_url = self.git_repo.inject_token(git_url, token) if token else git_url
|
||||
try:
|
||||
if repo_path.exists() and (repo_path / ".git").exists():
|
||||
repo_obj = git.Repo(repo_path)
|
||||
repo_obj.remotes.origin.pull(branch)
|
||||
else:
|
||||
repo_obj = git.Repo.clone_from(clone_url, repo_path, branch=branch)
|
||||
# Clear token from cached .git/config by resetting to non-token URL
|
||||
repo_obj.remotes.origin.set_url(git_url)
|
||||
except git.GitCommandError as e:
|
||||
raise RuntimeError(f"Failed to clone/pull marketplace repo: {e}") from e
|
||||
|
||||
# 6. Check for existing plugin
|
||||
plugin_dir = repo_path / "plugins" / skill_name
|
||||
if plugin_dir.exists() and not force:
|
||||
raise ValueError(
|
||||
f"Plugin '{skill_name}' already exists in marketplace '{marketplace_name}'. "
|
||||
"Use force=True to overwrite."
|
||||
)
|
||||
|
||||
# 7. Create plugin directory structure + commit + push
|
||||
# Wrap in try/finally to clean up partial plugin dir on failure
|
||||
plugin_created = False
|
||||
try:
|
||||
self._copy_skill_to_plugin(skill_dir, plugin_dir, skill_name)
|
||||
plugin_created = True
|
||||
|
||||
# 8. Generate plugin.json
|
||||
plugin_json = self._generate_plugin_json(skill_name, description, author)
|
||||
plugin_json_dir = plugin_dir / ".claude-plugin"
|
||||
plugin_json_dir.mkdir(parents=True, exist_ok=True)
|
||||
with open(plugin_json_dir / "plugin.json", "w", encoding="utf-8") as f:
|
||||
json.dump(plugin_json, f, indent=2, ensure_ascii=False)
|
||||
|
||||
# 9. Update marketplace.json
|
||||
self._update_marketplace_json(repo_path, skill_name, description, author, category)
|
||||
|
||||
# 10. Git commit and push
|
||||
repo = git.Repo(repo_path)
|
||||
|
||||
target_branch = branch
|
||||
if create_branch:
|
||||
target_branch = f"skill/{skill_name}"
|
||||
repo.git.checkout("-b", target_branch)
|
||||
|
||||
# Only stage the specific files we wrote (not the entire repo)
|
||||
files_to_stage = []
|
||||
# Stage the plugin directory (skill files + plugin.json)
|
||||
plugin_rel = str(plugin_dir.relative_to(repo_path))
|
||||
files_to_stage.append(plugin_rel)
|
||||
# Stage marketplace.json
|
||||
marketplace_json_rel = str(
|
||||
(repo_path / ".claude-plugin" / "marketplace.json").relative_to(repo_path)
|
||||
)
|
||||
files_to_stage.append(marketplace_json_rel)
|
||||
repo.index.add(files_to_stage)
|
||||
|
||||
action = "update" if force else "add"
|
||||
commit_msg = f"feat: {action} {skill_name} skill plugin"
|
||||
repo.index.commit(commit_msg)
|
||||
commit_sha = repo.head.commit.hexsha[:7]
|
||||
|
||||
push_url = self.git_repo.inject_token(git_url, token)
|
||||
repo.git.push(push_url, target_branch)
|
||||
|
||||
return {
|
||||
"success": True,
|
||||
"plugin_path": f"plugins/{skill_name}",
|
||||
"commit_sha": commit_sha,
|
||||
"branch": target_branch,
|
||||
"message": f"Published '{skill_name}' to marketplace '{marketplace_name}'",
|
||||
}
|
||||
|
||||
except git.GitCommandError as e:
|
||||
# Reset git state on push/commit failure
|
||||
try:
|
||||
repo = git.Repo(repo_path)
|
||||
repo.git.checkout("--", ".")
|
||||
repo.git.clean("-fd", "plugins/" + skill_name)
|
||||
except Exception:
|
||||
pass
|
||||
raise RuntimeError(f"Git operation failed: {e}") from e
|
||||
except Exception:
|
||||
# Clean up partial plugin directory on non-git failure
|
||||
if plugin_created and plugin_dir.exists():
|
||||
shutil.rmtree(plugin_dir, ignore_errors=True)
|
||||
raise
|
||||
|
||||
@staticmethod
|
||||
def _validate_skill_name(name: str) -> str:
|
||||
"""
|
||||
Validate skill name to prevent path traversal and injection.
|
||||
|
||||
Args:
|
||||
name: Skill name to validate
|
||||
|
||||
Returns:
|
||||
Validated skill name
|
||||
|
||||
Raises:
|
||||
ValueError: If name contains invalid characters
|
||||
"""
|
||||
if not name or not re.match(r"^[a-zA-Z0-9][a-zA-Z0-9._-]*$", name):
|
||||
raise ValueError(
|
||||
f"Invalid skill name '{name}'. "
|
||||
"Must start with alphanumeric and contain only alphanumeric, hyphens, underscores, or dots."
|
||||
)
|
||||
if ".." in name or "/" in name or "\\" in name:
|
||||
raise ValueError(f"Invalid skill name '{name}'. Path traversal characters not allowed.")
|
||||
return name
|
||||
|
||||
def _read_frontmatter(self, skill_md_path: Path) -> dict:
|
||||
"""Parse YAML frontmatter from SKILL.md."""
|
||||
content = skill_md_path.read_text(encoding="utf-8")
|
||||
if not content.startswith("---"):
|
||||
return {}
|
||||
|
||||
parts = content.split("---", 2)
|
||||
if len(parts) < 3:
|
||||
return {}
|
||||
|
||||
try:
|
||||
return yaml.safe_load(parts[1]) or {}
|
||||
except yaml.YAMLError:
|
||||
return {}
|
||||
|
||||
def _copy_skill_to_plugin(self, skill_dir: Path, plugin_dir: Path, skill_name: str) -> None:
|
||||
"""Copy skill files into plugin directory structure."""
|
||||
skills_dest = plugin_dir / "skills" / skill_name
|
||||
|
||||
if skills_dest.exists():
|
||||
shutil.rmtree(skills_dest)
|
||||
|
||||
skills_dest.mkdir(parents=True, exist_ok=True)
|
||||
shutil.copy2(skill_dir / "SKILL.md", skills_dest / "SKILL.md")
|
||||
|
||||
refs_src = skill_dir / "references"
|
||||
if refs_src.exists() and refs_src.is_dir():
|
||||
shutil.copytree(refs_src, skills_dest / "references", dirs_exist_ok=True)
|
||||
|
||||
def _generate_plugin_json(self, skill_name: str, description: str, author: dict) -> dict:
|
||||
"""Generate plugin.json content."""
|
||||
return {
|
||||
"name": skill_name,
|
||||
"description": description,
|
||||
"author": author,
|
||||
}
|
||||
|
||||
def _update_marketplace_json(
|
||||
self,
|
||||
repo_path: Path,
|
||||
skill_name: str,
|
||||
description: str,
|
||||
author: dict,
|
||||
category: str,
|
||||
) -> None:
|
||||
"""Update root .claude-plugin/marketplace.json with new plugin entry."""
|
||||
marketplace_json_path = repo_path / ".claude-plugin" / "marketplace.json"
|
||||
|
||||
if marketplace_json_path.exists():
|
||||
with open(marketplace_json_path, encoding="utf-8") as f:
|
||||
data = json.load(f)
|
||||
else:
|
||||
marketplace_json_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
data = {
|
||||
"$schema": "https://anthropic.com/claude-code/marketplace.schema.json",
|
||||
"name": repo_path.name,
|
||||
"description": "",
|
||||
"owner": author,
|
||||
"plugins": [],
|
||||
}
|
||||
|
||||
entry = {
|
||||
"name": skill_name,
|
||||
"description": description,
|
||||
"author": author,
|
||||
"source": f"./plugins/{skill_name}",
|
||||
"category": category,
|
||||
}
|
||||
|
||||
updated = False
|
||||
for i, plugin in enumerate(data["plugins"]):
|
||||
if plugin["name"] == skill_name:
|
||||
data["plugins"][i] = entry
|
||||
updated = True
|
||||
break
|
||||
|
||||
if not updated:
|
||||
data["plugins"].append(entry)
|
||||
|
||||
data["plugins"].sort(key=lambda p: p["name"])
|
||||
|
||||
with open(marketplace_json_path, "w", encoding="utf-8") as f:
|
||||
json.dump(data, f, indent=2, ensure_ascii=False)
|
||||
@@ -3,7 +3,7 @@
|
||||
Skill Seeker MCP Server (FastMCP Implementation)
|
||||
|
||||
Modern, decorator-based MCP server using FastMCP for simplified tool registration.
|
||||
Provides 34 tools for generating Claude AI skills from documentation.
|
||||
Provides 34 tools for generating LLM skills from documentation.
|
||||
|
||||
This is a streamlined alternative to server.py (2200 lines → 708 lines, 68% reduction).
|
||||
All tool implementations are delegated to modular tool files in tools/ directory.
|
||||
@@ -15,7 +15,8 @@ All tool implementations are delegated to modular tool files in tools/ directory
|
||||
* Scraping tools (11): estimate_pages, scrape_docs, scrape_github, scrape_pdf, scrape_video, scrape_codebase, detect_patterns, extract_test_examples, build_how_to_guides, extract_config_patterns, scrape_generic
|
||||
* Packaging tools (4): package_skill, upload_skill, enhance_skill, install_skill
|
||||
* Splitting tools (2): split_config, generate_router
|
||||
* Source tools (5): fetch_config, submit_config, add_config_source, list_config_sources, remove_config_source
|
||||
* Source tools (6): fetch_config, submit_config, push_config, add_config_source, list_config_sources, remove_config_source
|
||||
* Marketplace tools (4): add_marketplace, list_marketplaces, remove_marketplace, publish_to_marketplace
|
||||
* Vector Database tools (4): export_to_weaviate, export_to_chroma, export_to_faiss, export_to_qdrant
|
||||
* Workflow tools (5): list_workflows, get_workflow, create_workflow, update_workflow, delete_workflow
|
||||
|
||||
@@ -86,6 +87,11 @@ try:
|
||||
extract_test_examples_impl,
|
||||
# Source tools
|
||||
fetch_config_impl,
|
||||
# Marketplace tools
|
||||
add_marketplace_impl,
|
||||
list_marketplaces_impl,
|
||||
remove_marketplace_impl,
|
||||
publish_to_marketplace_impl,
|
||||
# Config tools
|
||||
generate_config_impl,
|
||||
generate_router_impl,
|
||||
@@ -133,6 +139,10 @@ except ImportError:
|
||||
extract_config_patterns_impl,
|
||||
extract_test_examples_impl,
|
||||
fetch_config_impl,
|
||||
add_marketplace_impl,
|
||||
list_marketplaces_impl,
|
||||
remove_marketplace_impl,
|
||||
publish_to_marketplace_impl,
|
||||
generate_config_impl,
|
||||
generate_router_impl,
|
||||
install_skill_impl,
|
||||
@@ -163,7 +173,7 @@ mcp = None
|
||||
if MCP_AVAILABLE and FastMCP is not None:
|
||||
mcp = FastMCP(
|
||||
name="skill-seeker",
|
||||
instructions="Skill Seeker MCP Server - Generate Claude AI skills from documentation",
|
||||
instructions="Skill Seeker MCP Server - Generate LLM skills from documentation",
|
||||
)
|
||||
|
||||
|
||||
@@ -338,7 +348,7 @@ async def estimate_pages(
|
||||
|
||||
|
||||
@safe_tool_decorator(
|
||||
description="Scrape documentation and build Claude skill. Supports both single-source (legacy) and unified multi-source configs. Creates SKILL.md and reference files. Automatically detects llms.txt files for 10x faster processing. Falls back to HTML scraping if not available."
|
||||
description="Scrape documentation and build LLM skill. Supports both single-source (legacy) and unified multi-source configs. Creates SKILL.md and reference files. Automatically detects llms.txt files for 10x faster processing. Falls back to HTML scraping if not available."
|
||||
)
|
||||
async def scrape_docs(
|
||||
config_path: str,
|
||||
@@ -349,12 +359,12 @@ async def scrape_docs(
|
||||
merge_mode: str | None = None,
|
||||
) -> str:
|
||||
"""
|
||||
Scrape documentation and build Claude skill.
|
||||
Scrape documentation and build LLM skill.
|
||||
|
||||
Args:
|
||||
config_path: Path to config JSON file (e.g., configs/react.json or configs/godot_unified.json)
|
||||
unlimited: Remove page limit - scrape all pages (default: false). Overrides max_pages in config.
|
||||
enhance_local: Open terminal for local enhancement with Claude Code (default: false)
|
||||
enhance_local: Open terminal for local enhancement with AI coding agent (default: false)
|
||||
skip_scrape: Skip scraping, use cached data (default: false)
|
||||
dry_run: Preview what will be scraped without saving (default: false)
|
||||
merge_mode: Override merge mode for unified configs: 'rule-based' or 'claude-enhanced' (default: from config)
|
||||
@@ -879,7 +889,7 @@ async def scrape_generic(
|
||||
)
|
||||
async def package_skill(
|
||||
skill_dir: str,
|
||||
target: str = "claude",
|
||||
target: str = "auto",
|
||||
auto_upload: bool = True,
|
||||
) -> str:
|
||||
"""
|
||||
@@ -887,12 +897,16 @@ async def package_skill(
|
||||
|
||||
Args:
|
||||
skill_dir: Path to skill directory to package (e.g., output/react/)
|
||||
target: Target platform (default: 'claude'). Options: claude, gemini, openai, markdown
|
||||
target: Target platform (default: 'auto'). Options: auto, claude, gemini, openai, markdown
|
||||
auto_upload: Auto-upload after packaging if API key is available (default: true). Requires platform-specific API key: ANTHROPIC_API_KEY, GOOGLE_API_KEY, or OPENAI_API_KEY.
|
||||
|
||||
Returns:
|
||||
Packaging results with file path and platform info.
|
||||
"""
|
||||
if target == "auto":
|
||||
from skill_seekers.cli.agent_client import AgentClient
|
||||
|
||||
target = AgentClient.detect_default_target()
|
||||
args = {
|
||||
"skill_dir": skill_dir,
|
||||
"target": target,
|
||||
@@ -909,7 +923,7 @@ async def package_skill(
|
||||
)
|
||||
async def upload_skill(
|
||||
skill_zip: str,
|
||||
target: str = "claude",
|
||||
target: str = "auto",
|
||||
api_key: str | None = None,
|
||||
) -> str:
|
||||
"""
|
||||
@@ -917,12 +931,16 @@ async def upload_skill(
|
||||
|
||||
Args:
|
||||
skill_zip: Path to skill package (.zip or .tar.gz, e.g., output/react.zip)
|
||||
target: Target platform (default: 'claude'). Options: claude, gemini, openai
|
||||
target: Target platform (default: 'auto'). Options: auto, claude, gemini, openai
|
||||
api_key: Optional API key (uses env var if not provided: ANTHROPIC_API_KEY, GOOGLE_API_KEY, or OPENAI_API_KEY)
|
||||
|
||||
Returns:
|
||||
Upload results with skill ID and platform URL.
|
||||
"""
|
||||
if target == "auto":
|
||||
from skill_seekers.cli.agent_client import AgentClient
|
||||
|
||||
target = AgentClient.detect_default_target()
|
||||
args = {
|
||||
"skill_zip": skill_zip,
|
||||
"target": target,
|
||||
@@ -937,11 +955,11 @@ async def upload_skill(
|
||||
|
||||
|
||||
@safe_tool_decorator(
|
||||
description="Enhance SKILL.md with AI using target platform's model. Local mode uses Claude Code Max (no API key). API mode uses platform API (requires key). Transforms basic templates into comprehensive 500+ line guides with examples."
|
||||
description="Enhance SKILL.md with AI using target platform's model. Local mode uses AI coding agent (no API key). API mode uses platform API (requires key). Transforms basic templates into comprehensive 500+ line guides with examples."
|
||||
)
|
||||
async def enhance_skill(
|
||||
skill_dir: str,
|
||||
target: str = "claude",
|
||||
target: str = "auto",
|
||||
mode: str = "local",
|
||||
api_key: str | None = None,
|
||||
) -> str:
|
||||
@@ -950,13 +968,17 @@ async def enhance_skill(
|
||||
|
||||
Args:
|
||||
skill_dir: Path to skill directory containing SKILL.md (e.g., output/react/)
|
||||
target: Target platform (default: 'claude'). Options: claude, gemini, openai
|
||||
mode: Enhancement mode (default: 'local'). Options: local (Claude Code, no API), api (uses platform API)
|
||||
target: Target platform (default: 'auto'). Options: auto, claude, gemini, openai
|
||||
mode: Enhancement mode (default: 'local'). Options: local (AI coding agent, no API), api (uses platform API)
|
||||
api_key: Optional API key for 'api' mode (uses env var if not provided: ANTHROPIC_API_KEY, GOOGLE_API_KEY, or OPENAI_API_KEY)
|
||||
|
||||
Returns:
|
||||
Enhancement results with backup location.
|
||||
"""
|
||||
if target == "auto":
|
||||
from skill_seekers.cli.agent_client import AgentClient
|
||||
|
||||
target = AgentClient.detect_default_target()
|
||||
args = {
|
||||
"skill_dir": skill_dir,
|
||||
"target": target,
|
||||
@@ -972,7 +994,7 @@ async def enhance_skill(
|
||||
|
||||
|
||||
@safe_tool_decorator(
|
||||
description="Complete one-command workflow: fetch config → scrape docs → AI enhance (MANDATORY) → package → upload. Enhancement required for quality (3/10→9/10). Takes 20-45 min depending on config size. Supports multiple LLM platforms: claude (default), gemini, openai, markdown. Auto-uploads if platform API key is set."
|
||||
description="Complete one-command workflow: fetch config → scrape docs → AI enhance (MANDATORY) → package → upload. Enhancement required for quality (3/10→9/10). Takes 20-45 min depending on config size. Supports multiple LLM platforms: auto (detects from environment), claude, gemini, openai, markdown. Auto-uploads if platform API key is set."
|
||||
)
|
||||
async def install_skill(
|
||||
config_name: str | None = None,
|
||||
@@ -981,7 +1003,7 @@ async def install_skill(
|
||||
auto_upload: bool = True,
|
||||
unlimited: bool = False,
|
||||
dry_run: bool = False,
|
||||
target: str = "claude",
|
||||
target: str = "auto",
|
||||
) -> str:
|
||||
"""
|
||||
Complete one-command workflow to install a skill.
|
||||
@@ -993,11 +1015,15 @@ async def install_skill(
|
||||
auto_upload: Auto-upload after packaging (requires platform API key). Default: true. Set to false to skip upload.
|
||||
unlimited: Remove page limits during scraping (default: false). WARNING: Can take hours for large sites.
|
||||
dry_run: Preview workflow without executing (default: false). Shows all phases that would run.
|
||||
target: Target LLM platform (default: 'claude'). Options: claude, gemini, openai, markdown. Requires corresponding API key: ANTHROPIC_API_KEY, GOOGLE_API_KEY, or OPENAI_API_KEY.
|
||||
target: Target LLM platform (default: 'auto'). Options: auto, claude, gemini, openai, markdown. Requires corresponding API key: ANTHROPIC_API_KEY, GOOGLE_API_KEY, or OPENAI_API_KEY.
|
||||
|
||||
Returns:
|
||||
Workflow results with all phase statuses.
|
||||
"""
|
||||
if target == "auto":
|
||||
from skill_seekers.cli.agent_client import AgentClient
|
||||
|
||||
target = AgentClient.detect_default_target()
|
||||
args = {
|
||||
"destination": destination,
|
||||
"auto_upload": auto_upload,
|
||||
@@ -1263,6 +1289,150 @@ async def remove_config_source(name: str) -> str:
|
||||
return str(result)
|
||||
|
||||
|
||||
@safe_tool_decorator(
|
||||
description="Push a config to a registered config source repository. Validates, places in category directory, commits, and pushes."
|
||||
)
|
||||
async def push_config(
|
||||
config_path: str,
|
||||
source_name: str,
|
||||
category: str = "auto",
|
||||
create_branch: bool = False,
|
||||
force: bool = False,
|
||||
) -> str:
|
||||
"""
|
||||
Push a config to a registered config source repository.
|
||||
|
||||
Args:
|
||||
config_path: Path to config JSON file. Example: 'configs/unity-spine.json'
|
||||
source_name: Registered source name. Example: 'spyke'
|
||||
category: Category directory (e.g., 'game-engines'). Auto-detected if 'auto'.
|
||||
create_branch: Create feature branch instead of pushing to main. Default: false
|
||||
force: Overwrite existing config if it exists. Default: false
|
||||
|
||||
Returns:
|
||||
Push results with commit SHA and config location.
|
||||
"""
|
||||
from skill_seekers.mcp.tools.source_tools import push_config_tool
|
||||
|
||||
result = await push_config_tool(
|
||||
{
|
||||
"config_path": config_path,
|
||||
"source_name": source_name,
|
||||
"category": category,
|
||||
"create_branch": create_branch,
|
||||
"force": force,
|
||||
}
|
||||
)
|
||||
if isinstance(result, list) and result:
|
||||
return result[0].text if hasattr(result[0], "text") else str(result[0])
|
||||
return str(result)
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# MARKETPLACE TOOLS (4 tools)
|
||||
# ============================================================================
|
||||
|
||||
|
||||
@safe_tool_decorator(
|
||||
description="Register a plugin marketplace repository. Allows publishing skills to private/team plugin repos. Supports GitHub, GitLab, Bitbucket with per-repo authentication."
|
||||
)
|
||||
async def add_marketplace(
|
||||
name: str,
|
||||
git_url: str,
|
||||
token_env: str = None,
|
||||
branch: str = "main",
|
||||
author_name: str = "",
|
||||
author_email: str = "",
|
||||
enabled: bool = True,
|
||||
) -> str:
|
||||
"""
|
||||
Register a plugin marketplace repository.
|
||||
|
||||
Args:
|
||||
name: Marketplace identifier (lowercase, alphanumeric + hyphens/underscores). Example: 'spyke'
|
||||
git_url: Git repository URL. Example: 'https://github.com/myorg/plugins.git'
|
||||
token_env: Environment variable name for auth token (auto-detected from URL). Example: 'GITHUB_TOKEN'
|
||||
branch: Git branch to use (default: "main")
|
||||
author_name: Default author name for generated plugin.json files
|
||||
author_email: Default author email for generated plugin.json files
|
||||
enabled: Whether marketplace is enabled (default: true)
|
||||
"""
|
||||
result = await add_marketplace_impl(
|
||||
{
|
||||
"name": name,
|
||||
"git_url": git_url,
|
||||
"token_env": token_env,
|
||||
"branch": branch,
|
||||
"author_name": author_name,
|
||||
"author_email": author_email,
|
||||
"enabled": enabled,
|
||||
}
|
||||
)
|
||||
if isinstance(result, list) and result:
|
||||
return result[0].text if hasattr(result[0], "text") else str(result[0])
|
||||
return str(result)
|
||||
|
||||
|
||||
@safe_tool_decorator(description="List all registered plugin marketplace repositories.")
|
||||
async def list_marketplaces(enabled_only: bool = False) -> str:
|
||||
"""List all registered plugin marketplace repositories."""
|
||||
result = await list_marketplaces_impl({"enabled_only": enabled_only})
|
||||
if isinstance(result, list) and result:
|
||||
return result[0].text if hasattr(result[0], "text") else str(result[0])
|
||||
return str(result)
|
||||
|
||||
|
||||
@safe_tool_decorator(
|
||||
description="Remove a registered plugin marketplace. Deletes from registry but not cached data."
|
||||
)
|
||||
async def remove_marketplace(name: str) -> str:
|
||||
"""Remove a registered plugin marketplace."""
|
||||
result = await remove_marketplace_impl({"name": name})
|
||||
if isinstance(result, list) and result:
|
||||
return result[0].text if hasattr(result[0], "text") else str(result[0])
|
||||
return str(result)
|
||||
|
||||
|
||||
@safe_tool_decorator(
|
||||
description="Publish a packaged skill to a plugin marketplace repository. Creates a Claude Code plugin in the target marketplace repo."
|
||||
)
|
||||
async def publish_to_marketplace(
|
||||
skill_dir: str,
|
||||
marketplace: str,
|
||||
category: str = "development",
|
||||
skill_name: str = None,
|
||||
description: str = None,
|
||||
create_branch: bool = False,
|
||||
force: bool = False,
|
||||
) -> str:
|
||||
"""
|
||||
Publish a skill to a plugin marketplace repository.
|
||||
|
||||
Args:
|
||||
skill_dir: Path to skill directory containing SKILL.md. Example: 'output/react/'
|
||||
marketplace: Registered marketplace name. Example: 'spyke'
|
||||
category: Plugin category (default: "development")
|
||||
skill_name: Override skill name (optional)
|
||||
description: Override description (optional)
|
||||
create_branch: Create feature branch instead of committing to main (default: false)
|
||||
force: Overwrite existing plugin (default: false)
|
||||
"""
|
||||
result = await publish_to_marketplace_impl(
|
||||
{
|
||||
"skill_dir": skill_dir,
|
||||
"marketplace": marketplace,
|
||||
"category": category,
|
||||
"skill_name": skill_name,
|
||||
"description": description,
|
||||
"create_branch": create_branch,
|
||||
"force": force,
|
||||
}
|
||||
)
|
||||
if isinstance(result, list) and result:
|
||||
return result[0].text if hasattr(result[0], "text") else str(result[0])
|
||||
return str(result)
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# VECTOR DATABASE TOOLS (4 tools)
|
||||
# ============================================================================
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Skill Seeker MCP Server
|
||||
Model Context Protocol server for generating Claude AI skills from documentation
|
||||
Model Context Protocol server for generating LLM skills from documentation
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
@@ -200,7 +200,7 @@ async def list_tools() -> list[Tool]:
|
||||
),
|
||||
Tool(
|
||||
name="scrape_docs",
|
||||
description="Scrape documentation and build Claude skill. Supports both single-source (legacy) and unified multi-source configs. Creates SKILL.md and reference files. Automatically detects llms.txt files for 10x faster processing. Falls back to HTML scraping if not available.",
|
||||
description="Scrape documentation and build LLM skill. Supports both single-source (legacy) and unified multi-source configs. Creates SKILL.md and reference files. Automatically detects llms.txt files for 10x faster processing. Falls back to HTML scraping if not available.",
|
||||
inputSchema={
|
||||
"type": "object",
|
||||
"properties": {
|
||||
@@ -215,7 +215,7 @@ async def list_tools() -> list[Tool]:
|
||||
},
|
||||
"enhance_local": {
|
||||
"type": "boolean",
|
||||
"description": "Open terminal for local enhancement with Claude Code (default: false)",
|
||||
"description": "Open terminal for local enhancement with AI coding agent (default: false)",
|
||||
"default": False,
|
||||
},
|
||||
"skip_scrape": {
|
||||
@@ -230,7 +230,7 @@ async def list_tools() -> list[Tool]:
|
||||
},
|
||||
"merge_mode": {
|
||||
"type": "string",
|
||||
"description": "Override merge mode for unified configs: 'rule-based' or 'claude-enhanced' (default: from config)",
|
||||
"description": "Override merge mode for unified configs: 'rule-based' or 'ai-enhanced' (default: from config)",
|
||||
},
|
||||
},
|
||||
"required": ["config_path"],
|
||||
@@ -238,7 +238,7 @@ async def list_tools() -> list[Tool]:
|
||||
),
|
||||
Tool(
|
||||
name="package_skill",
|
||||
description="Package a skill directory into a .zip file ready for Claude upload. Automatically uploads if ANTHROPIC_API_KEY is set.",
|
||||
description="Package a skill directory into a .zip file ready for upload. Automatically uploads if ANTHROPIC_API_KEY is set.",
|
||||
inputSchema={
|
||||
"type": "object",
|
||||
"properties": {
|
||||
@@ -257,7 +257,7 @@ async def list_tools() -> list[Tool]:
|
||||
),
|
||||
Tool(
|
||||
name="upload_skill",
|
||||
description="Upload a skill .zip file to Claude automatically (requires ANTHROPIC_API_KEY)",
|
||||
description="Upload a skill .zip file automatically (requires ANTHROPIC_API_KEY)",
|
||||
inputSchema={
|
||||
"type": "object",
|
||||
"properties": {
|
||||
@@ -340,7 +340,7 @@ async def list_tools() -> list[Tool]:
|
||||
),
|
||||
Tool(
|
||||
name="scrape_pdf",
|
||||
description="Scrape PDF documentation and build Claude skill. Extracts text, code, and images from PDF files.",
|
||||
description="Scrape PDF documentation and build LLM skill. Extracts text, code, and images from PDF files.",
|
||||
inputSchema={
|
||||
"type": "object",
|
||||
"properties": {
|
||||
@@ -370,7 +370,7 @@ async def list_tools() -> list[Tool]:
|
||||
),
|
||||
Tool(
|
||||
name="scrape_github",
|
||||
description="Scrape GitHub repository and build Claude skill. Extracts README, Issues, Changelog, Releases, and code structure.",
|
||||
description="Scrape GitHub repository and build LLM skill. Extracts README, Issues, Changelog, Releases, and code structure.",
|
||||
inputSchema={
|
||||
"type": "object",
|
||||
"properties": {
|
||||
@@ -425,7 +425,7 @@ async def list_tools() -> list[Tool]:
|
||||
),
|
||||
Tool(
|
||||
name="install_skill",
|
||||
description="Complete one-command workflow: fetch config → scrape docs → AI enhance (MANDATORY) → package → upload. Enhancement required for quality (3/10→9/10). Takes 20-45 min depending on config size. Automatically uploads to Claude if ANTHROPIC_API_KEY is set.",
|
||||
description="Complete one-command workflow: fetch config → scrape docs → AI enhance (MANDATORY) → package → upload. Enhancement required for quality (3/10→9/10). Takes 20-45 min depending on config size. Automatically uploads if ANTHROPIC_API_KEY is set.",
|
||||
inputSchema={
|
||||
"type": "object",
|
||||
"properties": {
|
||||
@@ -444,7 +444,7 @@ async def list_tools() -> list[Tool]:
|
||||
},
|
||||
"auto_upload": {
|
||||
"type": "boolean",
|
||||
"description": "Auto-upload to Claude after packaging (requires ANTHROPIC_API_KEY). Default: true. Set to false to skip upload.",
|
||||
"description": "Auto-upload after packaging (requires ANTHROPIC_API_KEY). Default: true. Set to false to skip upload.",
|
||||
"default": True,
|
||||
},
|
||||
"unlimited": {
|
||||
@@ -898,7 +898,7 @@ async def package_skill_tool(args: dict) -> list[TextContent]:
|
||||
if should_upload:
|
||||
# Upload succeeded
|
||||
output += "\n\n✅ Skill packaged and uploaded automatically!"
|
||||
output += "\n Your skill is now available in Claude!"
|
||||
output += "\n Your skill has been uploaded successfully!"
|
||||
elif auto_upload and not has_api_key:
|
||||
# User wanted upload but no API key
|
||||
output += "\n\n📝 Skill packaged successfully!"
|
||||
@@ -922,7 +922,7 @@ async def package_skill_tool(args: dict) -> list[TextContent]:
|
||||
|
||||
|
||||
async def upload_skill_tool(args: dict) -> list[TextContent]:
|
||||
"""Upload skill .zip to Claude"""
|
||||
"""Upload skill .zip file"""
|
||||
skill_zip = args["skill_zip"]
|
||||
|
||||
# Run upload_skill.py
|
||||
@@ -931,7 +931,7 @@ async def upload_skill_tool(args: dict) -> list[TextContent]:
|
||||
# Timeout: 5 minutes for upload
|
||||
timeout = 300
|
||||
|
||||
progress_msg = "📤 Uploading skill to Claude...\n"
|
||||
progress_msg = "📤 Uploading skill...\n"
|
||||
progress_msg += f"⏱️ Maximum time: {timeout // 60} minutes\n\n"
|
||||
|
||||
stdout, stderr, returncode = run_subprocess_with_streaming(cmd, timeout=timeout)
|
||||
@@ -1194,7 +1194,7 @@ async def scrape_pdf_tool(args: dict) -> list[TextContent]:
|
||||
|
||||
|
||||
async def scrape_github_tool(args: dict) -> list[TextContent]:
|
||||
"""Scrape GitHub repository to Claude skill (C1.11)"""
|
||||
"""Scrape GitHub repository to LLM skill (C1.11)"""
|
||||
repo = args.get("repo")
|
||||
config_path = args.get("config_path")
|
||||
name = args.get("name")
|
||||
@@ -1550,7 +1550,7 @@ async def install_skill_tool(args: dict) -> list[TextContent]:
|
||||
2. Scrape documentation
|
||||
3. AI Enhancement (MANDATORY - no skip option)
|
||||
4. Package to .zip
|
||||
5. Upload to Claude (optional)
|
||||
5. Upload (optional)
|
||||
|
||||
Args:
|
||||
config_name: Config to fetch from API (mutually exclusive with config_path)
|
||||
@@ -1743,7 +1743,7 @@ async def install_skill_tool(args: dict) -> list[TextContent]:
|
||||
output_lines.append(stdout)
|
||||
workflow_state["phases_completed"].append("enhance_skill")
|
||||
else:
|
||||
output_lines.append(" [DRY RUN] Would enhance SKILL.md with Claude Code")
|
||||
output_lines.append(" [DRY RUN] Would enhance SKILL.md with AI agent")
|
||||
|
||||
output_lines.append("")
|
||||
|
||||
@@ -1786,7 +1786,7 @@ async def install_skill_tool(args: dict) -> list[TextContent]:
|
||||
# ===== PHASE 5: Upload (Optional) =====
|
||||
if auto_upload:
|
||||
phase_num = "5/5" if config_name else "4/4"
|
||||
output_lines.append(f"📤 PHASE {phase_num}: Upload to Claude")
|
||||
output_lines.append(f"📤 PHASE {phase_num}: Upload Skill")
|
||||
output_lines.append("-" * 70)
|
||||
output_lines.append(f"Zip file: {workflow_state['zip_path']}")
|
||||
output_lines.append("")
|
||||
@@ -1817,7 +1817,7 @@ async def install_skill_tool(args: dict) -> list[TextContent]:
|
||||
output_lines.append(" 2. Click 'Upload Skill'")
|
||||
output_lines.append(f" 3. Select: {workflow_state['zip_path']}")
|
||||
else:
|
||||
output_lines.append(" [DRY RUN] Would upload to Claude (if API key set)")
|
||||
output_lines.append(" [DRY RUN] Would upload skill (if API key set)")
|
||||
|
||||
output_lines.append("")
|
||||
|
||||
@@ -1840,7 +1840,7 @@ async def install_skill_tool(args: dict) -> list[TextContent]:
|
||||
output_lines.append("")
|
||||
|
||||
if auto_upload and has_api_key:
|
||||
output_lines.append("🎉 Your skill is now available in Claude!")
|
||||
output_lines.append("🎉 Your skill has been uploaded successfully!")
|
||||
output_lines.append(" Go to https://claude.ai/skills to use it")
|
||||
elif auto_upload:
|
||||
output_lines.append("📝 Manual upload required (see instructions above)")
|
||||
|
||||
@@ -9,6 +9,7 @@ Tools are organized by functionality:
|
||||
- packaging_tools: Skill packaging and upload
|
||||
- splitting_tools: Config splitting and router generation
|
||||
- source_tools: Config source management (fetch, submit, add/remove sources)
|
||||
- marketplace_tools: Marketplace management (add, list, remove, publish)
|
||||
- vector_db_tools: Vector database export (Weaviate, Chroma, FAISS, Qdrant)
|
||||
"""
|
||||
|
||||
@@ -84,6 +85,18 @@ from .source_tools import (
|
||||
from .source_tools import (
|
||||
submit_config_tool as submit_config_impl,
|
||||
)
|
||||
from .marketplace_tools import (
|
||||
add_marketplace_tool as add_marketplace_impl,
|
||||
)
|
||||
from .marketplace_tools import (
|
||||
list_marketplaces_tool as list_marketplaces_impl,
|
||||
)
|
||||
from .marketplace_tools import (
|
||||
publish_to_marketplace_tool as publish_to_marketplace_impl,
|
||||
)
|
||||
from .marketplace_tools import (
|
||||
remove_marketplace_tool as remove_marketplace_impl,
|
||||
)
|
||||
from .splitting_tools import (
|
||||
generate_router as generate_router_impl,
|
||||
)
|
||||
@@ -147,6 +160,11 @@ __all__ = [
|
||||
# Splitting tools
|
||||
"split_config_impl",
|
||||
"generate_router_impl",
|
||||
# Marketplace tools
|
||||
"add_marketplace_impl",
|
||||
"list_marketplaces_impl",
|
||||
"remove_marketplace_impl",
|
||||
"publish_to_marketplace_impl",
|
||||
# Source tools
|
||||
"fetch_config_impl",
|
||||
"submit_config_impl",
|
||||
|
||||
226
src/skill_seekers/mcp/tools/marketplace_tools.py
Normal file
226
src/skill_seekers/mcp/tools/marketplace_tools.py
Normal file
@@ -0,0 +1,226 @@
|
||||
"""
|
||||
Marketplace management tools for MCP server.
|
||||
|
||||
This module contains tools for managing plugin marketplace repositories:
|
||||
- add_marketplace: Register a plugin marketplace repository
|
||||
- list_marketplaces: List all registered marketplace repositories
|
||||
- remove_marketplace: Remove a registered marketplace
|
||||
- publish_to_marketplace: Publish a packaged skill to a marketplace
|
||||
"""
|
||||
|
||||
# MCP types (imported conditionally)
|
||||
try:
|
||||
from mcp.types import TextContent
|
||||
|
||||
MCP_AVAILABLE = True
|
||||
except ImportError:
|
||||
|
||||
class TextContent:
|
||||
"""Fallback TextContent for when MCP is not installed"""
|
||||
|
||||
def __init__(self, type: str, text: str):
|
||||
self.type = type
|
||||
self.text = text
|
||||
|
||||
MCP_AVAILABLE = False
|
||||
|
||||
|
||||
async def add_marketplace_tool(args: dict) -> list[TextContent]:
|
||||
"""Register a plugin marketplace repository."""
|
||||
from skill_seekers.mcp.marketplace_manager import MarketplaceManager
|
||||
|
||||
name = args.get("name")
|
||||
git_url = args.get("git_url")
|
||||
token_env = args.get("token_env")
|
||||
branch = args.get("branch", "main")
|
||||
author_name = args.get("author_name", "")
|
||||
author_email = args.get("author_email", "")
|
||||
enabled = args.get("enabled", True)
|
||||
|
||||
try:
|
||||
if not name:
|
||||
return [TextContent(type="text", text="❌ Error: 'name' parameter is required")]
|
||||
if not git_url:
|
||||
return [TextContent(type="text", text="❌ Error: 'git_url' parameter is required")]
|
||||
|
||||
author = {"name": author_name, "email": author_email}
|
||||
|
||||
manager = MarketplaceManager()
|
||||
marketplace = manager.add_marketplace(
|
||||
name=name,
|
||||
git_url=git_url,
|
||||
token_env=token_env,
|
||||
branch=branch,
|
||||
author=author,
|
||||
enabled=enabled,
|
||||
)
|
||||
|
||||
is_update = marketplace["added_at"] != marketplace["updated_at"]
|
||||
|
||||
result = f"""✅ Marketplace {"updated" if is_update else "registered"} successfully!
|
||||
|
||||
📛 Name: {marketplace["name"]}
|
||||
📁 Repository: {marketplace["git_url"]}
|
||||
🌿 Branch: {marketplace["branch"]}
|
||||
🔑 Token env: {marketplace["token_env"]}
|
||||
👤 Author: {marketplace["author"]["name"]} <{marketplace["author"]["email"]}>
|
||||
✓ Enabled: {marketplace["enabled"]}
|
||||
🕒 Added: {marketplace["added_at"][:19]}
|
||||
|
||||
Usage:
|
||||
# Publish a skill to this marketplace
|
||||
publish_to_marketplace(skill_dir="output/my-skill", marketplace="{marketplace["name"]}")
|
||||
|
||||
# List all marketplaces
|
||||
list_marketplaces()
|
||||
|
||||
💡 Set {marketplace["token_env"]} environment variable for private repos
|
||||
"""
|
||||
return [TextContent(type="text", text=result)]
|
||||
|
||||
except ValueError as e:
|
||||
return [TextContent(type="text", text=f"❌ Validation Error: {str(e)}")]
|
||||
except Exception as e:
|
||||
return [TextContent(type="text", text=f"❌ Error: {str(e)}")]
|
||||
|
||||
|
||||
async def list_marketplaces_tool(args: dict) -> list[TextContent]:
|
||||
"""List all registered plugin marketplace repositories."""
|
||||
from skill_seekers.mcp.marketplace_manager import MarketplaceManager
|
||||
|
||||
enabled_only = args.get("enabled_only", False)
|
||||
|
||||
try:
|
||||
manager = MarketplaceManager()
|
||||
marketplaces = manager.list_marketplaces(enabled_only=enabled_only)
|
||||
|
||||
if not marketplaces:
|
||||
result = """📋 No marketplaces registered
|
||||
|
||||
To add a marketplace:
|
||||
add_marketplace(
|
||||
name="my-plugins",
|
||||
git_url="https://github.com/myorg/plugins.git",
|
||||
author_name="My Team",
|
||||
author_email="team@example.com"
|
||||
)
|
||||
|
||||
💡 Once added, use: publish_to_marketplace(skill_dir="...", marketplace="my-plugins")
|
||||
"""
|
||||
return [TextContent(type="text", text=result)]
|
||||
|
||||
result = f"📋 Plugin Marketplaces ({len(marketplaces)} total"
|
||||
if enabled_only:
|
||||
result += ", enabled only"
|
||||
result += ")\n\n"
|
||||
|
||||
for mp in marketplaces:
|
||||
status_icon = "✓" if mp.get("enabled", True) else "✗"
|
||||
author = mp.get("author", {})
|
||||
author_str = f"{author.get('name', '')} <{author.get('email', '')}>"
|
||||
result += f"{status_icon} **{mp['name']}**\n"
|
||||
result += f" 📁 {mp['git_url']}\n"
|
||||
result += f" 🌿 Branch: {mp['branch']} | 🔑 Token: {mp['token_env']}\n"
|
||||
result += f" 👤 Author: {author_str}\n"
|
||||
result += f" 🕒 Added: {mp['added_at'][:19]}\n"
|
||||
result += "\n"
|
||||
|
||||
result += """Usage:
|
||||
# Publish skill to a marketplace
|
||||
publish_to_marketplace(skill_dir="output/my-skill", marketplace="MARKETPLACE_NAME")
|
||||
|
||||
# Add new marketplace
|
||||
add_marketplace(name="...", git_url="...")
|
||||
|
||||
# Remove marketplace
|
||||
remove_marketplace(name="MARKETPLACE_NAME")
|
||||
"""
|
||||
return [TextContent(type="text", text=result)]
|
||||
|
||||
except Exception as e:
|
||||
return [TextContent(type="text", text=f"❌ Error: {str(e)}")]
|
||||
|
||||
|
||||
async def remove_marketplace_tool(args: dict) -> list[TextContent]:
|
||||
"""Remove a registered plugin marketplace."""
|
||||
from skill_seekers.mcp.marketplace_manager import MarketplaceManager
|
||||
|
||||
name = args.get("name")
|
||||
|
||||
try:
|
||||
if not name:
|
||||
return [TextContent(type="text", text="❌ Error: 'name' parameter is required")]
|
||||
|
||||
manager = MarketplaceManager()
|
||||
removed = manager.remove_marketplace(name)
|
||||
|
||||
if removed:
|
||||
result = f"""✅ Marketplace removed successfully!
|
||||
|
||||
📛 Removed: {name}
|
||||
|
||||
⚠️ Note: Cached repository data is NOT deleted
|
||||
To free disk space, manually delete: ~/.skill-seekers/cache/marketplace_{name}/
|
||||
"""
|
||||
return [TextContent(type="text", text=result)]
|
||||
else:
|
||||
sources = manager.list_marketplaces()
|
||||
available = [m["name"] for m in sources]
|
||||
result = f"""❌ Marketplace '{name}' not found
|
||||
|
||||
Available marketplaces: {", ".join(available) if available else "none"}
|
||||
"""
|
||||
return [TextContent(type="text", text=result)]
|
||||
|
||||
except Exception as e:
|
||||
return [TextContent(type="text", text=f"❌ Error: {str(e)}")]
|
||||
|
||||
|
||||
async def publish_to_marketplace_tool(args: dict) -> list[TextContent]:
|
||||
"""Publish a packaged skill to a plugin marketplace repository."""
|
||||
from skill_seekers.mcp.marketplace_publisher import MarketplacePublisher
|
||||
|
||||
skill_dir = args.get("skill_dir")
|
||||
marketplace = args.get("marketplace")
|
||||
category = args.get("category", "development")
|
||||
skill_name = args.get("skill_name")
|
||||
description = args.get("description")
|
||||
create_branch = args.get("create_branch", False)
|
||||
force = args.get("force", False)
|
||||
|
||||
try:
|
||||
if not skill_dir:
|
||||
return [TextContent(type="text", text="❌ Error: 'skill_dir' parameter is required")]
|
||||
if not marketplace:
|
||||
return [TextContent(type="text", text="❌ Error: 'marketplace' parameter is required")]
|
||||
|
||||
publisher = MarketplacePublisher()
|
||||
result = publisher.publish(
|
||||
skill_dir=skill_dir,
|
||||
marketplace_name=marketplace,
|
||||
category=category,
|
||||
skill_name=skill_name,
|
||||
description=description,
|
||||
create_branch=create_branch,
|
||||
force=force,
|
||||
)
|
||||
|
||||
if result["success"]:
|
||||
output = f"""✅ Skill published to marketplace successfully!
|
||||
|
||||
📦 Plugin: {result["plugin_path"]}
|
||||
🏪 Marketplace: {marketplace}
|
||||
🏷️ Category: {category}
|
||||
🌿 Branch: {result["branch"]}
|
||||
📝 Commit: {result["commit_sha"]}
|
||||
|
||||
{result["message"]}
|
||||
"""
|
||||
return [TextContent(type="text", text=output)]
|
||||
else:
|
||||
return [TextContent(type="text", text=f"❌ Publish failed: {result['message']}")]
|
||||
|
||||
except (FileNotFoundError, KeyError, ValueError, RuntimeError) as e:
|
||||
return [TextContent(type="text", text=f"❌ {str(e)}")]
|
||||
except Exception as e:
|
||||
return [TextContent(type="text", text=f"❌ Error: {str(e)}")]
|
||||
@@ -113,8 +113,8 @@ async def package_skill_tool(args: dict) -> list[TextContent]:
|
||||
args: Dictionary with:
|
||||
- skill_dir (str): Path to skill directory (e.g., output/react/)
|
||||
- auto_upload (bool): Try to upload automatically if API key is available (default: True)
|
||||
- target (str): Target platform (default: 'claude')
|
||||
Options: 'claude', 'gemini', 'openai', 'markdown'
|
||||
- target (str): Target platform (default: 'auto')
|
||||
Options: 'auto', 'claude', 'gemini', 'openai', 'markdown'
|
||||
|
||||
Returns:
|
||||
List of TextContent with packaging results
|
||||
@@ -123,7 +123,11 @@ async def package_skill_tool(args: dict) -> list[TextContent]:
|
||||
|
||||
skill_dir = args["skill_dir"]
|
||||
auto_upload = args.get("auto_upload", True)
|
||||
target = args.get("target", "claude")
|
||||
target = args.get("target", "auto")
|
||||
if target == "auto":
|
||||
from skill_seekers.cli.agent_client import AgentClient
|
||||
|
||||
target = AgentClient.detect_default_target()
|
||||
|
||||
# Get platform adaptor
|
||||
try:
|
||||
@@ -232,8 +236,8 @@ async def upload_skill_tool(args: dict) -> list[TextContent]:
|
||||
Args:
|
||||
args: Dictionary with:
|
||||
- skill_zip (str): Path to skill package (.zip or .tar.gz)
|
||||
- target (str): Target platform (default: 'claude')
|
||||
Options: 'claude', 'gemini', 'openai'
|
||||
- target (str): Target platform (default: 'auto')
|
||||
Options: 'auto', 'claude', 'gemini', 'openai'
|
||||
Note: 'markdown' does not support upload
|
||||
- api_key (str, optional): API key (uses env var if not provided)
|
||||
|
||||
@@ -243,7 +247,11 @@ async def upload_skill_tool(args: dict) -> list[TextContent]:
|
||||
from skill_seekers.cli.adaptors import get_adaptor
|
||||
|
||||
skill_zip = args["skill_zip"]
|
||||
target = args.get("target", "claude")
|
||||
target = args.get("target", "auto")
|
||||
if target == "auto":
|
||||
from skill_seekers.cli.agent_client import AgentClient
|
||||
|
||||
target = AgentClient.detect_default_target()
|
||||
api_key = args.get("api_key")
|
||||
|
||||
# Get platform adaptor
|
||||
@@ -296,11 +304,11 @@ async def enhance_skill_tool(args: dict) -> list[TextContent]:
|
||||
Args:
|
||||
args: Dictionary with:
|
||||
- skill_dir (str): Path to skill directory
|
||||
- target (str): Target platform (default: 'claude')
|
||||
Options: 'claude', 'gemini', 'openai'
|
||||
- target (str): Target platform (default: 'auto')
|
||||
Options: 'auto', 'claude', 'gemini', 'openai'
|
||||
Note: 'markdown' does not support enhancement
|
||||
- mode (str): Enhancement mode (default: 'local')
|
||||
'local': Uses Claude Code Max (no API key)
|
||||
'local': Uses AI coding agent (no API key)
|
||||
'api': Uses platform API (requires API key)
|
||||
- api_key (str, optional): API key for 'api' mode
|
||||
|
||||
@@ -310,7 +318,11 @@ async def enhance_skill_tool(args: dict) -> list[TextContent]:
|
||||
from skill_seekers.cli.adaptors import get_adaptor
|
||||
|
||||
skill_dir = Path(args.get("skill_dir"))
|
||||
target = args.get("target", "claude")
|
||||
target = args.get("target", "auto")
|
||||
if target == "auto":
|
||||
from skill_seekers.cli.agent_client import AgentClient
|
||||
|
||||
target = AgentClient.detect_default_target()
|
||||
mode = args.get("mode", "local")
|
||||
api_key = args.get("api_key")
|
||||
|
||||
@@ -348,8 +360,8 @@ async def enhance_skill_tool(args: dict) -> list[TextContent]:
|
||||
output_lines.append("")
|
||||
|
||||
if mode == "local":
|
||||
# Use local enhancement (Claude Code)
|
||||
output_lines.append("Using Claude Code Max (local, no API key required)")
|
||||
# Use local enhancement (AI coding agent)
|
||||
output_lines.append("Using AI coding agent (local, no API key required)")
|
||||
output_lines.append("Running enhancement in headless mode...")
|
||||
output_lines.append("")
|
||||
|
||||
@@ -437,7 +449,7 @@ async def install_skill_tool(args: dict) -> list[TextContent]:
|
||||
- auto_upload (bool): Upload after packaging (default: True)
|
||||
- unlimited (bool): Remove page limits (default: False)
|
||||
- dry_run (bool): Preview only (default: False)
|
||||
- target (str): Target LLM platform (default: "claude")
|
||||
- target (str): Target LLM platform (default: "auto")
|
||||
|
||||
Returns:
|
||||
List of TextContent with workflow progress and results
|
||||
@@ -455,7 +467,14 @@ async def install_skill_tool(args: dict) -> list[TextContent]:
|
||||
auto_upload = args.get("auto_upload", True)
|
||||
unlimited = args.get("unlimited", False)
|
||||
dry_run = args.get("dry_run", False)
|
||||
target = args.get("target", "claude")
|
||||
target = args.get("target", "auto")
|
||||
marketplace_arg = args.get("marketplace")
|
||||
marketplace_category = args.get("marketplace_category", "development")
|
||||
create_branch = args.get("create_branch", False)
|
||||
if target == "auto":
|
||||
from skill_seekers.cli.agent_client import AgentClient
|
||||
|
||||
target = AgentClient.detect_default_target()
|
||||
|
||||
# Get platform adaptor
|
||||
try:
|
||||
@@ -559,6 +578,7 @@ async def install_skill_tool(args: dict) -> list[TextContent]:
|
||||
with open(workflow_state["config_path"]) as f:
|
||||
config = json.load(f)
|
||||
workflow_state["skill_name"] = config.get("name", "unknown")
|
||||
workflow_state["config_data"] = config
|
||||
except Exception as e:
|
||||
return [
|
||||
TextContent(
|
||||
@@ -637,7 +657,7 @@ async def install_skill_tool(args: dict) -> list[TextContent]:
|
||||
output_lines.append(stdout)
|
||||
workflow_state["phases_completed"].append("enhance_skill")
|
||||
else:
|
||||
output_lines.append(" [DRY RUN] Would enhance SKILL.md with Claude Code")
|
||||
output_lines.append(" [DRY RUN] Would enhance SKILL.md with AI agent")
|
||||
|
||||
output_lines.append("")
|
||||
|
||||
@@ -702,6 +722,7 @@ async def install_skill_tool(args: dict) -> list[TextContent]:
|
||||
output_lines.append("")
|
||||
|
||||
# ===== PHASE 5: Upload (Optional) =====
|
||||
has_api_key = False # Initialize before conditional block
|
||||
if auto_upload:
|
||||
phase_num = "5/5" if config_name else "4/4"
|
||||
output_lines.append(f"📤 PHASE {phase_num}: Upload to {adaptor.PLATFORM_NAME}")
|
||||
@@ -767,6 +788,62 @@ async def install_skill_tool(args: dict) -> list[TextContent]:
|
||||
|
||||
output_lines.append("")
|
||||
|
||||
# ===== PHASE 6: Publish to Marketplace (Optional) =====
|
||||
marketplace_targets = []
|
||||
if marketplace_arg:
|
||||
marketplace_targets.append(
|
||||
{"marketplace": marketplace_arg, "category": marketplace_category}
|
||||
)
|
||||
else:
|
||||
cd = workflow_state.get("config_data", {})
|
||||
if isinstance(cd, dict):
|
||||
marketplace_targets = cd.get("marketplace_targets", [])
|
||||
|
||||
if marketplace_targets:
|
||||
phase_num = len(workflow_state["phases_completed"]) + 1
|
||||
output_lines.append(f"{'=' * 70}")
|
||||
output_lines.append(
|
||||
f"PHASE {phase_num}: Publish to Marketplace"
|
||||
f" ({len(marketplace_targets)} target{'s' if len(marketplace_targets) > 1 else ''})"
|
||||
)
|
||||
output_lines.append(f"{'=' * 70}")
|
||||
output_lines.append("")
|
||||
|
||||
if not dry_run:
|
||||
from .marketplace_tools import publish_to_marketplace_tool
|
||||
|
||||
for mp_target in marketplace_targets:
|
||||
mp_name = mp_target.get("marketplace", "")
|
||||
mp_cat = mp_target.get("category", "development")
|
||||
output_lines.append(f"Publishing to marketplace '{mp_name}'...")
|
||||
|
||||
try:
|
||||
pub_result = await publish_to_marketplace_tool(
|
||||
{
|
||||
"skill_dir": workflow_state["skill_dir"],
|
||||
"marketplace": mp_name,
|
||||
"category": mp_cat,
|
||||
"create_branch": create_branch,
|
||||
"force": True,
|
||||
}
|
||||
)
|
||||
pub_output = pub_result[0].text if pub_result else "No output"
|
||||
output_lines.append(pub_output)
|
||||
workflow_state["phases_completed"].append(
|
||||
f"publish_to_marketplace({mp_name})"
|
||||
)
|
||||
except Exception as e:
|
||||
output_lines.append(f"Failed to publish to '{mp_name}': {str(e)}")
|
||||
output_lines.append("")
|
||||
else:
|
||||
for mp_target in marketplace_targets:
|
||||
mp_name = mp_target.get("marketplace", "")
|
||||
mp_cat = mp_target.get("category", "development")
|
||||
output_lines.append(
|
||||
f" [DRY RUN] Would publish to marketplace '{mp_name}' (category: {mp_cat})"
|
||||
)
|
||||
output_lines.append("")
|
||||
|
||||
# ===== WORKFLOW SUMMARY =====
|
||||
output_lines.append("=" * 70)
|
||||
output_lines.append("✅ WORKFLOW COMPLETE")
|
||||
|
||||
@@ -826,3 +826,63 @@ To see all sources:
|
||||
|
||||
except Exception as e:
|
||||
return [TextContent(type="text", text=f"❌ Error: {str(e)}")]
|
||||
|
||||
|
||||
async def push_config_tool(args: dict) -> list[TextContent]:
|
||||
"""
|
||||
Push a config to a registered config source repository.
|
||||
|
||||
Validates the config, places it in the correct category directory,
|
||||
commits, and pushes to the source repo.
|
||||
|
||||
Args:
|
||||
args: Dictionary containing:
|
||||
- config_path: Path to config JSON file (required)
|
||||
- source_name: Registered source name, e.g., "spyke" (required)
|
||||
- category: Category directory (e.g., "game-engines"). Auto-detected if omitted.
|
||||
- create_branch: Create feature branch instead of pushing to main (default: false)
|
||||
- force: Overwrite existing config (default: false)
|
||||
|
||||
Returns:
|
||||
List of TextContent with push results
|
||||
"""
|
||||
config_path = args.get("config_path")
|
||||
source_name = args.get("source_name")
|
||||
category = args.get("category", "auto")
|
||||
create_branch = args.get("create_branch", False)
|
||||
force = args.get("force", False)
|
||||
|
||||
if not config_path:
|
||||
return [TextContent(type="text", text="❌ Missing required parameter: config_path")]
|
||||
if not source_name:
|
||||
return [TextContent(type="text", text="❌ Missing required parameter: source_name")]
|
||||
|
||||
try:
|
||||
from skill_seekers.mcp.config_publisher import ConfigPublisher
|
||||
|
||||
publisher = ConfigPublisher()
|
||||
result = publisher.publish(
|
||||
config_path=config_path,
|
||||
source_name=source_name,
|
||||
category=category,
|
||||
create_branch=create_branch,
|
||||
force=force,
|
||||
)
|
||||
|
||||
output = f"""✅ Config pushed successfully!
|
||||
|
||||
📄 Config: {result["config_name"]}
|
||||
📂 Path: {result["config_path"]}
|
||||
🏷️ Category: {result["category"]}
|
||||
📦 Source: {result["source"]}
|
||||
🔀 Branch: {result["branch"]}
|
||||
📝 Commit: {result["commit_sha"]}
|
||||
💬 Message: {result["message"]}
|
||||
|
||||
To fetch this config:
|
||||
fetch_config(source="{result["source"]}", config_name="{result["config_name"]}")
|
||||
"""
|
||||
return [TextContent(type="text", text=output)]
|
||||
|
||||
except Exception as e:
|
||||
return [TextContent(type="text", text=f"❌ Push failed: {str(e)}")]
|
||||
|
||||
131
src/skill_seekers/workflows/ai-assistant-internals.yaml
Normal file
131
src/skill_seekers/workflows/ai-assistant-internals.yaml
Normal file
@@ -0,0 +1,131 @@
|
||||
name: ai-assistant-internals
|
||||
description: Deep analysis of AI assistant CLI internals including tool registry, context pipeline, MCP integration, sub-agent coordination, and cost optimization
|
||||
version: "1.0"
|
||||
applies_to:
|
||||
- codebase_analysis
|
||||
- github_analysis
|
||||
variables:
|
||||
depth: comprehensive
|
||||
stages:
|
||||
- name: tool_registry_analysis
|
||||
type: custom
|
||||
target: tools
|
||||
uses_history: false
|
||||
enabled: true
|
||||
prompt: >
|
||||
Analyze the tool registry and dispatch system in this AI assistant codebase.
|
||||
|
||||
For each tool/agent capability:
|
||||
1. Tool name and purpose
|
||||
2. Input schema (parameters, types, required vs optional)
|
||||
3. Output schema (return types, error conditions)
|
||||
4. Dispatch mechanism (how the LLM selects and invokes it)
|
||||
5. Related tools that are commonly chained together
|
||||
|
||||
Output JSON with:
|
||||
- "tools": array of {name, purpose, input_schema, output_schema, dispatch_mechanism, related_tools[]}
|
||||
- "tool_categories": grouping by function (file_ops, web_search, agent_mgmt, shell_exec, etc.)
|
||||
- "dispatch_pattern": how tool calls are parsed, validated, and routed
|
||||
|
||||
- name: context_pipeline
|
||||
type: custom
|
||||
target: context
|
||||
uses_history: true
|
||||
enabled: true
|
||||
prompt: >
|
||||
Document how user queries, system context, conversation history, and file context are assembled into LLM prompts.
|
||||
|
||||
Cover:
|
||||
1. System prompt construction (static vs dynamic parts)
|
||||
2. User context collection (current directory, git state, open files)
|
||||
3. Conversation history management (truncation, summarization, checkpointing)
|
||||
4. File context injection (how code is read and formatted for the model)
|
||||
5. Context window optimization strategies
|
||||
|
||||
Output JSON with:
|
||||
- "system_prompt_structure": sections and their purposes
|
||||
- "context_sources": array of {source, description, priority, fallback_behavior}
|
||||
- "history_management": truncation/summary strategy
|
||||
- "file_context_format": how files are represented in prompts
|
||||
- "optimization_strategies": techniques to stay within token limits
|
||||
|
||||
- name: mcp_patterns
|
||||
type: custom
|
||||
target: mcp
|
||||
uses_history: true
|
||||
enabled: true
|
||||
prompt: >
|
||||
Extract Model Context Protocol (MCP) integration patterns from this codebase.
|
||||
|
||||
Document:
|
||||
1. MCP client initialization and configuration
|
||||
2. Server discovery and connection lifecycle
|
||||
3. Tool/resource exposure via MCP
|
||||
4. Authentication and security boundaries
|
||||
5. Error handling for MCP server failures
|
||||
|
||||
Output JSON with:
|
||||
- "mcp_client": initialization and config patterns
|
||||
- "server_management": discovery, connect, disconnect, health checks
|
||||
- "exposed_capabilities": tools/resources surfaced to MCP
|
||||
- "security_model": auth, sandboxing, permission boundaries
|
||||
- "failure_modes": retry, fallback, and user notification strategies
|
||||
|
||||
- name: agent_coordination
|
||||
type: custom
|
||||
target: agents
|
||||
uses_history: true
|
||||
enabled: true
|
||||
prompt: >
|
||||
Analyze sub-agent creation, task delegation, and multi-agent coordination patterns.
|
||||
|
||||
Identify:
|
||||
1. Agent lifecycle (create, run, pause, resume, terminate)
|
||||
2. Task delegation strategies (how work is split across agents)
|
||||
3. Inter-agent communication (message passing, shared state)
|
||||
4. Result aggregation and merging from multiple agents
|
||||
5. Parent-child relationship management and oversight
|
||||
|
||||
Output JSON with:
|
||||
- "agent_lifecycle": states and transitions
|
||||
- "delegation_patterns": strategies for splitting work
|
||||
- "communication_model": how agents share data
|
||||
- "result_aggregation": merging outputs from parallel agents
|
||||
- "oversight_mechanisms": how parent agents monitor children
|
||||
|
||||
- name: cost_optimization
|
||||
type: custom
|
||||
target: cost
|
||||
uses_history: true
|
||||
enabled: true
|
||||
prompt: >
|
||||
Review token usage tracking, cost estimation, and optimization strategies.
|
||||
|
||||
Document:
|
||||
1. How input/output tokens are measured per request
|
||||
2. Cost estimation methodology (pricing models, caching)
|
||||
3. Optimization techniques (prompt compression, selective context, model tiering)
|
||||
4. Budget controls and user-facing cost feedback
|
||||
5. Trade-offs between cost and response quality
|
||||
|
||||
Output JSON with:
|
||||
- "token_measurement": how tokens are counted and attributed
|
||||
- "cost_estimation": pricing logic and display
|
||||
- "optimization_techniques": array of {name, description, impact, implementation_location}
|
||||
- "budget_controls": limits, warnings, and enforcement
|
||||
- "quality_tradeoffs": when and how cost is prioritized over quality
|
||||
|
||||
post_process:
|
||||
reorder_sections:
|
||||
- overview
|
||||
- tool_registry
|
||||
- context_pipeline
|
||||
- mcp_integration
|
||||
- agent_coordination
|
||||
- cost_optimization
|
||||
- examples
|
||||
- advanced_topics
|
||||
add_metadata:
|
||||
enhanced: true
|
||||
workflow: ai-assistant-internals
|
||||
deep_analysis: true
|
||||
214
src/skill_seekers/workflows/unity-game-dev.yaml
Normal file
214
src/skill_seekers/workflows/unity-game-dev.yaml
Normal file
@@ -0,0 +1,214 @@
|
||||
name: unity-game-dev
|
||||
description: Unity game development patterns, MonoBehaviour lifecycle, component architecture, memory management, and C# best practices for Unity projects
|
||||
version: "1.0"
|
||||
applies_to:
|
||||
- codebase_analysis
|
||||
- github_analysis
|
||||
- documentation
|
||||
variables:
|
||||
depth: comprehensive
|
||||
|
||||
stages:
|
||||
- name: monobehaviour_lifecycle
|
||||
type: custom
|
||||
target: lifecycle
|
||||
uses_history: false
|
||||
enabled: true
|
||||
prompt: >
|
||||
Document all MonoBehaviour lifecycle patterns used in this codebase/library.
|
||||
|
||||
Cover:
|
||||
1. Initialization order (Awake vs Start vs OnEnable)
|
||||
2. Update loops (Update vs FixedUpdate vs LateUpdate) and when to use each
|
||||
3. Destruction and cleanup (OnDestroy, OnDisable, OnApplicationQuit)
|
||||
4. Coroutine lifecycle and cancellation patterns
|
||||
5. Script execution order dependencies
|
||||
|
||||
For each pattern found:
|
||||
- When to use it vs alternatives
|
||||
- Common mistakes and gotchas
|
||||
- Code example showing correct usage
|
||||
|
||||
Output JSON with "lifecycle_patterns" array of:
|
||||
{name, phase, use_case, gotchas[], code_example, execution_order}
|
||||
|
||||
- name: component_architecture
|
||||
type: custom
|
||||
target: components
|
||||
uses_history: true
|
||||
enabled: true
|
||||
prompt: >
|
||||
Document the component-based architecture patterns.
|
||||
|
||||
Cover:
|
||||
1. Component composition patterns (prefer composition over inheritance)
|
||||
2. GetComponent usage and caching strategies
|
||||
3. RequireComponent and dependency management
|
||||
4. Prefab architecture and nested prefab patterns
|
||||
5. ScriptableObject as data containers and event channels
|
||||
6. Inspector serialization ([SerializeField], [Header], [Tooltip], custom editors)
|
||||
|
||||
For each pattern:
|
||||
- Setup steps in Unity Editor
|
||||
- C# implementation
|
||||
- When to use vs alternatives
|
||||
|
||||
Output JSON with:
|
||||
- "component_patterns": array of architecture patterns
|
||||
- "scriptable_objects": data-driven design patterns
|
||||
- "serialization": Inspector setup patterns
|
||||
|
||||
- name: dependency_injection
|
||||
type: custom
|
||||
target: di_patterns
|
||||
uses_history: true
|
||||
enabled: true
|
||||
prompt: >
|
||||
Document dependency injection patterns for Unity.
|
||||
|
||||
Cover:
|
||||
1. Zenject/Extenject bindings (BindInterfacesAndSelfTo, FromComponentInHierarchy, AsSingle, AsCached)
|
||||
2. VContainer registration patterns (if used)
|
||||
3. Installer organization (MonoInstaller, ScriptableObjectInstaller, ProjectInstaller)
|
||||
4. Factory patterns with DI (PlaceholderFactory, IFactory)
|
||||
5. Signal/event bus patterns via DI
|
||||
6. Scene-scoped vs project-scoped containers
|
||||
|
||||
For each pattern:
|
||||
- Binding configuration
|
||||
- Injection usage ([Inject], constructor injection)
|
||||
- Lifecycle management
|
||||
- Testing with mock bindings
|
||||
|
||||
Output JSON with:
|
||||
- "di_bindings": binding patterns and when to use each
|
||||
- "installers": installer organization
|
||||
- "factories": factory patterns
|
||||
- "testing": how to mock dependencies
|
||||
|
||||
- name: async_patterns
|
||||
type: custom
|
||||
target: async
|
||||
uses_history: true
|
||||
enabled: true
|
||||
prompt: >
|
||||
Document async/concurrent programming patterns for Unity.
|
||||
|
||||
Cover:
|
||||
1. Coroutines (IEnumerator, yield return, StartCoroutine/StopCoroutine)
|
||||
2. UniTask patterns (async/await, UniTask vs Task, cancellation tokens)
|
||||
3. Async asset loading (Addressables, Resources.LoadAsync)
|
||||
4. Threading considerations (main thread requirement, UnitySynchronizationContext)
|
||||
5. DOTween async integration (AwaitForCompletion, ToUniTask)
|
||||
6. Progress reporting and loading screens
|
||||
|
||||
For each pattern:
|
||||
- When to use it vs alternatives
|
||||
- Cancellation and cleanup
|
||||
- Error handling
|
||||
- Performance implications
|
||||
|
||||
Output JSON with:
|
||||
- "coroutine_patterns": coroutine usage and gotchas
|
||||
- "unitask_patterns": modern async patterns
|
||||
- "loading_patterns": async loading strategies
|
||||
- "thread_safety": main thread considerations
|
||||
|
||||
- name: memory_and_performance
|
||||
type: custom
|
||||
target: unity_performance
|
||||
uses_history: true
|
||||
enabled: true
|
||||
prompt: >
|
||||
Document Unity-specific memory management and performance patterns.
|
||||
|
||||
Cover:
|
||||
1. Object pooling (instantiate/destroy avoidance, pool managers)
|
||||
2. GC pressure reduction (avoid allocations in Update, struct vs class, NativeArray)
|
||||
3. Asset memory lifecycle (loading, reference counting, unloading, Resources.UnloadUnusedAssets)
|
||||
4. Texture/mesh memory optimization (atlasing, LOD, compression)
|
||||
5. Draw call batching (static/dynamic batching, SRP Batcher, GPU instancing)
|
||||
6. Profiler-guided optimization (Unity Profiler, Memory Profiler, Frame Debugger)
|
||||
|
||||
For each optimization:
|
||||
- Problem it solves
|
||||
- Implementation approach
|
||||
- Measurable impact
|
||||
- Common mistakes
|
||||
|
||||
Output JSON with:
|
||||
- "pooling": object pool patterns
|
||||
- "gc_optimization": allocation reduction techniques
|
||||
- "asset_memory": asset lifecycle management
|
||||
- "rendering": draw call and GPU optimizations
|
||||
- "profiling": how to measure and verify
|
||||
|
||||
- name: unity_events
|
||||
type: custom
|
||||
target: events
|
||||
uses_history: true
|
||||
enabled: true
|
||||
prompt: >
|
||||
Document event and messaging patterns for Unity.
|
||||
|
||||
Cover:
|
||||
1. C# events and delegates (event Action, EventHandler)
|
||||
2. UnityEvent and Inspector-wired callbacks
|
||||
3. ScriptableObject event channels (SOA event pattern)
|
||||
4. Signal bus patterns (Zenject Signals, custom implementations)
|
||||
5. Observer pattern implementations
|
||||
6. Message broker / pub-sub patterns
|
||||
|
||||
For each pattern:
|
||||
- When to use it (tight coupling vs loose coupling trade-offs)
|
||||
- Memory leak prevention (unsubscribe in OnDestroy/OnDisable)
|
||||
- Thread safety considerations
|
||||
- Code example with subscribe/unsubscribe
|
||||
|
||||
Output JSON with:
|
||||
- "event_patterns": array of event/messaging approaches
|
||||
- "best_practices": subscription lifecycle management
|
||||
- "comparisons": when to use which pattern
|
||||
|
||||
- name: editor_integration
|
||||
type: custom
|
||||
target: editor
|
||||
uses_history: true
|
||||
enabled: true
|
||||
prompt: >
|
||||
Document Unity Editor integration patterns.
|
||||
|
||||
Cover:
|
||||
1. Custom Inspector/Editor scripts ([CustomEditor], EditorGUILayout)
|
||||
2. PropertyDrawer and DecoratorDrawer patterns
|
||||
3. EditorWindow for tool development
|
||||
4. Gizmos and Handles for scene visualization
|
||||
5. Menu items and context menus ([MenuItem], [ContextMenu])
|
||||
6. Build pipeline hooks (IPreprocessBuildWithReport, IPostprocessBuildWithReport)
|
||||
|
||||
For each pattern:
|
||||
- Use case and setup
|
||||
- Code example
|
||||
- Editor vs runtime separation (#if UNITY_EDITOR)
|
||||
|
||||
Output JSON with:
|
||||
- "custom_inspectors": inspector customization patterns
|
||||
- "editor_tools": tool development patterns
|
||||
- "build_pipeline": build automation patterns
|
||||
|
||||
post_process:
|
||||
reorder_sections:
|
||||
- monobehaviour_lifecycle
|
||||
- component_architecture
|
||||
- dependency_injection
|
||||
- async_patterns
|
||||
- memory_and_performance
|
||||
- unity_events
|
||||
- editor_integration
|
||||
add_metadata:
|
||||
enhanced: true
|
||||
workflow: unity-game-dev
|
||||
framework: unity
|
||||
has_lifecycle_docs: true
|
||||
has_di_patterns: true
|
||||
has_performance_guide: true
|
||||
Reference in New Issue
Block a user