Commit Graph

220 Commits

Author SHA1 Message Date
yusyus
72611af87d feat(v2.3.0): Add multi-agent installation support
Add automatic skill installation to 10+ AI coding agents with a single command.

New Features:
- New install-agent command for installing skills to any AI agent
- Support for 10+ agents: Claude Code, Cursor, VS Code, Amp, Goose, OpenCode, Letta, Aide, Windsurf
- Smart path resolution (global ~/.agent vs project-relative .agent/)
- Fuzzy agent name matching with suggestions
- --agent all flag to install to all agents at once
- --force flag to overwrite existing installations
- --dry-run flag to preview installations
- Comprehensive error handling and user feedback

Implementation:
- Created install_agent.py (379 lines) with core installation logic
- Updated main.py with install-agent subcommand
- Updated pyproject.toml with entry point
- Added 32 comprehensive tests (all passing, 603 total)
- No regressions in existing functionality

Documentation:
- Updated README.md with multi-agent installation guide
- Updated CLAUDE.md with install-agent examples
- Updated CHANGELOG.md with v2.3.0 release notes
- Added agent compatibility table

Technical Details:
- 100% own implementation (no external dependencies)
- Pure Python using stdlib (shutil, pathlib, argparse)
- Compatible with Agent Skills open standard (agentskills.io)
- Works offline

Closes #210

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2025-12-22 02:04:32 +03:00
yusyus
9cca9488e4 fix: Update version string in CLI to 2.2.0 2025-12-21 23:18:43 +03:00
yusyus
785fff087e feat: Add unified language detector for code analysis
- Created LanguageDetector class supporting 20+ programming languages
- Confidence-based detection with customizable thresholds (min_confidence parameter)
- Replaces duplicate language detection code in doc_scraper and pdf_extractor
- Comprehensive test suite with 100+ test cases

Changes:
- NEW: src/skill_seekers/cli/language_detector.py (17 KB)
  - Unified detector with pattern matching for 20+ languages
  - Confidence scoring (0.0-1.0 scale)
  - Supports: Python, JavaScript, TypeScript, Java, C++, C#, Go, Rust, PHP, Ruby, Swift, Kotlin, Shell, SQL, HTML, CSS, JSON, YAML, XML, and more

- NEW: tests/test_language_detector.py (20 KB)
  - 100+ test cases covering all supported languages
  - Edge case testing (mixed code, low confidence, etc.)

- MODIFIED: src/skill_seekers/cli/doc_scraper.py
  - Removed 80+ lines of duplicate detection code
  - Now uses shared LanguageDetector instance

- MODIFIED: src/skill_seekers/cli/pdf_extractor_poc.py
  - Removed 130+ lines of duplicate detection code
  - Now uses shared LanguageDetector instance

- MODIFIED: tests/test_pdf_extractor.py
  - Fixed imports to use proper package paths
  - Added manual detector initialization in test setup

Benefits:
- DRY: Single source of truth for language detection
- Maintainability: Add new languages in one place
- Consistency: Same detection logic across all scrapers
- Testability: Comprehensive test coverage
- Extensibility: Easy to add new languages or improve patterns

Addresses technical debt from having duplicate detection logic in multiple files.
2025-12-21 22:53:05 +03:00
Joseph Magly
0d0eda7149 feat(utils): add retry utilities with exponential backoff (#208)
Add retry_with_backoff() and retry_with_backoff_async() for network operations.

Features:
- Configurable max attempts (default: 3)
- Exponential backoff with configurable base delay
- Operation name for meaningful log messages
- Both sync and async versions

Addresses E2.6: Add retry logic for network failures

Co-authored-by: Joseph Magly <1159087+jmagly@users.noreply.github.com>
2025-12-21 22:31:38 +03:00
yusyus
65ded6c07c fix: Fix local repo extraction limitations (code analyzer, exclusions, enhancement)
This commit fixes three critical limitations discovered during local repository skill extraction testing:

**Fix 1: Code Analyzer Import Issue**
- Changed unified_scraper.py to use absolute imports instead of relative imports
- Fixed: `from github_scraper import` → `from skill_seekers.cli.github_scraper import`
- Fixed: `from pdf_scraper import` → `from skill_seekers.cli.pdf_scraper import`
- Result: CodeAnalyzer now available during extraction, deep analysis works

**Fix 2: Unity Library Exclusions**
- Updated should_exclude_dir() to accept and check full directory paths
- Updated _extract_file_tree_local() to pass both dir name and full path
- Added exclusion config passing from unified_scraper to github_scraper
- Result: exclude_dirs_additional now works (297 files excluded in test)

**Fix 3: AI Enhancement for Single Sources**
- Changed read_reference_files() to use rglob() for recursive search
- Now finds reference files in subdirectories (e.g., references/github/README.md)
- Result: AI enhancement works with unified skills that have nested references

**Test Results:**
- Code Analyzer:  Working (deep analysis running)
- Unity Exclusions:  Working (297 files excluded from 679)
- AI Enhancement:  Working (finds and reads nested references)

**Files Changed:**
- src/skill_seekers/cli/unified_scraper.py (Fix 1 & 2)
- src/skill_seekers/cli/github_scraper.py (Fix 2)
- src/skill_seekers/cli/utils.py (Fix 3)

**Test Artifacts:**
- configs/deck_deck_go_local.json (test configuration)
- docs/LOCAL_REPO_TEST_RESULTS.md (comprehensive test report)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2025-12-21 22:24:38 +03:00
yusyus
b7cd317efb feat(A1.7): Add install_skill MCP tool for one-command workflow automation
Implements complete end-to-end skill installation in a single command:
fetch_config → scrape_docs → enhance_skill_local → package_skill → upload_skill

Changes:
- MCP Tool: Added install_skill_tool() to server.py (~300 lines)
  - Input validation (config_name XOR config_path)
  - 5-phase orchestration with error handling
  - Dry-run mode for workflow preview
  - Mandatory AI enhancement (30-60 sec, 3/10→9/10 quality boost)
  - Auto-upload to Claude (if ANTHROPIC_API_KEY set)

- CLI Integration: New install command
  - Created install_skill.py CLI wrapper (~150 lines)
  - Updated main.py with install subcommand
  - Added entry point to pyproject.toml

- Testing: Comprehensive test suite
  - Created test_install_skill.py with 13 tests
  - Tests cover validation, dry-run, orchestration, error handling
  - All tests passing (13/13)

- Documentation: Updated all user-facing docs
  - CLAUDE.md: Added MCP tool (10 tools total) and CLI examples
  - README.md: Added prominent one-command workflow section
  - FLEXIBLE_ROADMAP.md: Marked A1.7 as complete

Features:
- Zero friction: One command instead of 5 separate steps
- Quality guaranteed: Mandatory enhancement ensures 9/10 quality
- Complete automation: From config to uploaded skill
- Intelligent: Auto-detects config type (name vs path)
- Flexible: Dry-run, unlimited, no-upload modes
- Well-tested: 13 unit tests with mocking

Usage:
  skill-seekers install --config react
  skill-seekers install --config configs/custom.json --no-upload
  skill-seekers install --config django --unlimited
  skill-seekers install --config react --dry-run

Closes #204

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2025-12-21 20:17:59 +03:00
yusyus
cbacdb0e66 release: v2.1.1 - GitHub Repository Analysis Enhancements
Major improvements:
- Configurable directory exclusions (Issue #203)
- Unlimited local repository analysis
- Skip llms.txt option (PR #198)
- 10+ bug fixes for GitHub scraper
- Test suite expanded to 427 tests

See CHANGELOG.md for full details.
2025-11-30 12:22:28 +03:00
yusyus
ea289cebe1 feat: Make EXCLUDED_DIRS configurable for local repository analysis
Closes #203

Adds configuration options to customize directory exclusions during local
repository analysis, while maintaining backward compatibility with smart
defaults.

**New Config Options:**

1. `exclude_dirs_additional` - Extend defaults (most common)
   - Adds custom directories to default exclusions
   - Example: ["proprietary", "legacy", "third_party"]
   - Total exclusions = defaults + additional

2. `exclude_dirs` - Replace defaults (advanced users)
   - Completely overrides default exclusions
   - Example: ["node_modules", ".git", "custom_vendor"]
   - Gives full control over exclusions

**Implementation:**

- Modified GitHubScraper.__init__() to parse exclude_dirs config
- Changed should_exclude_dir() to use instance variable instead of global
- Added logging for custom exclusions (INFO for extend, WARNING for replace)
- Maintains backward compatibility (no config = use defaults)

**Testing:**

- Added 12 comprehensive tests in test_excluded_dirs_config.py
  - 3 tests for defaults (backward compatibility)
  - 3 tests for extend mode
  - 3 tests for replace mode
  - 1 test for precedence
  - 2 tests for edge cases
- All 12 new tests passing 
- All 22 existing github_scraper tests passing 

**Documentation:**

- Updated CLAUDE.md config parameters section
- Added detailed "Configurable Directory Exclusions" feature section
- Included examples for both modes
- Listed common use cases (monorepos, enterprise, legacy codebases)

**Use Cases:**

- Monorepos with custom directory structures
- Enterprise projects with non-standard naming conventions
- Including unusual directories for analysis
- Minimal exclusions for small/simple projects

**Backward Compatibility:**

 Fully backward compatible - existing configs work unchanged
 Smart defaults maintained when no config provided
 All existing tests pass

Co-authored-by: jimmy058910 <jimmy058910@users.noreply.github.com>
2025-11-29 23:53:27 +03:00
yusyus
bd20b32470 Merge PR #198: Skip llms.txt Config Option
Merges feat/add-skip-llm-to-config by @sogoiii.

This PR adds a valuable configuration option to explicitly skip llms.txt
detection, useful when a site's llms.txt is incomplete, incorrect, or when
specific HTML scraping is needed.

Key features:
- New 'skip_llms_txt' config option (default: false, backward compatible)
- Boolean type validation with warning for invalid values
- Support in both sync and async scraping modes
- 17 comprehensive tests (15 feature tests + 2 config validation tests)

All tests passing after fixing import paths to use proper package names.

Test results:  17/17 tests passing
Full test suite:  391 tests passing

Co-authored-by: sogoiii <sogoiii@users.noreply.github.com>
2025-11-29 22:56:46 +03:00
yusyus
58ec69eb52 feat: Add unlimited local repository analysis with bug fixes (PR #195)
Merges PR #195 by @jimmy058910 with conflict resolution.

**New Features:**
- Local repository analysis via `local_repo_path` configuration
- Bypass GitHub API rate limits (50 → unlimited files)
- Auto-exclusion of virtual environments and build artifacts
- Support for analyzing large codebases (323 files vs 50 before)

**Improvements:**
- Code analysis coverage: 14% → 93.6% (+79.6pp)
- Files analyzed: 50 → 323 (+546%)
- Classes extracted: 55 → 585 (+964%)
- Functions extracted: 512 → 2,784 (+444%)
- AST parsing errors: 95 → 0 (-100%)

**Conflict Resolution:**
- Preserved logger initialization fix from development (Issue #190)
- Kept relative imports from development (Task 1.2 fix)
- Integrated EXCLUDED_DIRS and local repo features from PR
- Combined best of both implementations

**Testing:**
-  All 22 GitHub scraper tests passing
-  Syntax validation passed
-  Local repo analysis feature intact
-  Bug fixes from development preserved

Original implementation by @jimmy058910 in PR #195.
Conflict resolution preserves all bug fixes while adding local repo feature.

Co-authored-by: jimmy058910 <jimmy058910@users.noreply.github.com>
2025-11-29 22:46:31 +03:00
yusyus
414519b3c7 fix: Initialize logger before use in github_scraper.py
Fixes Issue #190 - "name 'logger' is not defined" error

**Problem:**
- Logger was used at line 40 (in code_analyzer import exception)
- Logger was defined at line 47
- Caused runtime error when code_analyzer import failed

**Solution:**
- Moved logging.basicConfig() and logger initialization to lines 34-39
- Now logger is defined BEFORE the code_analyzer import block
- Warning message now works correctly when code_analyzer is missing

**Testing:**
-  All 22 GitHub scraper tests pass
-  Logger warning appears correctly when code_analyzer missing
-  No similar issues found in other CLI files

Closes #190
2025-11-29 22:01:38 +03:00
yusyus
d7a4c51427 fix: Convert absolute imports to relative imports in cli modules
Fixes #193 - PDF scraping broken for PyPI users

Changed 3 files from absolute to relative imports to fix
ModuleNotFoundError when package is installed via pip:

1. pdf_scraper.py:22
   - from pdf_extractor_poc import → from .pdf_extractor_poc import
   - Fixes: skill-seekers pdf command failed with import error

2. github_scraper.py:36
   - from code_analyzer import → from .code_analyzer import
   - Proactive fix: prevents future import errors

3. test_unified_simple.py:17
   - from config_validator import → from .config_validator import
   - Proactive fix: test helper file

These absolute imports worked locally due to sys.path differences
but failed when installed via PyPI (pip install skill-seekers).

Tested with:
- skill-seekers pdf command now works 
- Extracted 32-page Godot Farming PDF successfully

All CLI commands should now work correctly when installed from PyPI.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-29 21:47:18 +03:00
sogoiii
a0b1c2f42f feat: add skip_llms_txt config option to bypass llms.txt detection
- Add skip_llms_txt config option (default: False)
- Validate value is boolean, warn and default to False if not
- Support in both sync and async scraping modes
- Add 17 tests for config, behavior, and edge cases
2025-11-20 13:55:46 -08:00
Jimmy Moceri
0b2a0d121e feat: Add unlimited local repository analysis and fix 10 critical bugs
Features:
- Add local_repo_path config parameter for unlimited file analysis
- Auto-exclude virtual environments and build artifacts (95% noise reduction)
- Enable comprehensive codebase analysis (50 → 323 files, 546% increase)

Bug Fixes:
- Fix logger initialization error (Issue #190)
- Fix NoneType subscriptable errors in release tag parsing (3 instances)
- Fix relative import paths causing ModuleNotFoundError
- Fix hardcoded 50-file analysis limit
- Fix GitHub API file tree limitation (140 → 345 files discovered)
- Fix AST parser 'not iterable' errors (95 → 0 parsing failures)
- Fix virtual environment file pollution (23,341 → 1,109 file tree items)
- Fix force_rescrape flag not checked before interactive prompt

Impact:
- Code coverage: 14% → 93.6% (+79.6pp)
- Files analyzed: 50 → 323 (+546%)
- Classes extracted: 55 → 585 (+964%)
- Functions extracted: 512 → 2,784 (+444%)
- AST errors: 95 → 0 (-100%)

Tested on JMo Security repository with 345 Python files.
2025-11-16 22:35:23 -05:00
yusyus
3272f9c59d feat: Add comprehensive quality checker for skills
Phase 2 & 3: Quality assurance before packaging

New module: quality_checker.py
- Enhancement verification (checks for template text, code examples, sections)
- Structure validation (SKILL.md, references/ directory)
- Content quality checks (YAML frontmatter, language tags, "When to Use" section)
- Link validation (internal markdown links)
- Quality scoring system (0-100 score + A-F grade)
- Detailed reporting with errors, warnings, and info messages
- CLI with --verbose and --strict modes

Integration in package_skill.py:
- Automatic quality checks before packaging
- Display quality report with score and grade
- Ask user to confirm if warnings/errors found
- Add --skip-quality-check flag to bypass checks
- Updated help examples

Benefits:
- Catch quality issues before packaging
- Ensure SKILL.md is properly enhanced
- Validate all links work
- Give users confidence in skill quality
- Comprehensive quality reports

Addresses user request: "check some sort of quality check at the end
like is links working, skill is good etc and give report the user"

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-12 23:01:28 +03:00
yusyus
e279ed6ca8 fix: Enhancement race condition - add headless mode that waits
Phase 1: Fix race condition where main console exits before enhancement completes

Changes to enhance_skill_local.py:
- Add headless mode (default) using subprocess.run() which WAITS for completion
- Add timeout protection (default 10 minutes, configurable)
- Verify SKILL.md was actually updated (check mtime and size)
- Add --interactive-enhancement flag to use old terminal mode
- Detailed progress messages and error handling
- Clean up temp files after completion

Changes to doc_scraper.py:
- Use skill-seekers-enhance entry point instead of direct python path
- Pass --interactive-enhancement flag through if requested
- Update help text to reflect new headless default behavior
- Show proper status messages (HEADLESS vs INTERACTIVE)

Benefits:
- Main console now waits for enhancement to complete
- No more "Package your skill" message while enhancement is running
- Timeout prevents infinite hangs
- Terminal mode still available for users who want it
- Better error messages and progress tracking

Fixes user request: "make sure 1. console wait for it to finish"

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-12 22:53:01 +03:00
yusyus
4cbd0a0a3c fix: Add anthropic-beta header and correct field name for skill uploads
Fixes #182

Changes:
- Add 'anthropic-beta: skills-2025-10-02' header (required by Anthropic Skills API)
- Change multipart field name from 'skill' to 'files[]' (correct API format)

Without these fixes, all upload attempts returned 404 errors.

Verified:
- All 379 tests passing (100%)
- No regressions in test suite
- Upload functionality corrected per API requirements

Co-authored-by: Straughter "BatmanOsama" Guthrie <straughterguthrie@gmail.com>
Original PR: #183
2025-11-11 23:39:56 +03:00
yusyus
530a68d1dc fix: Update test imports and merge_sources for v2.0.0 release
- Fix conflict_detector import in merge_sources.py (use relative import)
- Update test_mcp_server.py to use skill_seekers.mcp.server imports
- Fix @patch decorators to reference full module path
- Add MCP_AVAILABLE guards to test_unified_mcp_integration.py
- Add proper skipif decorators for MCP tests
- All 379 tests now passing (0 failures)

Resolves import errors that occurred during PyPI package testing.
2025-11-11 22:26:52 +03:00
yusyus
13ca374295 refactor: Update CLI commands to use new unified entry points
Updated all command examples in CLI scripts from old pattern:
  python3 cli/<script>.py → skill-seekers <command>

Changes:
- doc_scraper.py → skill-seekers scrape
- github_scraper.py → skill-seekers github
- pdf_scraper.py → skill-seekers pdf
- unified_scraper.py → skill-seekers unified
- enhance_skill.py → skill-seekers enhance
- enhance_skill_local.py → skill-seekers enhance
- package_skill.py → skill-seekers package
- estimate_pages.py → skill-seekers estimate

This reflects the new modern Python packaging with proper entry
points. Users can now use clean commands instead of file paths.

Files updated: 10 CLI scripts

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-07 01:23:17 +03:00
yusyus
ce1c07b437 feat: Add modern Python packaging - Phase 1 (Foundation)
Implements issue #168 - Modern Python packaging with uv support

This is Phase 1 of the modernization effort, establishing the core
package structure and build system.

## Major Changes

### 1. Migrated to src/ Layout
- Moved cli/ → src/skill_seekers/cli/
- Moved skill_seeker_mcp/ → src/skill_seekers/mcp/
- Created root package: src/skill_seekers/__init__.py
- Updated all imports: cli. → skill_seekers.cli.
- Updated all imports: skill_seeker_mcp. → skill_seekers.mcp.

### 2. Created pyproject.toml
- Modern Python packaging configuration
- All dependencies properly declared
- 8 CLI entry points configured:
  * skill-seekers (unified CLI)
  * skill-seekers-scrape
  * skill-seekers-github
  * skill-seekers-pdf
  * skill-seekers-unified
  * skill-seekers-enhance
  * skill-seekers-package
  * skill-seekers-upload
  * skill-seekers-estimate
- uv tool support enabled
- Build system: setuptools with wheel

### 3. Created Unified CLI (main.py)
- Git-style subcommands (skill-seekers scrape, etc.)
- Delegates to existing tool main() functions
- Full help system at top-level and subcommand level
- Backwards compatible with individual commands

### 4. Updated Package Versions
- cli/__init__.py: 1.3.0 → 2.0.0
- mcp/__init__.py: 1.2.0 → 2.0.0
- Root package: 2.0.0

### 5. Updated Test Suite
- Fixed test_package_structure.py for new layout
- All 28 package structure tests passing
- Updated all test imports for new structure

## Installation Methods (Working)

```bash
# Development install
pip install -e .

# Run unified CLI
skill-seekers --version  # → 2.0.0
skill-seekers --help

# Run individual tools
skill-seekers-scrape --help
skill-seekers-github --help
```

## Test Results
- Package structure tests: 28/28 passing 
- Package installs successfully 
- All entry points working 

## Still TODO (Phase 2)
- [ ] Run full test suite (299 tests)
- [ ] Update documentation (README, CLAUDE.md, etc.)
- [ ] Test with uv tool run/install
- [ ] Build and publish to PyPI
- [ ] Create PR and merge

## Breaking Changes
None - fully backwards compatible. Old import paths still work.

## Migration for Users
No action needed. Package works with both pip and uv.

Closes #168 (when complete)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-07 01:14:24 +03:00