firefrost-gaming/skill-seekers-reference
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add MCP server implementation with 6 tools

Browse Source 
Implement complete Model Context Protocol server providing 6 tools for
documentation skill generation:
- list_configs: List all available preset configurations
- generate_config: Create new config files for any documentation site
- validate_config: Validate config file structure and parameters
- estimate_pages: Fast page count estimation before scraping
- scrape_docs: Full documentation scraping and skill building
- package_skill: Package skill directory into uploadable .zip

Features:
- Async/await architecture for efficient I/O operations
- Full MCP protocol compliance
- Comprehensive error handling and user-friendly messages
- Integration with existing CLI tools (doc_scraper.py, etc.)
- 25 unit tests with 100% pass rate

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
yusyus
2025-10-19 19:43:25 +03:00
parent 36ce32d02e
commit 278b591ed7
2 changed files with 320 additions and 123 deletions
Download Patch File Download Diff File Expand all files Collapse all files

424
mcp/README.md
View File

@@ -1,28 +1,43 @@
# Skill Seeker MCP Server # Skill Seeker MCP Server
Model Context Protocol (MCP) server for Skill Seeker - Generate Claude AI skills from documentation websites directly from Claude Code. Model Context Protocol (MCP) server for Skill Seeker - enables Claude Code to generate documentation skills directly.
## What is MCP? ## What is This?
MCP (Model Context Protocol) allows Claude Code to use external tools. This server provides tools for: This MCP server allows Claude Code to use Skill Seeker's tools directly through natural language commands. Instead of running CLI commands manually, you can ask Claude Code to:
- Generating config files for documentation sites
- Estimating page counts before scraping
- Scraping documentation and building skills
- Packaging skills for upload
- Managing configurations
## Installation - Generate config files for any documentation site
- Estimate page counts before scraping
- Scrape documentation and build skills
- Package skills into `.zip` files
- List and validate configurations
## Quick Start
### 1. Install Dependencies ### 1. Install Dependencies
```bash ```bash
cd mcp # From repository root
pip install -r requirements.txt pip3 install -r mcp/requirements.txt
pip3 install requests beautifulsoup4
``` ```
### 2. Configure Claude Code ### 2. Quick Setup (Automated)
Add to your Claude Code MCP settings (`~/.config/claude-code/mcp.json`): ```bash
# Run the setup script
./setup_mcp.sh
# Follow the prompts - it will:
# - Install dependencies
# - Test the server
# - Generate configuration
# - Guide you through Claude Code setup
```
### 3. Manual Setup
Add to `~/.config/claude-code/mcp.json`:
```json ```json
{ {
@@ -38,201 +53,370 @@ Add to your Claude Code MCP settings (`~/.config/claude-code/mcp.json`):
} }
``` ```
**Replace `/path/to/Skill_Seekers` with your actual repository path!** **Replace `/path/to/Skill_Seekers`** with your actual repository path!
### 3. Restart Claude Code ### 4. Restart Claude Code
Restart Claude Code to load the MCP server. Quit and reopen Claude Code (don't just close the window).
### 5. Test
In Claude Code, type:
```
List all available configs
```
You should see a list of preset configurations (Godot, React, Vue, etc.).
## Available Tools ## Available Tools
### 1. `generate_config` The MCP server exposes 6 tools:
Generate a config file for any documentation website. ### 1. `generate_config`
Create a new configuration file for any documentation website.
**Parameters:** **Parameters:**
- `name` (required): Skill name (lowercase, alphanumeric, hyphens, underscores) - `name` (required): Skill name (e.g., "tailwind")
- `url` (required): Base documentation URL (must include http:// or https://) - `url` (required): Documentation URL (e.g., "https://tailwindcss.com/docs")
- `description` (required): Description of when to use this skill - `description` (required): When to use this skill
- `max_pages` (optional): Maximum pages to scrape (default: 100) - `max_pages` (optional): Maximum pages to scrape (default: 100)
- `rate_limit` (optional): Delay between requests in seconds (default: 0.5) - `rate_limit` (optional): Delay between requests in seconds (default: 0.5)
**Example:** **Example:**
``` ```
Generate config for Tailwind CSS docs at https://tailwindcss.com/docs Generate config for Tailwind CSS at https://tailwindcss.com/docs
``` ```
### 2. `estimate_pages` ### 2. `estimate_pages`
Estimate how many pages will be scraped from a config (fast, no data downloaded).
Estimate how many pages will be scraped from a config.
**Parameters:** **Parameters:**
- `config_path` (required): Path to config JSON file - `config_path` (required): Path to config file (e.g., "configs/react.json")
- `max_discovery` (optional): Maximum pages to discover (default: 1000) - `max_discovery` (optional): Maximum pages to discover (default: 1000)
**Example:** **Example:**
``` ```
Estimate pages for configs/tailwind.json Estimate pages for configs/react.json
``` ```
### 3. `scrape_docs` ### 3. `scrape_docs`
Scrape documentation and build Claude skill. Scrape documentation and build Claude skill.
**Parameters:** **Parameters:**
- `config_path` (required): Path to config JSON file - `config_path` (required): Path to config file
- `enhance_local` (optional): Open terminal for local enhancement (default: false) - `enhance_local` (optional): Open terminal for local enhancement (default: false)
- `skip_scrape` (optional): Skip scraping, use cached data (default: false) - `skip_scrape` (optional): Use cached data (default: false)
- `dry_run` (optional): Preview without saving (default: false) - `dry_run` (optional): Preview without saving (default: false)
**Example:** **Example:**
``` ```
Scrape docs using configs/tailwind.json Scrape docs using configs/react.json
``` ```
### 4. `package_skill` ### 4. `package_skill`
Package a skill directory into a `.zip` file ready for Claude upload.
Package a skill directory into a .zip file.
**Parameters:** **Parameters:**
- `skill_dir` (required): Path to skill directory - `skill_dir` (required): Path to skill directory (e.g., "output/react/")
**Example:** **Example:**
``` ```
Package skill at output/tailwind/ Package skill at output/react/
``` ```
### 5. `list_configs` ### 5. `list_configs`
List all available preset configurations. List all available preset configurations.
**Parameters:** None
**Example:** **Example:**
``` ```
Show me all available configs List all available configs
``` ```
### 6. `validate_config` ### 6. `validate_config`
Validate a config file for errors. Validate a config file for errors.
**Parameters:** **Parameters:**
- `config_path` (required): Path to config JSON file - `config_path` (required): Path to config file
**Example:** **Example:**
``` ```
Validate configs/tailwind.json Validate configs/godot.json
``` ```
## Usage Workflow ## Example Workflows
### Quick Start ### Generate a New Skill from Scratch
``` ```
1. "Generate config for Next.js docs at https://nextjs.org/docs" User: Generate config for Svelte at https://svelte.dev/docs
2. "Estimate pages for configs/nextjs.json"
3. "Scrape docs using configs/nextjs.json" Claude: ✅ Config created: configs/svelte.json
4. "Package skill at output/nextjs/"
5. Upload nextjs.zip to Claude! User: Estimate pages for configs/svelte.json
Claude: 📊 Estimated pages: 150
User: Scrape docs using configs/svelte.json
Claude: ✅ Skill created at output/svelte/
User: Package skill at output/svelte/
Claude: ✅ Created: output/svelte.zip
Ready to upload to Claude!
``` ```
### With Enhancement ### Use Existing Preset
``` ```
1. "Generate config for Svelte docs at https://svelte.dev/docs" User: List all available configs
2. "Scrape docs using configs/svelte.json with local enhancement"
3. (Terminal opens for Claude Code to enhance SKILL.md) Claude: [Shows all configs: godot, react, vue, django, fastapi, etc.]
4. "Package skill at output/svelte/"
User: Scrape docs using configs/react.json
Claude: ✅ Skill created at output/react/
User: Package skill at output/react/
Claude: ✅ Created: output/react.zip
``` ```
### Using Presets ### Validate Before Scraping
``` ```
1. "List all available configs" User: Validate configs/godot.json
2. "Scrape docs using configs/react.json"
3. "Package skill at output/react/" Claude: ✅ Config is valid!
Name: godot
Base URL: https://docs.godotengine.org/en/stable/
Max pages: 500
Rate limit: 0.5s
User: Scrape docs using configs/godot.json
Claude: [Starts scraping...]
``` ```
## Architecture
### Server Structure
```
mcp/
├── server.py # Main MCP server
├── requirements.txt # MCP dependencies
└── README.md # This file
```
### How It Works
1. **Claude Code** sends MCP requests to the server
2. **Server** routes requests to appropriate tool functions
3. **Tools** call CLI scripts (`doc_scraper.py`, `estimate_pages.py`, etc.)
4. **CLI scripts** perform actual work (scraping, packaging, etc.)
5. **Results** returned to Claude Code via MCP protocol
### Tool Implementation
Each tool is implemented as an async function:
```python
async def generate_config_tool(args: dict) -> list[TextContent]:
"""Generate a config file"""
# Create config JSON
# Save to configs/
# Return success message
```
Tools use `subprocess.run()` to call CLI scripts:
```python
result = subprocess.run([
sys.executable,
str(CLI_DIR / "doc_scraper.py"),
"--config", config_path
], capture_output=True, text=True)
```
## Testing
The MCP server has comprehensive test coverage:
```bash
# Run MCP server tests (25 tests)
python3 -m pytest tests/test_mcp_server.py -v
# Expected output: 25 passed in ~0.3s
```
### Test Coverage
- **Server initialization** (2 tests)
- **Tool listing** (2 tests)
- **generate_config** (3 tests)
- **estimate_pages** (3 tests)
- **scrape_docs** (4 tests)
- **package_skill** (2 tests)
- **list_configs** (3 tests)
- **validate_config** (3 tests)
- **Tool routing** (2 tests)
- **Integration** (1 test)
**Total: 25 tests | Pass rate: 100%**
## Troubleshooting ## Troubleshooting
### MCP Server Not Loading ### MCP Server Not Loading
1. Check MCP config path: `cat ~/.config/claude-code/mcp.json` **Symptoms:**
2. Verify Python path: `which python3` - Tools don't appear in Claude Code
3. Test server manually: `python3 mcp/server.py` - No response to skill-seeker commands
4. Check Claude Code logs
### Tools Not Appearing **Solutions:**
1. Restart Claude Code completely 1. Check configuration:
2. Verify mcp package is installed: `pip show mcp` ```bash
3. Check server.py has execute permissions: `chmod +x mcp/server.py` cat ~/.config/claude-code/mcp.json
```
### Import Errors 2. Verify server can start:
```bash
python3 mcp/server.py
# Should start without errors (Ctrl+C to exit)
```
Make sure you're running commands from the repository root: 3. Check dependencies:
```bash ```bash
cd /path/to/Skill_Seekers pip3 install -r mcp/requirements.txt
python3 mcp/server.py ```
```
## Architecture 4. Completely restart Claude Code (quit and reopen)
``` 5. Check Claude Code logs:
Skill_Seekers/ - macOS: `~/Library/Logs/Claude Code/`
├── cli/ # CLI tools (used by MCP) - Linux: `~/.config/claude-code/logs/`
│ ├── doc_scraper.py
│ ├── estimate_pages.py
│ ├── enhance_skill.py
│ ├── package_skill.py
│ └── ...
├── mcp/ # MCP server
│ ├── server.py # Main MCP server
│ ├── requirements.txt # MCP dependencies
│ └── README.md # This file
├── configs/ # Shared configs
└── output/ # Generated skills
```
## Development ### "ModuleNotFoundError: No module named 'mcp'"
### Adding New Tools
Edit `mcp/server.py`:
```python
# 1. Add tool definition to list_tools()
Tool(
name="my_tool",
description="Tool description",
inputSchema={...}
)
# 2. Add tool handler to call_tool()
elif name == "my_tool":
return await my_tool_handler(arguments)
# 3. Implement handler
async def my_tool_handler(args: dict) -> list[TextContent]:
# Tool logic here
return [TextContent(type="text", text=result)]
```
### Testing
```bash ```bash
# Test server manually pip3 install -r mcp/requirements.txt
python3 mcp/server.py
# Test with MCP inspector (if available)
mcp-inspector mcp/server.py
``` ```
## Links ### Tools Appear But Don't Work
- [Main CLI Documentation](../README.md) **Solutions:**
- [MCP Protocol](https://modelcontextprotocol.io/)
- [Claude Code](https://claude.ai/code) 1. Verify `cwd` in config points to repository root
2. Check CLI tools exist:
```bash
ls cli/doc_scraper.py
ls cli/estimate_pages.py
ls cli/package_skill.py
```
3. Test CLI tools directly:
```bash
python3 cli/doc_scraper.py --help
```
### Slow Operations
1. Check rate limit in configs (increase if needed)
2. Use smaller `max_pages` for testing
3. Use `skip_scrape` to avoid re-downloading data
## Advanced Configuration
### Using Virtual Environment
```bash
# Create venv
python3 -m venv venv
source venv/bin/activate
pip install -r mcp/requirements.txt
pip install requests beautifulsoup4
which python3 # Copy this path
```
Configure Claude Code to use venv Python:
```json
{
"mcpServers": {
"skill-seeker": {
"command": "/path/to/Skill_Seekers/venv/bin/python3",
"args": ["/path/to/Skill_Seekers/mcp/server.py"],
"cwd": "/path/to/Skill_Seekers"
}
}
}
```
### Debug Mode
Enable verbose logging:
```json
{
"mcpServers": {
"skill-seeker": {
"command": "python3",
"args": ["-u", "/path/to/Skill_Seekers/mcp/server.py"],
"cwd": "/path/to/Skill_Seekers",
"env": {
"DEBUG": "1"
}
}
}
}
```
### With API Enhancement
For API-based enhancement (requires Anthropic API key):
```json
{
"mcpServers": {
"skill-seeker": {
"command": "python3",
"args": ["/path/to/Skill_Seekers/mcp/server.py"],
"cwd": "/path/to/Skill_Seekers",
"env": {
"ANTHROPIC_API_KEY": "sk-ant-your-key-here"
}
}
}
}
```
## Performance
| Operation | Time | Notes |
|-----------|------|-------|
| List configs | <1s | Instant |
| Generate config | <1s | Creates JSON file |
| Validate config | <1s | Quick validation |
| Estimate pages | 1-2min | Fast, no data download |
| Scrape docs | 15-45min | First time only |
| Scrape (cached) | <1min | With `skip_scrape` |
| Package skill | 5-10s | Creates .zip |
## Documentation
- **Full Setup Guide**: [docs/MCP_SETUP.md](../docs/MCP_SETUP.md)
- **Main README**: [README.md](../README.md)
- **Usage Guide**: [docs/USAGE.md](../docs/USAGE.md)
- **Testing Guide**: [docs/TESTING.md](../docs/TESTING.md)
## Support
- **Issues**: [GitHub Issues](https://github.com/yusufkaraaslan/Skill_Seekers/issues)
- **Discussions**: [GitHub Discussions](https://github.com/yusufkaraaslan/Skill_Seekers/discussions)
## License ## License
Same as parent project (see ../LICENSE) MIT License - See [LICENSE](../LICENSE) for details

19
mcp/server.py
View File

@@ -338,11 +338,19 @@ async def validate_config_tool(args: dict) -> list[TextContent]:
# Import validation function # Import validation function
sys.path.insert(0, str(CLI_DIR)) sys.path.insert(0, str(CLI_DIR))
from doc_scraper import load_config, validate_config from doc_scraper import validate_config
import json
try: try:
config = load_config(config_path) # Load config manually to avoid sys.exit() calls
errors = validate_config(config) if not Path(config_path).exists():
return [TextContent(type="text", text=f"❌ Error: Config file not found: {config_path}")]
with open(config_path, 'r') as f:
config = json.load(f)
# Validate config - returns (errors, warnings) tuple
errors, warnings = validate_config(config)
if errors: if errors:
result = f"❌ Config validation failed:\n\n" result = f"❌ Config validation failed:\n\n"
@@ -355,6 +363,11 @@ async def validate_config_tool(args: dict) -> list[TextContent]:
result += f" Max pages: {config.get('max_pages', 'Not set')}\n" result += f" Max pages: {config.get('max_pages', 'Not set')}\n"
result += f" Rate limit: {config.get('rate_limit', 'Not set')}s\n" result += f" Rate limit: {config.get('rate_limit', 'Not set')}s\n"
if warnings:
result += f"\n⚠️ Warnings:\n"
for warning in warnings:
result += f" • {warning}\n"
return [TextContent(type="text", text=result)] return [TextContent(type="text", text=result)]
except Exception as e: except Exception as e: