Add comprehensive MCP setup guide and integration test template
**Documentation Added:** - docs/MCP_SETUP.md: Complete 400+ line setup guide - Prerequisites and installation steps - Configuration examples for Claude Code - Verification and troubleshooting - 3 usage examples and advanced configuration - End-to-end workflow and quick reference - tests/mcp_integration_test.md: Comprehensive test template - 10 test cases covering all MCP tools - Performance metrics table - Issue tracking and environment setup - Setup and cleanup scripts - .claude/mcp_config.example.json: Example MCP configuration **Documentation Updated:** - STRUCTURE.md: Complete monorepo structure documentation - CLAUDE.md: All Python script paths updated to cli/ prefix - docs/USAGE.md: All command examples updated for monorepo - TODO.md: Current sprint status and completed tasks **Summary:** - Issues #2 and #3 handled (MCP setup guide + integration tests) - All documentation now reflects monorepo structure (cli/ + mcp/) - Tests: 71/71 passing (100%) - Ready for MCP server testing with Claude Code 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
598
docs/MCP_SETUP.md
Normal file
598
docs/MCP_SETUP.md
Normal file
@@ -0,0 +1,598 @@
|
||||
# Complete MCP Setup Guide for Claude Code
|
||||
|
||||
Step-by-step guide to set up the Skill Seeker MCP server with Claude Code.
|
||||
|
||||
---
|
||||
|
||||
## Table of Contents
|
||||
|
||||
- [Prerequisites](#prerequisites)
|
||||
- [Installation](#installation)
|
||||
- [Configuration](#configuration)
|
||||
- [Verification](#verification)
|
||||
- [Usage Examples](#usage-examples)
|
||||
- [Troubleshooting](#troubleshooting)
|
||||
- [Advanced Configuration](#advanced-configuration)
|
||||
|
||||
---
|
||||
|
||||
## Prerequisites
|
||||
|
||||
### Required Software
|
||||
|
||||
1. **Python 3.7 or higher**
|
||||
```bash
|
||||
python3 --version
|
||||
# Should show: Python 3.7.x or higher
|
||||
```
|
||||
|
||||
2. **Claude Code installed**
|
||||
- Download from [claude.ai/code](https://claude.ai/code)
|
||||
- Requires Claude Pro or Claude Code Max subscription
|
||||
|
||||
3. **Skill Seeker repository cloned**
|
||||
```bash
|
||||
git clone https://github.com/yusufkaraaslan/Skill_Seekers.git
|
||||
cd Skill_Seekers
|
||||
```
|
||||
|
||||
### System Requirements
|
||||
|
||||
- **Operating System**: macOS, Linux, or Windows (WSL)
|
||||
- **Disk Space**: 100 MB for dependencies + space for generated skills
|
||||
- **Network**: Internet connection for documentation scraping
|
||||
|
||||
---
|
||||
|
||||
## Installation
|
||||
|
||||
### Step 1: Install Python Dependencies
|
||||
|
||||
```bash
|
||||
# Navigate to repository root
|
||||
cd /path/to/Skill_Seekers
|
||||
|
||||
# Install MCP server dependencies
|
||||
pip3 install -r mcp/requirements.txt
|
||||
|
||||
# Install CLI tool dependencies (for scraping)
|
||||
pip3 install requests beautifulsoup4
|
||||
```
|
||||
|
||||
**Expected output:**
|
||||
```
|
||||
Successfully installed mcp-0.9.0 requests-2.31.0 beautifulsoup4-4.12.3
|
||||
```
|
||||
|
||||
### Step 2: Verify Installation
|
||||
|
||||
```bash
|
||||
# Test MCP server can start
|
||||
python3 mcp/server.py
|
||||
|
||||
# You should see MCP protocol initialization (Ctrl+C to exit)
|
||||
```
|
||||
|
||||
### Step 3: Note Your Repository Path
|
||||
|
||||
```bash
|
||||
# Get absolute path
|
||||
pwd
|
||||
|
||||
# Example output: /Users/username/Projects/Skill_Seekers
|
||||
# or: /home/username/Skill_Seekers
|
||||
```
|
||||
|
||||
**Save this path** - you'll need it for configuration!
|
||||
|
||||
---
|
||||
|
||||
## Configuration
|
||||
|
||||
### Step 1: Locate Claude Code MCP Configuration
|
||||
|
||||
Claude Code stores MCP configuration in:
|
||||
|
||||
- **macOS**: `~/.config/claude-code/mcp.json`
|
||||
- **Linux**: `~/.config/claude-code/mcp.json`
|
||||
- **Windows (WSL)**: `~/.config/claude-code/mcp.json`
|
||||
|
||||
### Step 2: Create/Edit Configuration File
|
||||
|
||||
```bash
|
||||
# Create config directory if it doesn't exist
|
||||
mkdir -p ~/.config/claude-code
|
||||
|
||||
# Edit the configuration
|
||||
nano ~/.config/claude-code/mcp.json
|
||||
```
|
||||
|
||||
### Step 3: Add Skill Seeker MCP Server
|
||||
|
||||
**Full Configuration Example:**
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"skill-seeker": {
|
||||
"command": "python3",
|
||||
"args": [
|
||||
"/Users/username/Projects/Skill_Seekers/mcp/server.py"
|
||||
],
|
||||
"cwd": "/Users/username/Projects/Skill_Seekers",
|
||||
"env": {}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**IMPORTANT:** Replace `/Users/username/Projects/Skill_Seekers` with YOUR actual repository path!
|
||||
|
||||
**If you already have other MCP servers:**
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"existing-server": {
|
||||
"command": "node",
|
||||
"args": ["/path/to/existing/server.js"]
|
||||
},
|
||||
"skill-seeker": {
|
||||
"command": "python3",
|
||||
"args": [
|
||||
"/Users/username/Projects/Skill_Seekers/mcp/server.py"
|
||||
],
|
||||
"cwd": "/Users/username/Projects/Skill_Seekers"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Step 4: Save and Restart Claude Code
|
||||
|
||||
1. Save the file (`Ctrl+O` in nano, then `Enter`)
|
||||
2. Exit editor (`Ctrl+X` in nano)
|
||||
3. **Completely restart Claude Code** (quit and reopen)
|
||||
|
||||
---
|
||||
|
||||
## Verification
|
||||
|
||||
### Step 1: Check MCP Server Loaded
|
||||
|
||||
In Claude Code, type:
|
||||
```
|
||||
List all available MCP tools
|
||||
```
|
||||
|
||||
You should see 6 Skill Seeker tools:
|
||||
- `generate_config`
|
||||
- `estimate_pages`
|
||||
- `scrape_docs`
|
||||
- `package_skill`
|
||||
- `list_configs`
|
||||
- `validate_config`
|
||||
|
||||
### Step 2: Test a Simple Command
|
||||
|
||||
```
|
||||
List all available configs
|
||||
```
|
||||
|
||||
**Expected response:**
|
||||
```
|
||||
Available configurations:
|
||||
1. godot - Godot Engine documentation
|
||||
2. react - React framework
|
||||
3. vue - Vue.js framework
|
||||
4. django - Django web framework
|
||||
5. fastapi - FastAPI Python framework
|
||||
6. kubernetes - Kubernetes documentation
|
||||
7. steam-economy-complete - Steam Economy API
|
||||
```
|
||||
|
||||
### Step 3: Test Config Generation
|
||||
|
||||
```
|
||||
Generate a config for Tailwind CSS at https://tailwindcss.com/docs
|
||||
```
|
||||
|
||||
**Expected response:**
|
||||
```
|
||||
✅ Config created: configs/tailwind.json
|
||||
```
|
||||
|
||||
**Verify the file exists:**
|
||||
```bash
|
||||
ls configs/tailwind.json
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Usage Examples
|
||||
|
||||
### Example 1: Generate Skill from Scratch
|
||||
|
||||
```
|
||||
User: Generate config for Svelte docs at https://svelte.dev/docs
|
||||
|
||||
Claude: ✅ Config created: configs/svelte.json
|
||||
|
||||
User: Estimate pages for configs/svelte.json
|
||||
|
||||
Claude: 📊 Estimated pages: 150
|
||||
Recommended max_pages: 180
|
||||
|
||||
User: Scrape docs using configs/svelte.json
|
||||
|
||||
Claude: ✅ Skill created at output/svelte/
|
||||
Run: python3 cli/package_skill.py output/svelte/
|
||||
|
||||
User: Package skill at output/svelte/
|
||||
|
||||
Claude: ✅ Created: output/svelte.zip
|
||||
Ready to upload to Claude!
|
||||
```
|
||||
|
||||
### Example 2: Use Existing Config
|
||||
|
||||
```
|
||||
User: List all available configs
|
||||
|
||||
Claude: [Shows 7 configs]
|
||||
|
||||
User: Scrape docs using configs/react.json with max 50 pages
|
||||
|
||||
Claude: ✅ Skill created at output/react/
|
||||
|
||||
User: Package skill at output/react/
|
||||
|
||||
Claude: ✅ Created: output/react.zip
|
||||
```
|
||||
|
||||
### Example 3: Validate Before Scraping
|
||||
|
||||
```
|
||||
User: Validate configs/godot.json
|
||||
|
||||
Claude: ✅ Config is valid
|
||||
- Base URL: https://docs.godotengine.org/en/stable/
|
||||
- Max pages: 500
|
||||
- Rate limit: 0.5s
|
||||
- Categories: 3
|
||||
|
||||
User: Estimate pages for configs/godot.json
|
||||
|
||||
Claude: 📊 Estimated pages: 450
|
||||
Current max_pages (500) is sufficient
|
||||
|
||||
User: Scrape docs using configs/godot.json
|
||||
|
||||
Claude: [Scraping starts...]
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Issue: MCP Server Not Loading
|
||||
|
||||
**Symptoms:**
|
||||
- Skill Seeker tools don't appear in Claude Code
|
||||
- No response when asking about configs
|
||||
|
||||
**Solutions:**
|
||||
|
||||
1. **Check configuration path:**
|
||||
```bash
|
||||
cat ~/.config/claude-code/mcp.json
|
||||
```
|
||||
|
||||
2. **Verify Python path:**
|
||||
```bash
|
||||
which python3
|
||||
# Should show: /usr/bin/python3 or /usr/local/bin/python3
|
||||
```
|
||||
|
||||
3. **Test server manually:**
|
||||
```bash
|
||||
cd /path/to/Skill_Seekers
|
||||
python3 mcp/server.py
|
||||
# Should start without errors
|
||||
```
|
||||
|
||||
4. **Check Claude Code logs:**
|
||||
- macOS: `~/Library/Logs/Claude Code/`
|
||||
- Linux: `~/.config/claude-code/logs/`
|
||||
|
||||
5. **Completely restart Claude Code:**
|
||||
- Quit Claude Code (don't just close window)
|
||||
- Reopen Claude Code
|
||||
|
||||
### Issue: "ModuleNotFoundError: No module named 'mcp'"
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
pip3 install -r mcp/requirements.txt
|
||||
```
|
||||
|
||||
### Issue: "Permission denied" when running server
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
chmod +x mcp/server.py
|
||||
```
|
||||
|
||||
### Issue: Tools appear but don't work
|
||||
|
||||
**Symptoms:**
|
||||
- Tools listed but commands fail
|
||||
- "Error executing tool" messages
|
||||
|
||||
**Solutions:**
|
||||
|
||||
1. **Check working directory in config:**
|
||||
```json
|
||||
{
|
||||
"cwd": "/FULL/PATH/TO/Skill_Seekers"
|
||||
}
|
||||
```
|
||||
|
||||
2. **Verify CLI tools exist:**
|
||||
```bash
|
||||
ls cli/doc_scraper.py
|
||||
ls cli/estimate_pages.py
|
||||
ls cli/package_skill.py
|
||||
```
|
||||
|
||||
3. **Test CLI tools directly:**
|
||||
```bash
|
||||
python3 cli/doc_scraper.py --help
|
||||
```
|
||||
|
||||
### Issue: Slow or hanging operations
|
||||
|
||||
**Solutions:**
|
||||
|
||||
1. **Check rate limit in config:**
|
||||
- Default: 0.5 seconds
|
||||
- Increase if needed: 1.0 or 2.0 seconds
|
||||
|
||||
2. **Use smaller max_pages for testing:**
|
||||
```
|
||||
Generate config with max_pages=20 for testing
|
||||
```
|
||||
|
||||
3. **Check network connection:**
|
||||
```bash
|
||||
curl -I https://docs.example.com
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Advanced Configuration
|
||||
|
||||
### Custom Environment Variables
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"skill-seeker": {
|
||||
"command": "python3",
|
||||
"args": ["/path/to/Skill_Seekers/mcp/server.py"],
|
||||
"cwd": "/path/to/Skill_Seekers",
|
||||
"env": {
|
||||
"ANTHROPIC_API_KEY": "sk-ant-...",
|
||||
"PYTHONPATH": "/custom/path"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Multiple Python Versions
|
||||
|
||||
If you have multiple Python versions:
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"skill-seeker": {
|
||||
"command": "/usr/local/bin/python3.11",
|
||||
"args": ["/path/to/Skill_Seekers/mcp/server.py"],
|
||||
"cwd": "/path/to/Skill_Seekers"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Virtual Environment
|
||||
|
||||
To use a Python virtual environment:
|
||||
|
||||
```bash
|
||||
# Create venv
|
||||
cd /path/to/Skill_Seekers
|
||||
python3 -m venv venv
|
||||
source venv/bin/activate
|
||||
pip install -r mcp/requirements.txt
|
||||
pip install requests beautifulsoup4
|
||||
which python3
|
||||
# Copy this path for config
|
||||
```
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"skill-seeker": {
|
||||
"command": "/path/to/Skill_Seekers/venv/bin/python3",
|
||||
"args": ["/path/to/Skill_Seekers/mcp/server.py"],
|
||||
"cwd": "/path/to/Skill_Seekers"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Debug Mode
|
||||
|
||||
Enable verbose logging:
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"skill-seeker": {
|
||||
"command": "python3",
|
||||
"args": [
|
||||
"-u",
|
||||
"/path/to/Skill_Seekers/mcp/server.py"
|
||||
],
|
||||
"cwd": "/path/to/Skill_Seekers",
|
||||
"env": {
|
||||
"DEBUG": "1"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Complete Example Configuration
|
||||
|
||||
**Minimal (recommended for most users):**
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"skill-seeker": {
|
||||
"command": "python3",
|
||||
"args": [
|
||||
"/Users/username/Projects/Skill_Seekers/mcp/server.py"
|
||||
],
|
||||
"cwd": "/Users/username/Projects/Skill_Seekers"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**With API enhancement:**
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"skill-seeker": {
|
||||
"command": "python3",
|
||||
"args": [
|
||||
"/Users/username/Projects/Skill_Seekers/mcp/server.py"
|
||||
],
|
||||
"cwd": "/Users/username/Projects/Skill_Seekers",
|
||||
"env": {
|
||||
"ANTHROPIC_API_KEY": "sk-ant-your-key-here"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## End-to-End Workflow
|
||||
|
||||
### Complete Setup and First Skill
|
||||
|
||||
```bash
|
||||
# 1. Install
|
||||
cd ~/Projects
|
||||
git clone https://github.com/yusufkaraaslan/Skill_Seekers.git
|
||||
cd Skill_Seekers
|
||||
pip3 install -r mcp/requirements.txt
|
||||
pip3 install requests beautifulsoup4
|
||||
|
||||
# 2. Configure
|
||||
mkdir -p ~/.config/claude-code
|
||||
cat > ~/.config/claude-code/mcp.json << 'EOF'
|
||||
{
|
||||
"mcpServers": {
|
||||
"skill-seeker": {
|
||||
"command": "python3",
|
||||
"args": [
|
||||
"/Users/username/Projects/Skill_Seekers/mcp/server.py"
|
||||
],
|
||||
"cwd": "/Users/username/Projects/Skill_Seekers"
|
||||
}
|
||||
}
|
||||
}
|
||||
EOF
|
||||
# (Replace paths with your actual paths!)
|
||||
|
||||
# 3. Restart Claude Code
|
||||
|
||||
# 4. Test in Claude Code:
|
||||
```
|
||||
|
||||
**In Claude Code:**
|
||||
```
|
||||
User: List all available configs
|
||||
User: Scrape docs using configs/react.json with max 50 pages
|
||||
User: Package skill at output/react/
|
||||
```
|
||||
|
||||
**Result:** `output/react.zip` ready to upload!
|
||||
|
||||
---
|
||||
|
||||
## Next Steps
|
||||
|
||||
After successful setup:
|
||||
|
||||
1. **Try preset configs:**
|
||||
- React: `scrape docs using configs/react.json`
|
||||
- Vue: `scrape docs using configs/vue.json`
|
||||
- Django: `scrape docs using configs/django.json`
|
||||
|
||||
2. **Create custom configs:**
|
||||
- `generate config for [framework] at [url]`
|
||||
|
||||
3. **Test with small limits first:**
|
||||
- Use `max_pages` parameter: `scrape docs using configs/test.json with max 20 pages`
|
||||
|
||||
4. **Explore enhancement:**
|
||||
- Use `--enhance-local` flag for AI-powered SKILL.md improvement
|
||||
|
||||
---
|
||||
|
||||
## Getting Help
|
||||
|
||||
- **Documentation**: See [mcp/README.md](../mcp/README.md)
|
||||
- **Issues**: [GitHub Issues](https://github.com/yusufkaraaslan/Skill_Seekers/issues)
|
||||
- **Examples**: See [.github/ISSUES_TO_CREATE.md](../.github/ISSUES_TO_CREATE.md) for test cases
|
||||
|
||||
---
|
||||
|
||||
## Quick Reference Card
|
||||
|
||||
```
|
||||
SETUP:
|
||||
1. Install dependencies: pip3 install -r mcp/requirements.txt
|
||||
2. Configure: ~/.config/claude-code/mcp.json
|
||||
3. Restart Claude Code
|
||||
|
||||
VERIFY:
|
||||
- "List all available configs"
|
||||
- "Validate configs/react.json"
|
||||
|
||||
GENERATE SKILL:
|
||||
1. "Generate config for [name] at [url]"
|
||||
2. "Estimate pages for configs/[name].json"
|
||||
3. "Scrape docs using configs/[name].json"
|
||||
4. "Package skill at output/[name]/"
|
||||
|
||||
TROUBLESHOOTING:
|
||||
- Check: cat ~/.config/claude-code/mcp.json
|
||||
- Test: python3 mcp/server.py
|
||||
- Logs: ~/Library/Logs/Claude Code/
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
Happy skill creating! 🚀
|
||||
138
docs/USAGE.md
138
docs/USAGE.md
@@ -20,19 +20,19 @@ Comprehensive reference for all commands, options, and workflows.
|
||||
|
||||
```bash
|
||||
# 1. Estimate pages (fast, 1-2 min)
|
||||
python3 estimate_pages.py configs/react.json
|
||||
python3 cli/estimate_pages.py configs/react.json
|
||||
|
||||
# 2. Scrape documentation (20-40 min)
|
||||
python3 doc_scraper.py --config configs/react.json
|
||||
python3 cli/doc_scraper.py --config configs/react.json
|
||||
|
||||
# 3. Enhance with Claude Code (60 sec)
|
||||
python3 enhance_skill_local.py output/react/
|
||||
python3 cli/enhance_skill_local.py output/react/
|
||||
|
||||
# 4. Package to .zip (instant)
|
||||
python3 package_skill.py output/react/
|
||||
python3 cli/package_skill.py output/react/
|
||||
|
||||
# 5. Test everything (1 sec)
|
||||
python3 run_tests.py
|
||||
python3 cli/run_tests.py
|
||||
```
|
||||
|
||||
---
|
||||
@@ -70,16 +70,16 @@ options:
|
||||
|
||||
**1. Use Preset Config (Recommended)**
|
||||
```bash
|
||||
python3 doc_scraper.py --config configs/godot.json
|
||||
python3 doc_scraper.py --config configs/react.json
|
||||
python3 doc_scraper.py --config configs/vue.json
|
||||
python3 doc_scraper.py --config configs/django.json
|
||||
python3 doc_scraper.py --config configs/fastapi.json
|
||||
python3 cli/doc_scraper.py --config configs/godot.json
|
||||
python3 cli/doc_scraper.py --config configs/react.json
|
||||
python3 cli/doc_scraper.py --config configs/vue.json
|
||||
python3 cli/doc_scraper.py --config configs/django.json
|
||||
python3 cli/doc_scraper.py --config configs/fastapi.json
|
||||
```
|
||||
|
||||
**2. Interactive Mode**
|
||||
```bash
|
||||
python3 doc_scraper.py --interactive
|
||||
python3 cli/doc_scraper.py --interactive
|
||||
# Wizard walks you through:
|
||||
# - Skill name
|
||||
# - Base URL
|
||||
@@ -92,7 +92,7 @@ python3 doc_scraper.py --interactive
|
||||
|
||||
**3. Quick Mode (Minimal)**
|
||||
```bash
|
||||
python3 doc_scraper.py \
|
||||
python3 cli/doc_scraper.py \
|
||||
--name react \
|
||||
--url https://react.dev/ \
|
||||
--description "React framework for building UIs"
|
||||
@@ -100,7 +100,7 @@ python3 doc_scraper.py \
|
||||
|
||||
**4. Dry-Run (Preview)**
|
||||
```bash
|
||||
python3 doc_scraper.py --config configs/react.json --dry-run
|
||||
python3 cli/doc_scraper.py --config configs/react.json --dry-run
|
||||
# Shows what will be scraped without downloading data
|
||||
# No directories created
|
||||
# Fast validation
|
||||
@@ -108,7 +108,7 @@ python3 doc_scraper.py --config configs/react.json --dry-run
|
||||
|
||||
**5. Skip Scraping (Use Cached Data)**
|
||||
```bash
|
||||
python3 doc_scraper.py --config configs/godot.json --skip-scrape
|
||||
python3 cli/doc_scraper.py --config configs/godot.json --skip-scrape
|
||||
# Uses existing output/godot_data/
|
||||
# Fast rebuild (1-3 minutes)
|
||||
# Useful for testing changes
|
||||
@@ -116,7 +116,7 @@ python3 doc_scraper.py --config configs/godot.json --skip-scrape
|
||||
|
||||
**6. With Local Enhancement**
|
||||
```bash
|
||||
python3 doc_scraper.py --config configs/react.json --enhance-local
|
||||
python3 cli/doc_scraper.py --config configs/react.json --enhance-local
|
||||
# Scrapes + enhances in one command
|
||||
# Opens new terminal for Claude Code
|
||||
# No API key needed
|
||||
@@ -125,10 +125,10 @@ python3 doc_scraper.py --config configs/react.json --enhance-local
|
||||
**7. With API Enhancement**
|
||||
```bash
|
||||
export ANTHROPIC_API_KEY=sk-ant-...
|
||||
python3 doc_scraper.py --config configs/react.json --enhance
|
||||
python3 cli/doc_scraper.py --config configs/react.json --enhance
|
||||
|
||||
# Or with inline API key:
|
||||
python3 doc_scraper.py --config configs/react.json --enhance --api-key sk-ant-...
|
||||
python3 cli/doc_scraper.py --config configs/react.json --enhance --api-key sk-ant-...
|
||||
```
|
||||
|
||||
### Output Structure
|
||||
@@ -182,28 +182,28 @@ options:
|
||||
|
||||
**1. Quick Estimate (100 pages)**
|
||||
```bash
|
||||
python3 estimate_pages.py configs/react.json --max-discovery 100
|
||||
python3 cli/estimate_pages.py configs/react.json --max-discovery 100
|
||||
# Time: ~30-60 seconds
|
||||
# Good for: Quick validation
|
||||
```
|
||||
|
||||
**2. Standard Estimate (1000 pages - default)**
|
||||
```bash
|
||||
python3 estimate_pages.py configs/godot.json
|
||||
python3 cli/estimate_pages.py configs/godot.json
|
||||
# Time: ~1-2 minutes
|
||||
# Good for: Most use cases
|
||||
```
|
||||
|
||||
**3. Deep Estimate (2000 pages)**
|
||||
```bash
|
||||
python3 estimate_pages.py configs/vue.json --max-discovery 2000
|
||||
python3 cli/estimate_pages.py configs/vue.json --max-discovery 2000
|
||||
# Time: ~3-5 minutes
|
||||
# Good for: Large documentation sites
|
||||
```
|
||||
|
||||
**4. Custom Timeout**
|
||||
```bash
|
||||
python3 estimate_pages.py configs/django.json --timeout 60
|
||||
python3 cli/estimate_pages.py configs/django.json --timeout 60
|
||||
# Useful for slow servers
|
||||
```
|
||||
|
||||
@@ -259,8 +259,8 @@ Base URL: https://react.dev/
|
||||
|
||||
```bash
|
||||
# Usage
|
||||
python3 enhance_skill_local.py output/react/
|
||||
python3 enhance_skill_local.py output/godot/
|
||||
python3 cli/enhance_skill_local.py output/react/
|
||||
python3 cli/enhance_skill_local.py output/godot/
|
||||
|
||||
# What it does:
|
||||
# 1. Reads SKILL.md and references/
|
||||
@@ -283,10 +283,10 @@ pip3 install anthropic
|
||||
|
||||
# Usage with environment variable
|
||||
export ANTHROPIC_API_KEY=sk-ant-...
|
||||
python3 enhance_skill.py output/react/
|
||||
python3 cli/enhance_skill.py output/react/
|
||||
|
||||
# Usage with inline API key
|
||||
python3 enhance_skill.py output/godot/ --api-key sk-ant-...
|
||||
python3 cli/enhance_skill.py output/godot/ --api-key sk-ant-...
|
||||
|
||||
# What it does:
|
||||
# 1. Reads SKILL.md and references/
|
||||
@@ -307,8 +307,8 @@ python3 enhance_skill.py output/godot/ --api-key sk-ant-...
|
||||
|
||||
```bash
|
||||
# Usage
|
||||
python3 package_skill.py output/react/
|
||||
python3 package_skill.py output/godot/
|
||||
python3 cli/package_skill.py output/react/
|
||||
python3 cli/package_skill.py output/godot/
|
||||
|
||||
# What it does:
|
||||
# 1. Validates SKILL.md exists
|
||||
@@ -330,28 +330,28 @@ python3 package_skill.py output/godot/
|
||||
|
||||
```bash
|
||||
# Run all tests (default)
|
||||
python3 run_tests.py
|
||||
python3 cli/run_tests.py
|
||||
# 71 tests, ~1 second
|
||||
|
||||
# Verbose output
|
||||
python3 run_tests.py -v
|
||||
python3 run_tests.py --verbose
|
||||
python3 cli/run_tests.py -v
|
||||
python3 cli/run_tests.py --verbose
|
||||
|
||||
# Quiet output
|
||||
python3 run_tests.py -q
|
||||
python3 run_tests.py --quiet
|
||||
python3 cli/run_tests.py -q
|
||||
python3 cli/run_tests.py --quiet
|
||||
|
||||
# Stop on first failure
|
||||
python3 run_tests.py -f
|
||||
python3 run_tests.py --failfast
|
||||
python3 cli/run_tests.py -f
|
||||
python3 cli/run_tests.py --failfast
|
||||
|
||||
# Run specific test suite
|
||||
python3 run_tests.py --suite config
|
||||
python3 run_tests.py --suite features
|
||||
python3 run_tests.py --suite integration
|
||||
python3 cli/run_tests.py --suite config
|
||||
python3 cli/run_tests.py --suite features
|
||||
python3 cli/run_tests.py --suite integration
|
||||
|
||||
# List all tests
|
||||
python3 run_tests.py --list
|
||||
python3 cli/run_tests.py --list
|
||||
```
|
||||
|
||||
### Individual Tests
|
||||
@@ -434,13 +434,13 @@ python3 -m json.tool configs/godot.json
|
||||
|
||||
```bash
|
||||
# 1. Estimate (optional, 1-2 min)
|
||||
python3 estimate_pages.py configs/react.json
|
||||
python3 cli/estimate_pages.py configs/react.json
|
||||
|
||||
# 2. Scrape with local enhancement (25 min)
|
||||
python3 doc_scraper.py --config configs/react.json --enhance-local
|
||||
python3 cli/doc_scraper.py --config configs/react.json --enhance-local
|
||||
|
||||
# 3. Package (instant)
|
||||
python3 package_skill.py output/react/
|
||||
python3 cli/package_skill.py output/react/
|
||||
|
||||
# Result: output/react.zip
|
||||
# Upload to Claude!
|
||||
@@ -461,26 +461,26 @@ cat > configs/my-docs.json << 'EOF'
|
||||
EOF
|
||||
|
||||
# 2. Estimate
|
||||
python3 estimate_pages.py configs/my-docs.json
|
||||
python3 cli/estimate_pages.py configs/my-docs.json
|
||||
|
||||
# 3. Dry-run test
|
||||
python3 doc_scraper.py --config configs/my-docs.json --dry-run
|
||||
python3 cli/doc_scraper.py --config configs/my-docs.json --dry-run
|
||||
|
||||
# 4. Full scrape
|
||||
python3 doc_scraper.py --config configs/my-docs.json
|
||||
python3 cli/doc_scraper.py --config configs/my-docs.json
|
||||
|
||||
# 5. Enhance
|
||||
python3 enhance_skill_local.py output/my-docs/
|
||||
python3 cli/enhance_skill_local.py output/my-docs/
|
||||
|
||||
# 6. Package
|
||||
python3 package_skill.py output/my-docs/
|
||||
python3 cli/package_skill.py output/my-docs/
|
||||
```
|
||||
|
||||
### Workflow 3: Interactive Mode
|
||||
|
||||
```bash
|
||||
# 1. Start interactive wizard
|
||||
python3 doc_scraper.py --interactive
|
||||
python3 cli/doc_scraper.py --interactive
|
||||
|
||||
# 2. Answer prompts:
|
||||
# - Name: my-framework
|
||||
@@ -491,16 +491,16 @@ python3 doc_scraper.py --interactive
|
||||
# - Max pages: 100
|
||||
|
||||
# 3. Enhance
|
||||
python3 enhance_skill_local.py output/my-framework/
|
||||
python3 cli/enhance_skill_local.py output/my-framework/
|
||||
|
||||
# 4. Package
|
||||
python3 package_skill.py output/my-framework/
|
||||
python3 cli/package_skill.py output/my-framework/
|
||||
```
|
||||
|
||||
### Workflow 4: Quick Mode
|
||||
|
||||
```bash
|
||||
python3 doc_scraper.py \
|
||||
python3 cli/doc_scraper.py \
|
||||
--name vue \
|
||||
--url https://vuejs.org/ \
|
||||
--description "Vue.js framework" \
|
||||
@@ -512,13 +512,13 @@ python3 doc_scraper.py \
|
||||
```bash
|
||||
# Already scraped once?
|
||||
# Skip re-scraping, just rebuild
|
||||
python3 doc_scraper.py --config configs/godot.json --skip-scrape
|
||||
python3 cli/doc_scraper.py --config configs/godot.json --skip-scrape
|
||||
|
||||
# Try new enhancement
|
||||
python3 enhance_skill_local.py output/godot/
|
||||
python3 cli/enhance_skill_local.py output/godot/
|
||||
|
||||
# Re-package
|
||||
python3 package_skill.py output/godot/
|
||||
python3 cli/package_skill.py output/godot/
|
||||
```
|
||||
|
||||
### Workflow 6: Testing New Config
|
||||
@@ -535,13 +535,13 @@ cat > configs/test.json << 'EOF'
|
||||
EOF
|
||||
|
||||
# 2. Estimate
|
||||
python3 estimate_pages.py configs/test.json --max-discovery 50
|
||||
python3 cli/estimate_pages.py configs/test.json --max-discovery 50
|
||||
|
||||
# 3. Dry-run
|
||||
python3 doc_scraper.py --config configs/test.json --dry-run
|
||||
python3 cli/doc_scraper.py --config configs/test.json --dry-run
|
||||
|
||||
# 4. Small scrape
|
||||
python3 doc_scraper.py --config configs/test.json
|
||||
python3 cli/doc_scraper.py --config configs/test.json
|
||||
|
||||
# 5. Validate output
|
||||
ls output/test-site/
|
||||
@@ -572,7 +572,7 @@ ls output/test-site/references/
|
||||
|
||||
```bash
|
||||
# Estimate first
|
||||
python3 estimate_pages.py configs/my-config.json
|
||||
python3 cli/estimate_pages.py configs/my-config.json
|
||||
|
||||
# Set max_pages based on estimate
|
||||
# Add buffer: estimated + 50
|
||||
@@ -622,7 +622,7 @@ python3 -m unittest tests.test_config_validation.TestConfigValidation.test_name
|
||||
echo $ANTHROPIC_API_KEY
|
||||
|
||||
# Or use inline:
|
||||
python3 enhance_skill.py output/react/ --api-key sk-ant-...
|
||||
python3 cli/enhance_skill.py output/react/ --api-key sk-ant-...
|
||||
```
|
||||
|
||||
### Issue: "Package fails"
|
||||
@@ -632,7 +632,7 @@ python3 enhance_skill.py output/react/ --api-key sk-ant-...
|
||||
ls output/my-skill/SKILL.md
|
||||
|
||||
# If missing, build first:
|
||||
python3 doc_scraper.py --config configs/my-skill.json --skip-scrape
|
||||
python3 cli/doc_scraper.py --config configs/my-skill.json --skip-scrape
|
||||
```
|
||||
|
||||
### Issue: "Can't find output"
|
||||
@@ -773,9 +773,9 @@ Skill_Seekers/
|
||||
|
||||
```bash
|
||||
# Tool-specific help
|
||||
python3 doc_scraper.py --help
|
||||
python3 estimate_pages.py --help
|
||||
python3 run_tests.py --help
|
||||
python3 cli/doc_scraper.py --help
|
||||
python3 cli/estimate_pages.py --help
|
||||
python3 cli/run_tests.py --help
|
||||
|
||||
# Documentation
|
||||
cat CLAUDE.md # Quick reference for Claude Code
|
||||
@@ -793,18 +793,18 @@ cat README.md # Project overview
|
||||
|
||||
**Essential Commands:**
|
||||
```bash
|
||||
python3 estimate_pages.py configs/react.json # Estimate
|
||||
python3 doc_scraper.py --config configs/react.json # Scrape
|
||||
python3 enhance_skill_local.py output/react/ # Enhance
|
||||
python3 package_skill.py output/react/ # Package
|
||||
python3 run_tests.py # Test
|
||||
python3 cli/estimate_pages.py configs/react.json # Estimate
|
||||
python3 cli/doc_scraper.py --config configs/react.json # Scrape
|
||||
python3 cli/enhance_skill_local.py output/react/ # Enhance
|
||||
python3 cli/package_skill.py output/react/ # Package
|
||||
python3 cli/run_tests.py # Test
|
||||
```
|
||||
|
||||
**Quick Start:**
|
||||
```bash
|
||||
pip3 install requests beautifulsoup4
|
||||
python3 doc_scraper.py --config configs/react.json --enhance-local
|
||||
python3 package_skill.py output/react/
|
||||
python3 cli/doc_scraper.py --config configs/react.json --enhance-local
|
||||
python3 cli/package_skill.py output/react/
|
||||
# Upload output/react.zip to Claude!
|
||||
```
|
||||
|
||||
|
||||
Reference in New Issue
Block a user