[![MseeP.ai Security Assessment Badge](https://mseep.net/pr/yusufkaraaslan-skill-seekers-badge.png)](https://mseep.ai/app/yusufkaraaslan-skill-seekers)

# Skill Seeker

English | [简体中文](https://github.com/yusufkaraaslan/Skill_Seekers/blob/main/README.zh-CN.md)

[![Version](https://img.shields.io/badge/version-3.0.0-blue.svg)](https://github.com/yusufkaraaslan/Skill_Seekers/releases)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/)
[![MCP Integration](https://img.shields.io/badge/MCP-Integrated-blue.svg)](https://modelcontextprotocol.io)
[![Tested](https://img.shields.io/badge/Tests-1852%20Passing-brightgreen.svg)](tests/)
[![Project Board](https://img.shields.io/badge/Project-Board-purple.svg)](https://github.com/users/yusufkaraaslan/projects/2)
[![PyPI version](https://badge.fury.io/py/skill-seekers.svg)](https://pypi.org/project/skill-seekers/)
[![PyPI - Downloads](https://img.shields.io/pypi/dm/skill-seekers.svg)](https://pypi.org/project/skill-seekers/)
[![PyPI - Python Version](https://img.shields.io/pypi/pyversions/skill-seekers.svg)](https://pypi.org/project/skill-seekers/)
[![Website](https://img.shields.io/badge/Website-skillseekersweb.com-blue.svg)](https://skillseekersweb.com/)
[![Twitter Follow](https://img.shields.io/twitter/follow/_yUSyUS_?style=social)](https://x.com/_yUSyUS_)
[![GitHub Repo stars](https://img.shields.io/github/stars/yusufkaraaslan/Skill_Seekers?style=social)](https://github.com/yusufkaraaslan/Skill_Seekers)

**🧠 The data layer for AI systems.** Skill Seekers turns any documentation, GitHub repo, or PDF into structured knowledge assets—ready to power AI Skills (Claude, Gemini, OpenAI), RAG pipelines (LangChain, LlamaIndex, Pinecone), and AI coding assistants (Cursor, Windsurf, Cline) in minutes, not hours.

> 🌐 **[Visit SkillSeekersWeb.com](https://skillseekersweb.com/)** - Browse 24+ preset configs, share your configs, and access complete documentation!

> 📋 **[View Development Roadmap & Tasks](https://github.com/users/yusufkaraaslan/projects/2)** - 134 tasks across 10 categories, pick any to contribute!

## 🧠 The Data Layer for AI Systems

**Skill Seekers is the universal preprocessing layer** that sits between raw documentation and every AI system that consumes it. Whether you are building Claude skills, a LangChain RAG pipeline, or a Cursor `.cursorrules` file — the data preparation is identical. You do it once, and export to all targets.

```bash
# One command → structured knowledge asset
skill-seekers create https://docs.react.dev/
# or: skill-seekers create facebook/react
# or: skill-seekers create ./my-project

# Export to any AI system
skill-seekers package output/react --target claude      # → Claude AI Skill (ZIP)
skill-seekers package output/react --target langchain   # → LangChain Documents
skill-seekers package output/react --target llama-index # → LlamaIndex TextNodes
skill-seekers package output/react --target cursor      # → .cursorrules
```

### What gets built

| Output | Target | What it powers |
|--------|--------|---------------|
| **Claude Skill** (ZIP + YAML) | `--target claude` | Claude Code, Claude API |
| **Gemini Skill** (tar.gz) | `--target gemini` | Google Gemini |
| **OpenAI / Custom GPT** (ZIP) | `--target openai` | GPT-4o, custom assistants |
| **LangChain Documents** | `--target langchain` | QA chains, agents, retrievers |
| **LlamaIndex TextNodes** | `--target llama-index` | Query engines, chat engines |
| **Haystack Documents** | `--target haystack` | Enterprise RAG pipelines |
| **Pinecone-ready** (Markdown) | `--target markdown` | Vector upsert |
| **ChromaDB / FAISS / Qdrant** | `--format chroma/faiss/qdrant` | Local vector DBs |
| **Cursor** `.cursorrules` | `--target claude` → copy | Cursor IDE AI context |
| **Windsurf / Cline / Continue** | `--target claude` → copy | VS Code, IntelliJ, Vim |

### Why it matters

- ⚡ **99% faster** — Days of manual data prep → 15–45 minutes
- 🎯 **AI Skill quality** — 500+ line SKILL.md files with examples, patterns, and guides
- 📊 **RAG-ready chunks** — Smart chunking preserves code blocks and maintains context
- 🔄 **Multi-source** — Combine docs + GitHub + PDFs into one knowledge asset
- 🌐 **One prep, every target** — Export the same asset to 16 platforms without re-scraping
- ✅ **Battle-tested** — 1,880+ tests, 24+ framework presets, production-ready

## Quick Start

```bash
pip install skill-seekers

# Build an AI skill from any source
skill-seekers create https://docs.django.com/    # web docs
skill-seekers create django/django               # GitHub repo
skill-seekers create ./my-codebase               # local project
skill-seekers create manual.pdf                  # PDF

# Export for your use case
skill-seekers package output/django --target claude     # Claude AI Skill
skill-seekers package output/django --target langchain  # LangChain RAG
skill-seekers package output/django --target cursor     # Cursor IDE context
```

**Complete examples:**
- [Claude AI Skill](examples/claude-skill/) - Skills for Claude Code
- [LangChain RAG Pipeline](examples/langchain-rag-pipeline/) - QA chain with Chroma
- [Cursor IDE Context](examples/cursor-react-skill/) - Framework-aware AI coding

## What is Skill Seekers?

Skill Seekers is the **data layer for AI systems**. It transforms documentation websites, GitHub repositories, and PDF files into structured knowledge assets for every AI target:

| Use Case | What you get | Examples |
|----------|-------------|---------|
| **AI Skills** | Comprehensive SKILL.md + references | Claude Code, Gemini, GPT |
| **RAG Pipelines** | Chunked documents with rich metadata | LangChain, LlamaIndex, Haystack |
| **Vector Databases** | Pre-formatted data ready for upsert | Pinecone, Chroma, Weaviate, FAISS |
| **AI Coding Assistants** | Context files your IDE AI reads automatically | Cursor, Windsurf, Cline, Continue.dev |

Instead of spending days on manual preprocessing, Skill Seekers:

1. **Ingests** — docs, GitHub repos, local codebases, PDFs
2. **Analyzes** — deep AST parsing, pattern detection, API extraction
3. **Structures** — categorized reference files with metadata
4. **Enhances** — AI-powered SKILL.md generation (Claude, Gemini, or local)
5. **Exports** — 16 platform-specific formats from one asset

## Why Use This?

### For AI Skill Builders (Claude, Gemini, OpenAI)

- 🎯 **Production-grade Skills** — 500+ line SKILL.md files with code examples, patterns, and guides
- 🔄 **Enhancement Workflows** — Apply `security-focus`, `architecture-comprehensive`, or custom YAML presets
- 🎮 **Any Domain** — Game engines (Godot, Unity), frameworks (React, Django), internal tools
- 🔧 **Teams** — Combine internal docs + code into a single source of truth
- 📚 **Quality** — AI-enhanced with examples, quick reference, and navigation guidance

### For RAG Builders & AI Engineers

- 🤖 **RAG-ready data** — Pre-chunked LangChain `Documents`, LlamaIndex `TextNodes`, Haystack `Documents`
- 🚀 **99% faster** — Days of preprocessing → 15–45 minutes
- 📊 **Smart metadata** — Categories, sources, types → better retrieval accuracy
- 🔄 **Multi-source** — Combine docs + GitHub + PDFs in one pipeline
- 🌐 **Platform-agnostic** — Export to any vector DB or framework without re-scraping

### For AI Coding Assistant Users

- 💻 **Cursor / Windsurf / Cline** — Generate `.cursorrules` / `.windsurfrules` / `.clinerules` automatically
- 🎯 **Persistent context** — AI "knows" your frameworks without repeated prompting
- 📚 **Always current** — Update context in minutes when docs change

## Key Features

### 🌐 Documentation Scraping
- ✅ **llms.txt Support** - Automatically detects and uses LLM-ready documentation files (10x faster)
- ✅ **Universal Scraper** - Works with ANY documentation website
- ✅ **Smart Categorization** - Automatically organizes content by topic
- ✅ **Code Language Detection** - Recognizes Python, JavaScript, C++, GDScript, etc.
- ✅ **8 Ready-to-Use Presets** - Godot, React, Vue, Django, FastAPI, and more

### 📄 PDF Support (**v1.2.0**)
- ✅ **Basic PDF Extraction** - Extract text, code, and images from PDF files
- ✅ **OCR for Scanned PDFs** - Extract text from scanned documents
- ✅ **Password-Protected PDFs** - Handle encrypted PDFs
- ✅ **Table Extraction** - Extract complex tables from PDFs
- ✅ **Parallel Processing** - 3x faster for large PDFs
- ✅ **Intelligent Caching** - 50% faster on re-runs

### 🐙 GitHub Repository Scraping (**v2.0.0**)
- ✅ **Deep Code Analysis** - AST parsing for Python, JavaScript, TypeScript, Java, C++, Go
- ✅ **API Extraction** - Functions, classes, methods with parameters and types
- ✅ **Repository Metadata** - README, file tree, language breakdown, stars/forks
- ✅ **GitHub Issues & PRs** - Fetch open/closed issues with labels and milestones
- ✅ **CHANGELOG & Releases** - Automatically extract version history
- ✅ **Conflict Detection** - Compare documented APIs vs actual code implementation
- ✅ **MCP Integration** - Natural language: "Scrape GitHub repo facebook/react"

### 🔄 Unified Multi-Source Scraping (**NEW - v2.0.0**)
- ✅ **Combine Multiple Sources** - Mix documentation + GitHub + PDF in one skill
- ✅ **Conflict Detection** - Automatically finds discrepancies between docs and code
- ✅ **Intelligent Merging** - Rule-based or AI-powered conflict resolution
- ✅ **Transparent Reporting** - Side-by-side comparison with ⚠️ warnings
- ✅ **Documentation Gap Analysis** - Identifies outdated docs and undocumented features
- ✅ **Single Source of Truth** - One skill showing both intent (docs) and reality (code)
- ✅ **Backward Compatible** - Legacy single-source configs still work

### 🤖 Multi-LLM Platform Support (**NEW - v2.5.0**)
- ✅ **4 LLM Platforms** - Claude AI, Google Gemini, OpenAI ChatGPT, Generic Markdown
- ✅ **Universal Scraping** - Same documentation works for all platforms
- ✅ **Platform-Specific Packaging** - Optimized formats for each LLM
- ✅ **One-Command Export** - `--target` flag selects platform
- ✅ **Optional Dependencies** - Install only what you need
- ✅ **100% Backward Compatible** - Existing Claude workflows unchanged

| Platform | Format | Upload | Enhancement | API Key | Custom Endpoint |
|----------|--------|--------|-------------|---------|-----------------|
| **Claude AI** | ZIP + YAML | ✅ Auto | ✅ Yes | ANTHROPIC_API_KEY | ANTHROPIC_BASE_URL |
| **Google Gemini** | tar.gz | ✅ Auto | ✅ Yes | GOOGLE_API_KEY | - |
| **OpenAI ChatGPT** | ZIP + Vector Store | ✅ Auto | ✅ Yes | OPENAI_API_KEY | - |
| **Generic Markdown** | ZIP | ❌ Manual | ❌ No | - | - |

```bash
# Claude (default - no changes needed!)
skill-seekers package output/react/
skill-seekers upload react.zip

# Google Gemini
pip install skill-seekers[gemini]
skill-seekers package output/react/ --target gemini
skill-seekers upload react-gemini.tar.gz --target gemini

# OpenAI ChatGPT
pip install skill-seekers[openai]
skill-seekers package output/react/ --target openai
skill-seekers upload react-openai.zip --target openai

# Generic Markdown (universal export)
skill-seekers package output/react/ --target markdown
# Use the markdown files directly in any LLM
```

<details>
<summary>🔧 <strong>Environment Variables for Claude-Compatible APIs (e.g., GLM-4.7)</strong></summary>

Skill Seekers supports any Claude-compatible API endpoint:

```bash
# Option 1: Official Anthropic API (default)
export ANTHROPIC_API_KEY=sk-ant-...

# Option 2: GLM-4.7 Claude-compatible API
export ANTHROPIC_API_KEY=your-glm-47-api-key
export ANTHROPIC_BASE_URL=https://glm-4-7-endpoint.com/v1

# All AI enhancement features will use the configured endpoint
skill-seekers enhance output/react/
skill-seekers analyze --directory . --enhance
```

**Note**: Setting `ANTHROPIC_BASE_URL` allows you to use any Claude-compatible API endpoint, such as GLM-4.7 (智谱 AI) or other compatible services.

</details>

**Installation:**
```bash
# Install with Gemini support
pip install skill-seekers[gemini]

# Install with OpenAI support
pip install skill-seekers[openai]

# Install with all LLM platforms
pip install skill-seekers[all-llms]
```

### 🔗 RAG Framework Integrations (**NEW - v2.9.0**)

- ✅ **LangChain Documents** - Direct export to `Document` format with `page_content` + metadata
  - Perfect for: QA chains, retrievers, vector stores, agents
  - Example: [LangChain RAG Pipeline](examples/langchain-rag-pipeline/)
  - Guide: [LangChain Integration](docs/integrations/LANGCHAIN.md)

- ✅ **LlamaIndex TextNodes** - Export to `TextNode` format with unique IDs + embeddings
  - Perfect for: Query engines, chat engines, storage context
  - Example: [LlamaIndex Query Engine](examples/llama-index-query-engine/)
  - Guide: [LlamaIndex Integration](docs/integrations/LLAMA_INDEX.md)

- ✅ **Pinecone-Ready Format** - Optimized for vector database upsert
  - Perfect for: Production vector search, semantic search, hybrid search
  - Example: [Pinecone Upsert](examples/pinecone-upsert/)
  - Guide: [Pinecone Integration](docs/integrations/PINECONE.md)

- ✅ **AI Coding Assistants** - Expert context for 4+ IDE AI tools
  - **Cursor IDE** - `.cursorrules` format for VS Code fork | [Guide](docs/integrations/CURSOR.md)
  - **Windsurf** - `.windsurfrules` format for Codeium IDE | [Guide](docs/integrations/WINDSURF.md)
  - **Cline** - `.clinerules` + MCP for VS Code extension | [Guide](docs/integrations/CLINE.md)
  - **Continue.dev** - HTTP context providers for any IDE | [Guide](docs/integrations/CONTINUE_DEV.md)
  - Perfect for: Framework-specific code generation, consistent team patterns
  - Hub: [All AI Coding Integrations](docs/integrations/INTEGRATIONS.md)

**Quick Export:**
```bash
# LangChain Documents (JSON)
skill-seekers package output/django --target langchain
# → output/django-langchain.json

# LlamaIndex TextNodes (JSON)
skill-seekers package output/django --target llama-index
# → output/django-llama-index.json

# Markdown (Universal)
skill-seekers package output/django --target markdown
# → output/django-markdown/SKILL.md + references/
```

**Complete RAG Pipeline Guide:** [RAG Pipelines Documentation](docs/integrations/RAG_PIPELINES.md)

---

### 🧠 AI Coding Assistant Integrations (**NEW - v2.10.0**)

Transform any framework documentation into expert coding context for 4+ AI assistants:

- ✅ **Cursor IDE** - Generate `.cursorrules` for AI-powered code suggestions
  - Perfect for: Framework-specific code generation, consistent patterns
  - Works with: Cursor IDE (VS Code fork)
  - Guide: [Cursor Integration](docs/integrations/CURSOR.md)
  - Example: [Cursor React Skill](examples/cursor-react-skill/)

- ✅ **Windsurf** - Customize Windsurf's AI assistant context with `.windsurfrules`
  - Perfect for: IDE-native AI assistance, flow-based coding
  - Works with: Windsurf IDE by Codeium
  - Guide: [Windsurf Integration](docs/integrations/WINDSURF.md)
  - Example: [Windsurf FastAPI Context](examples/windsurf-fastapi-context/)

- ✅ **Cline (VS Code)** - System prompts + MCP for VS Code agent
  - Perfect for: Agentic code generation in VS Code, Cursor Composer equivalent
  - Works with: Cline extension for VS Code
  - Guide: [Cline Integration](docs/integrations/CLINE.md)
  - Example: [Cline Django Assistant](examples/cline-django-assistant/)

- ✅ **Continue.dev** - Context servers for IDE-agnostic AI
  - Perfect for: Multi-IDE environments (VS Code, JetBrains, Vim), custom LLM providers
  - Works with: Any IDE with Continue.dev plugin
  - Guide: [Continue Integration](docs/integrations/CONTINUE_DEV.md)
  - Example: [Continue Universal Context](examples/continue-dev-universal/)

**Quick Export for AI Coding Tools:**
```bash
# For any AI coding assistant (Cursor, Windsurf, Cline, Continue.dev)
skill-seekers scrape --config configs/django.json
skill-seekers package output/django --target markdown  # or --target claude

# Copy to your project (example for Cursor)
cp output/django-markdown/SKILL.md my-project/.cursorrules

# Or for Windsurf
cp output/django-markdown/SKILL.md my-project/.windsurf/rules/django.md

# Or for Cline
cp output/django-markdown/SKILL.md my-project/.clinerules

# Or for Continue.dev (HTTP server)
python examples/continue-dev-universal/context_server.py
# Configure in ~/.continue/config.json
```

**Multi-IDE Team Consistency:**
```bash
# Use Continue.dev for teams with mixed IDEs
skill-seekers scrape --config configs/react.json
python context_server.py --host 0.0.0.0 --port 8765

# Team members configure Continue.dev (same config works in ALL IDEs):
# VS Code, IntelliJ, PyCharm, WebStorm, Vim...
# Result: Identical AI suggestions across all environments!
```

**Integration Hub:** [All AI System Integrations](docs/integrations/INTEGRATIONS.md)

---

### 🌊 Three-Stream GitHub Architecture (**NEW - v2.6.0**)
- ✅ **Triple-Stream Analysis** - Split GitHub repos into Code, Docs, and Insights streams
- ✅ **Unified Codebase Analyzer** - Works with GitHub URLs AND local paths
- ✅ **C3.x as Analysis Depth** - Choose 'basic' (1-2 min) or 'c3x' (20-60 min) analysis
- ✅ **Enhanced Router Generation** - GitHub metadata, README quick start, common issues
- ✅ **Issue Integration** - Top problems and solutions from GitHub issues
- ✅ **Smart Routing Keywords** - GitHub labels weighted 2x for better topic detection
- ✅ **81 Tests Passing** - Comprehensive E2E validation (0.44 seconds)

**Three Streams Explained:**
- **Stream 1: Code** - Deep C3.x analysis (patterns, examples, guides, configs, architecture)
- **Stream 2: Docs** - Repository documentation (README, CONTRIBUTING, docs/*.md)
- **Stream 3: Insights** - Community knowledge (issues, labels, stars, forks)

```python
from skill_seekers.cli.unified_codebase_analyzer import UnifiedCodebaseAnalyzer

# Analyze GitHub repo with all three streams
analyzer = UnifiedCodebaseAnalyzer()
result = analyzer.analyze(
    source="https://github.com/facebook/react",
    depth="c3x",  # or "basic" for fast analysis
    fetch_github_metadata=True
)

# Access code stream (C3.x analysis)
print(f"Design patterns: {len(result.code_analysis['c3_1_patterns'])}")
print(f"Test examples: {result.code_analysis['c3_2_examples_count']}")

# Access docs stream (repository docs)
print(f"README: {result.github_docs['readme'][:100]}")

# Access insights stream (GitHub metadata)
print(f"Stars: {result.github_insights['metadata']['stars']}")
print(f"Common issues: {len(result.github_insights['common_problems'])}")
```

**See complete documentation**: [Three-Stream Implementation Summary](docs/IMPLEMENTATION_SUMMARY_THREE_STREAM.md)

### 🔐 Smart Rate Limit Management & Configuration (**NEW - v2.7.0**)
- ✅ **Multi-Token Configuration System** - Manage multiple GitHub accounts (personal, work, OSS)
  - Secure config storage at `~/.config/skill-seekers/config.json` (600 permissions)
  - Per-profile rate limit strategies: `prompt`, `wait`, `switch`, `fail`
  - Configurable timeout per profile (default: 30 min, prevents indefinite waits)
  - Smart fallback chain: CLI arg → Env var → Config file → Prompt
  - API key management for Claude, Gemini, OpenAI
- ✅ **Interactive Configuration Wizard** - Beautiful terminal UI for easy setup
  - Browser integration for token creation (auto-opens GitHub, etc.)
  - Token validation and connection testing
  - Visual status display with color coding
- ✅ **Intelligent Rate Limit Handler** - No more indefinite waits!
  - Upfront warning about rate limits (60/hour vs 5000/hour)
  - Real-time detection from GitHub API responses
  - Live countdown timers with progress
  - Automatic profile switching when rate limited
  - Four strategies: prompt (ask), wait (countdown), switch (try another), fail (abort)
- ✅ **Resume Capability** - Continue interrupted jobs
  - Auto-save progress at configurable intervals (default: 60 sec)
  - List all resumable jobs with progress details
  - Auto-cleanup of old jobs (default: 7 days)
- ✅ **CI/CD Support** - Non-interactive mode for automation
  - `--non-interactive` flag fails fast without prompts
  - `--profile` flag to select specific GitHub account
  - Clear error messages for pipeline logs
  - Exit codes for automation integration

**Quick Setup:**
```bash
# One-time configuration (5 minutes)
skill-seekers config --github

# Add multiple GitHub profiles
skill-seekers config
# → Select "1. GitHub Token Setup"
# → Add profiles for personal, work, OSS accounts

# Use specific profile for private repos
skill-seekers github --repo mycompany/private-repo --profile work

# CI/CD mode (fail fast, no prompts)
skill-seekers github --repo owner/repo --non-interactive

# View current configuration
skill-seekers config --show

# Test connections
skill-seekers config --test

# Resume interrupted job
skill-seekers resume --list
skill-seekers resume github_react_20260117_143022
```

**Rate Limit Strategies Explained:**
- **prompt** (default) - Ask what to do when rate limited (wait, switch, setup token, cancel)
- **wait** - Automatically wait with countdown timer (respects timeout)
- **switch** - Automatically try next available profile (for multi-account setups)
- **fail** - Fail immediately with clear error (perfect for CI/CD)

**See complete documentation**: [Configuration Guide](docs/guides/CONFIGURATION.md) (coming soon)

### 🎯 Bootstrap Skill - Self-Hosting (**NEW - v2.7.0**)

Generate skill-seekers as a Claude Code skill to use within Claude:

```bash
# Generate the skill
./scripts/bootstrap_skill.sh

# Install to Claude Code
cp -r output/skill-seekers ~/.claude/skills/

# Verify
ls ~/.claude/skills/skill-seekers/SKILL.md
```

**What you get:**
- ✅ **Complete skill documentation** - All CLI commands and usage patterns
- ✅ **CLI command reference** - Every tool and its options documented
- ✅ **Quick start examples** - Common workflows and best practices
- ✅ **Auto-generated API docs** - Code analysis, patterns, and examples
- ✅ **Robust validation** - YAML frontmatter and required fields checked
- ✅ **One-command bootstrap** - Combines manual header with auto-generated analysis

**How it works:**
1. Runs codebase analysis on skill-seekers itself (dogfooding!)
2. Combines handcrafted header (prerequisites, commands) with auto-generated content
3. Validates SKILL.md structure (frontmatter, required fields)
4. Outputs ready-to-use skill directory

**Result:** Use skill-seekers to create skills, from within Claude Code!

### 🔐 Private Config Repositories (**NEW - v2.2.0**)
- ✅ **Git-Based Config Sources** - Fetch configs from private/team git repositories
- ✅ **Multi-Source Management** - Register unlimited GitHub, GitLab, Bitbucket repos
- ✅ **Team Collaboration** - Share custom configs across 3-5 person teams
- ✅ **Enterprise Support** - Scale to 500+ developers with priority-based resolution
- ✅ **Secure Authentication** - Environment variable tokens (GITHUB_TOKEN, GITLAB_TOKEN)
- ✅ **Intelligent Caching** - Clone once, pull updates automatically
- ✅ **Offline Mode** - Work with cached configs when offline
- ✅ **Backward Compatible** - Existing API-based configs still work

### 🤖 Codebase Analysis & AI Enhancement (**C3.x - NEW!**)

**C3.4: Configuration Pattern Extraction with AI Enhancement**
- ✅ **9 Config Formats** - JSON, YAML, TOML, ENV, INI, Python, JavaScript, Dockerfile, Docker Compose
- ✅ **7 Pattern Types** - Database, API, logging, cache, email, auth, server configurations
- ✅ **AI Enhancement (NEW!)** - Optional dual-mode AI analysis (API + LOCAL, like C3.3)
  - Explains what each config does
  - Suggests best practices and improvements
  - **Security analysis** - Finds hardcoded secrets, exposed credentials
  - Migration suggestions - Consolidation opportunities
  - Context-aware documentation
- ✅ **Auto-Documentation** - Generates JSON + Markdown documentation of all configs
- ✅ **Type Inference** - Automatically detects setting types and environment variables
- ✅ **MCP Integration** - `extract_config_patterns` tool with enhancement support

**C3.3: AI-Enhanced How-To Guides**
- ✅ **Comprehensive AI Enhancement** - Transforms basic guides (⭐⭐) into professional tutorials (⭐⭐⭐⭐⭐)
- ✅ **5 Automatic Improvements** - Step descriptions, troubleshooting, prerequisites, next steps, use cases
- ✅ **Dual-Mode Support** - API mode (Claude API) or LOCAL mode (Claude Code CLI)
- ✅ **No API Costs with LOCAL Mode** - FREE enhancement using your Claude Code Max plan
- ✅ **Quality Transformation** - 75-line templates → 500+ line comprehensive guides

**What Gets Enhanced:**
- 🔍 **Step Descriptions** - Natural language explanations (not just syntax!)
- 🔧 **Troubleshooting** - Diagnostic flows + solutions for common errors
- 📋 **Prerequisites** - Why needed + setup instructions
- 🔗 **Next Steps** - Related guides, variations, learning paths
- 💡 **Use Cases** - Real-world scenarios showing when to use guide

**Usage:**
```bash
# Quick analysis (1-2 min, basic features only)
skill-seekers analyze --directory tests/ --quick

# Comprehensive analysis with AI (20-60 min, all features)
skill-seekers analyze --directory tests/ --comprehensive

# With AI enhancement
skill-seekers analyze --directory tests/ --enhance
```

**Full Documentation:** [docs/HOW_TO_GUIDES.md](docs/HOW_TO_GUIDES.md#ai-enhancement-new)

### 🔄 Enhancement Workflow Presets (**NEW!**)

Reusable YAML-defined enhancement pipelines that control how AI transforms your raw documentation into a polished skill.

- ✅ **5 Bundled Presets** — `default`, `minimal`, `security-focus`, `architecture-comprehensive`, `api-documentation`
- ✅ **User-Defined Presets** — add custom workflows to `~/.config/skill-seekers/workflows/`
- ✅ **Multiple Workflows** — chain two or more workflows in one command
- ✅ **Fully Managed CLI** — list, inspect, copy, add, remove, and validate workflows

```bash
# Apply a single workflow
skill-seekers create ./my-project --enhance-workflow security-focus

# Chain multiple workflows (applied in order)
skill-seekers create ./my-project \
  --enhance-workflow security-focus \
  --enhance-workflow minimal

# Manage presets
skill-seekers workflows list                          # List all (bundled + user)
skill-seekers workflows show security-focus           # Print YAML content
skill-seekers workflows copy security-focus           # Copy to user dir for editing
skill-seekers workflows add ./my-workflow.yaml        # Install a custom preset
skill-seekers workflows remove my-workflow            # Remove a user preset
skill-seekers workflows validate security-focus       # Validate preset structure

# Copy multiple at once
skill-seekers workflows copy security-focus minimal api-documentation

# Add multiple files at once
skill-seekers workflows add ./wf-a.yaml ./wf-b.yaml

# Remove multiple at once
skill-seekers workflows remove my-wf-a my-wf-b
```

**YAML preset format:**
```yaml
name: security-focus
description: "Security-focused review: vulnerabilities, auth, data handling"
version: "1.0"
stages:
  - name: vulnerabilities
    type: custom
    prompt: "Review for OWASP top 10 and common security vulnerabilities..."
  - name: auth-review
    type: custom
    prompt: "Examine authentication and authorisation patterns..."
    uses_history: true
```

### ⚡ Performance & Scale
- ✅ **Async Mode** - 2-3x faster scraping with async/await (use `--async` flag)
- ✅ **Large Documentation Support** - Handle 10K-40K+ page docs with intelligent splitting
- ✅ **Router/Hub Skills** - Intelligent routing to specialized sub-skills
- ✅ **Parallel Scraping** - Process multiple skills simultaneously
- ✅ **Checkpoint/Resume** - Never lose progress on long scrapes
- ✅ **Caching System** - Scrape once, rebuild instantly

### ✅ Quality Assurance
- ✅ **Fully Tested** - 1200+ tests with comprehensive coverage

---

## 📦 Now Available on PyPI!

**Skill Seekers is now published on the Python Package Index!** Install with a single command:

```bash
pip install skill-seekers
```

### Installation Options

Choose your installation profile based on which features you need:

```bash
# 1️⃣ CLI Only (Skill Generation)
pip install skill-seekers

# Features:
# • Scrape documentation websites
# • Analyze GitHub repositories
# • Extract from PDFs
# • Package skills for all platforms

# 2️⃣ MCP Integration (Claude Code, Cursor, Windsurf)
pip install skill-seekers[mcp]

# Features:
# • Everything from CLI Only
# • MCP server for Claude Code
# • One-command skill installation
# • HTTP/stdio transport modes

# 3️⃣ Multi-LLM Support (Gemini, OpenAI)
pip install skill-seekers[all-llms]

# Features:
# • Everything from CLI Only
# • Google Gemini support
# • OpenAI ChatGPT support
# • Enhanced AI features

# 4️⃣ Everything
pip install skill-seekers[all]

# Features:
# • All features enabled
# • Maximum flexibility
```

**Need help choosing?** Run the setup wizard:
```bash
skill-seekers-setup
```

The wizard shows all options with detailed feature lists and guides you through configuration.

Get started in seconds. No cloning, no setup - just install and run. See installation options below.

---

## Quick Start

### Option 1: Install from PyPI (Recommended)

```bash
# Install from PyPI (easiest method!)
pip install skill-seekers

# Use the unified CLI
skill-seekers scrape --config configs/react.json
skill-seekers github --repo facebook/react
skill-seekers enhance output/react/
skill-seekers package output/react/
```

**Time:** ~25 minutes | **Quality:** Production-ready | **Cost:** Free

📖 **New to Skill Seekers?** Check out our [Quick Start Guide](QUICKSTART.md) or [Bulletproof Guide](BULLETPROOF_QUICKSTART.md)

### Option 2: Install via uv (Modern Python Tool)

```bash
# Install with uv (fast, modern alternative)
uv tool install skill-seekers

# Or run directly without installing
uv tool run --from skill-seekers skill-seekers scrape --config https://raw.githubusercontent.com/yusufkaraaslan/Skill_Seekers/main/configs/react.json

# Unified CLI - simple commands
skill-seekers scrape --config configs/react.json
skill-seekers github --repo facebook/react
skill-seekers package output/react/
```

**Time:** ~25 minutes | **Quality:** Production-ready | **Cost:** Free

### Option 3: Development Install (From Source)

```bash
# Clone and install in editable mode
git clone https://github.com/yusufkaraaslan/Skill_Seekers.git
cd Skill_Seekers
pip install -e .

# Use the unified CLI
skill-seekers scrape --config configs/react.json
```

### Option 4: Use from Claude Code & 4 Other AI Agents (MCP Integration)

```bash
# One-time setup (5 minutes) - Auto-configures 5 AI agents!
./setup_mcp.sh

# Then in Claude Code, Cursor, Windsurf, VS Code + Cline, or IntelliJ IDEA, just ask:
"Generate a React skill from https://react.dev/"
"Scrape PDF at docs/manual.pdf and create skill"
```

**Time:** Automated | **Quality:** Production-ready | **Cost:** Free

**NEW in v2.4.0:** MCP server now supports 5 AI coding agents with automatic configuration!

### Option 5: Legacy CLI (Backwards Compatible)

```bash
# Install dependencies
pip3 install requests beautifulsoup4

# Run scripts directly (old method)
python3 src/skill_seekers/cli/doc_scraper.py --config configs/react.json

# Upload output/react.zip to Claude - Done!
```

**Time:** ~25 minutes | **Quality:** Production-ready | **Cost:** Free

---

## 🚀 **NEW!** One-Command Install Workflow (v2.1.1)

**The fastest way to go from config to uploaded skill - complete automation:**

```bash
# Install React skill from official configs (auto-uploads to Claude)
skill-seekers install --config react

# Install from local config file
skill-seekers install --config configs/custom.json

# Install without uploading (package only)
skill-seekers install --config django --no-upload

# Unlimited scraping (no page limits)
skill-seekers install --config godot --unlimited

# Preview workflow without executing
skill-seekers install --config react --dry-run
```

**Time:** 20-45 minutes total | **Quality:** Production-ready (9/10) | **Cost:** Free

### What it does automatically:

1. ✅ **Fetches config** from API (if config name provided)
2. ✅ **Scrapes documentation** (respects rate limits, handles pagination)
3. ✅ **AI Enhancement (MANDATORY)** - 30-60 sec, quality boost from 3/10 → 9/10
4. ✅ **Packages skill** to .zip file
5. ✅ **Uploads to Claude** (if ANTHROPIC_API_KEY set)

### Why use this?

- **Zero friction** - One command instead of 5 separate steps
- **Quality guaranteed** - Enhancement is mandatory, ensures professional output
- **Complete automation** - From config name to uploaded skill in Claude
- **Time savings** - Fully automated end-to-end workflow

### Phases executed:

```
📥 PHASE 1: Fetch Config (if config name provided)
📖 PHASE 2: Scrape Documentation
✨ PHASE 3: AI Enhancement (MANDATORY - no skip option)
📦 PHASE 4: Package Skill
☁️  PHASE 5: Upload to Claude (optional, requires API key)
```

**Requirements:**
- ANTHROPIC_API_KEY environment variable (for auto-upload)
- Claude Code Max plan (for local AI enhancement)

**Example:**
```bash
# Set API key once
export ANTHROPIC_API_KEY=sk-ant-your-key-here

# Run one command - sit back and relax!
skill-seekers install --config react

# Result: React skill uploaded to Claude in 20-45 minutes
```

---

## 📊 Feature Matrix

Skill Seekers supports **4 platforms** and **5 skill modes** with full feature parity.

**Platforms:** Claude AI, Google Gemini, OpenAI ChatGPT, Generic Markdown
**Skill Modes:** Documentation, GitHub, PDF, Unified Multi-Source, Local Repository

See [Complete Feature Matrix](docs/FEATURE_MATRIX.md) for detailed platform and feature support.

### Quick Platform Comparison

| Feature | Claude | Gemini | OpenAI | Markdown |
|---------|--------|--------|--------|----------|
| Format | ZIP + YAML | tar.gz | ZIP + Vector | ZIP |
| Upload | ✅ API | ✅ API | ✅ API | ❌ Manual |
| Enhancement | ✅ Sonnet 4 | ✅ 2.0 Flash | ✅ GPT-4o | ❌ None |
| All Skill Modes | ✅ | ✅ | ✅ | ✅ |

**Examples:**
```bash
# Package for all platforms (same skill)
skill-seekers package output/react/ --target claude
skill-seekers package output/react/ --target gemini
skill-seekers package output/react/ --target openai
skill-seekers package output/react/ --target markdown

# Install for specific platform
skill-seekers install --config django --target gemini
skill-seekers install --config fastapi --target openai
```

---

## Usage Examples

### Documentation Scraping

```bash
# Scrape documentation website
skill-seekers scrape --config configs/react.json

# Quick scrape without config
skill-seekers scrape --url https://react.dev --name react

# With async mode (3x faster)
skill-seekers scrape --config configs/godot.json --async --workers 8
```

### PDF Extraction

```bash
# Basic PDF extraction
skill-seekers pdf --pdf docs/manual.pdf --name myskill

# Advanced features
skill-seekers pdf --pdf docs/manual.pdf --name myskill \
    --extract-tables \        # Extract tables
    --parallel \              # Fast parallel processing
    --workers 8               # Use 8 CPU cores

# Scanned PDFs (requires: pip install pytesseract Pillow)
skill-seekers pdf --pdf docs/scanned.pdf --name myskill --ocr

# Password-protected PDFs
skill-seekers pdf --pdf docs/encrypted.pdf --name myskill --password mypassword
```

**Time:** ~5-15 minutes (or 2-5 minutes with parallel) | **Quality:** Production-ready | **Cost:** Free

### GitHub Repository Scraping

```bash
# Basic repository scraping
skill-seekers github --repo facebook/react

# Using a config file
skill-seekers github --config configs/react_github.json

# With authentication (higher rate limits)
export GITHUB_TOKEN=ghp_your_token_here
skill-seekers github --repo facebook/react

# Customize what to include
skill-seekers github --repo django/django \
    --include-issues \        # Extract GitHub Issues
    --max-issues 100 \        # Limit issue count
    --include-changelog \     # Extract CHANGELOG.md
    --include-releases        # Extract GitHub Releases
```

**Time:** ~5-10 minutes | **Quality:** Production-ready | **Cost:** Free

### Unified Multi-Source Scraping (**NEW - v2.0.0**)

**The Problem:** Documentation and code often drift apart. Docs might be outdated, missing features that exist in code, or documenting features that were removed.

**The Solution:** Combine documentation + GitHub + PDF into one unified skill that shows BOTH what's documented AND what actually exists, with clear warnings about discrepancies.

```bash
# Use existing unified configs
skill-seekers unified --config configs/react_unified.json
skill-seekers unified --config configs/django_unified.json

# Or create unified config (mix documentation + GitHub)
cat > configs/myframework_unified.json << 'EOF'
{
  "name": "myframework",
  "description": "Complete framework knowledge from docs + code",
  "merge_mode": "rule-based",
  "sources": [
    {
      "type": "documentation",
      "base_url": "https://docs.myframework.com/",
      "extract_api": true,
      "max_pages": 200
    },
    {
      "type": "github",
      "repo": "owner/myframework",
      "include_code": true,
      "code_analysis_depth": "surface"
    }
  ]
}
EOF

# Run unified scraper
skill-seekers unified --config configs/myframework_unified.json

# Package and upload
skill-seekers package output/myframework/
# Upload output/myframework.zip to Claude - Done!
```

**Time:** ~30-45 minutes | **Quality:** Production-ready with conflict detection | **Cost:** Free

**What Makes It Special:**

✅ **Conflict Detection** - Automatically finds 4 types of discrepancies:
- 🔴 **Missing in code** (high): Documented but not implemented
- 🟡 **Missing in docs** (medium): Implemented but not documented
- ⚠️ **Signature mismatch**: Different parameters/types
- ℹ️ **Description mismatch**: Different explanations

✅ **Transparent Reporting** - Shows both versions side-by-side:
```markdown
#### `move_local_x(delta: float)`

⚠️ **Conflict**: Documentation signature differs from implementation

**Documentation says:**
```
def move_local_x(delta: float)
```

**Code implementation:**
```python
def move_local_x(delta: float, snap: bool = False) -> None
```
```

✅ **Advantages:**
- **Identifies documentation gaps** - Find outdated or missing docs automatically
- **Catches code changes** - Know when APIs change without docs being updated
- **Single source of truth** - One skill showing intent (docs) AND reality (code)
- **Actionable insights** - Get suggestions for fixing each conflict
- **Development aid** - See what's actually in the codebase vs what's documented

**Example Unified Configs:**
- `configs/react_unified.json` - React docs + GitHub repo
- `configs/django_unified.json` - Django docs + GitHub repo
- `configs/fastapi_unified.json` - FastAPI docs + GitHub repo

**Full Guide:** See [docs/UNIFIED_SCRAPING.md](docs/UNIFIED_SCRAPING.md) for complete documentation.

### Private Config Repositories (**NEW - v2.2.0**)

**The Problem:** Teams need to share custom configs for internal documentation, but don't want to publish them publicly.

**The Solution:** Register private git repositories as config sources. Fetch configs from team repos just like the public API, with full authentication support.

```bash
# Setup: Set your GitHub token (one-time)
export GITHUB_TOKEN=ghp_your_token_here

# Option 1: Using MCP tools (recommended)
# Register your team's private repo
add_config_source(
    name="team",
    git_url="https://github.com/mycompany/skill-configs.git",
    token_env="GITHUB_TOKEN"
)

# Fetch config from team repo
fetch_config(source="team", config_name="internal-api")

# List all registered sources
list_config_sources()

# Remove source when no longer needed
remove_config_source(name="team")
```

**Direct Git URL mode** (no registration):
```bash
# Fetch directly from git URL
fetch_config(
    git_url="https://github.com/mycompany/configs.git",
    config_name="react-custom",
    token="ghp_your_token_here"
)
```

**Supported Platforms:**
- GitHub (token env: `GITHUB_TOKEN`)
- GitLab (token env: `GITLAB_TOKEN`)
- Gitea (token env: `GITEA_TOKEN`)
- Bitbucket (token env: `BITBUCKET_TOKEN`)
- Any git server (token env: `GIT_TOKEN`)

**Use Cases:**

📋 **Small Teams (3-5 people)**
```bash
# Team lead creates repo
gh repo create myteam/skill-configs --private

# Add configs to repo
cd myteam-skill-configs
cp ../Skill_Seekers/configs/react.json ./react-custom.json
# Edit selectors, categories for your internal docs...
git add . && git commit -m "Add custom React config" && git push

# Team members register (one-time)
add_config_source(name="team", git_url="https://github.com/myteam/skill-configs.git")

# Everyone can now fetch
fetch_config(source="team", config_name="react-custom")
```

🏢 **Enterprise (500+ developers)**
```bash
# IT pre-configures sources for everyone
add_config_source(name="platform", git_url="gitlab.company.com/platform/configs", priority=1)
add_config_source(name="mobile", git_url="gitlab.company.com/mobile/configs", priority=2)
add_config_source(name="official", git_url="api.skillseekersweb.com", priority=3)

# Developers use transparently
fetch_config(config_name="internal-platform")  # Finds in platform source
fetch_config(config_name="react")              # Falls back to official API
```

**Storage Locations:**
- Registry: `~/.skill-seekers/sources.json`
- Cache: `$SKILL_SEEKERS_CACHE_DIR` (default: `~/.skill-seekers/cache/`)

**Features:**
- ✅ **Shallow clone** - 10-50x faster, minimal disk space
- ✅ **Auto-pull** - Fetches latest changes automatically
- ✅ **Offline mode** - Works with cached repos when offline
- ✅ **Priority resolution** - Multiple sources with conflict resolution
- ✅ **Secure** - Tokens via environment variables only

**Example Team Repository:**

Try the included example:
```bash
# Test with file:// URL (no auth needed)
cd /path/to/Skill_Seekers

# Run the E2E test
python3 configs/example-team/test_e2e.py

# Or test manually
add_config_source(
    name="example",
    git_url="file://$(pwd)/configs/example-team",
    branch="master"
)

fetch_config(source="example", config_name="react-custom")
```

**Full Guide:** See [docs/GIT_CONFIG_SOURCES.md](docs/GIT_CONFIG_SOURCES.md) for complete documentation.

## How It Works

```mermaid
graph LR
    A[Documentation Website] --> B[Skill Seeker]
    B --> C[Scraper]
    B --> D[AI Enhancement]
    B --> E[Packager]
    C --> F[Organized References]
    D --> F
    F --> E
    E --> G[Claude Skill .zip]
    G --> H[Upload to Claude AI]
```

0. **Detect llms.txt** - Checks for llms-full.txt, llms.txt, llms-small.txt first
1. **Scrape**: Extracts all pages from documentation
2. **Categorize**: Organizes content into topics (API, guides, tutorials, etc.)
3. **Enhance**: AI analyzes docs and creates comprehensive SKILL.md with examples
4. **Package**: Bundles everything into a Claude-ready `.zip` file

## 📋 Prerequisites

**Before you start, make sure you have:**

1. **Python 3.10 or higher** - [Download](https://www.python.org/downloads/) | Check: `python3 --version`
2. **Git** - [Download](https://git-scm.com/) | Check: `git --version`
3. **15-30 minutes** for first-time setup

**First time user?** → **[Start Here: Bulletproof Quick Start Guide](BULLETPROOF_QUICKSTART.md)** 🎯

This guide walks you through EVERYTHING step-by-step (Python install, git clone, first skill creation).

---

## 🚀 Quick Start

### Method 1: MCP Server for 5 AI Agents (Easiest - **NEW v2.4.0!**)

Use Skill Seeker directly from **Claude Code, Cursor, Windsurf, VS Code + Cline, or IntelliJ IDEA** with natural language!

```bash
# Clone repository
git clone https://github.com/yusufkaraaslan/Skill_Seekers.git
cd Skill_Seekers

# One-time setup (5 minutes) - Auto-configures ALL 5 agents!
./setup_mcp.sh

# Restart your AI agent, then just ask:
```

**In Claude Code, Cursor, Windsurf, VS Code + Cline, or IntelliJ IDEA:**
```
List all available configs
Generate config for Tailwind at https://tailwindcss.com/docs
Scrape docs using configs/react.json
Package skill at output/react/
```

**Benefits:**
- ✅ No manual CLI commands
- ✅ Natural language interface
- ✅ Integrated with your workflow
- ✅ **18 tools** available instantly (up from 9!)
- ✅ **5 AI agents supported** - auto-configured with one command
- ✅ **Tested and working** in production

**NEW in v2.4.0:**
- ✅ **Upgraded to MCP SDK v1.25.0** - Latest features and performance
- ✅ **FastMCP Framework** - Modern, maintainable MCP implementation
- ✅ **HTTP + stdio transport** - Works with more AI agents
- ✅ **18 tools** (up from 9) - More capabilities
- ✅ **Multi-agent auto-configuration** - Setup all agents with one command

**Full guides:**
- 📘 [MCP Setup Guide](docs/MCP_SETUP.md) - Complete installation instructions
- 🧪 [MCP Testing Guide](docs/TEST_MCP_IN_CLAUDE_CODE.md) - Test all 18 tools
- 📦 [Large Documentation Guide](docs/LARGE_DOCUMENTATION.md) - Handle 10K-40K+ pages
- 📤 [Upload Guide](docs/UPLOAD_GUIDE.md) - How to upload skills to Claude

### Method 2: CLI (Traditional)

#### One-Time Setup: Create Virtual Environment

```bash
# Clone repository
git clone https://github.com/yusufkaraaslan/Skill_Seekers.git
cd Skill_Seekers

# Create virtual environment
python3 -m venv venv

# Activate virtual environment
source venv/bin/activate  # macOS/Linux
# OR on Windows: venv\Scripts\activate

# Install dependencies
pip install requests beautifulsoup4 pytest

# Save dependencies
pip freeze > requirements.txt

# Optional: Install anthropic for API-based enhancement (not needed for LOCAL enhancement)
# pip install anthropic
```

**Always activate the virtual environment before using Skill Seeker:**
```bash
source venv/bin/activate  # Run this each time you start a new terminal session
```

#### Easiest: Use a Preset

```bash
# Make sure venv is activated (you should see (venv) in your prompt)
source venv/bin/activate

# Optional: Estimate pages first (fast, 1-2 minutes)
skill-seekers estimate configs/godot.json

# Use Godot preset
skill-seekers scrape --config configs/godot.json

# Use React preset
skill-seekers scrape --config configs/react.json

# See all presets
ls configs/
```

### Interactive Mode

```bash
skill-seekers scrape --interactive
```

### Quick Mode

```bash
skill-seekers scrape \
  --name react \
  --url https://react.dev/ \
  --description "React framework for UIs"
```

## 📤 Uploading Skills to Claude

Once your skill is packaged, you need to upload it to Claude:

### Option 1: Automatic Upload (API-based)

```bash
# Set your API key (one-time)
export ANTHROPIC_API_KEY=sk-ant-...

# Package and upload automatically
skill-seekers package output/react/ --upload

# OR upload existing .zip
skill-seekers upload output/react.zip
```

**Benefits:**
- ✅ Fully automatic
- ✅ No manual steps
- ✅ Works from command line

**Requirements:**
- Anthropic API key (get from https://console.anthropic.com/)

### Option 2: Manual Upload (No API Key)

```bash
# Package skill
skill-seekers package output/react/

# This will:
# 1. Create output/react.zip
# 2. Open the output/ folder automatically
# 3. Show upload instructions

# Then manually upload:
# - Go to https://claude.ai/skills
# - Click "Upload Skill"
# - Select output/react.zip
# - Done!
```

**Benefits:**
- ✅ No API key needed
- ✅ Works for everyone
- ✅ Folder opens automatically

### Option 3: Claude Code (MCP) - Smart & Automatic

```
In Claude Code, just ask:
"Package and upload the React skill"

# With API key set:
# - Packages the skill
# - Uploads to Claude automatically
# - Done! ✅

# Without API key:
# - Packages the skill
# - Shows where to find the .zip
# - Provides manual upload instructions
```

**Benefits:**
- ✅ Natural language
- ✅ Smart auto-detection (uploads if API key available)
- ✅ Works with or without API key
- ✅ No errors or failures

---

## 🤖 Installing to AI Agents

Skill Seekers can automatically install skills to 10+ AI coding agents.

### Quick Start

```bash
# Install to specific agent
skill-seekers install-agent output/react/ --agent cursor

# Install to all agents at once
skill-seekers install-agent output/react/ --agent all

# Overwrite existing installation
skill-seekers install-agent output/react/ --agent claude --force

# Preview without installing
skill-seekers install-agent output/react/ --agent cursor --dry-run
```

### Supported Agents

| Agent | Path | Type |
|-------|------|------|
| **Claude Code** | `~/.claude/skills/` | Global |
| **Cursor** | `.cursor/skills/` | Project |
| **VS Code / Copilot** | `.github/skills/` | Project |
| **Amp** | `~/.amp/skills/` | Global |
| **Goose** | `~/.config/goose/skills/` | Global |
| **OpenCode** | `~/.opencode/skills/` | Global |
| **Letta** | `~/.letta/skills/` | Global |
| **Aide** | `~/.aide/skills/` | Global |
| **Windsurf** | `~/.windsurf/skills/` | Global |
| **Neovate Code** | `~/.neovate/skills/` | Global |

**Global paths** install to user's home directory (~/).
**Project paths** install to current project's root directory.

### Complete Workflow

```bash
# 1. Scrape documentation
skill-seekers scrape --config configs/react.json --enhance-local

# 2. Package skill
skill-seekers package output/react/

# 3. Install to your agent
skill-seekers install-agent output/react/ --agent cursor

# 4. Restart Cursor to load the skill
```

---

## 🤖 Multi-Agent MCP Support (NEW in v2.4.0)

**Skill Seekers MCP server now works with 5 leading AI coding agents!**

### Supported AI Agents

| Agent | Transport | Setup Difficulty | Auto-Configured |
|-------|-----------|------------------|-----------------|
| **Claude Code** | stdio | Easy | ✅ Yes |
| **VS Code + Cline** | stdio | Easy | ✅ Yes |
| **Cursor** | HTTP | Medium | ✅ Yes |
| **Windsurf** | HTTP | Medium | ✅ Yes |
| **IntelliJ IDEA** | HTTP | Medium | ✅ Yes |

### Quick Setup - All Agents at Once

```bash
# Clone repository
git clone https://github.com/yusufkaraaslan/Skill_Seekers.git
cd Skill_Seekers

# Run one command - auto-configures ALL 5 agents!
./setup_mcp.sh

# Restart your AI agent and start using natural language:
"List all available configs"
"Generate a React skill from https://react.dev/"
"Package the skill at output/react/"
```

**What `setup_mcp.sh` does:**
1. ✅ Installs MCP server dependencies
2. ✅ Configures Claude Code (stdio transport)
3. ✅ Configures VS Code + Cline (stdio transport)
4. ✅ Configures Cursor (HTTP transport)
5. ✅ Configures Windsurf (HTTP transport)
6. ✅ Configures IntelliJ IDEA (HTTP transport)
7. ✅ Shows next steps for each agent

**Time:** 5 minutes | **Result:** All agents configured and ready to use

### Transport Modes

Skill Seekers MCP server supports 2 transport modes:

#### stdio Transport (Claude Code, VS Code + Cline)

**How it works:** Agent launches MCP server as subprocess and communicates via stdin/stdout

**Benefits:**
- ✅ More secure (no network ports)
- ✅ Automatic lifecycle management
- ✅ Simpler configuration
- ✅ Better for single-user development

**Configuration example (Claude Code):**
```json
{
  "mcpServers": {
    "skill-seeker": {
      "command": "python3",
      "args": ["-m", "skill_seekers.mcp.server_fastmcp"],
      "cwd": "/path/to/Skill_Seekers"
    }
  }
}
```

#### HTTP Transport (Cursor, Windsurf, IntelliJ IDEA)

**How it works:** MCP server runs as HTTP service, agents connect as clients

**Benefits:**
- ✅ Multi-agent support (one server, multiple clients)
- ✅ Server can run independently
- ✅ Better for team collaboration
- ✅ Easier debugging and monitoring

**Configuration example (Cursor):**
```json
{
  "mcpServers": {
    "skill-seeker": {
      "url": "http://localhost:8765/sse"
    }
  }
}
```

**Starting HTTP server:**
```bash
# Start server manually (runs in background)
cd /path/to/Skill_Seekers
python3 -m skill_seekers.mcp.server_fastmcp --transport http --port 8765

# Or use auto-start script
./scripts/start_mcp_server.sh
```

### Agent-Specific Instructions

#### Claude Code (stdio)

```bash
# Already configured by setup_mcp.sh!
# Just restart Claude Code

# Config location: ~/.claude/claude_code_config.json
```

**Usage:**
```
In Claude Code:
"List all available configs"
"Scrape React docs at https://react.dev/"
```

#### VS Code + Cline Extension (stdio)

```bash
# Already configured by setup_mcp.sh!
# Just restart VS Code

# Config location: ~/Library/Application Support/Code/User/globalStorage/saoudrizwan.claude-dev/settings/cline_mcp_settings.json
```

**Usage:**
```
In Cline:
"Generate config for Tailwind"
"Package skill at output/tailwind/"
```

#### Cursor (HTTP)

```bash
# 1. Setup already configured HTTP settings
# Config location: ~/.cursor/mcp_settings.json

# 2. Start HTTP server (one-time per session)
./scripts/start_mcp_server.sh

# 3. Restart Cursor
```

**Usage:**
```
In Cursor:
"Show me all skill-seeker configs"
"Create Django skill from docs"
```

#### Windsurf (HTTP)

```bash
# 1. Setup already configured HTTP settings
# Config location: ~/.windsurf/mcp_settings.json

# 2. Start HTTP server (one-time per session)
./scripts/start_mcp_server.sh

# 3. Restart Windsurf
```

**Usage:**
```
In Windsurf:
"Estimate pages for Godot config"
"Build unified skill for FastAPI"
```

#### IntelliJ IDEA (HTTP)

```bash
# 1. Setup already configured HTTP settings
# Config location: ~/.intellij/mcp_settings.json

# 2. Start HTTP server (one-time per session)
./scripts/start_mcp_server.sh

# 3. Restart IntelliJ IDEA
```

**Usage:**
```
In IntelliJ IDEA:
"Validate my config file"
"Split large Godot config"
```

### Available MCP Tools (18 Total)

All agents have access to these 18 tools:

**Core Tools (9):**
1. `list_configs` - List all available preset configurations
2. `generate_config` - Generate new config for any docs site
3. `validate_config` - Validate config structure
4. `estimate_pages` - Estimate page count before scraping
5. `scrape_docs` - Scrape and build skill
6. `package_skill` - Package skill into .zip
7. `upload_skill` - Upload .zip to Claude
8. `split_config` - Split large documentation configs
9. `generate_router` - Generate router/hub skills

**Extended Tools (8 - NEW!):**
10. `scrape_github` - Scrape GitHub repositories
11. `scrape_pdf` - Extract content from PDFs
12. `unified_scrape` - Combine multiple sources
13. `merge_sources` - Merge documentation + code
14. `detect_conflicts` - Find doc/code discrepancies
15. `add_config_source` - Register private git repos
16. `fetch_config` - Fetch configs from git
17. `list_config_sources` - List registered sources

### What's New in v2.4.0

**MCP Infrastructure:**
- ✅ **Upgraded to MCP SDK v1.25.0** - Latest stable version
- ✅ **FastMCP Framework** - Modern, maintainable implementation
- ✅ **Dual Transport** - stdio + HTTP support
- ✅ **18 Tools** - Up from 9 (exactly 2x!)
- ✅ **Auto-Configuration** - One script configures all agents

**Agent Support:**
- ✅ **5 Agents Supported** - Claude Code, VS Code + Cline, Cursor, Windsurf, IntelliJ IDEA
- ✅ **Automatic Setup** - `./setup_mcp.sh` configures everything
- ✅ **Transport Detection** - Auto-selects stdio vs HTTP per agent
- ✅ **Config Management** - Handles all agent-specific config formats

**Developer Experience:**
- ✅ **One Setup Command** - Works for all agents
- ✅ **Natural Language** - Use plain English in any agent
- ✅ **No CLI Required** - All features via MCP tools
- ✅ **Full Testing** - All 18 tools tested and working

### Troubleshooting Multi-Agent Setup

**HTTP server not starting?**
```bash
# Check if port 8765 is in use
lsof -i :8765

# Use different port
python3 -m skill_seekers.mcp.server_fastmcp --transport http --port 9000

# Update agent config with new port
```

**Agent not finding MCP server?**
```bash
# Verify config file exists
cat ~/.claude/claude_code_config.json
cat ~/.cursor/mcp_settings.json

# Re-run setup
./setup_mcp.sh

# Check server logs
tail -f logs/mcp_server.log
```

**Tools not appearing in agent?**
```bash
# Restart agent completely (quit and relaunch)
# For HTTP transport, ensure server is running:
ps aux | grep "skill_seekers.mcp.server_fastmcp"

# Test server directly
curl http://localhost:8765/health
```

### Complete Multi-Agent Workflow

```bash
# 1. One-time setup (5 minutes)
git clone https://github.com/yusufkaraaslan/Skill_Seekers.git
cd Skill_Seekers
./setup_mcp.sh

# 2. For HTTP agents (Cursor/Windsurf/IntelliJ), start server
./scripts/start_mcp_server.sh

# 3. Restart your AI agent

# 4. Use natural language in ANY agent:
"List all available configs"
"Generate React skill from https://react.dev/"
"Estimate pages for Godot config"
"Package and upload skill at output/react/"

# 5. Result: Skills created without touching CLI!
```

**Full Guide:** See [docs/MCP_SETUP.md](docs/MCP_SETUP.md) for detailed multi-agent setup instructions.

---

## 📁 Simple Structure

```
doc-to-skill/
├── cli/
│   ├── doc_scraper.py      # Main scraping tool
│   ├── package_skill.py    # Package to .zip
│   ├── upload_skill.py     # Auto-upload (API)
│   └── enhance_skill.py    # AI enhancement
├── mcp/                    # MCP server for 5 AI agents
│   └── server.py           # 18 MCP tools (v2.7.0)
├── configs/                # Preset configurations
│   ├── godot.json         # Godot Engine
│   ├── react.json         # React
│   ├── vue.json           # Vue.js
│   ├── django.json        # Django
│   └── fastapi.json       # FastAPI
└── output/                 # All output (auto-created)
    ├── godot_data/        # Scraped data
    ├── godot/             # Built skill
    └── godot.zip          # Packaged skill
```

## ✨ Features

### 1. Fast Page Estimation (NEW!)

```bash
skill-seekers estimate configs/react.json

# Output:
📊 ESTIMATION RESULTS
✅ Pages Discovered: 180
📈 Estimated Total: 230
⏱️  Time Elapsed: 1.2 minutes
💡 Recommended max_pages: 280
```

**Benefits:**
- Know page count BEFORE scraping (saves time)
- Validates URL patterns work correctly
- Estimates total scraping time
- Recommends optimal `max_pages` setting
- Fast (1-2 minutes vs 20-40 minutes full scrape)

### 2. Auto-Detect Existing Data

```bash
skill-seekers scrape --config configs/godot.json

# If data exists:
✓ Found existing data: 245 pages
Use existing data? (y/n): y
⏭️  Skipping scrape, using existing data
```

### 3. Knowledge Generation

**Automatic pattern extraction:**
- Extracts common code patterns from docs
- Detects programming language
- Creates quick reference with real examples
- Smarter categorization with scoring

**Enhanced SKILL.md:**
- Real code examples from documentation
- Language-annotated code blocks
- Common patterns section
- Quick reference from actual usage examples

### 4. Smart Categorization

Automatically infers categories from:
- URL structure
- Page titles
- Content keywords
- With scoring for better accuracy

### 5. Code Language Detection

```python
# Automatically detects:
- Python (def, import, from)
- JavaScript (const, let, =>)
- GDScript (func, var, extends)
- C++ (#include, int main)
- And more...
```

### 5. Skip Scraping

```bash
# Scrape once
skill-seekers scrape --config configs/react.json

# Later, just rebuild (instant)
skill-seekers scrape --config configs/react.json --skip-scrape
```

### 6. Async Mode for Faster Scraping (2-3x Speed!)

```bash
# Enable async mode with 8 workers (recommended for large docs)
skill-seekers scrape --config configs/react.json --async --workers 8

# Small docs (~100-500 pages)
skill-seekers scrape --config configs/mydocs.json --async --workers 4

# Large docs (2000+ pages) with no rate limiting
skill-seekers scrape --config configs/largedocs.json --async --workers 8 --no-rate-limit
```

**Performance Comparison:**
- **Sync mode (threads):** ~18 pages/sec, 120 MB memory
- **Async mode:** ~55 pages/sec, 40 MB memory
- **Result:** 3x faster, 66% less memory!

**When to use:**
- ✅ Large documentation (500+ pages)
- ✅ Network latency is high
- ✅ Memory is constrained
- ❌ Small docs (< 100 pages) - overhead not worth it

**See full guide:** [ASYNC_SUPPORT.md](ASYNC_SUPPORT.md)

### 7. AI-Powered SKILL.md Enhancement

```bash
# Option 1: During scraping (API-based, requires API key)
pip3 install anthropic
export ANTHROPIC_API_KEY=sk-ant-...
skill-seekers scrape --config configs/react.json --enhance

# Option 2: During scraping (LOCAL, no API key - uses Claude Code Max)
skill-seekers scrape --config configs/react.json --enhance-local

# Option 3: After scraping (API-based, standalone)
skill-seekers enhance output/react/

# Option 4: After scraping (LOCAL, no API key, standalone)
skill-seekers enhance output/react/
```

**What it does:**
- Reads your reference documentation
- Uses Claude to generate an excellent SKILL.md
- Extracts best code examples (5-10 practical examples)
- Creates comprehensive quick reference
- Adds domain-specific key concepts
- Provides navigation guidance for different skill levels
- Automatically backs up original
- **Quality:** Transforms 75-line templates into 500+ line comprehensive guides

**LOCAL Enhancement (Recommended):**
- Uses your local coding agent (no API costs)
- **Multiple Agent Support:**
  - Claude Code (default) - Your Claude Code Max plan
  - Codex CLI - OpenAI Codex CLI
  - Copilot CLI - GitHub Copilot CLI
  - OpenCode CLI - OpenCode CLI
  - Custom agents - Use any CLI tool
- Choose agent with `--agent` flag:
  ```bash
  skill-seekers enhance output/react/ --agent codex
  skill-seekers enhance output/react/ --agent copilot
  skill-seekers enhance output/react/ --agent custom --agent-cmd "my-agent {prompt_file}"
  ```
- Environment variables for CI/CD:
  ```bash
  export SKILL_SEEKER_AGENT=codex
  export SKILL_SEEKER_AGENT_CMD="my-agent {prompt_file}"
  ```
- Analyzes reference files automatically
- Takes 30-60 seconds
- Quality: 9/10 (comparable to API version)

### 7. Large Documentation Support (10K-40K+ Pages)

**For massive documentation sites like Godot (40K pages), AWS, or Microsoft Docs:**

```bash
# 1. Estimate first (discover page count)
skill-seekers estimate configs/godot.json

# 2. Auto-split into focused sub-skills
python3 -m skill_seekers.cli.split_config configs/godot.json --strategy router

# Creates:
# - godot-scripting.json (5K pages)
# - godot-2d.json (8K pages)
# - godot-3d.json (10K pages)
# - godot-physics.json (6K pages)
# - godot-shaders.json (11K pages)

# 3. Scrape all in parallel (4-8 hours instead of 20-40!)
for config in configs/godot-*.json; do
  skill-seekers scrape --config $config &
done
wait

# 4. Generate intelligent router/hub skill
python3 -m skill_seekers.cli.generate_router configs/godot-*.json

# 5. Package all skills
python3 -m skill_seekers.cli.package_multi output/godot*/

# 6. Upload all .zip files to Claude
# Users just ask questions naturally!
# Router automatically directs to the right sub-skill!
```

**Split Strategies:**
- **auto** - Intelligently detects best strategy based on page count
- **category** - Split by documentation categories (scripting, 2d, 3d, etc.)
- **router** - Create hub skill + specialized sub-skills (RECOMMENDED)
- **size** - Split every N pages (for docs without clear categories)

**Benefits:**
- ✅ Faster scraping (parallel execution)
- ✅ More focused skills (better Claude performance)
- ✅ Easier maintenance (update one topic at a time)
- ✅ Natural user experience (router handles routing)
- ✅ Avoids context window limits

**Configuration:**
```json
{
  "name": "godot",
  "max_pages": 40000,
  "split_strategy": "router",
  "split_config": {
    "target_pages_per_skill": 5000,
    "create_router": true,
    "split_by_categories": ["scripting", "2d", "3d", "physics"]
  }
}
```

**Full Guide:** [Large Documentation Guide](docs/LARGE_DOCUMENTATION.md)

### 8. Checkpoint/Resume for Long Scrapes

**Never lose progress on long-running scrapes:**

```bash
# Enable in config
{
  "checkpoint": {
    "enabled": true,
    "interval": 1000  // Save every 1000 pages
  }
}

# If scrape is interrupted (Ctrl+C or crash)
skill-seekers scrape --config configs/godot.json --resume

# Resume from last checkpoint
✅ Resuming from checkpoint (12,450 pages scraped)
⏭️  Skipping 12,450 already-scraped pages
🔄 Continuing from where we left off...

# Start fresh (clear checkpoint)
skill-seekers scrape --config configs/godot.json --fresh
```

**Benefits:**
- ✅ Auto-saves every 1000 pages (configurable)
- ✅ Saves on interruption (Ctrl+C)
- ✅ Resume with `--resume` flag
- ✅ Never lose hours of scraping progress

## 🎯 Complete Workflows

### First Time (With Scraping + Enhancement)

```bash
# 1. Scrape + Build + AI Enhancement (LOCAL, no API key)
skill-seekers scrape --config configs/godot.json --enhance-local

# 2. Wait for new terminal to close (enhancement completes)
# Check the enhanced SKILL.md:
cat output/godot/SKILL.md

# 3. Package
skill-seekers package output/godot/

# 4. Done! You have godot.zip with excellent SKILL.md
```

**Time:** 20-40 minutes (scraping) + 60 seconds (enhancement) = ~21-41 minutes

### Using Existing Data (Fast!)

```bash
# 1. Use cached data + Local Enhancement
skill-seekers scrape --config configs/godot.json --skip-scrape
skill-seekers enhance output/godot/

# 2. Package
skill-seekers package output/godot/

# 3. Done!
```

**Time:** 1-3 minutes (build) + 60 seconds (enhancement) = ~2-4 minutes total

### Without Enhancement (Basic)

```bash
# 1. Scrape + Build (no enhancement)
skill-seekers scrape --config configs/godot.json

# 2. Package
skill-seekers package output/godot/

# 3. Done! (SKILL.md will be basic template)
```

**Time:** 20-40 minutes
**Note:** SKILL.md will be generic - enhancement strongly recommended!

## 📋 Available Presets

| Config | Framework | Description |
|--------|-----------|-------------|
| `godot.json` | Godot Engine | Game development |
| `react.json` | React | UI framework |
| `vue.json` | Vue.js | Progressive framework |
| `django.json` | Django | Python web framework |
| `fastapi.json` | FastAPI | Modern Python API |
| `ansible-core.json` | Ansible Core 2.19 | Automation & configuration |

### Using Presets

```bash
# Godot
skill-seekers scrape --config configs/godot.json

# React
skill-seekers scrape --config configs/react.json

# Vue
skill-seekers scrape --config configs/vue.json

# Django
skill-seekers scrape --config configs/django.json

# FastAPI
skill-seekers scrape --config configs/fastapi.json

# Ansible
skill-seekers scrape --config configs/ansible-core.json
```

## 🎨 Creating Your Own Config

### Option 1: Interactive

```bash
skill-seekers scrape --interactive
# Follow prompts, it will create the config for you
```

### Option 2: Copy and Edit

```bash
# Copy a preset
cp configs/react.json configs/myframework.json

# Edit it
nano configs/myframework.json

# Use it
skill-seekers scrape --config configs/myframework.json
```

### Where to Place Custom Configs

You have **three options** for placing your custom config files:

#### Option A: User Config Directory (Recommended for Personal Configs)

```bash
# Create your config in your home directory
mkdir -p ~/.config/skill-seekers/configs
cat > ~/.config/skill-seekers/configs/myproject.json << 'EOF'
{
  "name": "myproject",
  "base_url": "https://docs.myproject.com/",
  "max_pages": 50
}
EOF

# Use it (tool automatically finds it)
skill-seekers scrape --config myproject.json
# or
skill-seekers scrape --config configs/myproject.json
```

**Benefits:**
- ✅ Configs persist across project directories
- ✅ Separate from your project code
- ✅ Easy to manage personal configurations

#### Option B: Current Directory (Good for Project-Specific Configs)

```bash
# Create configs in your project folder
mkdir -p configs
cat > configs/myproject.json << 'EOF'
{
  "name": "myproject",
  "base_url": "https://docs.myproject.com/"
}
EOF

# Use it
skill-seekers scrape --config configs/myproject.json
```

**Benefits:**
- ✅ Config lives with your project
- ✅ Easy to commit to version control
- ✅ Team members can use the same config

#### Option C: Absolute Path (For Configs Stored Elsewhere)

```bash
# Use any file path
skill-seekers scrape --config /full/path/to/myconfig.json
```

**Config Resolution Order:**

The tool searches for configs in this order:
1. Exact path as provided
2. `./configs/` (current directory)
3. `~/.config/skill-seekers/configs/` (user config directory)
4. SkillSeekersWeb.com API (preset configs)

### Config Structure

```json
{
  "name": "myframework",
  "description": "When to use this skill",
  "base_url": "https://docs.myframework.com/",
  "selectors": {
    "main_content": "article",
    "title": "h1",
    "code_blocks": "pre code"
  },
  "url_patterns": {
    "include": ["/docs", "/guide"],
    "exclude": ["/blog", "/about"]
  },
  "categories": {
    "getting_started": ["intro", "quickstart"],
    "api": ["api", "reference"]
  },
  "rate_limit": 0.5,
  "max_pages": 500
}
```

## 📊 What Gets Created

```
output/
├── godot_data/              # Scraped raw data
│   ├── pages/              # JSON files (one per page)
│   └── summary.json        # Overview
│
└── godot/                   # The skill
    ├── SKILL.md            # Enhanced with real examples
    ├── references/         # Categorized docs
    │   ├── index.md
    │   ├── getting_started.md
    │   ├── scripting.md
    │   └── ...
    ├── scripts/            # Empty (add your own)
    └── assets/             # Empty (add your own)
```

## 🎯 Command Line Options

```bash
# Interactive mode
skill-seekers scrape --interactive

# Use config file
skill-seekers scrape --config configs/godot.json

# Quick mode
skill-seekers scrape --name react --url https://react.dev/

# Skip scraping (use existing data)
skill-seekers scrape --config configs/godot.json --skip-scrape

# With description
skill-seekers scrape \
  --name react \
  --url https://react.dev/ \
  --description "React framework for building UIs"
```

## 💡 Tips

### 1. Test Small First

Edit `max_pages` in config to test:
```json
{
  "max_pages": 20  // Test with just 20 pages
}
```

### 2. Reuse Scraped Data

```bash
# Scrape once
skill-seekers scrape --config configs/react.json

# Rebuild multiple times (instant)
skill-seekers scrape --config configs/react.json --skip-scrape
skill-seekers scrape --config configs/react.json --skip-scrape
```

### 3. Finding Selectors

```python
# Test in Python
from bs4 import BeautifulSoup
import requests

url = "https://docs.example.com/page"
soup = BeautifulSoup(requests.get(url).content, 'html.parser')

# Try different selectors
print(soup.select_one('article'))
print(soup.select_one('main'))
print(soup.select_one('div[role="main"]'))
```

### 4. Check Output Quality

```bash
# After building, check:
cat output/godot/SKILL.md  # Should have real examples
cat output/godot/references/index.md  # Categories
```

## 🐛 Troubleshooting

### No Content Extracted?
- Check your `main_content` selector
- Try: `article`, `main`, `div[role="main"]`

### Data Exists But Won't Use It?
```bash
# Force re-scrape
rm -rf output/myframework_data/
skill-seekers scrape --config configs/myframework.json
```

### Categories Not Good?
Edit the config `categories` section with better keywords.

### Want to Update Docs?
```bash
# Delete old data
rm -rf output/godot_data/

# Re-scrape
skill-seekers scrape --config configs/godot.json
```

## 📈 Performance

| Task | Time | Notes |
|------|------|-------|
| Scraping (sync) | 15-45 min | First time only, thread-based |
| Scraping (async) | 5-15 min | 2-3x faster with --async flag |
| Building | 1-3 min | Fast! |
| Re-building | <1 min | With --skip-scrape |
| Packaging | 5-10 sec | Final zip |

## ✅ Summary

**One tool does everything:**
1. ✅ Scrapes documentation
2. ✅ Auto-detects existing data
3. ✅ Generates better knowledge
4. ✅ Creates enhanced skills
5. ✅ Works with presets or custom configs
6. ✅ Supports skip-scraping for fast iteration

**Simple structure:**
- `doc_scraper.py` - The tool
- `configs/` - Presets
- `output/` - Everything else

**Better output:**
- Real code examples with language detection
- Common patterns extracted from docs
- Smart categorization
- Enhanced SKILL.md with actual examples

## 📚 Documentation

### Getting Started
- **[BULLETPROOF_QUICKSTART.md](BULLETPROOF_QUICKSTART.md)** - 🎯 **START HERE** if you're new!
- **[QUICKSTART.md](QUICKSTART.md)** - Quick start for experienced users
- **[TROUBLESHOOTING.md](TROUBLESHOOTING.md)** - Common issues and solutions

### Guides
- **[docs/LARGE_DOCUMENTATION.md](docs/LARGE_DOCUMENTATION.md)** - Handle 10K-40K+ page docs
- **[ASYNC_SUPPORT.md](ASYNC_SUPPORT.md)** - Async mode guide (2-3x faster scraping)
- **[docs/ENHANCEMENT.md](docs/ENHANCEMENT.md)** - AI enhancement guide
- **[docs/TERMINAL_SELECTION.md](docs/TERMINAL_SELECTION.md)** - Configure terminal app for local enhancement
- **[docs/UPLOAD_GUIDE.md](docs/UPLOAD_GUIDE.md)** - How to upload skills to Claude
- **[docs/MCP_SETUP.md](docs/MCP_SETUP.md)** - MCP integration setup

### Technical
- **[docs/CLAUDE.md](docs/CLAUDE.md)** - Technical architecture
- **[STRUCTURE.md](STRUCTURE.md)** - Repository structure

## 🎮 Ready?

```bash
# Try Godot
skill-seekers scrape --config configs/godot.json

# Try React
skill-seekers scrape --config configs/react.json

# Or go interactive
skill-seekers scrape --interactive
```

## 📝 License

MIT License - see [LICENSE](LICENSE) file for details

---

Happy skill building! 🚀