docs: complete documentation overhaul with v3.1.0 release notes and zh-CN translations
Documentation restructure: - New docs/getting-started/ guide (4 files: install, quick-start, first-skill, next-steps) - New docs/user-guide/ section (6 files: core concepts through troubleshooting) - New docs/reference/ section (CLI_REFERENCE, CONFIG_FORMAT, ENVIRONMENT_VARIABLES, MCP_REFERENCE) - New docs/advanced/ section (custom-workflows, mcp-server, multi-source) - New docs/ARCHITECTURE.md - system architecture overview - Archived legacy files (QUICKSTART.md, QUICK_REFERENCE.md, docs/guides/USAGE.md) to docs/archive/legacy/ Chinese (zh-CN) translations: - Full zh-CN mirror of all user-facing docs (getting-started, user-guide, reference, advanced) - GitHub Actions workflow for translation sync (.github/workflows/translate-docs.yml) - Translation sync checker script (scripts/check_translation_sync.sh) - Translation helper script (scripts/translate_doc.py) Content updates: - CHANGELOG.md: [Unreleased] → [3.1.0] - 2026-02-22 - README.md: updated with new doc structure links - AGENTS.md: updated agent documentation - docs/features/UNIFIED_SCRAPING.md: updated for unified scraper workflow JSON config Analysis/planning artifacts (kept for reference): - DOCUMENTATION_OVERHAUL_PLAN.md, DOCUMENTATION_OVERHAUL_SUMMARY.md - FEATURE_GAP_ANALYSIS.md, IMPLEMENTATION_GAPS_ANALYSIS.md, CREATE_COMMAND_COVERAGE_ANALYSIS.md - CHINESE_TRANSLATION_IMPLEMENTATION_SUMMARY.md, ISSUE_260_UPDATE.md Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
yusyus
432
docs/user-guide/01-core-concepts.md
Normal file
432
docs/user-guide/01-core-concepts.md
Normal file
@@ -0,0 +1,432 @@
|
||||
# Core Concepts
|
||||
|
||||
> **Skill Seekers v3.1.0**
|
||||
> **Understanding how Skill Seekers works**
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
Skill Seekers transforms documentation, code, and content into **structured knowledge assets** that AI systems can use effectively.
|
||||
|
||||
```
|
||||
Raw Content → Skill Seekers → AI-Ready Skill
|
||||
↓ ↓
|
||||
(docs, code, (SKILL.md +
|
||||
PDFs, repos) references)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## What is a Skill?
|
||||
|
||||
A **skill** is a structured knowledge package containing:
|
||||
|
||||
```
|
||||
output/my-skill/
|
||||
├── SKILL.md # Main file (400+ lines typically)
|
||||
├── references/ # Categorized content
|
||||
│ ├── index.md # Navigation
|
||||
│ ├── getting_started.md
|
||||
│ ├── api_reference.md
|
||||
│ └── ...
|
||||
├── .skill-seekers/ # Metadata
|
||||
└── assets/ # Images, downloads
|
||||
```
|
||||
|
||||
### SKILL.md Structure
|
||||
|
||||
```markdown
|
||||
# My Framework Skill
|
||||
|
||||
## Overview
|
||||
Brief description of the framework...
|
||||
|
||||
## Quick Reference
|
||||
Common commands and patterns...
|
||||
|
||||
## Categories
|
||||
- [Getting Started](#getting-started)
|
||||
- [API Reference](#api-reference)
|
||||
- [Guides](#guides)
|
||||
|
||||
## Getting Started
|
||||
### Installation
|
||||
```bash
|
||||
npm install my-framework
|
||||
```
|
||||
|
||||
### First Steps
|
||||
...
|
||||
|
||||
## API Reference
|
||||
...
|
||||
```
|
||||
|
||||
### Why This Structure?
|
||||
|
||||
| Element | Purpose |
|
||||
|---------|---------|
|
||||
| **Overview** | Quick context for AI |
|
||||
| **Quick Reference** | Common patterns at a glance |
|
||||
| **Categories** | Organized deep dives |
|
||||
| **Code Examples** | Copy-paste ready snippets |
|
||||
|
||||
---
|
||||
|
||||
## Source Types
|
||||
|
||||
Skill Seekers works with four types of sources:
|
||||
|
||||
### 1. Documentation Websites
|
||||
|
||||
**What:** Web-based documentation (ReadTheDocs, Docusaurus, GitBook, etc.)
|
||||
|
||||
**Examples:**
|
||||
- React docs (react.dev)
|
||||
- Django docs (docs.djangoproject.com)
|
||||
- Kubernetes docs (kubernetes.io)
|
||||
|
||||
**Command:**
|
||||
```bash
|
||||
skill-seekers create https://docs.example.com/
|
||||
```
|
||||
|
||||
**Best for:**
|
||||
- Framework documentation
|
||||
- API references
|
||||
- Tutorials and guides
|
||||
|
||||
---
|
||||
|
||||
### 2. GitHub Repositories
|
||||
|
||||
**What:** Source code repositories with analysis
|
||||
|
||||
**Extracts:**
|
||||
- Code structure and APIs
|
||||
- README and documentation
|
||||
- Issues and discussions
|
||||
- Releases and changelog
|
||||
|
||||
**Command:**
|
||||
```bash
|
||||
skill-seekers create owner/repo
|
||||
skill-seekers github --repo owner/repo
|
||||
```
|
||||
|
||||
**Best for:**
|
||||
- Understanding codebases
|
||||
- API implementation details
|
||||
- Contributing guidelines
|
||||
|
||||
---
|
||||
|
||||
### 3. PDF Documents
|
||||
|
||||
**What:** PDF manuals, papers, documentation
|
||||
|
||||
**Handles:**
|
||||
- Text extraction
|
||||
- OCR for scanned PDFs
|
||||
- Table extraction
|
||||
- Image extraction
|
||||
|
||||
**Command:**
|
||||
```bash
|
||||
skill-seekers create manual.pdf
|
||||
skill-seekers pdf --pdf manual.pdf
|
||||
```
|
||||
|
||||
**Best for:**
|
||||
- Product manuals
|
||||
- Research papers
|
||||
- Legacy documentation
|
||||
|
||||
---
|
||||
|
||||
### 4. Local Codebases
|
||||
|
||||
**What:** Your local projects and code
|
||||
|
||||
**Analyzes:**
|
||||
- Source code structure
|
||||
- Comments and docstrings
|
||||
- Test files
|
||||
- Configuration patterns
|
||||
|
||||
**Command:**
|
||||
```bash
|
||||
skill-seekers create ./my-project
|
||||
skill-seekers analyze --directory ./my-project
|
||||
```
|
||||
|
||||
**Best for:**
|
||||
- Your own projects
|
||||
- Internal tools
|
||||
- Code review preparation
|
||||
|
||||
---
|
||||
|
||||
## The Workflow
|
||||
|
||||
### Phase 1: Ingest
|
||||
|
||||
```
|
||||
┌─────────────┐ ┌──────────────┐
|
||||
│ Source │────▶│ Scraper │
|
||||
│ (URL/repo/ │ │ (extracts │
|
||||
│ PDF/local) │ │ content) │
|
||||
└─────────────┘ └──────────────┘
|
||||
```
|
||||
|
||||
- Detects source type automatically
|
||||
- Crawls and downloads content
|
||||
- Respects rate limits
|
||||
- Extracts text, code, metadata
|
||||
|
||||
---
|
||||
|
||||
### Phase 2: Structure
|
||||
|
||||
```
|
||||
┌──────────────┐ ┌──────────────┐
|
||||
│ Raw Data │────▶│ Builder │
|
||||
│ (pages/files/│ │ (organizes │
|
||||
│ commits) │ │ by category)│
|
||||
└──────────────┘ └──────────────┘
|
||||
```
|
||||
|
||||
- Categorizes content by topic
|
||||
- Extracts code examples
|
||||
- Builds navigation structure
|
||||
- Creates reference files
|
||||
|
||||
---
|
||||
|
||||
### Phase 3: Enhance (Optional)
|
||||
|
||||
```
|
||||
┌──────────────┐ ┌──────────────┐
|
||||
│ SKILL.md │────▶│ Enhancer │
|
||||
│ (basic) │ │ (AI improves │
|
||||
│ │ │ quality) │
|
||||
└──────────────┘ └──────────────┘
|
||||
```
|
||||
|
||||
- AI reviews and improves content
|
||||
- Adds examples and patterns
|
||||
- Fixes formatting
|
||||
- Enhances navigation
|
||||
|
||||
**Modes:**
|
||||
- **API:** Uses Claude API (fast, costs ~$0.10-0.30)
|
||||
- **LOCAL:** Uses Claude Code (free, requires Claude Code Max)
|
||||
|
||||
---
|
||||
|
||||
### Phase 4: Package
|
||||
|
||||
```
|
||||
┌──────────────┐ ┌──────────────┐
|
||||
│ Skill Dir │────▶│ Packager │
|
||||
│ (structured │ │ (creates │
|
||||
│ content) │ │ platform │
|
||||
│ │ │ format) │
|
||||
└──────────────┘ └──────────────┘
|
||||
```
|
||||
|
||||
- Formats for target platform
|
||||
- Creates archives (ZIP, tar.gz)
|
||||
- Optimizes for size
|
||||
- Validates structure
|
||||
|
||||
---
|
||||
|
||||
### Phase 5: Upload (Optional)
|
||||
|
||||
```
|
||||
┌──────────────┐ ┌──────────────┐
|
||||
│ Package │────▶│ Platform │
|
||||
│ (.zip/.tar) │ │ (Claude/ │
|
||||
│ │ │ Gemini/etc) │
|
||||
└──────────────┘ └──────────────┘
|
||||
```
|
||||
|
||||
- Uploads to target platform
|
||||
- Configures settings
|
||||
- Returns skill ID/URL
|
||||
|
||||
---
|
||||
|
||||
## Enhancement Levels
|
||||
|
||||
Control how much AI enhancement is applied:
|
||||
|
||||
| Level | What Happens | Use Case |
|
||||
|-------|--------------|----------|
|
||||
| **0** | No enhancement | Fast scraping, manual review |
|
||||
| **1** | SKILL.md only | Basic improvement |
|
||||
| **2** | + architecture/config | **Recommended** - good balance |
|
||||
| **3** | Full enhancement | Maximum quality, takes longer |
|
||||
|
||||
**Default:** Level 2
|
||||
|
||||
```bash
|
||||
# Skip enhancement (fastest)
|
||||
skill-seekers create <source> --enhance-level 0
|
||||
|
||||
# Full enhancement (best quality)
|
||||
skill-seekers create <source> --enhance-level 3
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Target Platforms
|
||||
|
||||
Package skills for different AI systems:
|
||||
|
||||
| Platform | Format | Use |
|
||||
|----------|--------|-----|
|
||||
| **Claude AI** | ZIP + YAML | Claude Code, Claude API |
|
||||
| **Gemini** | tar.gz | Google Gemini |
|
||||
| **OpenAI** | ZIP + Vector | ChatGPT, Assistants API |
|
||||
| **LangChain** | Documents | RAG pipelines |
|
||||
| **LlamaIndex** | TextNodes | Query engines |
|
||||
| **ChromaDB** | Collection | Vector search |
|
||||
| **Weaviate** | Objects | Vector database |
|
||||
| **Cursor** | .cursorrules | IDE AI assistant |
|
||||
| **Windsurf** | .windsurfrules | IDE AI assistant |
|
||||
|
||||
---
|
||||
|
||||
## Configuration
|
||||
|
||||
### Simple (Auto-Detect)
|
||||
|
||||
```bash
|
||||
# Just provide the source
|
||||
skill-seekers create https://docs.react.dev/
|
||||
```
|
||||
|
||||
### Preset Configs
|
||||
|
||||
```bash
|
||||
# Use predefined configuration
|
||||
skill-seekers create --config react
|
||||
```
|
||||
|
||||
**Available presets:** `react`, `vue`, `django`, `fastapi`, `godot`, etc.
|
||||
|
||||
### Custom Config
|
||||
|
||||
```bash
|
||||
# Create custom config
|
||||
cat > configs/my-docs.json << 'EOF'
|
||||
{
|
||||
"name": "my-docs",
|
||||
"base_url": "https://docs.example.com/",
|
||||
"max_pages": 200
|
||||
}
|
||||
EOF
|
||||
|
||||
skill-seekers create --config configs/my-docs.json
|
||||
```
|
||||
|
||||
See [Config Format](../reference/CONFIG_FORMAT.md) for full specification.
|
||||
|
||||
---
|
||||
|
||||
## Multi-Source Skills
|
||||
|
||||
Combine multiple sources into one skill:
|
||||
|
||||
```bash
|
||||
# Create unified config
|
||||
cat > configs/my-project.json << 'EOF'
|
||||
{
|
||||
"name": "my-project",
|
||||
"sources": [
|
||||
{"type": "docs", "base_url": "https://docs.example.com/"},
|
||||
{"type": "github", "repo": "owner/repo"},
|
||||
{"type": "pdf", "pdf_path": "manual.pdf"}
|
||||
]
|
||||
}
|
||||
EOF
|
||||
|
||||
# Run unified scraping
|
||||
skill-seekers unified --config configs/my-project.json
|
||||
```
|
||||
|
||||
**Benefits:**
|
||||
- Single skill with complete context
|
||||
- Automatic conflict detection
|
||||
- Cross-referenced content
|
||||
|
||||
---
|
||||
|
||||
## Caching and Resumption
|
||||
|
||||
### How Caching Works
|
||||
|
||||
```
|
||||
First scrape: Downloads all pages → saves to output/{name}_data/
|
||||
Second scrape: Reuses cached data → fast rebuild
|
||||
```
|
||||
|
||||
### Skip Scraping
|
||||
|
||||
```bash
|
||||
# Use cached data, just rebuild
|
||||
skill-seekers create --config react --skip-scrape
|
||||
```
|
||||
|
||||
### Resume Interrupted Jobs
|
||||
|
||||
```bash
|
||||
# List resumable jobs
|
||||
skill-seekers resume --list
|
||||
|
||||
# Resume specific job
|
||||
skill-seekers resume job-abc123
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Rate Limiting
|
||||
|
||||
Be respectful to servers:
|
||||
|
||||
```bash
|
||||
# Default: 0.5 seconds between requests
|
||||
skill-seekers create <source>
|
||||
|
||||
# Faster (for your own servers)
|
||||
skill-seekers create <source> --rate-limit 0.1
|
||||
|
||||
# Slower (for rate-limited sites)
|
||||
skill-seekers create <source> --rate-limit 2.0
|
||||
```
|
||||
|
||||
**Why it matters:**
|
||||
- Prevents being blocked
|
||||
- Respects server resources
|
||||
- Good citizenship
|
||||
|
||||
---
|
||||
|
||||
## Key Takeaways
|
||||
|
||||
1. **Skills are structured knowledge** - Not just raw text
|
||||
2. **Auto-detection works** - Usually don't need custom configs
|
||||
3. **Enhancement improves quality** - Level 2 is the sweet spot
|
||||
4. **Package once, use everywhere** - Same skill, multiple platforms
|
||||
5. **Cache saves time** - Rebuild without re-scraping
|
||||
|
||||
---
|
||||
|
||||
## Next Steps
|
||||
|
||||
- [Scraping Guide](02-scraping.md) - Deep dive into source options
|
||||
- [Enhancement Guide](03-enhancement.md) - AI enhancement explained
|
||||
- [Config Format](../reference/CONFIG_FORMAT.md) - Custom configurations
|
||||
409
docs/user-guide/02-scraping.md
Normal file
409
docs/user-guide/02-scraping.md
Normal file
@@ -0,0 +1,409 @@
|
||||
# Scraping Guide
|
||||
|
||||
> **Skill Seekers v3.1.0**
|
||||
> **Complete guide to all scraping options**
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
Skill Seekers can extract knowledge from four types of sources:
|
||||
|
||||
| Source | Command | Best For |
|
||||
|--------|---------|----------|
|
||||
| **Documentation** | `create <url>` | Web docs, tutorials, API refs |
|
||||
| **GitHub** | `create <repo>` | Source code, issues, releases |
|
||||
| **PDF** | `create <file.pdf>` | Manuals, papers, reports |
|
||||
| **Local** | `create <./path>` | Your projects, internal code |
|
||||
|
||||
---
|
||||
|
||||
## Documentation Scraping
|
||||
|
||||
### Basic Usage
|
||||
|
||||
```bash
|
||||
# Auto-detect and scrape
|
||||
skill-seekers create https://docs.react.dev/
|
||||
|
||||
# With custom name
|
||||
skill-seekers create https://docs.react.dev/ --name react-docs
|
||||
|
||||
# With description
|
||||
skill-seekers create https://docs.react.dev/ \
|
||||
--description "React JavaScript library documentation"
|
||||
```
|
||||
|
||||
### Using Preset Configs
|
||||
|
||||
```bash
|
||||
# List available presets
|
||||
skill-seekers estimate --all
|
||||
|
||||
# Use preset
|
||||
skill-seekers create --config react
|
||||
skill-seekers create --config django
|
||||
skill-seekers create --config fastapi
|
||||
```
|
||||
|
||||
**Available presets:** See `configs/` directory in repository.
|
||||
|
||||
### Custom Configuration
|
||||
|
||||
```bash
|
||||
# Create config file
|
||||
cat > configs/my-docs.json << 'EOF'
|
||||
{
|
||||
"name": "my-framework",
|
||||
"base_url": "https://docs.example.com/",
|
||||
"description": "My framework documentation",
|
||||
"max_pages": 200,
|
||||
"rate_limit": 0.5,
|
||||
"selectors": {
|
||||
"main_content": "article",
|
||||
"title": "h1"
|
||||
},
|
||||
"url_patterns": {
|
||||
"include": ["/docs/", "/api/"],
|
||||
"exclude": ["/blog/", "/search"]
|
||||
}
|
||||
}
|
||||
EOF
|
||||
|
||||
# Use config
|
||||
skill-seekers create --config configs/my-docs.json
|
||||
```
|
||||
|
||||
See [Config Format](../reference/CONFIG_FORMAT.md) for all options.
|
||||
|
||||
### Advanced Options
|
||||
|
||||
```bash
|
||||
# Limit pages (for testing)
|
||||
skill-seekers create <url> --max-pages 50
|
||||
|
||||
# Adjust rate limit
|
||||
skill-seekers create <url> --rate-limit 1.0
|
||||
|
||||
# Parallel workers (faster)
|
||||
skill-seekers create <url> --workers 5 --async
|
||||
|
||||
# Dry run (preview)
|
||||
skill-seekers create <url> --dry-run
|
||||
|
||||
# Resume interrupted
|
||||
skill-seekers create <url> --resume
|
||||
|
||||
# Fresh start (ignore cache)
|
||||
skill-seekers create <url> --fresh
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## GitHub Repository Scraping
|
||||
|
||||
### Basic Usage
|
||||
|
||||
```bash
|
||||
# By repo name
|
||||
skill-seekers create facebook/react
|
||||
|
||||
# With explicit flag
|
||||
skill-seekers github --repo facebook/react
|
||||
|
||||
# With custom name
|
||||
skill-seekers github --repo facebook/react --name react-source
|
||||
```
|
||||
|
||||
### With GitHub Token
|
||||
|
||||
```bash
|
||||
# Set token for higher rate limits
|
||||
export GITHUB_TOKEN=ghp_...
|
||||
|
||||
# Use token
|
||||
skill-seekers github --repo facebook/react
|
||||
```
|
||||
|
||||
**Benefits of token:**
|
||||
- 5000 requests/hour vs 60
|
||||
- Access to private repos
|
||||
- Higher GraphQL limits
|
||||
|
||||
### What Gets Extracted
|
||||
|
||||
| Data | Default | Flag to Disable |
|
||||
|------|---------|-----------------|
|
||||
| Source code | ✅ | `--scrape-only` |
|
||||
| README | ✅ | - |
|
||||
| Issues | ✅ | `--no-issues` |
|
||||
| Releases | ✅ | `--no-releases` |
|
||||
| Changelog | ✅ | `--no-changelog` |
|
||||
|
||||
### Control What to Fetch
|
||||
|
||||
```bash
|
||||
# Skip issues (faster)
|
||||
skill-seekers github --repo facebook/react --no-issues
|
||||
|
||||
# Limit issues
|
||||
skill-seekers github --repo facebook/react --max-issues 50
|
||||
|
||||
# Scrape only (no build)
|
||||
skill-seekers github --repo facebook/react --scrape-only
|
||||
|
||||
# Non-interactive (CI/CD)
|
||||
skill-seekers github --repo facebook/react --non-interactive
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## PDF Extraction
|
||||
|
||||
### Basic Usage
|
||||
|
||||
```bash
|
||||
# Direct file
|
||||
skill-seekers create manual.pdf --name product-manual
|
||||
|
||||
# With explicit command
|
||||
skill-seekers pdf --pdf manual.pdf --name docs
|
||||
```
|
||||
|
||||
### OCR for Scanned PDFs
|
||||
|
||||
```bash
|
||||
# Enable OCR
|
||||
skill-seekers pdf --pdf scanned.pdf --enable-ocr
|
||||
```
|
||||
|
||||
**Requirements:**
|
||||
```bash
|
||||
pip install skill-seekers[pdf-ocr]
|
||||
# Also requires: tesseract-ocr (system package)
|
||||
```
|
||||
|
||||
### Password-Protected PDFs
|
||||
|
||||
```bash
|
||||
# In config file
|
||||
{
|
||||
"name": "secure-docs",
|
||||
"pdf_path": "protected.pdf",
|
||||
"password": "secret123"
|
||||
}
|
||||
```
|
||||
|
||||
### Page Range
|
||||
|
||||
```bash
|
||||
# Extract specific pages (via config)
|
||||
{
|
||||
"pdf_path": "manual.pdf",
|
||||
"page_range": [1, 100]
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Local Codebase Analysis
|
||||
|
||||
### Basic Usage
|
||||
|
||||
```bash
|
||||
# Current directory
|
||||
skill-seekers create .
|
||||
|
||||
# Specific directory
|
||||
skill-seekers create ./my-project
|
||||
|
||||
# With explicit command
|
||||
skill-seekers analyze --directory ./my-project
|
||||
```
|
||||
|
||||
### Analysis Presets
|
||||
|
||||
```bash
|
||||
# Quick analysis (1-2 min)
|
||||
skill-seekers analyze --directory ./my-project --preset quick
|
||||
|
||||
# Standard analysis (5-10 min) - default
|
||||
skill-seekers analyze --directory ./my-project --preset standard
|
||||
|
||||
# Comprehensive (20-60 min)
|
||||
skill-seekers analyze --directory ./my-project --preset comprehensive
|
||||
```
|
||||
|
||||
### What Gets Analyzed
|
||||
|
||||
| Feature | Quick | Standard | Comprehensive |
|
||||
|---------|-------|----------|---------------|
|
||||
| Code structure | ✅ | ✅ | ✅ |
|
||||
| API extraction | ✅ | ✅ | ✅ |
|
||||
| Comments | - | ✅ | ✅ |
|
||||
| Patterns | - | ✅ | ✅ |
|
||||
| Test examples | - | - | ✅ |
|
||||
| How-to guides | - | - | ✅ |
|
||||
| Config patterns | - | - | ✅ |
|
||||
|
||||
### Language Filtering
|
||||
|
||||
```bash
|
||||
# Specific languages
|
||||
skill-seekers analyze --directory ./my-project \
|
||||
--languages Python,JavaScript
|
||||
|
||||
# File patterns
|
||||
skill-seekers analyze --directory ./my-project \
|
||||
--file-patterns "*.py,*.js"
|
||||
```
|
||||
|
||||
### Skip Features
|
||||
|
||||
```bash
|
||||
# Skip heavy features
|
||||
skill-seekers analyze --directory ./my-project \
|
||||
--skip-dependency-graph \
|
||||
--skip-patterns \
|
||||
--skip-test-examples
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Common Scraping Patterns
|
||||
|
||||
### Pattern 1: Test First
|
||||
|
||||
```bash
|
||||
# Dry run to preview
|
||||
skill-seekers create <source> --dry-run
|
||||
|
||||
# Small test scrape
|
||||
skill-seekers create <source> --max-pages 10
|
||||
|
||||
# Full scrape
|
||||
skill-seekers create <source>
|
||||
```
|
||||
|
||||
### Pattern 2: Iterative Development
|
||||
|
||||
```bash
|
||||
# Scrape without enhancement (fast)
|
||||
skill-seekers create <source> --enhance-level 0
|
||||
|
||||
# Review output
|
||||
ls output/my-skill/
|
||||
cat output/my-skill/SKILL.md
|
||||
|
||||
# Enhance later
|
||||
skill-seekers enhance output/my-skill/
|
||||
```
|
||||
|
||||
### Pattern 3: Parallel Processing
|
||||
|
||||
```bash
|
||||
# Fast async scraping
|
||||
skill-seekers create <url> --async --workers 5
|
||||
|
||||
# Even faster (be careful with rate limits)
|
||||
skill-seekers create <url> --async --workers 10 --rate-limit 0.2
|
||||
```
|
||||
|
||||
### Pattern 4: Resume Capability
|
||||
|
||||
```bash
|
||||
# Start scraping
|
||||
skill-seekers create <source>
|
||||
# ...interrupted...
|
||||
|
||||
# Resume later
|
||||
skill-seekers resume --list
|
||||
skill-seekers resume <job-id>
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting Scraping
|
||||
|
||||
### "No content extracted"
|
||||
|
||||
**Problem:** Wrong CSS selectors
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
# Find correct selectors
|
||||
curl -s <url> | grep -i 'article\|main\|content'
|
||||
|
||||
# Update config
|
||||
{
|
||||
"selectors": {
|
||||
"main_content": "div.content" // or "article", "main", etc.
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### "Rate limit exceeded"
|
||||
|
||||
**Problem:** Too many requests
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
# Slow down
|
||||
skill-seekers create <url> --rate-limit 2.0
|
||||
|
||||
# Or use GitHub token for GitHub repos
|
||||
export GITHUB_TOKEN=ghp_...
|
||||
```
|
||||
|
||||
### "Too many pages"
|
||||
|
||||
**Problem:** Site is larger than expected
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
# Estimate first
|
||||
skill-seekers estimate configs/my-config.json
|
||||
|
||||
# Limit pages
|
||||
skill-seekers create <url> --max-pages 100
|
||||
|
||||
# Adjust URL patterns
|
||||
{
|
||||
"url_patterns": {
|
||||
"exclude": ["/blog/", "/archive/", "/search"]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### "Memory error"
|
||||
|
||||
**Problem:** Site too large for memory
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
# Use streaming mode
|
||||
skill-seekers create <url> --streaming
|
||||
|
||||
# Or smaller chunks
|
||||
skill-seekers create <url> --chunk-size 500
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Performance Tips
|
||||
|
||||
| Tip | Command | Impact |
|
||||
|-----|---------|--------|
|
||||
| Use presets | `--config react` | Faster setup |
|
||||
| Async mode | `--async --workers 5` | 3-5x faster |
|
||||
| Skip enhancement | `--enhance-level 0` | Skip 60 sec |
|
||||
| Use cache | `--skip-scrape` | Instant rebuild |
|
||||
| Resume | `--resume` | Continue interrupted |
|
||||
|
||||
---
|
||||
|
||||
## Next Steps
|
||||
|
||||
- [Enhancement Guide](03-enhancement.md) - Improve skill quality
|
||||
- [Packaging Guide](04-packaging.md) - Export to platforms
|
||||
- [Config Format](../reference/CONFIG_FORMAT.md) - Advanced configuration
|
||||
432
docs/user-guide/03-enhancement.md
Normal file
432
docs/user-guide/03-enhancement.md
Normal file
@@ -0,0 +1,432 @@
|
||||
# Enhancement Guide
|
||||
|
||||
> **Skill Seekers v3.1.0**
|
||||
> **AI-powered quality improvement for skills**
|
||||
|
||||
---
|
||||
|
||||
## What is Enhancement?
|
||||
|
||||
Enhancement uses AI to improve the quality of generated SKILL.md files:
|
||||
|
||||
```
|
||||
Basic SKILL.md ──▶ AI Enhancer ──▶ Enhanced SKILL.md
|
||||
(100 lines) (60 sec) (400+ lines)
|
||||
↓ ↓
|
||||
Sparse Comprehensive
|
||||
examples with patterns,
|
||||
navigation, depth
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Enhancement Levels
|
||||
|
||||
Choose how much enhancement to apply:
|
||||
|
||||
| Level | What Happens | Time | Cost |
|
||||
|-------|--------------|------|------|
|
||||
| **0** | No enhancement | 0 sec | Free |
|
||||
| **1** | SKILL.md only | ~30 sec | Low |
|
||||
| **2** | + architecture/config | ~60 sec | Medium |
|
||||
| **3** | Full enhancement | ~2 min | Higher |
|
||||
|
||||
**Default:** Level 2 (recommended balance)
|
||||
|
||||
---
|
||||
|
||||
## Enhancement Modes
|
||||
|
||||
### API Mode (Default if key available)
|
||||
|
||||
Uses Claude API for fast enhancement.
|
||||
|
||||
**Requirements:**
|
||||
```bash
|
||||
export ANTHROPIC_API_KEY=sk-ant-...
|
||||
```
|
||||
|
||||
**Usage:**
|
||||
```bash
|
||||
# Auto-detects API mode
|
||||
skill-seekers create <source>
|
||||
|
||||
# Explicit
|
||||
skill-seekers enhance output/my-skill/ --agent api
|
||||
```
|
||||
|
||||
**Pros:**
|
||||
- Fast (~60 seconds)
|
||||
- No local setup needed
|
||||
|
||||
**Cons:**
|
||||
- Costs ~$0.10-0.30 per skill
|
||||
- Requires API key
|
||||
|
||||
---
|
||||
|
||||
### LOCAL Mode (Default if no key)
|
||||
|
||||
Uses Claude Code (free with Max plan).
|
||||
|
||||
**Requirements:**
|
||||
- Claude Code installed
|
||||
- Claude Code Max subscription
|
||||
|
||||
**Usage:**
|
||||
```bash
|
||||
# Auto-detects LOCAL mode (no API key)
|
||||
skill-seekers create <source>
|
||||
|
||||
# Explicit
|
||||
skill-seekers enhance output/my-skill/ --agent local
|
||||
```
|
||||
|
||||
**Pros:**
|
||||
- Free (with Claude Code Max)
|
||||
- Better quality (full context)
|
||||
|
||||
**Cons:**
|
||||
- Requires Claude Code
|
||||
- Slightly slower (~60-120 sec)
|
||||
|
||||
---
|
||||
|
||||
## How to Enhance
|
||||
|
||||
### During Creation
|
||||
|
||||
```bash
|
||||
# Default enhancement (level 2)
|
||||
skill-seekers create <source>
|
||||
|
||||
# No enhancement (fastest)
|
||||
skill-seekers create <source> --enhance-level 0
|
||||
|
||||
# Maximum enhancement
|
||||
skill-seekers create <source> --enhance-level 3
|
||||
```
|
||||
|
||||
### After Creation
|
||||
|
||||
```bash
|
||||
# Enhance existing skill
|
||||
skill-seekers enhance output/my-skill/
|
||||
|
||||
# With specific agent
|
||||
skill-seekers enhance output/my-skill/ --agent local
|
||||
|
||||
# With timeout
|
||||
skill-seekers enhance output/my-skill/ --timeout 1200
|
||||
```
|
||||
|
||||
### Background Mode
|
||||
|
||||
```bash
|
||||
# Run in background
|
||||
skill-seekers enhance output/my-skill/ --background
|
||||
|
||||
# Check status
|
||||
skill-seekers enhance-status output/my-skill/
|
||||
|
||||
# Watch in real-time
|
||||
skill-seekers enhance-status output/my-skill/ --watch
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Enhancement Workflows
|
||||
|
||||
Apply specialized AI analysis with preset workflows.
|
||||
|
||||
### Built-in Presets
|
||||
|
||||
| Preset | Stages | Focus |
|
||||
|--------|--------|-------|
|
||||
| `default` | 2 | General improvement |
|
||||
| `minimal` | 1 | Light touch-up |
|
||||
| `security-focus` | 4 | Security analysis |
|
||||
| `architecture-comprehensive` | 7 | Deep architecture |
|
||||
| `api-documentation` | 3 | API docs focus |
|
||||
|
||||
### Using Workflows
|
||||
|
||||
```bash
|
||||
# Apply workflow
|
||||
skill-seekers create <source> --enhance-workflow security-focus
|
||||
|
||||
# Chain multiple workflows
|
||||
skill-seekers create <source> \
|
||||
--enhance-workflow security-focus \
|
||||
--enhance-workflow api-documentation
|
||||
|
||||
# List available
|
||||
skill-seekers workflows list
|
||||
|
||||
# Show workflow content
|
||||
skill-seekers workflows show security-focus
|
||||
```
|
||||
|
||||
### Custom Workflows
|
||||
|
||||
Create your own YAML workflow:
|
||||
|
||||
```yaml
|
||||
# my-workflow.yaml
|
||||
name: my-custom
|
||||
stages:
|
||||
- name: overview
|
||||
prompt: "Add comprehensive overview section"
|
||||
- name: examples
|
||||
prompt: "Add practical code examples"
|
||||
```
|
||||
|
||||
```bash
|
||||
# Add workflow
|
||||
skill-seekers workflows add my-workflow.yaml
|
||||
|
||||
# Use it
|
||||
skill-seekers create <source> --enhance-workflow my-custom
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## What Enhancement Adds
|
||||
|
||||
### Level 1: SKILL.md Improvement
|
||||
|
||||
- Better structure and organization
|
||||
- Improved descriptions
|
||||
- Fixed formatting
|
||||
- Added navigation
|
||||
|
||||
### Level 2: Architecture & Config (Default)
|
||||
|
||||
Everything in Level 1, plus:
|
||||
|
||||
- Architecture overview
|
||||
- Configuration examples
|
||||
- Pattern documentation
|
||||
- Best practices
|
||||
|
||||
### Level 3: Full Enhancement
|
||||
|
||||
Everything in Level 2, plus:
|
||||
|
||||
- Deep code examples
|
||||
- Common pitfalls
|
||||
- Performance tips
|
||||
- Integration guides
|
||||
|
||||
---
|
||||
|
||||
## Enhancement Workflow Details
|
||||
|
||||
### Security-Focus Workflow
|
||||
|
||||
4 stages:
|
||||
1. **Security Overview** - Identify security features
|
||||
2. **Vulnerability Analysis** - Common issues
|
||||
3. **Best Practices** - Secure coding patterns
|
||||
4. **Compliance** - Security standards
|
||||
|
||||
### Architecture-Comprehensive Workflow
|
||||
|
||||
7 stages:
|
||||
1. **System Overview** - High-level architecture
|
||||
2. **Component Analysis** - Key components
|
||||
3. **Data Flow** - How data moves
|
||||
4. **Integration Points** - External connections
|
||||
5. **Scalability** - Performance considerations
|
||||
6. **Deployment** - Infrastructure
|
||||
7. **Maintenance** - Operational concerns
|
||||
|
||||
### API-Documentation Workflow
|
||||
|
||||
3 stages:
|
||||
1. **Endpoint Catalog** - All API endpoints
|
||||
2. **Request/Response** - Detailed examples
|
||||
3. **Error Handling** - Common errors
|
||||
|
||||
---
|
||||
|
||||
## Monitoring Enhancement
|
||||
|
||||
### Check Status
|
||||
|
||||
```bash
|
||||
# Current status
|
||||
skill-seekers enhance-status output/my-skill/
|
||||
|
||||
# JSON output (for scripting)
|
||||
skill-seekers enhance-status output/my-skill/ --json
|
||||
|
||||
# Watch mode
|
||||
skill-seekers enhance-status output/my-skill/ --watch --interval 10
|
||||
```
|
||||
|
||||
### Process Status Values
|
||||
|
||||
| Status | Meaning |
|
||||
|--------|---------|
|
||||
| `running` | Enhancement in progress |
|
||||
| `completed` | Successfully finished |
|
||||
| `failed` | Error occurred |
|
||||
| `pending` | Waiting to start |
|
||||
|
||||
---
|
||||
|
||||
## When to Skip Enhancement
|
||||
|
||||
Skip enhancement when:
|
||||
|
||||
- **Testing:** Quick iteration during development
|
||||
- **Large batches:** Process many skills, enhance best ones later
|
||||
- **Custom processing:** You have your own enhancement pipeline
|
||||
- **Time critical:** Need results immediately
|
||||
|
||||
```bash
|
||||
# Skip during creation
|
||||
skill-seekers create <source> --enhance-level 0
|
||||
|
||||
# Enhance best ones later
|
||||
skill-seekers enhance output/best-skill/
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Enhancement Best Practices
|
||||
|
||||
### 1. Use Level 2 for Most Cases
|
||||
|
||||
```bash
|
||||
# Default is usually perfect
|
||||
skill-seekers create <source>
|
||||
```
|
||||
|
||||
### 2. Apply Domain-Specific Workflows
|
||||
|
||||
```bash
|
||||
# Security review
|
||||
skill-seekers create <source> --enhance-workflow security-focus
|
||||
|
||||
# API focus
|
||||
skill-seekers create <source> --enhance-workflow api-documentation
|
||||
```
|
||||
|
||||
### 3. Chain for Comprehensive Analysis
|
||||
|
||||
```bash
|
||||
# Multiple perspectives
|
||||
skill-seekers create <source> \
|
||||
--enhance-workflow security-focus \
|
||||
--enhance-workflow architecture-comprehensive
|
||||
```
|
||||
|
||||
### 4. Use LOCAL Mode for Quality
|
||||
|
||||
```bash
|
||||
# Better results with Claude Code
|
||||
export ANTHROPIC_API_KEY="" # Unset to force LOCAL
|
||||
skill-seekers enhance output/my-skill/
|
||||
```
|
||||
|
||||
### 5. Enhance Iteratively
|
||||
|
||||
```bash
|
||||
# Create without enhancement
|
||||
skill-seekers create <source> --enhance-level 0
|
||||
|
||||
# Review and enhance
|
||||
skill-seekers enhance output/my-skill/
|
||||
# Review again...
|
||||
skill-seekers enhance output/my-skill/ # Run again for more polish
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### "Enhancement failed: No API key"
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
# Set API key
|
||||
export ANTHROPIC_API_KEY=sk-ant-...
|
||||
|
||||
# Or use LOCAL mode
|
||||
skill-seekers enhance output/my-skill/ --agent local
|
||||
```
|
||||
|
||||
### "Enhancement timeout"
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
# Increase timeout
|
||||
skill-seekers enhance output/my-skill/ --timeout 1200
|
||||
|
||||
# Or use background mode
|
||||
skill-seekers enhance output/my-skill/ --background
|
||||
```
|
||||
|
||||
### "Claude Code not found" (LOCAL mode)
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
# Install Claude Code
|
||||
# See: https://claude.ai/code
|
||||
|
||||
# Or switch to API mode
|
||||
export ANTHROPIC_API_KEY=sk-ant-...
|
||||
skill-seekers enhance output/my-skill/ --agent api
|
||||
```
|
||||
|
||||
### "Workflow not found"
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
# List available workflows
|
||||
skill-seekers workflows list
|
||||
|
||||
# Check spelling
|
||||
skill-seekers create <source> --enhance-workflow security-focus
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Cost Estimation
|
||||
|
||||
### API Mode Costs
|
||||
|
||||
| Skill Size | Level 1 | Level 2 | Level 3 |
|
||||
|------------|---------|---------|---------|
|
||||
| Small (< 50 pages) | $0.02 | $0.05 | $0.10 |
|
||||
| Medium (50-200 pages) | $0.05 | $0.10 | $0.20 |
|
||||
| Large (200-500 pages) | $0.10 | $0.20 | $0.40 |
|
||||
|
||||
*Costs are approximate and depend on actual content.*
|
||||
|
||||
### LOCAL Mode Costs
|
||||
|
||||
Free with Claude Code Max subscription (~$20/month).
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
| Approach | When to Use |
|
||||
|----------|-------------|
|
||||
| **Level 0** | Testing, batch processing |
|
||||
| **Level 2 (default)** | Most use cases |
|
||||
| **Level 3** | Maximum quality needed |
|
||||
| **API Mode** | Speed, no Claude Code |
|
||||
| **LOCAL Mode** | Quality, free with Max |
|
||||
| **Workflows** | Domain-specific needs |
|
||||
|
||||
---
|
||||
|
||||
## Next Steps
|
||||
|
||||
- [Workflows Guide](05-workflows.md) - Custom workflow creation
|
||||
- [Packaging Guide](04-packaging.md) - Export enhanced skills
|
||||
- [MCP Reference](../reference/MCP_REFERENCE.md) - Enhancement via MCP
|
||||
501
docs/user-guide/04-packaging.md
Normal file
501
docs/user-guide/04-packaging.md
Normal file
@@ -0,0 +1,501 @@
|
||||
# Packaging Guide
|
||||
|
||||
> **Skill Seekers v3.1.0**
|
||||
> **Export skills to AI platforms and vector databases**
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
Packaging converts your skill directory into a platform-specific format:
|
||||
|
||||
```
|
||||
output/my-skill/ ──▶ Packager ──▶ output/my-skill-{platform}.{format}
|
||||
↓ ↓
|
||||
(SKILL.md + Platform-specific (ZIP, tar.gz,
|
||||
references) formatting directories,
|
||||
FAISS index)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Supported Platforms
|
||||
|
||||
| Platform | Format | Extension | Best For |
|
||||
|----------|--------|-----------|----------|
|
||||
| **Claude AI** | ZIP + YAML | `.zip` | Claude Code, Claude API |
|
||||
| **Google Gemini** | tar.gz | `.tar.gz` | Gemini skills |
|
||||
| **OpenAI ChatGPT** | ZIP + Vector | `.zip` | Custom GPTs |
|
||||
| **LangChain** | Documents | directory | RAG pipelines |
|
||||
| **LlamaIndex** | TextNodes | directory | Query engines |
|
||||
| **Haystack** | Documents | directory | Enterprise RAG |
|
||||
| **Pinecone** | Markdown | `.zip` | Vector upsert |
|
||||
| **ChromaDB** | Collection | `.zip` | Local vector DB |
|
||||
| **Weaviate** | Objects | `.zip` | Vector database |
|
||||
| **Qdrant** | Points | `.zip` | Vector database |
|
||||
| **FAISS** | Index | `.faiss` | Local similarity |
|
||||
| **Markdown** | ZIP | `.zip` | Universal export |
|
||||
| **Cursor** | .cursorrules | file | IDE AI context |
|
||||
| **Windsurf** | .windsurfrules | file | IDE AI context |
|
||||
| **Cline** | .clinerules | file | VS Code AI |
|
||||
|
||||
---
|
||||
|
||||
## Basic Packaging
|
||||
|
||||
### Package for Claude (Default)
|
||||
|
||||
```bash
|
||||
# Default packaging
|
||||
skill-seekers package output/my-skill/
|
||||
|
||||
# Explicit target
|
||||
skill-seekers package output/my-skill/ --target claude
|
||||
|
||||
# Output: output/my-skill-claude.zip
|
||||
```
|
||||
|
||||
### Package for Other Platforms
|
||||
|
||||
```bash
|
||||
# Google Gemini
|
||||
skill-seekers package output/my-skill/ --target gemini
|
||||
# Output: output/my-skill-gemini.tar.gz
|
||||
|
||||
# OpenAI
|
||||
skill-seekers package output/my-skill/ --target openai
|
||||
# Output: output/my-skill-openai.zip
|
||||
|
||||
# LangChain
|
||||
skill-seekers package output/my-skill/ --target langchain
|
||||
# Output: output/my-skill-langchain/ directory
|
||||
|
||||
# ChromaDB
|
||||
skill-seekers package output/my-skill/ --target chroma
|
||||
# Output: output/my-skill-chroma.zip
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Multi-Platform Packaging
|
||||
|
||||
### Package for All Platforms
|
||||
|
||||
```bash
|
||||
# Create skill once
|
||||
skill-seekers create <source>
|
||||
|
||||
# Package for multiple platforms
|
||||
for platform in claude gemini openai langchain; do
|
||||
echo "Packaging for $platform..."
|
||||
skill-seekers package output/my-skill/ --target $platform
|
||||
done
|
||||
|
||||
# Results:
|
||||
# output/my-skill-claude.zip
|
||||
# output/my-skill-gemini.tar.gz
|
||||
# output/my-skill-openai.zip
|
||||
# output/my-skill-langchain/
|
||||
```
|
||||
|
||||
### Batch Packaging Script
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
SKILL_DIR="output/my-skill"
|
||||
PLATFORMS="claude gemini openai langchain llama-index chroma"
|
||||
|
||||
for platform in $PLATFORMS; do
|
||||
echo "▶️ Packaging for $platform..."
|
||||
skill-seekers package "$SKILL_DIR" --target "$platform"
|
||||
|
||||
if [ $? -eq 0 ]; then
|
||||
echo "✅ $platform done"
|
||||
else
|
||||
echo "❌ $platform failed"
|
||||
fi
|
||||
done
|
||||
|
||||
echo "🎉 All platforms packaged!"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Packaging Options
|
||||
|
||||
### Skip Quality Check
|
||||
|
||||
```bash
|
||||
# Skip validation (faster)
|
||||
skill-seekers package output/my-skill/ --skip-quality-check
|
||||
```
|
||||
|
||||
### Don't Open Output Folder
|
||||
|
||||
```bash
|
||||
# Prevent opening folder after packaging
|
||||
skill-seekers package output/my-skill/ --no-open
|
||||
```
|
||||
|
||||
### Auto-Upload After Packaging
|
||||
|
||||
```bash
|
||||
# Package and upload
|
||||
export ANTHROPIC_API_KEY=sk-ant-...
|
||||
skill-seekers package output/my-skill/ --target claude --upload
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Streaming Mode
|
||||
|
||||
For very large skills, use streaming to reduce memory usage:
|
||||
|
||||
```bash
|
||||
# Enable streaming
|
||||
skill-seekers package output/large-skill/ --streaming
|
||||
|
||||
# Custom chunk size
|
||||
skill-seekers package output/large-skill/ \
|
||||
--streaming \
|
||||
--chunk-size 2000 \
|
||||
--chunk-overlap 100
|
||||
```
|
||||
|
||||
**When to use:**
|
||||
- Skills > 500 pages
|
||||
- Limited RAM (< 8GB)
|
||||
- Batch processing many skills
|
||||
|
||||
---
|
||||
|
||||
## RAG Chunking
|
||||
|
||||
Optimize for Retrieval-Augmented Generation:
|
||||
|
||||
```bash
|
||||
# Enable semantic chunking
|
||||
skill-seekers package output/my-skill/ \
|
||||
--target langchain \
|
||||
--chunk \
|
||||
--chunk-tokens 512
|
||||
|
||||
# Custom chunk size
|
||||
skill-seekers package output/my-skill/ \
|
||||
--target chroma \
|
||||
--chunk-tokens 256 \
|
||||
--chunk-overlap 50
|
||||
```
|
||||
|
||||
**Chunking Options:**
|
||||
|
||||
| Option | Default | Description |
|
||||
|--------|---------|-------------|
|
||||
| `--chunk` | auto | Enable chunking |
|
||||
| `--chunk-tokens` | 512 | Tokens per chunk |
|
||||
| `--chunk-overlap` | 50 | Overlap between chunks |
|
||||
| `--no-preserve-code` | - | Allow splitting code blocks |
|
||||
|
||||
---
|
||||
|
||||
## Platform-Specific Details
|
||||
|
||||
### Claude AI
|
||||
|
||||
```bash
|
||||
skill-seekers package output/my-skill/ --target claude
|
||||
```
|
||||
|
||||
**Upload:**
|
||||
```bash
|
||||
# Auto-upload
|
||||
skill-seekers package output/my-skill/ --target claude --upload
|
||||
|
||||
# Manual upload
|
||||
skill-seekers upload output/my-skill-claude.zip --target claude
|
||||
```
|
||||
|
||||
**Format:**
|
||||
- ZIP archive
|
||||
- Contains SKILL.md + references/
|
||||
- Includes YAML manifest
|
||||
|
||||
---
|
||||
|
||||
### Google Gemini
|
||||
|
||||
```bash
|
||||
skill-seekers package output/my-skill/ --target gemini
|
||||
```
|
||||
|
||||
**Upload:**
|
||||
```bash
|
||||
export GOOGLE_API_KEY=AIza...
|
||||
skill-seekers upload output/my-skill-gemini.tar.gz --target gemini
|
||||
```
|
||||
|
||||
**Format:**
|
||||
- tar.gz archive
|
||||
- Optimized for Gemini's format
|
||||
|
||||
---
|
||||
|
||||
### OpenAI ChatGPT
|
||||
|
||||
```bash
|
||||
skill-seekers package output/my-skill/ --target openai
|
||||
```
|
||||
|
||||
**Upload:**
|
||||
```bash
|
||||
export OPENAI_API_KEY=sk-...
|
||||
skill-seekers upload output/my-skill-openai.zip --target openai
|
||||
```
|
||||
|
||||
**Format:**
|
||||
- ZIP with vector embeddings
|
||||
- Ready for Assistants API
|
||||
|
||||
---
|
||||
|
||||
### LangChain
|
||||
|
||||
```bash
|
||||
skill-seekers package output/my-skill/ --target langchain
|
||||
```
|
||||
|
||||
**Usage:**
|
||||
```python
|
||||
from langchain.document_loaders import DirectoryLoader
|
||||
|
||||
loader = DirectoryLoader("output/my-skill-langchain/")
|
||||
docs = loader.load()
|
||||
|
||||
# Use in RAG pipeline
|
||||
```
|
||||
|
||||
**Format:**
|
||||
- Directory of Document objects
|
||||
- JSON metadata
|
||||
|
||||
---
|
||||
|
||||
### ChromaDB
|
||||
|
||||
```bash
|
||||
skill-seekers package output/my-skill/ --target chroma
|
||||
```
|
||||
|
||||
**Upload:**
|
||||
```bash
|
||||
# Local ChromaDB
|
||||
skill-seekers upload output/my-skill-chroma.zip --target chroma
|
||||
|
||||
# With custom URL
|
||||
skill-seekers upload output/my-skill-chroma.zip \
|
||||
--target chroma \
|
||||
--chroma-url http://localhost:8000
|
||||
```
|
||||
|
||||
**Usage:**
|
||||
```python
|
||||
import chromadb
|
||||
|
||||
client = chromadb.HttpClient(host="localhost", port=8000)
|
||||
collection = client.get_collection("my-skill")
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Weaviate
|
||||
|
||||
```bash
|
||||
skill-seekers package output/my-skill/ --target weaviate
|
||||
```
|
||||
|
||||
**Upload:**
|
||||
```bash
|
||||
# Local Weaviate
|
||||
skill-seekers upload output/my-skill-weaviate.zip --target weaviate
|
||||
|
||||
# Weaviate Cloud
|
||||
skill-seekers upload output/my-skill-weaviate.zip \
|
||||
--target weaviate \
|
||||
--use-cloud \
|
||||
--cluster-url https://xxx.weaviate.network
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Cursor IDE
|
||||
|
||||
```bash
|
||||
# Package (actually creates .cursorrules file)
|
||||
skill-seekers package output/my-skill/ --target cursor
|
||||
|
||||
# Or install directly
|
||||
skill-seekers install-agent output/my-skill/ --agent cursor
|
||||
```
|
||||
|
||||
**Result:** `.cursorrules` file in your project root.
|
||||
|
||||
---
|
||||
|
||||
### Windsurf IDE
|
||||
|
||||
```bash
|
||||
skill-seekers install-agent output/my-skill/ --agent windsurf
|
||||
```
|
||||
|
||||
**Result:** `.windsurfrules` file in your project root.
|
||||
|
||||
---
|
||||
|
||||
## Quality Check
|
||||
|
||||
Before packaging, skills are validated:
|
||||
|
||||
```bash
|
||||
# Check quality
|
||||
skill-seekers quality output/my-skill/
|
||||
|
||||
# Detailed report
|
||||
skill-seekers quality output/my-skill/ --report
|
||||
|
||||
# Set minimum threshold
|
||||
skill-seekers quality output/my-skill/ --threshold 7.0
|
||||
```
|
||||
|
||||
**Quality Metrics:**
|
||||
- SKILL.md completeness
|
||||
- Code example coverage
|
||||
- Navigation structure
|
||||
- Reference file organization
|
||||
|
||||
---
|
||||
|
||||
## Output Structure
|
||||
|
||||
### After Packaging
|
||||
|
||||
```
|
||||
output/
|
||||
├── my-skill/ # Source skill
|
||||
│ ├── SKILL.md
|
||||
│ └── references/
|
||||
│
|
||||
├── my-skill-claude.zip # Claude package
|
||||
├── my-skill-gemini.tar.gz # Gemini package
|
||||
├── my-skill-openai.zip # OpenAI package
|
||||
├── my-skill-langchain/ # LangChain directory
|
||||
├── my-skill-chroma.zip # ChromaDB package
|
||||
└── my-skill-weaviate.zip # Weaviate package
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### "Package validation failed"
|
||||
|
||||
**Problem:** SKILL.md is missing or malformed
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
# Check skill structure
|
||||
ls output/my-skill/
|
||||
|
||||
# Rebuild if needed
|
||||
skill-seekers create --config my-config --skip-scrape
|
||||
|
||||
# Or recreate
|
||||
skill-seekers create <source>
|
||||
```
|
||||
|
||||
### "Target platform not supported"
|
||||
|
||||
**Problem:** Typo in target name
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
# Check available targets
|
||||
skill-seekers package --help
|
||||
|
||||
# Common targets: claude, gemini, openai, langchain, chroma, weaviate
|
||||
```
|
||||
|
||||
### "Upload failed"
|
||||
|
||||
**Problem:** Missing API key
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
# Set API key
|
||||
export ANTHROPIC_API_KEY=sk-ant-...
|
||||
export GOOGLE_API_KEY=AIza...
|
||||
export OPENAI_API_KEY=sk-...
|
||||
|
||||
# Try again
|
||||
skill-seekers upload output/my-skill-claude.zip --target claude
|
||||
```
|
||||
|
||||
### "Out of memory"
|
||||
|
||||
**Problem:** Skill too large for memory
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
# Use streaming mode
|
||||
skill-seekers package output/my-skill/ --streaming
|
||||
|
||||
# Smaller chunks
|
||||
skill-seekers package output/my-skill/ --streaming --chunk-size 1000
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Best Practices
|
||||
|
||||
### 1. Package Once, Use Everywhere
|
||||
|
||||
```bash
|
||||
# Create once
|
||||
skill-seekers create <source>
|
||||
|
||||
# Package for all needed platforms
|
||||
for platform in claude gemini langchain; do
|
||||
skill-seekers package output/my-skill/ --target $platform
|
||||
done
|
||||
```
|
||||
|
||||
### 2. Check Quality Before Packaging
|
||||
|
||||
```bash
|
||||
# Validate first
|
||||
skill-seekers quality output/my-skill/ --threshold 6.0
|
||||
|
||||
# Then package
|
||||
skill-seekers package output/my-skill/
|
||||
```
|
||||
|
||||
### 3. Use Streaming for Large Skills
|
||||
|
||||
```bash
|
||||
# Automatically detected, but can force
|
||||
skill-seekers package output/large-skill/ --streaming
|
||||
```
|
||||
|
||||
### 4. Keep Original Skill Directory
|
||||
|
||||
Don't delete `output/my-skill/` after packaging - you might want to:
|
||||
- Re-package for other platforms
|
||||
- Apply different workflows
|
||||
- Update and re-enhance
|
||||
|
||||
---
|
||||
|
||||
## Next Steps
|
||||
|
||||
- [Workflows Guide](05-workflows.md) - Apply workflows before packaging
|
||||
- [MCP Reference](../reference/MCP_REFERENCE.md) - Package via MCP
|
||||
- [Vector DB Integrations](../integrations/) - Platform-specific guides
|
||||
621
docs/user-guide/05-workflows.md
Normal file
621
docs/user-guide/05-workflows.md
Normal file
@@ -0,0 +1,621 @@
|
||||
# Workflows Guide
|
||||
|
||||
> **Skill Seekers v3.1.0**
|
||||
> **Enhancement workflow presets for specialized analysis**
|
||||
|
||||
---
|
||||
|
||||
## What are Workflows?
|
||||
|
||||
Workflows are **multi-stage AI enhancement pipelines** that apply specialized analysis to your skills:
|
||||
|
||||
```
|
||||
Basic Skill ──▶ Workflow: Security-Focus ──▶ Security-Enhanced Skill
|
||||
Stage 1: Overview
|
||||
Stage 2: Vulnerability Analysis
|
||||
Stage 3: Best Practices
|
||||
Stage 4: Compliance
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Built-in Presets
|
||||
|
||||
Skill Seekers includes 5 built-in workflow presets:
|
||||
|
||||
| Preset | Stages | Best For |
|
||||
|--------|--------|----------|
|
||||
| `default` | 2 | General improvement |
|
||||
| `minimal` | 1 | Light touch-up |
|
||||
| `security-focus` | 4 | Security analysis |
|
||||
| `architecture-comprehensive` | 7 | Deep architecture review |
|
||||
| `api-documentation` | 3 | API documentation focus |
|
||||
|
||||
---
|
||||
|
||||
## Using Workflows
|
||||
|
||||
### List Available Workflows
|
||||
|
||||
```bash
|
||||
skill-seekers workflows list
|
||||
```
|
||||
|
||||
**Output:**
|
||||
```
|
||||
Bundled Workflows:
|
||||
- default (built-in)
|
||||
- minimal (built-in)
|
||||
- security-focus (built-in)
|
||||
- architecture-comprehensive (built-in)
|
||||
- api-documentation (built-in)
|
||||
|
||||
User Workflows:
|
||||
- my-custom (user)
|
||||
```
|
||||
|
||||
### Apply a Workflow
|
||||
|
||||
```bash
|
||||
# During skill creation
|
||||
skill-seekers create <source> --enhance-workflow security-focus
|
||||
|
||||
# Multiple workflows (chained)
|
||||
skill-seekers create <source> \
|
||||
--enhance-workflow security-focus \
|
||||
--enhance-workflow api-documentation
|
||||
```
|
||||
|
||||
### Show Workflow Content
|
||||
|
||||
```bash
|
||||
skill-seekers workflows show security-focus
|
||||
```
|
||||
|
||||
**Output:**
|
||||
```yaml
|
||||
name: security-focus
|
||||
description: Security analysis workflow
|
||||
stages:
|
||||
- name: security-overview
|
||||
prompt: Analyze security features and mechanisms...
|
||||
|
||||
- name: vulnerability-analysis
|
||||
prompt: Identify common vulnerabilities...
|
||||
|
||||
- name: best-practices
|
||||
prompt: Document security best practices...
|
||||
|
||||
- name: compliance
|
||||
prompt: Map to security standards...
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Workflow Presets Explained
|
||||
|
||||
### Default Workflow
|
||||
|
||||
**Stages:** 2
|
||||
**Purpose:** General improvement
|
||||
|
||||
```yaml
|
||||
stages:
|
||||
- name: structure
|
||||
prompt: Improve overall structure and organization
|
||||
- name: content
|
||||
prompt: Enhance content quality and examples
|
||||
```
|
||||
|
||||
**Use when:** You want standard enhancement without specific focus.
|
||||
|
||||
---
|
||||
|
||||
### Minimal Workflow
|
||||
|
||||
**Stages:** 1
|
||||
**Purpose:** Light touch-up
|
||||
|
||||
```yaml
|
||||
stages:
|
||||
- name: cleanup
|
||||
prompt: Basic formatting and cleanup
|
||||
```
|
||||
|
||||
**Use when:** You need quick, minimal enhancement.
|
||||
|
||||
---
|
||||
|
||||
### Security-Focus Workflow
|
||||
|
||||
**Stages:** 4
|
||||
**Purpose:** Security analysis and recommendations
|
||||
|
||||
```yaml
|
||||
stages:
|
||||
- name: security-overview
|
||||
prompt: Identify and document security features...
|
||||
|
||||
- name: vulnerability-analysis
|
||||
prompt: Analyze potential vulnerabilities...
|
||||
|
||||
- name: security-best-practices
|
||||
prompt: Document security best practices...
|
||||
|
||||
- name: compliance-mapping
|
||||
prompt: Map to OWASP, CWE, and other standards...
|
||||
```
|
||||
|
||||
**Use for:**
|
||||
- Security libraries
|
||||
- Authentication systems
|
||||
- API frameworks
|
||||
- Any code handling sensitive data
|
||||
|
||||
**Example:**
|
||||
```bash
|
||||
skill-seekers create oauth2-server --enhance-workflow security-focus
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Architecture-Comprehensive Workflow
|
||||
|
||||
**Stages:** 7
|
||||
**Purpose:** Deep architectural analysis
|
||||
|
||||
```yaml
|
||||
stages:
|
||||
- name: system-overview
|
||||
prompt: Document high-level architecture...
|
||||
|
||||
- name: component-analysis
|
||||
prompt: Analyze key components...
|
||||
|
||||
- name: data-flow
|
||||
prompt: Document data flow patterns...
|
||||
|
||||
- name: integration-points
|
||||
prompt: Identify external integrations...
|
||||
|
||||
- name: scalability
|
||||
prompt: Document scalability considerations...
|
||||
|
||||
- name: deployment
|
||||
prompt: Document deployment patterns...
|
||||
|
||||
- name: maintenance
|
||||
prompt: Document operational concerns...
|
||||
```
|
||||
|
||||
**Use for:**
|
||||
- Large frameworks
|
||||
- Distributed systems
|
||||
- Microservices
|
||||
- Enterprise platforms
|
||||
|
||||
**Example:**
|
||||
```bash
|
||||
skill-seekers create kubernetes/kubernetes \
|
||||
--enhance-workflow architecture-comprehensive
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### API-Documentation Workflow
|
||||
|
||||
**Stages:** 3
|
||||
**Purpose:** API-focused enhancement
|
||||
|
||||
```yaml
|
||||
stages:
|
||||
- name: endpoint-catalog
|
||||
prompt: Catalog all API endpoints...
|
||||
|
||||
- name: request-response
|
||||
prompt: Document request/response formats...
|
||||
|
||||
- name: error-handling
|
||||
prompt: Document error codes and handling...
|
||||
```
|
||||
|
||||
**Use for:**
|
||||
- REST APIs
|
||||
- GraphQL services
|
||||
- SDKs
|
||||
- Library documentation
|
||||
|
||||
**Example:**
|
||||
```bash
|
||||
skill-seekers create https://api.example.com/docs \
|
||||
--enhance-workflow api-documentation
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Chaining Multiple Workflows
|
||||
|
||||
Apply multiple workflows sequentially:
|
||||
|
||||
```bash
|
||||
skill-seekers create <source> \
|
||||
--enhance-workflow security-focus \
|
||||
--enhance-workflow api-documentation
|
||||
```
|
||||
|
||||
**Execution order:**
|
||||
1. Run `security-focus` workflow
|
||||
2. Run `api-documentation` workflow on results
|
||||
3. Final skill has both security and API focus
|
||||
|
||||
**Use case:** API with security considerations
|
||||
|
||||
---
|
||||
|
||||
## Custom Workflows
|
||||
|
||||
### Create Custom Workflow
|
||||
|
||||
Create a YAML file:
|
||||
|
||||
```yaml
|
||||
# my-workflow.yaml
|
||||
name: performance-focus
|
||||
description: Performance optimization workflow
|
||||
|
||||
variables:
|
||||
target_latency: "100ms"
|
||||
target_throughput: "1000 req/s"
|
||||
|
||||
stages:
|
||||
- name: performance-overview
|
||||
type: builtin
|
||||
target: skill_md
|
||||
prompt: |
|
||||
Analyze performance characteristics of this framework.
|
||||
Focus on:
|
||||
- Benchmark results
|
||||
- Optimization opportunities
|
||||
- Scalability limits
|
||||
|
||||
- name: optimization-guide
|
||||
type: custom
|
||||
uses_history: true
|
||||
prompt: |
|
||||
Based on the previous analysis, create an optimization guide.
|
||||
Target latency: {target_latency}
|
||||
Target throughput: {target_throughput}
|
||||
|
||||
Previous results: {previous_results}
|
||||
```
|
||||
|
||||
### Install Workflow
|
||||
|
||||
```bash
|
||||
# Add to user workflows
|
||||
skill-seekers workflows add my-workflow.yaml
|
||||
|
||||
# With custom name
|
||||
skill-seekers workflows add my-workflow.yaml --name perf-guide
|
||||
```
|
||||
|
||||
### Use Custom Workflow
|
||||
|
||||
```bash
|
||||
skill-seekers create <source> --enhance-workflow performance-focus
|
||||
```
|
||||
|
||||
### Update Workflow
|
||||
|
||||
```bash
|
||||
# Edit the file, then:
|
||||
skill-seekers workflows add my-workflow.yaml --name performance-focus
|
||||
```
|
||||
|
||||
### Remove Workflow
|
||||
|
||||
```bash
|
||||
skill-seekers workflows remove performance-focus
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Workflow Variables
|
||||
|
||||
Pass variables to workflows at runtime:
|
||||
|
||||
### In Workflow Definition
|
||||
|
||||
```yaml
|
||||
variables:
|
||||
target_audience: "beginners"
|
||||
focus_area: "security"
|
||||
```
|
||||
|
||||
### Override at Runtime
|
||||
|
||||
```bash
|
||||
skill-seekers create <source> \
|
||||
--enhance-workflow my-workflow \
|
||||
--var target_audience=experts \
|
||||
--var focus_area=performance
|
||||
```
|
||||
|
||||
### Use in Prompts
|
||||
|
||||
```yaml
|
||||
stages:
|
||||
- name: customization
|
||||
prompt: |
|
||||
Tailor content for {target_audience}.
|
||||
Focus on {focus_area} aspects.
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Inline Stages
|
||||
|
||||
Add one-off enhancement stages without creating a workflow file:
|
||||
|
||||
```bash
|
||||
skill-seekers create <source> \
|
||||
--enhance-stage "performance:Analyze performance characteristics"
|
||||
```
|
||||
|
||||
**Format:** `name:prompt`
|
||||
|
||||
**Multiple stages:**
|
||||
```bash
|
||||
skill-seekers create <source> \
|
||||
--enhance-stage "perf:Analyze performance" \
|
||||
--enhance-stage "security:Check security" \
|
||||
--enhance-stage "examples:Add more examples"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Workflow Dry Run
|
||||
|
||||
Preview what a workflow will do without executing:
|
||||
|
||||
```bash
|
||||
skill-seekers create <source> \
|
||||
--enhance-workflow security-focus \
|
||||
--workflow-dry-run
|
||||
```
|
||||
|
||||
**Output:**
|
||||
```
|
||||
Workflow: security-focus
|
||||
Stages:
|
||||
1. security-overview
|
||||
- Will analyze security features
|
||||
- Target: skill_md
|
||||
|
||||
2. vulnerability-analysis
|
||||
- Will identify vulnerabilities
|
||||
- Target: skill_md
|
||||
|
||||
3. best-practices
|
||||
- Will document best practices
|
||||
- Target: skill_md
|
||||
|
||||
4. compliance
|
||||
- Will map to standards
|
||||
- Target: skill_md
|
||||
|
||||
Execution order: Sequential
|
||||
Estimated time: ~4 minutes
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Workflow Validation
|
||||
|
||||
Validate workflow syntax:
|
||||
|
||||
```bash
|
||||
# Validate bundled workflow
|
||||
skill-seekers workflows validate security-focus
|
||||
|
||||
# Validate file
|
||||
skill-seekers workflows validate ./my-workflow.yaml
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Copying Workflows
|
||||
|
||||
Copy bundled workflows to customize:
|
||||
|
||||
```bash
|
||||
# Copy single workflow
|
||||
skill-seekers workflows copy security-focus
|
||||
|
||||
# Copy multiple
|
||||
skill-seekers workflows copy security-focus api-documentation minimal
|
||||
|
||||
# Edit the copy
|
||||
nano ~/.config/skill-seekers/workflows/security-focus.yaml
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Best Practices
|
||||
|
||||
### 1. Start with Default
|
||||
|
||||
```bash
|
||||
# Default is good for most cases
|
||||
skill-seekers create <source>
|
||||
```
|
||||
|
||||
### 2. Add Specific Workflows as Needed
|
||||
|
||||
```bash
|
||||
# Security-focused project
|
||||
skill-seekers create auth-library --enhance-workflow security-focus
|
||||
|
||||
# API project
|
||||
skill-seekers create api-framework --enhance-workflow api-documentation
|
||||
```
|
||||
|
||||
### 3. Chain for Comprehensive Analysis
|
||||
|
||||
```bash
|
||||
# Large framework: architecture + security
|
||||
skill-seekers create kubernetes/kubernetes \
|
||||
--enhance-workflow architecture-comprehensive \
|
||||
--enhance-workflow security-focus
|
||||
```
|
||||
|
||||
### 4. Create Custom for Specialized Needs
|
||||
|
||||
```bash
|
||||
# Create custom workflow for your domain
|
||||
skill-seekers workflows add ml-workflow.yaml
|
||||
skill-seekers create ml-framework --enhance-workflow ml-focus
|
||||
```
|
||||
|
||||
### 5. Use Variables for Flexibility
|
||||
|
||||
```bash
|
||||
# Same workflow, different targets
|
||||
skill-seekers create <source> \
|
||||
--enhance-workflow my-workflow \
|
||||
--var audience=beginners
|
||||
|
||||
skill-seekers create <source> \
|
||||
--enhance-workflow my-workflow \
|
||||
--var audience=experts
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### "Workflow not found"
|
||||
|
||||
```bash
|
||||
# List available
|
||||
skill-seekers workflows list
|
||||
|
||||
# Check spelling
|
||||
skill-seekers create <source> --enhance-workflow security-focus
|
||||
```
|
||||
|
||||
### "Invalid workflow YAML"
|
||||
|
||||
```bash
|
||||
# Validate
|
||||
skill-seekers workflows validate ./my-workflow.yaml
|
||||
|
||||
# Common issues:
|
||||
# - Missing 'stages' key
|
||||
# - Invalid YAML syntax
|
||||
# - Undefined variable references
|
||||
```
|
||||
|
||||
### "Workflow stage failed"
|
||||
|
||||
```bash
|
||||
# Check stage details
|
||||
skill-seekers workflows show my-workflow
|
||||
|
||||
# Try with dry run
|
||||
skill-seekers create <source> \
|
||||
--enhance-workflow my-workflow \
|
||||
--workflow-dry-run
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Workflow Support Across All Scrapers
|
||||
|
||||
Workflows are supported by **all 5 scrapers** in Skill Seekers:
|
||||
|
||||
| Scraper | Command | Workflow Support |
|
||||
|---------|---------|------------------|
|
||||
| Documentation | `scrape` | ✅ Full support |
|
||||
| GitHub | `github` | ✅ Full support |
|
||||
| Local Codebase | `analyze` | ✅ Full support |
|
||||
| PDF | `pdf` | ✅ Full support |
|
||||
| Unified/Multi-Source | `unified` | ✅ Full support |
|
||||
| Create (Auto-detect) | `create` | ✅ Full support |
|
||||
|
||||
### Using Workflows with Different Sources
|
||||
|
||||
```bash
|
||||
# Documentation website
|
||||
skill-seekers scrape https://docs.example.com --enhance-workflow security-focus
|
||||
|
||||
# GitHub repository
|
||||
skill-seekers github --repo owner/repo --enhance-workflow api-documentation
|
||||
|
||||
# Local codebase
|
||||
skill-seekers analyze --directory ./my-project --enhance-workflow architecture-comprehensive
|
||||
|
||||
# PDF document
|
||||
skill-seekers pdf --pdf manual.pdf --enhance-workflow minimal
|
||||
|
||||
# Unified config (multi-source)
|
||||
skill-seekers unified --config configs/multi-source.json --enhance-workflow security-focus
|
||||
|
||||
# Auto-detect source type
|
||||
skill-seekers create ./my-project --enhance-workflow security-focus
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Workflows in Config Files
|
||||
|
||||
Unified configs support defining workflows at the top level:
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "my-skill",
|
||||
"description": "Complete skill with security enhancement",
|
||||
"workflows": ["security-focus", "api-documentation"],
|
||||
"workflow_stages": [
|
||||
{
|
||||
"name": "cleanup",
|
||||
"prompt": "Remove boilerplate and standardize formatting"
|
||||
}
|
||||
],
|
||||
"workflow_vars": {
|
||||
"focus_area": "performance",
|
||||
"detail_level": "comprehensive"
|
||||
},
|
||||
"sources": [
|
||||
{"type": "docs", "base_url": "https://docs.example.com/"}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
**Priority:** CLI flags override config values
|
||||
|
||||
```bash
|
||||
# Config has security-focus, CLI overrides with api-documentation
|
||||
skill-seekers unified config.json --enhance-workflow api-documentation
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
| Approach | When to Use |
|
||||
|----------|-------------|
|
||||
| **Default** | Most cases |
|
||||
| **Security-Focus** | Security-sensitive projects |
|
||||
| **Architecture** | Large frameworks, systems |
|
||||
| **API-Docs** | API frameworks, libraries |
|
||||
| **Custom** | Specialized domains |
|
||||
| **Chaining** | Multiple perspectives needed |
|
||||
|
||||
---
|
||||
|
||||
## Next Steps
|
||||
|
||||
- [Custom Workflows](../advanced/custom-workflows.md) - Advanced workflow creation
|
||||
- [Enhancement Guide](03-enhancement.md) - Enhancement fundamentals
|
||||
- [MCP Reference](../reference/MCP_REFERENCE.md) - Workflows via MCP
|
||||
619
docs/user-guide/06-troubleshooting.md
Normal file
619
docs/user-guide/06-troubleshooting.md
Normal file
@@ -0,0 +1,619 @@
|
||||
# Troubleshooting Guide
|
||||
|
||||
> **Skill Seekers v3.1.0**
|
||||
> **Common issues and solutions**
|
||||
|
||||
---
|
||||
|
||||
## Quick Fixes
|
||||
|
||||
| Issue | Quick Fix |
|
||||
|-------|-----------|
|
||||
| `command not found` | `export PATH="$HOME/.local/bin:$PATH"` |
|
||||
| `ImportError` | `pip install -e .` |
|
||||
| `Rate limit` | Add `--rate-limit 2.0` |
|
||||
| `No content` | Check selectors in config |
|
||||
| `Enhancement fails` | Set `ANTHROPIC_API_KEY` |
|
||||
| `Out of memory` | Use `--streaming` mode |
|
||||
|
||||
---
|
||||
|
||||
## Installation Issues
|
||||
|
||||
### "command not found: skill-seekers"
|
||||
|
||||
**Cause:** pip bin directory not in PATH
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
# Add to PATH
|
||||
export PATH="$HOME/.local/bin:$PATH"
|
||||
|
||||
# Or reinstall with --user
|
||||
pip install --user --force-reinstall skill-seekers
|
||||
|
||||
# Verify
|
||||
which skill-seekers
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### "No module named 'skill_seekers'"
|
||||
|
||||
**Cause:** Package not installed or wrong Python environment
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
# Install package
|
||||
pip install skill-seekers
|
||||
|
||||
# For development
|
||||
pip install -e .
|
||||
|
||||
# Verify
|
||||
python -c "import skill_seekers; print(skill_seekers.__version__)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### "Permission denied"
|
||||
|
||||
**Cause:** Trying to install system-wide
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
# Don't use sudo
|
||||
# Instead:
|
||||
pip install --user skill-seekers
|
||||
|
||||
# Or use virtual environment
|
||||
python3 -m venv venv
|
||||
source venv/bin/activate
|
||||
pip install skill-seekers
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Scraping Issues
|
||||
|
||||
### "Rate limit exceeded"
|
||||
|
||||
**Cause:** Too many requests to server
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
# Slow down
|
||||
skill-seekers create <url> --rate-limit 2.0
|
||||
|
||||
# For GitHub
|
||||
export GITHUB_TOKEN=ghp_...
|
||||
skill-seekers github --repo owner/repo
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### "No content extracted"
|
||||
|
||||
**Cause:** Wrong CSS selectors
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
# Find correct selectors
|
||||
curl -s <url> | grep -i 'article\|main\|content'
|
||||
|
||||
# Create config with correct selectors
|
||||
cat > configs/fix.json << 'EOF'
|
||||
{
|
||||
"name": "my-site",
|
||||
"base_url": "https://example.com/",
|
||||
"selectors": {
|
||||
"main_content": "article" # or "main", ".content", etc.
|
||||
}
|
||||
}
|
||||
EOF
|
||||
|
||||
skill-seekers create --config configs/fix.json
|
||||
```
|
||||
|
||||
**Common selectors:**
|
||||
| Site Type | Selector |
|
||||
|-----------|----------|
|
||||
| Docusaurus | `article` |
|
||||
| ReadTheDocs | `[role="main"]` |
|
||||
| GitBook | `.book-body` |
|
||||
| MkDocs | `.md-content` |
|
||||
|
||||
---
|
||||
|
||||
### "Too many pages"
|
||||
|
||||
**Cause:** Site larger than max_pages setting
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
# Estimate first
|
||||
skill-seekers estimate configs/my-config.json
|
||||
|
||||
# Increase limit
|
||||
skill-seekers create <url> --max-pages 1000
|
||||
|
||||
# Or limit in config
|
||||
{
|
||||
"max_pages": 1000
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### "Connection timeout"
|
||||
|
||||
**Cause:** Slow server or network issues
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
# Increase timeout
|
||||
skill-seekers create <url> --timeout 60
|
||||
|
||||
# Or in config
|
||||
{
|
||||
"timeout": 60
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### "SSL certificate error"
|
||||
|
||||
**Cause:** Certificate validation failure
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
# Set environment variable (not recommended for production)
|
||||
export PYTHONWARNINGS="ignore:Unverified HTTPS request"
|
||||
|
||||
# Or use requests settings in config
|
||||
{
|
||||
"verify_ssl": false
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Enhancement Issues
|
||||
|
||||
### "Enhancement failed: No API key"
|
||||
|
||||
**Cause:** ANTHROPIC_API_KEY not set
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
# Set API key
|
||||
export ANTHROPIC_API_KEY=sk-ant-...
|
||||
|
||||
# Or use LOCAL mode
|
||||
skill-seekers enhance output/my-skill/ --agent local
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### "Claude Code not found" (LOCAL mode)
|
||||
|
||||
**Cause:** Claude Code not installed
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
# Install Claude Code
|
||||
# See: https://claude.ai/code
|
||||
|
||||
# Or use API mode
|
||||
export ANTHROPIC_API_KEY=sk-ant-...
|
||||
skill-seekers enhance output/my-skill/ --agent api
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### "Enhancement timeout"
|
||||
|
||||
**Cause:** Enhancement taking too long
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
# Increase timeout
|
||||
skill-seekers enhance output/my-skill/ --timeout 1200
|
||||
|
||||
# Use background mode
|
||||
skill-seekers enhance output/my-skill/ --background
|
||||
skill-seekers enhance-status output/my-skill/ --watch
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### "Workflow not found"
|
||||
|
||||
**Cause:** Typo or workflow doesn't exist
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
# List available workflows
|
||||
skill-seekers workflows list
|
||||
|
||||
# Check spelling
|
||||
skill-seekers create <source> --enhance-workflow security-focus
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Packaging Issues
|
||||
|
||||
### "Package validation failed"
|
||||
|
||||
**Cause:** SKILL.md missing or malformed
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
# Check structure
|
||||
ls output/my-skill/
|
||||
|
||||
# Should contain:
|
||||
# - SKILL.md
|
||||
# - references/
|
||||
|
||||
# Rebuild if needed
|
||||
skill-seekers create --config my-config --skip-scrape
|
||||
|
||||
# Or recreate
|
||||
skill-seekers create <source>
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### "Target platform not supported"
|
||||
|
||||
**Cause:** Typo in target name
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
# List valid targets
|
||||
skill-seekers package --help
|
||||
|
||||
# Valid targets:
|
||||
# claude, gemini, openai, langchain, llama-index,
|
||||
# haystack, pinecone, chroma, weaviate, qdrant, faiss, markdown
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### "Out of memory"
|
||||
|
||||
**Cause:** Skill too large for available RAM
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
# Use streaming mode
|
||||
skill-seekers package output/my-skill/ --streaming
|
||||
|
||||
# Reduce chunk size
|
||||
skill-seekers package output/my-skill/ \
|
||||
--streaming \
|
||||
--chunk-size 1000
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Upload Issues
|
||||
|
||||
### "Upload failed: Invalid API key"
|
||||
|
||||
**Cause:** Wrong or missing API key
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
# Claude
|
||||
export ANTHROPIC_API_KEY=sk-ant-...
|
||||
|
||||
# Gemini
|
||||
export GOOGLE_API_KEY=AIza...
|
||||
|
||||
# OpenAI
|
||||
export OPENAI_API_KEY=sk-...
|
||||
|
||||
# Verify
|
||||
echo $ANTHROPIC_API_KEY
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### "Upload failed: Network error"
|
||||
|
||||
**Cause:** Connection issues
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
# Check connection
|
||||
ping api.anthropic.com
|
||||
|
||||
# Retry
|
||||
skill-seekers upload output/my-skill-claude.zip --target claude
|
||||
|
||||
# Or upload manually through web interface
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### "Upload failed: File too large"
|
||||
|
||||
**Cause:** Package exceeds platform limits
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
# Check size
|
||||
ls -lh output/my-skill-claude.zip
|
||||
|
||||
# Use streaming mode
|
||||
skill-seekers package output/my-skill/ --streaming
|
||||
|
||||
# Or split into smaller skills
|
||||
skill-seekers workflows split-config configs/my-config.json
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## GitHub Issues
|
||||
|
||||
### "GitHub API rate limit"
|
||||
|
||||
**Cause:** Unauthenticated requests limited to 60/hour
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
# Set token
|
||||
export GITHUB_TOKEN=ghp_...
|
||||
|
||||
# Create token: https://github.com/settings/tokens
|
||||
# Needs: repo, read:org (for private repos)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### "Repository not found"
|
||||
|
||||
**Cause:** Private repo or wrong name
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
# Check repo exists
|
||||
https://github.com/owner/repo
|
||||
|
||||
# Set token for private repos
|
||||
export GITHUB_TOKEN=ghp_...
|
||||
|
||||
# Correct format
|
||||
skill-seekers github --repo owner/repo
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### "No code found"
|
||||
|
||||
**Cause:** Empty repo or wrong branch
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
# Check repo has code
|
||||
|
||||
# Specify branch in config
|
||||
{
|
||||
"type": "github",
|
||||
"repo": "owner/repo",
|
||||
"branch": "main"
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## PDF Issues
|
||||
|
||||
### "PDF is encrypted"
|
||||
|
||||
**Cause:** Password-protected PDF
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
# Add password to config
|
||||
{
|
||||
"type": "pdf",
|
||||
"pdf_path": "protected.pdf",
|
||||
"password": "secret123"
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### "OCR failed"
|
||||
|
||||
**Cause:** Scanned PDF without OCR
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
# Enable OCR
|
||||
skill-seekers pdf --pdf scanned.pdf --enable-ocr
|
||||
|
||||
# Install OCR dependencies
|
||||
pip install skill-seekers[pdf-ocr]
|
||||
# System: apt-get install tesseract-ocr
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Configuration Issues
|
||||
|
||||
### "Invalid config JSON"
|
||||
|
||||
**Cause:** Syntax error in config file
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
# Validate JSON
|
||||
python -m json.tool configs/my-config.json
|
||||
|
||||
# Or use online validator
|
||||
# jsonlint.com
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### "Config not found"
|
||||
|
||||
**Cause:** Wrong path or missing file
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
# Check file exists
|
||||
ls configs/my-config.json
|
||||
|
||||
# Use absolute path
|
||||
skill-seekers create --config /full/path/to/config.json
|
||||
|
||||
# Or list available
|
||||
skill-seekers estimate --all
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Performance Issues
|
||||
|
||||
### "Scraping is too slow"
|
||||
|
||||
**Solutions:**
|
||||
```bash
|
||||
# Use async mode
|
||||
skill-seekers create <url> --async --workers 5
|
||||
|
||||
# Reduce rate limit (for your own servers)
|
||||
skill-seekers create <url> --rate-limit 0.1
|
||||
|
||||
# Skip enhancement
|
||||
skill-seekers create <url> --enhance-level 0
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### "Out of disk space"
|
||||
|
||||
**Solutions:**
|
||||
```bash
|
||||
# Check usage
|
||||
du -sh output/
|
||||
|
||||
# Clean old skills
|
||||
rm -rf output/old-skill/
|
||||
|
||||
# Use streaming mode
|
||||
skill-seekers create <url> --streaming
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### "High memory usage"
|
||||
|
||||
**Solutions:**
|
||||
```bash
|
||||
# Use streaming mode
|
||||
skill-seekers create <url> --streaming
|
||||
skill-seekers package output/my-skill/ --streaming
|
||||
|
||||
# Reduce workers
|
||||
skill-seekers create <url> --workers 1
|
||||
|
||||
# Limit pages
|
||||
skill-seekers create <url> --max-pages 100
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Getting Help
|
||||
|
||||
### Debug Mode
|
||||
|
||||
```bash
|
||||
# Enable verbose logging
|
||||
skill-seekers create <source> --verbose
|
||||
|
||||
# Or environment variable
|
||||
export SKILL_SEEKERS_DEBUG=1
|
||||
```
|
||||
|
||||
### Check Logs
|
||||
|
||||
```bash
|
||||
# Enable file logging
|
||||
export SKILL_SEEKERS_LOG_FILE=/tmp/skill-seekers.log
|
||||
|
||||
# Tail logs
|
||||
tail -f /tmp/skill-seekers.log
|
||||
```
|
||||
|
||||
### Create Minimal Reproduction
|
||||
|
||||
```bash
|
||||
# Create test config
|
||||
cat > test-config.json << 'EOF'
|
||||
{
|
||||
"name": "test",
|
||||
"base_url": "https://example.com/",
|
||||
"max_pages": 5
|
||||
}
|
||||
EOF
|
||||
|
||||
# Run with debug
|
||||
skill-seekers create --config test-config.json --verbose --dry-run
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Report an Issue
|
||||
|
||||
If none of these solutions work:
|
||||
|
||||
1. **Gather info:**
|
||||
```bash
|
||||
skill-seekers --version
|
||||
python --version
|
||||
pip show skill-seekers
|
||||
```
|
||||
|
||||
2. **Enable debug:**
|
||||
```bash
|
||||
skill-seekers <command> --verbose 2>&1 | tee debug.log
|
||||
```
|
||||
|
||||
3. **Create issue:**
|
||||
- https://github.com/yusufkaraaslan/Skill_Seekers/issues
|
||||
- Include: error message, command used, debug log
|
||||
|
||||
---
|
||||
|
||||
## Error Reference
|
||||
|
||||
| Error Code | Meaning | Solution |
|
||||
|------------|---------|----------|
|
||||
| `E001` | Config not found | Check path |
|
||||
| `E002` | Invalid config | Validate JSON |
|
||||
| `E003` | Network error | Check connection |
|
||||
| `E004` | Rate limited | Slow down or use token |
|
||||
| `E005` | Scraping failed | Check selectors |
|
||||
| `E006` | Enhancement failed | Check API key |
|
||||
| `E007` | Packaging failed | Check skill structure |
|
||||
| `E008` | Upload failed | Check API key |
|
||||
|
||||
---
|
||||
|
||||
## Still Stuck?
|
||||
|
||||
- **Documentation:** https://skillseekersweb.com/
|
||||
- **GitHub Issues:** https://github.com/yusufkaraaslan/Skill_Seekers/issues
|
||||
- **Discussions:** Share your use case
|
||||
|
||||
---
|
||||
|
||||
*Last updated: 2026-02-16*
|
||||
Reference in New Issue
Block a user