docs: Phase 6 - Add comprehensive multi-LLM platform documentation
Add three detailed platform guides: 1. **MULTI_LLM_SUPPORT.md** - Complete multi-platform overview - Supported platforms comparison table - Quick start for all platforms - Installation options - Complete workflow examples - Advanced usage and troubleshooting - Programmatic API usage examples 2. **GEMINI_INTEGRATION.md** - Google Gemini integration guide - Setup and API key configuration - Complete workflow with tar.gz packaging - Gemini-specific format differences - Files API + grounding usage - Cost estimation and best practices - Troubleshooting common issues 3. **OPENAI_INTEGRATION.md** - OpenAI ChatGPT integration guide - Setup and API key configuration - Complete workflow with Assistants API - Vector Store + file_search integration - Assistant instructions format - Cost estimation and best practices - Troubleshooting common issues All guides include: - Code examples for CLI and Python API - Platform-specific features and differences - Real-world usage patterns - Troubleshooting sections - Best practices Related to #179
This commit is contained in:
yusyus
435
docs/GEMINI_INTEGRATION.md
Normal file
435
docs/GEMINI_INTEGRATION.md
Normal file
@@ -0,0 +1,435 @@
|
||||
# Google Gemini Integration Guide
|
||||
|
||||
Complete guide for creating and deploying skills to Google Gemini using Skill Seekers.
|
||||
|
||||
## Overview
|
||||
|
||||
Skill Seekers packages documentation into Gemini-compatible formats optimized for:
|
||||
- **Gemini 2.0 Flash** for enhancement
|
||||
- **Files API** for document upload
|
||||
- **Grounding** for accurate, source-based responses
|
||||
|
||||
## Setup
|
||||
|
||||
### 1. Install Gemini Support
|
||||
|
||||
```bash
|
||||
# Install with Gemini dependencies
|
||||
pip install skill-seekers[gemini]
|
||||
|
||||
# Verify installation
|
||||
pip list | grep google-generativeai
|
||||
```
|
||||
|
||||
### 2. Get Google API Key
|
||||
|
||||
1. Visit [Google AI Studio](https://aistudio.google.com/)
|
||||
2. Click "Get API Key"
|
||||
3. Create new API key or use existing
|
||||
4. Copy the key (starts with `AIza`)
|
||||
|
||||
### 3. Configure API Key
|
||||
|
||||
```bash
|
||||
# Set as environment variable (recommended)
|
||||
export GOOGLE_API_KEY=AIzaSy...
|
||||
|
||||
# Or pass directly to commands
|
||||
skill-seekers upload --target gemini --api-key AIzaSy...
|
||||
```
|
||||
|
||||
## Complete Workflow
|
||||
|
||||
### Step 1: Scrape Documentation
|
||||
|
||||
```bash
|
||||
# Use any config (scraping is platform-agnostic)
|
||||
skill-seekers scrape --config configs/react.json
|
||||
|
||||
# Or use a unified config for multi-source
|
||||
skill-seekers unified --config configs/react_unified.json
|
||||
```
|
||||
|
||||
**Result:** `output/react/` skill directory with references
|
||||
|
||||
### Step 2: Enhance with Gemini (Optional but Recommended)
|
||||
|
||||
```bash
|
||||
# Enhance SKILL.md using Gemini 2.0 Flash
|
||||
skill-seekers enhance output/react/ --target gemini
|
||||
|
||||
# With API key specified
|
||||
skill-seekers enhance output/react/ --target gemini --api-key AIzaSy...
|
||||
```
|
||||
|
||||
**What it does:**
|
||||
- Analyzes all reference documentation
|
||||
- Extracts 5-10 best code examples
|
||||
- Creates comprehensive quick reference
|
||||
- Adds key concepts and usage guidance
|
||||
- Generates plain markdown (no YAML frontmatter)
|
||||
|
||||
**Time:** 20-40 seconds
|
||||
**Cost:** ~$0.01-0.05 (using Gemini 2.0 Flash)
|
||||
**Quality boost:** 3/10 → 9/10
|
||||
|
||||
### Step 3: Package for Gemini
|
||||
|
||||
```bash
|
||||
# Create tar.gz package for Gemini
|
||||
skill-seekers package output/react/ --target gemini
|
||||
|
||||
# Result: react-gemini.tar.gz
|
||||
```
|
||||
|
||||
**Package structure:**
|
||||
```
|
||||
react-gemini.tar.gz/
|
||||
├── system_instructions.md # Main documentation (plain markdown)
|
||||
├── references/ # Individual reference files
|
||||
│ ├── getting_started.md
|
||||
│ ├── hooks.md
|
||||
│ ├── components.md
|
||||
│ └── ...
|
||||
└── gemini_metadata.json # Platform metadata
|
||||
```
|
||||
|
||||
### Step 4: Upload to Gemini
|
||||
|
||||
```bash
|
||||
# Upload to Google AI Studio
|
||||
skill-seekers upload react-gemini.tar.gz --target gemini
|
||||
|
||||
# With API key
|
||||
skill-seekers upload react-gemini.tar.gz --target gemini --api-key AIzaSy...
|
||||
```
|
||||
|
||||
**Output:**
|
||||
```
|
||||
✅ Upload successful!
|
||||
Skill ID: files/abc123xyz
|
||||
URL: https://aistudio.google.com/app/files/abc123xyz
|
||||
Files uploaded: 15 files
|
||||
```
|
||||
|
||||
### Step 5: Use in Gemini
|
||||
|
||||
Access your uploaded files in Google AI Studio:
|
||||
|
||||
1. Go to [Google AI Studio](https://aistudio.google.com/)
|
||||
2. Navigate to **Files** section
|
||||
3. Find your uploaded skill files
|
||||
4. Use with Gemini API or AI Studio
|
||||
|
||||
## What Makes Gemini Different?
|
||||
|
||||
### Format: Plain Markdown (No YAML)
|
||||
|
||||
**Claude format:**
|
||||
```markdown
|
||||
---
|
||||
name: react
|
||||
description: React framework
|
||||
---
|
||||
|
||||
# React Documentation
|
||||
...
|
||||
```
|
||||
|
||||
**Gemini format:**
|
||||
```markdown
|
||||
# React Documentation
|
||||
|
||||
**Description:** React framework for building user interfaces
|
||||
|
||||
## Quick Reference
|
||||
...
|
||||
```
|
||||
|
||||
No YAML frontmatter - Gemini uses plain markdown for better compatibility.
|
||||
|
||||
### Package: tar.gz Instead of ZIP
|
||||
|
||||
Gemini uses `.tar.gz` compression for better Unix compatibility and smaller file sizes.
|
||||
|
||||
### Upload: Files API + Grounding
|
||||
|
||||
Files are uploaded to Google's Files API and made available for grounding in Gemini responses.
|
||||
|
||||
## Using Your Gemini Skill
|
||||
|
||||
### Option 1: Google AI Studio (Web UI)
|
||||
|
||||
1. Go to [Google AI Studio](https://aistudio.google.com/)
|
||||
2. Create new chat or app
|
||||
3. Reference your uploaded files in prompts:
|
||||
```
|
||||
Using the React documentation files, explain hooks
|
||||
```
|
||||
|
||||
### Option 2: Gemini API (Python)
|
||||
|
||||
```python
|
||||
import google.generativeai as genai
|
||||
|
||||
# Configure with your API key
|
||||
genai.configure(api_key='AIzaSy...')
|
||||
|
||||
# Create model
|
||||
model = genai.GenerativeModel('gemini-2.0-flash-exp')
|
||||
|
||||
# Use with uploaded files (automatic grounding)
|
||||
response = model.generate_content(
|
||||
"How do I use React hooks?",
|
||||
# Files automatically available via grounding
|
||||
)
|
||||
|
||||
print(response.text)
|
||||
```
|
||||
|
||||
### Option 3: Gemini API with File Reference
|
||||
|
||||
```python
|
||||
import google.generativeai as genai
|
||||
|
||||
# Configure
|
||||
genai.configure(api_key='AIzaSy...')
|
||||
|
||||
# Get your uploaded file
|
||||
files = genai.list_files()
|
||||
react_file = next(f for f in files if 'react' in f.display_name.lower())
|
||||
|
||||
# Use file in generation
|
||||
model = genai.GenerativeModel('gemini-2.0-flash-exp')
|
||||
response = model.generate_content([
|
||||
"Explain React hooks in detail",
|
||||
react_file
|
||||
])
|
||||
|
||||
print(response.text)
|
||||
```
|
||||
|
||||
## Advanced Usage
|
||||
|
||||
### Enhance with Custom Prompt
|
||||
|
||||
The enhancement process can be customized by modifying the adaptor:
|
||||
|
||||
```python
|
||||
from skill_seekers.cli.adaptors import get_adaptor
|
||||
from pathlib import Path
|
||||
|
||||
# Get Gemini adaptor
|
||||
adaptor = get_adaptor('gemini')
|
||||
|
||||
# Enhance with custom parameters
|
||||
success = adaptor.enhance(
|
||||
skill_dir=Path('output/react'),
|
||||
api_key='AIzaSy...'
|
||||
)
|
||||
```
|
||||
|
||||
### Programmatic Upload
|
||||
|
||||
```python
|
||||
from skill_seekers.cli.adaptors import get_adaptor
|
||||
from pathlib import Path
|
||||
|
||||
# Get adaptor
|
||||
gemini = get_adaptor('gemini')
|
||||
|
||||
# Package skill
|
||||
package_path = gemini.package(
|
||||
skill_dir=Path('output/react'),
|
||||
output_path=Path('output/react-gemini.tar.gz')
|
||||
)
|
||||
|
||||
# Upload
|
||||
result = gemini.upload(
|
||||
package_path=package_path,
|
||||
api_key='AIzaSy...'
|
||||
)
|
||||
|
||||
if result['success']:
|
||||
print(f"✅ Uploaded to: {result['url']}")
|
||||
print(f"Skill ID: {result['skill_id']}")
|
||||
else:
|
||||
print(f"❌ Upload failed: {result['message']}")
|
||||
```
|
||||
|
||||
### Manual Package Extraction
|
||||
|
||||
If you want to inspect or modify the package:
|
||||
|
||||
```bash
|
||||
# Extract tar.gz
|
||||
tar -xzf react-gemini.tar.gz -C extracted/
|
||||
|
||||
# View structure
|
||||
tree extracted/
|
||||
|
||||
# Modify files if needed
|
||||
nano extracted/system_instructions.md
|
||||
|
||||
# Re-package
|
||||
tar -czf react-gemini-modified.tar.gz -C extracted .
|
||||
```
|
||||
|
||||
## Gemini-Specific Features
|
||||
|
||||
### 1. Grounding Support
|
||||
|
||||
Gemini automatically grounds responses in your uploaded documentation files, providing:
|
||||
- Source attribution
|
||||
- Accurate citations
|
||||
- Reduced hallucination
|
||||
|
||||
### 2. Multimodal Capabilities
|
||||
|
||||
Gemini can process:
|
||||
- Text documentation
|
||||
- Code examples
|
||||
- Images (if included in PDFs)
|
||||
- Tables and diagrams
|
||||
|
||||
### 3. Long Context Window
|
||||
|
||||
Gemini 2.0 Flash supports:
|
||||
- Up to 1M token context
|
||||
- Entire documentation sets in single context
|
||||
- Better understanding of cross-references
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Issue: `google-generativeai not installed`
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
pip install skill-seekers[gemini]
|
||||
```
|
||||
|
||||
### Issue: `Invalid API key format`
|
||||
|
||||
**Error:** API key doesn't start with `AIza`
|
||||
|
||||
**Solution:**
|
||||
- Get new key from [Google AI Studio](https://aistudio.google.com/)
|
||||
- Verify you're using Google API key, not GCP service account
|
||||
|
||||
### Issue: `Not a tar.gz file`
|
||||
|
||||
**Error:** Wrong package format
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
# Use --target gemini for tar.gz format
|
||||
skill-seekers package output/react/ --target gemini
|
||||
|
||||
# NOT:
|
||||
skill-seekers package output/react/ # Creates .zip (Claude format)
|
||||
```
|
||||
|
||||
### Issue: `File upload failed`
|
||||
|
||||
**Possible causes:**
|
||||
- API key lacks permissions
|
||||
- File too large (check limits)
|
||||
- Network connectivity
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
# Verify API key works
|
||||
python3 -c "import google.generativeai as genai; genai.configure(api_key='AIza...'); print(list(genai.list_models())[:2])"
|
||||
|
||||
# Check file size
|
||||
ls -lh react-gemini.tar.gz
|
||||
|
||||
# Try with verbose output
|
||||
skill-seekers upload react-gemini.tar.gz --target gemini --verbose
|
||||
```
|
||||
|
||||
### Issue: Enhancement fails
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
# Check API quota
|
||||
# Visit: https://aistudio.google.com/apikey
|
||||
|
||||
# Try with smaller skill
|
||||
skill-seekers enhance output/react/ --target gemini --max-files 5
|
||||
|
||||
# Use without enhancement
|
||||
skill-seekers package output/react/ --target gemini
|
||||
# (Skip enhancement step)
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
### 1. Organize Documentation
|
||||
|
||||
Structure your SKILL.md clearly:
|
||||
- Start with overview
|
||||
- Add quick reference section
|
||||
- Group related concepts
|
||||
- Include practical examples
|
||||
|
||||
### 2. Optimize File Count
|
||||
|
||||
- Combine related topics into single files
|
||||
- Use clear file naming
|
||||
- Keep total under 100 files for best performance
|
||||
|
||||
### 3. Test with Gemini
|
||||
|
||||
After upload, test with sample questions:
|
||||
```
|
||||
1. How do I get started with [topic]?
|
||||
2. What are the core concepts?
|
||||
3. Show me a practical example
|
||||
4. What are common pitfalls?
|
||||
```
|
||||
|
||||
### 4. Update Regularly
|
||||
|
||||
```bash
|
||||
# Re-scrape updated documentation
|
||||
skill-seekers scrape --config configs/react.json
|
||||
|
||||
# Re-enhance and upload
|
||||
skill-seekers enhance output/react/ --target gemini
|
||||
skill-seekers package output/react/ --target gemini
|
||||
skill-seekers upload react-gemini.tar.gz --target gemini
|
||||
```
|
||||
|
||||
## Cost Estimation
|
||||
|
||||
**Gemini 2.0 Flash pricing:**
|
||||
- Input: $0.075 per 1M tokens
|
||||
- Output: $0.30 per 1M tokens
|
||||
|
||||
**Typical skill enhancement:**
|
||||
- Input: ~50K-200K tokens (docs)
|
||||
- Output: ~5K-10K tokens (enhanced SKILL.md)
|
||||
- Cost: $0.01-0.05 per skill
|
||||
|
||||
**File upload:** Free (no per-file charges)
|
||||
|
||||
## Next Steps
|
||||
|
||||
1. ✅ Install Gemini support: `pip install skill-seekers[gemini]`
|
||||
2. ✅ Get API key from Google AI Studio
|
||||
3. ✅ Scrape your documentation
|
||||
4. ✅ Enhance with Gemini
|
||||
5. ✅ Package for Gemini
|
||||
6. ✅ Upload and test
|
||||
|
||||
## Resources
|
||||
|
||||
- [Google AI Studio](https://aistudio.google.com/)
|
||||
- [Gemini API Documentation](https://ai.google.dev/docs)
|
||||
- [Gemini Pricing](https://ai.google.dev/pricing)
|
||||
- [Multi-LLM Support Guide](MULTI_LLM_SUPPORT.md)
|
||||
|
||||
## Feedback
|
||||
|
||||
Found an issue or have suggestions? [Open an issue](https://github.com/yusufkaraaslan/Skill_Seekers/issues)
|
||||
407
docs/MULTI_LLM_SUPPORT.md
Normal file
407
docs/MULTI_LLM_SUPPORT.md
Normal file
@@ -0,0 +1,407 @@
|
||||
# Multi-LLM Platform Support Guide
|
||||
|
||||
Skill Seekers supports multiple LLM platforms through a clean adaptor system. The core scraping and content organization remains universal, while packaging and upload are platform-specific.
|
||||
|
||||
## Supported Platforms
|
||||
|
||||
| Platform | Status | Format | Upload | Enhancement | API Key Required |
|
||||
|----------|--------|--------|--------|-------------|------------------|
|
||||
| **Claude AI** | ✅ Full Support | ZIP + YAML | ✅ Automatic | ✅ Yes | ANTHROPIC_API_KEY |
|
||||
| **Google Gemini** | ✅ Full Support | tar.gz | ✅ Automatic | ✅ Yes | GOOGLE_API_KEY |
|
||||
| **OpenAI ChatGPT** | ✅ Full Support | ZIP + Vector Store | ✅ Automatic | ✅ Yes | OPENAI_API_KEY |
|
||||
| **Generic Markdown** | ✅ Export Only | ZIP | ❌ Manual | ❌ No | None |
|
||||
|
||||
## Quick Start
|
||||
|
||||
### Claude AI (Default)
|
||||
|
||||
No changes needed! All existing workflows continue to work:
|
||||
|
||||
```bash
|
||||
# Scrape documentation
|
||||
skill-seekers scrape --config configs/react.json
|
||||
|
||||
# Package for Claude (default)
|
||||
skill-seekers package output/react/
|
||||
|
||||
# Upload to Claude
|
||||
skill-seekers upload react.zip
|
||||
```
|
||||
|
||||
### Google Gemini
|
||||
|
||||
```bash
|
||||
# Install Gemini support
|
||||
pip install skill-seekers[gemini]
|
||||
|
||||
# Set API key
|
||||
export GOOGLE_API_KEY=AIzaSy...
|
||||
|
||||
# Scrape documentation (same as always)
|
||||
skill-seekers scrape --config configs/react.json
|
||||
|
||||
# Package for Gemini
|
||||
skill-seekers package output/react/ --target gemini
|
||||
|
||||
# Upload to Gemini
|
||||
skill-seekers upload react-gemini.tar.gz --target gemini
|
||||
|
||||
# Optional: Enhance with Gemini
|
||||
skill-seekers enhance output/react/ --target gemini
|
||||
```
|
||||
|
||||
**Output:** `react-gemini.tar.gz` ready for Google AI Studio
|
||||
|
||||
### OpenAI ChatGPT
|
||||
|
||||
```bash
|
||||
# Install OpenAI support
|
||||
pip install skill-seekers[openai]
|
||||
|
||||
# Set API key
|
||||
export OPENAI_API_KEY=sk-proj-...
|
||||
|
||||
# Scrape documentation (same as always)
|
||||
skill-seekers scrape --config configs/react.json
|
||||
|
||||
# Package for OpenAI
|
||||
skill-seekers package output/react/ --target openai
|
||||
|
||||
# Upload to OpenAI (creates Assistant + Vector Store)
|
||||
skill-seekers upload react-openai.zip --target openai
|
||||
|
||||
# Optional: Enhance with GPT-4o
|
||||
skill-seekers enhance output/react/ --target openai
|
||||
```
|
||||
|
||||
**Output:** OpenAI Assistant created with file search enabled
|
||||
|
||||
### Generic Markdown (Universal Export)
|
||||
|
||||
```bash
|
||||
# Package as generic markdown (no dependencies)
|
||||
skill-seekers package output/react/ --target markdown
|
||||
|
||||
# Output: react-markdown.zip with:
|
||||
# - README.md
|
||||
# - references/*.md
|
||||
# - DOCUMENTATION.md (combined)
|
||||
```
|
||||
|
||||
**Use case:** Export for any LLM, documentation hosting, or manual distribution
|
||||
|
||||
## Installation Options
|
||||
|
||||
### Install Core Package Only
|
||||
|
||||
```bash
|
||||
# Default installation (Claude support only)
|
||||
pip install skill-seekers
|
||||
```
|
||||
|
||||
### Install with Specific Platform Support
|
||||
|
||||
```bash
|
||||
# Google Gemini support
|
||||
pip install skill-seekers[gemini]
|
||||
|
||||
# OpenAI ChatGPT support
|
||||
pip install skill-seekers[openai]
|
||||
|
||||
# All LLM platforms
|
||||
pip install skill-seekers[all-llms]
|
||||
|
||||
# Development dependencies (includes testing)
|
||||
pip install skill-seekers[dev]
|
||||
```
|
||||
|
||||
### Install from Source
|
||||
|
||||
```bash
|
||||
git clone https://github.com/yusufkaraaslan/Skill_Seekers.git
|
||||
cd Skill_Seekers
|
||||
|
||||
# Editable install with all platforms
|
||||
pip install -e .[all-llms]
|
||||
```
|
||||
|
||||
## Platform Comparison
|
||||
|
||||
### Format Differences
|
||||
|
||||
**Claude AI:**
|
||||
- Format: ZIP archive
|
||||
- SKILL.md: YAML frontmatter + markdown
|
||||
- Structure: `SKILL.md`, `references/`, `scripts/`, `assets/`
|
||||
- API: Anthropic Skills API
|
||||
- Enhancement: Claude Sonnet 4
|
||||
|
||||
**Google Gemini:**
|
||||
- Format: tar.gz archive
|
||||
- SKILL.md → `system_instructions.md` (plain markdown, no frontmatter)
|
||||
- Structure: `system_instructions.md`, `references/`, `gemini_metadata.json`
|
||||
- API: Google Files API + grounding
|
||||
- Enhancement: Gemini 2.0 Flash
|
||||
|
||||
**OpenAI ChatGPT:**
|
||||
- Format: ZIP archive
|
||||
- SKILL.md → `assistant_instructions.txt` (plain text)
|
||||
- Structure: `assistant_instructions.txt`, `vector_store_files/`, `openai_metadata.json`
|
||||
- API: Assistants API + Vector Store
|
||||
- Enhancement: GPT-4o
|
||||
|
||||
**Generic Markdown:**
|
||||
- Format: ZIP archive
|
||||
- Structure: `README.md`, `references/`, `DOCUMENTATION.md` (combined)
|
||||
- No API integration
|
||||
- No enhancement support
|
||||
- Universal compatibility
|
||||
|
||||
### API Key Configuration
|
||||
|
||||
**Claude AI:**
|
||||
```bash
|
||||
export ANTHROPIC_API_KEY=sk-ant-...
|
||||
```
|
||||
|
||||
**Google Gemini:**
|
||||
```bash
|
||||
export GOOGLE_API_KEY=AIzaSy...
|
||||
```
|
||||
|
||||
**OpenAI ChatGPT:**
|
||||
```bash
|
||||
export OPENAI_API_KEY=sk-proj-...
|
||||
```
|
||||
|
||||
## Complete Workflow Examples
|
||||
|
||||
### Workflow 1: Claude AI (Default)
|
||||
|
||||
```bash
|
||||
# 1. Scrape
|
||||
skill-seekers scrape --config configs/react.json
|
||||
|
||||
# 2. Enhance (optional but recommended)
|
||||
skill-seekers enhance output/react/
|
||||
|
||||
# 3. Package
|
||||
skill-seekers package output/react/
|
||||
|
||||
# 4. Upload
|
||||
skill-seekers upload react.zip
|
||||
|
||||
# Access at: https://claude.ai/skills
|
||||
```
|
||||
|
||||
### Workflow 2: Google Gemini
|
||||
|
||||
```bash
|
||||
# Setup (one-time)
|
||||
pip install skill-seekers[gemini]
|
||||
export GOOGLE_API_KEY=AIzaSy...
|
||||
|
||||
# 1. Scrape (universal)
|
||||
skill-seekers scrape --config configs/react.json
|
||||
|
||||
# 2. Enhance for Gemini
|
||||
skill-seekers enhance output/react/ --target gemini
|
||||
|
||||
# 3. Package for Gemini
|
||||
skill-seekers package output/react/ --target gemini
|
||||
|
||||
# 4. Upload to Gemini
|
||||
skill-seekers upload react-gemini.tar.gz --target gemini
|
||||
|
||||
# Access at: https://aistudio.google.com/files/
|
||||
```
|
||||
|
||||
### Workflow 3: OpenAI ChatGPT
|
||||
|
||||
```bash
|
||||
# Setup (one-time)
|
||||
pip install skill-seekers[openai]
|
||||
export OPENAI_API_KEY=sk-proj-...
|
||||
|
||||
# 1. Scrape (universal)
|
||||
skill-seekers scrape --config configs/react.json
|
||||
|
||||
# 2. Enhance with GPT-4o
|
||||
skill-seekers enhance output/react/ --target openai
|
||||
|
||||
# 3. Package for OpenAI
|
||||
skill-seekers package output/react/ --target openai
|
||||
|
||||
# 4. Upload (creates Assistant + Vector Store)
|
||||
skill-seekers upload react-openai.zip --target openai
|
||||
|
||||
# Access at: https://platform.openai.com/assistants/
|
||||
```
|
||||
|
||||
### Workflow 4: Export to All Platforms
|
||||
|
||||
```bash
|
||||
# Install all platforms
|
||||
pip install skill-seekers[all-llms]
|
||||
|
||||
# Scrape once
|
||||
skill-seekers scrape --config configs/react.json
|
||||
|
||||
# Package for all platforms
|
||||
skill-seekers package output/react/ --target claude
|
||||
skill-seekers package output/react/ --target gemini
|
||||
skill-seekers package output/react/ --target openai
|
||||
skill-seekers package output/react/ --target markdown
|
||||
|
||||
# Result:
|
||||
# - react.zip (Claude)
|
||||
# - react-gemini.tar.gz (Gemini)
|
||||
# - react-openai.zip (OpenAI)
|
||||
# - react-markdown.zip (Universal)
|
||||
```
|
||||
|
||||
## Advanced Usage
|
||||
|
||||
### Custom Enhancement Models
|
||||
|
||||
Each platform uses its default enhancement model, but you can customize:
|
||||
|
||||
```bash
|
||||
# Use specific model for enhancement (if supported)
|
||||
skill-seekers enhance output/react/ --target gemini --model gemini-2.0-flash-exp
|
||||
skill-seekers enhance output/react/ --target openai --model gpt-4o
|
||||
```
|
||||
|
||||
### Programmatic Usage
|
||||
|
||||
```python
|
||||
from skill_seekers.cli.adaptors import get_adaptor
|
||||
|
||||
# Get platform-specific adaptor
|
||||
gemini = get_adaptor('gemini')
|
||||
openai = get_adaptor('openai')
|
||||
claude = get_adaptor('claude')
|
||||
|
||||
# Package for specific platform
|
||||
gemini_package = gemini.package(skill_dir, output_path)
|
||||
openai_package = openai.package(skill_dir, output_path)
|
||||
|
||||
# Upload with API key
|
||||
result = gemini.upload(gemini_package, api_key)
|
||||
print(f"Uploaded to: {result['url']}")
|
||||
```
|
||||
|
||||
### Platform Detection
|
||||
|
||||
Check which platforms are available:
|
||||
|
||||
```python
|
||||
from skill_seekers.cli.adaptors import list_platforms, is_platform_available
|
||||
|
||||
# List all registered platforms
|
||||
platforms = list_platforms()
|
||||
print(platforms) # ['claude', 'gemini', 'openai', 'markdown']
|
||||
|
||||
# Check if platform is available
|
||||
if is_platform_available('gemini'):
|
||||
print("Gemini adaptor is available")
|
||||
```
|
||||
|
||||
## Backward Compatibility
|
||||
|
||||
**100% backward compatible** with existing workflows:
|
||||
|
||||
- All existing Claude commands work unchanged
|
||||
- Default behavior remains Claude-focused
|
||||
- Optional `--target` flag adds multi-platform support
|
||||
- No breaking changes to existing configs or workflows
|
||||
|
||||
## Platform-Specific Guides
|
||||
|
||||
For detailed platform-specific instructions, see:
|
||||
|
||||
- [Claude AI Integration](CLAUDE_INTEGRATION.md) (default)
|
||||
- [Google Gemini Integration](GEMINI_INTEGRATION.md)
|
||||
- [OpenAI ChatGPT Integration](OPENAI_INTEGRATION.md)
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Missing Dependencies
|
||||
|
||||
**Error:** `ModuleNotFoundError: No module named 'google.generativeai'`
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
pip install skill-seekers[gemini]
|
||||
```
|
||||
|
||||
**Error:** `ModuleNotFoundError: No module named 'openai'`
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
pip install skill-seekers[openai]
|
||||
```
|
||||
|
||||
### API Key Issues
|
||||
|
||||
**Error:** `Invalid API key format`
|
||||
|
||||
**Solution:** Check your API key format:
|
||||
- Claude: `sk-ant-...`
|
||||
- Gemini: `AIza...`
|
||||
- OpenAI: `sk-proj-...` or `sk-...`
|
||||
|
||||
### Package Format Errors
|
||||
|
||||
**Error:** `Not a tar.gz file: react.zip`
|
||||
|
||||
**Solution:** Use correct --target flag:
|
||||
```bash
|
||||
# Gemini requires tar.gz
|
||||
skill-seekers package output/react/ --target gemini
|
||||
|
||||
# OpenAI and Claude use ZIP
|
||||
skill-seekers package output/react/ --target openai
|
||||
```
|
||||
|
||||
## FAQ
|
||||
|
||||
**Q: Can I use the same scraped data for all platforms?**
|
||||
|
||||
A: Yes! The scraping phase is universal. Only packaging and upload are platform-specific.
|
||||
|
||||
**Q: Do I need separate API keys for each platform?**
|
||||
|
||||
A: Yes, each platform requires its own API key. Set them as environment variables.
|
||||
|
||||
**Q: Can I enhance with different models?**
|
||||
|
||||
A: Yes, each platform uses its own enhancement model:
|
||||
- Claude: Claude Sonnet 4
|
||||
- Gemini: Gemini 2.0 Flash
|
||||
- OpenAI: GPT-4o
|
||||
|
||||
**Q: What if I don't want to upload automatically?**
|
||||
|
||||
A: Use the `package` command without `upload`. You'll get the packaged file to upload manually.
|
||||
|
||||
**Q: Is the markdown export compatible with all LLMs?**
|
||||
|
||||
A: Yes! The generic markdown export creates universal documentation that works with any LLM or documentation system.
|
||||
|
||||
**Q: Can I contribute a new platform adaptor?**
|
||||
|
||||
A: Absolutely! See the [Contributing Guide](../CONTRIBUTING.md) for how to add new platform adaptors.
|
||||
|
||||
## Next Steps
|
||||
|
||||
1. Choose your target platform
|
||||
2. Install optional dependencies if needed
|
||||
3. Set up API keys
|
||||
4. Follow the platform-specific workflow
|
||||
5. Upload and test your skill
|
||||
|
||||
For more help, see:
|
||||
- [Quick Start Guide](../QUICKSTART.md)
|
||||
- [Troubleshooting Guide](../TROUBLESHOOTING.md)
|
||||
- [Platform-Specific Guides](.)
|
||||
515
docs/OPENAI_INTEGRATION.md
Normal file
515
docs/OPENAI_INTEGRATION.md
Normal file
@@ -0,0 +1,515 @@
|
||||
# OpenAI ChatGPT Integration Guide
|
||||
|
||||
Complete guide for creating and deploying skills to OpenAI ChatGPT using Skill Seekers.
|
||||
|
||||
## Overview
|
||||
|
||||
Skill Seekers packages documentation into OpenAI-compatible formats optimized for:
|
||||
- **Assistants API** for custom AI assistants
|
||||
- **Vector Store + File Search** for accurate retrieval
|
||||
- **GPT-4o** for enhancement and responses
|
||||
|
||||
## Setup
|
||||
|
||||
### 1. Install OpenAI Support
|
||||
|
||||
```bash
|
||||
# Install with OpenAI dependencies
|
||||
pip install skill-seekers[openai]
|
||||
|
||||
# Verify installation
|
||||
pip list | grep openai
|
||||
```
|
||||
|
||||
### 2. Get OpenAI API Key
|
||||
|
||||
1. Visit [OpenAI Platform](https://platform.openai.com/)
|
||||
2. Navigate to **API keys** section
|
||||
3. Click "Create new secret key"
|
||||
4. Copy the key (starts with `sk-proj-` or `sk-`)
|
||||
|
||||
### 3. Configure API Key
|
||||
|
||||
```bash
|
||||
# Set as environment variable (recommended)
|
||||
export OPENAI_API_KEY=sk-proj-...
|
||||
|
||||
# Or pass directly to commands
|
||||
skill-seekers upload --target openai --api-key sk-proj-...
|
||||
```
|
||||
|
||||
## Complete Workflow
|
||||
|
||||
### Step 1: Scrape Documentation
|
||||
|
||||
```bash
|
||||
# Use any config (scraping is platform-agnostic)
|
||||
skill-seekers scrape --config configs/react.json
|
||||
|
||||
# Or use a unified config for multi-source
|
||||
skill-seekers unified --config configs/react_unified.json
|
||||
```
|
||||
|
||||
**Result:** `output/react/` skill directory with references
|
||||
|
||||
### Step 2: Enhance with GPT-4o (Optional but Recommended)
|
||||
|
||||
```bash
|
||||
# Enhance SKILL.md using GPT-4o
|
||||
skill-seekers enhance output/react/ --target openai
|
||||
|
||||
# With API key specified
|
||||
skill-seekers enhance output/react/ --target openai --api-key sk-proj-...
|
||||
```
|
||||
|
||||
**What it does:**
|
||||
- Analyzes all reference documentation
|
||||
- Extracts 5-10 best code examples
|
||||
- Creates comprehensive assistant instructions
|
||||
- Adds response guidelines and search strategy
|
||||
- Formats as plain text (no YAML frontmatter)
|
||||
|
||||
**Time:** 20-40 seconds
|
||||
**Cost:** ~$0.15-0.30 (using GPT-4o)
|
||||
**Quality boost:** 3/10 → 9/10
|
||||
|
||||
### Step 3: Package for OpenAI
|
||||
|
||||
```bash
|
||||
# Create ZIP package for OpenAI Assistants
|
||||
skill-seekers package output/react/ --target openai
|
||||
|
||||
# Result: react-openai.zip
|
||||
```
|
||||
|
||||
**Package structure:**
|
||||
```
|
||||
react-openai.zip/
|
||||
├── assistant_instructions.txt # Main instructions for Assistant
|
||||
├── vector_store_files/ # Files for Vector Store + file_search
|
||||
│ ├── getting_started.md
|
||||
│ ├── hooks.md
|
||||
│ ├── components.md
|
||||
│ └── ...
|
||||
└── openai_metadata.json # Platform metadata
|
||||
```
|
||||
|
||||
### Step 4: Upload to OpenAI (Creates Assistant)
|
||||
|
||||
```bash
|
||||
# Upload and create Assistant with Vector Store
|
||||
skill-seekers upload react-openai.zip --target openai
|
||||
|
||||
# With API key
|
||||
skill-seekers upload react-openai.zip --target openai --api-key sk-proj-...
|
||||
```
|
||||
|
||||
**What it does:**
|
||||
1. Creates Vector Store for documentation
|
||||
2. Uploads reference files to Vector Store
|
||||
3. Creates Assistant with file_search tool
|
||||
4. Links Vector Store to Assistant
|
||||
|
||||
**Output:**
|
||||
```
|
||||
✅ Upload successful!
|
||||
Assistant ID: asst_abc123xyz
|
||||
URL: https://platform.openai.com/assistants/asst_abc123xyz
|
||||
Message: Assistant created with 15 knowledge files
|
||||
```
|
||||
|
||||
### Step 5: Use Your Assistant
|
||||
|
||||
Access your assistant in the OpenAI Platform:
|
||||
|
||||
1. Go to [OpenAI Platform](https://platform.openai.com/assistants)
|
||||
2. Find your assistant in the list
|
||||
3. Test in Playground or use via API
|
||||
|
||||
## What Makes OpenAI Different?
|
||||
|
||||
### Format: Assistant Instructions (Plain Text)
|
||||
|
||||
**Claude format:**
|
||||
```markdown
|
||||
---
|
||||
name: react
|
||||
---
|
||||
|
||||
# React Documentation
|
||||
...
|
||||
```
|
||||
|
||||
**OpenAI format:**
|
||||
```text
|
||||
You are an expert assistant for React.
|
||||
|
||||
Your Knowledge Base:
|
||||
- Getting started guide
|
||||
- React hooks reference
|
||||
- Component API
|
||||
|
||||
When users ask questions about React:
|
||||
1. Search the knowledge files
|
||||
2. Provide code examples
|
||||
...
|
||||
```
|
||||
|
||||
Plain text instructions optimized for Assistant API.
|
||||
|
||||
### Architecture: Assistant + Vector Store
|
||||
|
||||
OpenAI uses a two-part system:
|
||||
1. **Assistant** - The AI agent with instructions and tools
|
||||
2. **Vector Store** - Embedded documentation for semantic search
|
||||
|
||||
### Tool: file_search
|
||||
|
||||
The Assistant uses the `file_search` tool to:
|
||||
- Semantically search documentation
|
||||
- Find relevant code examples
|
||||
- Provide accurate, source-based answers
|
||||
|
||||
## Using Your OpenAI Assistant
|
||||
|
||||
### Option 1: OpenAI Playground (Web UI)
|
||||
|
||||
1. Go to [OpenAI Platform](https://platform.openai.com/assistants)
|
||||
2. Select your assistant
|
||||
3. Click "Test in Playground"
|
||||
4. Ask questions about your documentation
|
||||
|
||||
### Option 2: Assistants API (Python)
|
||||
|
||||
```python
|
||||
from openai import OpenAI
|
||||
|
||||
# Initialize client
|
||||
client = OpenAI(api_key='sk-proj-...')
|
||||
|
||||
# Create thread
|
||||
thread = client.beta.threads.create()
|
||||
|
||||
# Send message
|
||||
message = client.beta.threads.messages.create(
|
||||
thread_id=thread.id,
|
||||
role="user",
|
||||
content="How do I use React hooks?"
|
||||
)
|
||||
|
||||
# Run assistant
|
||||
run = client.beta.threads.runs.create(
|
||||
thread_id=thread.id,
|
||||
assistant_id='asst_abc123xyz' # Your assistant ID
|
||||
)
|
||||
|
||||
# Wait for completion
|
||||
while run.status != 'completed':
|
||||
run = client.beta.threads.runs.retrieve(thread_id=thread.id, run_id=run.id)
|
||||
|
||||
# Get response
|
||||
messages = client.beta.threads.messages.list(thread_id=thread.id)
|
||||
print(messages.data[0].content[0].text.value)
|
||||
```
|
||||
|
||||
### Option 3: Streaming Responses
|
||||
|
||||
```python
|
||||
from openai import OpenAI
|
||||
|
||||
client = OpenAI(api_key='sk-proj-...')
|
||||
|
||||
# Create thread and message
|
||||
thread = client.beta.threads.create()
|
||||
client.beta.threads.messages.create(
|
||||
thread_id=thread.id,
|
||||
role="user",
|
||||
content="Explain React hooks"
|
||||
)
|
||||
|
||||
# Stream response
|
||||
with client.beta.threads.runs.stream(
|
||||
thread_id=thread.id,
|
||||
assistant_id='asst_abc123xyz'
|
||||
) as stream:
|
||||
for event in stream:
|
||||
if event.event == 'thread.message.delta':
|
||||
print(event.data.delta.content[0].text.value, end='')
|
||||
```
|
||||
|
||||
## Advanced Usage
|
||||
|
||||
### Update Assistant Instructions
|
||||
|
||||
```python
|
||||
from openai import OpenAI
|
||||
|
||||
client = OpenAI(api_key='sk-proj-...')
|
||||
|
||||
# Update assistant
|
||||
client.beta.assistants.update(
|
||||
assistant_id='asst_abc123xyz',
|
||||
instructions="""
|
||||
You are an expert React assistant.
|
||||
|
||||
Focus on modern best practices using:
|
||||
- React 18+ features
|
||||
- Functional components
|
||||
- Hooks-based patterns
|
||||
|
||||
When answering:
|
||||
1. Search knowledge files first
|
||||
2. Provide working code examples
|
||||
3. Explain the "why" not just the "what"
|
||||
"""
|
||||
)
|
||||
```
|
||||
|
||||
### Add More Files to Vector Store
|
||||
|
||||
```python
|
||||
from openai import OpenAI
|
||||
|
||||
client = OpenAI(api_key='sk-proj-...')
|
||||
|
||||
# Upload new file
|
||||
with open('new_guide.md', 'rb') as f:
|
||||
file = client.files.create(file=f, purpose='assistants')
|
||||
|
||||
# Add to vector store
|
||||
client.beta.vector_stores.files.create(
|
||||
vector_store_id='vs_abc123',
|
||||
file_id=file.id
|
||||
)
|
||||
```
|
||||
|
||||
### Programmatic Package and Upload
|
||||
|
||||
```python
|
||||
from skill_seekers.cli.adaptors import get_adaptor
|
||||
from pathlib import Path
|
||||
|
||||
# Get adaptor
|
||||
openai_adaptor = get_adaptor('openai')
|
||||
|
||||
# Package skill
|
||||
package_path = openai_adaptor.package(
|
||||
skill_dir=Path('output/react'),
|
||||
output_path=Path('output/react-openai.zip')
|
||||
)
|
||||
|
||||
# Upload (creates Assistant + Vector Store)
|
||||
result = openai_adaptor.upload(
|
||||
package_path=package_path,
|
||||
api_key='sk-proj-...'
|
||||
)
|
||||
|
||||
if result['success']:
|
||||
print(f"✅ Assistant created!")
|
||||
print(f"ID: {result['skill_id']}")
|
||||
print(f"URL: {result['url']}")
|
||||
else:
|
||||
print(f"❌ Upload failed: {result['message']}")
|
||||
```
|
||||
|
||||
## OpenAI-Specific Features
|
||||
|
||||
### 1. Semantic Search (file_search)
|
||||
|
||||
The Assistant uses embeddings to:
|
||||
- Find semantically similar content
|
||||
- Understand intent vs. keywords
|
||||
- Surface relevant examples automatically
|
||||
|
||||
### 2. Citations and Sources
|
||||
|
||||
Assistants can provide:
|
||||
- Source attribution
|
||||
- File references
|
||||
- Quote extraction
|
||||
|
||||
### 3. Function Calling (Optional)
|
||||
|
||||
Extend your assistant with custom tools:
|
||||
|
||||
```python
|
||||
client.beta.assistants.update(
|
||||
assistant_id='asst_abc123xyz',
|
||||
tools=[
|
||||
{"type": "file_search"},
|
||||
{"type": "function", "function": {
|
||||
"name": "run_code_example",
|
||||
"description": "Execute React code examples",
|
||||
"parameters": {...}
|
||||
}}
|
||||
]
|
||||
)
|
||||
```
|
||||
|
||||
### 4. Multi-Modal Support
|
||||
|
||||
Include images in your documentation:
|
||||
- Screenshots
|
||||
- Diagrams
|
||||
- Architecture charts
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Issue: `openai not installed`
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
pip install skill-seekers[openai]
|
||||
```
|
||||
|
||||
### Issue: `Invalid API key format`
|
||||
|
||||
**Error:** API key doesn't start with `sk-`
|
||||
|
||||
**Solution:**
|
||||
- Get new key from [OpenAI Platform](https://platform.openai.com/api-keys)
|
||||
- Verify you're using API key, not organization ID
|
||||
|
||||
### Issue: `Not a ZIP file`
|
||||
|
||||
**Error:** Wrong package format
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
# Use --target openai for ZIP format
|
||||
skill-seekers package output/react/ --target openai
|
||||
|
||||
# NOT:
|
||||
skill-seekers package output/react/ --target gemini # Creates .tar.gz
|
||||
```
|
||||
|
||||
### Issue: `Assistant creation failed`
|
||||
|
||||
**Possible causes:**
|
||||
- API key lacks permissions
|
||||
- Rate limit exceeded
|
||||
- File too large
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
# Verify API key
|
||||
python3 -c "from openai import OpenAI; print(OpenAI(api_key='sk-proj-...').models.list())"
|
||||
|
||||
# Check rate limits
|
||||
# Visit: https://platform.openai.com/account/limits
|
||||
|
||||
# Reduce file count
|
||||
skill-seekers package output/react/ --target openai --max-files 20
|
||||
```
|
||||
|
||||
### Issue: Enhancement fails
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
# Check API quota and billing
|
||||
# Visit: https://platform.openai.com/account/billing
|
||||
|
||||
# Try with smaller skill
|
||||
skill-seekers enhance output/react/ --target openai --max-files 5
|
||||
|
||||
# Use without enhancement
|
||||
skill-seekers package output/react/ --target openai
|
||||
# (Skip enhancement step)
|
||||
```
|
||||
|
||||
### Issue: file_search not working
|
||||
|
||||
**Symptoms:** Assistant doesn't reference documentation
|
||||
|
||||
**Solution:**
|
||||
- Verify Vector Store has files
|
||||
- Check Assistant tool configuration
|
||||
- Test with explicit instructions: "Search the knowledge files for information about hooks"
|
||||
|
||||
## Best Practices
|
||||
|
||||
### 1. Write Clear Assistant Instructions
|
||||
|
||||
Focus on:
|
||||
- Role definition
|
||||
- Knowledge base description
|
||||
- Response guidelines
|
||||
- Search strategy
|
||||
|
||||
### 2. Organize Vector Store Files
|
||||
|
||||
- Keep files under 512KB each
|
||||
- Use clear, descriptive filenames
|
||||
- Structure content with headings
|
||||
- Include code examples
|
||||
|
||||
### 3. Test Assistant Behavior
|
||||
|
||||
Test with varied questions:
|
||||
```
|
||||
1. Simple facts: "What is React?"
|
||||
2. How-to questions: "How do I create a component?"
|
||||
3. Best practices: "What's the best way to manage state?"
|
||||
4. Troubleshooting: "Why isn't my hook working?"
|
||||
```
|
||||
|
||||
### 4. Monitor Token Usage
|
||||
|
||||
```python
|
||||
# Track tokens in API responses
|
||||
run = client.beta.threads.runs.retrieve(thread_id=thread.id, run_id=run.id)
|
||||
print(f"Input tokens: {run.usage.prompt_tokens}")
|
||||
print(f"Output tokens: {run.usage.completion_tokens}")
|
||||
```
|
||||
|
||||
### 5. Update Regularly
|
||||
|
||||
```bash
|
||||
# Re-scrape updated documentation
|
||||
skill-seekers scrape --config configs/react.json
|
||||
|
||||
# Re-enhance and upload (creates new Assistant)
|
||||
skill-seekers enhance output/react/ --target openai
|
||||
skill-seekers package output/react/ --target openai
|
||||
skill-seekers upload react-openai.zip --target openai
|
||||
```
|
||||
|
||||
## Cost Estimation
|
||||
|
||||
**GPT-4o pricing (as of 2024):**
|
||||
- Input: $2.50 per 1M tokens
|
||||
- Output: $10.00 per 1M tokens
|
||||
|
||||
**Typical skill enhancement:**
|
||||
- Input: ~50K-200K tokens (docs)
|
||||
- Output: ~5K-10K tokens (enhanced instructions)
|
||||
- Cost: $0.15-0.30 per skill
|
||||
|
||||
**Vector Store:**
|
||||
- $0.10 per GB per day (storage)
|
||||
- Typical skill: < 100MB = ~$0.01/day
|
||||
|
||||
**API usage:**
|
||||
- Varies by question volume
|
||||
- ~$0.01-0.05 per conversation
|
||||
|
||||
## Next Steps
|
||||
|
||||
1. ✅ Install OpenAI support: `pip install skill-seekers[openai]`
|
||||
2. ✅ Get API key from OpenAI Platform
|
||||
3. ✅ Scrape your documentation
|
||||
4. ✅ Enhance with GPT-4o
|
||||
5. ✅ Package for OpenAI
|
||||
6. ✅ Upload and create Assistant
|
||||
7. ✅ Test in Playground
|
||||
|
||||
## Resources
|
||||
|
||||
- [OpenAI Platform](https://platform.openai.com/)
|
||||
- [Assistants API Documentation](https://platform.openai.com/docs/assistants/overview)
|
||||
- [OpenAI Pricing](https://openai.com/pricing)
|
||||
- [Multi-LLM Support Guide](MULTI_LLM_SUPPORT.md)
|
||||
|
||||
## Feedback
|
||||
|
||||
Found an issue or have suggestions? [Open an issue](https://github.com/yusufkaraaslan/Skill_Seekers/issues)
|
||||
Reference in New Issue
Block a user