docs: add full UML architecture with 14 class diagrams synced from source code
- 14 StarUML diagrams covering all 13 modules (8 core + 5 utility) - ~200 classes with operations, attributes, and documentation from actual source - Package overview with 25 verified inter-module dependencies - Exported PNG diagrams in Docs/UML/exports/ - Architecture.md with embedded diagram descriptions - CLAUDE.md updated with architecture reference Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
43
CLAUDE.md
@@ -8,6 +8,8 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
|
||||
|
||||
**Version:** 3.3.0 | **Python:** 3.10+ | **Website:** https://skillseekersweb.com/
|
||||
|
||||
**Architecture:** See `Docs/Architecture.md` for UML diagrams and module overview. StarUML project at `Docs/UML/skill_seekers.mdj`.
|
||||
|
||||
## Essential Commands
|
||||
|
||||
```bash
|
||||
@@ -57,7 +59,8 @@ Entry point `src/skill_seekers/cli/main.py` maps subcommands to modules. The `cr
|
||||
```
|
||||
skill-seekers create <source> # Auto-detect: URL, owner/repo, ./path, file.pdf, etc.
|
||||
skill-seekers <type> [options] # Direct: scrape, github, pdf, word, epub, video, jupyter, html, openapi, asciidoc, pptx, rss, manpage, confluence, notion, chat
|
||||
skill-seekers package <dir> # Package for platform (--target claude/gemini/openai/markdown/minimax/opencode/kimi/deepseek/qwen/openrouter/together/fireworks, --format langchain/llama-index/haystack/chroma/faiss/weaviate/qdrant)
|
||||
skill-seekers analyze <dir> # Analyze local codebase (C3.x pipeline)
|
||||
skill-seekers package <dir> # Package for platform (--target claude/gemini/openai/markdown/minimax/opencode/kimi/deepseek/qwen/openrouter/together/fireworks, --format langchain/llama-index/haystack/chroma/faiss/weaviate/qdrant/pinecone)
|
||||
```
|
||||
|
||||
### Data Flow (5 phases)
|
||||
@@ -70,22 +73,25 @@ skill-seekers package <dir> # Package for platform (--target claude/gemini
|
||||
|
||||
### Platform Adaptor Pattern (Strategy + Factory)
|
||||
|
||||
Factory: `get_adaptor(platform, config)` in `adaptors/__init__.py` returns a `SkillAdaptor` instance. Base class `SkillAdaptor` + `SkillMetadata` in `adaptors/base.py`.
|
||||
|
||||
```
|
||||
src/skill_seekers/cli/adaptors/
|
||||
├── __init__.py # Factory: get_adaptor(target=..., format=...)
|
||||
├── base_adaptor.py # Abstract base: package(), upload(), enhance(), export()
|
||||
├── claude_adaptor.py # --target claude
|
||||
├── gemini_adaptor.py # --target gemini
|
||||
├── openai_adaptor.py # --target openai
|
||||
├── markdown_adaptor.py # --target markdown
|
||||
├── minimax_adaptor.py # --target minimax
|
||||
├── opencode_adaptor.py # --target opencode
|
||||
├── kimi_adaptor.py # --target kimi
|
||||
├── deepseek_adaptor.py # --target deepseek
|
||||
├── qwen_adaptor.py # --target qwen
|
||||
├── openrouter_adaptor.py # --target openrouter
|
||||
├── together_adaptor.py # --target together
|
||||
├── fireworks_adaptor.py # --target fireworks
|
||||
├── __init__.py # Factory: get_adaptor(platform, config), ADAPTORS registry
|
||||
├── base.py # Abstract base: SkillAdaptor, SkillMetadata
|
||||
├── openai_compatible.py # Shared base for OpenAI-compatible platforms
|
||||
├── claude.py # --target claude
|
||||
├── gemini.py # --target gemini
|
||||
├── openai.py # --target openai
|
||||
├── markdown.py # --target markdown
|
||||
├── minimax.py # --target minimax
|
||||
├── opencode.py # --target opencode
|
||||
├── kimi.py # --target kimi
|
||||
├── deepseek.py # --target deepseek
|
||||
├── qwen.py # --target qwen
|
||||
├── openrouter.py # --target openrouter
|
||||
├── together.py # --target together
|
||||
├── fireworks.py # --target fireworks
|
||||
├── langchain.py # --format langchain
|
||||
├── llama_index.py # --format llama-index
|
||||
├── haystack.py # --format haystack
|
||||
@@ -93,10 +99,11 @@ src/skill_seekers/cli/adaptors/
|
||||
├── faiss_helpers.py # --format faiss
|
||||
├── qdrant.py # --format qdrant
|
||||
├── weaviate.py # --format weaviate
|
||||
├── pinecone_adaptor.py # --format pinecone
|
||||
└── streaming_adaptor.py # --format streaming
|
||||
```
|
||||
|
||||
`--target` = LLM platforms, `--format` = RAG/vector DBs.
|
||||
`--target` = LLM platforms, `--format` = RAG/vector DBs. All adaptors are imported with `try/except ImportError` so missing optional deps don't break the registry.
|
||||
|
||||
### 17 Source Type Scrapers
|
||||
|
||||
@@ -208,8 +215,8 @@ GITHUB_TOKEN=ghp_... # Higher GitHub rate limits
|
||||
## Adding New Features
|
||||
|
||||
### New platform adaptor
|
||||
1. Create `src/skill_seekers/cli/adaptors/{platform}_adaptor.py` inheriting `BaseAdaptor`
|
||||
2. Register in `adaptors/__init__.py` factory
|
||||
1. Create `src/skill_seekers/cli/adaptors/{platform}.py` inheriting `SkillAdaptor` from `base.py`
|
||||
2. Register in `adaptors/__init__.py` (add try/except import + add to `ADAPTORS` dict)
|
||||
3. Add optional dep to `pyproject.toml`
|
||||
4. Add tests in `tests/`
|
||||
|
||||
|
||||
114
Docs/Architecture.md
Normal file
@@ -0,0 +1,114 @@
|
||||
# Skill Seekers Architecture
|
||||
|
||||
> Generated 2026-03-22 | StarUML project: `Docs/UML/skill_seekers.mdj`
|
||||
|
||||
## Overview
|
||||
|
||||
Skill Seekers converts documentation from 17 source types into production-ready formats for 24+ AI platforms. The architecture follows a layered module design with 8 core modules and 5 utility modules.
|
||||
|
||||
## Package Diagram
|
||||
|
||||

|
||||
|
||||
**Core Modules** (upper area):
|
||||
- **CLICore** -- Git-style command dispatcher, entry point for all `skill-seekers` commands
|
||||
- **Scrapers** -- 17 source-type extractors (web, GitHub, PDF, Word, EPUB, video, etc.)
|
||||
- **Adaptors** -- Strategy+Factory pattern for 20+ output platforms (Claude, Gemini, OpenAI, RAG frameworks)
|
||||
- **Analysis** -- C3.x codebase analysis pipeline (AST parsing, 10 GoF pattern detectors, guide builders)
|
||||
- **Enhancement** -- AI-powered skill improvement (API mode + LOCAL mode, --enhance-level 0-3)
|
||||
- **Packaging** -- Package, upload, and install skills to AI agent directories
|
||||
- **MCP** -- FastMCP server exposing 34 tools via stdio/HTTP transport
|
||||
- **Sync** -- Documentation change detection and re-scraping triggers
|
||||
|
||||
**Utility Modules** (lower area):
|
||||
- **Parsers** -- CLI argument parsers (30+ SubcommandParser subclasses)
|
||||
- **Storage** -- Cloud storage abstraction (S3, GCS, Azure)
|
||||
- **Embedding** -- Multi-provider vector embedding generation
|
||||
- **Benchmark** -- Performance measurement framework
|
||||
- **Utilities** -- Shared helpers (LanguageDetector, RAGChunker, MarkdownCleaner, etc.)
|
||||
|
||||
## Core Module Diagrams
|
||||
|
||||
### CLICore
|
||||

|
||||
|
||||
Entry point: `skill-seekers` CLI. `CLIDispatcher` maps subcommands to modules via `COMMAND_MODULES` dict. `CreateCommand` auto-detects source type via `SourceDetector`.
|
||||
|
||||
### Scrapers
|
||||

|
||||
|
||||
18 scraper classes implementing `IScraper`. Each has a `main()` entry point. Notable: `GitHubScraper` (3-stream fetcher) + `GitHubToSkillConverter` (builder), `UnifiedScraper` (multi-source orchestrator).
|
||||
|
||||
### Adaptors
|
||||

|
||||
|
||||
`SkillAdaptor` ABC with 3 abstract methods: `format_skill_md()`, `package()`, `upload()`. Two-level hierarchy: direct subclasses (Claude, Gemini, OpenAI, Markdown, OpenCode, RAG adaptors) and `OpenAICompatibleAdaptor` intermediate (MiniMax, Kimi, DeepSeek, Qwen, OpenRouter, Together, Fireworks).
|
||||
|
||||
### Analysis (C3.x Pipeline)
|
||||

|
||||
|
||||
`UnifiedCodebaseAnalyzer` controller orchestrates: `CodeAnalyzer` (AST, 9 languages), `PatternRecognizer` (10 GoF detectors via `BasePatternDetector`), `TestExampleExtractor`, `HowToGuideBuilder`, `ConfigExtractor`, `SignalFlowAnalyzer`, `DependencyAnalyzer`, `ArchitecturalPatternDetector`.
|
||||
|
||||
### Enhancement
|
||||

|
||||
|
||||
Two enhancement hierarchies: `AIEnhancer` (API mode, Claude API calls) and `UnifiedEnhancer` (C3.x pipeline enhancers). Each has specialized subclasses for patterns, test examples, guides, and configs. `WorkflowEngine` orchestrates multi-stage `EnhancementWorkflow`.
|
||||
|
||||
### Packaging
|
||||

|
||||
|
||||
`PackageSkill` delegates to adaptors for format-specific packaging. `UploadSkill` handles platform API uploads. `InstallSkill`/`InstallAgent` install to AI agent directories. `OpenCodeSkillSplitter` handles large file splitting.
|
||||
|
||||
### MCP Server
|
||||

|
||||
|
||||
`SkillSeekerMCPServer` (FastMCP) with 34 tools in 8 categories. Supporting classes: `SourceManager` (config CRUD), `AgentDetector` (environment detection), `GitConfigRepo` (community configs).
|
||||
|
||||
### Sync
|
||||

|
||||
|
||||
`SyncMonitor` controller schedules periodic checks via `ChangeDetector` (SHA-256 hashing, HTTP headers, content diffing). `Notifier` sends alerts when changes are found. Pydantic models: `PageChange`, `ChangeReport`, `SyncConfig`, `SyncState`.
|
||||
|
||||
## Utility Module Diagrams
|
||||
|
||||
### Parsers
|
||||

|
||||
|
||||
`SubcommandParser` ABC with 27 subclasses -- one per CLI subcommand (Create, Scrape, GitHub, PDF, Word, EPUB, Video, Unified, Analyze, Enhance, Package, Upload, Jupyter, HTML, OpenAPI, AsciiDoc, Pptx, RSS, ManPage, Confluence, Notion, Chat, Config, Estimate, Install, Stream, Quality, SyncConfig).
|
||||
|
||||
### Storage
|
||||

|
||||
|
||||
`BaseStorageAdaptor` ABC with `S3StorageAdaptor`, `GCSStorageAdaptor`, `AzureStorageAdaptor`. `StorageObject` dataclass for file metadata.
|
||||
|
||||
### Embedding
|
||||

|
||||
|
||||
`EmbeddingGenerator` (multi-provider: OpenAI, Sentence Transformers, Voyage AI). `EmbeddingPipeline` coordinates provider, caching, and cost tracking. `EmbeddingProvider` ABC with OpenAI and Local implementations.
|
||||
|
||||
### Benchmark
|
||||

|
||||
|
||||
`BenchmarkRunner` orchestrates `Benchmark` instances. `BenchmarkResult` collects timings/memory/metrics and produces `BenchmarkReport`. Supporting data types: `Metric`, `TimingResult`, `MemoryUsage`, `ComparisonReport`.
|
||||
|
||||
### Utilities
|
||||

|
||||
|
||||
16 shared helper classes: `LanguageDetector`, `MarkdownCleaner`, `RAGChunker`, `RateLimitHandler`, `ConfigManager`, `ConfigValidator`, `SkillQualityChecker`, `QualityAnalyzer`, `LlmsTxtDetector`/`Downloader`/`Parser`, `ConfigSplitter`, `ConflictDetector`, `IncrementalUpdater`, `MultiLanguageManager`, `StreamingIngester`.
|
||||
|
||||
## Key Design Patterns
|
||||
|
||||
| Pattern | Where | Classes |
|
||||
|---------|-------|---------|
|
||||
| Strategy + Factory | Adaptors | `SkillAdaptor` ABC + `get_adaptor()` factory + 20+ implementations |
|
||||
| Strategy + Factory | Storage | `BaseStorageAdaptor` ABC + S3/GCS/Azure |
|
||||
| Strategy + Factory | Embedding | `EmbeddingProvider` ABC + OpenAI/Local |
|
||||
| Command | CLI | `CLIDispatcher` + `COMMAND_MODULES` lazy dispatch |
|
||||
| Template Method | Pattern Detection | `BasePatternDetector` + 10 GoF detectors |
|
||||
| Template Method | Parsers | `SubcommandParser` + 27 subclasses |
|
||||
|
||||
## File Locations
|
||||
|
||||
- **StarUML project**: `Docs/UML/skill_seekers.mdj`
|
||||
- **Diagram exports**: `Docs/UML/exports/*.png`
|
||||
- **Source code**: `src/skill_seekers/`
|
||||
BIN
Docs/UML/exports/00_package_overview.png
Normal file
|
After Width: | Height: | Size: 108 KiB |
BIN
Docs/UML/exports/01_cli_core.png
Normal file
|
After Width: | Height: | Size: 43 KiB |
BIN
Docs/UML/exports/02_scrapers.png
Normal file
|
After Width: | Height: | Size: 486 KiB |
BIN
Docs/UML/exports/03_adaptors.png
Normal file
|
After Width: | Height: | Size: 776 KiB |
BIN
Docs/UML/exports/04_analysis.png
Normal file
|
After Width: | Height: | Size: 439 KiB |
BIN
Docs/UML/exports/05_enhancement.png
Normal file
|
After Width: | Height: | Size: 336 KiB |
BIN
Docs/UML/exports/06_packaging.png
Normal file
|
After Width: | Height: | Size: 129 KiB |
BIN
Docs/UML/exports/07_mcp_server.png
Normal file
|
After Width: | Height: | Size: 491 KiB |
BIN
Docs/UML/exports/08_sync.png
Normal file
|
After Width: | Height: | Size: 214 KiB |
BIN
Docs/UML/exports/09_parsers.png
Normal file
|
After Width: | Height: | Size: 219 KiB |
BIN
Docs/UML/exports/10_storage.png
Normal file
|
After Width: | Height: | Size: 261 KiB |
BIN
Docs/UML/exports/11_embedding.png
Normal file
|
After Width: | Height: | Size: 157 KiB |
BIN
Docs/UML/exports/12_benchmark.png
Normal file
|
After Width: | Height: | Size: 127 KiB |
BIN
Docs/UML/exports/13_utilities.png
Normal file
|
After Width: | Height: | Size: 223 KiB |