skill-seekers-reference

firefrost-gaming/skill-seekers-reference

Files

yusyus 4f9a5a553b feat: Phase 2 - Real upload capabilities for ChromaDB and Weaviate

Implemented complete upload functionality for vector databases, replacing
stub implementations with real upload capabilities including embedding
generation, multiple connection modes, and comprehensive error handling.

## ChromaDB Upload (chroma.py)
- ✅ Multiple connection modes (PersistentClient, HttpClient)
- ✅ 3 embedding strategies (OpenAI, sentence-transformers, default)
- ✅ Batch processing (100 docs per batch)
- ✅ Progress tracking for large uploads
- ✅ Collection management (create if not exists)

## Weaviate Upload (weaviate.py)
- ✅ Local and cloud connections
- ✅ Schema management (auto-create)
- ✅ Batch upload with progress tracking
- ✅ OpenAI embedding support

## Upload Command (upload_skill.py)
- ✅ Added 8 new CLI arguments for vector DBs
- ✅ Platform-specific kwargs handling
- ✅ Enhanced output formatting (collection/class names)
- ✅ Backward compatibility (LLM platforms unchanged)

## Dependencies (pyproject.toml)
- ✅ Added 4 optional dependency groups:
  - chroma = ["chromadb>=0.4.0"]
  - weaviate = ["weaviate-client>=3.25.0"]
  - sentence-transformers = ["sentence-transformers>=2.2.0"]
  - rag-upload = [all vector DB deps]

## Testing (test_upload_integration.py)
- ✅ 15 new tests across 4 test classes
- ✅ Works without optional dependencies installed
- ✅ Error handling tests (missing files, invalid JSON)
- ✅ Fixed 2 existing tests (chroma/weaviate adaptors)
- ✅ 37/37 tests passing

## User-Facing Examples

Local ChromaDB:
  skill-seekers upload output/react-chroma.json --target chroma \
    --persist-directory ./chroma_db

Weaviate Cloud:
  skill-seekers upload output/react-weaviate.json --target weaviate \
    --use-cloud --cluster-url https://xxx.weaviate.network

With OpenAI embeddings:
  skill-seekers upload output/react-chroma.json --target chroma \
    --embedding-function openai --openai-api-key $OPENAI_API_KEY

## Files Changed
- src/skill_seekers/cli/adaptors/chroma.py (250 lines)
- src/skill_seekers/cli/adaptors/weaviate.py (200 lines)
- src/skill_seekers/cli/upload_skill.py (50 lines)
- pyproject.toml (15 lines)
- tests/test_upload_integration.py (NEW - 293 lines)
- tests/test_adaptors/test_chroma_adaptor.py (1 line)
- tests/test_adaptors/test_weaviate_adaptor.py (1 line)

Total: 7 files, ~810 lines added/modified

See PHASE2_COMPLETION_SUMMARY.md for detailed documentation.

Time: ~7 hours (estimated 6-8h)
Status: ✅ COMPLETE - Ready for Phase 3

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

2026-02-08 01:30:04 +03:00

fixtures

Add unified scraping tests and example conflict data

2025-10-29 23:19:32 +03:00

test_adaptors

feat: Phase 2 - Real upload capabilities for ChromaDB and Weaviate

2026-02-08 01:30:04 +03:00

__init__.py

Add comprehensive test system with 71 tests (100% pass rate)

2025-10-19 02:08:58 +03:00

conftest.py

fix: Resolve remaining 188 linting errors (249 total fixed)

2026-01-17 23:02:11 +03:00

docker-compose.test.yml

feat: Add integration testing with real vector databases (Phase 5)

2026-02-07 22:55:02 +03:00

mcp_integration_test.md

Add comprehensive MCP setup guide and integration test template

2025-10-19 17:01:37 +03:00

test_adaptor_benchmarks.py

feat: Add comprehensive performance benchmarking (Phase 4)

2026-02-07 22:51:06 +03:00

test_analyze_command.py

style: Format code with ruff formatter

2026-02-03 21:37:54 +03:00

test_analyze_e2e.py

fix: Resolve PDF processing (#267 ), How-To Guide (#242 ), Chinese README (#260 ) + code quality (#273 )

2026-01-31 21:30:00 +03:00

test_api_reference_builder.py

change max lenght

2026-01-17 17:48:15 +00:00

test_architecture_scenarios.py

style: Run ruff format on 15 files (CI fix)

2026-01-18 00:01:30 +03:00

test_async_scraping.py

style: Run ruff format on 15 files (CI fix)

2026-01-18 00:01:30 +03:00

test_benchmark.py

fix: Enforce min_chunk_size in RAG chunker

2026-02-07 20:59:03 +03:00

test_bootstrap_skill_e2e.py

fix: Resolve remaining 188 linting errors (249 total fixed)

2026-01-17 23:02:11 +03:00

test_bootstrap_skill.py

docs: Update all documentation to use new 'analyze' command

2026-01-29 22:56:33 +03:00

test_c3_integration.py

fix: Fix remaining 61 ruff linting errors (SIM102, SIM117)

2026-01-17 23:25:12 +03:00

test_chunking_integration.py

feat: Complete Phase 1 - RAGChunker integration for all adaptors (v2.11.0)

2026-02-08 00:59:22 +03:00

test_cli_paths.py

fix: Update test version checks to 2.9.0 and remove whitespace

2026-02-03 21:00:34 +03:00

test_cloud_storage.py

fix: Enforce min_chunk_size in RAG chunker

2026-02-07 20:59:03 +03:00

test_code_analyzer.py

run ruff

2026-01-17 17:29:21 +00:00

test_codebase_scraper.py

fix: Remove duplicate documentation directories to save disk space (fixes #279 )

2026-02-05 21:27:41 +03:00

test_config_extractor.py

fix: Resolve remaining 188 linting errors (249 total fixed)

2026-01-17 23:02:11 +03:00

test_config_fetcher.py

fix: Resolve PDF processing (#267 ), How-To Guide (#242 ), Chinese README (#260 ) + code quality (#273 )

2026-01-31 21:30:00 +03:00

test_config_validation.py

change max lenght

2026-01-17 17:48:15 +00:00

test_constants.py

change max lenght

2026-01-17 17:48:15 +00:00

test_dependency_analyzer.py

fix: Resolve 61 critical linting errors

2026-01-17 22:54:40 +03:00

test_e2e_three_stream_pipeline.py

fix: Resolve remaining 188 linting errors (249 total fixed)

2026-01-17 23:02:11 +03:00

test_embedding_pipeline.py

feat: Add custom embedding pipeline (Task #17 )

2026-02-07 13:48:05 +03:00

test_embedding.py

fix: Enforce min_chunk_size in RAG chunker

2026-02-07 20:59:03 +03:00

test_enhance_skill_local.py

Add custom agent validation and tests

2026-02-04 10:14:20 +01:00

test_estimate_pages.py

change max lenght

2026-01-17 17:48:15 +00:00

test_excluded_dirs_config.py

fix: Resolve remaining 188 linting errors (249 total fixed)

2026-01-17 23:02:11 +03:00

test_framework_detection.py

fix: Framework detection now works by including import-only files (fixes #239 )

2026-02-05 22:02:06 +03:00

test_generate_router_github.py

change max lenght

2026-01-17 17:48:15 +00:00

test_git_repo.py

fix: Resolve remaining 188 linting errors (249 total fixed)

2026-01-17 23:02:11 +03:00

test_git_sources_e2e.py

fix: Resolve remaining 188 linting errors (249 total fixed)

2026-01-17 23:02:11 +03:00

test_github_fetcher.py

fix: Resolve remaining 188 linting errors (249 total fixed)

2026-01-17 23:02:11 +03:00

test_github_scraper.py

style: Run ruff format on 15 files (CI fix)

2026-01-18 00:01:30 +03:00

test_guide_enhancer.py

style: Run ruff format on 15 files (CI fix)

2026-01-18 00:01:30 +03:00

test_how_to_guide_builder.py

fix: Resolve PDF processing (#267 ), How-To Guide (#242 ), Chinese README (#260 ) + code quality (#273 )

2026-01-31 21:30:00 +03:00

test_incremental_updates.py

feat: Add incremental updates with change detection (Task #15 )

2026-02-07 13:42:14 +03:00

test_install_agent.py

style: Run ruff format on 15 files (CI fix)

2026-01-18 00:01:30 +03:00

test_install_multiplatform.py

change max lenght

2026-01-17 17:48:15 +00:00

test_install_skill_e2e.py

fix: Resolve remaining 188 linting errors (249 total fixed)

2026-01-17 23:02:11 +03:00

test_install_skill.py

change max lenght

2026-01-17 17:48:15 +00:00

test_integration_adaptors.py

feat: Add integration testing with real vector databases (Phase 5)

2026-02-07 22:55:02 +03:00

test_integration.py

chore: Post-merge cleanup - remove client docs and fix linter errors

2026-01-31 14:58:09 +03:00

test_issue_219_e2e.py

style: Format test_issue_219_e2e.py with ruff

2026-01-18 12:11:01 +03:00

test_issue_277_real_world.py

test: Add real-world integration tests for issue #277 (MikroORM case)

2026-02-04 21:20:23 +03:00

test_language_detector.py

run ruff

2026-01-17 17:29:21 +00:00

test_llms_txt_detector.py

fix: Resolve remaining 188 linting errors (249 total fixed)

2026-01-17 23:02:11 +03:00

test_llms_txt_downloader.py

style: Run ruff format on 15 files (CI fix)

2026-01-18 00:01:30 +03:00

test_llms_txt_parser.py

run ruff

2026-01-17 17:29:21 +00:00

test_markdown_parsing.py

fix: Fix list comprehension variable names (NameError in CI)

2026-01-17 23:33:34 +03:00

test_mcp_fastmcp.py

fix: Resolve remaining 188 linting errors (249 total fixed)

2026-01-17 23:02:11 +03:00

test_mcp_git_sources.py

fix: Resolve remaining 188 linting errors (249 total fixed)

2026-01-17 23:02:11 +03:00

test_mcp_server.py

fix: Resolve remaining 188 linting errors (249 total fixed)

2026-01-17 23:02:11 +03:00

test_mcp_vector_dbs.py

fix: Enforce min_chunk_size in RAG chunker

2026-02-07 20:59:03 +03:00

test_merge_sources_github.py

change max lenght

2026-01-17 17:48:15 +00:00

test_multi_source.py

change max lenght

2026-01-17 17:48:15 +00:00

test_multilang_support.py

feat: Add multi-language documentation support (Task #16 )

2026-02-07 13:45:01 +03:00

test_package_skill.py

change max lenght

2026-01-17 17:48:15 +00:00

test_package_structure.py

fix: Update test version checks to 2.9.0 and remove whitespace

2026-02-03 21:00:34 +03:00

test_parallel_scraping.py

change max lenght

2026-01-17 17:48:15 +00:00

test_pattern_recognizer.py

change max lenght

2026-01-17 17:48:15 +00:00

test_pdf_advanced_features.py

fix: Resolve 61 critical linting errors

2026-01-17 22:54:40 +03:00

test_pdf_extractor.py

Merge PR #275 : Add Dart, Scala, SCSS, SASS, Elixir, Lua, Perl language detection

2026-02-04 21:00:49 +03:00

test_pdf_scraper.py

fix: Remove duplicate import os statements causing UnboundLocalError

2026-01-27 21:11:04 +03:00

test_pr144_concerns.py

run ruff

2026-01-17 17:29:21 +00:00

test_quality_checker.py

change max lenght

2026-01-17 17:48:15 +00:00

test_quality_metrics.py

feat: Add quality metrics dashboard with 4-dimensional scoring (Task #18 - Week 2)

2026-02-07 13:54:44 +03:00

test_rag_chunker.py

feat: Add RAG chunking feature for semantic document splitting (Task 2.1)

2026-02-07 20:53:44 +03:00

test_rate_limit_handler.py

change max lenght

2026-01-17 17:48:15 +00:00

test_real_world_fastmcp.py

fix: Resolve remaining 188 linting errors (249 total fixed)

2026-01-17 23:02:11 +03:00

test_scraper_features.py

change max lenght

2026-01-17 17:48:15 +00:00

test_server_fastmcp_http.py

fix: Fix remaining 61 ruff linting errors (SIM102, SIM117)

2026-01-17 23:25:12 +03:00

test_setup_scripts.py

style: Format test_setup_scripts.py with ruff

2026-01-18 13:48:37 +03:00

test_skip_llms_txt.py

style: Run ruff format on 15 files (CI fix)

2026-01-18 00:01:30 +03:00

test_smart_summarization.py

fix: Resolve remaining 188 linting errors (249 total fixed)

2026-01-17 23:02:11 +03:00

test_source_manager.py

fix: Resolve remaining 188 linting errors (249 total fixed)

2026-01-17 23:02:11 +03:00

test_streaming_ingestion.py

feat: Add streaming ingestion for large docs (Task #14 )

2026-02-07 13:39:43 +03:00

test_swift_detection.py

fix: Resolve remaining 188 linting errors (249 total fixed)

2026-01-17 23:02:11 +03:00

test_terminal_detection.py

fix: Resolve remaining 188 linting errors (249 total fixed)

2026-01-17 23:02:11 +03:00

test_test_example_extractor.py

test: Add GDScript test extraction test case

2026-02-02 23:08:25 +03:00

test_unified_analyzer.py

fix: Resolve remaining 188 linting errors (249 total fixed)

2026-01-17 23:02:11 +03:00

test_unified_mcp_integration.py

fix: Resolve 61 critical linting errors

2026-01-17 22:54:40 +03:00

test_unified.py

fix: Fix remaining 61 ruff linting errors (SIM102, SIM117)

2026-01-17 23:25:12 +03:00

test_upload_integration.py

feat: Phase 2 - Real upload capabilities for ChromaDB and Weaviate

2026-02-08 01:30:04 +03:00

test_upload_skill.py

change max lenght

2026-01-17 17:48:15 +00:00

test_url_conversion.py

fix: Strip anchor fragments in URL conversion to prevent 404 errors (fixes #277 )

2026-02-04 21:16:13 +03:00

test_utilities.py

run ruff

2026-01-17 17:29:21 +00:00