yusyus b475b51ad1 feat: Add custom embedding pipeline (Task #17) ...

- Multi-provider support (OpenAI, Local)
- Batch processing with configurable batch size
- Memory and disk caching for efficiency
- Cost tracking and estimation
- Dimension validation
- 18 tests passing (100%)

Files:
- embedding_pipeline.py: Core pipeline engine
- test_embedding_pipeline.py: Comprehensive tests

Features:
- EmbeddingProvider abstraction
- OpenAIEmbeddingProvider with pricing
- LocalEmbeddingProvider (simulated)
- EmbeddingCache (memory + disk)
- CostTracker for API usage
- Batch processing optimization

Supported Models:
- text-embedding-ada-002 (1536d, $0.10/1M tokens)
- text-embedding-3-small (1536d, $0.02/1M tokens)
- text-embedding-3-large (3072d, $0.13/1M tokens)
- Local models (any dimension, free)

Week 2: 8/9 tasks complete (89%)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

2026-02-07 13:48:05 +03:00

..

adaptors

feat: Add streaming ingestion for large docs (Task #14)

2026-02-07 13:39:43 +03:00

__init__.py

refactor: Centralize version management in single source of truth

2026-01-31 17:03:40 +03:00

ai_enhancer.py

style: Format code with ruff formatter

2026-02-03 21:37:54 +03:00

api_reference_builder.py

change max lenght

2026-01-17 17:48:15 +00:00

architectural_pattern_detector.py

feat: Extend framework detection to 5 languages (JavaScript, Java, Ruby, PHP, C#)

2026-02-05 22:08:37 +03:00

code_analyzer.py

feat: Extend framework detection to 5 languages (JavaScript, Java, Ruby, PHP, C#)

2026-02-05 22:08:37 +03:00

codebase_scraper.py

feat: Add multi-level confidence filtering for pattern detection (fixes #240)

2026-02-05 22:18:27 +03:00

config_command.py

fix: Fix remaining 61 ruff linting errors (SIM102, SIM117)

2026-01-17 23:25:12 +03:00

config_enhancer.py

style: Format code with ruff formatter

2026-02-03 21:37:54 +03:00

config_extractor.py

style: Format code with ruff formatter

2026-02-03 21:37:54 +03:00

config_fetcher.py

fix: Resolve PDF processing (#267), How-To Guide (#242), Chinese README (#260) + code quality (#273)

2026-01-31 21:30:00 +03:00

config_manager.py

feat: C3.9 documentation extraction, AI enhancement optimization, and C# support

2026-01-31 14:56:00 +03:00

config_validator.py

feat: Complete Unity/game engine support and local source type validation

2026-02-02 21:06:01 +03:00

conflict_detector.py

fix: Fix remaining 61 ruff linting errors (SIM102, SIM117)

2026-01-17 23:25:12 +03:00

constants.py

run ruff

2026-01-17 17:29:21 +00:00

dependency_analyzer.py

style: Format code with ruff formatter

2026-02-03 21:37:54 +03:00

doc_scraper.py

fix: Strip anchor fragments in URL conversion to prevent 404 errors (fixes #277)

2026-02-04 21:16:13 +03:00

embedding_pipeline.py

feat: Add custom embedding pipeline (Task #17)

2026-02-07 13:48:05 +03:00

enhance_skill_local.py

Add custom agent validation and tests

2026-02-04 10:14:20 +01:00

enhance_skill.py

change max lenght

2026-01-17 17:48:15 +00:00

enhance_status.py

change max lenght

2026-01-17 17:48:15 +00:00

estimate_pages.py

fix: Fix remaining 61 ruff linting errors (SIM102, SIM117)

2026-01-17 23:25:12 +03:00

generate_router.py

fix: Resolve remaining 188 linting errors (249 total fixed)

2026-01-17 23:02:11 +03:00

github_fetcher.py

fix: Fix remaining 61 ruff linting errors (SIM102, SIM117)

2026-01-17 23:25:12 +03:00

github_scraper.py

feat: Complete Unity/game engine support and local source type validation

2026-02-02 21:06:01 +03:00

guide_enhancer.py

feat: Add GLM-4.7 support and fix PDF scraper issues (#266)

2026-01-27 21:10:40 +03:00

how_to_guide_builder.py

fix: Handle None ai_analysis in how-to guide builder (PR #247) (#274)

2026-01-31 22:17:19 +03:00

incremental_updater.py

feat: Add incremental updates with change detection (Task #15)

2026-02-07 13:42:14 +03:00

install_agent.py

fix: Resolve remaining 188 linting errors (249 total fixed)

2026-01-17 23:02:11 +03:00

install_skill.py

change max lenght

2026-01-17 17:48:15 +00:00

language_detector.py

docs: Add documentation for 7 new programming languages

2026-02-04 21:01:40 +03:00

llms_txt_detector.py

run ruff

2026-01-17 17:29:21 +00:00

llms_txt_downloader.py

style: Run black formatter on 16 files

2026-01-17 23:56:24 +03:00

llms_txt_parser.py

style: Run ruff format on 15 files (CI fix)

2026-01-18 00:01:30 +03:00

main.py

feat: Add Qdrant vector database adaptor (Task #13)

2026-02-05 23:50:02 +03:00

markdown_cleaner.py

run ruff

2026-01-17 17:29:21 +00:00

merge_sources.py

change max lenght

2026-01-17 17:48:15 +00:00

multilang_support.py

feat: Add multi-language documentation support (Task #16)

2026-02-07 13:45:01 +03:00

package_multi.py

run ruff

2026-01-17 17:29:21 +00:00

package_skill.py

feat: Add streaming ingestion for large docs (Task #14)

2026-02-07 13:39:43 +03:00

pattern_recognizer.py

feat: Add multi-level confidence filtering for pattern detection (fixes #240)

2026-02-05 22:18:27 +03:00

pdf_extractor_poc.py

fix: Resolve PDF processing (#267), How-To Guide (#242), Chinese README (#260) + code quality (#273)

2026-01-31 21:30:00 +03:00

pdf_scraper.py

style: Format code with ruff

2026-01-27 21:11:21 +03:00

quality_checker.py

fix: Fix list comprehension variable names (NameError in CI)

2026-01-17 23:33:34 +03:00

rate_limit_handler.py

change max lenght

2026-01-17 17:48:15 +00:00

resume_command.py

fix: Resolve remaining 188 linting errors (249 total fixed)

2026-01-17 23:02:11 +03:00

run_tests.py

change max lenght

2026-01-17 17:48:15 +00:00

setup_wizard.py

fix: Fix remaining 61 ruff linting errors (SIM102, SIM117)

2026-01-17 23:25:12 +03:00

signal_flow_analyzer.py

style: Format code with ruff formatter

2026-02-03 21:37:54 +03:00

split_config.py

change max lenght

2026-01-17 17:48:15 +00:00

streaming_ingest.py

feat: Add streaming ingestion for large docs (Task #14)

2026-02-07 13:39:43 +03:00

swift_patterns.py

change max lenght

2026-01-17 17:48:15 +00:00

test_example_extractor.py

style: Format code with ruff formatter

2026-02-03 21:37:54 +03:00

test_unified_simple.py

fix: Fix remaining 61 ruff linting errors (SIM102, SIM117)

2026-01-17 23:25:12 +03:00

unified_codebase_analyzer.py

style: Run ruff format on 15 files (CI fix)

2026-01-18 00:01:30 +03:00

unified_scraper.py

feat: Add GLM-4.7 support and fix PDF scraper issues (#266)

2026-01-27 21:10:40 +03:00

unified_skill_builder.py

fix: Resolve PDF processing (#267), How-To Guide (#242), Chinese README (#260) + code quality (#273)

2026-01-31 21:30:00 +03:00

upload_skill.py

fix: Resolve 61 critical linting errors

2026-01-17 22:54:40 +03:00

utils.py

fix: Resolve 61 critical linting errors

2026-01-17 22:54:40 +03:00