yusyus 5ce3ed4067 feat: Add streaming ingestion for large docs (Task #14) ...

- Memory-efficient streaming with chunking
- Progress tracking with real-time stats
- Batch processing and resume capability
- CLI integration with --streaming flag
- 10 tests passing (100%)

Files:
- streaming_ingest.py: Core streaming engine
- streaming_adaptor.py: Adaptor integration
- package_skill.py: CLI flags added
- test_streaming_ingestion.py: Comprehensive tests

Week 2: 5/9 tasks complete (56%)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

2026-02-07 13:39:43 +03:00

..

adaptors

feat: Add streaming ingestion for large docs (Task #14)

2026-02-07 13:39:43 +03:00

__init__.py

refactor: Centralize version management in single source of truth

2026-01-31 17:03:40 +03:00

ai_enhancer.py

style: Format code with ruff formatter

2026-02-03 21:37:54 +03:00

api_reference_builder.py

change max lenght

2026-01-17 17:48:15 +00:00

architectural_pattern_detector.py

feat: Extend framework detection to 5 languages (JavaScript, Java, Ruby, PHP, C#)

2026-02-05 22:08:37 +03:00

code_analyzer.py

feat: Extend framework detection to 5 languages (JavaScript, Java, Ruby, PHP, C#)

2026-02-05 22:08:37 +03:00

codebase_scraper.py

feat: Add multi-level confidence filtering for pattern detection (fixes #240)

2026-02-05 22:18:27 +03:00

config_command.py

fix: Fix remaining 61 ruff linting errors (SIM102, SIM117)

2026-01-17 23:25:12 +03:00

config_enhancer.py

style: Format code with ruff formatter

2026-02-03 21:37:54 +03:00

config_extractor.py

style: Format code with ruff formatter

2026-02-03 21:37:54 +03:00

config_fetcher.py

fix: Resolve PDF processing (#267), How-To Guide (#242), Chinese README (#260) + code quality (#273)

2026-01-31 21:30:00 +03:00

config_manager.py

feat: C3.9 documentation extraction, AI enhancement optimization, and C# support

2026-01-31 14:56:00 +03:00

config_validator.py

feat: Complete Unity/game engine support and local source type validation

2026-02-02 21:06:01 +03:00

conflict_detector.py

fix: Fix remaining 61 ruff linting errors (SIM102, SIM117)

2026-01-17 23:25:12 +03:00

constants.py

run ruff

2026-01-17 17:29:21 +00:00

dependency_analyzer.py

style: Format code with ruff formatter

2026-02-03 21:37:54 +03:00

doc_scraper.py

fix: Strip anchor fragments in URL conversion to prevent 404 errors (fixes #277)

2026-02-04 21:16:13 +03:00

enhance_skill_local.py

Add custom agent validation and tests

2026-02-04 10:14:20 +01:00

enhance_skill.py

change max lenght

2026-01-17 17:48:15 +00:00

enhance_status.py

change max lenght

2026-01-17 17:48:15 +00:00

estimate_pages.py

fix: Fix remaining 61 ruff linting errors (SIM102, SIM117)

2026-01-17 23:25:12 +03:00

generate_router.py

fix: Resolve remaining 188 linting errors (249 total fixed)

2026-01-17 23:02:11 +03:00

github_fetcher.py

fix: Fix remaining 61 ruff linting errors (SIM102, SIM117)

2026-01-17 23:25:12 +03:00

github_scraper.py

feat: Complete Unity/game engine support and local source type validation

2026-02-02 21:06:01 +03:00

guide_enhancer.py

feat: Add GLM-4.7 support and fix PDF scraper issues (#266)

2026-01-27 21:10:40 +03:00

how_to_guide_builder.py

fix: Handle None ai_analysis in how-to guide builder (PR #247) (#274)

2026-01-31 22:17:19 +03:00

install_agent.py

fix: Resolve remaining 188 linting errors (249 total fixed)

2026-01-17 23:02:11 +03:00

install_skill.py

change max lenght

2026-01-17 17:48:15 +00:00

language_detector.py

docs: Add documentation for 7 new programming languages

2026-02-04 21:01:40 +03:00

llms_txt_detector.py

run ruff

2026-01-17 17:29:21 +00:00

llms_txt_downloader.py

style: Run black formatter on 16 files

2026-01-17 23:56:24 +03:00

llms_txt_parser.py

style: Run ruff format on 15 files (CI fix)

2026-01-18 00:01:30 +03:00

main.py

feat: Add Qdrant vector database adaptor (Task #13)

2026-02-05 23:50:02 +03:00

markdown_cleaner.py

run ruff

2026-01-17 17:29:21 +00:00

merge_sources.py

change max lenght

2026-01-17 17:48:15 +00:00

package_multi.py

run ruff

2026-01-17 17:29:21 +00:00

package_skill.py

feat: Add streaming ingestion for large docs (Task #14)

2026-02-07 13:39:43 +03:00

pattern_recognizer.py

feat: Add multi-level confidence filtering for pattern detection (fixes #240)

2026-02-05 22:18:27 +03:00

pdf_extractor_poc.py

fix: Resolve PDF processing (#267), How-To Guide (#242), Chinese README (#260) + code quality (#273)

2026-01-31 21:30:00 +03:00

pdf_scraper.py

style: Format code with ruff

2026-01-27 21:11:21 +03:00

quality_checker.py

fix: Fix list comprehension variable names (NameError in CI)

2026-01-17 23:33:34 +03:00

rate_limit_handler.py

change max lenght

2026-01-17 17:48:15 +00:00

resume_command.py

fix: Resolve remaining 188 linting errors (249 total fixed)

2026-01-17 23:02:11 +03:00

run_tests.py

change max lenght

2026-01-17 17:48:15 +00:00

setup_wizard.py

fix: Fix remaining 61 ruff linting errors (SIM102, SIM117)

2026-01-17 23:25:12 +03:00

signal_flow_analyzer.py

style: Format code with ruff formatter

2026-02-03 21:37:54 +03:00

split_config.py

change max lenght

2026-01-17 17:48:15 +00:00

streaming_ingest.py

feat: Add streaming ingestion for large docs (Task #14)

2026-02-07 13:39:43 +03:00

swift_patterns.py

change max lenght

2026-01-17 17:48:15 +00:00

test_example_extractor.py

style: Format code with ruff formatter

2026-02-03 21:37:54 +03:00

test_unified_simple.py

fix: Fix remaining 61 ruff linting errors (SIM102, SIM117)

2026-01-17 23:25:12 +03:00

unified_codebase_analyzer.py

style: Run ruff format on 15 files (CI fix)

2026-01-18 00:01:30 +03:00

unified_scraper.py

feat: Add GLM-4.7 support and fix PDF scraper issues (#266)

2026-01-27 21:10:40 +03:00

unified_skill_builder.py

fix: Resolve PDF processing (#267), How-To Guide (#242), Chinese README (#260) + code quality (#273)

2026-01-31 21:30:00 +03:00

upload_skill.py

fix: Resolve 61 critical linting errors

2026-01-17 22:54:40 +03:00

utils.py

fix: Resolve 61 critical linting errors

2026-01-17 22:54:40 +03:00