yusyus a8ab462930 test: Add real-world integration tests for issue #277 (MikroORM case) ...

Added comprehensive integration tests using the exact MikroORM URLs that
caused 404 errors in the original bug report.

Test Coverage (6 integration tests):
1. test_mikro_orm_urls_from_issue_277
   - Tests exact URLs from the bug report
   - Verifies no malformed anchor fragments in results
   - Validates deduplication and correct URL transformation

2. test_no_404_causing_urls_generated
   - Verifies no URLs matching the 404 error pattern are generated
   - Tests all problematic patterns from the issue

3. test_deduplication_prevents_multiple_requests
   - Validates that multiple anchors on same page deduplicate correctly
   - Ensures bandwidth savings

4. test_md_files_with_anchors_preserved
   - Tests .md files with anchors are handled correctly
   - Verifies anchor stripping on .md URLs

5. test_real_scraping_scenario_no_404s
   - Integration test simulating full llms.txt parsing flow
   - Validates URL structure with regex patterns

6. test_issue_277_error_message_urls
   - Tests the exact malformed URLs from error output
   - Verifies correct URLs are generated instead

Results:
- 18/18 tests passing (12 unit + 6 integration)
- All MikroORM URLs from issue #277 handled correctly
- No 404-causing patterns generated

Related: #277

2026-02-04 21:20:23 +03:00

11 KiB

Raw Blame History

View Raw