diff --git a/CLAUDE.md b/CLAUDE.md index a15cfcc..64ab92e 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -15,7 +15,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co - **โœ… CI/CD Fixed**: All 5 test matrix jobs passing (Ubuntu + macOS, Python 3.10-3.12) - **๐Ÿ“š Documentation Complete**: README, CHANGELOG, FUTURE_RELEASES.md all updated - **๐Ÿš€ Unified CLI**: Single `skill-seekers` command with Git-style subcommands -- **๐Ÿงช Test Coverage**: 379 tests passing, 39% coverage +- **๐Ÿงช Test Coverage**: 391 tests passing, 39% coverage - **๐ŸŒ Community**: GitHub Discussion, Release notes, announcements published **๐Ÿš€ Unified Multi-Source Scraping (v2.0.0)** @@ -23,7 +23,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co - **NEW**: Automatic conflict detection between docs and code - **NEW**: Rule-based and AI-powered merging - **NEW**: 5 example unified configs (React, Django, FastAPI, Godot, FastAPI-test) -- **Status**: โš ๏ธ 12 unified tests need fixes (core functionality stable) +- **Status**: โœ… All 22 unified tests passing (18 core + 4 MCP integration) **โœ… Community Response (H1 Group):** - **Issue #8 Fixed** - Added BULLETPROOF_QUICKSTART.md and TROUBLESHOOTING.md for beginners @@ -40,16 +40,17 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co - ๐Ÿ“ Multi-source configs: django_unified, fastapi_unified, fastapi_unified_test, godot_unified, react_unified - ๐Ÿ“ Test/Example configs: godot_github, react_github, python-tutorial-test, example_pdf, test-manual -**๐Ÿ“‹ Next Up (Post-PyPI v2.0.0):** -- **โœ… DONE**: PyPI publication complete +**๐Ÿ“‹ Next Up (Post-v2.1.0):** +- **โœ… DONE**: PyPI publication complete (v2.0.0) - **โœ… DONE**: CI/CD fixed - all checks passing - **โœ… DONE**: Documentation updated (README, CHANGELOG, FUTURE_RELEASES.md) -- **Priority 1**: Fix 12 failing unified tests in tests/test_unified.py - - ConfigValidator expecting dict instead of file path - - ConflictDetector expecting dict pages, not list +- **โœ… DONE**: Quality Assurance + Race Condition Fixes (v2.1.0) +- **โœ… DONE**: All critical bugs fixed (Issues #190, #192, #193) +- **โœ… DONE**: Test suite stabilized (391 tests passing) +- **โœ… DONE**: Unified tests fixed (all 22 passing) +- **Priority 1**: Review and merge open PRs (#195, #196, #197, #198) - **Priority 2**: Task H1.3 - Create example project folder - **Priority 3**: Task A3.1 - GitHub Pages site (skillseekersweb.com) -- **Priority 4**: Task J1.1 - Install MCP package for testing **๐Ÿ“Š Roadmap Progress:** - 134 tasks organized into 22 feature groups @@ -325,12 +326,13 @@ Skill_Seekers/ โ”‚ โ”‚ โ””โ”€โ”€ conflict_detector.py # Conflict detection โ”‚ โ””โ”€โ”€ mcp/ # MCP server integration โ”‚ โ””โ”€โ”€ server.py -โ”œโ”€โ”€ tests/ # Test suite (379 tests passing) +โ”œโ”€โ”€ tests/ # Test suite (391 tests passing) โ”‚ โ”œโ”€โ”€ test_scraper_features.py โ”‚ โ”œโ”€โ”€ test_config_validation.py โ”‚ โ”œโ”€โ”€ test_integration.py โ”‚ โ”œโ”€โ”€ test_mcp_server.py -โ”‚ โ”œโ”€โ”€ test_unified.py # (12 tests need fixes) +โ”‚ โ”œโ”€โ”€ test_unified.py # Unified scraping tests (18 tests) +โ”‚ โ”œโ”€โ”€ test_unified_mcp_integration.py # (4 tests) โ”‚ โ””โ”€โ”€ ... โ”œโ”€โ”€ configs/ # Preset configurations (24 configs) โ”‚ โ”œโ”€โ”€ godot.json @@ -743,11 +745,11 @@ The correct command uses the local `cli/package_skill.py` in the repository root - โœ… `claude-code.json` - Claude Code documentation **NEW!** ### Unified Multi-Source Configs (5 configs - **NEW v2.0!**) -- โš ๏ธ `react_unified.json` - React (docs + GitHub + code analysis) -- โš ๏ธ `django_unified.json` - Django (docs + GitHub + code analysis) -- โš ๏ธ `fastapi_unified.json` - FastAPI (docs + GitHub + code analysis) -- โš ๏ธ `fastapi_unified_test.json` - FastAPI test config -- โš ๏ธ `godot_unified.json` - Godot (docs + GitHub + code analysis) +- โœ… `react_unified.json` - React (docs + GitHub + code analysis) +- โœ… `django_unified.json` - Django (docs + GitHub + code analysis) +- โœ… `fastapi_unified.json` - FastAPI (docs + GitHub + code analysis) +- โœ… `fastapi_unified_test.json` - FastAPI test config +- โœ… `godot_unified.json` - Godot (docs + GitHub + code analysis) ### Test/Example Configs (5 configs) - ๐Ÿ“ `godot_github.json` - GitHub-only scraping example @@ -756,8 +758,8 @@ The correct command uses the local `cli/package_skill.py` in the repository root - ๐Ÿ“ `example_pdf.json` - PDF extraction example - ๐Ÿ“ `test-manual.json` - Manual testing config -**Note:** โš ๏ธ = Unified configs have 12 failing tests that need fixing -**Last verified:** November 11, 2025 (v2.0.0 PyPI release) +**Note:** All configs verified and working! Unified configs fully tested with 22 passing tests. +**Last verified:** November 29, 2025 (Post-v2.1.0 bug fixes) ## Additional Documentation @@ -789,7 +791,7 @@ The correct command uses the local `cli/package_skill.py` in the repository root - โœ… **Modern Python Packaging**: pyproject.toml, src/ layout, entry points - โœ… **Unified CLI**: Single `skill-seekers` command with Git-style subcommands - โœ… **CI/CD Working**: All 5 test matrix jobs passing (Ubuntu + macOS, Python 3.10-3.12) -- โœ… **Test Coverage**: 379 tests passing, 39% coverage +- โœ… **Test Coverage**: 391 tests passing, 39% coverage - โœ… **Documentation**: Complete user and technical documentation **Architecture:** @@ -801,7 +803,7 @@ The correct command uses the local `cli/package_skill.py` in the repository root **Development Workflow:** 1. **Install**: `pip install -e .` (editable mode for development) -2. **Run tests**: `pytest tests/` (379 tests) +2. **Run tests**: `pytest tests/` (391 tests) 3. **Build package**: `uv build` or `python -m build` 4. **Publish**: `uv publish` (PyPI) diff --git a/README.md b/README.md index 1b3747a..2088137 100644 --- a/README.md +++ b/README.md @@ -86,7 +86,7 @@ Skill Seeker is an automated tool that transforms documentation websites, GitHub - โœ… **Caching System** - Scrape once, rebuild instantly ### โœ… Quality Assurance -- โœ… **Fully Tested** - 379 tests with comprehensive coverage +- โœ… **Fully Tested** - 391 tests with comprehensive coverage --- diff --git a/docs/CLAUDE.md b/docs/CLAUDE.md index 1bc3014..e5630ec 100644 --- a/docs/CLAUDE.md +++ b/docs/CLAUDE.md @@ -326,6 +326,40 @@ print(soup.select_one('main')) print(soup.select_one('div[role="main"]')) ``` +## Running Tests + +**IMPORTANT: You must install the package before running tests** + +```bash +# 1. Install package in editable mode (one-time setup) +pip install -e . + +# 2. Run all tests +pytest + +# 3. Run specific test files +pytest tests/test_config_validation.py +pytest tests/test_github_scraper.py + +# 4. Run with verbose output +pytest -v + +# 5. Run with coverage report +pytest --cov=src/skill_seekers --cov-report=html +``` + +**Why install first?** +- Tests import from `skill_seekers.cli` which requires the package to be installed +- Modern Python packaging best practice (PEP 517/518) +- CI/CD automatically installs with `pip install -e .` +- conftest.py will show helpful error if package not installed + +**Test Coverage:** +- 391+ tests passing +- 39% code coverage +- All core features tested +- CI/CD tests on Ubuntu + macOS with Python 3.10-3.12 + ## Troubleshooting **No content extracted**: Check `main_content` selector. Common values: `article`, `main`, `div[role="main"]`, `div.content` diff --git a/src/skill_seekers/cli/github_scraper.py b/src/skill_seekers/cli/github_scraper.py index 7a59253..67a38f3 100644 --- a/src/skill_seekers/cli/github_scraper.py +++ b/src/skill_seekers/cli/github_scraper.py @@ -31,6 +31,13 @@ except ImportError: print("Error: PyGithub not installed. Run: pip install PyGithub") sys.exit(1) +# Configure logging FIRST (before using logger) +logging.basicConfig( + level=logging.INFO, + format='%(asctime)s - %(levelname)s - %(message)s' +) +logger = logging.getLogger(__name__) + # Import code analyzer for deep code analysis try: from .code_analyzer import CodeAnalyzer @@ -39,13 +46,6 @@ except ImportError: CODE_ANALYZER_AVAILABLE = False logger.warning("Code analyzer not available - deep analysis disabled") -# Configure logging -logging.basicConfig( - level=logging.INFO, - format='%(asctime)s - %(levelname)s - %(message)s' -) -logger = logging.getLogger(__name__) - class GitHubScraper: """ diff --git a/tests/conftest.py b/tests/conftest.py index 5c432a0..77d483d 100644 --- a/tests/conftest.py +++ b/tests/conftest.py @@ -2,11 +2,28 @@ Pytest configuration for tests. Configures anyio to only use asyncio backend (not trio). +Checks that the skill_seekers package is installed before running tests. """ +import sys import pytest +def pytest_configure(config): + """Check if package is installed before running tests.""" + try: + import skill_seekers + except ModuleNotFoundError: + print("\n" + "=" * 70) + print("ERROR: skill_seekers package not installed") + print("=" * 70) + print("\nPlease install the package in editable mode first:") + print(" pip install -e .") + print("\nOr activate your virtual environment if you already installed it.") + print("=" * 70 + "\n") + sys.exit(1) + + @pytest.fixture(scope="session") def anyio_backend(): """Override anyio backend to only use asyncio (not trio).""" diff --git a/tests/test_constants.py b/tests/test_constants.py index 0f81b74..0eef01f 100644 --- a/tests/test_constants.py +++ b/tests/test_constants.py @@ -109,14 +109,14 @@ class TestConstantsUsage(unittest.TestCase): def test_doc_scraper_imports_constants(self): """Test that doc_scraper imports and uses constants.""" - from cli import doc_scraper + from skill_seekers.cli import doc_scraper # Check that doc_scraper can access the constants self.assertTrue(hasattr(doc_scraper, 'DEFAULT_RATE_LIMIT')) self.assertTrue(hasattr(doc_scraper, 'DEFAULT_MAX_PAGES')) def test_estimate_pages_imports_constants(self): """Test that estimate_pages imports and uses constants.""" - from cli import estimate_pages + from skill_seekers.cli import estimate_pages # Verify function signature uses constants import inspect sig = inspect.signature(estimate_pages.estimate_pages) @@ -125,7 +125,7 @@ class TestConstantsUsage(unittest.TestCase): def test_enhance_skill_imports_constants(self): """Test that enhance_skill imports constants.""" try: - from cli import enhance_skill + from skill_seekers.cli import enhance_skill # Check module loads without errors self.assertIsNotNone(enhance_skill) except (ImportError, SystemExit) as e: @@ -135,7 +135,7 @@ class TestConstantsUsage(unittest.TestCase): def test_enhance_skill_local_imports_constants(self): """Test that enhance_skill_local imports constants.""" - from cli import enhance_skill_local + from skill_seekers.cli import enhance_skill_local self.assertIsNotNone(enhance_skill_local) @@ -144,7 +144,7 @@ class TestConstantsExports(unittest.TestCase): def test_all_exports_exist(self): """Test that all items in __all__ exist.""" - from cli import constants + from skill_seekers.cli import constants self.assertTrue(hasattr(constants, '__all__')) for name in constants.__all__: self.assertTrue( @@ -154,7 +154,7 @@ class TestConstantsExports(unittest.TestCase): def test_all_exports_count(self): """Test that __all__ has expected number of exports.""" - from cli import constants + from skill_seekers.cli import constants # We defined 18 constants (added DEFAULT_ASYNC_MODE) self.assertEqual(len(constants.__all__), 18) diff --git a/tests/test_llms_txt_detector.py b/tests/test_llms_txt_detector.py index 5d474ac..68c8b43 100644 --- a/tests/test_llms_txt_detector.py +++ b/tests/test_llms_txt_detector.py @@ -6,7 +6,7 @@ def test_detect_llms_txt_variants(): """Test detection of llms.txt file variants""" detector = LlmsTxtDetector("https://hono.dev/docs") - with patch('cli.llms_txt_detector.requests.head') as mock_head: + with patch('skill_seekers.cli.llms_txt_detector.requests.head') as mock_head: mock_response = Mock() mock_response.status_code = 200 mock_head.return_value = mock_response @@ -22,7 +22,7 @@ def test_detect_no_llms_txt(): """Test detection when no llms.txt file exists""" detector = LlmsTxtDetector("https://example.com/docs") - with patch('cli.llms_txt_detector.requests.head') as mock_head: + with patch('skill_seekers.cli.llms_txt_detector.requests.head') as mock_head: mock_response = Mock() mock_response.status_code = 404 mock_head.return_value = mock_response @@ -36,7 +36,7 @@ def test_url_parsing_with_complex_paths(): """Test URL parsing handles non-standard paths correctly""" detector = LlmsTxtDetector("https://example.com/docs/v2/guide") - with patch('cli.llms_txt_detector.requests.head') as mock_head: + with patch('skill_seekers.cli.llms_txt_detector.requests.head') as mock_head: mock_response = Mock() mock_response.status_code = 200 mock_head.return_value = mock_response @@ -55,7 +55,7 @@ def test_detect_all_variants(): """Test detecting all llms.txt variants""" detector = LlmsTxtDetector("https://hono.dev/docs") - with patch('cli.llms_txt_detector.requests.head') as mock_head: + with patch('skill_seekers.cli.llms_txt_detector.requests.head') as mock_head: # Mock responses for different variants def mock_response(url, **kwargs): response = Mock()