feat: C3.9 documentation extraction, AI enhancement optimization, and C# support

Complete implementation of C3.9, granular AI enhancement control, performance optimizations, and bug fixes.

Features:
- C3.9 Project Documentation Extraction (markdown files)
- Granular AI enhancement control (--enhance-level 0-3)
- C# test extraction support
- 6-12x faster LOCAL mode with parallel execution
- Auto-enhancement UX improvements
- LOCAL mode fallback for all AI enhancements

Bug Fixes:
- C# language support
- Config type field compatibility
- LocalSkillEnhancer import

Documentation:
- Updated CHANGELOG.md
- Updated CLAUDE.md
- Removed client-specific files

Tests: All 1,257 tests passing
Critical linter errors: Fixed
This commit is contained in:
YusufKaraaslanSpyke
2026-01-31 14:56:00 +03:00
committed by GitHub
parent 5a78522dbc
commit aa57164d34
14 changed files with 3025 additions and 145 deletions

View File

@@ -297,9 +297,17 @@ skill-seekers analyze --directory . --skip-patterns --skip-how-to-guides
```
- Generates 300+ line standalone SKILL.md files from codebases
- All C3.x features integrated (patterns, tests, guides, config, architecture)
- All C3.x features integrated (patterns, tests, guides, config, architecture, docs)
- Complete codebase analysis without documentation scraping
**C3.9 Project Documentation Extraction** (`codebase_scraper.py`):
- Extracts and categorizes all markdown files from the project
- Auto-detects categories: overview, architecture, guides, workflows, features, etc.
- Integrates documentation into SKILL.md with summaries
- AI enhancement (level 2+) adds topic extraction and cross-references
- Controlled by depth: surface=raw copy, deep=parse+summarize, full=AI-enhanced
- Default ON, use `--skip-docs` to disable
**Key Architecture Decision (BREAKING in v2.5.2):**
- Changed from opt-in (`--build-*`) to opt-out (`--skip-*`) flags
- All analysis features now ON by default for maximum value

View File

@@ -0,0 +1,265 @@
# Spyke Games - Skill Seekers Integration Notes
> Discussion notes for Claude Code + Skill Seekers integration at Spyke Games
> Date: 2026-01-06
---
## Current State Analysis
### What They Have (Excellent Foundation)
```
knit-game-client/docs/
├── workflows/
│ └── feature-development-workflow.md # Complete dev workflow
├── templates/
│ ├── ANALYSIS-CHECKLIST.md # 13-section feature analysis
│ ├── DESIGN-TEMPLATE.md # Feature design template
│ ├── TDD-TEMPLATE.md # Technical design doc
│ ├── PR-CHECKLIST.md # Review checklist with pitfalls
│ └── ISSUE-TEMPLATE.md # GitHub issue structure
└── features/
└── area-cover-blocker/ # Example complete feature
├── DESIGN.md # 549 lines, comprehensive
├── EDGE-CASES.md
├── TASKS.md
└── TDD.md
```
### Key Observations
1. **Already using Claude Code skill references** in docs:
- `/knitgame-core` - Core gameplay patterns
- `/threadbox-blocker` - Grid blocker patterns
2. **Documented Common Pitfalls** (PR-CHECKLIST.md):
- UnityEngine in Controller/Model (MVC violation)
- Stale references after async
- Memory leaks from events (missing Dispose)
- Animation ID leaks (missing try-finally)
- Missing PrepareForReuse state reset
- Double-despawn race conditions
- Play-on under-restoration
3. **MVC Layer Rules** (CRITICAL):
| Layer | UnityEngine | Purpose |
|-------|-------------|---------|
| Model | NO | Pure C# data, state, logic |
| Controller | NO | Business logic, orchestration |
| View | YES | MonoBehaviour, visuals |
| Service | YES | Business logic needing Unity APIs |
4. **Test Patterns**:
- Reflection-based DI injection (no Zenject in tests)
- NSubstitute for mocking
- Real models, mocked dependencies
---
## Proposed Skill Layer Architecture
### Layer 1: Workflow Skills (HOW to develop)
| Skill | Source | Purpose |
|-------|--------|---------|
| `yarn-flow-workflow` | `docs/workflows/` | Feature development lifecycle |
| `yarn-flow-analysis` | `ANALYSIS-CHECKLIST.md` | Feature analysis patterns |
| `yarn-flow-pr-review` | `PR-CHECKLIST.md` | Review checklist, pitfalls |
| `yarn-flow-testing` | Test files + templates | Test patterns, reflection DI |
### Layer 2: Pattern Skills (WHAT to implement)
| Skill | Source | Purpose |
|-------|--------|---------|
| `yarn-flow-mvc` | Workflow docs + code | MVC layer rules |
| `yarn-flow-blockers` | Blocker implementations | Grid/Yarn/Bottom patterns |
| `yarn-flow-boosters` | Booster implementations | Booster patterns |
| `yarn-flow-async` | Code patterns | UniTask, cancellation, safety |
| `yarn-flow-pooling` | Generators | ObjectPool, PrepareForReuse |
| `yarn-flow-events` | Controllers | Event lifecycle (Init/Dispose) |
| `yarn-flow-di` | Installers | Zenject binding patterns |
### Layer 3: Reference Skills (Examples to follow)
| Skill | Source | Purpose |
|-------|--------|---------|
| `yarn-flow-threadbox` | ThreadBox implementation | Reference grid blocker |
| `yarn-flow-mystery` | Mystery implementation | Reference yarn blocker |
| `yarn-flow-areacover` | AreaCover + DESIGN.md | Recent, fully documented |
---
## Proposed Agent Architecture
### 1. Feature Analysis Agent
```
Trigger: "analyze feature {X}" or "what base class for {X}"
Skills: yarn-flow-analysis, yarn-flow-blockers, yarn-flow-boosters
Action:
- Runs ANALYSIS-CHECKLIST programmatically
- Identifies feature type (Grid/Yarn/Bottom Blocker, Booster)
- Suggests base class
- Maps system interactions
- Identifies edge cases
- Outputs gap analysis
```
### 2. Design Document Agent
```
Trigger: "create design doc for {X}" or when starting new feature
Skills: yarn-flow-workflow, yarn-flow-blockers, yarn-flow-reference
Action:
- Creates docs/features/{feature}/DESIGN.md from template
- Pre-populates interaction matrix based on feature type
- Suggests edge cases from similar features
- Creates EDGE-CASES.md skeleton
```
### 3. PR Review Agent
```
Trigger: PR created, "review PR", or pre-commit hook
Skills: yarn-flow-pr-review, yarn-flow-mvc, yarn-flow-async
Action:
- Scans for UnityEngine imports in Controller/Model
- Verifies IInitializable + IDisposable pair
- Checks event subscription/unsubscription balance
- Validates PrepareForReuse resets all state
- Checks async safety (CancellationToken, try-finally)
- Verifies test coverage for public methods
Output: Review comments with specific line numbers
```
### 4. Code Scaffold Agent
```
Trigger: "implement {type} {name}" after design approved
Skills: yarn-flow-blockers, yarn-flow-di, yarn-flow-pooling
Action:
- Generates Model extending correct base class
- Generates Controller with IInitializable, IDisposable
- Generates ModelGenerator with ObjectPool
- Generates View (MonoBehaviour)
- Adds DI bindings to installer
- Creates test file skeletons
Output: Complete scaffold following all patterns
```
---
## New Grad Pipeline Vision
```
FEATURE REQUEST
┌─────────────────────────────────────────┐
│ 1. ANALYSIS AGENT │
│ "Analyze feature ThreadCutter" │
│ → Suggests GridBlockerBaseModel │
│ → Maps interactions │
│ → Identifies 12 edge cases │
└─────────────────────────────────────────┘
┌─────────────────────────────────────────┐
│ 2. DESIGN AGENT │
│ "Create design doc" │
│ → Generates DESIGN.md (80% complete) │
│ → New grad fills in specifics │
└─────────────────────────────────────────┘
┌─────────────────────────────────────────┐
│ 3. CODE SCAFFOLD AGENT │
│ "Implement ThreadCutter" │
│ → Generates 6 files with patterns │
│ → All boilerplate correct │
│ → New grad fills in business logic │
└─────────────────────────────────────────┘
┌─────────────────────────────────────────┐
│ 4. NEW GRAD CODES │
│ Has correct structure │
│ Just writes the actual logic │
│ Skills loaded = answers questions │
└─────────────────────────────────────────┘
┌─────────────────────────────────────────┐
│ 5. PR REVIEW AGENT │
│ "Review my PR" │
│ → Catches MVC violations │
│ → Verifies async safety │
│ → Checks test coverage │
│ → Feedback before human review │
└─────────────────────────────────────────┘
SENIOR-QUALITY CODE FROM JUNIOR DEV
```
---
## Implementation Priority
### Phase 1: Core Skills (Week 1)
1. Generate skill from `knit-game-client` repo (full codebase)
2. Generate skill from `docs/` folder specifically
3. Install to Claude Code for all devs
### Phase 2: Specialized Skills (Week 2)
1. Split into workflow vs pattern skills
2. Create reference skills from best implementations
3. Test with actual feature development
### Phase 3: Agents (Week 3-4)
1. PR Review Agent (highest ROI - catches common pitfalls)
2. Analysis Agent (helps new devs start correctly)
3. Code Scaffold Agent (reduces boilerplate time)
### Phase 4: CI/CD Integration (Week 5+)
1. PR Review Agent as GitHub Action
2. Auto-regenerate skills when docs change
3. Team-wide skill distribution
---
## Questions to Resolve
1. **Confluence Integration**
- How stale is Confluence vs docs/ folder?
- Should we scrape Confluence or focus on in-repo docs?
- Can we set up sync from Confluence → docs/ → skills?
2. **Skill Granularity**
- One big `yarn-flow` skill vs many small skills?
- Recommendation: Start with 2-3 (workflow, patterns, reference)
- Split more if Claude context gets overloaded
3. **Agent Deployment**
- Local per-developer vs team server?
- GitHub Actions integration?
- Slack/Teams notifications?
4. **SDK Skills**
- Which SDKs cause most pain?
- Firebase? Analytics? Ads? IAP?
- Prioritize based on integration frequency
---
## Related Discussions
- Layered skill architecture (game → framework → external → base)
- New grad onboarding goal: "produce code near our standard"
- Manual review → automated agent review pipeline
- Confluence freshness concerns
---
## Next Steps
1. [ ] Generate skill from knit-game-client repo
2. [ ] Test with actual feature development
3. [ ] Identify highest-pain SDK for skill creation
4. [ ] Design PR Review Agent prompt
5. [ ] Pilot with 1-2 developers

View File

@@ -0,0 +1,774 @@
# Skill & Agent Integration Proposal
## Spyke Games - Claude Code Enhanced Development Workflow
> **Prepared for:** CTO Review
> **Date:** 2026-01-06
> **Status:** Proposal
---
## Executive Summary
This proposal outlines an AI-augmented development workflow using **Claude Code** with custom **Skills** and **Agents** to:
1. **Codify institutional knowledge** into reusable AI skills
2. **Automate quality gates** via specialized agents
3. **Enable pair programming** where Claude implements while developers observe and validate
4. **Ensure consistency** - any developer produces senior-quality code
**Expected Outcome:** New team members can produce production-ready code that follows all architectural patterns, passes review automatically, and matches team standards from day one.
---
## Current Workflow Challenges
| Challenge | Impact | Current Mitigation |
|-----------|--------|-------------------|
| MVC violations (UnityEngine in Controller) | Breaks testability, requires refactoring | Manual code review |
| Async safety issues (stale refs, missing CancellationToken) | Race conditions, hard-to-debug bugs | Senior developer knowledge |
| Missing PrepareForReuse/Dispose | Memory leaks, level replay bugs | PR checklist (manual) |
| Inconsistent patterns across developers | Technical debt accumulation | Documentation (not always read) |
| Onboarding time for new developers | 2-3 months to full productivity | Mentorship, pair programming |
---
## Proposed Solution: Skills + Agents
### What Are Skills?
Skills are **structured knowledge packages** that give Claude Code deep understanding of:
- Our codebase architecture (MVCN, Zenject, UniTask)
- Our coding patterns and conventions
- Our common pitfalls and how to avoid them
- Reference implementations to follow
**When loaded, Claude Code "knows" our codebase like a senior developer.**
### What Are Agents?
Agents are **automated specialists** that perform specific tasks:
- Analyze feature requirements against our architecture
- Generate code following our patterns
- Review PRs for violations before human review
- Scaffold new features with correct boilerplate
**Agents enforce consistency automatically.**
---
## Architecture Overview
```
┌─────────────────────────────────────────────────────────────────────┐
│ SKILL LAYERS │
├─────────────────────────────────────────────────────────────────────┤
│ │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ GAME LAYER - Yarn Flow Specific │ │
│ │ │ │
│ │ Workflow Skills: Pattern Skills: │ │
│ │ ├─ yarn-flow-workflow ├─ yarn-flow-mvc │ │
│ │ ├─ yarn-flow-analysis ├─ yarn-flow-blockers │ │
│ │ ├─ yarn-flow-pr-review ├─ yarn-flow-boosters │ │
│ │ └─ yarn-flow-testing ├─ yarn-flow-async │ │
│ │ ├─ yarn-flow-pooling │ │
│ │ Reference Skills: └─ yarn-flow-events │ │
│ │ ├─ yarn-flow-threadbox │ │
│ │ ├─ yarn-flow-mystery │ │
│ │ └─ yarn-flow-areacover │ │
│ └─────────────────────────────────────────────────────────────┘ │
│ ↓ │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ FRAMEWORK LAYER - UPM Packages │ │
│ │ ├─ upm-spyke-core │ │
│ │ ├─ upm-spyke-services │ │
│ │ ├─ upm-spyke-ui │ │
│ │ └─ upm-spyke-sdks │ │
│ └─────────────────────────────────────────────────────────────┘ │
│ ↓ │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ EXTERNAL LAYER - Third-Party │ │
│ │ ├─ zenject-skill │ │
│ │ ├─ unitask-skill │ │
│ │ ├─ dotween-skill │ │
│ │ └─ addressables-skill │ │
│ └─────────────────────────────────────────────────────────────┘ │
│ ↓ │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ BASE LAYER - Unity │ │
│ │ └─ unity-2022-lts-skill │ │
│ └─────────────────────────────────────────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────┐
│ AGENTS │
├─────────────────────────────────────────────────────────────────────┤
│ │
│ ┌───────────────┐ ┌───────────────┐ ┌───────────────┐ │
│ │ ANALYSIS │ │ SCAFFOLD │ │ PR REVIEW │ │
│ │ AGENT │ │ AGENT │ │ AGENT │ │
│ │ │ │ │ │ │ │
│ │ Analyzes │ │ Generates │ │ Reviews code │ │
│ │ requirements │ │ boilerplate │ │ for violations│ │
│ │ Suggests │ │ Creates files │ │ Catches │ │
│ │ architecture │ │ Adds DI │ │ pitfalls │ │
│ └───────────────┘ └───────────────┘ └───────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────┘
```
---
## Skill Definitions
### Workflow Skills
| Skill | Source | Purpose |
|-------|--------|---------|
| `yarn-flow-workflow` | `docs/workflows/feature-development-workflow.md` | Complete feature development lifecycle, phase gates |
| `yarn-flow-analysis` | `docs/templates/ANALYSIS-CHECKLIST.md` | 13-section feature analysis, system interaction matrix |
| `yarn-flow-pr-review` | `docs/templates/PR-CHECKLIST.md` | Review checklist, 20+ common pitfalls to catch |
| `yarn-flow-testing` | Test patterns from codebase | Reflection DI injection, NSubstitute mocking, test naming |
### Pattern Skills
| Skill | Source | Purpose |
|-------|--------|---------|
| `yarn-flow-mvc` | Controllers, Models, Views | MVC layer rules, UnityEngine boundaries |
| `yarn-flow-blockers` | Blocker implementations | Grid/Yarn/Bottom blocker patterns, base classes |
| `yarn-flow-boosters` | Booster implementations | BoosterControllerBase patterns, lifecycle |
| `yarn-flow-async` | Async code patterns | UniTask, CancellationToken, state revalidation |
| `yarn-flow-pooling` | Generators, PoolObjectBase | ObjectPool usage, PrepareForReuse, OnDespawn |
| `yarn-flow-events` | Controller lifecycle | IInitializable/IDisposable, event subscription balance |
### Reference Skills
| Skill | Source | Purpose |
|-------|--------|---------|
| `yarn-flow-threadbox` | ThreadBox implementation | Reference multi-cell grid blocker |
| `yarn-flow-mystery` | Mystery implementation | Reference yarn blocker with reveal |
| `yarn-flow-areacover` | AreaCover + DESIGN.md | Recent, fully documented blocker |
---
## Agent Specifications
### 1. Analysis Agent
**Purpose:** Analyze feature requirements and map to architecture
**Triggers:**
- "Analyze feature {name}"
- "What base class should I use for {description}"
- Starting any new feature
**Skills Loaded:**
- `yarn-flow-analysis`
- `yarn-flow-blockers`
- `yarn-flow-boosters`
**Input:** Feature name and description/ruleset
**Output:**
```markdown
## Feature Analysis: {Name}
### Classification
- Type: Grid Blocker
- Base Class: GridBlockerBaseModel
- Interface: IGridBlocker
### System Interactions
| System | Interaction | Details |
|--------|-------------|---------|
| Unstitch | Blocks covered cells | Exclude from TryGetUnstitchTarget() |
| Belt | No direct interaction | - |
| Play-On | Counter restoration | Save/restore checkpoint |
### Identified Edge Cases
1. Level ends during destruction animation
2. Multiple instances triggered simultaneously
3. Counter exceeds remaining (clamp to 0)
...
### Similar Implementations
- ThreadBox (multi-cell, direction-based entry)
- KnitCover (single-cell cover)
### Complexity Assessment: Medium
- Requires existing GridBlockerBaseModel patterns
- Direction-based entry adaptation needed
### Next Steps
1. Create DESIGN.md with full specifications
2. Get stakeholder approval on edge case behaviors
3. Proceed to implementation
```
---
### 2. Scaffold Agent
**Purpose:** Generate complete file structure following all patterns
**Triggers:**
- "Implement {type} {name}"
- "Create scaffold for {feature}"
- After design approval
**Skills Loaded:**
- `yarn-flow-blockers` or `yarn-flow-boosters` (based on type)
- `yarn-flow-di`
- `yarn-flow-pooling`
- `yarn-flow-events`
**Input:** Feature type, name, and approved DESIGN.md
**Output Files Generated:**
```
Assets/KnitGame/Scripts/
├── Model/Blockers/
│ └── {Feature}Model.cs
├── Controller/Blockers/{Feature}/
│ ├── {Feature}Controller.cs
│ └── I{Feature}Controller.cs
├── Controller/Generators/
│ └── {Feature}ModelGenerator.cs
├── View/Blockers/{Feature}/
│ ├── {Feature}View.cs
│ └── {Feature}ViewGroup.cs (if multi-cell)
└── Tests/
├── {Feature}ModelTests.cs
└── {Feature}ControllerTests.cs
+ DI bindings added to KnitGameInstaller.cs
```
**Code Quality Guarantees:**
- Models extend correct base class
- Controllers implement IInitializable, IDisposable
- PrepareForReuse implemented with all state reset
- ObjectPool used in generators
- Event subscriptions balanced (subscribe in Initialize, unsubscribe in Dispose)
- No UnityEngine imports in Model/Controller
- Test files with reflection DI helper
---
### 3. PR Review Agent
**Purpose:** Automated code review before human review
**Triggers:**
- PR created
- "Review my PR"
- "Check this code"
- Pre-commit hook (optional)
**Skills Loaded:**
- `yarn-flow-pr-review`
- `yarn-flow-mvc`
- `yarn-flow-async`
- `yarn-flow-pooling`
**Checks Performed:**
| Category | Check | Severity |
|----------|-------|----------|
| **MVC** | UnityEngine import in Controller | FAIL |
| **MVC** | UnityEngine import in Model | FAIL |
| **MVC** | Direct GameObject/Transform in Controller | FAIL |
| **Lifecycle** | IInitializable without IDisposable | WARN |
| **Lifecycle** | Event subscribe without unsubscribe | FAIL |
| **Lifecycle** | Missing PrepareForReuse | WARN |
| **Async** | Async method without CancellationToken | WARN |
| **Async** | State modification after await without check | FAIL |
| **Async** | Animation ID without try-finally | FAIL |
| **Async** | Missing `_gameModel.IsLevelEnded` check | WARN |
| **Pooling** | PoolObjectBase without OnDespawn override | WARN |
| **Pooling** | OnDespawn doesn't reset all fields | WARN |
| **Style** | Debug.Log instead of SpykeLogger | WARN |
| **Style** | Magic numbers without constants | INFO |
| **Testing** | Public method without test coverage | INFO |
**Output Format:**
```markdown
## PR Review: #{PR_NUMBER}
### Summary
- 2 FAIL (must fix)
- 3 WARN (should fix)
- 1 INFO (consider)
### Issues Found
#### FAIL: UnityEngine in Controller
`ThreadCutterController.cs:15`
```csharp
using UnityEngine; // VIOLATION: Controllers must be pure C#
```
**Fix:** Remove UnityEngine dependency, use interface for view interaction
#### FAIL: Missing CancellationToken Check
`ThreadCutterController.cs:89`
```csharp
await PlayAnimation();
UpdateState(); // UNSAFE: State may have changed during await
```
**Fix:** Add cancellation check before state modification:
```csharp
await PlayAnimation();
if (_levelCts.Token.IsCancellationRequested) return;
UpdateState();
```
#### WARN: Event Subscribe Without Unsubscribe
`ThreadCutterController.cs:45`
```csharp
_gridController.OnCellChanged += HandleCellChanged;
```
**Fix:** Add unsubscribe in Dispose():
```csharp
public void Dispose()
{
_gridController.OnCellChanged -= HandleCellChanged;
}
```
### Recommendations
1. Fix both FAIL issues before merge
2. Address WARN issues to prevent technical debt
3. Consider INFO items for code quality improvement
```
---
## Development Workflow with Agents
### Complete Feature Development Flow
```
┌─────────────────────────────────────────────────────────────────────┐
│ FEATURE REQUEST RECEIVED │
│ "Implement ThreadCutter blocker" │
└─────────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────┐
│ PHASE 1: ANALYSIS │
│ ────────────────── │
│ │
│ Developer: "Analyze feature ThreadCutter - a grid blocker that │
│ cuts threads when unstitch passes through" │
│ │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ ANALYSIS AGENT │ │
│ │ │ │
│ │ • Runs ANALYSIS-CHECKLIST │ │
│ │ • Classifies as Grid Blocker → GridBlockerBaseModel │ │
│ │ • Maps 8 system interactions │ │
│ │ • Identifies 14 edge cases │ │
│ │ • Suggests ThreadBox as reference │ │
│ │ • Complexity: Medium │ │
│ └─────────────────────────────────────────────────────────────┘ │
│ │
│ Developer: Reviews analysis, confirms understanding │
│ │
└─────────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────┐
│ PHASE 2: DESIGN │
│ ─────────────── │
│ │
│ Developer: "Create design document for ThreadCutter" │
│ │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ CLAUDE CODE │ │
│ │ (with skills loaded) │ │
│ │ │ │
│ │ • Creates docs/features/thread-cutter/DESIGN.md │ │
│ │ • Populates from DESIGN-TEMPLATE │ │
│ │ • Fills interaction matrix from analysis │ │
│ │ • Creates EDGE-CASES.md with 14 identified cases │ │
│ │ • Creates TDD.md skeleton │ │
│ └─────────────────────────────────────────────────────────────┘ │
│ │
│ Developer: Reviews design, adds game-specific details │
│ Stakeholders: Approve design document │
│ │
└─────────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────┐
│ PHASE 3: IMPLEMENTATION │
│ ─────────────────────── │
│ │
│ Developer: "Implement ThreadCutter grid blocker per DESIGN.md" │
│ │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ SCAFFOLD AGENT │ │
│ │ │ │
│ │ Generates 8 files: │ │
│ │ • ThreadCutterModel.cs │ │
│ │ • ThreadCutterController.cs + Interface │ │
│ │ • ThreadCutterModelGenerator.cs │ │
│ │ • ThreadCutterView.cs + ViewGroup │ │
│ │ • Test files │ │
│ │ • DI bindings │ │
│ └─────────────────────────────────────────────────────────────┘ │
│ │ │
│ ▼ │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ CLAUDE CODE │ │
│ │ (with skills loaded) │ │
│ │ │ │
│ │ • Implements business logic per DESIGN.md │ │
│ │ • Handles all edge cases │ │
│ │ • Writes comprehensive tests │ │
│ │ • Follows all patterns from loaded skills │ │
│ └─────────────────────────────────────────────────────────────┘ │
│ │
│ Developer: Observes, validates, runs tests, checks edge cases │
│ │
└─────────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────┐
│ PHASE 4: REVIEW │
│ ────────────── │
│ │
│ Developer: "Review my PR" or creates PR │
│ │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ PR REVIEW AGENT │ │
│ │ │ │
│ │ Automated checks: │ │
│ │ ✓ No UnityEngine in Controllers/Models │ │
│ │ ✓ All events properly subscribed/unsubscribed │ │
│ │ ✓ PrepareForReuse resets all state │ │
│ │ ✓ CancellationToken used in async methods │ │
│ │ ✓ Animation IDs cleaned up in finally blocks │ │
│ │ ✓ Tests cover public methods │ │
│ │ │ │
│ │ Result: 0 FAIL, 1 WARN, 2 INFO │ │
│ └─────────────────────────────────────────────────────────────┘ │
│ │
│ Developer: Addresses warnings, creates PR │
│ Senior: Quick review (most issues already caught) │
│ │
└─────────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────┐
│ MERGE & DEPLOY │
│ Production-ready code │
└─────────────────────────────────────────────────────────────────────┘
```
---
## Developer Role Transformation
### Before: Developer as Implementer
```
Developer receives task
→ Reads docs (maybe)
→ Writes code (varies by experience)
→ Makes mistakes (caught in review)
→ Refactors (wastes time)
→ Eventually passes review
Time: 3-5 days for feature
Quality: Depends on developer experience
```
### After: Developer as Validator
```
Developer receives task
→ Analysis Agent analyzes requirements
→ Claude Code creates design docs
→ Developer validates design
→ Scaffold Agent creates structure
→ Claude Code implements logic
→ Developer observes & validates
→ PR Review Agent checks automatically
→ Developer confirms & merges
Time: 1-2 days for feature
Quality: Consistent senior-level regardless of developer experience
```
### Developer Responsibilities
| Responsibility | Description |
|----------------|-------------|
| **Validate Analysis** | Confirm feature classification and edge cases |
| **Review Design** | Ensure design matches requirements |
| **Observe Implementation** | Watch Claude Code work, ask questions |
| **Test Functionality** | Run game, verify feature works correctly |
| **Verify Edge Cases** | Test each edge case from DESIGN.md |
| **Approve PR** | Final check before merge |
---
## Implementation Roadmap
### Phase 1: Foundation (Week 1-2)
| Task | Description | Owner |
|------|-------------|-------|
| Generate `yarn-flow-core` skill | Full codebase analysis | DevOps |
| Generate `yarn-flow-docs` skill | All docs/ content | DevOps |
| Install skills to team Claude Code | All developers | DevOps |
| Test with real feature | Validate skill quality | 1 Developer |
**Deliverable:** Skills working for all developers
### Phase 2: Skill Specialization (Week 3-4)
| Task | Description | Owner |
|------|-------------|-------|
| Split into workflow/pattern skills | Better context targeting | DevOps |
| Create reference skills | ThreadBox, Mystery, AreaCover | DevOps |
| Generate external skills | Zenject, UniTask, DOTween | DevOps |
| Validate skill loading | Test skill combinations | Team |
**Deliverable:** Specialized skills for different tasks
### Phase 3: Agent Development (Week 5-6)
| Task | Description | Owner |
|------|-------------|-------|
| Build PR Review Agent | Automated code checking | DevOps |
| Build Analysis Agent | Feature analysis automation | DevOps |
| Build Scaffold Agent | Code generation | DevOps |
| Integration testing | Agents with skills | Team |
**Deliverable:** Three working agents
### Phase 4: Workflow Integration (Week 7-8)
| Task | Description | Owner |
|------|-------------|-------|
| Update dev workflow docs | Incorporate agents | Tech Lead |
| Train team on new workflow | Hands-on sessions | Tech Lead |
| Pilot with 2-3 features | Real-world validation | Team |
| Iterate based on feedback | Refine agents/skills | DevOps |
**Deliverable:** Production-ready workflow
### Phase 5: CI/CD Integration (Week 9+)
| Task | Description | Owner |
|------|-------------|-------|
| PR Review as GitHub Action | Automated on PR create | DevOps |
| Skill auto-regeneration | When docs/code changes | DevOps |
| Team-wide skill sync | Central skill repository | DevOps |
| Metrics dashboard | Track quality improvements | DevOps |
**Deliverable:** Fully automated quality pipeline
---
## Success Metrics
### Quality Metrics
| Metric | Current | Target | Measurement |
|--------|---------|--------|-------------|
| MVC violations per PR | ~2-3 | 0 | PR Review Agent |
| Async safety issues per PR | ~1-2 | 0 | PR Review Agent |
| PR review iterations | 2-3 | 1 | Git history |
| Bugs from pattern violations | Unknown | -80% | Bug tracking |
### Efficiency Metrics
| Metric | Current | Target | Measurement |
|--------|---------|--------|-------------|
| Time to implement blocker | 3-5 days | 1-2 days | Sprint tracking |
| Code review time | 1-2 hours | 15-30 min | Time tracking |
| Onboarding to productivity | 2-3 months | 2-3 weeks | HR tracking |
### Consistency Metrics
| Metric | Current | Target | Measurement |
|--------|---------|--------|-------------|
| Pattern compliance | ~70% | 98%+ | PR Review Agent |
| Test coverage | Varies | 80%+ | Coverage tools |
| Documentation completeness | Partial | Full | Checklist |
---
## Risk Assessment
| Risk | Probability | Impact | Mitigation |
|------|-------------|--------|------------|
| Skills become stale | Medium | High | Auto-regenerate on code changes |
| Over-reliance on AI | Medium | Medium | Developers still validate all code |
| Agent false positives | Low | Low | Tune thresholds, allow overrides |
| Claude API downtime | Low | Medium | Local fallback, manual workflow |
| Context size limits | Medium | Low | Split skills, load contextually |
---
## Resource Requirements
### Tools
| Tool | Purpose | Cost |
|------|---------|------|
| Claude Code (Max) | AI pair programming | Existing subscription |
| Skill Seekers | Skill generation | Open source (free) |
| GitHub Actions | CI/CD integration | Existing |
### Time Investment
| Role | Initial Setup | Ongoing |
|------|---------------|---------|
| DevOps | 40 hours | 4 hours/week |
| Tech Lead | 16 hours | 2 hours/week |
| Developers | 4 hours training | Productivity gain |
### Expected ROI
| Investment | Return |
|------------|--------|
| 60 hours setup | 50% faster feature development |
| 6 hours/week maintenance | 80% fewer pattern violations |
| 4 hours training per dev | New devs productive in weeks, not months |
---
## Appendix A: Skill Generation Commands
```bash
# Generate core game skill
skill-seekers github \
--repo spyke/knit-game-client \
--name yarn-flow-core \
--code-analysis-depth full \
--enhance-local
# Generate docs skill
skill-seekers scrape \
--url file:///path/to/knit-game-client/docs \
--name yarn-flow-docs \
--enhance-local
# Install to Claude Code
skill-seekers install-agent output/yarn-flow-core/ --agent claude
skill-seekers install-agent output/yarn-flow-docs/ --agent claude
```
## Appendix B: Agent Prompt Templates
### PR Review Agent System Prompt
```
You are a code review agent for the Yarn Flow Unity game project.
Your job is to review code changes and identify violations of project standards.
LOADED SKILLS:
- yarn-flow-pr-review: PR checklist and common pitfalls
- yarn-flow-mvc: MVC layer rules
- yarn-flow-async: Async safety patterns
REVIEW CHECKLIST:
1. MVC Violations
- Controllers/Models must NOT import UnityEngine
- Views implement interfaces defined by controllers
2. Lifecycle Issues
- IInitializable requires IDisposable
- Events subscribed must be unsubscribed
- PrepareForReuse must reset ALL state
3. Async Safety
- CancellationToken must be passed and checked
- State must be revalidated after await
- Animation IDs must use try-finally
For each issue found, report:
- File and line number
- Severity (FAIL/WARN/INFO)
- Code snippet showing the problem
- Fix recommendation with corrected code
```
---
## Appendix C: Example Agent Output
### Analysis Agent Output Example
```markdown
## Feature Analysis: ThreadCutter
### Classification
| Property | Value |
|----------|-------|
| Type | Grid Blocker |
| Base Class | `GridBlockerBaseModel` |
| Interface | `IGridBlocker` |
| Shape | Single-cell |
| Collection | Direction-based (cuts thread when unstitch passes through) |
### System Interactions
| System | Interacts | Details |
|--------|-----------|---------|
| Unstitch | YES | Cuts thread, decreases remaining count |
| Belt/Tray | NO | No direct interaction |
| Grid | YES | Registered in GridModel |
| Play-On | YES | Counter restoration needed |
| Level Goals | YES | Required goal type |
| ThreadBox | YES | Can coexist on same row |
### Edge Cases Identified
1. **Timing**
- Level ends during cut animation
- Multiple cuts triggered same frame
2. **Spatial**
- ThreadCutter at grid edge
- Adjacent to another ThreadCutter
3. **State**
- Counter reaches 0 during animation
- Play-on during cut animation
### Complexity: Low-Medium
- Follows existing single-cell blocker patterns
- Direction-based collection similar to existing blockers
### Reference Implementations
- `KnitCover` - Single-cell grid blocker
- `ThreadBox` - Direction-based entry
```
---
## Conclusion
This proposal outlines a comprehensive system for AI-augmented game development that:
1. **Captures institutional knowledge** in reusable skills
2. **Automates quality enforcement** via specialized agents
3. **Enables pair programming** with Claude Code as implementer
4. **Ensures consistency** across all developers regardless of experience
The expected outcome is faster development, higher code quality, and dramatically reduced onboarding time for new team members.
---
**Prepared by:** Claude Code + Skill Seekers
**For review by:** CTO, Tech Lead
**Next step:** Approve and begin Phase 1 implementation

View File

@@ -0,0 +1,396 @@
# Spyke Games Confluence Documentation Analysis & Skill Generation Plan
## Executive Summary
**Total Pages**: 147
**Usable Content**: 127 pages (86%)
**Empty/Container**: 20 pages (14%)
**Legacy/Deprecated**: 17 pages (12%)
**Active & Valid**: ~110 pages (75%)
---
## Document Hierarchy Overview
```
Engineering (root)
├── R&D/
│ ├── Backend Architecture/ (5 docs)
│ ├── Client Architecture/ (9 docs + Addressables/5)
│ ├── Cloud Services/AWS notes/ (4 docs)
│ ├── Graphics/ (4 docs)
│ ├── Network Messaging/ (3 docs)
│ └── Tools/ (1 doc)
├── Backend Design/ (7 docs)
├── Team/ (4 docs)
├── Team Backend Notes/ (3 docs)
├── Cheatsheets/ (4 docs)
├── Tech Talks/ (3 docs)
├── Feature Flags/LiveOps Tooling/ (5+ docs)
├── Game Retrospectives/ (4 docs - legacy)
├── Reverse Engineering/ (7 docs - legacy)
├── Third Party SDKs/ (3 docs)
├── How To Add New Special Day Theme Assets/ (8 docs)
└── ~30 standalone pages
```
**Issues Found:**
- 3 orphaned docs (parent outside space)
- 20 empty container pages
- Inconsistent nesting (some topics deeply nested, others flat)
- Mixed languages (English + Turkish titles)
---
## Skill Generation Recommendations
### RECOMMENDED SKILLS TO GENERATE
Based on content depth, code examples, and practical value:
---
### 1. ⭐ SKILL: "spyke-unity-client" (HIGH VALUE)
**Content Sources**: 25 pages | ~59,000 chars | 12 with code
**Topics to Include**:
- UI Panel Transitions
- Screen Scaling for mobile
- Addressables (caching, bundles, catalog structure)
- Scriptable Objects as Architecture
- MVCVM Architecture pattern
- Fast Generic Observers (SignalBus alternative)
- Persistent Data management
- Animation & Particle Performance
- Shader development (MultiLayerText, Blur)
- URP vs Legacy Render Pipeline
**Why Generate**:
- Core Unity development patterns used across all games
- Reusable regardless of which game is active
- Good mix of code examples and explanations
**Improvements Needed Before Generating**:
1. Finalize "Slot Game X - Architecture (MVCVM) - (Draft)"
2. Add code examples to "Scriptable Objects as Architecture"
3. Update "Built-in (Legacy) Render Pipeline vs URP" - mark Legacy as deprecated
4. Consolidate Addressables docs into cohesive guide
---
### 2. ⭐ SKILL: "spyke-backend" (HIGH VALUE)
**Content Sources**: 16 pages | ~36,000 chars | 5 with code
**Topics to Include**:
- Database Version Control/Migration (Flyway)
- Database Access Layer patterns
- Spring/Gradle architecture
- Game Server architecture
- Load testing approaches
- Security measures
- MySQL/Aurora patterns
- Chat backend implementation
**Why Generate**:
- Backend patterns are game-agnostic
- Critical for onboarding backend devs
- Contains production-tested patterns
**Improvements Needed Before Generating**:
1. Finalize "Backend Code Structure (draft)"
2. Finalize "Chat Mysql (draft)"
3. Finalize "Help Call Backend Notes (Draft)"
4. Translate Turkish content: "bonanza ve lucky spin..." → English
5. Add more code examples to architecture docs
---
### 3. ⭐ SKILL: "spyke-aws" (MEDIUM VALUE)
**Content Sources**: 9 pages | ~22,000 chars | 3 with code
**Topics to Include**:
- AWS account/users/groups/policies
- Elastic Beanstalk setup
- Gateway and ALB configuration
- Aurora database notes
- Performance testing with k6
- AWS CLI access (secure)
- AWS Evidently for feature flags
- Cost saving strategies
**Why Generate**:
- Infrastructure knowledge critical for ops
- k6 performance testing guide is excellent
- AWS patterns are reusable
**Improvements Needed Before Generating**:
1. Finalize "Secure AWS CLI Access (DRAFT)"
2. Update AWS notes - verify if still using EB or migrated
3. Add more practical examples to account setup docs
---
### 4. SKILL: "spyke-onboarding" (MEDIUM VALUE)
**Content Sources**: 13 pages | ~26,000 chars | 4 with code
**Topics to Include**:
- Welcome To The Team
- Buddy System
- Code Review (How To)
- Release Manager responsibilities
- Git Submodule management
- New Project Setup from Bootstrap
- Unit Test Integration to Pipeline
- Mock Web Service Tool
**Why Generate**:
- Essential for new engineer onboarding
- Process documentation is evergreen
- Reduces tribal knowledge
**Improvements Needed Before Generating**:
1. Update "Welcome To The Team" with current tools/processes
2. Add current team structure to Team docs
3. Verify pipeline docs match current CI/CD
---
### 5. SKILL: "spyke-sdks" (LOW VALUE - CONSIDER SKIP)
**Content Sources**: 7 pages | ~7,000 chars | 5 with code
**Topics to Include**:
- MAX SDK integration
- OneSignal push notifications
- Braze platform notes
- AppsFlyer (if still used)
- i2 localization
- Huawei App Gallery
**Why Generate**: SDK integration guides save time
**Issues**:
- Most are version-specific and may be outdated
- Low content depth
- Better to link to official SDK docs
**Recommendation**: Skip or merge into onboarding skill
---
### 6. SKILL: "spyke-liveops" (LOW VALUE - NEEDS WORK)
**Content Sources**: ~10 pages | Content scattered
**Topics to Include**:
- Feature Flags overview
- Split.io vs Unleash vs AWS Evidently comparison
- A/B Test Infrastructure
- Configuration Management
**Issues**:
- Content is fragmented
- Many empty placeholder pages
- "The Choice and Things to Consider" has no conclusion
**Recommendation**: Consolidate before generating
---
## NOT RECOMMENDED FOR SKILLS
### Legacy/Deprecated (17 pages)
- Coin Master, Tile Busters, Royal Riches, Island King, Pirate King docs
- **Action**: Archive in Confluence, do NOT include in skills
- **Exception**: "Learnings From X" docs have reusable insights - extract generic patterns
### Empty Containers (20 pages)
- Engineering, R&D, Client, Backend, etc.
- **Action**: Either delete or add meaningful overview content
### Game-Specific Workflows
- "How to add new Endless Offers (Tile Busters)" - deprecated
- "Tile Busters Particle Optimizations" - game-specific
- **Action**: Generalize or archive
---
## Individual Document Improvements
### HIGH PRIORITY (Block skill generation)
| Document | Issue | Action |
|----------|-------|--------|
| Slot Game X - Architecture (MVCVM) - (Draft) | Still draft | Finalize or remove draft label |
| Backend Code Structure (draft) | Still draft | Finalize with current structure |
| Chat Mysql (draft) | Still draft | Finalize or archive |
| Secure AWS CLI Access (DRAFT) | Still draft | Finalize - important for security |
| Help Call Backend Notes (Draft) | Still draft | Finalize or archive |
| Submodule [Draft] | Still draft | Merge with Git Submodule doc |
| Creating New Team Event (DRAFT) | Still draft | Finalize |
| bonanza ve lucky spin... | Turkish title | Translate to English |
### MEDIUM PRIORITY (Improve quality)
| Document | Issue | Action |
|----------|-------|--------|
| Scriptable Objects as Architecture | No code examples | Add Unity C# examples |
| Built-in (Legacy) vs URP | Doesn't say which to use | Add clear recommendation: "Use URP" |
| Feature Flag System | No conclusion | Add recommendation on which system |
| The Choice and Things to Consider | Incomplete | Add final decision/recommendation |
| AWS notes (container) | Empty | Add overview or delete |
| Third Party SDKs (container) | Empty | Add overview or delete |
| All 20 empty containers | No content | Add overview content or delete |
### LOW PRIORITY (Nice to have)
| Document | Issue | Action |
|----------|-------|--------|
| Addressables (5 docs) | Scattered | Consolidate into single comprehensive guide |
| Animation Performance (2 docs) | Overlap | Merge benchmarks with tips |
| LiveOps Tools (5 docs) | Fragmented | Create summary comparison table |
| Game Retrospectives | Deprecated games | Extract generic learnings, archive rest |
---
## Recommended Skill Generation Order
1. **spyke-unity-client** (most value, good content)
2. **spyke-backend** (after drafts finalized)
3. **spyke-aws** (after drafts finalized)
4. **spyke-onboarding** (after process docs updated)
5. ~~spyke-sdks~~ (skip or merge)
6. ~~spyke-liveops~~ (needs consolidation first)
---
## Implementation Steps
### Phase 1: Content Cleanup
1. Finalize all 8 draft documents
2. Translate Turkish content to English
3. Delete or populate 20 empty container pages
4. Archive 17 legacy game docs
### Phase 2: Generate Skills
1. Create unified config for each skill
2. Use Skill Seekers with Confluence scraper (to be built)
3. Generate and package skills
### Phase 3: Ongoing Maintenance
1. Set up review schedule for docs
2. Add "Last Reviewed" date to each doc
3. Create Confluence template for new docs
---
## Confluence Scraper Feature (New Development)
To generate skills from Confluence, need to add:
```
src/skill_seekers/cli/confluence_scraper.py
```
Config format:
```json
{
"name": "spyke-unity-client",
"type": "confluence",
"domain": "spykegames.atlassian.net",
"space_key": "EN",
"page_ids": ["70811737", "8880129", ...],
"exclude_patterns": ["coin master", "tile busters"],
"auth": {
"email": "$CONFLUENCE_EMAIL",
"token": "$CONFLUENCE_TOKEN"
}
}
```
---
## Summary
| Metric | Count |
|--------|-------|
| Total Pages | 147 |
| Ready for Skills | ~80 |
| Need Improvement | ~30 |
| Archive/Delete | ~37 |
| Recommended Skills | 4 |
| Drafts to Finalize | 8 |
| Empty to Fix | 20 |
---
## ACTION CHECKLIST FOR DOC CLEANUP
### 1. Finalize Drafts (8 docs)
- [ ] [Slot Game X - Architecture (MVCVM) - (Draft)](https://spykegames.atlassian.net/wiki/spaces/EN/pages/63471723)
- [ ] [Backend Code Structure (draft)](https://spykegames.atlassian.net/wiki/spaces/EN/pages/637829184)
- [ ] [Chat Mysql (draft)](https://spykegames.atlassian.net/wiki/spaces/EN/pages/593330177)
- [ ] [Secure AWS CLI Access (DRAFT)](https://spykegames.atlassian.net/wiki/spaces/EN/pages/870744065)
- [ ] [Help Call Backend Notes (Draft)](https://spykegames.atlassian.net/wiki/spaces/EN/pages/695074823)
- [ ] [Submodule [Draft]](https://spykegames.atlassian.net/wiki/spaces/EN/pages/690356267)
- [ ] [Submodule View Management [Draft]](https://spykegames.atlassian.net/wiki/spaces/EN/pages/690126851)
- [ ] [Creating New Team Event (DRAFT)](https://spykegames.atlassian.net/wiki/spaces/EN/pages/759988225)
### 2. Translate to English (1 doc)
- [ ] [bonanza ve lucky spin bittikten sonra odeme gelmesi sorunsalı](https://spykegames.atlassian.net/wiki/spaces/EN/pages/831324161)
### 3. Delete or Populate Empty Containers (20 docs)
- [ ] Engineering (root page - add overview)
- [ ] R&D (add overview)
- [ ] Client (add overview or delete)
- [ ] Backend (add overview or delete)
- [ ] AWS notes (add overview or delete)
- [ ] Network Messaging (add overview or delete)
- [ ] Tools (add overview or delete)
- [ ] Cloud Services (add overview or delete)
- [ ] Graphics (add overview or delete)
- [ ] Client Architecture (add overview or delete)
- [ ] Backend Architecture (add overview or delete)
- [ ] Backend Design (add overview or delete)
- [ ] Third Party SDKs (add overview or delete)
- [ ] Tech Talks (add overview or delete)
- [ ] Cheatsheets (add overview or delete)
- [ ] Team (add overview or delete)
- [ ] Game Retrospectives (add overview or delete)
- [ ] Feature Flags / LiveOps Tooling (add overview or delete)
- [ ] How To Add New Special Day Theme Assets (add overview)
- [ ] Replacing Active App Icon On Player Settings (add content - only has link)
### 4. Archive Legacy Game Docs (17 docs)
Move to "Archive" or "Legacy" section:
- [ ] Coin Master
- [ ] Coin Master Notes
- [ ] Bot - Coin Master
- [ ] Coin Trip Notes
- [ ] Island King
- [ ] Pirate King
- [ ] Learnings From Royal Riches - Client
- [ ] Learnings From Royal Riches - Backend
- [ ] Learnings From Tile Busters - Client
- [ ] Learnings From Tile Busters - Backend
- [ ] How to add new Endless Offers (Tile Busters)
- [ ] Tile Busters Level/AB Update Flow
- [ ] Tile Busters Backend Git Branch/Deployment Cycle
- [ ] Tile Busters Backend Git Branch/Deployment Cycle (v2)
- [ ] Tile Busters Particle Optimizations
- [ ] Automated Play Test for Tile Busters
- [ ] Automated Purchase Testing for Tile Busters
### 5. Content Improvements (Optional but Recommended)
- [ ] Add code examples to "Scriptable Objects as Architecture"
- [ ] Add URP recommendation to "Built-in (Legacy) vs URP"
- [ ] Consolidate 5 Addressables docs into 1
- [ ] Add conclusion to "Feature Flag System"
- [ ] Create comparison table in LiveOps Tools
---
## AFTER CLEANUP: Come back and run skill generation
Once the above items are addressed, return and I will:
1. Build a Confluence scraper for Skill Seekers
2. Generate the 4 recommended skills
3. Package and upload them

View File

@@ -12,17 +12,34 @@ Features:
- Groups related examples into tutorials
- Identifies best practices
Modes:
- API mode: Uses Claude API (requires ANTHROPIC_API_KEY)
- LOCAL mode: Uses Claude Code CLI (no API key needed, uses your Claude Max plan)
- AUTO mode: Tries API first, falls back to LOCAL
Credits:
- Uses Claude AI (Anthropic) for analysis
- Graceful degradation if API unavailable
"""
import json
import logging
import os
import subprocess
import tempfile
from concurrent.futures import ThreadPoolExecutor, as_completed
from dataclasses import dataclass
from pathlib import Path
logger = logging.getLogger(__name__)
# Import config manager for settings
try:
from skill_seekers.cli.config_manager import get_config_manager
CONFIG_AVAILABLE = True
except ImportError:
CONFIG_AVAILABLE = False
@dataclass
class AIAnalysis:
@@ -47,29 +64,32 @@ class AIEnhancer:
api_key: Anthropic API key (uses ANTHROPIC_API_KEY env if None)
enabled: Enable AI enhancement (default: True)
mode: Enhancement mode - "auto" (default), "api", or "local"
- "auto": Use API if key available, otherwise disable
- "auto": Use API if key available, otherwise fall back to LOCAL
- "api": Force API mode (fails if no key)
- "local": Use Claude Code local mode (opens terminal)
- "local": Use Claude Code CLI (no API key needed)
"""
self.enabled = enabled
self.mode = mode
self.api_key = api_key or os.environ.get("ANTHROPIC_API_KEY")
self.client = None
# Get settings from config (with defaults)
if CONFIG_AVAILABLE:
config = get_config_manager()
self.local_batch_size = config.get_local_batch_size()
self.local_parallel_workers = config.get_local_parallel_workers()
else:
self.local_batch_size = 20 # Default
self.local_parallel_workers = 3 # Default
# Determine actual mode
if mode == "auto":
if self.api_key:
self.mode = "api"
else:
# For now, disable if no API key
# LOCAL mode for batch processing is complex
self.mode = "disabled"
self.enabled = False
logger.info(" AI enhancement disabled (no API key found)")
logger.info(
" Set ANTHROPIC_API_KEY to enable, or use 'skill-seekers enhance' for SKILL.md"
)
return
# Fall back to LOCAL mode (Claude Code CLI)
self.mode = "local"
logger.info(" No API key found, using LOCAL mode (Claude Code CLI)")
if self.mode == "api" and self.enabled:
try:
@@ -84,23 +104,44 @@ class AIEnhancer:
self.client = anthropic.Anthropic(**client_kwargs)
logger.info("✅ AI enhancement enabled (using Claude API)")
except ImportError:
logger.warning("⚠️ anthropic package not installed. AI enhancement disabled.")
logger.warning(" Install with: pip install anthropic")
self.enabled = False
logger.warning("⚠️ anthropic package not installed, falling back to LOCAL mode")
self.mode = "local"
except Exception as e:
logger.warning(f"⚠️ Failed to initialize AI client: {e}")
logger.warning(f"⚠️ Failed to initialize API client: {e}, falling back to LOCAL mode")
self.mode = "local"
if self.mode == "local" and self.enabled:
# Verify Claude CLI is available
if self._check_claude_cli():
logger.info("✅ AI enhancement enabled (using LOCAL mode - Claude Code CLI)")
else:
logger.warning("⚠️ Claude Code CLI not found. AI enhancement disabled.")
logger.warning(" Install with: npm install -g @anthropic-ai/claude-code")
self.enabled = False
elif self.mode == "local":
# LOCAL mode requires Claude Code to be available
# For patterns/examples, this is less practical than API mode
logger.info(" LOCAL mode not yet supported for pattern/example enhancement")
logger.info(
" Use API mode (set ANTHROPIC_API_KEY) or 'skill-seekers enhance' for SKILL.md"
def _check_claude_cli(self) -> bool:
"""Check if Claude Code CLI is available"""
try:
result = subprocess.run(
["claude", "--version"],
capture_output=True,
text=True,
timeout=5,
)
self.enabled = False
return result.returncode == 0
except (FileNotFoundError, subprocess.TimeoutExpired):
return False
def _call_claude(self, prompt: str, max_tokens: int = 1000) -> str | None:
"""Call Claude API with error handling"""
"""Call Claude (API or LOCAL mode) with error handling"""
if self.mode == "api":
return self._call_claude_api(prompt, max_tokens)
elif self.mode == "local":
return self._call_claude_local(prompt)
return None
def _call_claude_api(self, prompt: str, max_tokens: int = 1000) -> str | None:
"""Call Claude API"""
if not self.client:
return None
@@ -115,6 +156,82 @@ class AIEnhancer:
logger.warning(f"⚠️ AI API call failed: {e}")
return None
def _call_claude_local(self, prompt: str) -> str | None:
"""Call Claude using LOCAL mode (Claude Code CLI)"""
try:
# Create a temporary directory for this enhancement
with tempfile.TemporaryDirectory(prefix="ai_enhance_") as temp_dir:
temp_path = Path(temp_dir)
# Create prompt file
prompt_file = temp_path / "prompt.md"
output_file = temp_path / "response.json"
# Write prompt with instructions to output JSON
full_prompt = f"""# AI Analysis Task
IMPORTANT: You MUST write your response as valid JSON to this file:
{output_file}
## Task
{prompt}
## Instructions
1. Analyze the input carefully
2. Generate the JSON response as specified
3. Use the Write tool to save the JSON to: {output_file}
4. The JSON must be valid and parseable
DO NOT include any explanation - just write the JSON file.
"""
prompt_file.write_text(full_prompt)
# Run Claude CLI
result = subprocess.run(
["claude", "--dangerously-skip-permissions", str(prompt_file)],
capture_output=True,
text=True,
timeout=120, # 2 minute timeout per call
cwd=str(temp_path),
)
if result.returncode != 0:
logger.warning(f"⚠️ Claude CLI returned error: {result.returncode}")
return None
# Read output file
if output_file.exists():
response_text = output_file.read_text()
# Try to extract JSON from response
try:
# Validate it's valid JSON
json.loads(response_text)
return response_text
except json.JSONDecodeError:
# Try to find JSON in the response
import re
json_match = re.search(r'\[[\s\S]*\]|\{[\s\S]*\}', response_text)
if json_match:
return json_match.group()
logger.warning("⚠️ Could not parse JSON from LOCAL response")
return None
else:
# Look for any JSON file created
for json_file in temp_path.glob("*.json"):
if json_file.name != "prompt.json":
return json_file.read_text()
logger.warning("⚠️ No output file from LOCAL mode")
return None
except subprocess.TimeoutExpired:
logger.warning("⚠️ Claude CLI timeout (2 minutes)")
return None
except Exception as e:
logger.warning(f"⚠️ LOCAL mode error: {e}")
return None
class PatternEnhancer(AIEnhancer):
"""Enhance design pattern detection with AI analysis"""
@@ -132,20 +249,68 @@ class PatternEnhancer(AIEnhancer):
if not self.enabled or not patterns:
return patterns
logger.info(f"🤖 Enhancing {len(patterns)} detected patterns with AI...")
# Batch patterns to minimize API calls (max 5 per batch)
batch_size = 5
enhanced = []
# Use larger batch size for LOCAL mode (configurable)
if self.mode == "local":
batch_size = self.local_batch_size
parallel_workers = self.local_parallel_workers
logger.info(
f"🤖 Enhancing {len(patterns)} patterns with AI "
f"(LOCAL mode: {batch_size} per batch, {parallel_workers} parallel workers)..."
)
else:
batch_size = 5 # API mode uses smaller batches
parallel_workers = 1 # API mode is sequential
logger.info(f"🤖 Enhancing {len(patterns)} detected patterns with AI...")
# Create batches
batches = []
for i in range(0, len(patterns), batch_size):
batch = patterns[i : i + batch_size]
batch_results = self._enhance_pattern_batch(batch)
enhanced.extend(batch_results)
batches.append(patterns[i : i + batch_size])
# Process batches (parallel for LOCAL, sequential for API)
if parallel_workers > 1 and len(batches) > 1:
enhanced = self._enhance_patterns_parallel(batches, parallel_workers)
else:
enhanced = []
for batch in batches:
batch_results = self._enhance_pattern_batch(batch)
enhanced.extend(batch_results)
logger.info(f"✅ Enhanced {len(enhanced)} patterns")
return enhanced
def _enhance_patterns_parallel(self, batches: list[list[dict]], workers: int) -> list[dict]:
"""Process pattern batches in parallel using ThreadPoolExecutor."""
results = [None] * len(batches) # Preserve order
with ThreadPoolExecutor(max_workers=workers) as executor:
# Submit all batches
future_to_idx = {
executor.submit(self._enhance_pattern_batch, batch): idx
for idx, batch in enumerate(batches)
}
# Collect results as they complete
completed = 0
total = len(batches)
for future in as_completed(future_to_idx):
idx = future_to_idx[future]
try:
results[idx] = future.result()
completed += 1
if completed % 5 == 0 or completed == total:
logger.info(f" Progress: {completed}/{total} batches completed")
except Exception as e:
logger.warning(f"⚠️ Batch {idx} failed: {e}")
results[idx] = batches[idx] # Return unenhanced on failure
# Flatten results
enhanced = []
for batch_result in results:
if batch_result:
enhanced.extend(batch_result)
return enhanced
def _enhance_pattern_batch(self, patterns: list[dict]) -> list[dict]:
"""Enhance a batch of patterns"""
# Prepare prompt
@@ -176,8 +341,6 @@ Format as JSON array matching input order. Be concise and actionable.
return patterns
try:
import json
analyses = json.loads(response)
# Merge AI analysis into patterns
@@ -223,20 +386,68 @@ class TestExampleEnhancer(AIEnhancer):
if not self.enabled or not examples:
return examples
logger.info(f"🤖 Enhancing {len(examples)} test examples with AI...")
# Batch examples to minimize API calls
batch_size = 5
enhanced = []
# Use larger batch size for LOCAL mode (configurable)
if self.mode == "local":
batch_size = self.local_batch_size
parallel_workers = self.local_parallel_workers
logger.info(
f"🤖 Enhancing {len(examples)} test examples with AI "
f"(LOCAL mode: {batch_size} per batch, {parallel_workers} parallel workers)..."
)
else:
batch_size = 5 # API mode uses smaller batches
parallel_workers = 1 # API mode is sequential
logger.info(f"🤖 Enhancing {len(examples)} test examples with AI...")
# Create batches
batches = []
for i in range(0, len(examples), batch_size):
batch = examples[i : i + batch_size]
batch_results = self._enhance_example_batch(batch)
enhanced.extend(batch_results)
batches.append(examples[i : i + batch_size])
# Process batches (parallel for LOCAL, sequential for API)
if parallel_workers > 1 and len(batches) > 1:
enhanced = self._enhance_examples_parallel(batches, parallel_workers)
else:
enhanced = []
for batch in batches:
batch_results = self._enhance_example_batch(batch)
enhanced.extend(batch_results)
logger.info(f"✅ Enhanced {len(enhanced)} examples")
return enhanced
def _enhance_examples_parallel(self, batches: list[list[dict]], workers: int) -> list[dict]:
"""Process example batches in parallel using ThreadPoolExecutor."""
results = [None] * len(batches) # Preserve order
with ThreadPoolExecutor(max_workers=workers) as executor:
# Submit all batches
future_to_idx = {
executor.submit(self._enhance_example_batch, batch): idx
for idx, batch in enumerate(batches)
}
# Collect results as they complete
completed = 0
total = len(batches)
for future in as_completed(future_to_idx):
idx = future_to_idx[future]
try:
results[idx] = future.result()
completed += 1
if completed % 5 == 0 or completed == total:
logger.info(f" Progress: {completed}/{total} batches completed")
except Exception as e:
logger.warning(f"⚠️ Batch {idx} failed: {e}")
results[idx] = batches[idx] # Return unenhanced on failure
# Flatten results
enhanced = []
for batch_result in results:
if batch_result:
enhanced.extend(batch_result)
return enhanced
def _enhance_example_batch(self, examples: list[dict]) -> list[dict]:
"""Enhance a batch of examples"""
# Prepare prompt
@@ -268,8 +479,6 @@ Format as JSON array matching input order. Focus on educational value.
return examples
try:
import json
analyses = json.loads(response)
# Merge AI analysis into examples

View File

@@ -75,6 +75,53 @@ LANGUAGE_EXTENSIONS = {
".php": "PHP",
}
# Markdown extension mapping
MARKDOWN_EXTENSIONS = {".md", ".markdown", ".mdown", ".mkd"}
# Common documentation folders to scan
DOC_FOLDERS = {"docs", "doc", "documentation", "wiki", ".github"}
# Root-level doc files → category mapping
ROOT_DOC_CATEGORIES = {
"readme": "overview",
"contributing": "contributing",
"changelog": "changelog",
"history": "changelog",
"license": "license",
"authors": "authors",
"code_of_conduct": "community",
"security": "security",
"architecture": "architecture",
"design": "architecture",
}
# Folder name → category mapping
FOLDER_CATEGORIES = {
"architecture": "architecture",
"arch": "architecture",
"design": "architecture",
"guides": "guides",
"guide": "guides",
"tutorials": "guides",
"tutorial": "guides",
"howto": "guides",
"how-to": "guides",
"workflows": "workflows",
"workflow": "workflows",
"templates": "templates",
"template": "templates",
"api": "api",
"reference": "api",
"examples": "examples",
"example": "examples",
"specs": "specifications",
"spec": "specifications",
"rfcs": "specifications",
"rfc": "specifications",
"features": "features",
"feature": "features",
}
# Default directories to exclude
DEFAULT_EXCLUDED_DIRS = {
"node_modules",
@@ -216,6 +263,469 @@ def walk_directory(
return sorted(files)
def walk_markdown_files(
root: Path,
gitignore_spec: pathspec.PathSpec | None = None,
excluded_dirs: set | None = None,
) -> list[Path]:
"""
Walk directory tree and collect markdown documentation files.
Args:
root: Root directory to walk
gitignore_spec: Optional PathSpec object for .gitignore rules
excluded_dirs: Set of directory names to exclude
Returns:
List of markdown file paths
"""
if excluded_dirs is None:
excluded_dirs = DEFAULT_EXCLUDED_DIRS
files = []
root = Path(root).resolve()
for dirpath, dirnames, filenames in os.walk(root):
current_dir = Path(dirpath)
# Filter out excluded directories (in-place modification)
dirnames[:] = [d for d in dirnames if not should_exclude_dir(d, excluded_dirs)]
for filename in filenames:
file_path = current_dir / filename
# Check .gitignore rules
if gitignore_spec:
try:
rel_path = file_path.relative_to(root)
if gitignore_spec.match_file(str(rel_path)):
logger.debug(f"Skipping (gitignore): {rel_path}")
continue
except ValueError:
continue
# Check if markdown file
if file_path.suffix.lower() not in MARKDOWN_EXTENSIONS:
continue
files.append(file_path)
return sorted(files)
def categorize_markdown_file(file_path: Path, root: Path) -> str:
"""
Categorize a markdown file based on its location and filename.
Args:
file_path: Path to the markdown file
root: Root directory of the project
Returns:
Category name (e.g., 'overview', 'guides', 'architecture')
"""
try:
rel_path = file_path.relative_to(root)
except ValueError:
return "other"
# Check root-level files by filename
if len(rel_path.parts) == 1:
filename_lower = file_path.stem.lower().replace("-", "_").replace(" ", "_")
for key, category in ROOT_DOC_CATEGORIES.items():
if key in filename_lower:
return category
return "overview" # Default for root .md files
# Check folder-based categorization
for part in rel_path.parts[:-1]: # Exclude filename
part_lower = part.lower().replace("-", "_").replace(" ", "_")
for key, category in FOLDER_CATEGORIES.items():
if key in part_lower:
return category
# Default category
return "other"
def extract_markdown_structure(content: str) -> dict[str, Any]:
"""
Extract structure from markdown content (headers, code blocks, links).
Args:
content: Markdown file content
Returns:
Dictionary with extracted structure
"""
import re
structure = {
"title": None,
"headers": [],
"code_blocks": [],
"links": [],
"word_count": len(content.split()),
"line_count": len(content.split("\n")),
}
lines = content.split("\n")
# Extract headers
for i, line in enumerate(lines):
header_match = re.match(r"^(#{1,6})\s+(.+)$", line)
if header_match:
level = len(header_match.group(1))
text = header_match.group(2).strip()
structure["headers"].append({
"level": level,
"text": text,
"line": i + 1,
})
# First h1 is the title
if level == 1 and structure["title"] is None:
structure["title"] = text
# Extract code blocks (fenced)
code_block_pattern = re.compile(r"```(\w*)\n(.*?)```", re.DOTALL)
for match in code_block_pattern.finditer(content):
language = match.group(1) or "text"
code = match.group(2).strip()
if len(code) > 0:
structure["code_blocks"].append({
"language": language,
"code": code[:500], # Truncate long code blocks
"full_length": len(code),
})
# Extract links
link_pattern = re.compile(r"\[([^\]]+)\]\(([^)]+)\)")
for match in link_pattern.finditer(content):
structure["links"].append({
"text": match.group(1),
"url": match.group(2),
})
return structure
def generate_markdown_summary(content: str, structure: dict[str, Any], max_length: int = 500) -> str:
"""
Generate a summary of markdown content.
Args:
content: Full markdown content
structure: Extracted structure from extract_markdown_structure()
max_length: Maximum summary length
Returns:
Summary string
"""
# Start with title if available
summary_parts = []
if structure.get("title"):
summary_parts.append(f"**{structure['title']}**")
# Add header outline (first 5 h2/h3 headers)
h2_h3 = [h for h in structure.get("headers", []) if h["level"] in (2, 3)][:5]
if h2_h3:
sections = [h["text"] for h in h2_h3]
summary_parts.append(f"Sections: {', '.join(sections)}")
# Extract first paragraph (skip headers and empty lines)
lines = content.split("\n")
first_para = []
in_para = False
for line in lines:
stripped = line.strip()
if stripped.startswith("#") or stripped.startswith("```"):
if in_para:
break
continue
if stripped:
in_para = True
first_para.append(stripped)
elif in_para:
break
if first_para:
para_text = " ".join(first_para)
if len(para_text) > 200:
para_text = para_text[:200] + "..."
summary_parts.append(para_text)
# Add stats
stats = f"({structure.get('word_count', 0)} words, {len(structure.get('code_blocks', []))} code blocks)"
summary_parts.append(stats)
summary = "\n".join(summary_parts)
if len(summary) > max_length:
summary = summary[:max_length] + "..."
return summary
def process_markdown_docs(
directory: Path,
output_dir: Path,
depth: str = "deep",
gitignore_spec: pathspec.PathSpec | None = None,
enhance_with_ai: bool = False,
ai_mode: str = "none",
) -> dict[str, Any]:
"""
Process all markdown documentation files in a directory.
Args:
directory: Root directory to scan
output_dir: Output directory for processed docs
depth: Processing depth ('surface', 'deep', 'full')
gitignore_spec: Optional .gitignore spec
enhance_with_ai: Whether to use AI enhancement
ai_mode: AI mode ('none', 'auto', 'api', 'local')
Returns:
Dictionary with processed documentation data
"""
logger.info("Scanning for markdown documentation...")
# Find all markdown files
md_files = walk_markdown_files(directory, gitignore_spec)
logger.info(f"Found {len(md_files)} markdown files")
if not md_files:
return {"files": [], "categories": {}, "total_files": 0}
# Process each file
processed_docs = []
categories = {}
for md_path in md_files:
try:
content = md_path.read_text(encoding="utf-8", errors="ignore")
rel_path = str(md_path.relative_to(directory))
category = categorize_markdown_file(md_path, directory)
doc_data = {
"path": rel_path,
"filename": md_path.name,
"category": category,
"size_bytes": len(content.encode("utf-8")),
}
# Surface depth: just path and category
if depth == "surface":
processed_docs.append(doc_data)
else:
# Deep/Full: extract structure and summary
structure = extract_markdown_structure(content)
summary = generate_markdown_summary(content, structure)
doc_data.update({
"title": structure.get("title") or md_path.stem,
"structure": structure,
"summary": summary,
"content": content if depth == "full" else None,
})
processed_docs.append(doc_data)
# Track categories
if category not in categories:
categories[category] = []
categories[category].append(rel_path)
except Exception as e:
logger.warning(f"Failed to process {md_path}: {e}")
continue
# AI Enhancement (if enabled and enhance_level >= 2)
if enhance_with_ai and ai_mode != "none" and processed_docs:
logger.info("🤖 Enhancing documentation analysis with AI...")
try:
processed_docs = _enhance_docs_with_ai(processed_docs, ai_mode)
logger.info("✅ AI documentation enhancement complete")
except Exception as e:
logger.warning(f"⚠️ AI enhancement failed: {e}")
# Save processed docs to output
docs_output_dir = output_dir / "documentation"
docs_output_dir.mkdir(parents=True, exist_ok=True)
# Copy files organized by category
for doc in processed_docs:
try:
src_path = directory / doc["path"]
category = doc["category"]
category_dir = docs_output_dir / category
category_dir.mkdir(parents=True, exist_ok=True)
# Copy file to category folder
dest_path = category_dir / doc["filename"]
import shutil
shutil.copy2(src_path, dest_path)
except Exception as e:
logger.debug(f"Failed to copy {doc['path']}: {e}")
# Save documentation index
index_data = {
"total_files": len(processed_docs),
"categories": categories,
"files": processed_docs,
}
index_json = docs_output_dir / "documentation_index.json"
with open(index_json, "w", encoding="utf-8") as f:
json.dump(index_data, f, indent=2, default=str)
logger.info(f"✅ Processed {len(processed_docs)} documentation files in {len(categories)} categories")
logger.info(f"📁 Saved to: {docs_output_dir}")
return index_data
def _enhance_docs_with_ai(docs: list[dict], ai_mode: str) -> list[dict]:
"""
Enhance documentation analysis with AI.
Args:
docs: List of processed document dictionaries
ai_mode: AI mode ('api' or 'local')
Returns:
Enhanced document list
"""
# Try API mode first
if ai_mode in ("api", "auto"):
api_key = os.environ.get("ANTHROPIC_API_KEY")
if api_key:
return _enhance_docs_api(docs, api_key)
# Fall back to LOCAL mode
if ai_mode in ("local", "auto"):
return _enhance_docs_local(docs)
return docs
def _enhance_docs_api(docs: list[dict], api_key: str) -> list[dict]:
"""Enhance docs using Claude API."""
try:
import anthropic
client = anthropic.Anthropic(api_key=api_key)
# Batch documents for efficiency
batch_size = 10
for i in range(0, len(docs), batch_size):
batch = docs[i:i + batch_size]
# Create prompt for batch
docs_text = "\n\n".join([
f"## {d.get('title', d['filename'])}\nCategory: {d['category']}\nSummary: {d.get('summary', 'N/A')}"
for d in batch if d.get("summary")
])
if not docs_text:
continue
prompt = f"""Analyze these documentation files and provide:
1. A brief description of what each document covers
2. Key topics/concepts mentioned
3. How they relate to each other
Documents:
{docs_text}
Return JSON with format:
{{"enhancements": [{{"filename": "...", "description": "...", "key_topics": [...], "related_to": [...]}}]}}"""
response = client.messages.create(
model="claude-sonnet-4-20250514",
max_tokens=2000,
messages=[{"role": "user", "content": prompt}]
)
# Parse response and merge enhancements
try:
import re
json_match = re.search(r"\{.*\}", response.content[0].text, re.DOTALL)
if json_match:
enhancements = json.loads(json_match.group())
for enh in enhancements.get("enhancements", []):
for doc in batch:
if doc["filename"] == enh.get("filename"):
doc["ai_description"] = enh.get("description")
doc["ai_topics"] = enh.get("key_topics", [])
doc["ai_related"] = enh.get("related_to", [])
except Exception:
pass
except Exception as e:
logger.warning(f"API enhancement failed: {e}")
return docs
def _enhance_docs_local(docs: list[dict]) -> list[dict]:
"""Enhance docs using Claude Code CLI (LOCAL mode)."""
import subprocess
import tempfile
# Prepare batch of docs for enhancement
docs_with_summary = [d for d in docs if d.get("summary")]
if not docs_with_summary:
return docs
docs_text = "\n\n".join([
f"## {d.get('title', d['filename'])}\nCategory: {d['category']}\nPath: {d['path']}\nSummary: {d.get('summary', 'N/A')}"
for d in docs_with_summary[:20] # Limit to 20 docs
])
prompt = f"""Analyze these documentation files from a codebase and provide insights.
For each document, provide:
1. A brief description of what it covers
2. Key topics/concepts
3. Related documents
Documents:
{docs_text}
Output JSON only:
{{"enhancements": [{{"filename": "...", "description": "...", "key_topics": ["..."], "related_to": ["..."]}}]}}"""
try:
with tempfile.NamedTemporaryFile(mode="w", suffix=".txt", delete=False) as f:
f.write(prompt)
prompt_file = f.name
result = subprocess.run(
["claude", "--dangerously-skip-permissions", "-p", prompt],
capture_output=True,
text=True,
timeout=120,
)
os.unlink(prompt_file)
if result.returncode == 0 and result.stdout:
import re
json_match = re.search(r"\{.*\}", result.stdout, re.DOTALL)
if json_match:
enhancements = json.loads(json_match.group())
for enh in enhancements.get("enhancements", []):
for doc in docs:
if doc["filename"] == enh.get("filename"):
doc["ai_description"] = enh.get("description")
doc["ai_topics"] = enh.get("key_topics", [])
doc["ai_related"] = enh.get("related_to", [])
except Exception as e:
logger.warning(f"LOCAL enhancement failed: {e}")
return docs
def analyze_codebase(
directory: Path,
output_dir: Path,
@@ -229,8 +739,8 @@ def analyze_codebase(
extract_test_examples: bool = True,
build_how_to_guides: bool = True,
extract_config_patterns: bool = True,
enhance_with_ai: bool = True,
ai_mode: str = "auto",
extract_docs: bool = True,
enhance_level: int = 0,
) -> dict[str, Any]:
"""
Analyze local codebase and extract code knowledge.
@@ -248,12 +758,26 @@ def analyze_codebase(
extract_test_examples: Extract usage examples from test files
build_how_to_guides: Build how-to guides from workflow examples (C3.3)
extract_config_patterns: Extract configuration patterns from config files (C3.4)
enhance_with_ai: Enhance patterns and examples with AI analysis (C3.6)
ai_mode: AI enhancement mode for how-to guides (auto, api, local, none)
extract_docs: Extract and process markdown documentation files (default: True)
enhance_level: AI enhancement level (0=off, 1=SKILL.md only, 2=+config+arch+docs, 3=full)
Returns:
Analysis results dictionary
"""
# Determine AI enhancement settings based on level
# Level 0: No AI enhancement
# Level 1: SKILL.md only (handled in main.py)
# Level 2: Architecture + Config AI enhancement
# Level 3: Full AI enhancement (patterns, tests, config, architecture)
enhance_patterns = enhance_level >= 3
enhance_tests = enhance_level >= 3
enhance_config = enhance_level >= 2
enhance_architecture = enhance_level >= 2
ai_mode = "auto" if enhance_level > 0 else "none"
if enhance_level > 0:
level_names = {1: "SKILL.md only", 2: "SKILL.md+Architecture+Config", 3: "full"}
logger.info(f"🤖 AI Enhancement Level: {enhance_level} ({level_names.get(enhance_level, 'unknown')})")
# Resolve directory to absolute path to avoid relative_to() errors
directory = Path(directory).resolve()
@@ -405,7 +929,7 @@ def analyze_codebase(
logger.info("Detecting design patterns...")
from skill_seekers.cli.pattern_recognizer import PatternRecognizer
pattern_recognizer = PatternRecognizer(depth=depth, enhance_with_ai=enhance_with_ai)
pattern_recognizer = PatternRecognizer(depth=depth, enhance_with_ai=enhance_patterns)
pattern_results = []
for file_path in files:
@@ -447,7 +971,7 @@ def analyze_codebase(
min_confidence=0.5,
max_per_file=10,
languages=languages,
enhance_with_ai=enhance_with_ai,
enhance_with_ai=enhance_tests,
)
# Extract examples from directory
@@ -486,8 +1010,8 @@ def analyze_codebase(
try:
from skill_seekers.cli.how_to_guide_builder import HowToGuideBuilder
# Create guide builder
guide_builder = HowToGuideBuilder(enhance_with_ai=enhance_with_ai)
# Create guide builder (uses same enhance level as test examples)
guide_builder = HowToGuideBuilder(enhance_with_ai=enhance_tests)
# Build guides from workflow examples
tutorials_dir = output_dir / "tutorials"
@@ -505,7 +1029,7 @@ def analyze_codebase(
examples_list,
grouping_strategy="ai-tutorial-group",
output_dir=tutorials_dir,
enhance_with_ai=enhance_with_ai,
enhance_with_ai=enhance_tests,
ai_mode=ai_mode,
)
@@ -538,8 +1062,8 @@ def analyze_codebase(
# Convert to dict for enhancement
result_dict = config_extractor.to_dict(extraction_result)
# AI Enhancement (if enabled)
if enhance_with_ai and ai_mode != "none":
# AI Enhancement (if enabled - level 2+)
if enhance_config and ai_mode != "none":
try:
from skill_seekers.cli.config_enhancer import ConfigEnhancer
@@ -591,7 +1115,7 @@ def analyze_codebase(
logger.info("Analyzing architectural patterns...")
from skill_seekers.cli.architectural_pattern_detector import ArchitecturalPatternDetector
arch_detector = ArchitecturalPatternDetector(enhance_with_ai=enhance_with_ai)
arch_detector = ArchitecturalPatternDetector(enhance_with_ai=enhance_architecture)
arch_report = arch_detector.analyze(directory, results["files"])
if arch_report.patterns:
@@ -610,6 +1134,33 @@ def analyze_codebase(
else:
logger.info("No clear architectural patterns detected")
# Extract markdown documentation (C3.9)
docs_data = None
if extract_docs:
logger.info("Extracting project documentation...")
try:
# Determine AI enhancement for docs (level 2+)
enhance_docs_ai = enhance_level >= 2
docs_data = process_markdown_docs(
directory=directory,
output_dir=output_dir,
depth=depth,
gitignore_spec=gitignore_spec,
enhance_with_ai=enhance_docs_ai,
ai_mode=ai_mode,
)
if docs_data and docs_data.get("total_files", 0) > 0:
logger.info(
f"✅ Extracted {docs_data['total_files']} documentation files "
f"in {len(docs_data.get('categories', {}))} categories"
)
else:
logger.info("No markdown documentation files found")
except Exception as e:
logger.warning(f"Documentation extraction failed: {e}")
docs_data = None
# Generate SKILL.md and references/ directory
logger.info("Generating SKILL.md and references...")
_generate_skill_md(
@@ -622,6 +1173,8 @@ def analyze_codebase(
detect_patterns=detect_patterns,
extract_test_examples=extract_test_examples,
extract_config_patterns=extract_config_patterns,
extract_docs=extract_docs,
docs_data=docs_data,
)
return results
@@ -637,6 +1190,8 @@ def _generate_skill_md(
detect_patterns: bool,
extract_test_examples: bool,
extract_config_patterns: bool,
extract_docs: bool = True,
docs_data: dict[str, Any] | None = None,
):
"""
Generate rich SKILL.md from codebase analysis results.
@@ -716,7 +1271,10 @@ Use this skill when you need to:
skill_content += "- ✅ Test Examples (C3.2)\n"
if extract_config_patterns:
skill_content += "- ✅ Configuration Patterns (C3.4)\n"
skill_content += "- ✅ Architectural Analysis (C3.7)\n\n"
skill_content += "- ✅ Architectural Analysis (C3.7)\n"
if extract_docs:
skill_content += "- ✅ Project Documentation (C3.9)\n"
skill_content += "\n"
# Add design patterns if available
if detect_patterns:
@@ -747,6 +1305,12 @@ Use this skill when you need to:
if config_content:
skill_content += config_content
# Add project documentation if available
if extract_docs and docs_data:
docs_content = _format_documentation_section(output_dir, docs_data)
if docs_content:
skill_content += docs_content
# Available references
skill_content += "## 📚 Available References\n\n"
skill_content += "This skill includes detailed reference documentation:\n\n"
@@ -776,6 +1340,9 @@ Use this skill when you need to:
if (output_dir / "architecture").exists():
skill_content += "- **Architecture**: `references/architecture/` - Architectural patterns\n"
refs_added = True
if extract_docs and (output_dir / "documentation").exists():
skill_content += "- **Documentation**: `references/documentation/` - Project documentation\n"
refs_added = True
if not refs_added:
skill_content += "No additional references generated (analysis features disabled).\n"
@@ -1005,6 +1572,75 @@ def _format_config_section(output_dir: Path) -> str:
return content
def _format_documentation_section(output_dir: Path, docs_data: dict[str, Any]) -> str:
"""Format project documentation section from extracted markdown files."""
if not docs_data or docs_data.get("total_files", 0) == 0:
return ""
categories = docs_data.get("categories", {})
files = docs_data.get("files", [])
content = "## 📖 Project Documentation\n\n"
content += "*Extracted from markdown files in the project (C3.9)*\n\n"
content += f"**Total Documentation Files:** {docs_data['total_files']}\n"
content += f"**Categories:** {len(categories)}\n\n"
# List documents by category (most important first)
priority_order = ["overview", "architecture", "guides", "workflows", "features", "api", "examples"]
# Sort categories by priority
sorted_categories = []
for cat in priority_order:
if cat in categories:
sorted_categories.append(cat)
for cat in sorted(categories.keys()):
if cat not in sorted_categories:
sorted_categories.append(cat)
for category in sorted_categories[:6]: # Limit to 6 categories in SKILL.md
cat_files = categories[category]
content += f"### {category.title()}\n\n"
# Get file details for this category
cat_docs = [f for f in files if f.get("category") == category]
for doc in cat_docs[:5]: # Limit to 5 docs per category
title = doc.get("title") or doc.get("filename", "Unknown")
path = doc.get("path", "")
# Add summary if available (deep/full depth)
if doc.get("ai_description"):
content += f"- **{title}**: {doc['ai_description']}\n"
elif doc.get("summary"):
# Extract first sentence from summary
summary = doc["summary"].split("\n")[0]
if len(summary) > 100:
summary = summary[:100] + "..."
content += f"- **{title}**: {summary}\n"
else:
content += f"- **{title}** (`{path}`)\n"
if len(cat_files) > 5:
content += f"- *...and {len(cat_files) - 5} more*\n"
content += "\n"
# AI-enhanced topics if available
all_topics = []
for doc in files:
all_topics.extend(doc.get("ai_topics", []))
if all_topics:
# Deduplicate and count
from collections import Counter
topic_counts = Counter(all_topics)
top_topics = [t for t, _ in topic_counts.most_common(10)]
content += f"**Key Topics:** {', '.join(top_topics)}\n\n"
content += "*See `references/documentation/` for all project documentation*\n\n"
return content
def _generate_references(output_dir: Path):
"""
Generate references/ directory structure by symlinking analysis output.
@@ -1023,6 +1659,7 @@ def _generate_references(output_dir: Path):
"tutorials": "tutorials",
"config_patterns": "config_patterns",
"architecture": "architecture",
"documentation": "documentation",
}
for source, target in mappings.items():
@@ -1132,6 +1769,12 @@ Examples:
default=False,
help="Skip configuration pattern extraction from config files (JSON, YAML, TOML, ENV, etc.) (default: enabled)",
)
parser.add_argument(
"--skip-docs",
action="store_true",
default=False,
help="Skip project documentation extraction from markdown files (README, docs/, etc.) (default: enabled)",
)
parser.add_argument(
"--ai-mode",
choices=["auto", "api", "local", "none"],
@@ -1147,6 +1790,19 @@ Examples:
)
parser.add_argument("--no-comments", action="store_true", help="Skip comment extraction")
parser.add_argument("--verbose", action="store_true", help="Enable verbose logging")
parser.add_argument(
"--enhance-level",
type=int,
choices=[0, 1, 2, 3],
default=0,
help=(
"AI enhancement level: "
"0=off (default), "
"1=SKILL.md only, "
"2=SKILL.md+Architecture+Config, "
"3=full (patterns, tests, config, architecture, SKILL.md)"
),
)
# Check for deprecated flags
deprecated_flags = {
@@ -1232,8 +1888,8 @@ Examples:
extract_test_examples=not args.skip_test_examples,
build_how_to_guides=not args.skip_how_to_guides,
extract_config_patterns=not args.skip_config_patterns,
enhance_with_ai=True, # Auto-disables if no API key present
ai_mode=args.ai_mode, # NEW: AI enhancement mode for how-to guides
extract_docs=not args.skip_docs,
enhance_level=args.enhance_level, # AI enhancement level (0-3)
)
# Print summary

View File

@@ -165,12 +165,16 @@ class ConfigEnhancer:
for cf in config_files[:10]: # Limit to first 10 files
settings_summary = []
for setting in cf.get("settings", [])[:5]: # First 5 settings per file
# Support both "type" (from config_extractor) and "value_type" (legacy)
value_type = setting.get("type", setting.get("value_type", "unknown"))
settings_summary.append(
f" - {setting['key']}: {setting['value']} ({setting['value_type']})"
f" - {setting['key']}: {setting['value']} ({value_type})"
)
# Support both "type" (from config_extractor) and "config_type" (legacy)
config_type = cf.get("type", cf.get("config_type", "unknown"))
config_summary.append(f"""
File: {cf["relative_path"]} ({cf["config_type"]})
File: {cf["relative_path"]} ({config_type})
Purpose: {cf["purpose"]}
Settings:
{chr(10).join(settings_summary)}
@@ -252,124 +256,184 @@ Focus on actionable insights that help developers understand and improve their c
def _enhance_via_local(self, result: dict) -> dict:
"""Enhance configs using Claude Code CLI"""
try:
# Create temporary prompt file
with tempfile.NamedTemporaryFile(mode="w", suffix=".md", delete=False) as f:
prompt_file = Path(f.name)
f.write(self._create_local_prompt(result))
# Create a temporary directory for this enhancement session
with tempfile.TemporaryDirectory(prefix="config_enhance_") as temp_dir:
temp_path = Path(temp_dir)
# Create output file path
output_file = prompt_file.parent / f"{prompt_file.stem}_enhanced.json"
# Define output file path (absolute path that Claude will write to)
output_file = temp_path / "config_enhancement.json"
logger.info("🖥️ Launching Claude Code CLI for config analysis...")
logger.info("⏱️ This will take 30-60 seconds...")
# Create prompt file with the output path embedded
prompt_file = temp_path / "enhance_prompt.md"
prompt_content = self._create_local_prompt(result, output_file)
prompt_file.write_text(prompt_content)
# Run Claude Code CLI
result_data = self._run_claude_cli(prompt_file, output_file)
logger.info("🖥️ Launching Claude Code CLI for config analysis...")
logger.info("⏱️ This will take 30-60 seconds...")
# Clean up
prompt_file.unlink()
if output_file.exists():
output_file.unlink()
# Run Claude Code CLI
result_data = self._run_claude_cli(prompt_file, output_file, temp_path)
if result_data:
# Merge LOCAL enhancements
result["ai_enhancements"] = result_data
logger.info("✅ LOCAL enhancement complete")
return result
else:
logger.warning("⚠️ LOCAL enhancement produced no results")
return result
if result_data:
# Merge LOCAL enhancements
result["ai_enhancements"] = result_data
logger.info("✅ LOCAL enhancement complete")
return result
else:
logger.warning("⚠️ LOCAL enhancement produced no results")
return result
except Exception as e:
logger.error(f"❌ LOCAL enhancement failed: {e}")
return result
def _create_local_prompt(self, result: dict) -> str:
"""Create prompt file for Claude Code CLI"""
def _create_local_prompt(self, result: dict, output_file: Path) -> str:
"""Create prompt file for Claude Code CLI
Args:
result: Config extraction result dict
output_file: Absolute path where Claude should write the JSON output
Returns:
Prompt content string
"""
config_files = result.get("config_files", [])
# Format config data for Claude
# Format config data for Claude (limit to 15 files for reasonable prompt size)
config_data = []
for cf in config_files[:10]:
for cf in config_files[:15]:
# Support both "type" (from config_extractor) and "config_type" (legacy)
config_type = cf.get("type", cf.get("config_type", "unknown"))
settings_preview = []
for s in cf.get("settings", [])[:3]: # Show first 3 settings
settings_preview.append(f" - {s.get('key', 'unknown')}: {str(s.get('value', ''))[:50]}")
config_data.append(f"""
### {cf["relative_path"]} ({cf["config_type"]})
### {cf["relative_path"]} ({config_type})
- Purpose: {cf["purpose"]}
- Patterns: {", ".join(cf.get("patterns", []))}
- Settings count: {len(cf.get("settings", []))}
- Patterns: {", ".join(cf.get("patterns", [])) or "none detected"}
- Settings: {len(cf.get("settings", []))} total
{chr(10).join(settings_preview) if settings_preview else " (no settings)"}
""")
prompt = f"""# Configuration Analysis Task
I need you to analyze these configuration files and provide AI-enhanced insights.
IMPORTANT: You MUST write the output to this EXACT file path:
{output_file}
## Configuration Files ({len(config_files)} total)
## Configuration Files ({len(config_files)} total, showing first 15)
{chr(10).join(config_data)}
## Your Task
Analyze these configs and create a JSON file with the following structure:
Analyze these configuration files and write a JSON file to the path specified above.
The JSON must have this EXACT structure:
```json
{{
"file_enhancements": [
{{
"file_path": "path/to/file",
"explanation": "What this config does",
"best_practice": "Suggested improvements",
"security_concern": "Security issues (if any)",
"migration_suggestion": "Consolidation opportunities",
"context": "Pattern explanation"
"file_path": "relative/path/to/config.json",
"explanation": "Brief explanation of what this config file does",
"best_practice": "Suggested improvement or 'None'",
"security_concern": "Security issue if any, or 'None'",
"migration_suggestion": "Consolidation opportunity or 'None'",
"context": "What pattern or purpose this serves"
}}
],
"overall_insights": {{
"config_count": {len(config_files)},
"security_issues_found": 0,
"consolidation_opportunities": [],
"recommended_actions": []
"consolidation_opportunities": ["List of suggestions"],
"recommended_actions": ["List of actions"]
}}
}}
```
Please save the JSON output to a file named `config_enhancement.json` in the current directory.
## Instructions
Focus on actionable insights:
1. Explain what each config does
2. Suggest best practices
3. Identify security concerns (hardcoded secrets, exposed credentials)
4. Suggest consolidation opportunities
5. Explain the detected patterns
1. Use the Write tool to create the JSON file at: {output_file}
2. Include an enhancement entry for each config file shown above
3. Focus on actionable insights:
- Explain what each config does in 1-2 sentences
- Identify any hardcoded secrets or security issues
- Suggest consolidation if configs have overlapping settings
- Note any missing best practices
DO NOT explain your work - just write the JSON file directly.
"""
return prompt
def _run_claude_cli(self, prompt_file: Path, _output_file: Path) -> dict | None:
"""Run Claude Code CLI and wait for completion"""
def _run_claude_cli(
self, prompt_file: Path, output_file: Path, working_dir: Path
) -> dict | None:
"""Run Claude Code CLI and wait for completion
Args:
prompt_file: Path to the prompt markdown file
output_file: Expected path where Claude will write the JSON output
working_dir: Working directory to run Claude from
Returns:
Parsed JSON dict if successful, None otherwise
"""
import time
try:
# Run claude command
start_time = time.time()
# Run claude command with --dangerously-skip-permissions to bypass all prompts
# This allows Claude to write files without asking for confirmation
logger.info(f" Running: claude --dangerously-skip-permissions {prompt_file.name}")
logger.info(f" Output expected at: {output_file}")
result = subprocess.run(
["claude", str(prompt_file)],
["claude", "--dangerously-skip-permissions", str(prompt_file)],
capture_output=True,
text=True,
timeout=300, # 5 minute timeout
cwd=str(working_dir),
)
elapsed = time.time() - start_time
logger.info(f" Claude finished in {elapsed:.1f} seconds")
if result.returncode != 0:
logger.error(f"❌ Claude CLI failed: {result.stderr}")
logger.error(f"❌ Claude CLI failed (exit code {result.returncode})")
if result.stderr:
logger.error(f" Error: {result.stderr[:200]}")
return None
# Try to find output file (Claude might save it with different name)
# Look for JSON files created in the last minute
import time
# Check if the expected output file was created
if output_file.exists():
try:
with open(output_file) as f:
data = json.load(f)
if "file_enhancements" in data or "overall_insights" in data:
logger.info(f"✅ Found enhancement data in {output_file.name}")
return data
else:
logger.warning("⚠️ Output file exists but missing expected keys")
except json.JSONDecodeError as e:
logger.error(f"❌ Failed to parse output JSON: {e}")
return None
# Fallback: Look for any JSON files created in the working directory
logger.info(" Looking for JSON files in working directory...")
current_time = time.time()
potential_files = []
for json_file in prompt_file.parent.glob("*.json"):
if current_time - json_file.stat().st_mtime < 120: # Created in last 2 minutes
for json_file in working_dir.glob("*.json"):
# Check if created recently (within last 2 minutes)
if current_time - json_file.stat().st_mtime < 120:
potential_files.append(json_file)
# Try to load the most recent JSON file
for json_file in sorted(potential_files, key=lambda f: f.stat().st_mtime, reverse=True):
# Try to load the most recent JSON file with expected structure
for json_file in sorted(
potential_files, key=lambda f: f.stat().st_mtime, reverse=True
):
try:
with open(json_file) as f:
data = json.load(f)
@@ -380,11 +444,17 @@ Focus on actionable insights:
continue
logger.warning("⚠️ Could not find enhancement output file")
logger.info(f" Expected file: {output_file}")
logger.info(f" Files in dir: {list(working_dir.glob('*'))}")
return None
except subprocess.TimeoutExpired:
logger.error("❌ Claude CLI timeout (5 minutes)")
return None
except FileNotFoundError:
logger.error("'claude' command not found. Is Claude Code CLI installed?")
logger.error(" Install with: npm install -g @anthropic-ai/claude-code")
return None
except Exception as e:
logger.error(f"❌ Error running Claude CLI: {e}")
return None

View File

@@ -34,6 +34,11 @@ class ConfigManager:
},
"resume": {"auto_save_interval_seconds": 60, "keep_progress_days": 7},
"api_keys": {"anthropic": None, "google": None, "openai": None},
"ai_enhancement": {
"default_enhance_level": 1, # Default AI enhancement level (0-3)
"local_batch_size": 20, # Patterns per Claude CLI call (default was 5)
"local_parallel_workers": 3, # Concurrent Claude CLI calls
},
"first_run": {"completed": False, "version": "2.7.0"},
}
@@ -378,6 +383,43 @@ class ConfigManager:
if deleted_count > 0:
print(f"🧹 Cleaned up {deleted_count} old progress file(s)")
# AI Enhancement Settings
def get_default_enhance_level(self) -> int:
"""Get default AI enhancement level (0-3)."""
return self.config.get("ai_enhancement", {}).get("default_enhance_level", 1)
def set_default_enhance_level(self, level: int):
"""Set default AI enhancement level (0-3)."""
if level not in [0, 1, 2, 3]:
raise ValueError("enhance_level must be 0, 1, 2, or 3")
if "ai_enhancement" not in self.config:
self.config["ai_enhancement"] = {}
self.config["ai_enhancement"]["default_enhance_level"] = level
self.save_config()
def get_local_batch_size(self) -> int:
"""Get batch size for LOCAL mode AI enhancement."""
return self.config.get("ai_enhancement", {}).get("local_batch_size", 20)
def set_local_batch_size(self, size: int):
"""Set batch size for LOCAL mode AI enhancement."""
if "ai_enhancement" not in self.config:
self.config["ai_enhancement"] = {}
self.config["ai_enhancement"]["local_batch_size"] = size
self.save_config()
def get_local_parallel_workers(self) -> int:
"""Get number of parallel workers for LOCAL mode AI enhancement."""
return self.config.get("ai_enhancement", {}).get("local_parallel_workers", 3)
def set_local_parallel_workers(self, workers: int):
"""Set number of parallel workers for LOCAL mode AI enhancement."""
if "ai_enhancement" not in self.config:
self.config["ai_enhancement"] = {}
self.config["ai_enhancement"]["local_parallel_workers"] = workers
self.save_config()
# First Run Experience
def is_first_run(self) -> bool:
@@ -443,6 +485,14 @@ class ConfigManager:
print(f" • Auto-switch profiles: {self.config['rate_limit']['auto_switch_profiles']}")
print(f" • Keep progress for: {self.config['resume']['keep_progress_days']} days")
# AI Enhancement settings
level_names = {0: "off", 1: "SKILL.md only", 2: "standard", 3: "full"}
default_level = self.get_default_enhance_level()
print("\nAI Enhancement:")
print(f" • Default level: {default_level} ({level_names.get(default_level, 'unknown')})")
print(f" • Batch size: {self.get_local_batch_size()} patterns per call")
print(f" • Parallel workers: {self.get_local_parallel_workers()} concurrent calls")
# Resumable jobs
jobs = self.list_resumable_jobs()
if jobs:

View File

@@ -34,6 +34,7 @@ Examples:
import argparse
import sys
from pathlib import Path
from skill_seekers.cli import __version__
@@ -299,7 +300,14 @@ For more information: https://github.com/yusufkaraaslan/Skill_Seekers
)
analyze_parser.add_argument("--file-patterns", help="Comma-separated file patterns")
analyze_parser.add_argument(
"--enhance", action="store_true", help="Enable AI enhancement (auto-detects API or LOCAL)"
"--enhance", action="store_true", help="Enable AI enhancement (default level 1 = SKILL.md only)"
)
analyze_parser.add_argument(
"--enhance-level",
type=int,
choices=[0, 1, 2, 3],
default=None,
help="AI enhancement level: 0=off, 1=SKILL.md only (default), 2=+Architecture+Config, 3=full"
)
analyze_parser.add_argument("--skip-api-reference", action="store_true", help="Skip API docs")
analyze_parser.add_argument("--skip-dependency-graph", action="store_true", help="Skip dep graph")
@@ -307,6 +315,7 @@ For more information: https://github.com/yusufkaraaslan/Skill_Seekers
analyze_parser.add_argument("--skip-test-examples", action="store_true", help="Skip test examples")
analyze_parser.add_argument("--skip-how-to-guides", action="store_true", help="Skip guides")
analyze_parser.add_argument("--skip-config-patterns", action="store_true", help="Skip config")
analyze_parser.add_argument("--skip-docs", action="store_true", help="Skip project docs (README, docs/)")
analyze_parser.add_argument("--no-comments", action="store_true", help="Skip comments")
analyze_parser.add_argument("--verbose", action="store_true", help="Verbose logging")
@@ -547,9 +556,9 @@ def main(argv: list[str] | None = None) -> int:
if args.output:
sys.argv.extend(["--output", args.output])
# Handle preset flags (new)
# Handle preset flags (depth and features)
if args.quick:
# Quick = surface depth + skip advanced features
# Quick = surface depth + skip advanced features + no AI
sys.argv.extend([
"--depth", "surface",
"--skip-patterns",
@@ -558,17 +567,35 @@ def main(argv: list[str] | None = None) -> int:
"--skip-config-patterns",
])
elif args.comprehensive:
# Comprehensive = full depth + all features + AI
sys.argv.extend(["--depth", "full", "--ai-mode", "auto"])
# Comprehensive = full depth + all features (AI level is separate)
sys.argv.extend(["--depth", "full"])
elif args.depth:
sys.argv.extend(["--depth", args.depth])
# Determine enhance_level (independent of --comprehensive)
# Priority: explicit --enhance-level > --enhance (uses config default) > --quick (level 0) > 0
if args.enhance_level is not None:
enhance_level = args.enhance_level
elif args.quick:
enhance_level = 0 # Quick mode disables AI
elif args.enhance:
# Use default from config (default: 1)
try:
from skill_seekers.cli.config_manager import get_config_manager
config = get_config_manager()
enhance_level = config.get_default_enhance_level()
except Exception:
enhance_level = 1 # Fallback to level 1
else:
enhance_level = 0 # Default: no AI
# Pass enhance_level to codebase_scraper
sys.argv.extend(["--enhance-level", str(enhance_level)])
if args.languages:
sys.argv.extend(["--languages", args.languages])
if args.file_patterns:
sys.argv.extend(["--file-patterns", args.file_patterns])
if args.enhance:
sys.argv.extend(["--ai-mode", "auto"])
# Pass through skip flags
if args.skip_api_reference:
@@ -583,12 +610,51 @@ def main(argv: list[str] | None = None) -> int:
sys.argv.append("--skip-how-to-guides")
if args.skip_config_patterns:
sys.argv.append("--skip-config-patterns")
if args.skip_docs:
sys.argv.append("--skip-docs")
if args.no_comments:
sys.argv.append("--no-comments")
if args.verbose:
sys.argv.append("--verbose")
return analyze_main() or 0
result = analyze_main() or 0
# Enhance SKILL.md if enhance_level >= 1
if result == 0 and enhance_level >= 1:
skill_dir = Path(args.output)
skill_md = skill_dir / "SKILL.md"
if skill_md.exists():
print("\n" + "=" * 60)
print(f"ENHANCING SKILL.MD WITH AI (Level {enhance_level})")
print("=" * 60 + "\n")
try:
from skill_seekers.cli.enhance_skill_local import LocalSkillEnhancer
enhancer = LocalSkillEnhancer(str(skill_dir), force=True)
# Use headless mode (runs claude directly, waits for completion)
success = enhancer.run(
headless=True,
timeout=600, # 10 minute timeout
)
if success:
print("\n✅ SKILL.md enhancement complete!")
# Re-read line count
with open(skill_md) as f:
lines = len(f.readlines())
print(f" Enhanced SKILL.md: {lines} lines")
else:
print("\n⚠️ SKILL.md enhancement did not complete")
print(" You can retry with: skill-seekers enhance " + str(skill_dir))
except Exception as e:
print(f"\n⚠️ SKILL.md enhancement failed: {e}")
print(" You can retry with: skill-seekers enhance " + str(skill_dir))
else:
print(f"\n⚠️ SKILL.md not found at {skill_md}, skipping enhancement")
return result
elif args.command == "install-agent":
from skill_seekers.cli.install_agent import main as install_agent_main

View File

@@ -651,9 +651,20 @@ class GenericTestAnalyzer:
"test_function": r"@Test\s+public\s+void\s+(\w+)\(\)",
},
"csharp": {
"instantiation": r"var\s+(\w+)\s*=\s*new\s+(\w+)\(([^)]*)\)",
"assertion": r"Assert\.(?:AreEqual|IsTrue|IsFalse|IsNotNull)\(([^)]+)\)",
"test_function": r"\[Test\]\s+public\s+void\s+(\w+)\(\)",
# Object instantiation patterns (var, explicit type, generic)
"instantiation": r"(?:var|[\w<>]+)\s+(\w+)\s*=\s*new\s+([\w<>]+)\(([^)]*)\)",
# NUnit assertions (Assert.AreEqual, Assert.That, etc.)
"assertion": r"Assert\.(?:AreEqual|AreNotEqual|IsTrue|IsFalse|IsNull|IsNotNull|That|Throws|DoesNotThrow|Greater|Less|Contains)\(([^)]+)\)",
# NUnit test attributes ([Test], [TestCase], [TestCaseSource])
"test_function": r"\[(?:Test|TestCase|TestCaseSource|Theory|Fact)\(?[^\]]*\)?\]\s*(?:\[[\w\(\)\"',\s]+\]\s*)*public\s+(?:async\s+)?(?:Task|void)\s+(\w+)\s*\(",
# Setup/Teardown patterns
"setup": r"\[(?:SetUp|OneTimeSetUp|TearDown|OneTimeTearDown)\]\s*public\s+(?:async\s+)?(?:Task|void)\s+(\w+)\s*\(",
# Mock/substitute patterns (NSubstitute, Moq)
"mock": r"(?:Substitute\.For<([\w<>]+)>|new\s+Mock<([\w<>]+)>|MockRepository\.GenerateMock<([\w<>]+)>)\(",
# Dependency injection patterns (Zenject, etc.)
"injection": r"Container\.(?:Bind|BindInterfacesTo|BindInterfacesAndSelfTo)<([\w<>]+)>",
# Configuration/setup dictionaries
"config": r"(?:var|[\w<>]+)\s+\w+\s*=\s*new\s+(?:Dictionary|List|HashSet)<[^>]+>\s*\{[\s\S]{20,500}?\}",
},
"php": {
"instantiation": r"\$(\w+)\s*=\s*new\s+(\w+)\(([^)]*)\)",
@@ -667,11 +678,21 @@ class GenericTestAnalyzer:
},
}
# Language name normalization mapping
LANGUAGE_ALIASES = {
"c#": "csharp",
"c++": "cpp",
"c plus plus": "cpp",
}
def extract(self, file_path: str, code: str, language: str) -> list[TestExample]:
"""Extract examples from test file using regex patterns"""
examples = []
language_lower = language.lower()
# Normalize language name (e.g., "C#" -> "csharp")
language_lower = self.LANGUAGE_ALIASES.get(language_lower, language_lower)
if language_lower not in self.PATTERNS:
logger.warning(f"Language {language} not supported for regex extraction")
return []
@@ -715,6 +736,54 @@ class GenericTestAnalyzer:
)
examples.append(example)
# Extract mock/substitute patterns (if pattern exists)
if "mock" in patterns:
for mock_match in re.finditer(patterns["mock"], test_body):
example = self._create_example(
test_name=test_name,
category="setup",
code=mock_match.group(0),
language=language,
file_path=file_path,
line_number=code[: start_pos + mock_match.start()].count("\n") + 1,
)
examples.append(example)
# Extract dependency injection patterns (if pattern exists)
if "injection" in patterns:
for inject_match in re.finditer(patterns["injection"], test_body):
example = self._create_example(
test_name=test_name,
category="setup",
code=inject_match.group(0),
language=language,
file_path=file_path,
line_number=code[: start_pos + inject_match.start()].count("\n") + 1,
)
examples.append(example)
# Also extract setup/teardown methods (outside test functions)
if "setup" in patterns:
for setup_match in re.finditer(patterns["setup"], code):
setup_name = setup_match.group(1)
# Get setup function body
setup_start = setup_match.end()
# Find next method (setup or test)
next_pattern = patterns.get("setup", patterns["test_function"])
next_setup = re.search(next_pattern, code[setup_start:])
setup_end = setup_start + next_setup.start() if next_setup else min(setup_start + 500, len(code))
setup_body = code[setup_start:setup_end]
example = self._create_example(
test_name=setup_name,
category="setup",
code=setup_match.group(0) + setup_body[:200], # Include some of the body
language=language,
file_path=file_path,
line_number=code[: setup_match.start()].count("\n") + 1,
)
examples.append(example)
return examples
def _create_example(

View File

@@ -441,8 +441,10 @@ async def scrape_codebase_tool(args: dict) -> list[TextContent]:
Analyze local codebase and extract code knowledge.
Walks directory tree, analyzes code files, extracts signatures,
docstrings, and optionally generates API reference documentation
and dependency graphs.
docstrings, and generates API reference documentation, dependency graphs,
design patterns, test examples, and how-to guides.
All features are ON by default. Use skip_* parameters to disable specific features.
Args:
args: Dictionary containing:
@@ -451,8 +453,18 @@ async def scrape_codebase_tool(args: dict) -> list[TextContent]:
- depth (str, optional): Analysis depth - surface, deep, full (default: deep)
- languages (str, optional): Comma-separated languages (e.g., "Python,JavaScript,C++")
- file_patterns (str, optional): Comma-separated file patterns (e.g., "*.py,src/**/*.js")
- build_api_reference (bool, optional): Generate API reference markdown (default: False)
- build_dependency_graph (bool, optional): Generate dependency graph and detect circular dependencies (default: False)
- enhance_level (int, optional): AI enhancement level 0-3 (default: 0)
- 0: No AI enhancement
- 1: SKILL.md enhancement only
- 2: SKILL.md + Architecture + Config enhancement
- 3: Full enhancement (patterns, tests, config, architecture, SKILL.md)
- skip_api_reference (bool, optional): Skip API reference generation (default: False)
- skip_dependency_graph (bool, optional): Skip dependency graph (default: False)
- skip_patterns (bool, optional): Skip design pattern detection (default: False)
- skip_test_examples (bool, optional): Skip test example extraction (default: False)
- skip_how_to_guides (bool, optional): Skip how-to guide generation (default: False)
- skip_config_patterns (bool, optional): Skip config pattern extraction (default: False)
- skip_docs (bool, optional): Skip project documentation extraction (default: False)
Returns:
List[TextContent]: Tool execution results
@@ -461,8 +473,12 @@ async def scrape_codebase_tool(args: dict) -> list[TextContent]:
scrape_codebase(
directory="/path/to/repo",
depth="deep",
build_api_reference=True,
build_dependency_graph=True
enhance_level=1
)
scrape_codebase(
directory="/path/to/repo",
enhance_level=2,
skip_patterns=True
)
"""
directory = args.get("directory")
@@ -473,8 +489,16 @@ async def scrape_codebase_tool(args: dict) -> list[TextContent]:
depth = args.get("depth", "deep")
languages = args.get("languages", "")
file_patterns = args.get("file_patterns", "")
build_api_reference = args.get("build_api_reference", False)
build_dependency_graph = args.get("build_dependency_graph", False)
enhance_level = args.get("enhance_level", 0)
# Skip flags (features are ON by default)
skip_api_reference = args.get("skip_api_reference", False)
skip_dependency_graph = args.get("skip_dependency_graph", False)
skip_patterns = args.get("skip_patterns", False)
skip_test_examples = args.get("skip_test_examples", False)
skip_how_to_guides = args.get("skip_how_to_guides", False)
skip_config_patterns = args.get("skip_config_patterns", False)
skip_docs = args.get("skip_docs", False)
# Build command
cmd = [sys.executable, "-m", "skill_seekers.cli.codebase_scraper"]
@@ -488,15 +512,38 @@ async def scrape_codebase_tool(args: dict) -> list[TextContent]:
cmd.extend(["--languages", languages])
if file_patterns:
cmd.extend(["--file-patterns", file_patterns])
if build_api_reference:
cmd.append("--build-api-reference")
if build_dependency_graph:
cmd.append("--build-dependency-graph")
if enhance_level > 0:
cmd.extend(["--enhance-level", str(enhance_level)])
timeout = 600 # 10 minutes for codebase analysis
# Skip flags
if skip_api_reference:
cmd.append("--skip-api-reference")
if skip_dependency_graph:
cmd.append("--skip-dependency-graph")
if skip_patterns:
cmd.append("--skip-patterns")
if skip_test_examples:
cmd.append("--skip-test-examples")
if skip_how_to_guides:
cmd.append("--skip-how-to-guides")
if skip_config_patterns:
cmd.append("--skip-config-patterns")
if skip_docs:
cmd.append("--skip-docs")
# Adjust timeout based on enhance_level
timeout = 600 # 10 minutes base
if enhance_level >= 2:
timeout = 1200 # 20 minutes with AI enhancement
if enhance_level >= 3:
timeout = 3600 # 60 minutes for full enhancement
level_names = {0: "off", 1: "SKILL.md only", 2: "standard", 3: "full"}
progress_msg = "🔍 Analyzing local codebase...\n"
progress_msg += f"📁 Directory: {directory}\n"
progress_msg += f"📊 Depth: {depth}\n"
if enhance_level > 0:
progress_msg += f"🤖 AI Enhancement: Level {enhance_level} ({level_names.get(enhance_level, 'unknown')})\n"
progress_msg += f"⏱️ Maximum time: {timeout // 60} minutes\n\n"
stdout, stderr, returncode = run_subprocess_with_streaming(cmd, timeout=timeout)

View File

@@ -74,7 +74,8 @@ class TestAnalyzeSubcommand(unittest.TestCase):
"--skip-patterns",
"--skip-test-examples",
"--skip-how-to-guides",
"--skip-config-patterns"
"--skip-config-patterns",
"--skip-docs"
])
self.assertTrue(args.skip_api_reference)
self.assertTrue(args.skip_dependency_graph)
@@ -82,6 +83,7 @@ class TestAnalyzeSubcommand(unittest.TestCase):
self.assertTrue(args.skip_test_examples)
self.assertTrue(args.skip_how_to_guides)
self.assertTrue(args.skip_config_patterns)
self.assertTrue(args.skip_docs)
def test_backward_compatible_depth_flag(self):
"""Test that deprecated --depth flag still works."""

View File

@@ -21,10 +21,17 @@ sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "src"))
from skill_seekers.cli.codebase_scraper import (
DEFAULT_EXCLUDED_DIRS,
FOLDER_CATEGORIES,
MARKDOWN_EXTENSIONS,
ROOT_DOC_CATEGORIES,
categorize_markdown_file,
detect_language,
extract_markdown_structure,
generate_markdown_summary,
load_gitignore,
should_exclude_dir,
walk_directory,
walk_markdown_files,
)
@@ -201,6 +208,191 @@ class TestGitignoreLoading(unittest.TestCase):
self.assertIsNotNone(spec)
class TestMarkdownDocumentation(unittest.TestCase):
"""Tests for markdown documentation extraction (C3.9)"""
def setUp(self):
"""Set up test environment"""
self.temp_dir = tempfile.mkdtemp()
self.root = Path(self.temp_dir)
def tearDown(self):
"""Clean up test environment"""
shutil.rmtree(self.temp_dir, ignore_errors=True)
def test_markdown_extensions(self):
"""Test that markdown extensions are properly defined."""
self.assertIn(".md", MARKDOWN_EXTENSIONS)
self.assertIn(".markdown", MARKDOWN_EXTENSIONS)
def test_root_doc_categories(self):
"""Test root document category mapping."""
self.assertEqual(ROOT_DOC_CATEGORIES.get("readme"), "overview")
self.assertEqual(ROOT_DOC_CATEGORIES.get("changelog"), "changelog")
self.assertEqual(ROOT_DOC_CATEGORIES.get("architecture"), "architecture")
def test_folder_categories(self):
"""Test folder category mapping."""
self.assertEqual(FOLDER_CATEGORIES.get("guides"), "guides")
self.assertEqual(FOLDER_CATEGORIES.get("tutorials"), "guides")
self.assertEqual(FOLDER_CATEGORIES.get("workflows"), "workflows")
self.assertEqual(FOLDER_CATEGORIES.get("architecture"), "architecture")
def test_walk_markdown_files(self):
"""Test walking directory for markdown files."""
# Create test markdown files
(self.root / "README.md").write_text("# Test README")
(self.root / "test.py").write_text("print('test')")
docs_dir = self.root / "docs"
docs_dir.mkdir()
(docs_dir / "guide.md").write_text("# Guide")
files = walk_markdown_files(self.root)
# Should find markdown files only
self.assertEqual(len(files), 2)
filenames = [f.name for f in files]
self.assertIn("README.md", filenames)
self.assertIn("guide.md", filenames)
def test_categorize_root_readme(self):
"""Test categorizing root README file."""
readme_path = self.root / "README.md"
readme_path.write_text("# Test")
category = categorize_markdown_file(readme_path, self.root)
self.assertEqual(category, "overview")
def test_categorize_changelog(self):
"""Test categorizing CHANGELOG file."""
changelog_path = self.root / "CHANGELOG.md"
changelog_path.write_text("# Changelog")
category = categorize_markdown_file(changelog_path, self.root)
self.assertEqual(category, "changelog")
def test_categorize_docs_guide(self):
"""Test categorizing file in docs/guides folder."""
guides_dir = self.root / "docs" / "guides"
guides_dir.mkdir(parents=True)
guide_path = guides_dir / "getting-started.md"
guide_path.write_text("# Getting Started")
category = categorize_markdown_file(guide_path, self.root)
self.assertEqual(category, "guides")
def test_categorize_architecture(self):
"""Test categorizing architecture documentation."""
arch_dir = self.root / "docs" / "architecture"
arch_dir.mkdir(parents=True)
arch_path = arch_dir / "overview.md"
arch_path.write_text("# Architecture")
category = categorize_markdown_file(arch_path, self.root)
self.assertEqual(category, "architecture")
class TestMarkdownStructureExtraction(unittest.TestCase):
"""Tests for markdown structure extraction"""
def test_extract_headers(self):
"""Test extracting headers from markdown."""
content = """# Main Title
## Section 1
Some content
### Subsection
More content
## Section 2
"""
structure = extract_markdown_structure(content)
self.assertEqual(structure["title"], "Main Title")
self.assertEqual(len(structure["headers"]), 4)
self.assertEqual(structure["headers"][0]["level"], 1)
self.assertEqual(structure["headers"][1]["level"], 2)
def test_extract_code_blocks(self):
"""Test extracting code blocks from markdown."""
content = """# Example
```python
def hello():
print("Hello")
```
```javascript
console.log("test");
```
"""
structure = extract_markdown_structure(content)
self.assertEqual(len(structure["code_blocks"]), 2)
self.assertEqual(structure["code_blocks"][0]["language"], "python")
self.assertEqual(structure["code_blocks"][1]["language"], "javascript")
def test_extract_links(self):
"""Test extracting links from markdown."""
content = """# Links
Check out [Example](https://example.com) and [Another](./local.md).
"""
structure = extract_markdown_structure(content)
self.assertEqual(len(structure["links"]), 2)
self.assertEqual(structure["links"][0]["text"], "Example")
self.assertEqual(structure["links"][0]["url"], "https://example.com")
def test_word_and_line_count(self):
"""Test word and line count."""
content = "First line\nSecond line\nThird line"
structure = extract_markdown_structure(content)
self.assertEqual(structure["line_count"], 3)
self.assertEqual(structure["word_count"], 6) # First, line, Second, line, Third, line
class TestMarkdownSummaryGeneration(unittest.TestCase):
"""Tests for markdown summary generation"""
def test_generate_summary_with_title(self):
"""Test summary includes title."""
content = "# My Title\n\nSome content here."
structure = extract_markdown_structure(content)
summary = generate_markdown_summary(content, structure)
self.assertIn("**My Title**", summary)
def test_generate_summary_with_sections(self):
"""Test summary includes section names."""
content = """# Main
## Getting Started
Content
## Installation
Content
## Usage
Content
"""
structure = extract_markdown_structure(content)
summary = generate_markdown_summary(content, structure)
self.assertIn("Sections:", summary)
def test_generate_summary_truncation(self):
"""Test summary is truncated to max length."""
content = "# Title\n\n" + "Long content. " * 100
structure = extract_markdown_structure(content)
summary = generate_markdown_summary(content, structure, max_length=200)
self.assertLessEqual(len(summary), 210) # Allow some buffer for truncation marker
if __name__ == "__main__":
# Run tests with verbose output
unittest.main(verbosity=2)

View File

@@ -307,6 +307,82 @@ fn test_subtract() {
self.assertGreater(len(examples), 0)
self.assertEqual(examples[0].language, "Rust")
def test_extract_csharp_nunit_tests(self):
"""Test C# NUnit test extraction"""
code = """
using NUnit.Framework;
using NSubstitute;
[TestFixture]
public class GameControllerTests
{
private IGameService _gameService;
private GameController _controller;
[SetUp]
public void SetUp()
{
_gameService = Substitute.For<IGameService>();
_controller = new GameController(_gameService);
}
[Test]
public void StartGame_ShouldInitializeBoard()
{
var config = new GameConfig { Rows = 8, Columns = 8 };
var board = new GameBoard(config);
_controller.StartGame(board);
Assert.IsTrue(board.IsInitialized);
Assert.AreEqual(64, board.CellCount);
}
[TestCase(1, 2)]
[TestCase(3, 4)]
public void MovePlayer_ShouldUpdatePosition(int x, int y)
{
var player = new Player("Test");
_controller.MovePlayer(player, x, y);
Assert.AreEqual(x, player.X);
Assert.AreEqual(y, player.Y);
}
}
"""
examples = self.analyzer.extract("GameControllerTests.cs", code, "C#")
# Should extract test functions and instantiations
self.assertGreater(len(examples), 0)
self.assertEqual(examples[0].language, "C#")
# Check that we found some instantiations
instantiations = [e for e in examples if e.category == "instantiation"]
self.assertGreater(len(instantiations), 0)
# Check for setup extraction
setups = [e for e in examples if e.category == "setup"]
# May or may not have setups depending on extraction
def test_extract_csharp_with_mocks(self):
"""Test C# mock pattern extraction (NSubstitute)"""
code = """
[Test]
public void ProcessOrder_ShouldCallPaymentService()
{
var paymentService = Substitute.For<IPaymentService>();
var orderProcessor = new OrderProcessor(paymentService);
orderProcessor.ProcessOrder(100);
paymentService.Received().Charge(100);
}
"""
examples = self.analyzer.extract("OrderTests.cs", code, "C#")
# Should extract instantiation and mock
self.assertGreater(len(examples), 0)
def test_language_fallback(self):
"""Test handling of unsupported languages"""
code = """