feat(A1.1): Add Config API endpoint with FastAPI backend

Implements Task A1.1 - Config Sharing JSON API

Features:
- FastAPI backend with 6 endpoints
- Config analyzer with auto-categorization
- Full metadata extraction (24 fields per config)
- Category/tag/type filtering
- Direct config download endpoint
- Render deployment configuration

Endpoints:
- GET / - API information
- GET /api/configs - List all configs (filterable)
- GET /api/configs/{name} - Get specific config
- GET /api/categories - List categories with counts
- GET /api/download/{config_name} - Download config file
- GET /health - Health check

Metadata:
- name, description, type (single-source/unified)
- category (8 auto-detected categories)
- tags (language, domain, tech)
- primary_source (URL/repo)
- max_pages, file_size, last_updated
- download_url (skillseekersweb.com)

Categories:
- web-frameworks (12 configs)
- game-engines (4 configs)
- devops (2 configs)
- css-frameworks (1 config)
- development-tools (1 config)
- gaming (1 config)
- testing (2 configs)
- uncategorized (1 config)

Deployment:
- Configured for Render via render.yaml
- Domain: skillseekersweb.com
- Auto-deploys from main branch

Tests:
-  All endpoints tested locally
-  24 configs discovered and analyzed
-  Filtering works (category/tag/type)
-  Download works for all configs

Issue: #9
Roadmap: FLEXIBLE_ROADMAP.md Task A1.1
This commit is contained in:
yusyus
2025-11-30 13:15:34 +03:00
parent a4e5025dd1
commit 13bcb6beda
7 changed files with 888 additions and 0 deletions

267
api/README.md Normal file
View File

@@ -0,0 +1,267 @@
# Skill Seekers Config API
FastAPI backend for discovering and downloading Skill Seekers configuration files.
## 🚀 Endpoints
### Base URL
- **Production**: `https://skillseekersweb.com`
- **Local**: `http://localhost:8000`
### Available Endpoints
#### 1. **GET /** - API Information
Returns API metadata and available endpoints.
```bash
curl https://skillseekersweb.com/
```
**Response:**
```json
{
"name": "Skill Seekers Config API",
"version": "1.0.0",
"endpoints": {
"/api/configs": "List all available configs",
"/api/configs/{name}": "Get specific config details",
"/api/categories": "List all categories",
"/docs": "API documentation"
},
"repository": "https://github.com/yusufkaraaslan/Skill_Seekers",
"website": "https://skillseekersweb.com"
}
```
---
#### 2. **GET /api/configs** - List All Configs
Returns list of all available configs with metadata.
**Query Parameters:**
- `category` (optional) - Filter by category (e.g., `web-frameworks`)
- `tag` (optional) - Filter by tag (e.g., `javascript`)
- `type` (optional) - Filter by type (`single-source` or `unified`)
```bash
# Get all configs
curl https://skillseekersweb.com/api/configs
# Filter by category
curl https://skillseekersweb.com/api/configs?category=web-frameworks
# Filter by tag
curl https://skillseekersweb.com/api/configs?tag=javascript
# Filter by type
curl https://skillseekersweb.com/api/configs?type=unified
```
**Response:**
```json
{
"version": "1.0.0",
"total": 24,
"filters": null,
"configs": [
{
"name": "react",
"description": "React framework for building user interfaces...",
"type": "single-source",
"category": "web-frameworks",
"tags": ["javascript", "frontend", "documentation"],
"primary_source": "https://react.dev/",
"max_pages": 300,
"file_size": 1055,
"last_updated": "2025-11-30T09:26:07+00:00",
"download_url": "https://skillseekersweb.com/api/download/react.json",
"config_file": "react.json"
}
]
}
```
---
#### 3. **GET /api/configs/{name}** - Get Specific Config
Returns detailed information about a specific config.
```bash
curl https://skillseekersweb.com/api/configs/react
```
**Response:**
```json
{
"name": "react",
"description": "React framework for building user interfaces...",
"type": "single-source",
"category": "web-frameworks",
"tags": ["javascript", "frontend", "documentation"],
"primary_source": "https://react.dev/",
"max_pages": 300,
"file_size": 1055,
"last_updated": "2025-11-30T09:26:07+00:00",
"download_url": "https://skillseekersweb.com/api/download/react.json",
"config_file": "react.json"
}
```
---
#### 4. **GET /api/categories** - List Categories
Returns all available categories with config counts.
```bash
curl https://skillseekersweb.com/api/categories
```
**Response:**
```json
{
"total_categories": 5,
"categories": {
"web-frameworks": 7,
"game-engines": 2,
"devops": 2,
"css-frameworks": 1,
"uncategorized": 12
}
}
```
---
#### 5. **GET /api/download/{config_name}** - Download Config File
Downloads the actual config JSON file.
```bash
# Download react config
curl -O https://skillseekersweb.com/api/download/react.json
# Download with just name (auto-adds .json)
curl -O https://skillseekersweb.com/api/download/react
```
---
#### 6. **GET /health** - Health Check
Health check endpoint for monitoring.
```bash
curl https://skillseekersweb.com/health
```
**Response:**
```json
{
"status": "healthy",
"service": "skill-seekers-api"
}
```
---
#### 7. **GET /docs** - API Documentation
Interactive OpenAPI documentation (Swagger UI).
Visit: `https://skillseekersweb.com/docs`
---
## 📦 Metadata Fields
Each config includes the following metadata:
| Field | Type | Description |
|-------|------|-------------|
| `name` | string | Config identifier (e.g., "react") |
| `description` | string | What the config is used for |
| `type` | string | "single-source" or "unified" |
| `category` | string | Auto-categorized (e.g., "web-frameworks") |
| `tags` | array | Relevant tags (e.g., ["javascript", "frontend"]) |
| `primary_source` | string | Main documentation URL or repo |
| `max_pages` | int | Estimated page count for scraping |
| `file_size` | int | Config file size in bytes |
| `last_updated` | string | ISO 8601 date of last update |
| `download_url` | string | Direct download link |
| `config_file` | string | Filename (e.g., "react.json") |
---
## 🏗️ Categories
Configs are auto-categorized into:
- **web-frameworks** - Web development frameworks (React, Django, FastAPI, etc.)
- **game-engines** - Game development engines (Godot, Unity, etc.)
- **devops** - DevOps tools (Kubernetes, Ansible, etc.)
- **css-frameworks** - CSS frameworks (Tailwind, etc.)
- **development-tools** - Dev tools (Claude Code, etc.)
- **gaming** - Gaming platforms (Steam, etc.)
- **uncategorized** - Other configs
---
## 🏷️ Tags
Common tags include:
- **Language**: `javascript`, `python`, `php`
- **Domain**: `frontend`, `backend`, `devops`, `game-development`
- **Type**: `documentation`, `github`, `pdf`, `multi-source`
- **Tech**: `css`, `testing`, `api`
---
## 🚀 Local Development
### Setup
```bash
# Install dependencies
cd api
pip install -r requirements.txt
# Run server
python main.py
```
API will be available at `http://localhost:8000`
### Testing
```bash
# Test health check
curl http://localhost:8000/health
# List all configs
curl http://localhost:8000/api/configs
# Get specific config
curl http://localhost:8000/api/configs/react
# Download config
curl -O http://localhost:8000/api/download/react.json
```
---
## 📝 Deployment
### Render
This API is configured for Render deployment via `render.yaml`.
1. Push to GitHub
2. Connect repository to Render
3. Render auto-deploys from `render.yaml`
4. Configure custom domain: `skillseekersweb.com`
---
## 🔗 Links
- **API Documentation**: https://skillseekersweb.com/docs
- **GitHub Repository**: https://github.com/yusufkaraaslan/Skill_Seekers
- **Main Project**: https://github.com/yusufkaraaslan/Skill_Seekers#readme

6
api/__init__.py Normal file
View File

@@ -0,0 +1,6 @@
"""
Skill Seekers Config API
FastAPI backend for discovering and downloading config files
"""
__version__ = "1.0.0"

348
api/config_analyzer.py Normal file
View File

@@ -0,0 +1,348 @@
#!/usr/bin/env python3
"""
Config Analyzer - Extract metadata from Skill Seekers config files
"""
import json
import os
import subprocess
from pathlib import Path
from typing import List, Dict, Any, Optional
from datetime import datetime
class ConfigAnalyzer:
"""Analyzes Skill Seekers config files and extracts metadata"""
# Category mapping based on config content
CATEGORY_MAPPING = {
"web-frameworks": [
"react", "vue", "django", "fastapi", "laravel", "astro", "hono"
],
"game-engines": [
"godot", "unity", "unreal"
],
"devops": [
"kubernetes", "ansible", "docker", "terraform"
],
"css-frameworks": [
"tailwind", "bootstrap", "bulma"
],
"development-tools": [
"claude-code", "vscode", "git"
],
"gaming": [
"steam"
],
"testing": [
"pytest", "jest", "test"
]
}
# Tag extraction keywords
TAG_KEYWORDS = {
"javascript": ["react", "vue", "astro", "hono", "javascript", "js", "node"],
"python": ["django", "fastapi", "ansible", "python", "flask"],
"php": ["laravel", "php"],
"frontend": ["react", "vue", "astro", "tailwind", "frontend", "ui"],
"backend": ["django", "fastapi", "laravel", "backend", "server", "api"],
"css": ["tailwind", "css", "styling"],
"game-development": ["godot", "unity", "unreal", "game"],
"devops": ["kubernetes", "ansible", "docker", "k8s", "devops"],
"documentation": ["docs", "documentation"],
"testing": ["test", "testing", "pytest", "jest"]
}
def __init__(self, config_dir: Path, base_url: str = "https://skillseekersweb.com"):
"""
Initialize config analyzer
Args:
config_dir: Path to configs directory
base_url: Base URL for download links
"""
self.config_dir = Path(config_dir)
self.base_url = base_url
if not self.config_dir.exists():
raise ValueError(f"Config directory not found: {self.config_dir}")
def analyze_all_configs(self) -> List[Dict[str, Any]]:
"""
Analyze all config files and extract metadata
Returns:
List of config metadata dicts
"""
configs = []
# Find all JSON files in configs directory
for config_file in sorted(self.config_dir.glob("*.json")):
try:
metadata = self.analyze_config(config_file)
if metadata: # Skip invalid configs
configs.append(metadata)
except Exception as e:
print(f"Warning: Failed to analyze {config_file.name}: {e}")
continue
return configs
def analyze_config(self, config_path: Path) -> Optional[Dict[str, Any]]:
"""
Analyze a single config file and extract metadata
Args:
config_path: Path to config JSON file
Returns:
Config metadata dict or None if invalid
"""
try:
# Read config file
with open(config_path, 'r') as f:
config_data = json.load(f)
# Skip if no name field
if "name" not in config_data:
return None
name = config_data["name"]
description = config_data.get("description", "")
# Determine config type
config_type = self._determine_type(config_data)
# Get primary source (base_url or repo)
primary_source = self._get_primary_source(config_data, config_type)
# Auto-categorize
category = self._categorize_config(name, description, config_data)
# Extract tags
tags = self._extract_tags(name, description, config_data)
# Get file metadata
file_size = config_path.stat().st_size
last_updated = self._get_last_updated(config_path)
# Generate download URL
download_url = f"{self.base_url}/api/download/{config_path.name}"
# Get max_pages (for estimation)
max_pages = self._get_max_pages(config_data)
return {
"name": name,
"description": description,
"type": config_type,
"category": category,
"tags": tags,
"primary_source": primary_source,
"max_pages": max_pages,
"file_size": file_size,
"last_updated": last_updated,
"download_url": download_url,
"config_file": config_path.name
}
except json.JSONDecodeError as e:
print(f"Invalid JSON in {config_path.name}: {e}")
return None
except Exception as e:
print(f"Error analyzing {config_path.name}: {e}")
return None
def get_config_by_name(self, name: str) -> Optional[Dict[str, Any]]:
"""
Get config metadata by name
Args:
name: Config name (e.g., "react", "django")
Returns:
Config metadata or None if not found
"""
configs = self.analyze_all_configs()
for config in configs:
if config["name"] == name:
return config
return None
def _determine_type(self, config_data: Dict[str, Any]) -> str:
"""
Determine if config is single-source or unified
Args:
config_data: Config JSON data
Returns:
"single-source" or "unified"
"""
# Unified configs have "sources" array
if "sources" in config_data:
return "unified"
# Check for merge_mode (another indicator of unified configs)
if "merge_mode" in config_data:
return "unified"
return "single-source"
def _get_primary_source(self, config_data: Dict[str, Any], config_type: str) -> str:
"""
Get primary source URL/repo
Args:
config_data: Config JSON data
config_type: "single-source" or "unified"
Returns:
Primary source URL or repo name
"""
if config_type == "unified":
# Get first source
sources = config_data.get("sources", [])
if sources:
first_source = sources[0]
if first_source.get("type") == "documentation":
return first_source.get("base_url", "")
elif first_source.get("type") == "github":
return f"github.com/{first_source.get('repo', '')}"
elif first_source.get("type") == "pdf":
return first_source.get("pdf_url", "PDF file")
return "Multiple sources"
# Single-source configs
if "base_url" in config_data:
return config_data["base_url"]
elif "repo" in config_data:
return f"github.com/{config_data['repo']}"
elif "pdf_url" in config_data or "pdf" in config_data:
return "PDF file"
return "Unknown"
def _categorize_config(self, name: str, description: str, config_data: Dict[str, Any]) -> str:
"""
Auto-categorize config based on name and content
Args:
name: Config name
description: Config description
config_data: Full config data
Returns:
Category name
"""
name_lower = name.lower()
# Check against category mapping
for category, keywords in self.CATEGORY_MAPPING.items():
if any(keyword in name_lower for keyword in keywords):
return category
# Check description for hints
desc_lower = description.lower()
if "framework" in desc_lower or "library" in desc_lower:
if any(word in desc_lower for word in ["web", "frontend", "backend", "api"]):
return "web-frameworks"
if "game" in desc_lower or "engine" in desc_lower:
return "game-engines"
if "devops" in desc_lower or "deployment" in desc_lower or "infrastructure" in desc_lower:
return "devops"
# Default to uncategorized
return "uncategorized"
def _extract_tags(self, name: str, description: str, config_data: Dict[str, Any]) -> List[str]:
"""
Extract relevant tags from config
Args:
name: Config name
description: Config description
config_data: Full config data
Returns:
List of tags
"""
tags = set()
name_lower = name.lower()
desc_lower = description.lower()
# Check against tag keywords
for tag, keywords in self.TAG_KEYWORDS.items():
if any(keyword in name_lower or keyword in desc_lower for keyword in keywords):
tags.add(tag)
# Add config type as tag
config_type = self._determine_type(config_data)
if config_type == "unified":
tags.add("multi-source")
# Add source type tags
if "base_url" in config_data or (config_type == "unified" and any(s.get("type") == "documentation" for s in config_data.get("sources", []))):
tags.add("documentation")
if "repo" in config_data or (config_type == "unified" and any(s.get("type") == "github" for s in config_data.get("sources", []))):
tags.add("github")
if "pdf" in config_data or "pdf_url" in config_data or (config_type == "unified" and any(s.get("type") == "pdf" for s in config_data.get("sources", []))):
tags.add("pdf")
return sorted(list(tags))
def _get_max_pages(self, config_data: Dict[str, Any]) -> Optional[int]:
"""
Get max_pages value from config
Args:
config_data: Config JSON data
Returns:
max_pages value or None
"""
# Single-source configs
if "max_pages" in config_data:
return config_data["max_pages"]
# Unified configs - get from first documentation source
if "sources" in config_data:
for source in config_data["sources"]:
if source.get("type") == "documentation" and "max_pages" in source:
return source["max_pages"]
return None
def _get_last_updated(self, config_path: Path) -> str:
"""
Get last updated date from git history
Args:
config_path: Path to config file
Returns:
ISO format date string
"""
try:
# Try to get last commit date for this file
result = subprocess.run(
["git", "log", "-1", "--format=%cI", str(config_path)],
cwd=config_path.parent.parent,
capture_output=True,
text=True,
timeout=5
)
if result.returncode == 0 and result.stdout.strip():
return result.stdout.strip()
except Exception:
pass
# Fallback to file modification time
mtime = config_path.stat().st_mtime
return datetime.fromtimestamp(mtime).isoformat()

209
api/main.py Normal file
View File

@@ -0,0 +1,209 @@
#!/usr/bin/env python3
"""
Skill Seekers Config API
FastAPI backend for listing available skill configs
"""
from fastapi import FastAPI, HTTPException
from fastapi.middleware.cors import CORSMiddleware
from fastapi.responses import JSONResponse, FileResponse
from typing import List, Dict, Any, Optional
import os
from pathlib import Path
from config_analyzer import ConfigAnalyzer
app = FastAPI(
title="Skill Seekers Config API",
description="API for discovering and downloading Skill Seekers configuration files",
version="1.0.0",
docs_url="/docs",
redoc_url="/redoc"
)
# CORS middleware - allow all origins for public API
app.add_middleware(
CORSMiddleware,
allow_origins=["*"],
allow_credentials=True,
allow_methods=["*"],
allow_headers=["*"],
)
# Initialize config analyzer
CONFIG_DIR = Path(__file__).parent.parent / "configs"
analyzer = ConfigAnalyzer(CONFIG_DIR)
@app.get("/")
async def root():
"""Root endpoint - API information"""
return {
"name": "Skill Seekers Config API",
"version": "1.0.0",
"endpoints": {
"/api/configs": "List all available configs",
"/api/configs/{name}": "Get specific config details",
"/api/categories": "List all categories",
"/docs": "API documentation",
},
"repository": "https://github.com/yusufkaraaslan/Skill_Seekers",
"website": "https://skillseekersweb.com"
}
@app.get("/api/configs")
async def list_configs(
category: Optional[str] = None,
tag: Optional[str] = None,
type: Optional[str] = None
) -> Dict[str, Any]:
"""
List all available configs with metadata
Query Parameters:
- category: Filter by category (e.g., "web-frameworks")
- tag: Filter by tag (e.g., "javascript")
- type: Filter by type ("single-source" or "unified")
Returns:
- version: API version
- total: Total number of configs
- filters: Applied filters
- configs: List of config metadata
"""
try:
# Get all configs
all_configs = analyzer.analyze_all_configs()
# Apply filters
configs = all_configs
filters_applied = {}
if category:
configs = [c for c in configs if c.get("category") == category]
filters_applied["category"] = category
if tag:
configs = [c for c in configs if tag in c.get("tags", [])]
filters_applied["tag"] = tag
if type:
configs = [c for c in configs if c.get("type") == type]
filters_applied["type"] = type
return {
"version": "1.0.0",
"total": len(configs),
"filters": filters_applied if filters_applied else None,
"configs": configs
}
except Exception as e:
raise HTTPException(status_code=500, detail=f"Error analyzing configs: {str(e)}")
@app.get("/api/configs/{name}")
async def get_config(name: str) -> Dict[str, Any]:
"""
Get detailed information about a specific config
Path Parameters:
- name: Config name (e.g., "react", "django")
Returns:
- Full config metadata including all fields
"""
try:
config = analyzer.get_config_by_name(name)
if not config:
raise HTTPException(
status_code=404,
detail=f"Config '{name}' not found"
)
return config
except HTTPException:
raise
except Exception as e:
raise HTTPException(status_code=500, detail=f"Error loading config: {str(e)}")
@app.get("/api/categories")
async def list_categories() -> Dict[str, Any]:
"""
List all available categories with config counts
Returns:
- categories: Dict of category names to config counts
- total_categories: Total number of categories
"""
try:
configs = analyzer.analyze_all_configs()
# Count configs per category
category_counts = {}
for config in configs:
cat = config.get("category", "uncategorized")
category_counts[cat] = category_counts.get(cat, 0) + 1
return {
"total_categories": len(category_counts),
"categories": category_counts
}
except Exception as e:
raise HTTPException(status_code=500, detail=f"Error analyzing categories: {str(e)}")
@app.get("/api/download/{config_name}")
async def download_config(config_name: str):
"""
Download a specific config file
Path Parameters:
- config_name: Config filename (e.g., "react.json", "django.json")
Returns:
- JSON file for download
"""
try:
# Validate filename (prevent directory traversal)
if ".." in config_name or "/" in config_name or "\\" in config_name:
raise HTTPException(status_code=400, detail="Invalid config name")
# Ensure .json extension
if not config_name.endswith(".json"):
config_name = f"{config_name}.json"
config_path = CONFIG_DIR / config_name
if not config_path.exists():
raise HTTPException(
status_code=404,
detail=f"Config file '{config_name}' not found"
)
return FileResponse(
path=config_path,
media_type="application/json",
filename=config_name
)
except HTTPException:
raise
except Exception as e:
raise HTTPException(status_code=500, detail=f"Error downloading config: {str(e)}")
@app.get("/health")
async def health_check():
"""Health check endpoint for monitoring"""
return {"status": "healthy", "service": "skill-seekers-api"}
if __name__ == "__main__":
import uvicorn
uvicorn.run(app, host="0.0.0.0", port=8000)

3
api/requirements.txt Normal file
View File

@@ -0,0 +1,3 @@
fastapi==0.115.0
uvicorn[standard]==0.32.0
python-multipart==0.0.12

15
render.yaml Normal file
View File

@@ -0,0 +1,15 @@
services:
# Config API Service
- type: web
name: skill-seekers-api
runtime: python
plan: free
buildCommand: pip install -r api/requirements.txt
startCommand: cd api && uvicorn main:app --host 0.0.0.0 --port $PORT
envVars:
- key: PYTHON_VERSION
value: 3.10
- key: PORT
generateValue: true
healthCheckPath: /health
autoDeploy: true

40
test_api.py Normal file
View File

@@ -0,0 +1,40 @@
#!/usr/bin/env python3
"""Quick test of the config analyzer"""
import sys
sys.path.insert(0, 'api')
from pathlib import Path
from api.config_analyzer import ConfigAnalyzer
# Initialize analyzer
config_dir = Path('configs')
analyzer = ConfigAnalyzer(config_dir, base_url="https://skillseekersweb.com")
# Test analyzing all configs
print("Testing config analyzer...")
print("-" * 60)
configs = analyzer.analyze_all_configs()
print(f"\n✅ Found {len(configs)} configs")
# Show first 3 configs
print("\n📋 Sample Configs:")
for config in configs[:3]:
print(f"\n Name: {config['name']}")
print(f" Type: {config['type']}")
print(f" Category: {config['category']}")
print(f" Tags: {', '.join(config['tags'])}")
print(f" Source: {config['primary_source'][:50]}...")
print(f" File Size: {config['file_size']} bytes")
# Test category counts
print("\n\n📊 Categories:")
categories = {}
for config in configs:
cat = config['category']
categories[cat] = categories.get(cat, 0) + 1
for cat, count in sorted(categories.items()):
print(f" {cat}: {count} configs")
print("\n✅ All tests passed!")