New skill: douban-skill - Full export of Douban (豆瓣) book/movie/music/game collections via Frodo API - RSS incremental sync for daily updates - Python stdlib only, zero dependencies, cross-platform (macOS/Windows/Linux) - Documented 7 failed approaches (PoW anti-scraping) and why Frodo API is the only working solution - Pre-flight user validation, KeyboardInterrupt handling, pagination bug fix skill-creator enhancements: - Add development methodology reference (8-phase process with prior art research, counter review, and real failure case studies) - Sync upstream changes: improve_description.py now uses `claude -p` instead of Anthropic SDK (no ANTHROPIC_API_KEY needed), remove stale "extended thinking" ref - Add "Updating an existing skill" guidance to Claude.ai and Cowork sections - Restore test case heuristic guidance for objective vs subjective skills README updates: - Document fork advantages vs upstream with quality comparison table (65 vs 42) - Bilingual (EN + ZH-CN) with consistent content Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
132 lines
4.7 KiB
Markdown
132 lines
4.7 KiB
Markdown
---
|
|
name: douban-skill
|
|
description: >
|
|
Export and sync Douban (豆瓣) book/movie/music/game collections to local CSV files via Frodo API.
|
|
Supports full export (all history) and RSS incremental sync (recent items).
|
|
Use when the user wants to export Douban reading/watching/listening/gaming history,
|
|
back up their Douban data, set up incremental sync, or mentions 豆瓣/douban collections.
|
|
Triggers on: 豆瓣, douban, 读书记录, 观影记录, 书影音, 导出豆瓣, export, backup, sync, collection.
|
|
---
|
|
|
|
# Douban Collection Export
|
|
|
|
Export Douban user collections (books, movies, music, games) to CSV files.
|
|
Douban has no official data export; the official API shut down in 2018.
|
|
|
|
## What This Skill Can Do
|
|
|
|
- Full export of all book/movie/music/game collections via Frodo API
|
|
- RSS incremental sync for daily updates (last ~10 items)
|
|
- CSV output with UTF-8 BOM (Excel-compatible), cross-platform (macOS/Windows/Linux)
|
|
- No login, no cookies, no browser required
|
|
- Pre-flight user ID validation (fail fast on wrong ID)
|
|
|
|
## What This Skill Cannot Do
|
|
|
|
- Cannot export reviews (长评), notes (读书笔记), or broadcasts (广播)
|
|
- Cannot filter by single category in one run (exports all 4 types together)
|
|
- Cannot access private profiles (returns 0 items silently)
|
|
|
|
## Why Frodo API (Do NOT Use Web Scraping)
|
|
|
|
Douban uses PoW (Proof of Work) challenges on web pages, blocking all HTTP scraping.
|
|
We tested 7 approaches — only the Frodo API works. **Do NOT attempt** web scraping,
|
|
`browser_cookie3`+`requests`, `curl` with cookies, or Jina Reader.
|
|
|
|
See [references/troubleshooting.md](references/troubleshooting.md) for the complete
|
|
failure log of all 7 tested approaches and why each failed.
|
|
|
|
## Security & Privacy
|
|
|
|
The API key and HMAC secret in the script are Douban's **public mobile app credentials**,
|
|
extracted from the APK. They are shared by all Douban app users and do not identify you.
|
|
No personal credentials are used or stored. Data is fetched only from `frodo.douban.com`.
|
|
|
|
## Full Export (Primary Method)
|
|
|
|
```bash
|
|
DOUBAN_USER=<user_id> python3 scripts/douban-frodo-export.py
|
|
```
|
|
|
|
**Finding the user ID:** Profile URL `douban.com/people/<ID>/` — the ID is after `/people/`.
|
|
If the user provides a full URL, the script auto-extracts the ID.
|
|
|
|
**Environment variables:**
|
|
- `DOUBAN_USER` (required): Douban user ID (alphanumeric or numeric, or full profile URL)
|
|
- `DOUBAN_OUTPUT_DIR` (optional): Override output directory
|
|
|
|
**Default output** (auto-detected per platform):
|
|
- macOS: `~/Downloads/douban-sync/<user_id>/`
|
|
- Windows: `%USERPROFILE%\Downloads\douban-sync\<user_id>\`
|
|
- Linux: `~/Downloads/douban-sync/<user_id>/`
|
|
|
|
**Dependencies:** Python 3.6+ standard library only (works with `python3` or `uv run`).
|
|
|
|
**Example console output:**
|
|
```
|
|
Douban Export for user: your_douban_id
|
|
Output directory: /Users/you/Downloads/douban-sync/your_douban_id
|
|
|
|
=== 读过 (book) ===
|
|
Total: 639
|
|
Fetched 0-50 (50/639)
|
|
Fetched 50-100 (100/639)
|
|
...
|
|
Fetched 597-639 (639/639)
|
|
Collected: 639
|
|
|
|
=== 在读 (book) ===
|
|
Total: 75
|
|
...
|
|
|
|
--- Writing CSV files ---
|
|
书.csv: 996 rows
|
|
影视.csv: 238 rows
|
|
音乐.csv: 0 rows
|
|
游戏.csv: 0 rows
|
|
|
|
Done! 1234 total items exported to /Users/you/Downloads/douban-sync/your_douban_id
|
|
```
|
|
|
|
## RSS Incremental Sync (Complementary)
|
|
|
|
```bash
|
|
DOUBAN_USER=<user_id> node scripts/douban-rss-sync.mjs
|
|
```
|
|
|
|
RSS returns only the latest ~10 items (no pagination). Use Full Export first, then RSS for daily updates.
|
|
|
|
## Output Format
|
|
|
|
Four CSV files per user:
|
|
|
|
```
|
|
Downloads/douban-sync/<user_id>/
|
|
├── 书.csv (读过 + 在读 + 想读)
|
|
├── 影视.csv (看过 + 在看 + 想看)
|
|
├── 音乐.csv (听过 + 在听 + 想听)
|
|
└── 游戏.csv (玩过 + 在玩 + 想玩)
|
|
```
|
|
|
|
Columns: `title, url, date, rating, status, comment`
|
|
- `rating`: ★ to ★★★★★ (empty if unrated)
|
|
- `date`: YYYY-MM-DD (when the user marked it)
|
|
- Safe to run multiple times (overwrites with fresh data)
|
|
- Row counts may be slightly below Douban's displayed count due to delisted items
|
|
|
|
## Workflow
|
|
|
|
1. Ask for Douban user ID (from profile URL, or accept full URL)
|
|
2. Run: `DOUBAN_USER=<id> python3 scripts/douban-frodo-export.py`
|
|
3. Verify: row counts in console output should match, check with `wc -l <output_dir>/*.csv`
|
|
4. (Optional) Set up RSS sync for daily incremental updates
|
|
|
|
## Troubleshooting
|
|
|
|
See [references/troubleshooting.md](references/troubleshooting.md) for:
|
|
- Frodo API auth details (HMAC-SHA1 signature computation)
|
|
- Common errors (code 996 signature error, rate limits, pagination quirks)
|
|
- Complete failure log of all 7 tested approaches with root causes
|
|
- Alternative approaches (豆伴 extension, Tampermonkey script, browser console)
|
|
- API endpoint reference with response format
|