claude-code-skills-reference/douban-skill/references/troubleshooting.md

Troubleshooting & Technical Reference

How Frodo API Auth Works

The Frodo API is Douban's mobile app backend at frodo.douban.com. It uses HMAC-SHA1 signature authentication instead of the PoW challenges used on web pages.

Signature computation:

1. Build raw string: GET + & + URL-encoded(path only) + & + timestamp
2. HMAC-SHA1 with secret key bf7dddc7c9cfe6f7
3. Base64-encode the result → this is _sig

Critical: Sign only the URL path (e.g., /api/v2/user/xxx/interests), never the full URL with query parameters. This was our first signature error — code 996.

Required query parameters:

• apiKey: 0dad551ec0f84ed02907ff5c42e8ec70 (Douban mobile app's public API key)
• _ts: Unix timestamp in seconds (string)
• _sig: The computed HMAC-SHA1 signature
• os_rom: android

Required headers:

• User-Agent: Must look like a Douban Android app client string

Python implementation:

import hmac, hashlib, base64, urllib.parse

def compute_signature(url_path, timestamp):
    raw = '&'.join(['GET', urllib.parse.quote(url_path, safe=''), timestamp])
    sig = hmac.new(b'bf7dddc7c9cfe6f7', raw.encode(), hashlib.sha1)
    return base64.b64encode(sig.digest()).decode()

Common Errors

Signature Error (code 996)

{"msg": "invalid_request_996", "code": 996}

Cause: The _sig parameter doesn't match the expected value.

Debug checklist:

1. Are you signing only the path, not the full URL with query params?
2. Does _ts in the signature match _ts in the query params exactly?
3. Is _ts a string of Unix seconds (not milliseconds)?
4. Are you using urllib.parse.quote(path, safe='') (encoding / as %2F)?

Pagination Returns Fewer Items Than Expected

Some pages return fewer than the requested count (e.g., 48 instead of 50). This happens when items have been delisted from Douban's catalog but still count toward the total.

This was our biggest silent bug. The first version of the export script used len(page_items) < count_per_page as the stop condition. Result: only 499 out of 639 books were exported, with no error message. The fix:

# WRONG: stops early when a page has fewer items due to delisted content
if len(interests) < count_per_page:
    break

# CORRECT: check against the total count reported by the API
if len(all_items) >= total:
    break
start += len(interests)  # advance by actual count, not page_size

Rating Scale Confusion

The Frodo API returns two different ratings per item:

Field	Scale	Meaning
interest.rating	{value: 1-5, max: 5}	User's personal rating
subject.rating	{value: 0-10, max: 10}	Douban community average

Our first version divided all values by 2, which halved the user's rating (2 stars → 1 star). The fix: check max field to determine scale.

# Correct conversion
if max_val <= 5:
    stars = int(val)      # value is already 1-5
else:
    stars = int(val / 2)  # value is 2-10, convert to 1-5

HTTP 403 / Rate Limiting

The Frodo API is generally tolerant, but excessive requests may trigger rate limiting.

Tested intervals:

• 1.5s between pages + 2s between categories: 1234 items exported without issues
• 0s (no delay): Not tested, not recommended

If you hit 403, increase delays to 3s/5s and retry after a few minutes.

Detailed Failure Log: All 7 Tested Approaches

Approach 1: requests + browser_cookie3 (Python)

What we tried: Extract Chrome cookies via browser_cookie3, use requests with those cookies.

What happened:

1. First request succeeded — we saw "639 books" in the page title
2. Subsequent requests returned "禁止访问" (Forbidden) page
3. The HTML contained no items despite HTTP 200 status

Root cause: Douban's PoW challenge. The first request sometimes passes (cached/grace period), but subsequent requests trigger the PoW redirect to sec.douban.com. Python requests cannot execute the SHA-512 proof-of-work JavaScript.

Approach 2: curl with browser cookies

What we tried: Export cookies from Chrome, use curl with full browser headers (User-Agent, Accept, Referer, Accept-Language).

What happened: HTTP 302 redirect to https://www.douban.com/misc/sorry?original-url=...

Root cause: Same PoW issue. Even with NO_PROXY set to bypass local proxy, the IP was already rate-limited from approach 1's requests.

Approach 3: Jina Reader (r.jina.ai)

What we tried: curl -s "https://r.jina.ai/https://book.douban.com/people/<user_id>/collect"

What happened: HTTP 200 but content was "403 Forbidden" — Jina's server got blocked.

Root cause: Jina's scraping infrastructure also cannot solve Douban's PoW challenges.

Approach 4: Chrome DevTools MCP (Playwright browser)

What we tried: Navigate to Douban pages in the Playwright browser via Chrome DevTools MCP. Injected cookies via document.cookie in evaluate_script.

What happened:

1. mcp__chrome-devtools__navigate_page → page title was "403 Forbidden"
2. After cookie injection → still redirected to /misc/sorry

Root cause: The Chrome DevTools MCP connects to a Playwright browser instance, not the user's actual Chrome. Even after injecting cookies, the IP was already banned from earlier requests. Also, HttpOnly cookies (like dbcl2) can't be set via document.cookie.

Approach 5: opencli douban marks

What we tried: opencli douban marks --uid <user_id> --status all --limit 0 -f csv

What happened: Partial success — exported 24 movie records successfully.

Limitation: opencli douban only implements marks (movies). No book/music/game support. The opencli generate and opencli cascade commands failed to discover APIs for book.douban.com because Douban books use server-rendered HTML with no discoverable API.

Approach 6: Agent Reach

What we tried: Installed agent-reach (17-platform CLI tool). Checked for Douban support.

What happened: Agent Reach has no Douban channel. Its web reader (Jina) also gets 403.

Approach 7: Node.js HTTP scraper (from douban-sync-skill)

What we tried: The douban-scraper.mjs from the cosformula/douban-sync-skill.

Status: User rejected the command before it ran — based on prior failures, it would hit the same PoW blocking. The script uses fetch() with a fake User-Agent, which is exactly what approaches 1-3 proved doesn't work.

Alternative Approaches (Not Blocked)

These approaches work but have different tradeoffs compared to the Frodo API:

豆伴 (Tofu) Chrome Extension (605 stars)

• GitHub: doufen-org/tofu
• Uses Douban's Rexxar API (m.douban.com/rexxar/api/v2/user/{uid}/interests)
• Most comprehensive: backs up books, movies, music, games, reviews, notes, photos, etc.
• Current status (April 2026): Mainline v0.12.x is broken due to MV3 migration + anti-scraping. PR #121 (v0.13.0) fixes both issues but is not yet merged.
• Risk: Makes many API calls as logged-in user → may trigger account lockout

Tampermonkey Userscript (bambooom/douban-backup, 162 stars)

• Greasemonkey/Tampermonkey: https://greasyfork.org/en/scripts/420999
• Runs inside real browser → inherits PoW-solved session
• Adds "export" button on collection pages → auto-paginates → downloads CSV
• Suitable for one-time manual export

Browser Console Script (built into old skill)

• Paste fetch()-based extraction script into browser DevTools console
• Zero blocking risk (same-origin request from authenticated session)
• Most manual approach — user must paste script and copy clipboard

API Endpoint Reference

User Interests (Collections)

GET https://frodo.douban.com/api/v2/user/{user_id}/interests
  ?type={book|movie|music|game}
  &status={done|doing|mark}
  &start={offset}
  &count={page_size, max 50}
  &apiKey=0dad551ec0f84ed02907ff5c42e8ec70
  &_ts={unix_timestamp_seconds}
  &_sig={hmac_sha1_signature}
  &os_rom=android

Response:

{
  "count": 50,
  "start": 0,
  "total": 639,
  "interests": [
    {
      "comment": "短评文本",
      "rating": {"value": 4, "max": 5, "star_count": 4.0},
      "create_time": "2026-03-21 18:23:10",
      "status": "done",
      "id": 4799352304,
      "subject": {
        "id": "36116375",
        "title": "书名",
        "url": "https://book.douban.com/subject/36116375/",
        "rating": {"value": 7.8, "max": 10, "count": 14}
      }
    }
  ]
}

Important distinctions:

• interest.rating = user's personal rating (max 5)
• subject.rating = Douban community average (max 10)
• interest.create_time = when the user marked it (not the item's publish date)
• status: done = 读过/看过/听过/玩过, doing = 在读/在看/在听/在玩, mark = 想读/想看/想听/想玩

Other Known Frodo Endpoints (Not Used by This Skill)

Endpoint	Returns
/api/v2/book/{id}	Book detail
/api/v2/movie/{id}	Movie detail
/api/v2/group/{id}/topics	Group discussion topics
/api/v2/group/topic/{id}	Single topic with comments
/api/v2/subject_collection/{type}/items	Douban curated lists

Mouban Proxy Service (Third-Party)

mouban.mythsman.com is a Go service that pre-crawls Douban data. If a user has been indexed, it returns data instantly without hitting Douban directly. Endpoints:

Endpoint	Returns
GET /guest/check_user?id={douban_id}	User profile + counts
GET /guest/user_book?id={id}&action={wish|do|collect}	Book entries
GET /guest/user_movie?id={id}&action=...	Movie entries

Caveat: Data freshness depends on when the service last crawled the user. First request for a new user triggers a background crawl (takes minutes to hours). Third-party dependency.