From 91692db87c32be951e90b697eb93412b85b5a5ff Mon Sep 17 00:00:00 2001
From: sogoiii <sogoiii@gmail.com>
Date: Thu, 20 Nov 2025 14:00:55 -0800
Subject: [PATCH] =?UTF-8?q?=F0=9F=93=9D=20docs:=20add=20skip=5Fllms=5Ftxt?=
 =?UTF-8?q?=20to=20config=20parameters=20documentation?=
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

---
 CLAUDE.md | 1 +
 1 file changed, 1 insertion(+)

diff --git a/CLAUDE.md b/CLAUDE.md
index a15cfcc..2177795 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -434,6 +434,7 @@ Config files (`configs/*.json`) define scraping behavior:
 - `categories`: Keyword mapping for categorization
 - `rate_limit`: Delay between requests (seconds)
 - `max_pages`: Maximum pages to scrape
+- `skip_llms_txt`: Skip llms.txt detection, force HTML scraping (default: false)
 
 ## Key Features & Implementation