Agent skill
crawl4ai-pipeline-builder
当需要使用 Crawl4AI 进行多 URL 抓取、Markdown 生成、结构化抽取、CLI 批量 crawl、会话复用、内容过滤或无 LLM 的 schema 提取流水线时使用。适用于“批量抓多个页面”“把文档站转成 markdown”“用 CSS schema 抽结构化数据”“做可复用的 crawl pipeline”等场景。
Install this agent skill to your Project
npx add-skill https://github.com/majiayu000/claude-skill-registry/tree/main/skills/other/other/crawl4ai-prorise-cool-prorise-claude-skill
SKILL.md
Resource Map
基准路径:
.claude/skills/scraping-specialist/references/domains/crawl4ai/
crawl4ai/
├── references/
│ ├── cli-and-sdk-quick-ref.md
│ └── extraction-strategies.md
├── scripts/
│ ├── basic_crawler.py
│ ├── batch_crawler.py
│ ├── extraction_pipeline.py
│ └── google_search.py
├── requirements.txt
└── SKILL.md
Crawl4AI 抓取流水线
此技能用于明确要走 crawl4ai 的场景。它适合批量抓取、Markdown 化、结构化提取和可编排的 Python / CLI 抓取流水线。
先装环境
pip install crawl4ai
crawl4ai-setup
接口选择
- 快速任务：优先
crwlCLI - 可编排自动化：使用 Python SDK
- 结构化提取：优先 CSS / XPath schema，只有规则不稳定时才考虑 LLM
推荐流程
- 快速试跑时，先看 CLI 与 SDK 速览。
- 设计抽取方案时，优先看 抽取与批量策略。
- 直接复用
scripts/下的样例脚本，而不是从零手写。
自带脚本
scripts/basic_crawler.pyscripts/batch_crawler.pyscripts/extraction_pipeline.pyscripts/google_search.py
使用规则
- 先用 CLI 或最小脚本验证，再做批量化。
- 优先 CSS schema 提取，避免不必要的 LLM 成本。
- 需要会话、滚动或等待条件时，再补 browser/crawler 配置。
Recommended Agent Skills
Expand your agent's capabilities with these related and highly-rated skills.
majiayu000/claude-skill-registry
agent-ops-spec
Manage specification documents in .agent/specs/. Use when user provides requirements, acceptance criteria, or feature descriptions that need to be tracked and validated against implementation.
majiayu000/claude-skill-registry
agent-ops-state
Maintain .agent state files. Use at session start, after meaningful steps, and before concluding: read/update constitution/memory/focus/issues/baseline consistently.
majiayu000/claude-skill-registry
agent-ops-spec
Manage specification documents in .agent/specs/. Use when user provides requirements, acceptance criteria, or feature descriptions that need to be tracked and validated against implementation.
majiayu000/claude-skill-registry
agent-ops-testing
Test strategy, execution, and coverage analysis. Use when designing tests, running test suites, or analyzing test results beyond baseline checks.
majiayu000/claude-skill-registry
agent-ops-testing
Test strategy, execution, and coverage analysis. Use when designing tests, running test suites, or analyzing test results beyond baseline checks.
majiayu000/claude-skill-registry
agent-ops-state
Maintain .agent state files. Use at session start, after meaningful steps, and before concluding: read/update constitution/memory/focus/issues/baseline consistently.
Didn't find tool you were looking for?