Agent skill

tavily-map

Discover and list all URLs on a website without extracting content, via the Tavily CLI. Use this skill when the user wants to find a specific page on a large site, list all URLs, see the site structure, find where something is on a domain, or says "map the site", "find the URL for", "what pages are on", "list all pages", or "site structure". Faster than crawling — returns URLs only. Essential when you know the site but not the exact page. Combine with extract for targeted content retrieval.

Stars 174
Forks 17

Install this agent skill to your Project

npx add-skill https://github.com/tavily-ai/skills/tree/main/skills/tavily-map

SKILL.md

tavily map

Discover URLs on a website without extracting content. Faster than crawling.

Before running any command

If tvly is not found on PATH, install it first:

bash
curl -fsSL https://cli.tavily.com/install.sh | bash && tvly login

Do not skip this step or fall back to other tools.

See tavily-cli for alternative install methods and auth options.

When to use

  • You need to find a specific subpage on a large site
  • You want a list of all URLs before deciding what to extract or crawl
  • Step 3 in the workflow: search → extract → map → crawl → research

Quick start

bash
# Discover all URLs
tvly map "https://docs.example.com" --json

# With natural language filtering
tvly map "https://docs.example.com" --instructions "Find API docs and guides" --json

# Filter by path
tvly map "https://example.com" --select-paths "/blog/.*" --limit 500 --json

# Deep map
tvly map "https://example.com" --max-depth 3 --limit 200 --json

Options

Option Description
--max-depth Levels deep (1-5, default: 1)
--max-breadth Links per page (default: 20)
--limit Max URLs to discover (default: 50)
--instructions Natural language guidance for URL filtering
--select-paths Comma-separated regex patterns to include
--exclude-paths Comma-separated regex patterns to exclude
--select-domains Comma-separated regex for domains to include
--exclude-domains Comma-separated regex for domains to exclude
--allow-external / --no-external Include external links
--timeout Max wait (10-150 seconds)
-o, --output Save output to file
--json Structured JSON output

Map + Extract pattern

Use map to find the right page, then extract it. This is often more efficient than crawling an entire site:

bash
# Step 1: Find the authentication docs
tvly map "https://docs.example.com" --instructions "authentication" --json

# Step 2: Extract the specific page you found
tvly extract "https://docs.example.com/api/authentication" --json

Tips

  • Map is URL discovery only — no content extraction. Use extract or crawl for content.
  • Map + extract beats crawl when you only need a few specific pages from a large site.
  • Use --instructions for semantic filtering when path patterns aren't enough.

See also

  • tavily-extract — extract content from URLs you discover
  • tavily-crawl — bulk extract when you need many pages

Expand your agent's capabilities with these related and highly-rated skills.

tavily-ai/skills

tavily-search

Search the web with LLM-optimized results via the Tavily CLI. Use this skill when the user wants to search the web, find articles, look up information, get recent news, discover sources, or says "search for", "find me", "look up", "what's the latest on", "find articles about", or needs current information from the internet. Returns relevant results with content snippets, relevance scores, and metadata — optimized for LLM consumption. Supports domain filtering, time ranges, and multiple search depths.

174 17
Explore
tavily-ai/skills

tavily-best-practices

Build production-ready Tavily integrations with best practices baked in. Reference documentation for developers using coding assistants (Claude Code, Cursor, etc.) to implement web search, content extraction, crawling, and research in agentic workflows, RAG systems, or autonomous agents.

174 17
Explore
tavily-ai/skills

tavily-crawl

Crawl websites and extract content from multiple pages via the Tavily CLI. Use this skill when the user wants to crawl a site, download documentation, extract an entire docs section, bulk-extract pages, save a site as local markdown files, or says "crawl", "get all the pages", "download the docs", "extract everything under /docs", "bulk extract", or needs content from many pages on the same domain. Supports depth/breadth control, path filtering, semantic instructions, and saving each page as a local markdown file.

174 17
Explore
tavily-ai/skills

tavily-research

Conduct comprehensive AI-powered research with citations via the Tavily CLI. Use this skill when the user wants deep research, a detailed report, a comparison, market analysis, literature review, or says "research", "investigate", "analyze in depth", "compare X vs Y", "what does the market look like for", or needs multi-source synthesis with explicit citations. Returns a structured report grounded in web sources. Takes 30-120 seconds. For quick fact-finding, use tavily-search instead.

174 17
Explore
tavily-ai/skills

tavily-cli

Web search, content extraction, crawling, and deep research via the Tavily CLI. Use this skill whenever the user wants to search the web, find articles, research a topic, look something up online, extract content from a URL, grab text from a webpage, crawl documentation, download a site's pages, discover URLs on a domain, or conduct in-depth research with citations. Also use when they say "fetch this page", "pull the content from", "get the page at https://", "find me articles about", or reference extracting data from external websites. This provides LLM-optimized web search, content extraction, site crawling, URL discovery, and AI-powered deep research — capabilities beyond what agents can do natively. Do NOT trigger for local file operations, git commands, deployments, or code editing tasks.

174 17
Explore
tavily-ai/skills

tavily-extract

Extract clean markdown or text content from specific URLs via the Tavily CLI. Use this skill when the user has one or more URLs and wants their content, says "extract", "grab the content from", "pull the text from", "get the page at", "read this webpage", or needs clean text from web pages. Handles JavaScript-rendered pages, returns LLM-optimized markdown, and supports query-focused chunking for targeted extraction. Can process up to 20 URLs in a single call.

174 17
Explore

Didn't find tool you were looking for?

Be as detailed as possible for better results