面向 AI 应用的大规模网页抓取与上下文 API

大规模搜索、抓取网页并与之交互的 API。 🔥 面向 AI 智能体的网页上下文 API——查找来源、提取内容，并将其转换为干净的 Markdown 或结构化数据，供你的 AI 智能体直接使用。开源项目，同时提供托管服务。

嘿。如果你读到这里，欢迎给我们点个星标 :)

行业领先的可靠性：覆盖 96% 的网页，包括重度 JavaScript 渲染页面——无需操心代理服务器，只有干净的数据（详见基准测试）

极速响应：数百万页面 P95 延迟仅 3.4 秒，为实时 AI 智能体和动态应用而生

LLM 就绪的输出：干净的 Markdown、结构化 JSON、截图等——消耗更少 token，构建更优质的 AI 应用

我们搞定脏活累活：轮换代理、编排、速率限制、JS 拦截内容等——零配置

AI 智能体就绪：用一条命令将 Firecrawl 连接到任意 AI 智能体或 MCP 客户端

媒体解析：从网页托管的 PDF、DOCX 等格式中解析提取内容

Actions：提取内容前支持点击、滚动、输入、等待和按键操作

开源：透明协作开发——欢迎加入我们的社区

访问 firecrawl.dev 注册以获取 API key。也可以尝试 playground 进行测试。

搜索网页并获取结果的完整内容。

from firecrawl import Firecrawl

app = Firecrawl(api_key="fc-YOUR_API_KEY")

search_result = app.search("firecrawl", limit=5)

import { Firecrawl } from 'firecrawl';

const app = new Firecrawl({apiKey: "fc-YOUR_API_KEY"});

app.search("firecrawl", { limit: 5 })

curl -X POST 'https://api.firecrawl.dev/v2/search' \
-H 'Authorization: Bearer fc-YOUR_API_KEY' \
-H 'Content-Type: application/json' \
-d '{
  "query": "firecrawl",
  "limit": 5
}'

firecrawl search "firecrawl" --limit 5

[
  {
    "url": "https://firecrawl.dev",
    "title": "Firecrawl",
    "markdown": "Turn websites into..."
  },
  {
    "url": "https://docs.firecrawl.dev",
    "title": "Firecrawl Docs",
    "markdown": "# Getting Started..."
  }
]

从任意网站获取 LLM 就绪的数据——Markdown、JSON、截图等。

from firecrawl import Firecrawl

app = Firecrawl(api_key="fc-YOUR_API_KEY")

result = app.scrape('firecrawl.dev')

import { Firecrawl } from 'firecrawl';

const app = new Firecrawl({ apiKey: "fc-YOUR_API_KEY" });

app.scrape('firecrawl.dev')

curl -X POST 'https://api.firecrawl.dev/v2/scrape' \
-H 'Authorization: Bearer fc-YOUR_API_KEY' \
-H 'Content-Type: application/json' \
-d '{
  "url": "firecrawl.dev"
}'

firecrawl scrape https://firecrawl.dev
firecrawl https://firecrawl.dev --only-main-content

# Firecrawl

Firecrawl helps AI agents search, scrape, and interact with the web.

## Features
- Search: Find information across the web
- Scrape: Clean data from any page
- Interact: Click, navigate, and operate pages
- Agent: Autonomous data gathering

抓取页面后，使用 AI 提示词或代码与之交互。

from firecrawl import Firecrawl

app = Firecrawl(api_key="fc-YOUR_API_KEY")

result = app.scrape("https://amazon.com")
scrape_id = result.metadata.scrape_id

app.interact(scrape_id, prompt="Search for 'mechanical keyboard'")
app.interact(scrape_id, prompt="Click the first result")

import { Firecrawl } from 'firecrawl';

const app = new Firecrawl({apiKey: "fc-YOUR_API_KEY"});

const result = await app.scrape("https://amazon.com");

await app.interact(result.metadata.scrapeId, {
  prompt: "Search for 'mechanical keyboard'"
});
await app.interact(result.metadata.scrapeId, {
  prompt: "Click the first result"
});

# 1. Scrape the page
curl -X POST 'https://api.firecrawl.dev/v2/scrape' \
-H 'Authorization: Bearer fc-YOUR_API_KEY' \
-H 'Content-Type: application/json' \
-d '{"url": "https://amazon.com"}'

# 2. Interact with the page (use scrapeId from step 1)
curl -X POST 'https://api.firecrawl.dev/v2/scrape/SCRAPE_ID/interact' \
-H 'Authorization: Bearer fc-YOUR_API_KEY' \
-H 'Content-Type: application/json' \
-d '{"prompt": "Search for mechanical keyboard"}'

firecrawl scrape https://amazon.com
firecrawl interact exec --prompt "Search for 'mechanical keyboard'"
firecrawl interact exec --prompt "Click the first result"

{
  "success": true,
  "output": "Keyboard available at $100",
  "liveViewUrl": "https://liveview.firecrawl.dev/..."
}

几分钟内将 Firecrawl 连接到任意 AI 智能体或 MCP 客户端。

用一条命令让你的 AI 智能体轻松访问实时网页数据。

npx -y firecrawl-cli@latest init --all --browser

安装后重启你的 AI 智能体。支持 Claude Code、Antigravity、OpenCode 等。

将任意 MCP 兼容客户端在几秒内连接到互联网。

{
  "mcpServers": {
    "firecrawl-mcp": {
      "command": "npx",
      "args": ["-y", "firecrawl-mcp"],
      "env": {
        "FIRECRAWL_API_KEY": "fc-YOUR_API_KEY"
      }
    }
  }
}

你是 AI 智能体？获取此 Skill 为你的用户注册、获取 API key，开始使用 Firecrawl 构建。

curl -s https://firecrawl.dev/agent-onboarding/SKILL.md

查看 Skill + CLI 文档以了解所有可用命令。关于 MCP，请参阅 firecrawl-mcp-server。

从网页获取数据最简单的方式。描述你需要什么，我们的 AI 智能体会搜索、导航并检索回来。无需提供 URL。

Agent 是我们 /extract 端点的进化版：更快、更可靠，且无需你预先知道 URL。

curl -X POST 'https://api.firecrawl.dev/v2/agent' \
  -H 'Authorization: Bearer fc-YOUR_API_KEY' \
  -H 'Content-Type: application/json' \
  -d '{
    "prompt": "Find the pricing plans for Notion"
  }'

{
  "success": true,
  "data": {
    "result": "Notion offers the following pricing plans:\n\n1. Free - $0/month...\n2. Plus - $10/seat/month...\n3. Business - $18/seat/month...",
    "sources": ["https://www.notion.so/pricing"]
  }
}

使用 schema 获取结构化数据：

from firecrawl import Firecrawl
from pydantic import BaseModel, Field
from typing import List, Optional

app = Firecrawl(api_key="fc-YOUR_API_KEY")

class Founder(BaseModel):
    name: str = Field(description="Full name of the founder")
    role: Optional[str] = Field(None, description="Role or position")

class FoundersSchema(BaseModel):
    founders: List[Founder] = Field(description="List of founders")

result = app.agent(
    prompt="Find the founders of Firecrawl",
    schema=FoundersSchema
)

print(result.data)

{
  "founders": [
    {"name": "Eric Ciarla", "role": "Co-founder"},
    {"name": "Nicolas Camara", "role": "Co-founder"},
    {"name": "Caleb Peffer", "role": "Co-founder"}
  ]
}

将 Agent 聚焦在特定页面上：

result = app.agent(
    urls=["https://docs.firecrawl.dev", "https://firecrawl.dev/pricing"],
    prompt="Compare the features and pricing information"
)

根据需求在两个模型之间选择：

result = app.agent(
    prompt="Compare enterprise features across Firecrawl, Apify, and ScrapingBee",
    model="spark-1-pro"
)

跨多个网站对比数据

从具有复杂导航或需要登录认证的网站提取

需要 AI 智能体探索多条路径的研究任务

对准确性要求极高的关键数据

在我们的 Agent 文档中了解更多关于 Spark 模型的信息。

抓取整个网站并获取所有页面的内容。

curl -X POST 'https://api.firecrawl.dev/v2/crawl' \
  -H 'Authorization: Bearer fc-YOUR_API_KEY' \
  -H 'Content-Type: application/json' \
  -d '{
    "url": "https://docs.firecrawl.dev",
    "limit": 100,
    "scrapeOptions": {
      "formats": ["markdown"]
    }
  }'

{
  "success": true,
  "id": "123-456-789",
  "url": "https://api.firecrawl.dev/v2/crawl/123-456-789"
}

curl -X GET 'https://api.firecrawl.dev/v2/crawl/123-456-789' \
  -H 'Authorization: Bearer fc-YOUR_API_KEY'

{
  "status": "completed",
  "total": 50,
  "completed": 50,
  "creditsUsed": 50,
  "data": [
    {
      "markdown": "# Page Title\n\nContent...",
      "metadata": {"title": "Page Title", "sourceURL": "https://..."}
    }
  ]
}

Note: SDK 会自动处理轮询，以提供更好的开发者体验。

即时发现网站上的所有 URL。

curl -X POST 'https://api.firecrawl.dev/v2/map' \
  -H 'Authorization: Bearer fc-YOUR_API_KEY' \
  -H 'Content-Type: application/json' \
  -d '{"url": "https://firecrawl.dev"}'

{
  "success": true,
  "links": [
    {"url": "https://firecrawl.dev", "title": "Firecrawl", "description": "Turn websites into LLM-ready data"},
    {"url": "https://firecrawl.dev/pricing", "title": "Pricing", "description": "Firecrawl pricing plans"},
    {"url": "https://firecrawl.dev/blog", "title": "Blog", "description": "Firecrawl blog"}
  ]
}

在站点内查找特定 URL：

from firecrawl import Firecrawl

app = Firecrawl(api_key="fc-YOUR_API_KEY")

result = app.map("https://firecrawl.dev", search="pricing")
# 返回按与 "pricing" 相关性排序的 URL

批量抓取多个 URL：

from firecrawl import Firecrawl

app = Firecrawl(api_key="fc-YOUR_API_KEY")

job = app.batch_scrape([
    "https://firecrawl.dev",
    "https://docs.firecrawl.dev",
    "https://firecrawl.dev/pricing"
], formats=["markdown"])

for doc in job.data:
    print(doc.metadata.source_url)

我们的 SDK 提供了一种便捷的方式来使用 Firecrawl 的所有功能，并自动处理异步操作的轮询。

pip install firecrawl-py

from firecrawl import Firecrawl

app = Firecrawl(api_key="fc-YOUR_API_KEY")

# 抓取单个 URL
doc = app.scrape("https://firecrawl.dev", formats=["markdown"])
print(doc.markdown)

# 使用 AI 智能体进行自主数据采集
result = app.agent(prompt="Find the founders of Stripe")
print(result.data)

# 爬取网站（自动等待完成）
docs = app.crawl("https://docs.firecrawl.dev", limit=50)
for doc in docs.data:
    print(doc.metadata.source_url, doc.markdown[:100])

# 网页搜索
results = app.search("best AI data tools 2024", limit=10)
print(results)

npm install firecrawl

import { Firecrawl } from 'firecrawl';

const app = new Firecrawl({ apiKey: 'fc-YOUR_API_KEY' });

// 抓取单个 URL
const doc = await app.scrape('https://firecrawl.dev', { formats: ['markdown'] });
console.log(doc.markdown);

// 使用 AI 智能体进行自主数据采集
const result = await app.agent({ prompt: 'Find the founders of Stripe' });
console.log(result.data);

// 爬取网站（自动等待完成）
const docs = await app.crawl('https://docs.firecrawl.dev', { limit: 50 });
docs.data.forEach(doc => {
    console.log(doc.metadata.sourceURL, doc.markdown.substring(0, 100));
});

// 网页搜索
const results = await app.search('best AI data tools 2024', { limit: 10 });
results.data.web.forEach(result => {
    console.log(`${result.title}: ${result.url}`);
});

go get github.com/firecrawl/firecrawl/apps/go-sdk

package main

import (
	"context"
	"fmt"
	"log"

firecrawl "github.com/firecrawl/firecrawl/apps/go-sdk"
	"github.com/firecrawl/firecrawl/apps/go-sdk/option"
)

func main() {
	// 创建客户端（从环境变量读取 FIRECRAWL_API_KEY）
	client, err := firecrawl.NewClient(option.WithAPIKey("fc-YOUR_API_KEY"))
	if err != nil {
		log.Fatal(err)
	}

ctx := context.Background()

// 抓取单个 URL
	doc, err := client.Scrape(ctx, "https://firecrawl.dev", &firecrawl.ScrapeOptions{
		Formats: []string{"markdown"},
	})
	if err != nil {
		log.Fatal(err)
	}
	fmt.Println(doc.Markdown)

// 使用 AI 智能体进行自主数据采集
	agent, err := client.Agent(ctx, &firecrawl.AgentOptions{
		Prompt: "Find the founders of Stripe",
	})
	if err != nil {
		log.Fatal(err)
	}
	fmt.Println(agent.Data)

// 爬取网站（自动等待完成）
	job, err := client.Crawl(ctx, "https://docs.firecrawl.dev", &firecrawl.CrawlOptions{
		Limit: firecrawl.Int(50),
	})
	if err != nil {
		log.Fatal(err)
	}
	fmt.Printf("Crawled %d pages\n", len(job.Data))

// 网页搜索
	results, err := client.Search(ctx, "best AI data tools 2024", &firecrawl.SearchOptions{
		Limit: firecrawl.Int(10),
	})
	if err != nil {
		log.Fatal(err)
	}
	fmt.Println(results)
}

添加依赖（Gradle/Maven）：

repositories {
    mavenCentral()
    maven { url 'https://jitpack.io' }
}

dependencies {
    implementation 'com.github.firecrawl:firecrawl-java-sdk:2.0'
}

import dev.firecrawl.client.FirecrawlClient;
import dev.firecrawl.model.*;

FirecrawlClient client = new FirecrawlClient(
    System.getenv("FIRECRAWL_API_KEY"), null, null
);

// 抓取单个 URL
ScrapeParams scrapeParams = new ScrapeParams();
scrapeParams.setFormats(new String[]{"markdown"});
FirecrawlDocument doc = client.scrapeURL("https://firecrawl.dev", scrapeParams);
System.out.println(doc.getMarkdown());

// 使用 AI 智能体进行自主数据采集
AgentParams agentParams = new AgentParams("Find the founders of Stripe");
AgentResponse start = client.createAgent(agentParams);
AgentStatusResponse result = client.getAgentStatus(start.getId());
System.out.println(result.getData());

// 爬取网站（轮询直到完成）
CrawlParams crawlParams = new CrawlParams();
crawlParams.setLimit(50);
CrawlStatusResponse job = client.crawlURL("https://docs.firecrawl.dev", crawlParams, null, 10);
for (FirecrawlDocument page : job.getData()) {
    System.out.println(page.getMetadata().get("sourceURL"));
}

// 网页搜索
SearchParams searchParams = new SearchParams("best AI data tools 2024");
searchParams.setLimit(10);
SearchResponse results = client.search(searchParams);
for (SearchResult r : results.getResults()) {
    System.out.println(r.getTitle() + ": " + r.getUrl());
}

def deps do
  [
    {:firecrawl, "~> 1.0"}
  ]
end

# 抓取 URL
{:ok, response} = Firecrawl.scrape_and_extract_from_url(
  url: "https://firecrawl.dev",
  formats: ["markdown"]
)

# 爬取网站
{:ok, response} = Firecrawl.crawl_urls(
  url: "https://docs.firecrawl.dev",
  limit: 50
)

# 网页搜索
{:ok, response} = Firecrawl.search_and_scrape(
  query: "best AI data tools 2024",
  limit: 10
)

# 映射 URL
{:ok, response} = Firecrawl.map_urls(url: "https://example.com")

[dependencies]
firecrawl = "2"
tokio = { version = "1", features = ["macros", "rt-multi-thread"] }

use firecrawl::{Client, ScrapeOptions, Format, CrawlOptions};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let client = Client::new("fc-YOUR_API_KEY")?;

// 抓取 URL
    let document = client.scrape("https://firecrawl.dev", None).await?;
    println!("{:?}", document.markdown);

// 爬取网站
    let options = CrawlOptions {
        limit: Some(50),
        ..Default::default()
    };
    let result = client.crawl("https://docs.firecrawl.dev", options).await?;
    println!("Crawled {} pages", result.data.len());

// 网页搜索
    let response = client.search("best web scraping tools 2024", None).await?;
    println!("{:?}", response.data);

Ok(())
}

gem install firecrawl-sdk

require "firecrawl"

client = Firecrawl::Client.new(api_key: "fc-YOUR_API_KEY")

# 抓取单个 URL
doc = client.scrape("https://firecrawl.dev",
  Firecrawl::Models::ScrapeOptions.new(formats: ["markdown"]))
puts doc.markdown

# 使用 AI 智能体进行自主数据采集
result = client.agent(
  Firecrawl::Models::AgentOptions.new(prompt: "Find the founders of Stripe"))
puts result.data

# 爬取网站（自动等待完成）
job = client.crawl("https://docs.firecrawl.dev",
  Firecrawl::Models::CrawlOptions.new(limit: 50))
job.data.each { |d| puts d.metadata.source_url }

# 网页搜索
results = client.search("best AI data tools 2024",
  Firecrawl::Models::SearchOptions.new(limit: 10))
puts results

dotnet add package firecrawl-sdk

using Firecrawl;
using Firecrawl.Models;

var client = new FirecrawlClient("fc-YOUR_API_KEY");

// 抓取单个 URL
var doc = await client.ScrapeAsync("https://firecrawl.dev",
    new ScrapeOptions { Formats = new List<object> { "markdown" } });
Console.WriteLine(doc.Markdown);

// 爬取网站（自动等待完成）
var job = await client.CrawlAsync("https://docs.firecrawl.dev",
    new CrawlOptions { Limit = 50 });
Console.WriteLine($"Crawled {job.Data.Count} pages");

// 网页搜索
var results = await client.SearchAsync("best AI data tools 2024",
    new SearchOptions { Limit = 10 });
Console.WriteLine(results);

composer require firecrawl/firecrawl-sdk

<?php

use Firecrawl\Client\FirecrawlClient;
use Firecrawl\Models\ScrapeOptions;
use Firecrawl\Models\CrawlOptions;
use Firecrawl\Models\SearchOptions;

$client = FirecrawlClient::create(apiKey: 'fc-YOUR_API_KEY');

// 抓取单个 URL
$doc = $client->scrape("https://firecrawl.dev",
    new ScrapeOptions(formats: ["markdown"]));
echo $doc->markdown;

// 使用 AI 智能体进行自主数据采集
$result = $client->agent(
    new AgentOptions(prompt: "Find the founders of Stripe"));
echo $result->data;

// 爬取网站（自动轮询等待完成）
$job = $client->crawl("https://docs.firecrawl.dev",
    new CrawlOptions(limit: 50));
foreach ($job->data as $page) {
    echo $page->metadata->sourceURL;
}

// 网页搜索
$results = $client->search("best AI data tools 2024",
    new SearchOptions(limit: 10));
print_r($results);

// 抓取单个 URL
$doc = $client->scrape('https://firecrawl.dev', ScrapeOptions::with(
    formats: ['markdown'],
));
echo $doc->getMarkdown();

// 爬取网站（自动等待完成）
$job = $client->crawl('https://docs.firecrawl.dev', CrawlOptions::with(limit: 50));
foreach ($job->getData() as $page) {
    echo $page->getMetadata()['sourceURL'] . "\n";
}

// 搜索网页
$results = $client->search('best AI data tools 2024', SearchOptions::with(limit: 10));
print_r($results);

查看所有集成 →

你喜欢的工具没有列出来？提交一个 issue 告诉我们！

Firecrawl 基于 AGPL-3.0 许可证开源。firecrawl.dev 上的云版本包含更多功能：

本地运行方式请参阅贡献指南。自托管方式请参阅自托管指南。

我们欢迎各种贡献！提交 pull request 前请先阅读我们的贡献指南。

本项目主要基于 GNU Affero General Public License v3.0（AGPL-3.0）授权。SDK 和部分 UI 组件基于 MIT 许可证授权。具体目录中的 LICENSE 文件中有详细信息。

最终用户有责任在爬取时遵守各网站的政策。用户应遵守适用的隐私政策和使用条款。Firecrawl 默认遵守 robots.txt 指令。使用 Firecrawl 即表示你同意遵守这些条件。