Bright Data Web Scraper API
69
AI ReadinessAI-Aware (B+)69/100
Official vendor OpenAPI document · agent-readyAnalyticsData Pipelinesbearer6 EndpointsREST
For Agents
Trigger web scraping collections, monitor snapshots, and download structured datasets, or scrape a single URL on demand. Returns collected web data for an agent to process.
Use for: Trigger a Bright Data collection for a list of product URLs, Check the progress of a running scraping snapshot, Download the results of a completed data collection, Scrape a single web page and return its structured data
Not supported: Does not handle proxy network configuration, browser automation, or long-term data storage. Use for web data collection and scraping only.
The Bright Data Web Scraper API triggers asynchronous web data collections, monitors their progress, and downloads the resulting snapshots. It also scrapes individual URLs on demand and returns their structured content, and supports webhook delivery for completed batch jobs. Agents can run large scraping jobs against Bright Data's managed datasets without operating proxy rotation or scraper infrastructure themselves.
Install Jentic One Beta
Connect the Bright Data Web Scraper API to your agent
Jentic One is a self-hosted execution layer for AI agents. It lets your agent call the Bright Data Web Scraper API, or any other public or private API you need. You set the rules, the agent never sees your credentials, and every call is logged.
Two steps, two machines. Install the instance in a safe environment, then register your agent from wherever it runs.
1
Step 1: Jentic One Host machine
# On the machine that will host your Jentic One instance:
curl -fsSL "https://jentic.com/install.sh?src=apis&api=%2Fapis%2Fbrightdata.com%2Fbrightdata" | sh2
Step 2: Agent machine
# On the machine where your agent runs (keep this separate from the instance):
curl -fsSL "https://jentic.com/install.sh?src=apis&api=%2Fapis%2Fbrightdata.com%2Fbrightdata" | sh
jentic register # connects your agent to your Jentic One instanceJentic One is in public beta. The setup above keeps your agent separate from the instance, which is what you want before using real credentials: an agent running as the same OS user as Jentic One can read its stored keys directly. Just evaluating? A single local install is fine to start. See the secure deployment guide for the tiers.
Capabilities
What an agent can do with Bright Data Web Scraper API.
Trigger asynchronous data collection jobs against Bright Data scraper datasets
Monitor collection progress for a running snapshot
Download completed snapshot data once a collection finishes
Scrape a single URL on demand and return its structured content
List existing scraping snapshots and cancel collections that are still running
Use Cases
Patterns agents use Bright Data Web Scraper API for, with concrete tasks.
★ Agent-Driven Web Data Collection
An AI agent gathers structured web data on demand by triggering a Bright Data collection, polling the snapshot until it completes, and downloading the results. Because collection is asynchronous with webhook delivery, the agent can start a large job and pick up the output later without holding a connection open. This turns open-ended research and monitoring tasks into a repeatable data pipeline the agent drives itself.
Trigger a data collection for a set of product page URLs, poll the snapshot until it reports complete, then download the structured results as JSON
Bulk Dataset Collection
Teams scrape large batches of pages by submitting a collection request and letting Bright Data run it asynchronously, with a webhook signalling when the snapshot is ready to download. Progress can be monitored per snapshot and a running job can be cancelled if the inputs were wrong. This suits recurring catalogue, pricing, or listing refreshes where the input set changes but the extraction shape stays the same.
Submit a batch collection for a list of category URLs, wait for the completion webhook, and download the snapshot once progress reports finished
On-Demand Single Page Scraping
For one-off lookups an agent scrapes a single URL synchronously and receives the page's structured content directly, without setting up a dataset collection. This is the fast path for answering a question about one page: a product detail, an article, or a listing. It keeps quick fetches separate from the heavier asynchronous batch flow.
Scrape a single product URL and return the extracted structured content for the agent to summarise
Key Endpoints
6 endpoints — the bright data web scraper api triggers asynchronous web data collections, monitors their progress, and downloads the resulting snapshots.
METHOD
PATH
DESCRIPTION
POST
/datasets/v3/trigger
Trigger a data collection
GET
/datasets/v3/progress/{snapshot_id}
Monitor collection progress
GET
/datasets/v3/snapshot/{snapshot_id}
Download snapshot data
GET
/datasets/v3/snapshots
List snapshots
POST
/datasets/v3/snapshot/{snapshot_id}/cancel
Cancel a running collection
POST
/request
Scrape a single URL
POST
/datasets/v3/trigger
Trigger a data collection
GET
/datasets/v3/progress/{snapshot_id}
Monitor collection progress
GET
/datasets/v3/snapshot/{snapshot_id}
Download snapshot data
GET
/datasets/v3/snapshots
List snapshots
POST
/datasets/v3/snapshot/{snapshot_id}/cancel
Cancel a running collection
POST
/request
Scrape a single URL
Jentic AI Readiness Score
This API is usable in Jentic One now. Its AI-readiness score against Jentic's framework shows where it stands today and where improvements would make it even easier for agents to use.
B
Bright Data Web Scraper API
- AI-Aware (B+)69/100
90
Foundational Compliance63
Developer Experience & Jentic Compatibility53
AI-Readiness & Agent Experience94
Agent Usability60
Security83
AI DiscoverabilityPowered by JenticScoring Framework 1.0.0 | Scoring Engine 0.4.2
Show dimension breakdown
90
Foundational Compliance
Base layer of spec validity and structural soundness.
Grade: ASignals: 4
98%
Lint Results
Aggregated quality score from linter diagnostics, weighted by severity.
100%
Resolution Completeness
Percentage of `$ref` references that resolve successfully.
100%
Specification Validity
Checks whether the API description parses successfully and conforms to its declared specification (e.g., OpenAPI).
60%
Structural Integrity
Structural correctness score based on schema issues using logarithmic dampening.
63
Developer Experience & Jentic Compatibility
Clarity, completeness, and ingestion readiness for developers and tooling.
Grade: BSignals: 4
3%
Example Density
How richly the API is illustrated with examples.
100%
Example Validity
Percentage of examples that conform to their schemas.
50%
Response Coverage
Percentage of operations with complete response definitions (success, client error, server error).
100%
Tooling Readiness
Health of API ingestion, bundling, and resolution within Jentic pipelines.
53
AI-Readiness & Agent Experience
Semantic breadth, depth, and agent comprehension for AI systems.
Grade: C-Signals: 4
60%
Description Coverage
Coverage of descriptions across API elements.
0%
Error Standardization
Coverage of RFC 9457 Problem Details for error responses.
100%
OperationId Quality
Coverage, uniqueness, and casing consistency of operationIds for AI inference.
50%
Summary Coverage
Coverage of summaries across operations/tags/info.
94
Agent Usability
Functional utility, complexity comfort, and AI orchestration readiness.
Grade: A+Signals: 1
94%
Complexity Comfort
Agent comfort level based on API operational and structural complexity.
60
Security
Trust, risk posture, and security compliance.
Grade: B-Signals: 1
60%
Authentication Strength
Average quality of security schemes based on authentication method strength (weakest link for OAuth2).
83
AI Discoverability
Findability, semantic richness, and reasoning readiness.
Grade: ASignals: 1
83%
Descriptive Richness
Clarity and depth of descriptions across API elements.
Score it yourself
Every API in the directory is allowlisted, so you can re-score it with no key required.
npx @jentic/api-scorecard-cli score <openapi-url>Why Jentic?
What agents get from Jentic-routed access to this vendor.
Setup
Wiring the Bright Data Web Scraper API by hand means handling its bearer token, kicking off asynchronous collections, and polling snapshot progress before you can download anything. Through Jentic you install once, import Bright Data from the API Directory, store the token once, and your agent triggers and retrieves collections.
Permission scoping
Bright Data puts the snapshot id in the URL path (/datasets/v3/snapshot/{snapshot_id}), so a rule can bound your agent to reading and downloading snapshots. You choose the operations it may call, so triggering new collections or cancelling a run are not included unless you add them.
Credential isolation
Your Bright Data token is stored once, encrypted, by your own Jentic One instance and injected at execution time. It never enters the agent's prompt, logs, or context.
Intent-based discovery
Agents search Jentic by intent such as 'scrape a web page' or 'download a dataset snapshot', and Jentic returns the matching Bright Data operation with its input schema so the agent calls the right endpoint without reading the reference docs.
Related APIs
Alternatives and complements available in the Jentic catalogue.
FAQs
Specific to using Bright Data Web Scraper API through Jentic.
What authentication does the Bright Data Web Scraper API use?
The Bright Data Web Scraper API authenticates with an HTTP bearer token per its OpenAPI spec: you send your API key from your Bright Data account settings as an Authorization Bearer header on every request. Through Jentic the token is held encrypted by your own Jentic One instance and attached at call time, so it never reaches the agent's context.
Can I scrape a single URL with the Bright Data Web Scraper API?
Yes. The API exposes a single-URL scrape operation that returns a page's structured content directly, alongside the dataset collection flow for larger asynchronous jobs. An agent can call it for one-off pages without setting up a full collection.
What are the rate limits for the Bright Data Web Scraper API?
The OpenAPI spec does not specify rate limits. Check the Bright Data documentation at https://docs.brightdata.com/scraping-automation/web-scraper-api/overview for current limits and concurrency guidance.
Can I limit what my agent is allowed to do with the Bright Data Web Scraper API?
Yes. Write a rule that allows only the trigger, progress, and snapshot download operations, so the agent can run and retrieve collections but cannot cancel jobs or reach any operation you did not grant, and every call it makes is logged.
How do I trigger and download a Bright Data collection through Jentic?
Search Jentic for 'scrape a web page' or 'trigger a data collection', add the Bright Data Web Scraper API from the directory, and store your token once. Your agent then triggers a collection, polls the snapshot until it is ready, and downloads the results. To run it on your own infrastructure, install Jentic One from its GitHub repo.
Is there a Bright Data MCP server?
You don't need an MCP server to give your agent the Bright Data Web Scraper API. Jentic connects it directly from the API Directory: import it, store your token once, and your agent triggers and downloads collections. Operations are discovered on demand, so no extra server's tool definitions sit in the agent's context.
GET STARTED