Automated clients
GAIP's user-agents and what they read.
Outbound requests from GAIP identify themselves with one of the tokens below. None sends credentials, logs in or reads beyond public metadata and public endpoints.
| User-agent | Purpose | When it runs | Rate limits | robots.txt |
|---|---|---|---|---|
GAIP-Observatory/1.0 (+https://www.gaipagents.com/bots) | Agent Observatory: reads public Official MCP Registry listings, sends the MCP discovery handshake (initialize, tools/list; never tools/call) to remote endpoints listed there, and reads public A2A agent cards in GAIP's readiness index. | Scheduled. Registry listing at most 3 pages every 6 hours; each target at most once every 24 hours. | At most 400 requests per hour in total and 8 per target observation; at least 60 seconds between targets on the same host. | Honoured: the GAIP-Observatory group, otherwise the * group. If robots.txt cannot be read (other than 404/410), the host is skipped. Hosts on the opt-out list are skipped. |
GAIP-Conformance-Check/1.0 (+https://www.gaipagents.com/bots) | Check an agent by URL: reads the agent card, MCP server or OpenAPI document a caller submitted and checks it against the published conformance rules. | Only when a caller asks for a check. | One bounded read per submitted URL and discovery path. | Reads only the public URL a caller submitted (and its standard discovery paths). |
GAIP-Citation-Witness/1.0 (+https://www.gaipagents.com/bots) | Citation witness: checks that pages cited by a caller exist and contain the quoted text. | Only when a caller asks, up to 20 citations per call. | At most 10 page reads per host per call and 256 KiB per page; keeps a hash and at most 300 characters of matched excerpt. | Honoured (RFC 9309) for the GAIP-Citation-Witness group, otherwise the * group. |
GAIP-Supplier-Watch/1.0 (+https://www.gaipagents.com/bots) | Supplier Watch: compares published agent cards of a small public cohort over time for material change. | Scheduled, once a day. | At most 20 targets per run and 64 KiB per response; redirects are not followed. | Reads only public agent-card discovery paths (/.well-known/agent-card.json, /.well-known/agent.json). |
GAIP-Watch/1.0 (+https://www.gaipagents.com/bots) | Watch-lists: re-checks an agent card, supplier interface or delivery terms that a caller registered, and sends a signed webhook POST to the caller's own endpoint when it changes. | Scheduled per watch, at most once every 24 hours. | At least 600 seconds between reads of the same host; at most 120 webhook attempts per hour in total. | Reads only the URL the caller registered. |
GAIP-Broker/1.0 (+https://www.gaipagents.com/bots) | Opportunity Broker: reads public agent cards to check which candidates declare a fit for a bounded task. | Only when a caller asks. | At most 256 KiB per document; 5-second timeout. | Reads only the public URL a caller submitted (and its standard discovery paths). |
GAIP-Discovery/1.0 (+https://www.gaipagents.com/bots) | Discovery: reads public agent registry listings to find public agent cards for the aggregate readiness index. | Scheduled. | A bounded number of listing pages per run. | Reads public registry listing pages only. |
GAIP-Adoption-Evidence/1.0 (+https://www.gaipagents.com/bots) | Evidence records: a bounded public reachability read made while recording a caller-requested check. | Only when a caller asks. | One read of at most 32 KiB; public HTTPS on port 443 only. | Reads only the public URL a caller submitted (and its standard discovery paths). |
GAIP-Venture-Scout/1.0 (+https://www.gaipagents.com/bots) | Internal planning: reads GAIP's own public aggregate of requested task kinds. | Scheduled. | One read per run. | Reads GAIP's own endpoint only. |
GAIP-Shadow-Qualification/1.0 (+https://www.gaipagents.com/bots) | Service qualification: a bounded live read of the public services GAIP is listed on (for example registry listings) to record whether they respond as declared. | Scheduled. | One read per listed service per run. | Reads only explicitly approved public service URLs. |
GAIP-Capability-Assurance/1.0 (+https://www.gaipagents.com/bots) | Capability assurance: runs bounded synthetic tasks against GAIP's own public routes. | Scheduled. | A bounded number of synthetic cases per run. | Calls GAIP's own endpoints only. |
GAIP-Verify-Before-Call/2.0 (+https://www.gaipagents.com/bots) | Verify before call: a single optional reachability read of the endpoint a caller named, only when the caller sets live_public_probe. | Only when a caller asks. | One read of at most 32 KiB; public HTTPS on port 443 only; redirects are not followed. | Reads only the public URL a caller submitted (and its standard discovery paths). |
Opt out or correct
To stop scheduled reads of your hosts or to correct an observation, use corrections and opt-out or the machine route POST /v1/free/observatory/corrections. To block one token, add a robots.txt group for it, for example User-agent: GAIP-Observatory followed by Disallow: /.
Machine-readable: /bots.json · Methodology