Flowpane reads your governance files the way any member of the public can: over HTTPS, from your canonical origin, without signing in. Its requests identify themselves with a stable user agent and carry a cryptographic signature, so a firewall can verify they really came from Flowpane. When a firewall stands between Flowpane and a file, the result is Blocked, never Missing.
Public HTTPS only
Governance evidence is only useful if it matches what the public receives. A logged-in view, a staging server or an editor's preview can all differ from the live file, so Flowpane resolves the site's canonical origin, for example https://www.example.com after redirects, and fetches from there.
When it checks your site, Flowpane never signs in, submits forms or sends credentials, and checking never writes anything to your origin. If a file is visible only after login, it is not public, and Flowpane reports what the public receives.
Identity
Governance checks and Deep Scan crawls send this user agent:
Flowpane/1.0 (+https://crawler.flowpane.com)The robots.txt product token is Flowpane. A Mozilla-compatible form of the same identity is also defined for firewalls that reject non-browser strings: Mozilla/5.0 (compatible; Flowpane/1.0; +https://crawler.flowpane.com). Rules that match on the Flowpane token cover both.
One deliberate exception applies. On a manually started check, after the normal signed fetch of a sitemap has succeeded, the sitemap reliability test requests it three more times: once as Flowpane, and once each, unsigned, with a generic browser user agent and a search-crawler user agent. It exists to detect a firewall or cache that serves different clients different content.
Web Bot Auth
A user agent string is a claim that anyone can copy. Flowpane backs it with proof: requests are signed with HTTP Message Signatures (RFC 9421) using Ed25519 keys, following the Web Bot Auth approach. Each signed request carries three headers:
Signature-Agentpoints to crawler.flowpane.com, where the public keys are published.Signature-Inputlists the signed components, the creation and expiry times, the key ID, a nonce and theweb-bot-authtag.Signatureholds the Ed25519 signature itself.
The key directory is at https://crawler.flowpane.com/.well-known/http-message-signatures-directory. A firewall or CDN that supports Web Bot Auth fetches the directory, selects the key by its ID and verifies the signature. Each signature is created per request and is valid for five minutes, which limits the value of a captured header.
# Illustrative only. Every value below is a placeholder, not a real key or signature.
GET /robots.txt HTTP/1.1
Host: www.example.com
User-Agent: Flowpane/1.0 (+https://crawler.flowpane.com)
Signature-Agent: https://crawler.flowpane.com/.well-known/http-message-signatures-directory
Signature-Input: sig1=("@authority");created=1790000000;keyid="<key-id>";alg="ed25519";expires=1790000300;nonce="<nonce>";tag="web-bot-auth"
Signature: sig1=:<base64-signature>:Where your firewall supports it, verify the signature rather than allow-listing the user agent string. If you must allow-list by string, keep the exception narrow: the governance file paths and the verification path, not the whole site.
Allowing Flowpane in robots.txt
User-agent: Flowpane
Allow: /
Crawl-delay: 5
User-agent: *
Disallow: /admin/Under RFC 9309, a crawler follows only the most specific group that matches it. Once a Flowpane group exists, Flowpane ignores the * group entirely, so copy across any Disallow lines you still want applied.
Blocked is not Missing
Every fetch ends in one of four states, and the difference between them is what keeps the evidence honest.
| State | What happened | Effect |
|---|---|---|
| Present | The file was served and assessed | Scored |
| Missing | The server said the file does not exist, such as 404 | Scores 0 |
| Blocked | A challenge, block page or refusal stood in the way | Not scored |
| Error | Timeout, server error or transport failure | Last good result stands |
Blocked means Flowpane did not see the file, so it cannot say the file is absent. Reporting Missing would send a security team to publish a security.txt that already exists while the real cause, their own firewall, goes unexamined. If a required artefact is blocked, the site is shown as unscorable rather than given a partial score. The presence of a CDN or WAF is never treated as a challenge on its own: proxy headers on an ordinary response are attribution, not evidence. Blocked needs challenge, block or rate-limit evidence.
Ownership verification
Monitoring starts only after you prove control of the site, by one of three methods:
- A file at
/.well-known/flowpane-verify-<id>.txt. - A DNS TXT record at
_flowpane.<host>. No HTTP request is involved, so this works even when a firewall blocks Flowpane. - A
flowpane-site-verificationmeta tag in the homepage<head>.
Deep Scan crawls
On Starter and above, Deep Scan crawls the site breadth-first from the homepage, following internal links within the plan's depth and page limits.
- It reads robots.txt first and obeys the group that matches the
Flowpanetoken, or*when there is none. - It honours
Crawl-delayfor that group, capped at 10 seconds, and follows redirects one hop at a time, checking robots.txt for each. - A 404 for robots.txt means no restrictions, as RFC 9309 specifies. A robots.txt that is blocked, times out or errors stops the crawl on that origin: Flowpane does not guess the rules.
- It is never scheduled automatically. At most one completed crawl per site per hour.
Search Console
With read-only access (webmasters.readonly), Flowpane imports your submitted sitemaps and their counts and runs bounded URL Inspection. Flowpane reads from Search Console. It never submits sitemaps.
That gives three evidence layers: the public pull (what is on the wire), the crawl (what is reachable) and Search Console (what Google accepted). The value is in the difference between them.