ai.txt

ai.txt is a plain-text file at the root of a site that states how the organisation would like AI systems to use its content: for training, as input to generated answers, in AI search and for automated scraping. There is no single ratified standard for it, so Flowpane documents and checks one explicit dialect. It states preferences; it does not enforce them.

What it is for

robots.txt answers who may fetch a page. ai.txt answers what they may do with it once fetched. The readers are AI operators whose crawlers choose to honour declared preferences, and the people who need a written position they can point to: legal and policy teams, partners, licensees and journalists.

Its practical value is clarity of record. A site that publishes AI-Training: no next to a named contact and a policy link has made a deliberate, dated statement. A site with nothing has left its position to be inferred from crawler rules that were written for a different purpose.

No single standard

Several proposals share the name ai.txt, and current IETF work on AI usage preferences attaches them in other ways, such as robots.txt directives and HTTP headers. Flowpane does not pretend otherwise. It supports one documented root-file dialect: Key: Value lines, with # for comments.

Fields outside this list are preserved and shown as Information. Flowpane never removes or penalises them.

How it relates to robots.txt

robots.txt carries two AI signals of its own. User-agent groups for named crawlers control access. The Content-Signal directive declares usage preferences with three keys: search, ai-train and ai-input. That last pair overlaps directly with AI-Training and AI-Input in ai.txt, and nothing forces the two files to agree.

They are usually owned by different people. The platform team edits robots.txt during a release; the policy owner signs off ai.txt once a year.

# robots.txt
User-agent: *
Content-Signal: search=yes, ai-train=yes
Allow: /

# ai.txt
AI-Training: no

Each file is valid on its own. Together they make two opposite statements about training, and a reader has no way to tell which one is current. That is worse than silence, because it undermines the credibility of both.

How it goes stale

• Divergence. One file is updated for a partnership or a policy change and the other is not.
• A passed Expires date. The file was meant to be reviewed by a date, and nobody did.
• A dead contact. Contact names a mailbox that belonged to someone who has left.
• Invalid values. AI-Training: maybe or AI-Training: disallow is not a stance any reader can act on.
• An HTML fallback. A catch-all route returns a web page at /ai.txt with status 200.

What true looks like

# ai.txt for www.example.com
# Reviewed by the policy owner on 2026-09-01
AI-Training: no
AI-Input: yes
AI-Search: yes
AI-Scraping: no
AI-Policy: https://www.example.com/ai-policy
Contact: mailto:ai-policy@example.com
License: https://www.example.com/content-licence
Canonical: https://www.example.com/ai.txt
Expires: 2027-09-01

• Every stance is exactly yes, no or conditional.
• The stances match the Content-Signal line in robots.txt, or the difference is deliberate and recorded in the policy.
• Contact is a monitored role address, not a person's inbox.
• Expires is a real review date, renewed each time the declaration is reviewed.

How Flowpane checks it

Flowpane fetches /ai.txt over public HTTPS. The file carries a weight of 6 in the site score.

• Missing, empty, or HTML. Each scores 0.
• Syntax. Lines that are not Key: Value, a file with no supported fields, and stance values other than yes, no or conditional are Issues.
• Training stance. No AI-Training field is a Recommendation. Flowpane will not choose a stance for you.
• Contact. A missing Contact is a small deduction. Invalid syntax is an Issue. Flowpane checks syntax only; it does not test that the address is reachable.
• Expires. An invalid date is an Issue. A date in the past is flagged as possibly outdated: a prompt to review, not an RFC expiry rule.
• Coherence. The coherence engine compares AI-Training and AI-Input with Content-Signal in robots.txt, and Contact with security.txt.

The starter Flowpane drafts for a missing file is comments only: it chooses no stance and invents no contact, licence or policy link. AI Assist can draft wording, but it cannot choose a stance, and its output must pass the same factual-safety and deterministic validation as anything you write. No draft changes the score; the next public check after you publish does.

Related

• robots.txt
• llms.txt
• Cross-file coherence
• security.txt
• AI crawlers