# AI crawler policy

Whether models may crawl, train on, or quote your content is now a business decision. For most sites it is made by accident, in a robots.txt nobody has reviewed since AI crawlers arrived.

## What goes wrong

- Some AI crawlers are blocked and others are not, with no stated reason.
- ai.txt says no to training while robots.txt lets the training crawlers in.
- llms.txt points assistants at pages that robots.txt forbids them to read.

## What Flowpane does

- Assesses robots.txt against a directory of named AI crawlers and reads Content-Signal directives.
- Flags conflicts between robots.txt, ai.txt and llms.txt as cross-file findings.
- Grades AI readiness from A to F, with the checks behind the grade.

## Go deeper

- https://flowpane.com/learn/robots-txt.md
- https://flowpane.com/learn/ai-txt.md
- https://flowpane.com/learn/llms-txt.md
- https://flowpane.com/for/operators.md
- https://flowpane.com/for/agencies.md
- https://flowpane.com/for/enterprise.md