Data story · A reading of 3 October 2026
What software saw at 24 federal agency home sites
One program, plainly labelled, asked the home sites of the 24 CFO Act agencies for their pages on one day.[1] 10 refused it at the door. None of the 14 that answered had a page written for AI assistants.[2]
14 of 24
home sites answered the labelled program with the page; 10 answered with HTTP 403, a refusal.[2]
What the 14 that answered offered
- 14 of 14 had a robots.txt; 1 of them names an AI crawler.[2]
- 13 of 14 had a site map.[2]
- 0 had an llms.txt, 0 returned Markdown when asked, and 0 listed a tool an AI agent could use.[2]
| Agency | Home page | robots.txt | Site map | llms.txt |
|---|---|---|---|---|
| USDA | refused (403) | - | - | - |
| Commerce | refused (403) | - | - | - |
| Defense | refused (403) | - | - | - |
| Education | refused (403) | - | - | - |
| Energy | answered | yes | yes | - |
| HHS | refused (403) | - | - | - |
| DHS | answered | yes | yes | - |
| HUD | answered | yes | - | - |
| Interior | answered | yes | yes | - |
| Justice | answered | yes | yes | - |
| Labor | refused (403) | - | - | - |
| State | answered | yes | yes | - |
| Transportation | refused (403) | - | - | - |
| Treasury | answered | yes | yes | - |
| VA | answered | yes | yes | - |
| EPA | answered | yes | yes | - |
| NASA | answered | yes | yes | - |
| USAID | refused (403) | - | - | - |
| GSA | answered | yes | yes | - |
| NSF | answered | yes | yes | - |
| NRC | refused (403) | - | - | - |
| OPM | answered | yes | yes | - |
| SBA | answered | yes | yes | - |
| SSA | refused (403) | - | - | - |
Data: sites-2026-10-03.csv (one row per site, every column the analysis used)
What this does not say
- It does not say these sites block AI agents. It says 10 refused one plainly labelled program on one day.
- It does not rank agencies. It reads home sites only, once, from one network address.
- Three known-answer pages were read the same way and came out as expected.[2]
- The agencies are the 24 named in the CFO Act list.[1]
A recount is planned for the monthly brief. This page stays a dated reading of 3 October 2026.
Sources
- United States Code, 31 U.S.C. 901(b), the list of CFO Act agencies (Legal Information Institute). Read 28 October 2026.
- Seven Sky Consulting, data-to-social, story 2026-10-03_federal-sites-ai-agents: the collection program, every rule and the hashes of the saved responses. Read 28 October 2026.