llms.txt for Universities and .edu Sites
No other site type is asked so many factual questions it never sees. Fees, deadlines, entry requirements, program lists — all of it gets answered by assistants from whatever page they can reach. Here is what 33 education domains serve today, the three ways a university file fails silently, and the structure that works.
What Education Domains Serve Today
We fetched /llms.txt from this server on October 3, 2026 across 33
university, library and online-learning domains, first with a default user agent and then
with a desktop browser user agent to separate policy from bot mitigation.
| Response | Count | Examples |
|---|---|---|
200, text/plain — a real Markdown file | 3 | imperial.ac.uk (4.2 KB), coursera.org (6.8 KB), edx.org (3.6 KB) |
200, but no file behind it | 4 | khanacademy.org, utexas.edu, utoronto.ca (JS challenge page); gla.ac.uk (a 43.5 KB HTML page) |
404 | 19 | harvard.edu, cam.ac.uk, cmu.edu, ucla.edu, duke.edu, washington.edu, ed.ac.uk, unsw.edu.au, tudelft.nl, leeds.ac.uk, u-tokyo.ac.jp, ocw.mit.edu, cs50.harvard.edu, loc.gov, futurelearn.com, udacity.com and others |
403 (edge block, unchanged with a browser user agent) | 6 | mit.edu, berkeley.edu, ox.ac.uk, monash.edu, open.ac.uk, si.edu |
405 Method Not Allowed | 1 | nyu.edu |
Three of 33 is a lower hit rate than the developer-documentation world, where the file has become routine. Two of the three that do publish are commercial learning platforms rather than universities, and the one university file we found — Imperial College London — is also the best-written file in the sample, which is not a coincidence.
Three Ways a .edu File Fails Before Anyone Reads It
The interesting results are not the 404s. They are the responses that pass a naive status-code check and still deliver nothing a model can use.
- The JavaScript challenge served as 200. utexas.edu, utoronto.ca and
khanacademy.org all answered
200— with a 3,038-byte bot-protection interstitial titled “Client Challenge”. Two of the three bodies were byte-identical. A monitoring script that only counts status codes reports this as healthy; an assistant receives a script tag. This is the single most misleading failure in the sample. - The CMS catch-all. gla.ac.uk returned
200andcontent-type: text/htmlwith a 43.5 KB body — a real University of Glasgow School of Law postgraduate page, apparently served because the CMS treats unknown paths as internal navigation. Nothing is broken and nothing is a file. The tell is the content type and a body that opens with<!doctype html>. - The WAF that hides the real answer. imperial.ac.uk answered
403to our default fetch and200 text/plainto a browser user agent: the file exists and was simply not served to an unrecognized client. The reverse also happens — stanford.edu answered403by default and404with a browser user agent, meaning the block was masking a genuine absence. One request per domain proves nothing in either direction.
The Six Buckets a University File Needs
Government, commercial and academic sites have different priorities, but a university file organizes around the questions prospective students, current students and researchers actually ask. Six buckets cover it:
- Institution core. About, leadership and governance, faculties and departments, contact and campus locations.
- Study and admissions. The undergraduate and postgraduate entry points, how to apply, and the official key-dates page — linked, never transcribed.
- Programs and course catalogue. The catalogue root plus one link per faculty catalogue. With thousands of program pages regenerated annually, document the URL pattern instead of listing slugs.
- Fees, funding and scholarships. The official fee-status and funding pages only. A hardcoded tuition figure in this file becomes a wrong answer the moment the fee schedule changes.
- Research and people. Research portal, institutional repository, staff profile directory, research ethics and integrity policy.
- Library and open resources. Library home, discovery layer, open-access repository, open courseware, research data catalogue.
News, events, jobs, alumni and giving belong in an ## Optional section, so a
client under a tight context budget can skip them. A compressed example of the shape:
# Example University
> Example University is a public research university with three campuses,
> 21,000 students and faculties of science, engineering, medicine and law.
> The pages below are the official sources for admissions, fees, programs
> and research. Program URLs follow /study/programs/<faculty>/<slug>/.
> Fees and deadlines change annually: the linked pages are the source of
> truth, not this file. Admissions queries: /study/contact/
## Institution
- [About](https://example.edu/about/): history, leadership, governance and campuses
## Study and admissions
- [Undergraduate admissions](https://example.edu/study/undergraduate/): requirements and how to apply
- [Key dates](https://example.edu/study/key-dates/): application deadlines and decision rounds
## Programs and catalogue
- [Programme catalogue](https://example.edu/study/programs/): every degree, filterable by faculty
## Fees and funding
- [Fees and funding](https://example.edu/study/fees/): current fee schedule and fee status rules
- [Scholarships](https://example.edu/study/scholarships/): eligibility and application windows
## Research and people
- [Research portal](https://example.edu/research/): groups, projects and publications
- [Staff profiles](https://example.edu/people/): academic staff directory
## Library and open resources
- [Library](https://www.example.edu/library/): discovery, e-resources and subject guides
- [Open repository](https://repository.example.edu/): open-access papers, theses and data
## Optional
- [News](https://example.edu/news/) · [Events](https://example.edu/events/) · [Jobs](https://example.edu/jobs/) Two Production Patterns Worth Copying
The two things that make Imperial’s file better than a link list are procedural. First, it
places an instruction block before the first ##, telling assistants what the
file does not authorize: prefer official pages over summaries and archived content,
prefer department and policy pages over news for factual questions, and do not infer
admissions decisions, visa advice, fee status, scholarship eligibility or policy exceptions
from summaries. That is the highest-value sentence a university can publish here, because
the failure mode for an admissions answer is not omission — it is a confident guess.
Second, both Imperial and Coursera name a maintenance checkpoint inside the file: Imperial
routes time-sensitive topics (deadlines, fees, vacancies) to the page itself, and Coursera
states that the file is a curated entry point, that
sitemap.xml is the exhaustive list, and that it was last reviewed on
2026-08-13. edX does the same trick with numbers — its “Key facts” section points at the live
About page instead of freezing a statistic that will decay. All three decisions say one
thing to the reader: trust the linked page, not the index.
Subpath Splits, Ownership and the Academic Calendar
A university is really a federation of independently edited sites, and the spec’s
most-specific-path rule lets your file structure follow that reality. Put the root file at
example.edu/llms.txt, then add /library/llms.txt,
/research/llms.txt and a file per branch campus or teaching language — the same
pattern covered in
llms.txt for multilingual websites,
where the subfile wins for anything under its path.
Ownership is the part that decides whether the file survives its second year. The web team can write it, but the content owners are the registrar (dates, catalogues), the finance office (fee schedules) and the library (resources). Bind the review to events you already have: fee schedule publication, admissions cycle opening, annual catalogue regeneration. Two of those three dates move annually, and a link check that fails your build is cheaper than discovering in March that the file still points at last cycle’s deadline page — the routine the maintenance pass covers.
What Must Not Appear — and How to Verify the Rest
A .edu file sits on the most trusted domain in the sector, so the disclosure bar is higher than on a marketing site. Keep out applicant and student portals behind SSO, student record and learning-management routes, internal staff and intranet pages, unpublished exam papers or question banks, licensed library e-resources that only work from inside the institution, and any draft or unreleased strategy document. The exclusion checklist covers the general categories; for universities the assessment-integrity items are the ones that cause real damage.
# 1. Is it served as text, or is a CMS or challenge page answering?
curl -sI https://example.edu/llms.txt | grep -iE "^(HTTP|content-type|location)"
curl -s https://example.edu/llms.txt | head -1
# 2. Would an admitted crawler get the same answer?
curl -s -o /dev/null -w "%{http_code} %{content_type}\n" \
-A "OAI-SearchBot" https://example.edu/llms.txt
# 3. Every listed URL should answer 200 without a redirect chain
curl -s https://example.edu/llms.txt | grep -oE "https://[^)]+" | while read u; do
printf "%s %s\n" "$(curl -s -o /dev/null -w '%{http_code}' -L "$u")" "$u"
done | grep -v "^200" Then run the file through the llms.txt checker for structure and the summary-slot rules, and build the first draft from your admissions and catalogue sitemaps with the generator — the curation is the work, not the formatting.
Status codes, byte sizes and file contents referenced were captured from this server on October 3, 2026, with both a default curl user agent and a desktop browser user agent. Edge bot rules and published files change without notice, so re-check your own paths rather than treating this table as current configuration for any institution named. A 4xx or a 200 tells you nothing about the body until you read it.