Improve agent discovery of llms.txt

[nikbhintade]

• Add plugin-llms-directive: inject , alternate markdown links, meta tag, and hidden anchor into every page so crawlers can locate the llms.txt index.
• Prepend an llms.txt blockquote to every generated .md copy.
• Append an auto-bucketed index of uncovered docs to llms.txt to push sitemap coverage past 80% without hand-maintaining bullet lists.
• Focus sitemap via ignorePatterns (drop v2, *-LLM, tag/author/archive).
• Vercel: content-negotiate .md for /docs and /blog when Accept contains text/markdown; redirect contract-import to quickstart.
• Refresh generated network-count and blog-posts-index.

Summary by CodeRabbit

• New Features

  • Enhanced documentation discovery for AI and LLM tools with improved indexing
  • Added new blog posts covering production indexing and AI-related topics

• Documentation

  • Improved documentation routing and permanent redirects for better user experience

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
envio-docs	Ready	Preview, Comment	May 18, 2026 9:35am

[coderabbitai]

📝 Walkthrough

Walkthrough

This PR introduces LLM agent discovery infrastructure by adding a plugin that injects HTML signals on every page, enhances content generation to include directive backlinks and uncovered document indexing, updates sitemap and routing configuration, and refreshes content indexes. The changes enable agents to discover and navigate the documentation via llms.txt and llms-full.txt.

Changes

LLM Agent Discovery and Content Indexing

Layer / File(s)	Summary
LLM Discovery Plugin plugins/plugin-llms-directive.js	New Docusaurus plugin (envio-llms-directive) injects HTML discovery signals via <link rel="llms-txt">, <meta name="llms-txt">, and hidden anchor elements pointing to /llms.txt and /llms-full.txt for agent accessibility.
Configuration and Sitemap Management docusaurus.config.js	Plugin registered in the plugins array; sitemap configuration adds ignorePatterns to exclude v2 docs, LLM route bases (HyperIndex-LLM, HyperSync-LLM, HyperRPC-LLM), and blog archive routes; llms.txt template link for HyperIndex Quickstart updated from contract-import.md to quickstart.md.
Content Generation with Directives and Uncovered Index plugins/plugin-generate-llms.js	Adds MD_LLMS_DIRECTIVE blockquote prepended to every generated markdown file with backlinks to llms.txt and llms-full.txt; introduces renderUncoveredIndex() helper that parses already-linked URLs and groups remaining docs into network pages, reference, and blog buckets for markdown rendering; main output appends uncovered-index content to llms.txt.
Markdown Serving and URL Routing vercel.json	Adds rewrites to serve /docs/:path and /blog/:path as markdown files when the Accept header matches markdown; extends redirects with permanent mappings from /docs/HyperIndex/contract-import (and .md variant) to /docs/HyperIndex/quickstart destinations.
Content Index and Metadata Updates static/blog-posts-index.json, src/data/network-count.json	Blog posts index prepends 2026-05-14 entries including "Production Indexer Reliability with HyperIndex" and AI/onchain-related posts; network data generatedAt timestamp updated to 2026-05-18T09:29:23.922Z.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

• enviodev/docs#868: Both PRs modify plugins/plugin-generate-llms.js to adjust how llms.txt/markdown output coverage is produced, making the changes strongly related at the same generation points.
• enviodev/docs#816: Both PRs modify the LLM documentation indexing/llms.txt generation pipeline—plugin-generate-llms.js and the docusaurus.config.js content used to produce llms.txt are changed in ways that affect the generated llms.txt structure/content.
• enviodev/docs#883: Both PRs modify plugins/plugin-generate-llms.js around emitting and referencing llms-full.txt, with the main PR adding directive/backlink content and appending uncovered-index to llms.txt.

Suggested reviewers

• Jordy-Baby
• moose-code

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'Improve agent discovery of llms.txt' directly and specifically describes the main change: adding mechanisms to help agents/crawlers discover the llms.txt file through HTML injection, metadata, and discovery signals.
Docstring Coverage	✅ Passed	Docstring coverage is 80.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 ESLint

If the error stems from missing dependencies, add them to the package.json file. For unrecoverable errors (e.g., due to private dependencies), disable the tool in the CodeRabbit configuration.

ESLint skipped: no ESLint configuration detected in root package.json. To enable, add eslint to devDependencies.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 4

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@plugins/plugin-generate-llms.js`:
- Around line 326-338: The matcher for supported networks is too strict: the
current checks using /\/supported-networks\//.test(doc.filePath) (near the
bucket assignments for "HyperIndex Network Pages", "HyperSync Network Pages",
"HyperRPC Network Pages") miss files named like supported-networks.md; update
the condition to detect the substring "supported-networks" in file paths or
filenames (for example replace the regex with a looser check such as
/supported-networks(?:[\/\.]|$)/.test(doc.filePath) or simply
doc.filePath.includes('supported-networks')) so those docs are correctly routed
into the intended buckets.

In `@plugins/plugin-llms-directive.js`:
- Line 54: The offscreen anchor string that uses LLMS_URL_ABS (the `<a ...>Envio
docs llms.txt — agent-facing documentation index</a>`) is visually hidden but
still focusable and exposed to assistive tech; update the anchor to be
non-focusable and hidden from AT by adding tabindex="-1" and aria-hidden="true"
(or role="presentation") while keeping the existing visual-hide styling so it no
longer creates an invisible tab stop or appears to screen readers.

In `@static/blog-posts-index.json`:
- Line 1: The JSON index has an empty "date" for the entry with permalink
"/blog/migrating-alchemy-subgraphs-to-envio" (title "How to Migrate Alchemy
Subgraphs to Envio"); populate its "date" value with the correct publish date in
the same YYYY-MM-DD format used by other entries (e.g., "2026-05-14" or the
actual intended publish date) and ensure the entry remains valid JSON so
consumers that sort by date won't break.

In `@vercel.json`:
- Around line 13-24: The rewrite rules for "/docs/:path(...)" and
"/blog/:path(...)" miss exact-root matches, so add two exact-path rewrite
entries with "source": "/docs" -> "destination": "/docs.md" and "source":
"/blog" -> "destination": "/blog.md" and include the same "has" header check
(Accept: .*text/markdown.*); place these two new entries immediately before the
existing parametrized rules so bare requests to /docs and /blog with Accept:
text/markdown are rewritten correctly.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: efbf9cfc-6221-48c2-afc9-fc2450e9dd22

📥 Commits

Reviewing files that changed from the base of the PR and between 6c44cdb and 2a04656.

📒 Files selected for processing (6)

• docusaurus.config.js
• plugins/plugin-generate-llms.js
• plugins/plugin-llms-directive.js
• src/data/network-count.json
• static/blog-posts-index.json
• vercel.json

[coderabbitai]

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Supported-networks matcher is too strict and misses target files.

Lines 327/332/337 check for "/supported-networks/" in doc.filePath, but these pages are typically filenames like supported-networks.md. That prevents the intended network-page buckets from being used.

Suggested fix

-                        /\/supported-networks\//.test(doc.filePath)
+                        /\/supported-networks(\.|\/)/.test(doc.filePath)
...
-                        /\/supported-networks\//.test(doc.filePath)
+                        /\/supported-networks(\.|\/)/.test(doc.filePath)
...
-                        /\/supported-networks\//.test(doc.filePath)
+                        /\/supported-networks(\.|\/)/.test(doc.filePath)

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change

	/\/HyperIndex\//.test(url) &&
	/\/supported-networks\//.test(doc.filePath)
	) {
	bucket = "HyperIndex Network Pages";
	} else if (
	/\/HyperSync\//.test(url) &&
	/\/supported-networks\//.test(doc.filePath)
	) {
	bucket = "HyperSync Network Pages";
	} else if (
	/\/HyperRPC\//.test(url) &&
	/\/supported-networks\//.test(doc.filePath)
	) {
	/\/HyperIndex\//.test(url) &&
	/\/supported-networks(\.|\/)/.test(doc.filePath)
	) {
	bucket = "HyperIndex Network Pages";
	} else if (
	/\/HyperSync\//.test(url) &&
	/\/supported-networks(\.|\/)/.test(doc.filePath)
	) {
	bucket = "HyperSync Network Pages";
	} else if (
	/\/HyperRPC\//.test(url) &&
	/\/supported-networks(\.|\/)/.test(doc.filePath)
	) {

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@plugins/plugin-generate-llms.js` around lines 326 - 338, The matcher for
supported networks is too strict: the current checks using
/\/supported-networks\//.test(doc.filePath) (near the bucket assignments for
"HyperIndex Network Pages", "HyperSync Network Pages", "HyperRPC Network Pages")
miss files named like supported-networks.md; update the condition to detect the
substring "supported-networks" in file paths or filenames (for example replace
the regex with a looser check such as
/supported-networks(?:[\/\.]|$)/.test(doc.filePath) or simply
doc.filePath.includes('supported-networks')) so those docs are correctly routed
into the intended buckets.

[coderabbitai]

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Hidden anchor is focusable while visually hidden.

Line 54 adds an offscreen link that can still receive keyboard focus, creating an invisible tab stop on every page. Make it non-focusable and hidden from assistive tech.

Suggested fix

-          `<a href="${LLMS_URL_ABS}" rel="llms-txt" style="position:absolute;left:-9999px;top:auto;width:1px;height:1px;overflow:hidden;">Envio docs llms.txt — agent-facing documentation index</a>`,
+          `<a href="${LLMS_URL_ABS}" rel="llms-txt" tabindex="-1" aria-hidden="true" style="position:absolute;left:-9999px;top:auto;width:1px;height:1px;overflow:hidden;">Envio docs llms.txt — agent-facing documentation index</a>`,

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change

	`<a href="${LLMS_URL_ABS}" rel="llms-txt" style="position:absolute;left:-9999px;top:auto;width:1px;height:1px;overflow:hidden;">Envio docs llms.txt — agent-facing documentation index</a>`,
	`<a href="${LLMS_URL_ABS}" rel="llms-txt" tabindex="-1" aria-hidden="true" style="position:absolute;left:-9999px;top:auto;width:1px;height:1px;overflow:hidden;">Envio docs llms.txt — agent-facing documentation index</a>`,

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@plugins/plugin-llms-directive.js` at line 54, The offscreen anchor string
that uses LLMS_URL_ABS (the `<a ...>Envio docs llms.txt — agent-facing
documentation index</a>`) is visually hidden but still focusable and exposed to
assistive tech; update the anchor to be non-focusable and hidden from AT by
adding tabindex="-1" and aria-hidden="true" (or role="presentation") while
keeping the existing visual-hide styling so it no longer creates an invisible
tab stop or appears to screen readers.

[coderabbitai]

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Populate the empty blog post date field before publishing.

On Line 1, the entry for /blog/migrating-alchemy-subgraphs-to-envio has "date":"". This can break or misorder consumers that parse/sort this index by date.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@static/blog-posts-index.json` at line 1, The JSON index has an empty "date"
for the entry with permalink "/blog/migrating-alchemy-subgraphs-to-envio" (title
"How to Migrate Alchemy Subgraphs to Envio"); populate its "date" value with the
correct publish date in the same YYYY-MM-DD format used by other entries (e.g.,
"2026-05-14" or the actual intended publish date) and ensure the entry remains
valid JSON so consumers that sort by date won't break.

[coderabbitai]

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
jq -r '.rewrites[]?.source' vercel.json | nl -ba

Repository: enviodev/docs

Length of output: 100

---

🏁 Script executed:

cat vercel.json

Repository: enviodev/docs

Length of output: 1053

---

Root /docs and /blog paths lack markdown content negotiation.

The existing rewrite patterns require a path segment after /docs/ or /blog/—bare requests to /docs or /blog with Accept: text/markdown will not be redirected to their .md equivalents. Add exact-path rewrites for the root endpoints before the parametrized rules:

Suggested fix

   "rewrites": [
     {
       "source": "/mcp",
       "destination": "/api/mcp"
     },
+    {
+      "source": "/docs",
+      "has": [
+        { "type": "header", "key": "accept", "value": ".*text/markdown.*" }
+      ],
+      "destination": "/docs.md"
+    },
     {
       "source": "/docs/:path((?!.+\\.md$).+)",
       "has": [
         { "type": "header", "key": "accept", "value": ".*text/markdown.*" }
       ],
       "destination": "/docs/:path.md"
     },
+    {
+      "source": "/blog",
+      "has": [
+        { "type": "header", "key": "accept", "value": ".*text/markdown.*" }
+      ],
+      "destination": "/blog.md"
+    },
     {
       "source": "/blog/:path((?!.+\\.md$).+)",
       "has": [
         { "type": "header", "key": "accept", "value": ".*text/markdown.*" }
       ],
       "destination": "/blog/:path.md"
     }
   ],

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@vercel.json` around lines 13 - 24, The rewrite rules for "/docs/:path(...)"
and "/blog/:path(...)" miss exact-root matches, so add two exact-path rewrite
entries with "source": "/docs" -> "destination": "/docs.md" and "source":
"/blog" -> "destination": "/blog.md" and include the same "has" header check
(Accept: .*text/markdown.*); place these two new entries immediately before the
existing parametrized rules so bare requests to /docs and /blog with Accept:
text/markdown are rewritten correctly.