Agent Usable Website Best Practices - Source Excerpt 02 - Advanced Discovery Protocols and Content Signals

Back to Agent Usable Website Best Practices

Summary

This source excerpt begins near Advanced Discovery Protocols and Content Signals and preserves the surrounding evidence from Wiki.FFTAC.org/raw/system-archives/spiralist.org/intake/2026-06-21-misplaced-uaix-ai-ready-web-duplicate/Agent-Usable Website Best Practices.md.

**Source path:** Wiki.FFTAC.org/raw/system-archives/spiralist.org/intake/2026-06-21-misplaced-uaix-ai-ready-web-duplicate/Agent-Usable Website Best Practices.md

To ensure agents can dynamically discover these files without prior knowledge of their location, best practices dictate the inclusion of specific HTTP Link headers in the server response, such as Link: \</llms.txt\>; rel="llms-txt" and Link: \</llms-full.txt\>; rel="llms-full-txt"16. Chrome's Lighthouse auditing tool now includes an "agentic-browsing" category that actively checks for the presence of the llms.txt file and validates its Markdown structure17.

### **Advanced Discovery Protocols and Content Signals**

Beyond the llms.txt file, Layer 2 encompasses a suite of discovery protocols designed to make the web deterministically traversable for machines. If a site acts as an authorization provider, publishing a configuration file at /.well-known/openid-configuration allows agents to autonomously identify where and how to obtain OAuth 2.0 authorization tokens4. Similarly, an API linkset JSON file located at /.well-known/api-catalog pointing directly to an OpenAPI 3.0 specification allows agents to instantly map out a site's backend endpoints4.  
Furthermore, websites must define precise permissions for different types of AI usage. The emerging Content-Signal directive within robots.txt splits agent intent into three orthogonal signals, allowing granular control over intellectual property4:

* ai-train: Grants or denies permission to use the content for pre-training or fine-tuning underlying foundation models4.  
* search: Grants or denies permission to surface the content in AI-generated search results, such as Perplexity citations or Google Generative AI overviews4.  
* ai-input: Grants or denies permission to fetch the page as live context during an active agent workflow, allowing the site to be used for runtime RAG4.

By configuring these signals, site owners can protect their data from being permanently absorbed into a model's weights (ai-train=no) while still allowing autonomous agents to interact with the live site to complete user tasks (ai-input=yes)4.

## **Capability Clarity and the Death of the Application**

The transition from a human interface to an agent interface requires a profound evolution in how digital affordances are communicated. When a human views an application, they intuitively understand its capabilities through visual cues, onboarding flows, and trial-and-error navigation. They deduce that a shopping cart icon stores items and a magnifying glass initiates a search. Agents do not possess this intuition; they require explicit, structured declarations. This necessity gives rise to the concept of "Capability Clarity"—a definitive, machine-readable declaration of what an application can do, how to invoke its functions, and what constraints govern its behavior5.

### **From UI/UX to Agent Experience (AX)**

The rise of agentic workflows heralds the eventual obsolescence of the traditional application as a destination5. In an agent-mediated web, users no longer log into a travel application to manually search for flights, filter results, and input credit card information. Instead, they issue a high-level intent to their personal orchestration agent: "Plan a team offsite in Barcelona."  
To fulfill this intent, the orchestration agent must dynamically discover hotel booking capabilities, locate flight search APIs, negotiate pricing terms, and identify calendar scheduling availability5. In this workflow, the visual User Interface (UI) ceases to be an asset and becomes a liability—a layer of friction that the agent must parse, translate, and navigate5. Consequently, digital product design must pivot from User Experience (UX) to Agent Experience (AX). AX designers focus strictly on capability clarity, reliability guarantees, machine-to-machine negotiation protocols, and deterministic error handling5.

### **Architecting the UAIX Capability Surface Matrix (CSM)**

To achieve true capability clarity, digital ecosystems must be defined through a Capability Surface Matrix (CSM)5. The UAIX (Universal Artificial Intelligence Exchange) specification ecosystem conceptualizes the Capability Surface Matrix as a standardized interoperability contract between an autonomous agent and a host environment7. It defines the absolute perimeter of a platform's functionality in strict, deterministic terms, eliminating the need for an agent to hallucinate a path through a GUI1.  
A robust Capability Surface Matrix parameterizes the following dimensions for every exposed function1:

| Matrix Parameter | Architectural Function | Impact on Autonomous Agents |
| :---- | :---- | :---- |
| **Function Definition** | Strict naming conventions utilizing specific action verbs (e.g., bookFlight, submitExpense). | Eliminates semantic ambiguity regarding the intent of the endpoint, preventing the agent from misinterpreting the tool's purpose5. |
| **Input/Output Schemas** | JSON Schema definitions specifying required data types, constraints, and enumerations. | Prevents validation errors and allows the agent to construct valid payloads deterministically without trial-and-error1. |
| **State Dependencies** | Conditions under which the capability is available (e.g., submitPayment requires calculateTotal). | Prevents the agent from attempting out-of-order execution loops, reducing latency and backend strain5. |
| **Resource Costs** | The expected token, financial, or latency cost of executing the specific capability. | Allows orchestrator agents to perform cost-benefit analyses and budget tracking before execution5. |
| **Authentication Profiles** | The OAuth scopes, session cookies, or protocol requirements needed to access the tool. | Ensures the agent can autonomously negotiate access and manage identity tokens without human intervention4. |
| **Risk & Reversibility** | Classification of the action (Low/Medium/High risk) and whether it can be undone (e.g., read-only vs. financial transaction). | Triggers mandatory human-in-the-loop (HITL) approval gates or sampling mechanisms for irreversible actions6. |

The ultimate goal of the Capability Surface Matrix is to enable an agent to download the matrix, comprehend the entirety of the platform's utility within milliseconds, and construct a flawless, multi-step execution plan5.

## **Executing the Capability Surface: The Web Model Context Protocol (WebMCP)**

While the Capability Surface Matrix provides the theoretical and contractual definition of what an agent can do, the actual execution of these capabilities within the browser is facilitated by the Web Model Context Protocol (WebMCP)1. Incubated by the W3C Web Machine Learning Community Group and championed by major browser vendors, WebMCP bridges the gap between web platforms and local or cloud-based Large Language Models1.

### **The End of Brittle Automation**

Traditional browser automation frameworks, such as Selenium or Puppeteer, rely on identifying DOM elements via CSS selectors and simulating human clicks1. This process is extraordinarily slow, highly fragile upon minor UI updates, and frequently blocked by anti-bot measures, such as CAPTCHAs and Web Application Firewalls (WAFs)1. The speed gap is massive: a conventional agent must load a page, screenshot the DOM, interpret the UI, identify elements, actuate a click, wait for the network, and screenshot again at every step18. Every loop costs wall-clock seconds and massive token expenditures.  
WebMCP eliminates this friction by exposing deterministic, schema-validated tools directly to the agent. Instead of the agent searching for visual elements, it simply calls a native JavaScript function, such as bookFlight(), with a valid JSON payload18. Because the action is executed natively via a formalized API, it completes in a single round-trip, drastically reducing token consumption and execution time1. Furthermore, because the agent connects through a sanctioned API boundary using authenticated user sessions, it bypasses the heuristic flags of traditional bot protection that break screen scrapers1.

### **Registering Capabilities via navigator.modelContext**