Architecture, Failure, Verification and Real-World Agentic Accessibility

28. The Assistive Agent as a Layered System

An assistive agent is easy to imagine as a single artificial intelligence that “uses the computer.” In practice, reliable systems are better understood as layers. The language model is only one component. Around it sit perception channels, tool connectors, permission systems, application interfaces, memory stores, confirmation policies, logging mechanisms, error detectors and user-interface components. Accessibility is determined by the behavior of the whole system, not by the language model alone.

The first layer is state acquisition. An agent cannot act safely if it has an inaccurate picture of the environment. State can come from screenshots, the browser document object model, accessibility trees, operating-system automation APIs, application APIs, files, databases or structured tool responses. For blind users, the accessibility tree deserves particular attention because it already expresses interfaces in nonvisual semantic terms: roles, names, values, states and relationships. A screenshot may show a blue rectangle; the accessibility tree may say that the same object is a checked checkbox labeled “Email me updates.” The latter is closer to what both a screen reader and an agent need.

The second layer is intent interpretation. Natural language instructions are often incomplete. “Book the cheapest flight” leaves unstated whether stops are acceptable, whether an overnight itinerary is reasonable, whether baggage should be included and whether an unfamiliar airport is acceptable. Humans fill these gaps using context and shared assumptions. An agent has to decide whether it can safely infer, whether it should retrieve a stored preference, or whether it must ask. Accessibility makes this problem more consequential because the user may not see all options that the agent sees.

The third layer is planning. A good plan is not merely a hidden chain of internal reasoning. From the user’s perspective, the important artifact is an operational outline: which services will be accessed, which actions are reversible, where confirmation will occur and what the completion condition will be. A user does not need to hear every internal token generated by the model. They do need to know, for example, that the agent will compare prices across three stores and stop before purchase.

The fourth layer is execution. Execution can occur through APIs, semantic UI actions, keyboard input, pointer events or generated code. In accessibility-oriented systems, the order matters. APIs and semantic actions are generally preferable because they expose stable targets and machine-readable success states. Keyboard interaction can mirror how a screen-reader user works and preserve compatibility. Pixel-based clicks are useful as a fallback when semantics are missing, but they are fragile because layout can move and the visual target may be ambiguous.

The fifth layer is verification. An execution step is not complete merely because an input event occurred. A save operation should be followed by checking that the file exists. A calendar operation should verify that the event appears with the expected date and participants. A web form should identify a confirmation state. The verification channel should be as independent as practical from the mechanism that proposed the action. If the same model both guesses that a button means “submit” and later guesses that the resulting screen means “success,” the system may compound one hallucination with another.

The sixth layer is user communication. This layer is often neglected because sighted developers can watch a cursor and infer what the agent is doing. Blind users need an explicit representation of trajectory state. They need to know whether the agent is searching, waiting for a page to load, comparing options, blocked by authentication, uncertain about a control or ready for a decision. The communication should be concise by default, more detailed on request and available through screen-reader-compatible status regions rather than visual animation alone.

The seventh layer is recovery. Agents will fail. The question is whether failure leaves the user stranded. A strong system preserves checkpoints, maintains a history of actions, distinguishes reversible from irreversible steps and offers a clear handoff. It may say, “I could not confirm whether the form submitted. I stopped before retrying. Focus is currently on the Submit button, and all entered values remain on the page.” That statement is more useful than a generic error.

Finally, there is governance: permissions, logging, identity, retention, safety boundaries and organizational rules. NIST’s agent-security work and the UC Berkeley Agentic AI Risk-Management profile emphasize that autonomy must exist inside controls, not outside them. In an assistive context, governance should be visible enough that the user understands what the agent can do without making every interaction feel like a compliance ceremony.

This layered model matters because it prevents a common category error. Better language-model reasoning will improve agents, but it will not solve every accessibility problem. A model can be intelligent and still receive a broken accessibility tree. It can interpret the page correctly and still lack permission to act. It can execute successfully and still communicate the result poorly. Assistive agent design is systems engineering.

29. A Failure Taxonomy for Accessible Agents

To build trustworthy systems, failures need names. Treating every unsuccessful workflow as “the AI made a mistake” obscures important differences. Accessible agents can fail at perception, grounding, planning, memory, execution, authentication, verification, explanation, preference handling or termination. Each category requires different mitigation.

29.1 Perception failure

A perception failure occurs when the agent does not correctly observe the environment. The screenshot may omit content outside the viewport. The accessibility tree may contain incorrect labels. A loading overlay may hide the actual page. Text may be embedded in an image. A screen magnifier may alter what is visible. The A11y-CUA findings are instructive because agent performance fell sharply under assistive-technology conditions, showing that a state representation optimized for standard sighted interaction may not survive accessibility transformations.

Mitigation begins with sensor diversity. An agent can combine semantic structure, visual pixels, OCR and application-specific APIs. It should prefer sources that are both stable and inspectable. Conflicts between sources should trigger caution: if a screenshot appears to show a button labeled “Pay now” while the accessibility tree exposes “Continue,” the system should not silently assume they are the same control.

29.2 Grounding failure

Grounding is the process of connecting an instruction to the correct object. An agent may understand the sentence “open the second attachment” but target the second link on the page rather than the second attachment. Grounding errors are dangerous because the overall plan can be correct while the local target is wrong. Semantic identifiers, labels, relationships and region boundaries reduce this risk.

For blind users, grounding explanations should be linguistic. Instead of saying “I clicked the item at coordinates 742, 513,” the system should say “I opened the attachment named invoice-september.pdf in the message from Accounting.” This allows the user to verify the target without sight.

29.3 Planning failure

Planning failures happen when the agent selects an inappropriate sequence even though it understands the interface. It may take a costly route, omit a necessary prerequisite or choose a shortcut that changes meaning. In travel, it might optimize for ticket price while ignoring an overnight airport transfer. In file management, it might rename a folder when the user intended a single file. Plans should therefore encode user constraints and expose consequential assumptions.

29.4 Constraint-tracking failure

Long workflows require the agent to remember constraints across steps. The 2026 OLLA study specifically identified constraint-tracking failures. A user may say, “Find a flight under $500, no overnight layover, one checked bag, and do not book anything.” The agent may respect the price but forget the layover rule after navigating several pages. Structured task state is safer than relying entirely on conversational memory. Critical constraints should be represented explicitly and rechecked before commitment.

29.5 Preference failure

A preference failure is not necessarily a technical error. The agent may complete a valid task but choose an option the user would not have chosen. Morae’s work makes this failure visible: tasks can contain ambiguity that a benchmark marks as success even though the user loses meaningful choice. Preference failure is especially relevant to accessibility because the user may be unable to perceive the option set directly. The agent must therefore distinguish “any valid option” from “a decision that belongs to the user.”

29.6 Execution failure

The agent may target the correct control but the action may not take effect. A click can miss. A keystroke can be swallowed. A browser may block a popup. A page may reject a field. Robust systems inspect the resulting state after each important action instead of assuming inputs succeeded.

29.7 Authentication failure

Authentication often sits at the boundary between automation and human identity. Password managers, one-time codes, biometric prompts and CAPTCHAs can interrupt agents. An accessible system should not attempt to circumvent security. It should pause, explain what authentication is required and hand control to the user through an accessible channel. After authentication, the agent can resume from a stable checkpoint.

29.8 Verification failure

Verification failure occurs when the agent performs an action but cannot establish the outcome. This is common on poorly structured websites that change visually without a machine-readable confirmation. Retrying blindly can duplicate a purchase or submission. Safe behavior is conservative: if the system cannot distinguish failure from success, it should stop and describe the ambiguity.

29.9 Explanation failure

An agent can be operationally correct and still fail accessibility if its explanation is unusable. “Completed task” is insufficient if the user needs to know which option was selected. Conversely, narrating hundreds of low-level events can overwhelm speech users. Explanation quality is therefore a distinct performance dimension. It requires selecting the information necessary for oversight.

29.10 Termination failure

The OLLA study also identified termination problems: agents may continue acting after the goal is satisfied or stop before the goal is actually reached. In accessible systems, termination should be tied to verified success criteria. Once those criteria are met, the agent should stop unless the user explicitly requests a follow-on action.

29.11 Recovery failure

A failed step becomes far more harmful if recovery is poor. An agent may leave focus in an unexpected place, close a document without saving, lose form data or provide no way to undo. Recovery should be designed before deployment, not improvised after errors occur. Workflows should favor reversible primitives, staging and checkpoints.

29.12 Accessibility-feedback failure

This final category is unique to collaborative accessibility. The agent may operate successfully while failing to expose what it is doing to assistive technology. A sighted user can watch. A blind user may hear nothing. Agents should therefore emit structured status events compatible with screen readers and braille, including current step, waiting states, errors and decision requests.

A taxonomy like this changes evaluation. Instead of reporting only a completion percentage, researchers can ask which failure classes dominate and whether those classes impose different costs on blind users. It also encourages better engineering. Improving perception does not fix preference failures; adding confirmations does not fix grounding; better model reasoning does not fix inaccessible status output.

30. Verification Engineering: Making Agent Actions Auditable Without Sight

Verification is the hinge between usefulness and trust. In ordinary software, users often verify outcomes directly. A saved file appears in a folder. A sent message moves to Sent. A transaction shows a confirmation screen. Sight provides a broad and immediate channel for these cues. Nonvisual users can access the same information when interfaces expose it correctly, but agentic systems add another mediation layer. The agent itself may be responsible for reporting the cue.

A strong verification architecture separates three questions: Did the action execute? Did the environment reach the intended state? Can the user understand the evidence? These are different. Pressing a submit button answers only the first. Seeing a confirmation number answers the second. Presenting that number in accessible text answers the third.

Deterministic checks should be preferred whenever possible. If a calendar API returns a newly created event identifier, that is stronger evidence than a model visually interpreting a calendar screen. If a file system reports the exact path and checksum of a saved document, that is stronger than a screenshot of a folder. If a payment service returns an immutable transaction receipt, that is stronger than a language model saying that the page “looks successful.”

Graphical user interfaces still matter because many services expose no public API. In those cases, verification can combine signals. The agent can inspect the DOM for confirmation text, compare the accessibility tree before and after the action, look for a URL change, detect an enabled/disabled state transition, and use vision as an additional cue. Agreement across independent signals increases confidence. Disagreement should decrease it.

Verification should also be consequence-aware. A low-stakes search filter does not need a spoken receipt. A bank transfer does. Designers can define verification levels. Level 0 may require only local state consistency. Level 1 may produce a silent machine check. Level 2 may provide an accessible post-action summary. Level 3 may require the user to review critical fields before action. Level 4 may require both user confirmation and an external acknowledgement after action.

Accessible evidence should be concise but inspectable. A post-action summary might say: “Transfer submitted. From: Checking ending 2841. To: Savings ending 9017. Amount: $200. Fee: $0. Confirmation: 8F4Q2. Date: September 29.” Each element can be navigated individually in a structured region for users who want detail. The summary should not rely on a green checkmark or visually animated success banner.

For file edits, a semantic diff can be more useful than a screenshot. The agent can report: “Three lines changed in settings.json: timeout increased from 30 to 60 seconds; retries changed from 2 to 3; no other keys changed.” For a document, it can list headings modified and provide before/after text. For a webpage, it can report the element that changed and its new state. Verification becomes accessible when change is represented symbolically.

Rollback is a form of verification insurance. The 2026 user-side accessibility-repair work found that unverified generative changes could produce regressions, whereas a verify-and-reject loop dramatically improved reliability in the seeded evaluation. The broader principle applies to agents: speculative action becomes safer when the system can test its effects and discard changes that violate invariants.

Invariants are rules that must remain true throughout a workflow. A file should not be deleted. A draft should not be sent. A total cost should remain below a stated amount. A selected date should remain within the requested week. A website should still expose a valid main heading. Agents can check invariants after each step, turning vague intent into concrete constraints.

Blind users should be able to define personal invariants as well. “Never send an email with an attachment unless you announce the attachment name.” “Never purchase an item if shipping exceeds the item price.” “Never change my screen-reader settings.” “Never disclose my home address unless I approve the recipient.” Such rules convert accessibility and privacy preferences into enforceable policy.

Verification histories also support trust calibration. If the system records when it was uncertain, when a user corrected it and when a step required fallback, users can develop a realistic model of capability. Developers can identify recurring problems. Organizations can audit consequential workflows. The history should be human-readable, searchable and privacy-preserving.

There is a temptation to equate more verification with more safety. That can backfire. If the user must approve every keystroke, the agent recreates manual interaction with extra friction. The objective is not maximum confirmation but maximum assurance per interruption. Verification should be largely automatic for mechanical steps and highly visible around commitments.

This balance is especially important in speech interfaces because interruptions are expensive. A visual agent can display subtle progress indicators in parallel. A screen-reader workflow should avoid constant announcements that interrupt reading. Polite live regions, on-demand status commands and milestone-based updates allow the user to choose when to hear detail.

The final verification principle is humility. Some states cannot be established confidently. A website may be broken. A visual scene may be ambiguous. A third-party system may time out after receiving a request. In those cases, “I cannot verify whether this succeeded” is a successful system behavior. False certainty is the failure.

31. Workflow Case Study I: Researching a Complex Topic

Research is an ideal example of agentic accessibility because it combines high information volume with relatively low transactional risk. A blind researcher can already use accessible browsers, search engines and reference managers, but the workflow often involves repetitive navigation across websites with inconsistent structure. An agent can compress this work without making substantive judgments on behalf of the researcher.

Suppose the goal is to investigate current evidence on autonomous AI tools used by screen-reader users. A conventional workflow might involve searching several databases, opening dozens of results, finding abstracts, locating participant counts, checking publication status, copying citation details and organizing notes. Every source may use a different layout. Cookie dialogs, PDF viewers and dynamic search interfaces add friction.

An accessibility-aligned research agent begins by clarifying scope: peer-reviewed work only, or are accepted papers and preprints acceptable? What date range? Which populations? It then issues searches through structured APIs or accessible web interfaces. The agent should not merely return a synthesized answer. It should preserve provenance. Each extracted claim should be connected to a source record containing title, authors, venue, date, identifier and the specific passage or field that supported the claim.

The user can ask high-level questions such as “Which studies included actual blind participants rather than simulated impairment?” or “Which papers measured task success?” The agent searches the gathered corpus and produces a comparison. This saves extensive navigation while keeping the evidence inspectable. The user can open the underlying paper at any time.

The agent should distinguish publication types. A peer-reviewed CHI paper, an arXiv preprint, a standards document and a vendor blog are not interchangeable. Source classification should be explicit. When a preprint is cited because no final version exists, that status should be visible. When the agent relies on a vendor’s own documentation to describe a product capability, it should not present the claim as independent evidence of effectiveness.

Verification in this workflow focuses on citation integrity. The agent should confirm DOIs, dates and participant counts directly from source metadata or full text rather than inventing them from memory. If an extraction parser produces contradictory numbers, the system should flag the discrepancy. The research agent can also maintain a source ledger showing which documents were fully reviewed, skimmed or used only for discovery.

The human retains interpretive authority. The agent can identify patterns and draft summaries, but the researcher decides what the evidence means. This division of labor is especially attractive for accessibility: the machine absorbs page-level friction while the person retains scholarly judgment.

32. Workflow Case Study II: Email, Scheduling and Administrative Work

Administrative work contains many small tasks that are individually accessible but collectively expensive. Reading a message, locating an attachment, checking a calendar, proposing times, creating an event, filing the thread and adding a reminder can require movement across several applications. Agentic AI can combine these steps into one coherent workflow.

Consider the instruction: “Find the latest email from the conference organizer, summarize what they need from me, check my calendar for two possible times this week, and draft a reply. Do not send it.” The agent must access email, identify the correct sender and thread, extract requests, read calendar availability, propose times and create a draft. The final prohibition—do not send—is a critical constraint.

A well-designed agent represents that constraint structurally rather than trusting conversational memory. The email connector should expose a draft action separately from a send action, and the agent’s current permission set should exclude sending for this workflow. This is a concrete example of least privilege improving both safety and user confidence.

The output should be accessible at the level of the goal: “The organizer is asking for a 30-minute planning call before Friday and needs your speaker bio. You are free Wednesday at 2:00 p.m. and Thursday at 11:30 a.m. I drafted a reply offering those times and attached the bio from your Press folder. Nothing has been sent.” The user can then review the exact draft, recipients and attachment.

Notice what the agent did not need to narrate: every folder navigation, every focus movement or every calendar cell visited. The machine handles mechanics. The user receives the information necessary to verify commitments. If the system found two files named bio.docx, that ambiguity belongs to the user and should trigger a question.

This pattern can generalize to routine administration without becoming a black box. The user might authorize automatic filing of newsletters while requiring confirmation for messages to humans. The agent can summarize calendar conflicts but should not move another person’s meeting without permission. Rules can become increasingly personalized as trust is earned.

33. Workflow Case Study III: Shopping, Travel and Choice-Rich Interfaces

Shopping and travel are more difficult for agents because the main problem is not merely navigation. It is preference. A user can specify price and dates while leaving dozens of meaningful factors unstated. The design challenge is to automate comparison without turning ambiguous criteria into silent decisions.

Suppose a traveler asks, “Find me the cheapest nonstop flight from San Francisco to Manila next month with one checked bag.” The agent can search airlines and aggregators, normalize prices, identify nonstop routes and incorporate baggage fees. But it still encounters questions: Which departure times are acceptable? Is a basic-economy fare acceptable? Are seat-selection fees important? Does the user care about arrival time, aircraft, refundability or airport transfer?

An agent should not ask twenty questions before starting. That would destroy the value of automation. Instead, it can search using known constraints and pause only when the unresolved factors materially affect the top options. It might say: “I found two nonstop options within $18 of each other. Option A is $742, departs at 11:20 p.m., and includes one checked bag. Option B is $760, departs at 4:10 p.m., includes the bag, and is changeable for a fee. Which matters more: lowest price or earlier departure?”

This is the mixed-initiative principle demonstrated by Morae at a larger scale. The agent performs broad mechanical comparison and returns compact decision points. The user remains the source of values.

Accessibility information should be treated as first-class data rather than a generic “accessible” label. For a blind traveler, relevant factors might include airport assistance procedures, transit complexity, hotel navigation, availability of digital keys, elevator labeling, accessible ride-hailing and staff support. An agent can search policy pages and reviews, but it should distinguish official guarantees from anecdotal reports.

The final purchase step should include a structured confirmation of itinerary, names, dates, airports, baggage, total price and cancellation terms. If any value was inferred or remains uncertain, it should be identified. The agent should not proceed merely because the webpage exposes a prominent “Buy” button.

34. Workflow Case Study IV: Software Development and Technical Work

Software development illustrates why agentic AI can be especially useful to blind professionals. Code is fundamentally textual, yet modern development environments surround code with visually dense interfaces: tree views, inline diagnostics, graphical debuggers, dashboards, design mocks, diagrams and web-based collaboration systems. Accessibility varies widely.

A coding agent can operate at the repository level rather than the widget level. The user can ask, “Find why the login form stopped accepting keyboard submission after yesterday’s changes.” The agent can search version history, inspect the form component, run tests, identify a likely regression and prepare a patch. Instead of forcing the user through numerous panels, the agent returns a semantic explanation and a diff.

This workflow should remain reviewable. The agent can state which files it changed, why, which tests it ran and which tests passed. The user can inspect the diff with a screen reader or braille display. If the change affects accessibility semantics, the system can run automated checks but should acknowledge that automated tooling cannot establish full WCAG conformance.

The accessibility benefits go beyond speed. AI can translate visual artifacts into code-level or textual representations. A screenshot of a broken layout can be described. A chart from a monitoring dashboard can be summarized. An architecture diagram can be converted into a list of components and relationships. Command-line errors can be interpreted. These functions reduce dependence on sighted coworkers for incidental information.

Research on visually impaired developers also warns about overload. Coding assistants can generate too many suggestions and force users to move between AI output and source code repeatedly. An agent should therefore support “quiet” modes: work in the background, then present a compact patch summary. For complex changes, it can offer progressively deeper explanations rather than streaming every intermediate thought.

Repository permissions should follow least privilege. The agent may be able to edit a branch but not merge to production. It can create a pull request but not approve it. This separation mirrors good software engineering and makes agentic coding more compatible with professional governance.

35. Workflow Case Study V: Education and Accessible Learning

Education is a domain where accessibility support and intellectual authorship must be carefully distinguished. AI can make inaccessible material usable without doing the student’s academic thinking for them. Agentic systems can fetch readings, restructure documents, describe figures, organize deadlines and navigate learning platforms, while the student remains responsible for analysis and assessed work.

Imagine a course in which weekly readings are distributed as inconsistent PDFs. A student can ask an agent to download the assigned files, identify whether they contain selectable text, run OCR when necessary, reconstruct headings, extract references and create an accessible reading package. The agent can describe charts and images while linking each description to the relevant page. It can also identify passages that appear ambiguous and advise the student to inspect the original or ask the instructor.

The system might then synchronize assignment deadlines with a calendar and summarize instructor announcements. These are administrative supports. If the student asks the agent to explain a concept, that becomes tutoring. If the student asks it to write a graded essay for submission, the task crosses into authorship and should be governed by course policy. Accessibility services and academic-integrity rules need enough nuance to recognize this continuum.

There is a fairness issue if institutions ban AI broadly while distributing inaccessible content that AI can remediate. Students with disabilities should not be forced to choose between equal access and compliance. Institutions can instead define approved accessibility uses, provide secure tools and ensure that accommodations are documented. The goal is not special permission to outsource learning. It is equal access to the material and workflow.

Agentic learning systems can also help users develop screen-reader skill. Rather than always performing a task, the agent can offer step-by-step instructions based on the current application state, as AskEase demonstrates. The student decides whether to learn the interface or delegate the task. This flexibility is valuable because some tools are used daily and worth mastering; others are one-time bureaucratic obstacles.

36. Workflow Case Study VI: Financial and Legal Tasks

Financial and legal workflows demonstrate the upper end of consequence. Agentic assistance can be enormously helpful because forms and account interfaces are often complex, but mistakes can create monetary or legal harm. These tasks require conservative automation.

A useful pattern is prepare, verify, commit. In the prepare phase, the agent gathers information and fills fields that are objectively known: name, address, account number, dates or values from source documents. In the verify phase, it reads back critical information, identifies declarations and highlights discrepancies. In the commit phase, the user explicitly approves submission.

An agent helping with a bank transfer can identify the correct account, prepare the amount and fill the recipient, but it should not click the final transfer button until the user hears a structured confirmation. If the bank later shows a receipt, the agent captures the confirmation number and reports it. If the page times out after submission and no reliable receipt exists, the agent should not immediately retry because duplicate transactions are possible.

Legal forms require even greater care. The agent can improve access to instructions, identify required fields and translate complex layouts into a linear representation. It can point out that a signature certifies certain statements. It should not determine the truth of those statements without user input. The distinction between clerical assistance and substantive representation must remain clear.

This is also a domain where human escalation can be essential. An agent can say, “This section asks whether you are a resident for tax purposes, and the definition is not clear from the materials I have. I recommend that you answer this yourself or seek qualified advice.” The ability to recognize a boundary is a feature, not a weakness.

37. Workflow Case Study VII: Publishing, Creative Work and Media

Creative workflows show another side of accessible agency. The challenge is not simply correctness. It is authorship, aesthetic control and interpretation. A blind creator may use AI to describe artwork, inspect layouts, manage metadata, operate publishing interfaces and compare how content appears across devices. The agent should increase the creator’s reach without becoming the hidden author.

For a long research paper such as this one, an agent can manage citations, generate a table of contents, calculate reading time, create downloadable citation files, validate heading structure and run link checks. These are mechanical publishing tasks. The author remains responsible for the argument, interpretation and claims.

In image-heavy media, the agent can describe compositions and identify likely accessibility problems such as text embedded in images without alternatives. It can propose alt text but should let the creator decide how much description fits the communicative purpose. Alternative text is not merely object recognition; it is editorial judgment about what information matters.

Version control is particularly valuable. An agent can make formatting changes in a staged draft, run checks and show a diff before publication. If the change creates a regression, the previous version remains available. This workflow operationalizes the principle of reversible agency.

Creative agents should also expose uncertainty around subjective judgments. “The text may be difficult to read against the background because contrast appears low” is more appropriate than pretending there is a single aesthetic truth. When objective standards exist, such as contrast thresholds, the agent can measure them directly.

38. Screen Readers and Agents as Complementary Technologies

Predictions that AI will replace screen readers misunderstand what screen readers provide. A screen reader is not simply a voice that describes the screen. It is a direct interaction layer tied to structured application state. It gives users deterministic navigation, immediate access to text, precise control and a fallback independent of model interpretation. Those properties remain valuable even if agents become extremely capable.

Agents are strongest at abstraction, synthesis and delegation. Screen readers are strongest at direct inspection and exact manipulation. Combining them produces a more powerful system than either alone. The user can ask the agent for an overview, then use the screen reader to inspect a specific control. They can navigate independently until reaching an inaccessible widget, then delegate that step. They can let the agent complete repetitive form fields and take over at the certification statement.

This hybrid architecture also improves learning. Users can ask, “How would I do this manually with my screen reader?” The agent can explain shortcuts and navigation patterns based on current context. Over time, the user may choose to automate some workflows and master others. Independence is enhanced because the person controls the division of labor.

Technically, integration requires careful focus management. An agent that clicks around a page can change the active element and disorient the screen reader. The system should announce significant focus changes and, when returning control, place focus at a predictable location. If the agent navigates to a new page, it can summarize the page title and main region before yielding.

Speech contention is another issue. If the agent speaks while the screen reader is already reading, information can overlap. A unified accessibility layer could route agent status through screen-reader APIs or accessible live regions, allowing established speech and braille settings to control output. Users should not have to manage two competing voices.

Braille integration deserves special attention. Braille can present exact spelling, punctuation, code and structured data with less temporal pressure than speech. Agent summaries, diffs and confirmation fields can be exposed in braille-friendly formats. Research should include braille users rather than assuming auditory output represents all blind interaction.

Magnification users require another approach. An agent may need to respect the magnified viewport and avoid moving focus unexpectedly. It can also provide spoken summaries while preserving the user’s visual context. A11y-CUA’s performance decline under magnification shows that this area cannot be treated as an afterthought.

The future screen reader may therefore become more agent-aware without becoming an agent itself. It could expose a standardized semantic channel for agent actions, status and confirmations. Agents, in turn, could use screen-reader infrastructure as a trusted representation of the user interface. This would turn accessibility APIs into a bridge between human and machine interaction.

39. Multimodal Agents in the Physical World

Agentic AI becomes more difficult when it leaves the predictable structure of software and enters the physical world. A webpage has discrete controls. A kitchen, street or hotel lobby contains continuous space, moving people, ambiguous objects and safety-critical uncertainty. Multimodal models are increasingly capable of describing these environments, but current evidence does not support treating them as autonomous substitutes for established mobility and safety techniques.

The 2026 visual-assistance diary study found that MLLM systems can provide useful conversational access while still producing a meaningful rate of incorrect answers. The 2025 ACL evaluation found weaknesses in cultural context, multilingual interpretation, braille and assistive-object recognition. These limitations become more consequential when the user is moving through space or handling objects.

A physically situated assistive agent therefore needs a stronger uncertainty policy. It should distinguish static informational tasks from safety-critical guidance. Reading a restaurant menu from a photograph is different from telling a person when to cross a street. Describing the apparent location of a door is different from certifying that a path is free of hazards.

Camera framing is itself an accessibility problem. A blind user may not know whether the relevant object is inside the frame, in focus or obscured. Agents can help by providing directional framing guidance: “Move the phone slightly left; the document is partly cut off,” or “I can see the control panel but not the bottom row.” This is a form of interactive perception rather than one-shot captioning.

Temporal consistency also matters. If the user asks a follow-up question, the agent should know whether the camera has moved. It should not combine observations from different frames as though the scene were unchanged. A trustworthy system can timestamp observations and say when its description may be stale.

Physical agents may eventually control robots, appliances or smart-home devices. Here the advantage of structured APIs reappears. It is safer to ask a smart oven for its current temperature through an authenticated interface than to infer the display visually. Multimodal perception remains valuable for devices with no digital integration, but it should be treated as an uncertain sensor.

The physical-world opportunity is still substantial. AI can reduce reliance on sighted assistance for reading labels, sorting objects, understanding photographs and inspecting unfamiliar environments. The design objective should be to expand information and choice without encouraging users to replace proven safety practices with uncertain perception.

40. Agentic Programming and Disabled People as Technology Authors

One of the most radical implications of agentic AI is that accessibility users may increasingly be able to create their own software. Traditional assistive-technology development requires programming skill, platform knowledge, testing resources and distribution infrastructure. Those barriers mean that many highly specific needs never become commercial products.

Agentic programming changes the economics of prototyping. A user can describe a desired tool in natural language, ask an agent to build a small application, test it and revise it through conversation. ProgramAT’s 2026 preprint offers early evidence: five technically experienced blind and low-vision co-designers created 37 camera-based prototypes across 68 iterations over two months. The tools were often highly personal, addressing situations that mainstream assistive applications did not cover.

The significance is not that every blind person will become a software engineer. It is that the distance between a need and a working prototype may shrink. A teacher could build a quick tool that reads a specific piece of laboratory equipment. A musician could create a converter for an inaccessible metadata workflow. A traveler could create a small app that reorganizes a particular airline’s itinerary emails. Community organizations could share templates.

User-authored assistive technology also creates new risks. Generated code can contain security flaws. Camera apps may upload sensitive images. A tool that works for one person may fail for another. Distribution creates maintenance obligations. Agentic development environments should therefore include accessible testing, permissions review, dependency scanning and clear warnings when a prototype is not ready for safety-critical use.

There is a parallel with spreadsheets and macros. Millions of people automate work without considering themselves programmers. Agentic systems could make accessibility scripting similarly ordinary. Screen readers already support scripts and add-ons created by expert users. Natural-language agents may open that tradition to a broader population.

This direction also changes accessibility research. Instead of asking only what tools disabled users need, researchers can study what users build when given flexible creation systems. Those artifacts may reveal priorities that commercial markets have ignored. The research agenda shifts from designing for users toward designing with and by users.

41. Organizational Deployment: Accessibility, Procurement and Governance

Individual users can adopt consumer AI quickly. Organizations move more slowly because they must consider security, privacy, compliance, integration and support. This can create a new accessibility gap. Sighted employees may use unofficial tools informally while disabled employees need formal permission to use AI as an accommodation.

Organizations should therefore evaluate agentic accessibility explicitly. Procurement checklists can ask whether the AI interface itself meets accessibility standards, whether agent status is exposed to assistive technology, whether keyboard-only operation is supported, whether confirmation dialogs are accessible, whether logs can be reviewed nonvisually and whether enterprise policies permit approved assistive use cases.

Security review should distinguish between model access and agent permissions. A text assistant that can summarize pasted material has a different risk profile from an agent with email, calendar and file-system access. Permissions should be scoped by role and task. An employee using an agent to read inaccessible dashboards may not need authorization to modify those dashboards.

Organizations also need a process for exceptions. A blanket prohibition on generative AI may unintentionally remove an effective accessibility accommodation. Conversely, unrestricted use of external AI services may expose confidential information. A mature policy can provide approved tools, local processing for sensitive content, data-classification rules and an accommodation pathway.

Training should include disability perspectives. Employees and managers need to understand that AI can be an assistive technology, not merely a productivity shortcut. At the same time, users should receive realistic guidance about hallucination and agent risk. Trust should be calibrated, not promoted through hype.

Procurement can influence the broader market. When governments, universities and large employers require both conventional accessibility and accessible agent controls, vendors have an incentive to build these features into mainstream platforms. This is more scalable than relying on individual users to repair every product after purchase.

Auditability is particularly important in enterprise settings. The system should be able to distinguish actions initiated by the human from actions executed by the agent, preserving accountability without blaming users for opaque model behavior. NIST’s emerging identity and authorization work points toward explicit non-human identities and delegated permissions. Accessible interfaces for these controls should be part of implementation.

Organizations should also measure outcomes. Did the agent reduce time spent on inaccessible workflows? Did disabled employees report greater independence? Were errors easy to detect and reverse? Did support requests decline? A deployment that technically “includes AI” but increases verification workload has not succeeded.

42. Economic Inclusion, Market Structure and the Cost of Agentic Access

Accessibility technology has always been shaped by economics. Specialized hardware and software can be expensive because markets are small and development costs are high. Mainstream integration—such as built-in screen readers on smartphones and operating systems—has dramatically expanded access. Agentic AI could follow either path: it could become a broadly available layer of computing, or it could become a premium accessibility advantage available primarily to those who can afford top-tier models and services.

The OECD’s analysis of AI and disability employment emphasizes that AI can reduce costs and embed accessibility into mainstream tools, but it also notes funding, commercialization and adoption barriers. Agentic AI intensifies these questions because high-capability systems consume significant compute and often require subscriptions. The most useful accessibility features may therefore be tied to service tiers.

This creates a policy challenge even without entering partisan politics. If advanced agentic access becomes necessary to interact efficiently with essential digital services, should it be treated like optional productivity software or like assistive technology? Employers may have obligations to provide accommodations. Schools may need to supply approved tools. Public libraries and disability-service programs may become access points. Standards and procurement can encourage baseline functionality in widely available products.

Competition matters as well. Users should not have to choose between accessibility and platform lock-in. Portable preference formats, interoperable agent protocols and exportable action histories could reduce dependence on one provider. If a user has spent years teaching an agent preferred verbosity, confirmation boundaries and accessibility conventions, those settings should ideally be transferable.

Open-source models and local agents can improve autonomy, but they are not automatically accessible. Setup complexity can be a barrier, and local models may require expensive hardware. Community projects need accessible installers, documentation and interfaces. Public investment in open accessibility infrastructure could have outsized benefits because it supports both specialized and mainstream developers.

There is also a labor-market distribution question. AI may increase productivity for disabled workers, but productivity gains do not automatically translate into opportunity. Employers could use automation to reduce headcount, raise performance expectations or intensify monitoring. Disability inclusion requires institutional choices about training, accommodation and job design. Technology can remove barriers but cannot substitute for fair employment practices.

The most hopeful economic scenario is one in which accessibility features developed for disabled users become mainstream capabilities. Voice control, text-to-speech and predictive text have followed this pattern before. Agentic systems that summarize complex interfaces, automate repetitive workflows and expose semantic state can benefit everyone. Broad usefulness can support the investment needed for high-quality accessibility, provided disabled users remain involved in design rather than becoming incidental beneficiaries.

43. Part II Synthesis: From Automation to Accessible Delegation

The engineering question running through Part II can be stated simply: how can a blind user delegate work without delegating away awareness, preference and authority?

The answer is not a single model capability. It is an architecture. The agent needs multiple perception channels, explicit task state, structured constraints, consequence-sensitive confirmation, independent verification, accessible progress communication, stable interruption, reversible actions and a compatible screen-reader handoff. It needs permission boundaries that limit what it can do and action histories that explain what it has done. It needs the humility to stop when evidence is insufficient.

Real-world workflows demonstrate why these components matter. Research benefits from source provenance. Email benefits from separating draft from send. Shopping and travel benefit from ambiguity detection. Financial and legal tasks benefit from prepare-verify-commit patterns. Coding benefits from diffs, tests and branch protections. Education benefits from separating access support from authorship. Creative work benefits from reversibility and explicit treatment of subjective judgment.

These are not isolated accessibility tricks. They are general principles of good agent design made more visible by disability. Blind users expose the weaknesses of systems that assume visual monitoring. If an agent can be safely supervised without sight, its state is probably clearer for everyone. If its actions can be verified semantically, the system is more robust. If it preserves choice rather than silently optimizing, it respects human agency broadly.

Part III turns from operational design to long-term questions: security models for high-autonomy agents, autonomy tiers, trust calibration, cognitive load, standards, research methodology, global deployment and a detailed agenda for future accessible-agent research.