glass 1.1 · open-core · Apache-2.0

Your agent closes the loop.

An MCP server that gives an AI coding agent a closed build → see → interact → debug loop over native GUI apps — so it can build and debug UI itself instead of asking “does this look right?”.

Get started View on GitHub ↗
agent · glassstdio
  1. glass_start { run:["python3","tasks_demo.py"], a11y:true } → ok · 360×420 · 0.4s
  2. glass_a11y_snapshot → #4 TextArea · #5 Button "Add" · #7 List (empty)
  3. glass_click_element #4 (New task) → ok · field focused
  4. glass_type "Buy milk" → ok
  5. glass_click_element #5 (Add) → #7 List still (empty) — Add did nothing
  6. glass_stop → session ended
  7. edit tasks_demo.py + add.connect("clicked", self.on_add)
  8. glass_start { … a11y:true } → ok · a fresh, empty app
  9. glass_click_element #4 (New task) → ok · field focused
  10. glass_type "Buy milk" → ok
  11. glass_click_element #5 (Add) → #9 Label "Buy milk" — the same click now lands
Tasks (GTK4)360×420 · x11
The Tasks app launched — a text field, an Add button, and an empty list
The accessibility tree, drawn over the app: a field #4, an Add button #5, an empty list
Selecting the text field #4 before typing
Typing a task into field #4
Clicking element #5 (Add) — the list is still empty, the bug, read from the tree
The buggy app stopped — now fix the code
The one-line fix inserted into the source: the Add button was never wired to its handler
Re-launched: a fresh, empty app
Selecting the text field #4 again before typing
Typing the task again into field #4
After the fix, clicking #5 adds the task — it appears in the list; the app works
4
5
examples/tasks-demo/tasks_demo.py
  1. self.entry.set_hexpand(True)
  2. add = Gtk.Button(label="Add")
  3. +add.connect("clicked", self.on_add)
  4. input_row.append(self.entry)
  5. input_row.append(add)
  6. box.append(input_row)
A real glass session — an agent debugging a GTK app via its accessibility tree, replayed.

The loop

  1. 01 build glass launches your app (sandboxed by default) and captures its logs.
  2. 02 see Read the accessibility tree, screenshot the window, or diff regions — as text, so an agent checks a change without spending vision tokens.
  3. 03 interact Inject mouse, keyboard, scroll, and drag at window-relative coordinates.
  4. 04 debug Read the app's stdout/stderr and wait for elements, logs, or regions to settle. 
glass_start           { "run": ["python3", "app.py"], "a11y": true }   // launch + private a11y bus
glass_a11y_snapshot                        // the tree: role, name, #id, bounds — as text
glass_click_element   { "id": 5 }          // click by #id, not pixels
glass_wait_for_element { "name": "Save", "condition": "enabled" }       // wait on state — no polling
glass_set_value       { "id": 4, "value": "hello" }   // set a field / toggle / dropdown
glass_logs                                 // read the app's stderr

Or drive by pixels — glass_screenshot, glass_click{x,y}, glass_diff — for a canvas or custom-rendered app with no accessibility tree.

Why it matters

Any app, any toolkit

glass drives apps as an external black box, so it works with any native GUI app regardless of toolkit or language — no SDK, no app integration. Drive it by pixels, or by its accessibility tree where one is exposed.

Cross-platform

Linux (X11 and Wayland), Windows, macOS, Android (an AVD emulator), and iOS (the Simulator) backends, behind a platform-agnostic core.

Cheap to verify

Text-only diffs and waits let an agent check between screenshots without spending vision tokens on every step.

Sandboxed by default

Launched apps are contained by default; the default level fails closed rather than running an app unconfined.

Quickstart

Download the binary, verify the environment, then register it with your agent.

# 1. Download the latest release for your platform and extract it
#    → github.com/fixed-width/glass/releases/latest
tar xzf glass-mcp-*-x86_64-linux-gnu.tar.gz
cd glass-mcp-*-x86_64-linux-gnu

# 2. Verify the environment (prints a remedy for any gap)
./glass-mcp doctor

# 3. Register it with your agent (stdio)
claude mcp add glass --scope user -- /absolute/path/to/glass-mcp

Drive it well: install the open glass-drive Agent Skill ↗ — npx skills add fixed-width/skills -s glass-drive — so your agent arrives knowing the verify-cheaply-then-look loop instead of spending its first turns rediscovering it. Optional, but the single highest-leverage thing you can add.

First drive: grab the bundled examples/tasks-demo/tasks_demo.py ↗ (a tiny app with a bug), then ask your agent: “Use glass to run examples/tasks-demo/tasks_demo.py with accessibility on. Clicking Add doesn't add the task — reproduce it from the accessibility tree, fix the code, and verify.” Watch it close the loop.

On macOS? Install the notarized .dmg ↗ and double-click it instead. Prefer to build from source ↗? It's a single cargo build. Full setup, sandboxing, and network/HTTP detail are in the docs ↗.

Install glass ↗ Read the docs ↗