Case Study
Proactive UI for AI Glasses
A UX study in when an assistant should speak, and when it should stay quiet
A city street seen through a near-future pair of AI glasses, wired into a deterministic decision engine that chooses what to show a person as they walk, how much, and when to stay quiet. Content shrinks as they speed up, yields when they check for traffic and saves itself for the red light. Every decision is logged with its reason, replayable in an inspector and scored by an eval suite.
Overview
An assistant on a screen waits to be asked. Glasses have to decide when to speak at all, to someone who is walking, watching for traffic and can't afford to read. This project is a study of that decision. One looping city block renders in real time in Three.js. The wearer walks it on rails and looks where they like, with a viewfinder standing in for eye tracking. Ten times a second a rules engine reads a snapshot of the walk (pace, gaze and dwell, the crossing and its signal, nearby cars and people, messages and the goal) and returns decisions, each with a level, a timing, a placement and a reason. Gates beat everything: in a crossing the glass steps back and the safety text shows. A settled view lights the labels of the places in sight, a held look grows one into its card and then its expanded view, and turning away plays it back down a level at a time. The brain is a rules policy, not a model: no LLM, no keys, no server, and a seeded simulation, so the same inputs make the same decisions every run. That is what keeps the proof exact. The inspector replays every decision with its reason, and the eval suite runs two recorded walks and 41 focused cases through the engine headless, scoring them against hand-written expectations. Claude Code on Opus built it from a written spec across 31 gated phases. In the final rounds a second Claude Code session reviewed each phase and reran its checks, and I played each build before the next one was written.
Key Features
- 01Thirteen walking rules: the clear center of view grows with pace, unread labels fold themselves in 2.5 seconds at a walk, and a brisk pace gets one captioned line in place of a pane
- 02Deterministic decision engine: a 10 Hz tick in a fixed order of pace band, gates, scored candidates and the attention ladder, with no Math.random and no wall clock
- 03Attention climbs with the look: a held look grows a label to its card in half a second and to its expanded view two seconds later, pinned where the look landed. Turn away and it plays back down
- 04The world has right of way: crossings and nearby vehicles step the glass back, content yields to a head check at the curb, and a pane over a person or car dims and moves aside. A face is never covered
- 05The pocket and the budget: a pinch while walking opens nothing, it saves the place for the next red light. Seven proactive shows per walk, earned back by 40 seconds of quiet
- 06An inspector as proof: every decision with its reason, a scrubbable timeline, live signals, overrides, the eval report and an export that replays through a fresh engine and must match exactly
- 07Evals as the real test suite: 44 suites and over 1,000 checks, with invariants on every run. Every crossing decision is a hold, no reason cites the wearer's body or habits, and two runs produce identical logs
- 08Measured in a real browser: scripted laps aim with real drags and judge every landing for jumps, every one of fourteen doors is walked and chosen, and thirteen screens from a 390 px phone to 4K are rendered