TL;DR: As CTO of Nystag Healthcare XR I led the development of a clinical eye-tracking diagnostics application built on Augmented Reality and the Vive Focus 3 headset. It is one thing to build XR that looks good in a demo, and a very different thing to build XR a clinician will trust to read something off a patient's eyes. This is the honest account of what that gap looks like: why a headset is a genuinely good instrument for eye-tracking, why clinical-grade means "boring and repeatable" long before it means "impressive," and the engineering discipline that carries straight over into the AI agents I build now.
Most XR write-ups are about spectacle. This one is about the opposite. In a diagnostic tool the wow factor is a liability, because the moment the experience draws attention to itself it stops being a measurement instrument and becomes a toy. The whole job was to make the headset disappear and let a clean, repeatable signal come out the other side.
Why put an eye-tracking diagnostic in a headset at all
Eye movement is one of the most information-dense signals the body produces. The way the eyes hold a target, drift, jerk back, or fail to track a moving point tells a clinician a great deal, which is why nystagmus and gaze testing are established parts of a vestibular and neurological workup. The classic version of this test involves a technician, a light bar or a wall, and a fair amount of manual observation. It works, but it is hard to standardize, hard to record cleanly, and it varies with the room and the person running it.
A modern standalone headset like the Vive Focus 3 changes the economics of that test in a few concrete ways:
- A controlled visual world. Inside the headset the stimulus is exactly the same every time: same target, same trajectory, same contrast, same speed, with no stray light from a window or a badly lit clinic room. Standardization is half the battle in any diagnostic, and a headset gives it to you for free.
- Integrated eye tracking. The hardware already watches the eyes at a useful rate, so the same device that shows the stimulus also records the response. There is no separate camera rig to align.
- Portability. A standalone headset is a clinic-in-a-bag compared to a fixed lab setup. The test can move to where the patient is instead of the other way round.
That combination, a perfectly repeatable stimulus and a synchronized recording of the eye's response, is the real reason to build this in XR. Not immersion. Measurement.
Clinical-grade means boring, repeatable and honest
The hardest adjustment moving into healthcare XR is that almost every instinct a consumer app trains into you is wrong. In a game you smooth things, you interpolate, you hide jitter, you add a little juice to make motion feel good. In a diagnostic you must do none of that, because every one of those tricks is you editing the very signal you are trying to measure.
I ran the build around three rules.
The stimulus is the specification
If the test says a target moves at a fixed speed across a fixed arc, then it has to move at exactly that speed on every run, on every headset, regardless of frame rate hiccups or thermal throttling. A stimulus that drifts because the render loop stuttered is a corrupted test, even if no human eye could see the stutter. That meant driving stimulus motion off a stable time base rather than raw frame timing, and treating any deviation as a fault to surface, not a cosmetic issue to smooth over.
Raw data first, pretty pictures second
The temptation in eye tracking is to filter the signal until the on-screen trace looks clean. Clean is exactly what you do not want at the recording layer. I kept the captured gaze data as close to raw as the hardware allowed, timestamped and unsmoothed, and did any presentation filtering in a separate, clearly labelled layer downstream. The rule was simple: never throw away signal you cannot get back. A clinician can always smooth a raw trace later; nobody can un-smooth a filtered one.
Design for being wrong
Eye tracking loses the eyes. A blink, a headset shift, a patient with glasses or heavy lashes, and for a stretch of milliseconds the tracker simply does not know where the eye is. The wrong response is to guess and paper over the gap. The right one is to record honestly that tracking was lost for that window and show it. A trace that admits "no data here" is far safer than one that invents a plausible line through the missing stretch, because a confident fabrication is exactly the kind of thing that leads to a wrong read. I designed the capture to mark its own uncertainty rather than hide it.
That last rule is the same one I learned building AR for firefighters at ARCortex: a system that signals its own uncertainty is safer than one that is always confident. It shows up in public-safety AR and in clinical XR for the same reason. When a real decision hangs off your output, honesty about what you do not know is a feature, not a weakness.
The engineering shape of it
Under the clinical framing, a lot of this is disciplined systems work, and it rhymes with the cross-platform and real-time problems I have written about elsewhere.
A strict time budget, like any real-time system. The stimulus render and the gaze capture have to stay locked to a shared, reliable clock so the response can be aligned to the exact moment of the stimulus that caused it. If the timestamps drift apart, the whole recording is meaningless. This is the same latency-as-a-property discipline that AR for a moving operator demands, just pointed at measurement instead of overlay.
Separate the capture core from the presentation. I kept the part that runs the protocol and records the data cleanly separated from the part that draws things for the operator, the same share-the-logic-never-the-platform boundary I leaned on building the Repocket cross-platform SDK at Geonode. The protocol should not care how a chart is drawn, and the chart should never be able to corrupt the protocol.
Make the operator's path foolproof. A clinician is not going to debug your app between patients. The flow to start a test, seat the headset, run the protocol and get a result had to be short, hard to do wrong, and obvious when something was off, such as poor tracking quality before you have wasted a patient's time on a bad recording. Getting that operator experience right mattered as much as the signal processing.
Handle the hardware honestly. A standalone headset throttles when it gets warm, batteries sag, and eye-tracking calibration drifts per person. None of that can be allowed to silently degrade a measurement. Where a consumer app would quietly drop quality, a diagnostic has to notice and tell someone.
Why I do not overclaim what it is
I want to be precise about scope, because healthcare is exactly the wrong field to inflate. What I led was the engineering of a clinical eye-tracking diagnostic application on the Vive Focus 3: the stimulus protocols, the synchronized capture, the operator flow and the data pipeline. Turning captured eye-movement data into a validated clinical determination is a regulated, evidence-heavy process that sits with clinicians and the regulatory pathway, not with a headset app on its own. My job was to make the instrument trustworthy: repeatable stimulus, honest capture, clean data out. That boundary, build the measurement well and be clear about where measurement ends and diagnosis begins, is the responsible way to work in this space.
What clinical XR taught me that I use in AI agents now
I have moved a lot of my work into AI voice agents, and people are sometimes surprised that a clinical XR project informs that at all. It informs almost all of it.
The through-line is human-in-the-loop by design. In the diagnostic, the headset measures and a clinician decides. In the AI voice agents I build for clinics and service businesses, the agent handles the routine and hands off the moment it hits anything that needs a human judgement, exactly the way I wired the emergency line in the vet-clinic agents at Fortell AI. Same shape: automate the repeatable part flawlessly, know precisely where your competence ends, and make the handoff clean.
The other carryover is honesty about uncertainty. A gaze trace that marks its own dropouts and a voice agent that says "let me get a human" when it is out of its depth are the same design instinct. Systems that admit what they do not know are the ones people end up trusting, whether the person on the other side is a clinician reading eyes or a business owner watching their phones get answered.
If you are building something a professional has to trust
The lesson from Nystag that I would hand to anyone building a serious instrument, XR or AI, is this: make it boring on purpose. Repeatable beats impressive. Raw and honest beats smoothed and pretty. And the clearest thing your system can do is know exactly where its job ends and a human's begins.
I have led clinical XR as a CTO and now build production AI systems and voice agents for teams in the US, UK and Europe. If you are building something that a professional has to be able to trust, from a measurement instrument to an AI agent with real stakes, book a call and let's talk about doing it properly.