TL;DR: Putting live data into augmented reality is easy to demo and hard to justify. At ARCortex I built two experimental AR viewers alongside the public-safety work: a CCXT Crypto Viewer that pulled market data through the open-source CCXT library and rendered it in AR, and an AR NFT Viewer that placed digital collectibles into a real room. They taught me the question that matters before any AR data product: does the third dimension carry information, or is it decoration? Beyond that, the engineering lessons were consistent. Text is the hardest thing to render well in AR, live data needs a normalization layer and an honest staleness signal, the network must never touch the frame loop, and the asset pipeline decides whether a viewer feels magical or broken. Here is what I learned, and when I would still say yes to an AR data project.
Most of my time at ARCortex went into the ERIS XR platform for the fire service and the geospatial tools around it, AR Planes, AR Properties and MQ-9 Reaper tracking on AR maps, which I wrote about in the coordinate problem behind geospatial AR. Next to that serious work sat two smaller, more exploratory builds: the crypto viewer and the NFT viewer.
They were the kind of project a company builds to answer a question: what does AR add to data people already look at on a screen all day?
The first question: does depth carry information?
Every AR data idea should survive one test before anyone opens Unity. Remove the camera feed and the spatial placement, and render the same data on a flat phone screen. What did you lose?
For the fire service the answer was obvious. The position of a hazard relative to the person looking at it is the data. Flatten it and you lose the thing that makes the overlay useful. Same for an aircraft labelled in the sky: the label only matters because it sits on top of the real plane.
For market data, the honest answer is that you lose very little. A price is one number. A candlestick chart is a two-dimensional object, and it was designed for a flat surface. Wrapping it in 3D, extruding bars, floating tickers around the room, adds spectacle but mostly subtracts readability. That was the most useful thing the crypto viewer taught me.
The NFT viewer landed differently. There the object itself is the point. Seeing a piece of digital art at real scale on your actual wall, from different angles, is something a flat screen genuinely cannot give you. The spatial placement is not decoration, it is the experience.
That gives a simple split I still use:
- Depth is the data: spatial relations, real-world positions, objects at true scale. AR earns its place.
- Depth is the frame: numbers, tables, charts, dashboards. AR is usually a worse screen, and you need a very specific reason to use it anyway.
Text is the hardest thing you will render in AR
Coming from game development, I assumed the hard part would be the 3D. It was the text. A trading view is mostly text and numbers, and text in AR fights you on every axis:
- You do not control the background. A white label that reads perfectly against a dark office wall disappears against a window. Every piece of text needs its own backing panel or outline, because the scene behind it will change.
- Distance changes legibility constantly. Text anchored in world space shrinks as the user steps back. Either you scale it with distance, which breaks the illusion that it belongs to the world, or you accept that it becomes unreadable, which breaks the product.
- Viewing angle matters. A panel read from the side is useless, so labels usually need to face the camera. Face it too aggressively and the scene feels like a flat UI glued to your lens.
The pragmatic answer was a split. Anything the user needs to read precisely, a price or a percentage, belongs in screen space or on a camera-facing panel with a solid background. Anything the user needs to perceive, a direction of movement, a relative size, belongs in the world. Trying to make precise numbers live in world space is where AR data viewers go to die.
Live data needs a normalization layer and an honest clock
The reason the crypto viewer used CCXT is the reason every live data product needs a layer like it. Exchanges disagree about almost everything: how a trading pair is named, how many decimal places a price has, how often you may call them before you get rate limited, and what shape the response takes. CCXT exists to flatten much of that into one consistent interface, which meant the viewer could treat several sources as one kind of thing.
What a normalization layer cannot flatten is time. Every value on screen was true at some moment, and that moment is not now. On a flat trading screen people have learned to read that implicitly. In AR, where a floating number looks like a physical fact, a stale value is more misleading than it would be in a browser tab.
So I treated staleness as part of the data:
- Every value carries its timestamp, and the renderer knows how old it is.
- Stale values look stale. Dimmed, or visibly marked, rather than rendered with the same crispness as a fresh one.
- Failures degrade quietly. If one source stops responding, its values age visibly instead of freezing at a confident last number.
It is the same principle I took from geospatial AR, where a crisp marker for an old position is worse than no marker at all. Never render confidence you do not have.
Keep the network out of the frame loop
This is the Unity lesson that applies to every live-data AR app, and it is invisible until it ruins everything.
AR is unforgiving about frame timing. A dropped frame makes virtual content visibly swim against the room, and people feel it physically.
Live data invites exactly the mistakes that cause hitches: waiting on a network call on the main thread, parsing a large response in one go, allocating fresh objects on every update and letting the garbage collector pause the frame at the worst moment. None show up in the editor. All show up on a mid-range phone.
The rule was strict separation. Fetching and parsing happen off the render path. The render loop reads the most recent snapshot, never waits for one. Data arrives when it arrives, a few times a second at most, and the visuals interpolate between snapshots so the scene keeps running at full frame rate regardless of what the network is doing. Data rate and frame rate are two different clocks, and the moment you couple them the whole experience inherits the worst latency on the network.
For the NFT viewer, the asset pipeline was the product
The NFT viewer had the opposite profile. The data was simple, one object at a time, and the difficulty moved entirely into the assets.
Collectibles come in whatever format the creator chose: still images, animated images, video, sometimes 3D models, at wildly varying sizes and often hosted on slow, decentralized storage. An AR viewer has to take all of that and present it as if it were a physical object that has always been there.
What mattered most:
- Show something immediately. Place a frame or a low-resolution preview at once and swap in the full asset when it arrives. A blank surface while a large file downloads reads as a broken app.
- Respect mobile memory. A single very large texture can push a phone into trouble. Downscaling to what the device can actually display is not a compromise, it is the only way the app survives a collector with a big wallet.
- Real scale is a decision. Digital art usually has no physical size. Picking a sensible default and letting the user resize it was more important than any rendering effect.
- Anchor believably. Detect a surface, place the piece on it, and make it stay there. A framed piece of art that drifts a few centimetres as you walk around it destroys the illusion faster than low resolution ever will.
The build and release discipline behind it is the same one I described for Unity CI/CD pipelines.
Novelty decays, utility compounds
Both viewers impressed people the first time they saw them. That is the trap with AR data products. The first session is a wow. The question that decides whether it is a product is what the user does on the tenth day.
For the crypto viewer, by the tenth day most users would reasonably go back to a flat chart, because that is the better tool for reading prices. For the NFT viewer, the use case was repeatable in a narrower way: showing a piece to someone else, seeing how it would look in a space. Smaller, but real.
That is the lens I use now whenever someone asks me about AR for data, and the same one I apply when choosing an XR headset for an enterprise deployment:
- Does depth carry information? If not, build a good 2D screen first.
- Is the precise part readable? If the key values live in world space, rethink it.
- Is staleness visible? If an old value looks like a fresh one, fix that before adding features.
- Is the frame loop isolated from the network? Test on a mid-range phone, not the editor.
- What does day ten look like? If the answer is "they show a friend once", it is a demo, and that is fine as long as everyone knows it.
What carried over into AI work
I moved from Unity and AR into building AI voice agents and my own products, and these two small viewers left more behind than their size suggests.
The normalization layer is everywhere in agent work. Between a voice agent and a client's CRM there is always a layer, usually n8n in my case, whose job is to make messy systems look consistent, exactly like CCXT did for exchanges. I wrote about that layer in connecting Retell AI to n8n.
The medium has to earn its place. "Should this be AR?" and "should this be a voice agent or a chat agent?" are the same question: where does the information actually live, and which channel carries it without loss? That is the whole argument of chat agent vs voice agent.
And staleness is honesty. An AR label showing an old price as fact and an agent confidently quoting a price list from last quarter are the same bug. The fix is also the same: know how old your information is, and let it show.
I have built AR and XR systems for public safety, defense and healthcare, including clinical eye-tracking diagnostics, and now build production AI agents and products for startups across the US, UK and Europe. If you are weighing whether a data product belongs in AR, on a screen, or in a conversation, more about my background is here, or book a call.