With iOS 11, ARKit has actually put augmented reality in people’s hands. And as a leader, with my team actively developing projects in it, it was important I understood how ideas come to life in the framework.
So I took some time in between meetings and on a Sunday to try and make something with ARKit. Here’s a little travelogue with some broad thoughts on AR, products, and using experiments to gather data.
My first thought was, “OK, where to start?” AR is a broad set of technologies, and attempting to learn all of them without a concise goal in mind is daunting.
What I knew was that ARKit existed. What we needed to figure out was what the product was going to be, what was something I could learn a thing or two from. We’re seeing a lot of fun, novel stuff right now, but it feels like a framework that will become very practical very quickly.
I figured I’d recreate the part in Beetlejuice with the waiting room to the afterlife. Because with AR, it’s all very flashy and wonderful and famous, but there’s quite a bit of tedium. And in Beetlejuice, the afterlife winds up being pretty boring. Like the cliche, as a developer, my job mostly winds up being mostly staring at text.
Additionally, AR has the ability to make your current environment a piece of fantasy. And who doesn’t love Beetlejuice? Halloween’s coming up. So what better way to do that than lean on the themes of the one and only Tim Burton?
What is a product?
Products narrow and focus and sharpen a particular slice of a technology stack into a tangible point that can be measured against. Ideally, at each fidelity level, a product prototype creates a notch from which you can measure future progress.
One of the things we do well here at Uncorked is visualizing that end point, thinking about the product, and using that vision to make a hypothesis about what some incremental data could be gained.
And that notion of a product can be really simple; it just matters that some kind of measurement comes off it.
Your goal cannot be greatness the first time you measure. You have to underperform, get some data, and try it again, or you’ll never progress.
Digging into ARKit
Most of this is super boring. But understanding any language starts with understanding the low-level mechanics of what the framework offers. With ARKit, that’s plane detection. That’s how the system tells me the state of what’s going on. At first glance, it seemed like it was going to be a lot of hacks and patchwork to get something that was presentable, because I hadn’t acclimated to the pattern it was providing. A big part of learning a new framework is learning how to not fight the framework.
I learned resource constraints from device to device have a big effect on the process. The iPhone 7 has a bigger screen and is a more powerful phone than the SE, which I was working on, but the experience was more pleasant on the SE. The resource constraints meant it was pushing fewer pixels, it was becoming more certain about planes before it would notify me. It would notify me about fewer planes, so I had to contend with less data.
You actually probably want the multi-plane, but since I’m only just starting to know the framework I benefitted from it having a built-in governor in what I was having to mentally deal with. Using a dumber device was like having training wheels, and helped me get over my fears of the broader architecture in a bigger effort, and let me pull back to focus on the idea I had. My anxiety about solving for “8 million planes detected” disappeared, because it would only give me one.
After that, it was “I don’t know anything about SceneKit,” which is Apple’s 3D modeling environment. So I had to get up to speed on it. Which is beyond the scope of this document. Suffice to say it’s a surprisingly powerful tool built into Xcode which is shockingly uncomfortable for a programmer to use. Let’s leave it at that.
Putting polish in the process
After that I decided I needed a goal, I now had enough understanding of the pieces of this thing that I needed a thing to drive toward to keep myself engaged and figure out all the other elements. So the goal was to do something fun and funny and Beetlejuice emerged.
The mechanics of the portal effect were well documented, so that was easy. Can I do interesting lighting and sound in SceneKit? That was key; the idea that I’m physically moving toward an object making sound was vital to convey. That became more about the experience of the scene, the tangibility of this unreal thing, because now it had taken on these attributes of reality. Whoa.
The polish had to be part of the process, because I had to prove to myself how within reach next-level polish was. If I, as an amateur, can achieve something that felt next-level in terms of polish, that means a dedicated team over time can push this much further than I can.
I think this proves the entertainment theory pretty well. People seem to like this. And I can easily envision entertaining uses for AR. The challenge, given the technology, is to figure out what utility looks like. What is genuinely useful in AR? How do you merge in other technologies in a non-novelty way?
This company is equipped to think about that, and execute on that, but finding a truly transformative solve to the challenge is going to take a partnership between people who are interested in executing on this and making it good and people who are interested to find and solve big business challenges using the technology.
For me personally, I’m going to keep messing with it. I want to do something that’s less jokey. Maybe not completely less jokey. I’m also a big fan of the movie Mannequin, with Andrew McCarthy. So maybe there’s something there. Eighties films might be a deep well when it comes to AR concepts.