If you need evidence that Apple is working on a mixed-reality headset, take a spin with the HoloKit X. Created by Botao Amber Hu, a developer who has worked at companies like DJI, Google, and Twitter and is now CEO and founder of Holo Interactive, this headset relies entirely on existing capabilities of the iPhone to create interactive hands-free augmented reality experiences. It’s a powerful showcase of what’s possible if Apple ever made a headset using the tech already embedded in its smartphone.
Any such headset to come out of Cupertino would almost certainly cost more than a thousand dollars. (This is Apple, after all.) Look at Meta’s newest mixed-reality headset for reference; it starts at $1,499. Headsets in Microsoft’s XR platform cost between $600 and $1,000. These high prices are why the HoloKit X exists. Hu, who has long had a special interest in future computing and new media art, says he wants to “democratize” the world of mixed reality. As such, the HoloKit X costs $129, and all you need is a recent iPhone (excluding iPhone Mini and iPhone SE models) to power it.
An iPhone on Your Head
The HoloKit X is a plasticky headset with optical lenses inside. There’s no technology here (save for an NFC sensor, but more on that later). Just think of it as a viewer, not unlike old-school View-Masters. Similar to mobile virtual reality headsets like Lenovo’s AR set for Star Wars games, or the now-defunct Google Daydream, you need to mount an iPhone onto the HoloKit X.
Unlike VR headsets, you’re not staring at a screen. The iPhone is mounted up and away from your eyes. Instead, you’re looking through the glass in a 60-degree field of view and can see the physical world as well as the people around you. The iPhone’s screen, while using the rear cameras to manage these AR experiences, is mirrored in stereoscopic vision to the lenses, making it so that you can effectively see virtual 3D objects embedded in the real world.
Exactly what you can do with the HoloKit X is limited right now. There are just a handful of experiences—what Hu calls “Realities”—in the HoloKit app, one of which is a multiplayer dueling game where you cast spells at an enemy. The visuals are clear, colorful, and pretty sharp, and the platform supports six degrees of freedom via Apple’s ARKit framework. Because of this, you can move around virtual objects and they will stay anchored in the real-world places where you position them. And when you’re playing a game, you can even duck to dodge blasts. The “enemy” can be another person using a HoloKit X in a shared space, a virtual character, or even a character controlled by someone with just an iPhone.
Since it’s entirely powered by an iPhone, the HoloKit app is leveraging existing technologies. The ability to play a game with other HoloKit X users, for example, doesn’t rely on cellular data or Wi-Fi, but rather the local networking technology that powers AirDrop. This is also what powers “Spectator View,” which allows anyone to use an iPhone and the HoloKit app to view your augmented reality experience in real time by pointing their phone at the scene. (You can record and share this to social media, or cast it via AirPlay to a TV for others to see.) Hu says Holo Interactive is also working on a Puppeteer mode that would enable someone else to direct your AR experience.
There are a few ways to interact with the augmented reality experience. The HoloKit app uses Apple’s Vision framework technology to identify and track your hand. I didn’t see a demo of this, but the idea is that you can just use your hands to interact with objects and the iPhone’s cameras will recognize your hand movements. Hu says HoloKit also supports any Bluetooth device that can connect to the iPhone, like PlayStation controllers.
What I did demo was the ability to use an Apple Watch’s gyroscope as a motion controller, just like a Wiimote. Hu strapped an Apple Watch to my wrist (it works with Watch Series 4 and newer) with the HoloKit watch app installed and running, and gave me a wand purely so I could feel like I was using it to shoot out spells. Lo and behold, I was able to cast spells with mere gestures or a flick of the wrist. I could even point my wand downward to load a charging bar and trigger a more powerful spell. Aiding the immersion is the use of spatial audio via any of Apple’s headphones that support that feature, so you can hear a spell whizzing past your right ear. The iPhone’s haptic vibration adds another layer of sensory input, but since the phone is mounted in the headset, it’s only vibrating up near your forehead, so you may not immediately sense it.
You can use the HoloKit X with an iPhone XS, XS Max, iPhone 11, iPhone 11 Pro and 11 Pro Max, iPhone 12, iPhone 12 Pro and 12 Pro Max, iPhone 13, and iPhone 13 and 13 Pro Max, iPhone 14, and iPhone 14 and 14 Pro Max. (You’ll need to take off your case so it will fit.) You’ll get the best experience with an iPhone that has a lidar sensor, which became a staple on the Pro models—starting with the iPhone 12 series.
#HoloKit #Headset #Advances #HandsFree #iPhones