Apple confirmed it’s launching VisionOS 2 on September 16th, which is coming along with a ton of previously teased features announced back at WWDC in June. Three features are listed as “coming later this year” though. Here’s what you’ll be missing come Monday.

We wrapped up the list of VisionOS 2 stuff below, however three pretty interesting features don’t have release dates yet:

Panoramic Productivity with Mac Virtual Display

This one we really hoped would come along with VisionOS 2, but we’ll just have to wait a bit longer. Later this year Apple will finally allow Mac Virtual Display for a more immersive, ultrawide screen experience, something the company says simulates two 4K displays side by side for enhanced multitasking.

It looks pretty simple too. The video above shows a UI toggle to go from ‘Normal’ to ‘Wide’ and ‘Ultrawide’ modes when using Mac Virtual Display, which could tip the scales of finally bringing Vision Pro (along with your Mac) to the coffee shop to get work done.

Spatial Videos Support for Final Cut Pro

Apple announced back at WWDC that Final Cut Pro will finally play nice with MV-HVEC videos captured by Vision Pro, iPhone 15 Pro, and now the entire iPhone 16 line thanks to support for of spatial video, photo and audio capture. At some point, Apple says you’ll be able to import and edit spatial videos in Final Cut Pro on your Mac, add “immersive titles and effects,” and then view the final product in Vision Pro.

We haven’t seen it in action yet, but if Apple is investing sufficient resources, we’re really holding out hope for a seamless editing and live viewing experience, which would be great for workflow.

Apple TV Sports Multiview

At some point Vision Pro is getting the Multiview feature on Vision Pro’s Apple TV app, putting it more in line with the capabilities of Apple TV 4K.

The addition of Multiview means you’ll be able to watch up to five MLS or MLB games simultaneously, with customizable layouts and the ability to switch between different game views. On the toilet.

In case you missed it, here’s a list of some headlining features coming to VisionOS 2 next week:

Coming Next Week:

• Create Spatial Photos from 2D Images: Instantly transform existing photos into spatial photos with depth and dimension by tapping on them in your library.
• SharePlay for Photos: On FaceTime, share and interact with spatial photos, videos, and panoramas with friends and family, allowing you to relive memories together in life-size scale.
• Redesigned Photos App: A refreshed and organized Photos app makes it easier to find photos and featured spatial photos with a visually appealing and user-friendly layout.
• Trim Videos at Any Scale: Trim videos from your photo library with ease, adjusting the scale to fit your preferences.
• See Your Keyboard in Any Environment: When immersed in an Environment, your Magic Keyboard or MacBook keyboard will be recognized, allowing uninterrupted typing.
• Cinematic Video Experience in Safari: Enjoy watching videos on a massive screen in Safari, with enhanced cinematic effects, such as screen glow and reflections.
• Guest User Mode: Easily let friends and family try Apple Vision Pro with Guest User mode, which now saves their eye and hand data for quick setup in future sessions.
• Look to Dictate in Messages: A new dictation method in Messages where you can simply look at the text field to start dictating.
• AirPlay Receiver: Use AirPlay to mirror your devices to Apple Vision Pro, expanding sharing and viewing capabilities.
• Train Support in Travel Mode: Work with multiple apps or watch shows while commuting, enjoying a massive screen in a serene virtual Environment.

Apple announced its new iPhone 16 is getting a little more spatial with the addition of both spatial photos and audio, essentially putting it in line with Vision Pro’s capture capabilities.

The company’s big event on Monday didn’t bring any giant revelations for Vision Pro, although the company announced a host of products, including its AI-infused iPhone 16 line, as well as all-new Apple Watch Series 10 and AirPods 4.

The iPhone 16 introduces a new pill-shaped camera design, enhanced AI features, and a couple of new buttons (Action and Camera Control) compared to the iPhone 15. It retains the same 60Hz display (120Hz on iPhone 16 Pro), with improvements being pretty incremental overall.

The entire iPhone 16 line however also comes with the ability to not only capture spatial video like on iPhone 15 Pro, but now spatial photos and audio. Vision Pro can do all of these things, although you’d have a hard time fitting it in your pocket.

Regardless of what phone you have, if you own a Vision Pro all of your photos can be spatialized thanks to an update earlier this summer which automatically converts 2D photos to 3D—and pretty convincingly so.

Notably, spatial videos can also be viewed on other XR devices that support the MV-HVEC video codec, such as Meta Quest.

Vision Hack, the world’s first global visionOS hackathon, is being held this week from September 13–15th. The organizer has announced $25K in cash & prizes on offer for winners.

Guest Article by Cosmo Scharf

Cosmo Scharf is an Emmy-nominated product designer and entrepreneur with a decade of experience in XR. He co-founded Vision Hack and visionOS Dev Partners to support visionOS developers. Previously, Scharf created MovieBot, an AI-powered 3D animation app, and Mindshow, a VR platform for animated content creation. His also started VRLA, one of the world’s largest immersive technology expos.

Vision Hack is happening this week starting on Friday the 13th. All skill levels are welcome and there’s still time to sign up to participate. The first two readers to use code RoadToVRF5M3T can sign up for free.

We have some exciting updates for you about the event.

Prize Categories

There will be four main prize categories:

• Game & Entertainment
• Productivity & Utility
• Wellbeing & Lifestyle
• XR Demoparty – Presented by XR.vc

Each category will have three winners: 1st Place ($3,000), 2nd Place ($1,500), and 3rd Place ($500).

Schedule

Here’s a schedule of Vision Hack’s key moments (all times in EST):

Day One – Friday, September 13th

5:00 PM – Opening Ceremony
6:00 PM – Hackathon Begins

Day Two – Saturday, September 14th

10:00 AM – Check-in & Workshop Kickoff
1:00 PM – Teamwork & Mentor Office Hours
7:00 PM – Evening Coding Jam

Day Three – Sunday, September 15th

10:00 AM – Check In
6:00 PM – Project Submission Deadline
6:30 PM – Closing Celebration

Day Four – Monday, September 16th

10:00 AM – Winners Announced

Local Meetup Update

We’ve added Manchester and Stockholm to our existing lineup of NYC, SF, Paris, & Montreal. RSVP here. Can’t wait to see you IRL!

Thank You Sponsors & Media Partners

Thank you to our sponsors XR.vc, The Venture Reality Fund, Syntech, CXR Agency, Annapro, & Globular Cluster! Thank you as well to our media partners Apple Insider, Apps for Apple Vision Pro, Road to VR, XReality Zone, r/VisionPro, and Immersive Tech Week!

Sounds fun? Sign up for Vision Hack here.

Road to VR is a proud media partner of the Vision Hack event

Google Earth VR was an early and enchanting app for the first generation of modern PC VR headsets, but unfortunately it never made the leap to the standalone era. Now a new app, FLY, brings the magic of exploring Google Earth in VR to Quest and Vision Pro.

Now available on Quest 2, Quest Pro, Quest 3, and Vision Pro, Fly aims to bring the magic of exploring Google Earth in 3D in a fully immersive way. Beyond just being able to explore the globe to your heart’s content, Fly aims for a more immersive feel than its spiritual predecessor. While the original Google Earth VR essentially had you floating around as a disembodied camera, Fly is thematically built around the idea that players are piloting a drone that can take them anywhere.

“Fly is our idea of a personal flight simulator that lets you travel on ground, above cities, and across countries to explore places you’ve never seen. Our focus has been on a seamless experience with intuitive comfortable controls, smooth loading, and detailed rendering of Google Earth Tiles,” says Eric Malafeew, CTO and Co-Founder of VirZOOM, the studio behind the app.

The studio says that Fly uses Google Earth’s 3D Map Tiles, offering a massive Earth-sized space for players to explore, including full 3D geometry in many major cities. The project was purportedly started in collaboration with Google, but was put on hold for years.

“Originally created in collaboration with Google for a PC VR release in 2018, we had to shelve it after they paused Google Earth API development,” says VirZOOM Co-Founder and CEO Eric Janszen. “So when we saw them announce the Google Earth Tiles API this past year we were excited to complete the product. This new version takes what we created in 2018 and benefits from our years of knowledge in VR development since then. This version of Fly is everything we had hoped for in PC VR, but available on standalone headsets.”

While the app is available on Quest headsets—and gets a boost in quality on Quest 3—the developer says the added headroom on Vision Pro means it can deliver the optimal experience.

“The 3D Earth tiles, which Google provides, have the same maximum quality on both Quest 3 and Vision Pro. But the Vision Pro version is rendered at 2.5x the max resolution of Quest 3, and we draw tile details out to 2.5x greater distance, with none of the performance hits you get from maxing out the Quality option on Quest 3,” says Robert Collins, COO of VirZOOM. “This means the tiles, UI and overall experience in the game look significantly better on Vision Pro and provide a smoother experience as the user moves through the world.”

MixRift, a mixed reality game studio, today announced it’s secured a $1.6 million pre-seed investment round, something the team says will be used to develop more MR games.

The funding round was led by Outsized Ventures, Underline Ventures, and SOSV. According to CrunchBase data, MixRift received an undisclosed round from SOSV in May 2024.

Founded in 2023, the team is led by Bobby Voicu, David Pripas and Andrei Vaduva. Voicu, who is CEO, previously launched mobile games studio Mavenhut; Pripas is an official Meta AR Partner and the company’s CPO, and Vaduva is the company’s CTO.

Known for its indie XR titles Hell Horde (2024) on Quest and Fractured (2024) on both Quest and Apple Vision Pro, MixRift says it’s working to create the Angry Birds of mixed reality—something that’s easy to pick up but hard to put down.

“We’re at the very beginning of what mixed reality could do for gamers,” Voicu says. “We know there is a huge untapped market, and our founding team knows what it takes to create the games that hook those players in. This puts us in a unique position to capitalise on the opportunity.”

MixRift says it will use the investment to continue its development and plans to release further titles later in 2024, noting that it’s focusing on sticky game mechanics over “specific titles” to best identify what resonates with audiences.

“We are committed to rapidly prototyping games, getting them into the market, and gathering real user feedback as quickly as possible,” Voicu says. “We use those insights to improve titles that connect with audiences and replace those that don’t. This investment gives us the support we need to take that approach and demonstrates experienced, knowledgeable backers’ buy-in to our philosophy.”

Vision Hack is a forthcoming remote hackathon for developers building apps for visionOS. Open to all skill levels, the event is set to take place September 13–15th.

Guest Article by Cosmo Scharf

Cosmo Scharf is an Emmy-nominated product designer and entrepreneur with a decade of experience in XR. He co-founded Vision Hack and visionOS Dev Partners to support visionOS developers. Previously, Scharf created MovieBot, an AI-powered 3D animation app, and Mindshow, a VR platform for animated content creation. His also started VRLA, one of the world’s largest immersive technology expos.

Imagine building the first iPhone apps in 2008. That’s where we are with visionOS today. Vision Hack is your launchpad into the new frontier of spatial computing.

Over the past decade, I’ve had the privilege of witnessing the virtual reality industry evolve from a niche technology to a transformative medium. From the early days of clunky prototypes to the sleek, powerful devices we have today, the journey has been nothing short of extraordinary. Now, with the introduction of Apple Vision Pro, we’re standing at the threshold of a new era.

As one of the organizers of Vision Hack, I’m thrilled to announce the launch of the first global visionOS hackathon. Scheduled for September 13–15th, this event represents a significant milestone in our industry’s progression. It’s an opportunity to explore and shape the future of spatial computing as Apple Vision Pro continues its global rollout.

Vision Hack is designed to be a truly immersive experience. We’re encouraging teams to communicate in Vision Pro itself using spatial Personas, in addition to Discord. This approach not only showcases the device’s capabilities but also provides participants with authentic, hands-on experience in visionOS development.

Our three-day program caters to both seasoned spatial computing developers and newcomers:

• Day 1: Workshops and team formation
• Day 2: Intensive development with mentorship from industry experts
• Day 3: Development, project presentations, and awards

To foster collaboration while ensuring focused development, we’ve capped team sizes at 5 people each. Understanding the global nature of our community, we’re organizing local meetups in various cities so developers can connect in person.

While we highly recommend access to a Vision Pro for the full experience, it’s not a strict requirement for participation. However, developers will need a Mac with an Apple chip to run the visionOS simulator. This setup will enable meaningful participation even without the physical device.

The organizing team brings extensive experience from major VR expos (VRLA), Metaverse hackathons, and XR startups.

As spatial computing evolves, we believe early developer engagement is crucial in building a robust ecosystem. Vision Hack aims to play a key role in nurturing the visionOS developer community, potentially influencing the trajectory of spatial computing applications.

For developers keen on exploring visionOS, Vision Hack offers a unique opportunity to dive into this emerging platform. There’s a $25 registration fee, which helps us cover some of the event costs and ensures committed participation.

For companies interested in being at the forefront of spatial computing development, we offer various sponsorship opportunities. These partnerships not only support the event but also provide sponsors with direct access to a pool of talented developers working on cutting-edge spatial computing applications.

More details, registration information, and sponsorship opportunities can be found at visionoshackathon.com. We’re excited to see the innovative projects and ideas that will emerge from this event, and we look forward to welcoming you to the next chapter of spatial computing development.

In this edition of XR News Bits we see a newly released Walkabout Mini Golf DLC course, a big update to Contractors Showdown, game director Hideo Kojima exploring Vision Pro, the release of VR action-arcade game Thrasher, and signs of life from cooperative WWII VR bomber game The Mighty Eighth.

XR News Bits

An incredible amount of exciting XR news comes our way every day. Very often we see news come across our desk that’s worth highlighting, but we don’t always have the bandwidth for a full article on every one of them. XR News Bits is our occasional roundup of stories we think are worth a shout-out, even if we can’t take you in-depth.

Walkabout Mini Golf Gets New ‘Wallace & Gromit’ Course

Walkabout Mini Golf has maintained an impressive cadence with the release of interesting courses to give players something new to come back to. The latest course is based the perennial stop-motion franchise Wallace & Gromit, which first premiered in 1989.

The new course is said to be packed full of locations and references that long standing fans of the show will appreciate. The new course is paid DLC which can be bought inside of the game for $4. Thoughtfully, Walkabout Mini Golf includes a ‘Guest Pass’ feature which means only one person in the room needs to own the course for the whole party to play.

Battle Royale Contractors Showdown Gets Big ‘Season 1’ Update

Hot of its release in April, VR battle royale Contractors Showdown is already dropping its first big update, borrowing the ‘season-based’ format that’s become the norm for live-service games.

The ‘Season 1’ update includes new vehicles, weapons, map locations, and dynamic weather. There’s also a new game mode called ‘Reborn’: a 30-player brawl where players get an immediate second chance after their first elimination. The original mode also now adapts the ‘Gulag’ feature from Call of Duty: Warzone where eliminated players duke it out in a quick 1v1 battle, with the winner earning the right to respawn into the battle royale match and the loser being fully eliminated.

Thrasher Launches on Quest and Apple Vision Pro

Thrasher, a unique action-arcade game from one of the creators of the cult action-arcade hit Thumper, has launched on Quest and Vision Pro.

The game sees players guiding a “magnificent space eel” through and around obstacles which grow in difficulty as time goes on. The creators call it a “mind-melting arcade action odyssey and visceral audiovisual experience.”

Thrasher is available now for $20 on Quest and Vision Pro, and the game will on launch on PC VR via Steam in December.

Metal Gear Solid Creator Hideo Kojima Checks out Vision Pro

Legendary videogame director Hideo Kojima posted several photos of himself unboxing and trying out Apple Vision Pro.

Kojima is best known as a creative force behind the Metal Gear Solid series and his newer venture, Death Stranding.

Kojima has demonstrated a clear interest in VR tech over the years, having been spotted checking out the latest developments here and there. So far he hasn’t released or announced any VR projects, but perhaps Vision Pro will inspire new interest in the tech.

Cooperative VR Bomber Game The Mighty Eighth VR Shows Signs of Life

Cooperative WWII VR bomber game The Mighty Eighth was announced way back in 2020, but we hadn’t heard much from the project in the intervening years.

Earlier this year the game’s Steam page sprung to life with some updates on its development. The latest update includes early footage showing some of the game’s core systems.

Planned for eventual launch on PC VR, The Mighty Eighth VR has no release date for now, but the developer is promising more progress updates to come.

Thanks to Daniel Fearon for the tip on this one.

Meta today announced plans to update Quest 3 with AI vision features similar to what’s on the company’s latest Meta Ray-Ban glasses. Meanwhile, Apple still hasn’t confirmed if Apple Intelligence features will hit Vision Pro at the same time as other Apple devices this Fall.

Meta has pointed to AI investments as its other major area of R&D alongside XR, and now the company is starting to bring the two together with consumer-facing AI features.

Sometime later this Summer in the US and Canada, Meta says it will roll out an update to Quest 3 enabling the ‘Meta AI with Vision’ feature. The feature will give the headset AI voice chat capabilities and also allow the headset to ‘see’ what’s in the user’s real-world field-of-view. Users can ask general questions but also inquire about things in front of them. Meta gives some examples in its announcement:

Let’s say you’re watching YouTube videos of some breathtaking hikes in mixed reality while packing for your upcoming trip to Joshua Tree. You can ask Meta AI for advice on how to best dress for the summer weather. Or you could hold up a pair of shorts and say, “Look and tell me what kind of top would complete this outfit.” You can get the forecast so you can prep for the weather ahead and even ask for local restaurant recommendations to indulge your inner foodie.

Or say you’re in-headset and listening to music while working on a paper for school on a massive virtual monitor. You could ask Meta AI to identify some of the most memorable quotes from Shakespeare’s Hamlet and explain the significance of the “to be or not to be” soliloquy or the play within a play.

You might be playing Assassin’s Creed® Nexus VR, parkouring across rooftops when your curiosity is piqued. Why not ask Meta AI whether or not there were actual assassins in colonial Boston? The answer may surprise you…

For the time being, the camera can only see what’s in the real world, but won’t have awareness of virtual content shown by the headset. Meta is alluding that the Meta AI with Vision feature may eventually include both real world and virtual world awareness.

Unfortunately Meta confirms the feature won’t be coming to Quest 2 or older devices (and it’s likely Quest Pro won’t get it either).

Meta didn’t talk about whether requests for this feature are being processed on-device or in the cloud, nor did it address things like encryption, though it did confirm the feature is based on Bing AI. The company hasn’t yet responded to our request for more info on privacy architecture.

While Meta is rapidly deploying AI capabilities to its devices, Apple has yet to confirm if its so-called ‘Apple Intelligence’ features are coming to Vision Pro.

Earlier this year Apple announced a range of Apple Intelligence features coming to iPhones, iPads, and Macs in beta this fall. Despite announcing visionOS 2 at the same time, no Apple Intelligence features have been confirmed for Vision Pro. That leaves it up in the air whether the headset will get any Apple Intelligence features at the same time as other Apple devices, or if users will need to wait until later versions of VisionOS.

Apple says many Apple Intelligence features are processed on-device, but some requests—including those which lean on ChatGPT—will request off-device processing. Apple claims off-device requests “never store your data,” “are used only for your requests,” and that Apple will make the behind-the-scenes code available for privacy auditing.

Apple is launching a new slate of immersive video content for Vision Pro using Apple Immersive Video, a 180-degree video format combining 8K 3D video and spatial audio for immersive experiences.

Premiering on July 18th is Boundless, which features unique travel adventures, starting with a hot air balloon journey over Cappadocia, Türkiye—the episode suitably named ‘Hot Air Balloons’.

In August, Vision Pro users will get to see the next installment of Wild Life, a nature documentary series captured at Kenya’s Sheldrick Wildlife Trust. Coming in September is Elevated, which promises aerial views of stunning landscapes, beginning with Hawaii.

Additionally, Apple will release an immersive experience featuring The Weeknd, the first scripted short film Submerged by Edward Berger, and exclusive sports content like behind-the-scenes access to the 2024 NBA All-Star Weekend.

A new sports series with Red Bull called Big-Wave Surfing will also be available, showcasing elite surfers tackling massive waves in Tahiti.

Apple says it’s collaborating with Blackmagic Design to enhance the production of Apple Immersive Video with new tools and workflows, including the URSA Cine Immersive camera.

All of these immersive videos will be accessible for free via the Apple TV app in several countries.

Vision Pro doesn’t have controllers, although that’s not stopping third-party hardware creators from trying to beat Apple to the punch. Enter Surreal Touch, a motion controller that’s just launched a Kickstarter campaign, aiming to leverage Vision Pro’s ability to play PC VR games.

Update (August 27th, 2024): The Kickstarter for Surreal Interactive’s controller Surreal Touch for Vision Pro is now live. It’s already broken past its initial $50,000 funding goal, with an estimated ship date in December 2024.

The lowest funding tier, priced at $249, is still available at the time of this writing. The campaign maintains that this represents a 30% savings off its MRSP of $369. All funding tiers feature a single set of controllers, however mix and match Steam keys for Contractors Showdown (2024) and/or Grimlord (2023).

Check out the product trailer below to see some of the main features in action:

Original Article (July 3rd, 2024): Surreal Interactive, the startup behind Surreal Touch, is hoping to bring the controllers to market some point next month (see update), with its price revealed at the time of launch.

Similar to Meta’s Touch Pro controllers, Surreal Touch contains optical sensors with fisheye lenses that allow the device to track independently from the headset.

Also very similar to Meta’s Touch Pro, you’ll find a ring-less design thanks to its inside-out optical sensors, onboard SLAM tracking allowing for 6DOF input, and a standard button configuration that anyone with a modern VR headset will instantly recognize.

Speaking to CEO and founder Zhenfei Yang, Road to VR learned that Surreal Touch will ship with a SteamVR compatibility mode, which supports all existing PC VR games. That means you’ll need a VR-ready PC to play games like Half-Life: Alyx (2020), which the company shows working below:

And just how will that work? Surreal Interactive says it’s currently developing a Vision Pro-native streaming app, which promises to let you play SteamVR games from your computer at “4K, 120 Hz.”

While PC VR gameplay is undoubtedly a big focus, the company says it’s also working on its own native SDK, which will allow Vision Pro app developers the ability to support Surreal Touch.

“We’re currently collaborating with several studios to port their games to the AVP platform. Additionally, we have plans to reach out to and partner with more developers in the near future,” Yang told Road to VR.

Surreal Interactive also teased it’s working on a ‘Tool Mode’, which presumably could also allow for basic UI selection when in Vision Pro, however the company hasn’t revealed its exact function, stating only that the additional mode is “still undergoing in-house development. Stay tuned for more details.”

There’s still plenty to learn about, including battery life, real-world tracking accuracy, compatibility with other headsets, price, and more. We’re hoping to learn more closer to launch, so check back soon. In the meantime, check out the spec sheet below:

Specs

The 2D to 3D photo conversion feature coming to Vision Pro in VisionOS 2.0 makes a novel capability meaningful for the first time.

Cue “Apple didn’t even do it first!” in the comments.

You’re not wrong. There’s been seemingly a hundred different startups over the years that have promised to turn 2D photos into 3D.

Even Meta had a go at it when it added 2D to 3D photo conversion to Facebook several years ago. But they never really caught on… probably because seeing 3D photos on a smartphone isn’t that exciting, even if Facebook added a little ‘wiggle’ animation to show the depth on 2D displays.

When it comes to features that people actually want to use—it doesn’t matter who does it first. It matter who does it well.

This headline says “the real deal,” because Apple has, in fact, actually done it well with Vision Pro. The 2D to 3D conversion doesn’t just look good, the feature is actually implemented in a way that takes it beyond the novelty of previous attempts.

The feature is part of visionOS 2.0, which is currently available in a developer beta. Apple says the feature creates “spatial photos” from your existing 2D images (which of course just means stereoscopic ‘3D’).

Granted, even though it’s “just stereoscopic,” seeing your own photos in 3D really adds a layer of depth to them (figuratively and literally). While a 2D photo can remind us of memories, a 3D photo feels much closer to actually visiting the memory… or at least seeing it through a window.

In VisionOS 2.0, just go to the usual Photos app, then open any photo and spot the little cube icon at the top left. Click it and the headset analyzes and converts it to 3D in just two or three seconds. With a click you can also return to the original.

The results aren’t perfect but they’re very impressive. It’s unfortunate I can’t actually show them to you here—since I have no way to embed a 3D photo in this page, and 99.9% of you are probably reading this on a 2D display anyway—but it’s the best automatic 2D to 3D photo conversion that I’ve personally seen.

The speed and accuracy is doubly impressive because the conversion is happening 100% on-device. Apple isn’t sending your photos off to a server to crank out a 3D version with cloud processing resources and then sending it back to your headset. That makes the feature secure by default (and available offline), which is especially important when it comes to a dataset that’s as personal as someone’s photo library.

Across the photos you’d find in the average person’s library—pictures of people, pets, places, and occasionally things—the conversion algorithm seems to handle a wide range of these very well.

While the feature works best on real-life photography, you can also use it on synthetic imagery, like digital artwork, AI-generated photos, 3D renderings, and the like. Results vary, but I overall I was impressed with the feature’s ability to create plausible 3D depth even from synthetic imagery which itself never actually had any 3D depth in the first place.

The thing the algorithm seems to struggle with the most is highly reflective and translucent surfaces. It often ends up ‘painting’ the reflections right onto the reflecting object, rather than projecting them ‘into’ the object with correct depth.

The only major limitation at the moment is that 2D to 3D photo conversion doesn’t seem to want to work on panoramic images. On Vision Pro panoramas can already be blown up and wrapped around you in a way that feels life-sized, but they would still get another layer of emotional impact from being 3D-ified.

It’s unclear why this limitation exists at present, but it’s likely either because panoramas tend to be very high resolution (and would take longer than a few seconds to convert), or Apple’s 2D to 3D algorithm needs more training on wide field-of-view imagery.

Beyond that limitation, the thing that really makes this feature… a feature (not just a ‘technical possibility’), is that it’s built right in and works in the places and ways you’d expect.

Not only can you send spatial photos to other users who can view them in 3D on their own headset, you can also start a SharePlay session and view them together—an incredible way to share moments and memories with the people that matter to you.

And its easy to actually get the photos you want onto your headset for viewing.

Many people will have their iCloud photos library synced with their headset, so they’ll already have all their favorite photos ready to view in 3D. I personally don’t use iCloud photos, but I was easily able to select some of my favorite photos from my iPhone and AirDrop them, which automatically opened the Photos app so they were right in front of me in the headset.

Further, you can just save any old photo to your headset—be it from Facebook, a website, or another app—and use the 2D to 3D conversion feature to view them with a new layer of intrigue.

And this is what makes this visionOS 2.0 feature different than 2D to 3D conversion software that has come before it. It’s not that Apple has any groundbreaking quality advantage in the conversion… it’s the fact that they made the experience good enough and easy enough that people will actually want to use it.

Apple Vision Pro isn’t exactly brimming with XR games yet, although starting next month you’ll be able to jump into a classic city builder that has been updated to include full mixed reality support.

Magic Fuel Games is releasing Cityscapes: Sim Builder (2023) for Vision Pro soon, bringing with it a new immersive view so you can manage your city as the god-like mayor you always wanted to be. It’s slated to arrive on July 3rd via Apple Arcade.

“They’ll use their hands to lay down roads, build neighborhoods, and ultimately grow their city into a thriving, profitable, and sustainable place to live,” Apple says in a press statement.

Cityscapes: Sim Builder has been available across all Apple devices since its initial iOS launch in mid-2023, taking the Apple Arcade Game of the Year as well as an Apple Design Award finalist.

While we still think the platform could benefit from more ports (and proper motion controllers) this follows the release of a few top mixed reality games that make use of the headset’s hand-tracking, including Game Room, WHAT THE GOLF?, Super Fruit Ninja, Where Cards Fall, Crossy Road Castle, and Solitaire Stories. 

Maze Theory today announced that its upcoming immersive puzzle-adventure game Infinite Inside is set to arrive next month, coming to all major headsets.

Initially released as a demo on Quest earlier this year, the full game is set to arrive on July 12th across Vision Pro, Quest 2/3, Pico, SteamVR headsets and PSVR 2.

On mixed reality-supported devices (re: not PC VR or PSVR 2), Infinite Inside is said to blend VR and MR, making these “multi-dimensions a fundamental part of the storytelling and gameplay.”

Check out the trailer below:

When Infinite Inside launches next month, Maze Theory says it will feature five “peaceful and soothing labyrinths filled with mysteries,” environmental storytelling, and “challenging and accessible” puzzles.

Here’s how the studio describes the action:

When the enigmatic ancient artifact known as the ‘Plinth’ mysteriously materialises in your home, it opens a portal to a tranquil, dream-like world, filled with impossible architecture and hidden secrets waiting to be discovered. Guided by the echoes of past explorers, you will collect shards, solve three-dimensional puzzles, and assemble keys to unlock the mysteries of a secret society committed to maintaining the balance of Order and Chaos.

Can you solve the puzzles and reveal the truth about the society, the legacy of the Plinth, and ultimately, your own place in this mystical world?

Maze Theory, a subsidiary of Saltwater Games, is the London-based team behind a slew of XR games, including Peaky Blinders: The Kings Ransom, Doctor Who: The Edge of Time and Doctor Who: The Lonely Assassins.

Infinite Inside marks one of Maze Theory’s (and likely the industry’s) broadest releases to date, covering nearly all major VR/MR platforms. You can find it on Steam, Quest, PSVR 2, and Pico. An App Store link for the Vision Pro version should arrive closer to launch, so make sure to keep your eyes peeled.

At $3,500, Vision Pro is undoubtedly expensive, which many are rightfully hoping will be remedied in a prospective follow-up. Now, according to a report from The Information, Apple may be ditching the ‘Pro’ aspect of its next-gen Vision headsets altogether, instead aiming to release a single “more affordable” device in late 2025.

It’s rumored that Apple was slated to release two headsets: an expensive Pro-style device and a cheaper version targeted more squarely at consumers, much like how the company positions iPhone in its lineup today.

Now, citing an employee at a manufacturer that makes key components for the Vision Pro, The Information reports Apple has suspended work on that high-end follow-up due to slowing sales of the $3,500 Vision Pro.

There may be hope though, at least for anyone without the budget to shell out what amounts to a good used Honda Civic. According sources both involved in the supply chain and in the manufacturing of the headset, the company is “still working on releasing a more affordable Vision product with fewer features before the end of 2025.”

Granted, it’s important to note that Apple often leaks incorrect information in a bid to nail prospective leakers, so this (and any Apple report for that matter) should be taken with a heaping handful of salt.

This follows Apple’s announcement it was getting set to release Vision Pro outside of the US for the first time, which includes mainland China, Hong Kong, Japan, Singapore, Australia, Canada, France, Germany, and the UK.

– – — – –

Whether it’s “more affordable” or not, there’s a lot Apple can do to appeal to the masses without drastically sacrificing quality. Check out our article on the 6 Things Vision Pro Needs Before It Can Go Mainstream to see how.

Did you know you can technically play SteamVR games like Half-Life: Alyx (2020) on Vision Pro? Well, now you do, as the leading app dedicated to bringing SteamVR connectivity to a ton of standalone headsets is now available for Vision Pro on the App Store.

Previously offered as an invite-only beta, ALVR is now free on the App Store, allowing Vision Pro users to stream OpenVR games from your PC via Wi-Fi.

This includes support for Vision Pro’s hand gestures, however more than likely you’ll want some sort of controller to really make use of SteamVR content. Developer Max Thomas mentions ALVR supports dual controllers such as the Nintendo Switch Joy-Con controllers, in addition to existing Lighthouse base station setups for enthusiasts already in the SteamVR tracking ecosystem.

To get up and running, you’ll of course need a VR-ready computer, either running Windows or Linux. Check here if your computer is capable of running VR games.

Another thing to check is that Vision Pro is on the same local network as the computer, which for best performance ought to be 5GHz 802.11ac router with speeds greater than 300Mbps, placed within 20 feet (6 meters) from your playspace. You can get away with less, but you may experience some pretty annoying stuttering.

Whatever the case, network activity from other devices should be at a minimal to keep bandwidth clear for PC-streaming, and the host computer should be connected directly to the router via Ethernet. Lastly, make sure AirDrop and Handoff are disabled during gameplay to reduce stuttering.

The official release of ALVR on Vision Pro follows early third-party attempts to hack the software into working, which spurred indie developer Max Thomas (aka ‘Shiny Quagsire’) to start experimenting with bringing the streaming tool officially to visionOS.

We’ve know that Apple planned to support WebXR for quite some time, but with VisionOS 2, the company is enabling the feature for all users. WebXR allows developers to deliver cross-platform XR experiences directly from the web, with no gatekeepers to approve or reject content.

WebXR is the widely supported web standard that allows developers to deliver AR and VR content from the web.

Just like anyone can host a webpage online without any explicit approval from another party, WebXR allows the same for AR and VR content. And because it’s delivered through the browser, accessing and sharing WebXR experiences is as easy as clicking or sending a link—like this one.

Vision Pro has supported initial WebXR support since its launch, but it required users to manually enable the feature by digging into Safari’s settings.

With VisionOS 2—available today as a developer preview, and coming to all this Fall—WebXR will be enabled by default, making it easy for anyone to access WebXR through the headset. Vision Pro thus joins headsets like Quest, HoloLens 2, and Magic Leap 2 in supporting WebXR content.

Though WebXR is “supported” on VisionOS 2, our understanding is that it only support VR (or ‘fully immersive’) experiences. WebXR is also capable of delivering AR experiences (where virtual content is merged with a view of the real world), but VisionOS 2 doesn’t yet support that portion of the standard.

There’s many reasons why developers might want to use WebXR to build experiences over native apps that are distributed through a headset’s proprietary store.

For one, any headset with WebXR support can run any compatible WebXR experience, meaning developers can build one experience that works across many headsets, rather than needing to make multiple builds for multiple headsets, then uploading and managing those builds across multiple platform stores.

Like a webpage, WebXR content can also be updated at any time, allowing developers to tweak and enhance the experience on the fly, without needing to upload new builds to multiple stores, or for users to download a new version.

WebXR also has no gatekeepers. So content that wouldn’t be allowed on, say, Apple or Meta’s app stores—either for technical or content-related reasons—can still reach users on those headsets. That could include adult content that’s explicitly forbidden on most platform app stores.

For basic Vision Pro interactions like navigating apps and scrolling web pages, the headset’s look-and-pinch input system works like magic. But if you want to go more ‘hands-on’ with virtual content, the headset’s full hand-tracking leaves much to be desired.

Compared to Quest 3, Vision Pro’s full hand-tracking has notably more latency. That means when moving your hands it takes longer for the headset to register the movement. Especially in interactive content where you directly grab virtual objects, this can make the objects feel like they lag behind your hand.

Changes coming in VisionOS 2 stand to improve hand-tracking. Apple detailed the changes in a developer session at WWDC 2024 this week.

For one, the headset will now report estimated hand positions at 90Hz instead of the previous 30Hz. That means the system can reflect changes in hand position in one-third of the time, also making the movement of the hand smoother thanks to more frequent updates. This only applies to a small portion of the overall latency pipeline (which we previously estimated at a total of 127.7ms) but it could reduce hand-tracking latency by as much as 22ms in the best case scenario.

Here’s a look at that in action:

It’s an improvement, but you can still easily see the latency of the teapod compared to the hand, even with this slow movement.

For a snappier experience, VisionOS 2 will alternatively allow developers to enable hand-tracking prediction, which provides an estimate of the user’s future hand position. While this doesn’t truly reduce latency, it can reduce perceived latency in many cases. Similar prediction techniques are common across various XR tracking systems; it’s quite surprising that Vision Pro wasn’t already employing it—or at least not making it available to developers.

Here’s a look at predictions in action:

Now we can see the virtual teapot staying much more aligned to the user’s hand. Granted, this isn’t likely to look quite as good with faster motions.

We’ll be looking forward to putting Vision Pro’s hand-tracking latency to the test with VisionOS 2 soon!

From the artist and composer behind indie hit Thumper (2016), the forthcoming Thrasher is officially launching on July 25th on Quest and Vision Pro. A PC VR and flatscreen version is planned for release at a later date.

We’ve known Thrasher was in the works for some time now, but this unique looking game now has an official release date of July 25th, priced at $20. The game is coming to Quest and is now confirmed to be releasing on Vision Pro the very same day. The PC VR and flatscreen version will launch at a later date.

A new trailer shows us a better look at the game’s chaotic yet mesmerizing gameplay:

Apparently based purely on hand-tracking (though likely falling back to controllers on PC VR), the gameplay looks undeniably unique. Here’s how the developers describe it:

TRANSCEND SPACETIME

Immerse yourself in a dazzling odyssey where music, visuals, and gameplay mesh into one transcendent experience. Journey from the depths of primordial gloom to the heights of celestial bliss, culminating in a heart pounding reckoning with a cosmic baby god.

YOU V THE UNIVERSE

Swoop, dash and thrash at breakneck speed, busting through obstacles and stacking up combos, leading to nine jaw dropping encounters with mysterious leviathans that will challenge your skills, and your sanity.

POWER UP

Deploy power-ups to supercharge your space eel and max out your combos. Create a destructive rainbow spray of bullets, bulldoze everything in a blaze of color and light, or slow things down to create a perfect path through the chaos.

SOUND AND FURY

Lose yourself in the enthralling soundtrack created by designer Brian Gibson, bassist for the band Lightning Bolt. THRASHER is a spatial audio showcase, creating a stunning sensory experience.

CHILL OR CHALLENGE

Vibe out and enjoy the wild journey, or push yourself to the limit by chaining together massive combos to worm your way up the rankings.

Will you be giving Thrasher a try?

VisionOS 2 is bringing a range of new development features, but some of the most significant are restricted to enterprise applications.

VisionOS 2 will bring some of the top requested development features to the headset, but Apple says its reserving some of them for enterprise applications only.

Developers that want to use the features will need ‘Enterprise’ status, which means having at least 100 employees and being accepted into the Apple Developer Enterprise Program ($300 per year).

Apple says the restriction on the new dev capabilities is to protect privacy and ensure a predictable experience for everyday users.

Here’s a breakdown of the enterprise-only development features coming to VisionOS 2, which Apple detailed in a WWDC session.

Vision Pro Camera Access

Up to this point, developers building apps for Vision Pro and VisionOS couldn’t actually ‘see’ the user’s environment through the headset’s cameras. That limits the ability for developers to create Vision Pro apps that directly detect and interact with the world around the user.

With approval from Apple, developers building Vision Pro enterprise apps can now access the headset’s camera feed. This can be used to detect things in the scene, or to stream the view for use elsewhere. This is popular for ‘see what I see’ use-cases, where a remote person can see the video feed of someone at a work site in order to give them help or instruction.

Developers could also use the headset’s camera feed with a computer vision algorithm to detect things in view. This might be used to automatically identify a part, or verify that something was repaired correctly.

Even with Apple’s blessing to use the feature, enterprise apps will need to explicitly ask the user for camera access each time it is used.

Barcode and QR Code Detection

Being able to use the headset’s camera feed naturally opens the door for reading QR codes and barcodes, which allow structured data to be transmitted to the headset visually.

Apple is providing a readymade system for developers to detect, track, and read barcodes using Vision Pro.

The company says this could be useful for workers to retrieve an item in a warehouse and immediately know they’ve found the right thing by looking at a barcode on the box. Or to scan a barcode to easily pull up instructions for assembling something.

Neural Engine Access

Enterprise developers will have the option to tap into Vision Pro’s neural processor to accelerate machine learning tasks. Previously developers could only access the compute resources of the headset’s CPU and GPU.

Object Tracking

Although the new Object Tracking feature is coming to VisionOS 2 more broadly, there are additional enhancements to this feature that will only be available to enterprise developers.

Object Tracking allows apps to include reference models of real-world objects (for instance, a model of a can of soda), which can be detected and tracked once they’re in view of the headset.

Enterprise developers will have greater control over this feature, including the ability to tweak the max number of tracked objects, deciding to track only static or dynamic objects, and changing the object detection rate.

Greater Control Over Vision Pro Performance

Enterprise developers working with VisionOS 2 will have more control over the headset’s performance.

Apple explains that, out of the box, Vision Pro is designed to strike a balance between battery life, performance, and fan noise.

But some specific use-cases might need a different balance of those factors.

Enterprise developers will have the option to increase performance by sacrificing battery life and fan noise. Or perhaps stretch battery life by reducing performance, if that’s best for the given use-case.

There’s more new developer features coming to Vision Pro in VisionOS 2, but these above will be restricted to enterprise developers only.