How AI could supercharge the Vision Pro

By CNN Newsource

Published June 4, 2024 5:00 AM

By Samantha Murphy Kelly, CNN

(CNN) — Apple’s Vision Pro mixed reality headset was announced one year ago this week, and it’s arguably already due for a major upgrade to add artificial intelligence.

Apple is expected to announce its first batch of generative AI tools for the iPhone and iPad next week, as part of a greater effort to breathe new life into its struggling product lines. Although the Vision Pro mixed reality headset has only been on the market for a few months, some experts believe the same technology could soon find its way onto the device.

Generative AI, the buzzy form of artificial intelligence that can provide thoughtful and thorough responses to questions and prompts, could potentially propel the headset into another level of personalization and immersion, and usher in new use cases for businesses, particularly in the education and medical space. It could also boost sales of the pricey Vision Pro, which has reportedly and unexpectedly received a production cut.

At its annual Worldwide Developers Conference at its Cupertino, California, headquarters, which kicks off Monday June 10, CEO Tim Cook will likely lay out the company’s vision for this type of AI. It’s also expected to partner with ChatGPT maker OpenAI to unlock new layers of interaction across its product lines. All this comes during a time when the company is under pressure to catch up to rivals such as Google and Samsung, which are already using the technology in its smartphones.

The easy way in

The most obvious way to integrate generative AI into the Vision Pro would be through a virtual assistant similar to a much smarter Siri, allowing users to ask questions about what they’re watching or automating tasks such as sending texts by voice or turning on the lights.

Other use cases include real-time language translation, deeper collaboration in a workspace environment or personalized fitness plans and guided meditations, according to experts.

But the experience could be even richer for businesses. During its most recent earnings call, CEO Tim Cook said more than half of the Fortune 100 companies have already bought an Apple Vision Pro. “ … [We] are exploring innovative ways to use it to do things that weren’t possible before”, before moving on to other topics,” he added.

Tuong Nygugen, a director analyst at market research firm Gartner, said any company which is interested in this market “has to be thinking about it.”

“Nobody really knows what’s a winner yet, so they’re pulling together all these use cases right now, with a focus on enterprise, and making a case for why this is the future of experiences and devices,” he said.

But while it’s unclear when generative AI will come to the Vision Pro, Nygugen said: “It’s only a matter of time.”

That’s because the Vision Pro is merely just another interface – much like a tablet, laptop, or watch, that touches Apple’s ecosystem.

“The leg up isn’t in if Apple uses it – because I would expect everyone to use it – or even if they use it first,” Nygugen said. “It’s about how they use it and how they bring value to the user.”

The future of Vision Pro

Apple’s first new product in seven years entered the market in February just as the extended reality (XR) market — a category that includes augmented, virtual and mixed reality — plateaued with little mainstream consumer adoption. The $3,499 headset had limited apps out of the gate and is tethered to a battery pack the size of an iPhone, offering about 2.5 hours of battery life on a single charge.

But both consumers and businesses widely agreed the user experience was unmatched when it comes to watching immersive videos and interacting with the world around you.

Apple has yet to release specific sales data on the device but Apple analyst Ming Chi-Kuo said production was being cut to as low as 400,000 units per year, compared to early industry projections of 700,000. Adding generative AI to the experience, however, could help developers of all skill levels create more engaging content to attract a greater audience, according to Jeremy Bailenson, founding director of Stanford University’s Virtual Human Interaction Lab.

“Although most headsets today are cheaper than smartphones and laptops, there is very limited use by consumers due in part to a lack of compelling content that is designed specifically for headsets like the Vision Pro or the Meta Quest,” he said. “AI will help by allowing people without any programming experience to generate 3D models, immersive scenes, and even avatar bodies and animations.”

In the lab at Stanford, researchers are already using new software which allows a user to describe any scene by speaking out loud, such as “make me a beautiful park in the middle of a big city with cows roaming through it.” The VR results emerge in seconds.

He sees strong use cases around this particularly in education and gaming.

“AI allows teachers to quickly create VR content that fits into their curricula and lesson plans, solving the biggest roadblock of VR in classrooms,” he said. “VR now makes a teacher’s job easier to create lessons, whereas before generative AI, the content creation was so expensive and difficult it often made their jobs harder.”

Generative AI could also change gaming on headsets. Social VR – a concept where users gather together to talk, collaborate and play – tends to be lonely, Bailenson said. Although there are thousands of virtual scenes on a typical platform, there are far fewer users.

“Generative AI can add non-player characters to make places feel less empty,” he said.

Ramon Llamas, a director at market research firm IDC, believes Apple will start with the commercialization of gen AI with Vision Pro, but the bigger impact will ultimately impact businesses.

Practical business applications

For example, he said a field service worker could troubleshoot machinery in real time with the help of the headset and generative AI. In theory, it could examine and summarize the collected data, and provide an interactive diagram, hologram or video that takes the user through how to diagnose and fix the problem, he said.

“This is the bigger play and the magic of generative AI,” he said. “The worker would no longer be looking at a generic video; the tech would go through stacks of files, data, videos and pictures to put something very specific to address the problem.”

Another use case, he said, could be for worker training, such as showing someone how to repair an airplane engine based on the model and other specifications.

“I think we’ll start to see this more in the next couple of product cycles,” he said. “The race is on to see how quickly we can get generative AI into these devices.”

Meta, for example, has already expressed interest in teaming up with Microsoft, which has been heavily integrating its Copilot technology – powered by ChatGPT – into its own suite of products. Llamas said a likely partnership would allow Meta’s Quest headset lineup to utilize Copilot as an assistant or for training purposes. Google might follow in the same footsteps.

Tuong Nygugen, a director analyst at market research firm Gartner, said “anyone who is invested in this market has to be thinking about it.”

For now, however, the focus will likely be on getting people comfortable using generative AI on the devices users are already using, especially the ones they hold in their hand.

“Vision Pro just came out a few months ago and it needs to get its feet under it before it really starts to take off,” Llamas said. “With iOS, we already have a base of users and products, so this is probably where Apple will accelerate now.”

Article Topic Follows: CNN - Business/Consumer

Jump to comments ↓