Everything Apple announced at its big AI event

By Samantha Murphy Kelly, CNN

Cupertino, California (CNN) — Apple just kicked off a brand new era for the company with the introduction of its first generative AI features for the iPhone.

At its annual Worldwide Developers Conference on Monday, the company unveiled the first batch of tools powered by “Apple Intelligence,” from personalized Genmoji – Apple’s AI-generated emoji – to a significantly smarter Siri, which can answer questions about your schedule, what’s in your email and what time your loved one’s flight is landing.

It also announced a partnership with ChatGPT creator OpenAI to power the features. Although the partnership will likely give the company a much-needed boost, Apple aligning itself with a company and a technology that have yet to win public trust, facing criticism for taking user inputs and incorporating them into ChatGPT’s data set may cause some challenges down the road.

Apple emphasized privacy and security during its presentation, noting most AI functions will be done on the phone, keeping inputs away from a far-off server cloud.

A big push into AI could spur growth for iPhone sales and services for years to come, as users are now waiting longer to upgrade their devices and an uncertain economic environment weighs on consumers, particularly in China. The company also faces regulatory scrutiny in Washington and was passed this week by chip maker Nvidia as the second-largest public company in the US.

The company said it’s been impressed with the generative AI tools already on the market, but wanted to make it more personal with privacy in mind.

“As we look to build in these incredible new capabilities, we want to ensure that the outcome reflects the principles at the core of our products, it has to be powerful enough to help with the things that matter most to you,” Apple CEO Tim Cook said during the keynote. “It has to be intuitive and easy to use. It has to be deeply integrated into your product experiences.”

He added: “Most importantly, it has to understand you and be grounded in your personal context, like your routine, your relationships, your communications, and more. And, of course, it has to be built with privacy, from the ground up.”

The timing is also noteworthy: Apple is not always first to adopt and integrate emerging technologies — it typically researches, develops and aims to perfect new tech for years before including it in new products — but the speed at which the world is adopting generative AI is perhaps expediting the company’s need to have a smartphone with the most cutting-edge technology.

Here’s a closer look at what Apple announced at its big event:

AI, AI and more AI

Apple is embracing generative AI – the buzzy form of artificial intelligence that can provide thoughtful and thorough responses to questions – through Siri, the company’s virtual assistant with a hit-or-miss track record, essentially turning it into an iPhone chatbot.

This could enable Siri to perform specific tasks such as recalling a picture taken years ago on the device or answering detailed questions about the weather, the news or trivia. It can also perform more advanced tasks, such as answering when a user’s mom’s plane is landing, by analyzing information previously sent in an email. Over time, it could learn the user’s preferences and respond accordingly.

This is not unlike what some competitors have already introduced in generative tools. Siri will also likely adapt automatically and seamlessly to users, based on voice, audio and natural language, along with images and contextual cues.

Users will also be able to create personalized photos, such as taking a picture of your mom and making it into a stylized, cartoon-y version, adding a superhero cape. It can take action across apps, including asking the software to pull up all photos of a family member, and retrieve and analyze data from across your apps, such as factoring in what’s on your screen like email or a calendar.

If a meeting is being rescheduled, Apple Intelligence can process relevant personal data and see the email your kid sent days before about a recital, alerting users if there may be a conflict.

In a Q&A following the keynote, Cook once again said the company is taking privacy and security very seriously with the rollout of the new technology.

“We think AI’s role is not to replace users but to empower them,” he said. “When you think about what’s possible, it has to be integrated with the experience and be intuitive, and informed by your personal context and knowledge of you. If you’re going to do that, there’s a lot of responsibility.”

Beyond AI

Apple kicked off its iOS 18 portion of the event by highlighting new controls and personalization coming to iOS 18, including a revamped look for all icons when it goes into Dark Mode, a new tint color that compliments wallpaper and a redesigned control center that is available from the lock screen, so you can swap the flashlight icon for other tools.

Apple said it is also doubling down on privacy and security, such as giving users the option to “lock” certain apps, making them accessible only via Face ID, Touch ID or a user’s passcode. The feature could be especially helpful to protect apps like a banking app, an insurance app, or if you’re handing your phone over to show off photos or get someone’s phone number.

Users can now also “hide” an app, so it will show up only in a locked, hidden folder. And media from hidden apps won’t show up anywhere else on their phone.

Text messages are getting a bump, too. The company said it will soon let people send texts via satellite, even if they don’t have a cell or WiFi connection. Apple is also adding what it called one of the most requested features in iMessage: the ability to schedule messages.

IPhone users will now be able to record and create transcripts of calls right from the phone app. All parties to the call will be notified when it is being recorded.

Other product updates

The company also made software updates to other products, including the Mac, AirPods and Apple Watch.

Apple showed off its latest MacOS software called Sequoia, with a handful of new productivity features. Phone mirroring, for example, will bring iPhone alerts directly to the laptop that show up directly next to Mac notifications.

MacOS will also support the ability to arrange windows open on screen in certain areas, such as side by side — a concept popularized by Microsoft Windows.

Video conferencing is getting a presenter preview, so users can see what they’re going to share before they share it. Apple is also introducing background replacements on calls, so users can hide the laundry behind them.

Meanwhile, AirPods users will be able to answer or decline a call with just a nod or shake of their head. And new vital sign tracking on Apple Watch can notify users when they may be getting sick, based on signals like body temperature and heart rate.

Vision Pro update

Just a few months after its launch, Apple showed off a few upgrades coming to the software that powers its Vision Pro mixed reality headset.

During its most recent earnings call, Cook said more than half of the Fortune 100 companies have already bought an Apple Vision Pro. “[We] are exploring innovative ways to use it to do things that weren’t possible before,” he added.

Vision OS 2 promises to offer more rich experiences for users, such as a wider display for work stations, new hand gestures that let you check battery life or open up the home screen, and get deeper inside the photos in your library, thanks to advanced machine learning that provides more natural depth.

The company said it is also adding more capability for users to capture spatial videos.

Apple is also bringing Vision Pro to more countries, including the UK, China, Japan Singapore and Australia, this summer.

Upgrades could also boost sales of the pricey Vision Pro, which starts at $3,499. Demand for Apple’s new headset has reportedly been weak.

CNN’s Clare Duffy contributed to this report

The-CNN-Wire
™ & © 2024 Cable News Network, Inc., a Warner Bros. Discovery Company. All rights reserved.