UPGRADE TO AI
Apple announced a raft of AI tools and features aimed at helping it keep up with competitors like Google and Microsoft, which have boasted in recent months about why AI makes their phones, laptops and software better than Apple’s. The deal with OpenAI integrates ChatGPT into Apple’s products, including the Siri voice assistant. For OpenAI, the deal is a validation of the importance of its tech and puts its tools in front of millions of iPhone users who might not know about or want to use its chatbots and image generators directly.
Now, onto the announcements.
The big stuff
- APPLE INTELLIGENCE:
- CEO Tim Cook promises it will be powerful, intuitive, integrated, personal, and private (PIIPP?)
- iOS 18, iPad 18, and MacOS Sequoia will debut a new personal intelligence system that puts generative models at the core of your experience, to give you relevant information and protect your privacy in every step
- The A17 Pro chip, and the M1 through M4 chip, all support Apple Intelligence (that’s iPhone 15 and later, and all Apple Silicon Macs)
- Siri can plug into ChatGPT (more on this below) for free, but if you already subscribe, you’ll get access to its premium features
- Capabilities:
- At its core, Apple Intelligence can create language and images, and take action
- It can prioritize your notifications to make sure you don’t miss anything important
- Writing tools can edit and rewrite your notes, messages, and essays
- You can create original images in unique styles to make everyday conversations more enjoyable (like sending a picture of your mom in a superhero costume to wish her a happy birthday)
- It can take action, like “play the song that Ray texted me,” or “flip my camera”
- It understands personal context so when you make a request, it can look at all of your relevant personal data: Let’s say a meeting gets rescheduled and you’re wondering if you’ll be able to make it to your daughter’s concert in time, Apple Intelligence can look at past messages, times, dates, and even traffic information, to help you problem-solve
- Experiences:
- Siri is getting a major upgrade after its debut 13 years ago to be more natural, contextually relevant, and personal to you
- Siri looks different: It’s now a glowing light that wraps around the edge of your screen
- Even if you stumble over your words, Siri will understand what you’re talking about
- Even if you don’t know the name of a particular iPhone feature, you can describe what you want to do and Siri will help you out
- On-screen awareness: If your friend sends you an address, just say “add this address to his contact card” and Siri will automatically know who “his” is and do the work for you
- Siri can take hundreds of new actions in and across apps, like asking for photos of your friend in New York wearing her pink coat. If you say “make this pop,” it’ll enhance your photo.
- Siri can draft a summary of your meeting notes and send them off to your teammates
- Thanks to its ability to index all of your apps and data, Siri can find and look through things it never could before, like finding information from an email sent years ago, filling out forms from your ID or driver’s license, and more
- Siri can rewrite messages so you can choose which message you like best, which is incredibly helpful for people who want their cover letters to come across perfectly
- Apple Intelligence can change the tone of your writing to be more friendly, or even rewrite your message as a poem
- Siri can write a TL;DR for a long message at the top of your messages to coworkers and friends by summarizing your notes in bullet points; it can also summarize emails sent to you to help you cut to the chase
- Notifications are simplified, prioritized, and summarized to help you reduce noise
- GENMOJI! Instead of using emoji to represent how you feel, you can create an emoji of your own design (!) just by describing what you want, like a T-rex riding a wave on a surfboard
- Playground lets you create brand-new AI images in three different styles (animation, sketch, or illustration) and gives you a preview of an image you’ve described before sending it off to your friends via Messages
- A dedicated Image Playground app lets you experiment with creating your own images to share via Messages, Freeform, Keynote, or social media
- If you make an original sketch of your own, just circle it and you can create an AI-generated spiffy version of that same image
- A Clean Up tool in the Photos app can help you remove unwanted aspects of your photos
- It’s easier to search photos, and videos, or create a memory movie just by describing what it is you want to see
- You can record and transcribe audio in the Notes app, and when it’s done, Apple Intelligence gives you a summary
- Siri can tap into ChatGPT to help you answer certain questions, so if you have a bunch of ingredients and want an interesting recipe, you can confirm it’s okay to share your question with ChatGPT, and it’ll bring you answers without leaving your conversation with Siri
- ChatGPT can help you create bedtime stories, and even illustrate your bedtime story (!)
- iOS 18, the newest iPhone software coming this fall, comes with:
- The ability to customize app widgets and apps, including the ability to change the color tint of all your app icons
- A Dark Mode for the entire iOS homescreen interface
- A new customizable Control Center: multiple pages to swipe between your phone, home, and music controls; the ability to resize your controls; and a new gallery to choose which controls you want to see and toggle
- The ability to lock an app, so friends and family can’t open certain apps or see sensitive information (this is going to test a lot of people’s trust in their relationships)
- The ability to react to any message in Messages with any emoji or sticker and you can format text to be bold, underlined, and more
- The ability to schedule Messages (!)
- The Mail app intelligently 👀 separates your mail into categories: primary mail, transactions, social media updates, and promotions
- The ability to exchange Apple Cash without needing to share phone numbers or email addresses; just hold your phones together to pay
- A redesigned Photos app that organizes your photos into categories like trips, people & pets, and more; a carousel for your favorite photos and featured photos (surfaced by the app)
The smaller stuff
- You can nod your head yes, or shake your head no, to interact with Siri using your AirPods—for those occasions where you don’t want to speak in public, for instance
- AirPods Pro gets voice isolation to intelligently 👀 remove background noise and enhance call quality
- Apple TV+ is celebrating its fifth anniversary this year. The company showed off some new original movies and shows featuring big names like Matt Damon, Vince Vaughn, Harrison Ford, and more
- Apple TV+ gets a new feature called InSight, which can tell you which actors are on screen, or what song is playing (like Amazon’s X-Ray feature for Prime Video)
- Mac computers get new software, called MacOS Sequoia, which comes with:
- iPhone Mirroring, which lets you see what’s on your iPhone—and control it—on your Mac
- The ability to place all of your windows in smart tiles and arrange them, so you don’t have to resize everything by hand all the time
- A new Passwords app, which syncs across your devices; no more needing to dig through your Settings app
- WatchOS 11, the new software for Apple Watch, comes with:
- The ability to intelligently 👀 surface information it thinks you’ll need with the Smart Stack, like when your Uber is arriving, or the translation app if you’re traveling
- An effort rating to tell you how hard you’re working out compared to your average
- More details about your metrics and vitals to help you understand how changes may be linked to other aspects of your life like elevation or illness
- Cycle tracking now supports gestational age to help you track your pregnancy
- iPadOS 18, the next-generation iPad software, comes with:
- All of the big iOS 18 features, including a customizable Control Center and home screen
- A floating app bar to improve navigation, which can be customized so each app can feel distinct
- The ability to share your screen with someone else, so you can teach a friend how to do things on their iPad with your own (grandparents rejoice!)
- Math notes, where you can use handwriting to do complex math—and your iPad will solve your math problems for you, using your own handwriting (!), once you scribble an equal sign (RIP homework)
- The ability to refine your handwriting as you scribble and take notes. It’ll also spellcheck for you. You can also paste text and it’ll appear in your handwriting (this is bonkers)
- Apple Vision Pro is getting its first major software update since its February release. VisionOS2 offers:
- The ability to create spatial photos from photos you’ve already taken
- Show you the time and battery level with simple hand gestures
- Travel Mode, so you can keep working or watching entertainment in a moving vehicle
- New developer APIs and frameworks to support experiences for the enterprise or tabletop
- Support for third parties to create spatial video experiences, so you can shoot immersive video with a special Canon camera, or enjoy 3D videos on sites like Vimeo
- New immersive content, including an Apple short film, and videos from Red Bull and The Weeknd
- Apple Vision Pro is coming to more countries, including Japan, Singapore, China, Australia, Canada, and more, this summer
Apple finally showed off its contribution to the AI wars, and there are some pretty cool use cases for it.
The company made a splash with its AI announcement for Apple Intelligence during a keynote Monday at its annual WWDC event. CEO Tim Cook called Apple Intelligence "the new personal intelligence system that makes your most personal products even more useful and delightful."
Apple Intelligence can be used across iPhones, iPads, and Macs. On your phone, for example, it can organize notifications by priority to make sure you don't miss something important. New writing tools available through Apple Intelligence can also help rewrite, proofread, and summarize text for you across apps like Mail, Notes, and Safari.
Apple Intelligence can also lend a hand when it comes to image generation, another hot frontier for the AI race. You can create personalized images to add to your conversations in one of three styles — sketch, illustration, or animation.
One example shown in the demo highlights that Photos learns to recognize people regularly in your pictures. For example, it can create an image of your friend blowing out candles that you can send when you wish them a happy birthday.
This works in apps like Messages, Notes, Freeform, Keynote, and Pages.
Apple Intelligence can also help with tasks that require knowledge of your "personal context," said Craig Federighi, Apple's senior vice president of software engineering.
"Apple Intelligence is grounded in your personal information and context, with the ability to retrieve and analyze the most relevant data from across your apps as well as to reference the content on your screen like an email or calendar event you're looking at."
As an example, Federighi imagined a meeting was rescheduled and he was wondering if it'd prevent him from getting to his daughter's play on time.
Apple Intelligence can "understand who my daughter is, the play details she sent several days ago, the time and location for my meeting, and predicted traffic between my office and the theater," he said in the demo.
Apple Intelligence will also open up a new world of possibilities for Siri, said Kelsey Peterson, Apple's director of machine learning and AI.
It'll allow you to speak more conversationally with Siri; if you stumble on your words or accidentally misspeak before correcting yourself, Siri will still understand what you mean.
Siri will also maintain conversational context, so you can follow up in a conversation without having to spell everything out for Siri again in each question or command you give.
If you don't want to talk to Siri, you'll be able to double-tap at the bottom of your screen and type your questions or commands in there.
In addition, you can ask Siri questions about settings or features on your iPhone, even if you don't know their specific name, and she'll show you the answer or relevant result in the iPhone User Guide.
Apple Intelligence will also give Siri on-screen awareness to understand and act upon what's on your screen.
If a friend texts you his new address, for example, you can tell Siri from the Messages app, "Add this address to his contact card." Siri will take the address from the message on-screen, as well as the name of your friend, and carry out the task.
In the demo, Siri was also able to handle a request to "show me my photos of Stacey in New York wearing her pink coat" and surface the photos.
Managing your email app — and even just searching for an email you know is in there somewhere — can be unbearable.
But Apple is trying to change that.
At Apple's WWDC event on Monday, company executives announced several new features that will likely make the Mail app much easier to use across Apple devices, and even make it function more like an executive assistant.
Here's what you need to know about the latest email updates Apple is rolling out.
Mail app categorization
Apple is introducing new categories into its Mail app that will help you better manage and organize incoming emails.
Some of this may sound familiar to Gmail users, but there are also some differences.
A "primary" category will gather together your most important messages at the top, like emails from people you know and time-sensitive emails, Ronak Shah, Apple's director of internet technologies product marketing, said at the event.
Everything else will be divided into new categories like "'transactions' for receipts and order confirmations, 'updates' for newsletters and social media notices, and 'promotions' for marketing and sales messages," Shah explained.
In addition to these categories, the Mail app will also soon have a "digest" view that will assemble all emails from a business in one place. For example, all your emails from United Airlines will appear in one grouping so you can see all your flight information without having to search or scour your inbox.
You'll also still be able to see all your emails in one place, and you can recategorize your messages as much as you want, Ronak said, adding that the categorization feature will become available later this year.
Writing tools integrated into Mail
Thanks to Apple's new generative AI system, Apple Intelligence, some smart AI-powered writing tools will be integrated across Apple apps, including the Mail app.
With Apple Intelligence, you'll be able to pull up a sidebar from the Mail app that will help you rewrite an email draft. For example, you can choose options to make the tone of your email more "friendly," more "professional," or more "concise."
You can also choose the "summary" or "key points" options to add TL; DRs to your emails.
Smart Reply can also help you identify and answer questions you were asked in a previous email.
And, in the Mail app, you'll also be able to see summaries of emails instead of just short previews.
New Siri capabilities with the Mail app
Apple's new and improved Siri can also help you track down that hard-to-remember item.
When you're trying to remember something, like a book recommendation a friend sent you — but can't remember if it was from an email or a text message or a shared note — Siri will be able to surface it down for you, Kelsey Peterson, Apple's director of machine learning and AI, said at the conference.
You'll also be able to ask Siri things like, "When is my mom's flight landing?" And in response, Siri will find the email with your mom's flight information, and cross-reference it with real-time flight-tracking info to give you her most accurate arrival time, Kelsey explained.