All the news on Microsoft’s latest Copilot and Windows AI features

At Microsoft’s New York City event today, it’s announcing an all-new Copilot experience. The new Copilot design includes a new card-based look across mobile, web, and Windows. Copilot is getting more personalized with features like Copilot Vision, which adds the ability to see what you’re looking at, an OpenAI-like natural voice conversation mode, and a virtual news presenter mode that can read the headlines to you.

Windows 11 is getting new features like Phone Link status in the start menu that can show notifications and your phone’s battery life. And both Paint and Photos are getting fun new features like Generative Fill and Erase. Copilot Plus PCs are getting a revamped AI-powered Windows Search that includes a Google Circle to Search-like “Click to Do” feature and the ability to search for a photo using just a text description.

Earlier this year, an exec reorganization put Pavan Davuluri in charge of Windows and Surface and made Mustafa Suleyman the new CEO of AI. Now, a full year after Panos Panay’s abrupt departure, we will find out more about where Microsoft’s “AI PC” push is headed.

You can read all the updates from the event below.

• Read Microsoft’s optimistic memo about the future of AI companions

  

  Image: Getty Images

  Microsoft is launching a redesigned version of Copilot today, intent on becoming an AI assistant or companion. To celebrate this, Microsoft’s new AI CEO, Mustafa Suleyman, has penned a 700-plus-word memo on what he describes as a “technological paradigm shift” toward AI models that can understand what humans see and hear.

  Suleyman joined Microsoft earlier this year as the CEO of its new Microsoft AI division, amid the software giant’s hiring of a number of key Inflection AI staff. In June, Suleyman sparked controversy after brazenly claiming that anything published on the web is “freeware” that can be copied, recreated, and reproduced by AI models. Now, he’s optimistic that AI — under Microsoft’s stewardship — will create a “calmer, more helpful and supportive era of technology, quite unlike anything we’ve seen before.”

  Read Article >

• Microsoft is using AI to improve Windows search

  

  Illustration by Alex Castro / The Verge

  Microsoft is using AI models to greatly improve Windows search on its new Copilot Plus PCs, including the addition of a new Click to Do feature that’s very similar to Google’s Circle to Search. These search improvements will make it easier to find and interact with images, emails, documents, and even videos and are just a few of the AI-based features coming to Copilot Plus PCs starting in November.

  The improved Windows search will first show up in File Explorer on Copilot Plus PCs next month, allowing you to search for pictures using words, even if the search word isn’t found in the photo or file name.

  Read Article >

• Microsoft Paint is getting Photoshop-like generative AI fill and erase features

  

  Generative Erase is like Google Pixel’s Magic Eraser feature but for your desktop PC.

  Image: Microsoft

  Microsoft is bringing some new AI-powered Paint and Photos features to Copilot Plus PCs that could make creatives less reliant on more powerful image editing software. Generative Fill and Generative Erase — which appear to be heavily inspired by similar AI tools in Adobe Photoshop — are being introduced to Paint, allowing users to precisely add or remove objects in their images.

  Both tools utilize a size-adjustable brush to “paint” over specific areas of an image to edit. Generative Erase will remove unwanted figures, objects like background clutter, and other distractions, similar to the Magic Eraser feature on Google’s Pixel phones. Generative Fill allows Paint users to add new AI-generated assets to an image using a text description and select precisely where they should be placed — much like the Photoshop tool that shares the same name.

  Read Article >

• Microsoft starts rolling out its Windows 11 2024 update with lots of useful improvements

  

  Image: Microsoft

  Microsoft is starting to release its Windows 11 2024 update today, also known as version 24H2. This update includes a number of small but useful additions to Windows 11 that improve the Start menu, File Explorer, Settings, and much more.

  The Start menu has a big change for those using Microsoft’s Phone Link software. The Windows 11 2024 update adds a side panel to the Start menu that provides information on your phone’s battery status and notifications as well as quick access to messages, calls, and photos. The floating panel is a neat addition for Phone Link users.

  Read Article >

• Microsoft gives Copilot a voice and vision in its biggest redesign yet

  

  Image: Microsoft

  Microsoft is unveiling a big overhaul of its Copilot experience today, adding voice and vision capabilities to transform it into a more personalized AI assistant. As I exclusively revealed in my Notepad newsletter last week, Copilot’s new capabilities include a virtual news presenter mode to read you the headlines, the ability for Copilot to see what you’re looking at, and a voice feature that lets you talk to Copilot in a natural way, much like OpenAI’s Advanced Voice Mode.

  Copilot is being redesigned across mobile, web, and the dedicated Windows app into a user experience that’s more card-based and looks very similar to the work Inflection AI has done with its Pi personalized AI assistant. Microsoft hired a bunch of folks from Inflection AI earlier this year, including Google DeepMind cofounder Mustafa Suleyman, who is now CEO of Microsoft AI. This is Suleyman’s first big change to Copilot since taking over the consumer side of the AI assistant.

  Read Article >

• Microsoft teases a new Copilot design.

  If you open up Copilot today on the web you’ll find a teaser for an “updated Copilot” that Microsoft says will be faster, simpler, and more personalized. I exclusively revealed last week in my Notepad newsletter that Microsoft is getting ready to launch a completely redesigned Copilot that will even read you the daily news headlines. Microsoft has also teased that it has some “exciting announcements” coming today.

  
  

  

  

  Image: Microsoft

• Microsoft has some ‘exciting announcements’ today.

  Yusuf Mehdi, executive vice president and consumer chief marketing officer at Microsoft, is teasing some “exciting announcements” from the company today. Microsoft last held a “special event” in New York City in September 2023 that focused on Copilot and Surface hardware. The software giant hasn’t announced a similar event for this year, but something is clearly happening today 👀

  
  

• 

  Microsoft is turning Copilot into an AI presenter to read you the news

  

  Image: The Verge

  Microsoft is working on an overhaul of its Copilot mobile app that includes a new feature that will transform the AI assistant into a virtual news presenter. Multiple sources familiar with Microsoft’s plans tell me that the software giant has been testing a completely redesigned Copilot app in recent weeks that looks unlike any of Microsoft’s other apps.

  The Copilot redesign surfaces topics you can choose from based on your own interests or your history of asking Copilot questions. The AI assistant might offer to generate a story for you one day, ask if you to do a workout the next, or simply surface the latest sports scores without you having to ask for them.

  Read Article >