Microsoft aims to win over artificial intelligence developers with its new Windows Copilot Runtime

Windows is the most open platform for AI for developers. A keynote presentation by Microsoft CEO Pavan Davuluri (Pavavien)

All of these improvements inside Windows for developers are the very early building blocks for more AI-powered apps on top of its new Arm-powered systems and the NPUs coming from AMD and Intel soon. Microsoft is banking on the upcoming decade of Windows development being impacted by the platform that developers will use to create Artificial Intelligence apps. Onstage at Build today, Davuluri stood in front of a slide that read “Windows is the most open platform for AI,” signaling just how important this moment is for Microsoft.

Adding contextual info to apps that feeds into the database will allow developers to improve the Recall feature in Windows. “This integration helps users pick up where they left off in your app, improving app engagement and users’ seamless flow between Windows and your app,” says Davuluri.

“Windows Copilot Library consists of ready-to-use AI APIs like Studio Effects, Live Captions Translations, OCR, Recall with User Activity, and Phi Silica, which will be available to developers in June,” explains Windows and Surface chief Pavan Davuluri.

Developers will be able to integrate things like filters, portrait blur and other features in their apps with the help of the Windows Copilot Library. Meta is adding the Windows Studio Effects into WhatsApp, so you’ll get features like background blur and eye contact during video calls. Live Captions and the new AI- powered translation feature are available to developers with little to no code.

The Windows 11 Advanced Paste feature for keeping track of your coding projects using the Copilot Plus AI feature and its integration with Git in the file system browser

The recall artificial intelligence feature was demonstrated by Microsoft yesterday, allowing Copilot Plus PCs to store and remember everything that you do on your PC. This is all powered by a new Windows Semantic Index that stores this data locally, and Microsoft plans to allow developers to build something similar.

“We will make this capability available for developers with Vector Embeddings API to build their own vector store and RAG within their applications and with their app data,” says Davuluri.

The Advanced Paste feature, which is included in the PowerToys suite for Windows 11, allows you to copy the contents of your clipboard as you go. You’ll be able to trigger the Advanced Paste menu by pressing Windows Key + Shift + V and, from there, convert your paste to formats like plaintext, markdown, or JSON, using further keyboard shortcuts. The prompt box allows you to modify or summarize the text before you paste it. The catch: you’ll need an OpenAI API key and credits in your OpenAI account for the AI part.

You will soon be able to keep track of your coding projects using the File Explorer, as Microsoft integrates Git into the file system browser. The company said that developers would be able to keep track of their files and commits from within File Explorer. The app supports both 7-ZIP and TAR compression.

Phi-3-Vision: An Artificial Intelligence Family of Models for Real-Time Video Dubing and Analysis on a Mobile Device

Qualcomm’s roughly Mac Mini-sized $899 Snapdragon Dev Kit for Windows has a Snapdragon X Elite chip inside. It has a large amount of memory, ports and storage, though it is not certain if anyone will buy it.

Microsoft Teams will soon have the option to add your own cartoonish characters, like parrots and cutouts of your teammates. admins can limit who can add emojis and they will not be seen outside of your organization. They’re coming in July.

Microsoft’s Edge browser is getting an AI-powered real-time video translation feature that can dub videos from sites like YouTube, LinkedIn, Reuters, and Coursera. The feature works with a handful of languages, offering translation from Spanish to English or vice versa — or from English to German, Hindi, Italian, and Russian. A new feature will soon be added by Microsoft and more languages and video platforms will follow in the future.

It was announced in April that there was a new version of the AI model. It’s multimodal and can read text and look at pictures, but it’s a small language model that’s compact enough to work on a mobile device. One of the biggest use cases that artificial intelligence companies have been pushing has been image analysis, and the ideal place to do it is on a phone. Phi-3-vision is part of Microsoft’s Phi-3 family of models that the company announced in April and is available in preview now.