Copilot Vision and Voice enable task automation across productivity apps

Users will soon be able to utilize artificial intelligence (AI) assistant features on their computers. Microsoft (MS) has announced plans to integrate AI capabilities into 'Windows 11.' While companies like Google and Samsung have previously released 'AI smartphones' with built-in AI features, the era of 'AI PCs' -- where AI communicates seamlessly with users and performs tasks on their behalf -- has now arrived.

On the 16th, local time, MS unveiled details of the Windows update centered around AI assistant functionality. The core addition is 'Copilot Vision,' an AI feature that recognizes on-screen content. Copilot can now better assist with tasks by recognizing what the user is viewing, such as guiding app usage. Notably, it is fully integrated with productivity apps like Word, Excel, and PowerPoint, automatically reviewing entire slides or document content. For instance, if a user displays a draft report and says, "Copilot, translate the English text in this document into Korean and check for typos," Copilot will recognize the screen and execute the request.

Additionally, the 'Copilot Voice' feature, which recognizes voice commands, has been added. Similar to interacting with OpenAI's ChatGPT via text or voice, users can initiate conversations with the AI assistant by typing or saying, "Hey Copilot."

These features are available on all Windows 11 devices that support Copilot. To enhance accessibility, MS has added an 'Ask Copilot' button to the taskbar, enabling users to seek AI assistance at any time.

MS has also experimentally introduced 'Copilot Actions,' described as a "smarter AI assistant." This feature allows AI to perform tasks directly based on user commands, such as organizing files or extracting specific information from PDFs. These capabilities are currently in testing and available only to a limited group of users.

Microsoft Integrates AI Assistant into All Windows 11 Devices