Microsoft just showed how artificial intelligence can find its way into many software applications — by coding quickly.
In Microsoft Build developer conference today, the company’s chief technology officer, Kevin Scottdemonstrated an AI helper for the game Minecraft. Non-player characters in the game are powered by the same machine learning technology that Microsoft has been testing for automated software code generation. The feat hints at how recent advances in AI could transform personal computing in the years to come by replacing the interfaces where you tap, type, and click to navigate to the interfaces you simply need. conversation.
The Minecraft agent responds appropriately to entered commands by converting them into code that works behind the scenes using the software API for the game. The bot-driven AI model was trained on large amounts of code and natural language text, and then showed the API specifications for Minecraft, along with some usage examples. For example, when a player asks it to “come here”, the AI model below generates the code needed for the agent to move towards the player. In the demo shown at Build, the bot can also perform more complex tasks, like taking items and combining them to create something new. And because the model has been trained in natural language as well as code, it can even answer simple questions about how to build things.
While it’s not clear how reliably the system can work outside of the demo, the same tricks can be used to make other apps respond to spoken or typed commands.
Microsoft has built an AI encryption engine called GitHub Copilot on the same technology. It automatically suggests code when a developer starts typing or in response to comments added to a piece of code. Scott said Copilot is the first example of what will likely be a range of “AI-first” products in the coming years, from Microsoft and others. Coding AI “allows you to think about software development in a different way — so you can express an intent about something you want to accomplish,” he said.
Scott doesn’t provide specific examples, but this could one day mean a version of Windows locating a particular document and emailing it to a colleague when you request it, or a version Excel is imbued with AI to turn datasets into graphs when you ask. “We’re going to see lots and lots of big productivity wins on all sorts of routine cognitive tasks that none of us particularly enjoy,” says Scott.
In recent years, AI has proven adept at tasks like image classification, audio transcription, and text translation. Recent algorithmic advances, combined with vast amounts of computing power, have yielded new AI programs capable of performing more complex feats, including produce coherent text—Such as computer code.
Minecraft bots built using an AI model called Codex developed by OpenAIan AI company that received funding from Microsoft in 2019. The Codex is trained in natural language text pulled from the web, as well as billions of lines of code from GitHub, a popular repository for software owned by Microsoft.
Microsoft says Microsoft’s Copilot was made available to a limited number of testers in June 2021 and is currently being used by more than 10,000 developers who are producing an average of about 35% of their code in these languages. Popularities like Python and Java use Copilot, Microsoft said. The company plans to make Copilot available to anyone to download this summer. To build something like a Minecraft bot, developers will need to work with the underlying AI model, the Codex.
Both Codex and Copilot are cause for concern for developers who fear that they might have their work automated. Minecraft demos may inspire similar concerns. But Scott says that feedback on Copilot has been largely positive, suggesting that it simply automates more tedious coding tasks. “If you talk to a developer who actually uses Copilot, they’ll say ‘this is a great tool,’” he says.
