Facebook researchers published a paper in July on training an AI assistant using the wildly popular video game Minecraft. Their “virtual assistant bot” is available for people to download, and will learn from humans communicating with it in-game. The researchers say Minecraft is a good place for an AI to learn about human language because it’s a rigid environment in which players have almost endless free rein in what the do and build.
Facebook is hoping it can train an AI assistant to understand a broad range of human commands with a little help from one of the biggest games in the world — Minecraft.
A group of Facebook researchers published a paper in July explaining why they think Minecraft is the perfect place for an AI to learn about human communication. The key lies in the fact that Minecraft is what’s known as a “sandbox” game, where players can roam around with relatively free rein as to what they want to do or build, while also following a set of relatively simple rules.
The researchers also hope that the natural curiosity of Minecraft players will give the AI plenty of humans to practise with. “Since we work in a game environment, players may enjoy interacting with the assistants as they are developed, yielding a rich resource for human-in-the-loop research,” the paper says. Minecraft has 91 million monthly active users, so the potential pool of humans who could help train the AI is pretty vast.
Players are already able to download Facebook’s Minecraft “virtual assistant bot” on Github. The idea is still nascent, but the researchers hope the bot could get to the stage where a player could tell it what to do in-game, for example “build a tower 15 blocks tall and then put a giant smiley on top.” Although the command may seem simple enough, an AI learning the principles of human language from scratch has a lot to interpret.
The paper also gives an example of a dialogue where a player might help the bot learn what a gargoyle is:
PLAYER: go over to that tower with the gargoyles[assistant does not know what “gargoyles are”, but sees there is only one tower; surmise that that tower must be related to “gargoyles”] [assistant walks to the tower]
ASSISTANT: does this tower have gargoyles?
ASSISTANT: What is a gargoyle?
PLAYER: its the monster on top of the tower [assistant recognizes is it is a part of the tower because of “top” which it already understands]
ASSISTANT: is it this? [assistant moves to the top of the tower]
Games have been used previously to research AI and its capability to learn independently. OpenAI created bots which thrashed the top “Dora 2” e-sports team, and going beyond video games Facebook built an AI capable of beating professional poker players.
Facebook’s Minecraft paper doesn’t give any specific goal for the assistant, and describes itself as a “call-to-arms” for researchers interested in machine-learning, but any insight gleaned from its Minecraft bot could conceivably go towards building a more all-purpose AI assistant. Facebook had a go at introducing an AI assistant, called M, in 2015, but shut down most of its functionality in 2018.
Who knows, perhaps Minecraft could one day give M new life.