MIT Wants to Teach Robots to Do Your Chores

Teaching a robot basic human tasks is more of a challenge than it seems. To teach a robot to pour you a glass of orange juice, for instance, the 'bot has to not just recognize the command to take the juice out of the fridge and pour it into a glass, but it has to understand the many tiny aspects of the task that the human brain infers—like, say, the steps where you have to walk into the kitchen, open the cupboard, and grab an empty glass.

VirtualHome, a 3D virtual environment created by MIT's Computer Science and Artificial Intelligence Laboratory with researchers at the University of Toronto, is designed to teach robots exactly how to accomplish household tasks like pouring juice. The simulator acts as a training ground for artificial intelligence, turning a large set of household tasks into robot-friendly, sequence-by-sequence programs.

First, researchers created a knowledge base that the AI would use to perform tasks [PDF]. The researchers asked participants on Amazon's Mechanical Turk to come up with descriptions of household activities, like making coffee or turning on the television, and describe the steps. Their descriptions naturally didn't include some of the steps that a robot would need, since they were composed as if speaking to another human—the "watch TV" command didn't include some obvious steps a robot might need, like "walk over to the TV" or "sit on the sofa and watch." They then had the same participants generate programs for these tasks using a simple system designed to teach young kids how to code. All told, they created more than 2800 programs for household tasks.

Then, the researchers tested these programs in a Sims-inspired virtual home to see if the crowd-sourced instructions could work to train robots. They turned the programs into videos in which a virtual agent would execute the household task based on the code.

The researchers were focused on creating a virtual environment that could serve as a dataset for future AI training, rather than training any actual robots right now. But their model is designed so that one day, artificial intelligence could be trained by someone who isn't a robotics expert, converting natural language commands into robot-friendly code.

In the future, they hope to be able to turn videos from real life into similar programs, so that a robot could learn to do simple tasks by watching a YouTube video. An artificial intelligence system like Amazon's Alexa wouldn't need to be programmed by its manufacturer to do every single task—it could learn on the fly, without waiting for a developer to create a new skill.