We’re at the start of the integration of robotics and large language models, commonly called AI even if that’s not the most accurate description. Some experts think this will presage a “step change” in robotics — the beginning of all-purpose robots imbued with common sense that will be able to do just about anything a human co do, but without needing breaks or coffee. Other equally qualified experts say there’s an enormous gap between showy demonstration models and anything that will actually be useful. When it comes to embodied AI, there is no clear path at this point.
Google DeepMind’s Robotic Transformer 2 (RT-2) is one AI-powered robot that is causing optimism. Trained on enormous internet data sets and extensive video from other robots, RT-2 is able to do some surprising things. For example, it can choose the “extinct animal” in a random group of toys, picking up the dinosaur figure without any more specific information. There’s not a lot of obvious value in that action, which is a common thread when we look at AI-powered robots, but it is still a pretty surprising thing for a robot to be able to do.
Here’s Figure 1 doing something similar, giving a human an apple when he asks for “something to eat.” The robot in this video also says “uh” and repeated a word in what sounds like an entirely human stumble over a word. We find both of these things unlikely in a robot, so we have some qualms about the video. But we can imagine situations in which the ability to make broad classifications of observed objects could be useful for a robot.
“Could be useful”
The fact that things could be useful is a sticking point. AI for Good lists some things that have to be sorted before AI-powered robots can actually do what we’ve imagined they could do:
- Navigate safely in human spaces
- Communicate with natural language
- Have the ability to correct themselves
Embodied AI could theoretically learn these things from simulations and hone their skills from practical experience in the real world.
Looking ahead
Some of the things that have stumped robots — opening doors, folding clothes, sorting objects with natural variations, like fruit — have largely been overcome by now. Embodied AI may be able to speed up the development of robots in ways that are hard for us to imagine now.