AS I mentioned in my last blog post, I’ve started to do some work using Generative AI. One of the great things about this is that it’s one of those rare areas of IT where people in the “outside world” are actually aware of AI and what it can potentially do.
People are now sending me examples of clever uses of AI and I have to say that this is the most impressive so far….
In the demo a human has a conversation with a humanoid robot called “Figure One” developed by OpenAI.
The human asked Figure One what it could see in the environment, and Figure One described seeing a red apple on a plate, a drying rack with cups and plates, and the human standing nearby. When the human asked for something to eat, Figure One gave them the red apple, explaining it was the only edible item available on the table.
Figure One then asked the human where the dirty dishes on the table should go next, and the human correctly deduced they should go in the drying rack. When instructed, Figure One picked up the dishes and placed them in the drying rack.
This demonstration is impressive for a few key reasons:
- Environmental awareness: The robot showed it could visually perceive and describe the objects and layout of its environment, including identifying edible items versus dishes etc. This spatial awareness and object recognition is a fundamental requirement for any AI system operating in the real world.
- Task reasoning beyond simple perception: The robot demonstrated basic reasoning about the appropriate actions to take given the context. It inferred that the human wanted food, so it provided the apple. It also correctly deduced that used dishes belong in the drying rack after the human prompted it.
- Instruction following: The robot could understand and follow the human’s spoken instructions, like “put them in the drying rack.” This ability to comprehend natural language commands and translate them into physical actions can be difficult for AI systems.
- Dexterous manipulation: To successfully pick up the dishes and place them in the rack required visual/mechanical coordination and dexterity.
While this is a simple scenario, Figure One exhibited several core competencies that AI and robotics researchers have been working towards for decades – perception, reasoning, language understanding, and dexterity. Integrating all these capabilities into one system that can interact in an environment is an impressive feat of artificial intelligence. It suggests progress towards more capable household/service robots that can understand their surroundings and assist with tasks based on dialogue.
