Robot chef learns to cook by watching people make the recipes

As a lover of good food, but someone who doesn’t like to cook, I’ve always fantasized about having a robot chef in my house. Now, thanks to the work of researchers at the University of Cambridge, my dream may soon come true.

The research team has succeeded in training a robot to watch cooking videos, learn from them and then recreate dishes. “We wanted to see if we could train a robot chef to learn in the same step-by-step way that humans do – by identifying the ingredients and how they go together in the dish,” says Grzegorz Sochacki from Cambridge’s Department of Engineering, the study leading author.

To test that, the scientists created eight simple salad recipes and recorded videos of themselves preparing them. They then trained the robot using a publicly available neural network, which was pre-programmed to identify various objects, including the fruits and vegetables used in the salads.

The robot chief analyzed every frame of the video using computer vision methods. Not only was it able to identify various objects, such as knives and ingredients, but also the actions of the human demonstrator. The recipes and the videos were converted into vectors and by making mathematical correlations, the robot recognized similarities between a demonstration and a vector. This allowed it to be determined which of the recipes were prepared.

robot chef