Content provided by IBM and TNW.
Today’s AI systems are rapidly evolving to become man’s new best friend. We now have AIs that can craft award-winning whiskey, write poetry, and help doctors perform highly accurate surgical procedures. But one thing they can’t do – which at first glance is much simpler than all those other things – is use common sense.
Common sense differs from intelligence in that it is usually something innate and natural in people that helps them navigate daily life, and cannot really be taught. In 1906, philosopher GK Chesterton wrote that “common sense is a wild thing, savage and beyond rules.”
Greetings, Humanoids
Sign up for our newsletter now for a weekly roundup of our favorite AI stories delivered to your inbox.
Robots, of course, run on algorithms that are just that: rules.
So no, robots can’t use common sense just yet. But thanks to current efforts in the field, we can now measure an AI’s core psychological reasoning ability, bringing us one step closer.
So why does it matter if we teach AI common sense?
The bottom line is that common sense will make AI better at solving real-world problems. Many argue that AI-driven solutions are designed for complex problems, such as: diagnosis of Covid-19 treatments often fail, for example, because the system cannot easily adapt to a real-life situation where the problems are unpredictable, vague and undefined by rules.
Common sense includes not only social skills and reasoning, but also a “naive sense of physics.”
Injecting common sense into AI could mean big things for people; better customer service, where a robot can actually help a disgruntled customer, other than send them into an endless “Choose from the following” loop. It can make autonomous cars more responsive to unexpected incidents on the road. It might even help the military information about life or death signs of intelligence.
So why have scientists so far failed to crack the common sense code?
Called the “dark matter from AICommon sense is critical to the future development of AI and hitherto elusive. In fact, equipping computers with common sense has been a goal of computer science from the beginning; in 1958, pioneering computer scientist John McCarthy published a paper titled “Common Sense Programs”, which looked at how logic could be used as a method of representing information in computer memory. But since then, we haven’t come much closer to realizing it.
Common sense includes not only social skills and reasoning, but also a “naive sense of physics” – this means that we know certain things about physics without having to work through physics equations, like why you shouldn’t place a bowling ball on a slanted surface. It also includes basic knowledge of abstract matters such as time and space, allowing us to plan, estimate and organize. “It is knowledge that you should to have,” say Michael Witbrock, AI researcher at the University of Auckland.
All this means that common sense is not one precise thing and therefore cannot be easily defined by rules.
Secret agent
We’ve found that common sense requires a computer to infer things based on complex real-world situations — something that’s easy on humans and starts to form from childhood.
Computer scientists are making (slow) but steady progress in building AI agents that can infer mental states, predict future actions, and work with humans. But to see how close we are to each other, we first need a rigorous benchmark for evaluating an AI’s “common sense,” or its psychological reasoning ability.
Researchers from IBM, MIT and Harvard have created just that: AGENT, which stands for Aaction-Gul-Eefficiency coNvoltage-uTility. After testing and validation, this benchmark has been shown to be able to evaluate the core psychological reasoning ability of an AI model. This means it can provide a sense of social awareness and interact with people in a real world.
To demonstrate common sense, an AI model must have built-in representations of how people plan.
So what is AGENT? AGENT is a large-scale dataset of 3D animations, inspired by experiments studying cognitive development in children. The animations show someone interacting with different objects under different physical constraints. According to IBM:
“The videos consist of several trials, each containing one or more ‘familiar’ videos of an agent’s typical behavior in a particular physical environment, combined with ‘test’ videos of the same agent’s behavior in a new environment, which are labeled as either ‘expected’ or ‘surprising’, given the agent’s behavior in the accompanying introductory videos.”
A model must then assess how surprising the agent’s behavior in the “test” videos is, based on the actions it has learned in the “trust” videos. Using the AGENT benchmark, that model is then validated against large-scale trials of human assessments, where people rated the ‘surprising’ ‘test’ videos as more surprising than the ‘expected’ test videos.
Common sense?
IBM’s trial shows that to demonstrate common sense, an AI model must have built-in representations of how people plan. This means combining both a basic feel for physics and ‘cost-rewards’, meaning understanding how people take actions”based on utilitybalancing the rewards of the goal against the costs of achieving it.”
While not perfect yet, the findings show that AGENT is a promising diagnostic tool for developing and evaluating common sense in AI, something IBM is also working on. It also shows that we can use similar traditional developmental psychology methods to those used to teach human children how objects and ideas are related.
In the future, this could significantly reduce the need for training in these models, helping businesses save computing power, time and money.
Robots do not yet understand human consciousness, but with the development of benchmarking tools such as AGENT, we can measure how close we are getting.
Contents
