ChatGPT's Theory of Mind: How OpenAI's AI System is Moving Closer to Human-Like Intelligence

The ChatGPT AI chatbot created by Open AI has undergone multiple tests by Michal Kosinski, a computational psychologist at Stanford University, to assess its ability to pass the well-known Theory of Mind Test. In his paper published on the arXiv preprint server, Kosinski notes that the latest version of ChatGPT managed to pass the test at the same level as a typical nine-year-old child.

The capacities of AI chatbots like ChatGPT are incredibly sophisticated, including writing complete essays for secondary and higher education students. With continuous advancements in their abilities, some people have observed that communicating with certain software applications is nearly identical to engaging with an unknown human. As a result, there are questions in the psychology community about the potential consequences of such apps for both individuals and society. In this recent research, Kosinski has explored whether these chatbots are getting close to passing the Theory of Mind Test.

As the name implies, the purpose of the Theory of Mind Test is to evaluate the theory of mind, which aims to describe or comprehend the mental state of an individual. In other words, it proposes that individuals have the ability to "guess" what is happening in another person's mind based on the available information, but only to a certain degree. For instance, if someone displays a particular facial expression, many individuals can deduce that they are experiencing a specific emotion, but only those who possess certain knowledge about the circumstances that led to the facial cues will likely understand the reason for it, and as a result, be able to predict the thoughts in that person's head.

Previous investigations have indicated that these abilities arise during childhood and improve with age. As a result, the study of these theories has resulted in the development of assessments to evaluate them. For instance, one test involves providing one person with a box labeled to describe its contents, only for the person to discover upon opening it that it is something entirely different. Then, a matching box is given to another individual, while the first person is asked to forecast what is going on in the second person's mind—namely, that the second individual will assume that the box contains what is indicated on the label.

Kosinski tested an earlier version of ChatGPT that was released prior to 2022 and determined that it had no capability to pass Theory of Mind tests. Subsequently, he tried a later version that was released shortly afterward and found that it was capable of solving around 70% of the theoretical evaluations—equivalent to the capabilities of a 7-year-old child. Then, in November of the same year, he evaluated the most recent version and observed that it had the capacity to solve 93% of the tasks—roughly similar to the abilities of a 9-year-old child.

Microsoft has integrated ChatGPT functionalities into its Bing chatbot and appears to have acknowledged the findings. Bing's AI chatbot has reportedly instituted a filter on correlated inquiries—if questioned about its capacity to pass the Theory of Mind test, it responded with, "I'm regretful, but I would rather not continue with this discussion. As I'm still learning, I value your understanding and forbearance."