How a new science is decoding the hidden workings of AI through a multidisciplinary framework for understanding artificial minds
In 2023, large language models like ChatGPT became cultural phenomena, demonstrating capabilities that rival—and sometimes surpass—human performance. They write poetry, write code, and ace professional exams. Yet, a profound mystery lies beneath these impressive feats: we don't truly understand how they work 4 .
"Large language models can do jaw-dropping things. But nobody knows exactly why" 4 .
This knowledge gap presents a critical challenge for researchers, developers, and society at large. As Matthew Hutson noted in Nature, the opacity of AI systems raises serious concerns about their reliability, potential biases, and the ethical implications of their widespread use 4 .
In response to this pressing need, researchers have proposed a novel multidisciplinary framework called psychomatics 1 . This emerging field, whose name fuses "psychology" and "informatics," aims to bridge the gap between artificial and biological intelligence by combining insights from cognitive science, linguistics, and computer science 7 .
Modern AI systems demonstrate remarkable capabilities while remaining largely opaque in their internal decision-making processes.
Psychomatics represents a formal approach to studying artificial minds, particularly large language models (LLMs). It establishes a systematic methodology for exploring how AI systems acquire, learn, remember, and use information to produce their outputs 1 7 .
The central question driving psychomatics is deceptively simple: Is the process of language development and use different in humans and LLMs? 1 7 To answer this, researchers employ a comparative methodology that examines not just observable behaviors but the underlying cognitive and processing frameworks that facilitate these behaviors 7 .
Grammatical structure analysis comparing how humans and AI process language rules and patterns.
Meaning analysis examining how symbols and words convey meaning in both biological and artificial systems.
Allowing the model to understand syntagmatic relations by weighing the importance of individual words in context 7 .
Simulating associative relations by linking concepts across different sequences, similar to how human language understanding processes relationships between words without a linear structure 7 .
Through systematic comparison, psychomatics has identified several fundamental distinctions between human and artificial cognition:
Humans acquire language through a gradual process of social, emotional, and linguistic interactions that begin in infancy and continue throughout life. In contrast, LLMs are trained on vast, pre-existing datasets in a relatively short time frame 4 .
Human cognition is deeply rooted in physical embodiment and direct experiences with the world. Our understanding of language and concepts is shaped by our sensory perceptions, emotions, and interactions with our environment. LLMs, lacking physical bodies and sensory experiences, rely solely on statistical patterns in their training data to approximate meaning 4 .
Humans can use imagination and combine existing knowledge in unique ways to generate entirely new ideas and meanings. LLMs, while adept at recombining existing information in impressive ways, are ultimately limited to the patterns present in their training data. They cannot truly create novel meanings in the way that humans can 4 .
Humans excel at interpreting subtle contextual cues, understanding sarcasm, and navigating complex social situations. While LLMs can often produce contextually appropriate responses, they struggle with more nuanced aspects of communication, such as detecting sarcasm or understanding social faux pas without explicit training in these areas 4 .
Perhaps most importantly, psychomatics reveals that while LLMs can produce output that aligns with Grice's Cooperative Principle (making conversation relevant and informative), they do so through learned statistical correlations rather than genuine understanding 7 . Human communication draws from multiple sources of meaning—experiential, emotional, and imaginative—that transcend mere language processing and are rooted in our social and developmental trajectories 1 .
To illustrate how psychomatics research operates in practice, consider a groundbreaking Stanford study that simulated the personalities of 1,052 individuals with impressive accuracy 6 .
The researchers designed a comprehensive approach to create what they term "generative agents" that could accurately mirror real human personalities, beliefs, and decision-making patterns 6 :
1,052 participants representative of U.S. population demographics
2-hour interviews with follow-up questions based on responses
LLM evaluation from perspectives of different experts
Interview transcripts and personality synthesis added to agent memories
To determine whether the generative agents accurately captured the study participants' views and personalities, both the participants and their AI counterparts were given identical tasks 6 :
| Assessment Type | Accuracy Level | Correlation Note |
|---|---|---|
| General Social Survey | 85% | As accurate as participants matching their own answers two weeks later |
| Big Five Personality Inventory | 80% correlation | Strong alignment with personality traits |
| Economic Games | 66% correlation | Notable consistency in decision-making patterns |
The researchers also demonstrated that their interview-based approach significantly outperformed alternatives. Agents with only demographic information or short self-written biographies were substantially less accurate and more prone to biased, stereotype-based responses 6 .
| Data Type | Accuracy | Bias Tendency | Key Finding |
|---|---|---|---|
| Interview-based | High | Low | Captures idiosyncrasies, minimizes stereotypes |
| Demographic-only | Moderate | High | Relies on broad demographic generalizations |
| Self-written biography | Low-Moderate | Moderate | Limited by self-awareness and expression |
As lead researcher Joon Sung Park noted: "The beauty of having interview data is that it includes people's idiosyncrasies and therefore the language models don't resort to making race-based generalizations as often" 6 .
Psychomatics research employs a diverse set of methodologies drawn from its constituent fields. Here are the key tools and approaches:
| Tool/Method | Function | Application in Psychomatics |
|---|---|---|
| Comparative Methodology | Systematically compare processes across humans and AI | Fundamental approach for identifying similarities and differences in cognition 1 7 |
| Theory-Driven Research Questions | Anchor investigations in established theoretical frameworks | Ensures research addresses foundational questions about language and intelligence 1 |
| Cognitive Task Batteries | Assess various aspects of intelligence and behavior | Enable direct performance comparison between humans and AI on identical tasks 6 |
| Interview-Based Profiling | Capture rich, qualitative data about individuals | Creates detailed personality datasets for training and validation 6 |
| Statistical Learning Analysis | Examine how patterns in data drive AI capabilities | Reveals differences between human experiential learning and AI statistical training 4 |
| Transformer Architecture Analysis | Study the technical foundation of modern LLMs | Investigates how self-attention and cross-attention mechanisms enable language processing 7 |
Psychomatics holds transformative potential for the future of artificial intelligence. By systematically comparing the cognitive processes of AI systems and biological minds, this framework can inform the development of more robust, reliable, and potentially more human-like AI systems 1 4 .
The practical applications are significant. Psychomatics could help make AI systems more transparent and interpretable, inform ethical discussions about AI deployment, and foster valuable interdisciplinary collaboration across cognitive science, linguistics, and computer science 4 .
However, significant challenges remain. The sheer complexity of modern LLMs makes it difficult to fully unravel their inner workings, and as AI systems continue to evolve rapidly, frameworks like psychomatics will need to adapt quickly 4 .
Future research may focus on developing more sophisticated comparative methodologies, exploring the potential for AI systems with greater embodied understanding, and investigating ways to imbue AI with more human-like capabilities for generating novel meanings and understanding context 4 .
As we continue to push the boundaries of artificial intelligence, psychomatics will play a crucial role in helping us understand these powerful but opaque systems. By bridging the gap between artificial and biological cognition, we can work toward developing AI that is not only more capable but also more aligned with human values and understanding 4 .
The journey to truly understand artificial minds has just begun, but psychomatics provides a promising roadmap for this exploration—one that might ultimately reveal as much about our own minds as it does about the machines we create.