Google Researchers can create an AI that thinks like you after just a two-hour interview

Stanford University researchers paid 1,052 people $60 to read the first two lines The Great Gatsby to the program. So the AI, which looked like a 2D sprite from the SNES-era Final Fantasy game, asked participants to tell the story of their lives. The scientists took those interviews and processed them into an artificial intelligence that replicated the participants’ behavior with 85% accuracy.

study called 1000 Human Generative Agent SimulationsIt is a joint venture between scientists working at Stanford and Google’s DeepMind AI research lab. The idea is that creating AI agents based on random people can help politicians and businesspeople better understand the public. Why use focus groups or survey the public when you can talk to them once, earn an LLM based on that conversation, and then own their thoughts and opinions forever? Or, at least, as close to those thoughts and feelings as an LLM can recreate.

“This work paves the way for new tools that can help investigate individual and collective behavior,” the paper’s abstract reads.

“For example, how might different sets of individuals respond to new public health policies and messages, react to product launches, or respond to major shocks?” The paper continued. “When simulated individuals are combined into collectives, these simulations can help pilot interventions, develop complex theories that capture nuanced causal and contextual interactions, and expand our understanding of structures such as institutions and networks in fields such as economics, sociology, organizations, and political science. “

All these opportunities, based on a two-hour interview, are fed into the LLM, where you basically answer the same questions as your real-life counterparts.

Most of the process was automated. The researchers contracted with market research firm Bovitz to recruit participants. The goal was to obtain a broad sample of the US population while limiting it to 1,000 people. To complete the study, users signed up for an account in a purpose-built interface, created a 2D sprite avatar, and began speaking with an AI interviewer.

The questions and interview style are a modified version of what he used Voices of America Project, A joint project between Stanford and Princeton University is interviewing people across the country.

Each interview began with the participants reading the first two lines The Great Gatsby (“During my younger and more vulnerable years, my father gave me advice that I’ve carried with me ever since. ‘Whenever you feel like criticizing someone,’ he told me, ‘remember that everyone in this world has the advantages you have. not.’”) as an audio calibration method.

According to the paper, “The interview interface displayed a 2-D sprite avatar representing the interviewer’s agent in the center, while the participant’s avatar was shown below, moving toward the goal post to indicate progress. When the AI ​​interviewing agent is speaking, this is indicated by a vibrating animation of the center circle with the interviewer’s avatar.

The two-hour interviews produced transcripts that averaged 6,491 words in length. He asked questions about race, gender, politics, income, social media use, job stress, and family structure. The researchers published the script of the interview and the questions asked by the AI.

These transcripts, each less than 10,000 words, were then fed into another LLM, which the researchers used to spin generative agents to replicate the participants. The researchers then put both the participants and the AI ​​clones through more questions and economic games to see how they would compare. “When an agent is questioned, the entire transcript of the interview is inserted into the model query and the model is instructed to impersonate that person based on the interview information,” the newspaper said.

This part of the process was as close to control as possible. The researchers used General Social Survey (GSS) and Big five personality inventory (BFI) to test how LLMs live up to their inspiration. He then ran five economic games to see how participants and LLMs would compare.

The results were mixed. The AI ​​agents answered about 85% of the questions like real-world participants in the GSS. They scored 80% in BFI. But the numbers dropped when agents started playing economic games. The researchers offered real-life participants monetary rewards for playing games The prisoner’s dilemma and Dictator’s game.

In Prisoner’s Dilemma, participants can choose to work together and both succeed or defeat their partner for a chance to win big. In the Dictator Game, participants must choose how to allocate resources to other participants. Real-life subjects paid more than the original $60 to play these.

Artificial intelligence clones of humans who encountered these economic games also did not replicate their real-world counterparts. “On average, generative agents achieved a normalized correlation of 0.66, or about 60%.

If you’re interested in how academics think about AI agents and the public, the whole paper is worth reading. It didn’t take long for researchers to transform a human personality into an LLM that behaved in the same way. Given time and energy, they could probably bring the two closer together.

This worries me. Not because I don’t want to see the inexplicable human soul turned into an electronic spreadsheet, but I know that this kind of technology will be used for diseases. We’ve already seen stupid LLMs trained in public recordings trick grannies into giving their bank details to an AI relative after a quick phone call. What happens when the script is on those machines? What happens when they have access to targeted profiles based on social media activity and other public data?

What happens when a corporation or a politician decides that the public wants and needs something based on their approach to it rather than their negotiated will?