Abstract:
AI understands you so well, it’s almost creepy! In the past few days,
an "empty room" experiment has become popular again
, attracting millions of people across the Internet to watch. Netizen Jakub Szymczuk also followed the trend and tested it. He casually threw a photo of a gray rough house with four bare walls to ChatGPT, and typed only one line: "Fill this room with the information you know about me
".
In just ten seconds, the originally empty room was instantly transformed into a "panoramic monitoring room" full of
personal privacy
.The most amazing thing is a detail on the computer desk. There is actually a bug "silverfish" in the glass jar.
It took him a long time to realize that he had photographed this little bug a year ago and asked what ChatGPT was. Even this kind of casual question that had long been forgotten was written down in the notebook by AI.
It’s one thing to chat with AI every day and know that it will accumulate data.
But when these fragments turn into a large high-definition picture and are clearly displayed in front of our eyes, it still makes people feel numb.
A picture of "empty room" flooded the entire internet
Soon, netizens around the world couldn't sit still. Someone posted the results and called them "pretty accurate"
.Someone added "but make it exaggerated" after the prompt word, and the room directly turned into a large character exhibition hall.
Someone else got a swamp and two cats.
Analyst Patrick Moorhead simply conducted a round of reviews on multiple models, and the conclusion was straightforward: the best is ChatGPT, and the worst is Claude.
In this regard, a netizen summed it up brilliantly. AI is the cheapest decorator you can find, and it is also the one that loves to spy.
Why does ChatGPT understand you so well?
96% conversation memory, ChatGPT saves it yourself
Actually, this is no secret.
Everything in the room is almost something you slipped out of your mouth while chatting.
Behind this is precisely the "memory" of
ChatGPT
Function. To put it simply, it has two core functions——The first is "saved memory", which remembers clear information such as your name, preferences, and goals;
The other is "Reference Chat History", which can dig out useful context from a large number of your past conversations.
This year, a paper selected for ACM Web Conference 2026 did something that no one had done before.
Researchers from the Max Planck Institute, Ruhr University Bochum and other institutions found 80 real ChatGPT users, asked them to export their data in accordance with the EU GDPR, and then donated their chat records.
How do you know what the AI secretly memorized? Next, the researchers came up with a clever idea.
ChatGPT has a small tool dedicated to writing memories, which is equivalent to the notepad it carries with you. Every time you write, a line of "Model set context updated (model context updated)" record will be left in the background.
The researcher went back through this line of text to see which sentence the user made that made the AI write.
In this way, they extracted 2,050 memories from 1,058 conversations and more than 20,000 rounds of confrontations between 65 people.
The paper gives these memories a name, called "algorithmic self-portraits."
The three sets of data extracted as a result can make people feel chills down their backs.
First, 96% of the memories are secretly saved by ChatGPT itself.
How to tell if the user took the initiative to ask it to remember? The researchers specifically searched for words like "remember" and "forget" in the sentence that triggered the memory.
As a result, only 84 of the 2050 memories were matched. Calculated, for every 24 pieces of information that AI saves about you, only 1 piece of information comes from your nod.
Second, 52% of memories are secretly "psychological profiling" for you.
The researchers borrowed the "theory of mind" framework from psychology and divided these contents into categories such as desires, intentions, emotions, and beliefs.
Among them, "wishes" dominate, accounting for 73%; intentions account for 16%, emotions account for 14%, and beliefs account for 11%.
To put it bluntly, AI doesn’t care whether you are happy today or not. What it really wants to find out is what you long for in the long term and what you plan to do next. Rather than your temporary joys and sorrows, it wants to see through what kind of person you are.
Third, more sensitive privacy is recorded than you think.
28% of the memories belong to the personal privacy recognized by the EU GDPR, and specific names alone appear in 41% of the memories.
What is even more exaggerated is that 7% directly stepped into the "high-risk red lines" such as health status, religious beliefs, and political leanings, of which health privacy accounted for more than 60%.
If it were spread to everyone, 91% of people would have personal data stored, and more than half (54%) of people would have highly sensitive privacy records recorded.
The paper directly points out that this is obviously contradictory to OpenAI's public commitment of "not actively saving sensitive information."
The paper also conducted a more detailed and terrifying experiment.
The researchers used open source models such as Gemma and GPT-OSS to "copy" a ChatGPT memory extractor and learned six images.
Then they hypothesized that what would happen if the AI retrieved the memory of every sentence the user said?
The answer is that the proportion of sensitive data will increase from 28% to 31%~35%, and the proportion of psychological portraits will increase from 52% to 55%~59%.
You must know that the old version of ChatGPT is just a "note".
What Dreaming is about to do next is to read through your chat history in the background.
It turns out to be "dreaming", AI repeatedly rewrites personal portraits
In June of this year, OpenAI gave ChatGPT a major overhaul of its memory, giving it a mysterious name - Dreaming.
To put it bluntly, just like when a person sleeps, the brain automatically sorts out the day's experiences, ChatGPT will also quietly "review" the days of chatting in the background, constantly refreshing its understanding of you.
For example, you said "I will go to Singapore in July". Once the day passes, this memory will be automatically rewritten as "I went to Singapore in July 2026".
OpenAI has conducted internal tests and found that this new mechanism makes its record accuracy soar:
From 41.5% in 2024 and 67.9% in 2025, it has directly reached 82.8% of the V3 version today.
What’s even more cruel is that the cost of backend computing power has been cut to one-fifth of its original value.
But what is really scary to think about is that deleting a ChatGPT conversation will not delete the memories that have been extracted from it.
Among these, the most worrying thing is actually your health privacy.
In September, a paper published on arXiv examined nearly 180,000 chat records of 1,057 users and found that more than one-fifth (21.31%) of the conversations discussed personal health issues.
What’s even more worrying is that more than 95% of the personal portraits in it were also secretly recorded by AI while the user was not paying attention.
This ratio is almost exactly the same as the 96% measured in the previous paper.
When AI "brain supplements" and merges memories in the background, it is very easy to weld your temporary symptoms into a permanent "medical record label".
For example, if you just casually said you have a headache one day, in the eyes of AI, you may have become a "migraine patient."
"Second Brain"?
We always think that we are training a "second brain" that becomes more and more sensible.
It wasn’t until the picture of the room filled with privacy was thrown in my face that I was shocked: After thousands of conversations, it turned out that we were the only ones who had been stripped of our trump cards.
In the eyes of algorithms, there is never so-called empathy, only the "context" that is clearly priced.
Comments