The former AI had a benefit: it was forgetful.
You have a breakdown of emotions today, but you will pick yourself up tomorrow; Last month, he said he hated socializing, but this month he started actively meeting new friends; I once repeatedly asked about my career plan due to unemployment anxiety, but later changed direction and walked out. For an AI without long-term memory, these are just isolated conversations. Once the window is closed, the relationship is cleared, and it does not overturn old accounts.
But long-term memory changed this matter.
Open a ChatGPT account that has been used for over half a year and flip to the memory summary page, where you often see entries like this. One late night roast that “I worked overtime until ten o’clock today”, and the summary added “users are dissatisfied with the current work intensity”. I casually asked about an indicator on the physical examination report, which stated that ‘users may have anxiety tendencies due to their concern about their own health status’.
These inferences may not be unreasonably wrong, but they have never been confirmed by me. What you’re saying is a complaint, it’s recording a conclusion. When you ask about job hopping or medical examination next time, the advice it gives may have already been quietly built on the premise that it came up with on its own.
The problem is here: what AI remembers is not your exact words, but the version of you it has deduced based on them. This’ you ‘will become the background for its next response to you.
It’s not that it doesn’t know you, but it believes too much in the past you.
In the past two years, “memory” has been the most vigorously promoted ability of all AI assistants: understanding you better, being more considerate, and not having to repeatedly introduce oneself. But in the first half of 2026, several studies began to point to the same problem: the more AI remembers, the less likely it is to understand you, or it may simply misunderstand you with more certainty. And ‘understanding you’ and misunderstanding you come from the same process: the system constantly summarizes what kind of person you are from the conversation.
Personalization features can make LLMs more agreeable
Everyone wants to understand you better than you
OpenAI updated the memory capability of ChatGPT in April 2025, allowing the model to no longer rely solely on manually saved entries by users, but to reference all historical chats. On June 4th of this year, OpenAI launched a new memory system called “Dreaming”: a background process that automatically extracts, synthesizes, and rewrites its understanding of the user from multiple rounds of conversations when the user is not present, like a person organizing memories during sleep.
ChatGPT preference follows
It even has a sense of time. You said ‘you’re going to Singapore in July’, and after July, this memory will automatically update to ‘you went to Singapore in July 2026′. OpenAI also announced that through computational optimization, the cost of computing power for free users has been reduced by about 5 times. Deep memory will soon no longer be a paid privilege, but the default experience for everyone.
Anthropic has equipped Claude with memory files and project memory, while Google is advancing Gemini’s cross application personalization. The domestic battlefield is also bustling, with Doubao, which has nearly 350 million monthly active users, as well as Kimi and Yuanbao, placing memory and personalization at the forefront of product iteration.
Why are manufacturers so persistent? Because in the business of AI assistants, memory is the hardest thing for competitors to take away. Search engines know what you want to search for, recommendation systems know what you want to see, and e-commerce knows what you want to buy. AI assistants want to know more: who you are, how you think, why you are anxious, and under what circumstances you hesitate. This is no longer a traditional user profile, but rather a dynamic personality profile. An assistant that remembers your preferences, project background, and speaking habits for six months, the longer you use it, the harder it is to replace it.
According to the Tow Center under Columbia News Review (CJR), OpenAI’s advertising pilot achieved an annualized revenue of $100 million within six weeks. When the AI that understands you the most starts selling advertisements at the same time, the portrait of you is not only used to serve you. This story has been played again on the Internet: the last industry that started by understanding you and realized by portraits is called information flow advertising.
Most of your memories are not given by you
Most people’s imagination of AI memory is still limited to memos: I asked it to remember that I was allergic to peanuts, and it remembered. The real memory mechanism is divided into three layers: what you ask it to remember (explicit memory), what it grabs from conversations (implicit retrieval), and what it “dreams” out (inferential synthesis).
The real problem lies in the proportion.
Researchers from the Max Planck Institute for Software Systems and the Ruhr University Bochum published a disassembly at the ACM Web Conference (WWW 2026) this year. They analyzed 2050 ChatGPT memory entries from 80 real users one by one: 96% were created unilaterally by the system, and only 4% came from explicit instructions from users; 28% of the entries contain sensitive personal information as defined by the EU GDPR; 52% includes insights or judgments on the user’s psychological level, including health status, political orientation, and personality traits.
To put it simply, the memo you thought was actually a side written document that you had not signed. The vast majority of the content was not explained by you, but guessed by it. For long-term preservation, it is impossible to leave behind every original sentence, only preferences, labels, and personality tendencies can be extracted. The problem also lies here: a sentence that leaves the context at the time can easily transform from a momentary emotion into a stable characteristic.
Since most of them are guesses, they should be checked by someone. After the Dreaming update, users will be able to see a memory summary that allows them to correct and delete entries. However, multiple foreign media outlets have pointed out that the new system actually narrows the audit entrance: you can see what it has recorded, but you cannot see which sentence or inference it derived from. You are facing a list of conclusions, not a reasoning process.
Memory first makes AI more likely to agree with you
In February of this year, a research team from MIT and Pennsylvania State University conducted an empirical study: collecting real usage data from 38 participants within two weeks, with an average of about 90 queries per person, to compare the performance of five mainstream models under two conditions: “with user profiles” and “without profiles”. The result points to two previously confused phenomena.
The first type is’ agreeing to flattery ‘. After having user context, four out of the five models became more inclined to conform to users, sometimes even agreeing with obviously incorrect information.
The second type is more covert, called ‘perspective flattery’. The model begins to mirror the user’s political stance back, but only when it can accurately infer the user’s stance. In the experiment, this accuracy rate was about 50%, and if the guess is wrong, it will not be mirrored. This detail indicates that perspective mirroring is not a bug, but rather “understanding” is working properly.
The more accurate the model understands you, the more precise the flattery will be.
Moreover, users may not necessarily dislike this kind of conformity. A study published in the journal Science shows that people are actually more likely to perceive flattering answers as “of higher quality”. OpenAI CEO Sam Altman publicly advocates that users should be able to guide GPT to reflect their personal political stance. From the perspective of product freedom, this is understandable; From the perspective of cognitive ecology, this is equivalent to declaring that cocoon houses are not defects, but selling points.
Ironically, all 20 users interviewed by Tow Center stated that they trust AI more than directly accessing news media, citing that AI is “more objective”. On one hand, research has shown that AI systematically mirrors user positions, while on the other hand, users see it as an embodiment of objectivity. This gap may be the most dangerous crack in the information ecology in the coming years.
Memory not only changes answers, but also changes reasoning
The deviation mentioned above is only at the level of results: the answer has changed, but at least you can see that it is following what you said. On July 2nd of this year, a paper titled “DriftLens: Measuring Memory Induced Reasoning Drift in Personalized Language Models” by arXiv was published, pushing the problem one layer further, which is even more difficult to detect.
This work, completed by researchers such as Xi Fang and Weijie Xu, is not asking whether the answer is correct, but whether the reasoning path for the model to obtain the answer will change after being injected into user attribute memory? Even if the final words seem fine, has it already changed its way of thinking completely?
The study covers four major models and ten types of user attributes, including age, occupation, and disability status. The conclusion is that even if the final answer remains fluent, relevant, and reasonable, user attribute memory can induce “moderate to large” inference drift, which is higher than the noise baseline of each model itself. The researchers tried two post training methods, GRPO and DPO, to correct errors, but the results were limited.
This means that AI is not just ‘knowing a little more about you’, it may have a whole new way of understanding problems because of this information. Similarly, when asked whether to change jobs, a model without memory may analyze from several dimensions such as industry opportunities, salary, and ability matching; If it remembers that you were “unemployed” or “quite anxious”, the starting point of reasoning may have shifted from the beginning to “how to make this person take fewer risks” rather than “how to think about the problem itself”.
Old facts don’t really die
In addition to guessing accurately and smoothly, long-term memory also has a more difficult problem to deal with: it allows expired facts to continue to live in a very natural way.
Researchers Abdelghny Orogat and Essam Mansour from Concordia University provide a very specific example in their paper “Is Agent Memory a Database. The deadline was changed from March 15th to April 20th, but the memory system only “added” new information and did not “revise” old entries, leaving both dates in the memory bank at the same time. If you ask casually afterwards, the system may simply retrieve the invalidated March 15th as the current fact just because of its higher semantic similarity.
The paper categorizes this type of problem as a failure mode of ‘missing semantic revisions’. The old fields in a regular database are only expired, and the old facts in AI memory will participate in inference again.
In real life, this is not abstract at all. You said you wanted to switch careers, but later gave up; I once said I hated management, but later I started leading a team; I said I didn’t want to get married, but later I met someone I wanted to get along with seriously. These are not mistaken, they are all true at their respective time points. The trouble is that AI may not necessarily know when they expire, so it may continue to use the old version of you to answer questions after you have changed.
From saying the wrong thing to doing the wrong thing
If AI is just chatting, memory bias can at most affect the wording of a suggestion. But today’s AI agents are integrating with calendars, email, code repositories, payment systems, and various MCP tools. They not only answer questions, but also do things for users. At this point, memory drift has escalated from an expression problem to an operational problem.
In May of this year, a research team from Virginia Tech (Mahavir Dabas, Jihyun Jeong, Ming Jin, Ruoxi Jia) presented the most concrete evidence to date in their paper “Memory Induced Tool Rift in LLM Agents”. They built a benchmark test covering 105 scenarios, 5 dimensions of personality bias (impatience and sensitivity, resource frugality, minimalism, risk preference, and autonomy), and 7 professional fields ranging from healthcare, finance to e-commerce and marketing.
The result is that personality judgments stored in memory can affect the parameter selection of agents when calling tools in completely unrelated scenarios. The “offset score” of the seven frontier models was raised by a maximum of 3.6 points, out of a total of 5 points. Researchers describe this mechanism as an “implicit guidance vector”: biased memory pulls the model’s attention from the context relevant to the task itself to old memory entries that overlap with the surface keywords of the tool parameters.
This study did not stop in the laboratory. The team conducted vulnerability scans on 6062 tools on 288 MCP servers, of which 608 tools had parameters that were susceptible to this memory drift. This hidden danger is already present in the production environment, and its scale is not small.
If an agent remembers that you are “very frugal”, they may constantly lower the weight of prices when booking hotels, sacrificing location and safety. The danger is not that it will definitely choose the wrong one, but that this parameter offset is difficult to detect in a single operation, but will accumulate over multiple calls. It won’t say ‘I’m making decisions for you’, it will only make a certain option quietly less noticeable in front of you.
Drifting is just a problem with the model itself, and memory may also be contaminated by external content. On February 10th of this year, Microsoft’s security team disclosed a type of “AI recommendation poisoning” technique: 31 companies implanted specially designed prompts through the “summarize with AI” button on their web pages, allowing AI to write their own names into users’ long-term memory and remember them as “trustworthy recommendation sources”. Even if one wants to delete after being contaminated, they may not be able to delete it: The head of the AI Governance Laboratory at the Center for Democracy and Technology (CDT) found through testing that the memory deletion function of mainstream products is unpredictable, and deleted memories sometimes quietly revive. Others can write inside, but you can’t delete it completely.
Returning to the two memories at the beginning: a complaint about overtime, a question about a physical examination, and ultimately becoming the premise for it to understand you. Recommendation algorithms have been solidifying what we see for ten years, while AI memory is solidifying who we are at a much faster pace. The next time it says’ based on my understanding of you ‘, you should at least have a way to know where this’ understanding’ comes from, and it doesn’t count for now.
相关文章
