In March 2023, three Samsung engineers did something a lot of people would do without thinking twice. They pasted confidential source code and private meeting notes into ChatGPT, just to get help finishing their work faster. Within twenty days, Samsung found out and banned the tool company-wide. But the code was already gone. Not stolen, exactly. Just sitting on someone else's servers, completely outside Samsung's control.
That one mistake, made by ordinary employees trying to save time, is still one of the most talked-about case studies in AI privacy. And it points to a question most people never really stop to ask: when you type something into a chatbot, where does it actually go?
Free and paid accounts don't work the same way
On ChatGPT's free and Plus plans, your conversations can be used to train future versions of the model. You can turn this off in settings, but most people never do. Business accounts work differently. On Team, Enterprise, and the API, OpenAI does not train on your data by default. Google and Anthropic follow roughly the same pattern: personal accounts usually feed training unless you switch it off, while paid business accounts with a signed agreement usually don't.
That means the exact same sentence, typed into the exact same chatbot, gets treated completely differently depending on which account you happen to be logged into. Almost nobody checks this before they start typing.
"Deleted" comes with more conditions than you'd expect
Even when a company says it deletes your data, that usually means “within about 30 days,” not right away. And even that 30-day promise isn't absolute. In May 2025, a US federal judge ordered OpenAI to stop deleting ChatGPT logs entirely, as part of a copyright lawsuit brought by The New York Times. The order covered chats people had already deleted, and even chats from users who had opted out of training. OpenAI called it a conflict with its own privacy commitments and pushed back, but had to comply for about four and a half months, until the order was lifted in late September 2025. Business customers on Enterprise or with a Zero Data Retention agreement were excluded from this. Free and personal accounts were not.
The point here isn't that this exact thing will happen to you. It's that “deleted” describes a company's normal policy, not a guarantee. Legal, security, or safety exceptions can override that policy, and as a user, you usually have no way of knowing when that's happening.
What companies with something to protect do differently
Organisations that take this seriously don't just tell employees to “be careful” and hope for the best. They use business-tier accounts with real contracts attached. These usually include a written agreement not to train on the company's data, retention settings an admin controls, encryption, and sometimes a Zero Data Retention option, where prompts and answers aren't stored at all beyond the moment they're processed.
This is exactly the gap that caused the Samsung incident in the first place. Employees were using personal, consumer accounts for work, which put company secrets on the training-by-default tier without IT ever knowing. After the story broke, so many companies restricted or banned employee chatbot use, including Apple, JPMorgan Chase, Verizon, and Amazon, that it became close to a standard corporate reaction almost overnight.
The kind of information that causes real problems
The riskiest habits tend to be the most tempting ones. Pasting a client contract to get a fast summary. Uploading a resume with a home address and ID number so AI can “improve” it. Typing in medical symptoms to ask what they might mean. Copying a company's internal financial numbers into a prompt just to build a chart. None of it feels risky in the moment. All of it can end up stored somewhere outside your control, on a system that was built to answer questions, not to keep secrets.
A simple rule helps here: if you wouldn't post it publicly online, don't paste it into a chatbot on a personal account. Enterprise tools with a real data agreement are a different situation. But the free version most people default to isn't built for anything confidential.
What this means for you
If you're on a personal plan, open your account's data settings and turn off training. It takes about thirty seconds, and most people have never even opened that menu. At work, use whatever business or enterprise tool your company has actually licensed, not your personal login. That's the entire reason the business tier exists. And treat anything you type as something you can't fully take back, not because every AI company is careless with your data, but because retention promises can be overridden by forces outside the company's control, let alone yours.
None of this means you should avoid AI tools. They're genuinely useful, and most everyday questions carry no real risk at all. It just means treating the input box the way you'd treat any place you don't fully control: fine for most things, and not the place for anything you truly can't afford to have leak.
EDITOR'S TAKEAWAYWhat happens to information typed into a chatbot depends heavily on account type: free and personal accounts often train on your data by default, while paid business tiers typically don't. Even 'deleted' data can be overridden by legal orders, as shown by a 2025 court case involving OpenAI. Sensitive personal or workplace information should only go into accounts with an actual data agreement in place. |