A Chatbot Crisis Warning Does Not Tell You Who Sees the Chat

The consequential sentence is not inside the crisis card. It sits in a law-enforcement policy most users will never open: information may be disclosed during an “emergency involving danger of death or serious physical injury.” Variations of that phrase appear across the documentation surrounding ChatGPT, Claude, Google Gemini and Microsoft Copilot.

On screen, the exchange looks simpler. A person types language associated with self-harm. The chatbot acknowledges distress, avoids providing harmful instructions and may display a crisis line or urge contact with emergency services. That is the visible product behavior.

Behind it are separate systems for safety classification, account enforcement, data retention and requests from authorities.

Those systems do not make the same promise. A supportive response is not evidence that somebody has been contacted. An emergency-disclosure clause is not proof that every flagged conversation reaches a human. A crisis number on screen does not tell the user whether the underlying text will remain on a company server, enter a review queue or become available when police submit an emergency request.

This is not a guide to crisis care, and it cannot establish how any company would handle an individual case. It is a reading of the paperwork. The paperwork shows a wide gap between what the chatbot says in the moment and what the company reserves the right to do afterward.

The card is only the first layer

Major chatbot makers train or instruct their systems to recognize language associated with self-harm and respond supportively. An automated classifier, software that assigns content to a risk category, may also assess the prompt or conversation separately from the model generating the reply. Companies disclose the existence of safety systems in broad terms, but they seldom publish the thresholds that move a conversation from an ordinary response into additional review.

That distinction matters. The model can produce a crisis-oriented answer without creating an external report. A classifier can flag text without a person reading it. A trust-and-safety team can examine a conversation without contacting authorities.

Police can request account information even if the company never initiated contact. The interface folds these events into one reassuring surface, although each has a different trigger and a different privacy consequence.

OpenAI’s published explanation is unusually direct on one point: it says the company does not currently refer self-harm cases to law enforcement, citing the importance of respecting people’s privacy. The same explanation distinguishes threats against other people, for which OpenAI says it has been developing escalation to law enforcement in cases involving an imminent threat of serious physical harm.

That is a meaningful boundary. It is not a permanent guarantee. OpenAI’s privacy policy and its policy for government requests still preserve emergency disclosure where the company believes information is needed to prevent death or serious physical injury. The sentence behind the crisis card remains in force.

Anthropic’s public materials similarly frame Claude’s desired response around care, de-escalation and encouragement to seek human support. Its privacy and government-request documentation also allows disclosure when necessary to protect safety or address an emergency involving imminent danger of death or serious physical injury. The published documents do not promise that self-harm language automatically causes such disclosure. They preserve the option.

Google separates Gemini’s conversational safety behavior from the rules governing Gemini Apps data. Its policies restrict assistance that meaningfully facilitates self-harm, while its privacy materials explain that conversations may be stored, sampled for review and used to improve services depending on account settings. Google also maintains a channel through which authorities can seek emergency disclosure when they believe information may prevent death or serious physical harm.

Microsoft does much the same through different documents. Copilot’s safety materials describe restrictions and supportive handling for self-harm content, while the Microsoft Privacy Statement permits data disclosure to protect customers or the public and comply with valid legal demands. Microsoft also accepts emergency requests from law enforcement under limited circumstances. Nothing in the ordinary crisis response explains which route, if any, a particular conversation has entered.

Retention comes before reporting

The more routine privacy issue is storage.

An ordinary ChatGPT conversation generally remains associated with the account until the user deletes it. OpenAI says deletion normally starts a process intended to remove the chat from its systems within 30 days, subject to legal, security and de-identification exceptions. Temporary Chat is designed not to appear in history or train models, but OpenAI says it may retain a copy for up to 30 days for safety purposes.

Google’s Gemini documentation is more explicit about settings and human review. When Gemini Apps Activity is enabled, conversations can remain in the account according to the chosen auto-delete period. If activity is turned off, Google says conversations may still be held for a short period so the service can respond and protect users. Conversations selected for human review can be disconnected from the account and retained separately for longer, which means deleting visible activity does not necessarily retrieve every review copy.

Claude and Copilot also provide controls for deleting conversation history and managing whether consumer interactions help improve models. Their privacy documents retain exceptions for fraud, abuse prevention, security, legal duties and enforcement. A delete button controls the version the user can see. It is not a live view of every backup, safety record or legally preserved copy.

This is where the argument about reporting can become misleading. The dramatic image is a chatbot silently calling the police. The ordinary mechanism is less cinematic: a highly sensitive disclosure becomes account data, receives automated labels, may be sampled under a review program and remains available long enough for a later legal or emergency request to reach the company.

The cost is not only exposure. It is uncertainty. A person in distress cannot inspect the classifier’s score, determine whether a reviewer has opened the exchange or see whether the company preserved the chat under a safety exception. They receive a phone number and general encouragement.

The data rules remain elsewhere.

An emergency request is not an automatic alarm

The repeated phrase, “danger of death or serious physical injury,” comes from a legal framework for emergency disclosure. In the United States, providers may voluntarily disclose certain customer records to government authorities when they believe an emergency justifies it. Companies build request portals and internal review procedures around that permission.

Usually, an emergency request begins with law enforcement. An agency identifies an account, describes the danger, specifies the information sought and asks the provider to disclose it without the ordinary delay of a warrant or other standard process. The company then assesses whether the request meets its policy and legal threshold. Some providers also reserve the ability to initiate contact when their own teams identify a sufficiently grave threat.

The documentation does not tell a user how often these routes are used for chatbot conversations. It does not publish a shared definition of imminence, disclose the confidence score required for escalation or guarantee that a trained clinician makes the decision. Companies may issue aggregate transparency reports about government demands, but those totals rarely let the public isolate crisis conversations with consumer AI systems.

Location creates another unresolved problem. A chatbot may know an account email, an approximate network location or billing information, but that does not mean it has an accurate address for the person typing. Shared devices, travel, virtual private networks and false account details complicate identification. A company can have enough information to expose a conversation without having enough reliable information to send useful help.

The policy phrase therefore does two jobs. It creates a narrow-sounding standard for disclosure, and it leaves the company room to interpret the facts. The user sees neither interpretation.

The missing notice

None of the four companies needs to turn a crisis response into a wall of legal text. A concise notice could still state that the conversation may be stored or reviewed under safety policies, that emergency disclosure is possible in limited circumstances, and that displaying crisis resources does not mean an outside service has been contacted.

That notice would force the product to admit that support and surveillance can occupy the same exchange. It could also affect what a distressed person chooses to disclose, which is precisely why companies have little incentive to foreground it. Friction is unwelcome when the product depends on intimate, continuous conversation.

Privacy policies place the burden elsewhere. Users are expected to study retention schedules before the emergency, understand government-request procedures during it and remember which account setting governs model improvement afterward. The crisis card keeps its clean edges. The sentence about death or serious physical injury stays several links away.

Questions people ask

Does

ChatGPT automatically call police after self-harm language?

OpenAI says it does not currently refer self-harm cases to law enforcement. Its policies still allow emergency disclosure in limited circumstances involving danger of death or serious physical injury, so the published position should not be mistaken for an unconditional privacy guarantee.

Can a human reviewer read a crisis conversation?

Potentially. Major chatbot companies use automated safety systems and reserve some form of human review for safety, abuse prevention or service improvement. Whether a particular chat is reviewed depends on the product, account settings and internal thresholds that the companies do not fully publish.

Does deleting the chat erase every copy immediately?

No major provider makes that blanket promise. Deletion usually begins a removal process, while legal holds, security investigations, de-identified records, backups or separately retained review data may remain under the exceptions described in company policies.

Does seeing a crisis line mean the chatbot reported the conversation?

No. A crisis line or emergency-services prompt is part of the chatbot’s response behavior and does not, by itself, show that a human reviewed the chat or that an outside organization was contacted. The interface generally does not reveal whether any separate escalation occurred.