Legal experts warn that AI prompts could become evidence in litigation
As people increasingly turn to artificial intelligence for advice, some US lawyers are telling their clients not to treat AI chatbots like trusted confidants when their freedom or legal liability is on the line.
These warnings became more urgent after a federal judge in New York ruled this year that the former CEO of a bankrupt financial services company could not shield his AI chats from prosecutors pursuing securities fraud charges against him.
In the wake of the ruling, attorneys have been advising that conversations with chatbots like Anthropic’s Claude and OpenAI’s ChatGPT could be demanded by prosecutors in criminal cases or by litigation adversaries in civil cases.
“We are telling our clients: You should proceed with caution here,” said Alexandria Gutiérrez Swette, a lawyer at New York-based law firm Kobre & Kim.
People’s discussions with their lawyers are almost always deemed confidential under US law. But AI chatbots are not lawyers, and attorneys are instructing clients to take steps that could keep their communications with AI tools more private.
In emails to clients and advisories posted on their websites, more than a dozen major US law firms have outlined advice for people and companies to decrease the chances of AI chats winding up in court.
Similar warnings are also appearing in hiring agreements by some firms with their clients. For instance, New York-based firm Sher Tremonte stated in a recent client contract that sharing a lawyer’s advice or communications with a chatbot could erase the legal protection known as attorney-client privilege that usually shields communications between lawyers and their clients.
A JUDICIAL RULING
The case that helped set off the alarm bells involved Bradley Heppner, the former chair of bankrupt financial services company GWG Holdings and founder of alternative asset firm Beneficent (BENF.O). Heppner was charged by federal prosecutors last November with securities and wire fraud, and pleaded not guilty.
Heppner had used Anthropic’s chatbot Claude to prepare reports about his case to share with his attorneys, who later argued that his AI exchanges should be withheld because they contained details from the lawyers related to his defense.
Prosecutors argued that they had a right to demand material that Heppner created with Claude because his defense lawyers were not directly involved, and because attorney-client privilege does not apply to chatbots.
Voluntarily revealing information from a lawyer to any third party can jeopardize the customary legal protections for those attorney communications.
Manhattan-based US District Judge Jed Rakoff ruled in February that Heppner must hand over 31 documents generated by Anthropic’s chatbot Claude related to the case.
No attorney-client relationship exists “or could exist, between an AI user and a platform such as Claude,” Rakoff wrote.
Lawyers for Heppner did not immediately respond to requests for comment. A spokesperson for the US attorney’s office in Manhattan declined to comment.
Courts already are grappling with the growing use of artificial intelligence by lawyers and people representing themselves in legal cases, which among other things has led to legal filings containing made-up cases invented by AI.
Rakoff’s decision was an important early test in the AI chatbot era for bedrock legal protections governing attorney-client communications and materials prepared for litigation.
On the same day as Rakoff’s ruling, US Magistrate Judge Anthony Patti in Michigan said a woman representing herself in a lawsuit she brought against her former company did not have to hand over her chats with OpenAI’s ChatGPT about the employment claims made in the case.
Patti treated the woman’s AI chats as part of her own personal “work-product” for the case, rather than as conversations with a person who her employer could seek to use for its defense.
ChatGPT and other generative AI programs “are tools, not persons,” Patti wrote in his order.
The privacy and usage terms for both OpenAI and Anthropic state that the companies can share data involving their users with third parties. Both also state that they require users to consult a qualified professional before relying on their chatbots for legal advice.
Rakoff at a February hearing in Heppner’s case noted that Claude “expressly provided that users have no expectation of privacy in their inputs.”
Representatives for OpenAI and Anthropic did not immediately respond to requests for comment.
LAWYERS RACE TO SET GUARDRAILS
The advice from lawyers has ranged from telling clients to select their AI platforms carefully to suggesting specific language to use in chatbot prompts.
Los Angeles-based O’Melveny & Myers and other firms said in client advisories that “closed” AI systems designed for corporate use could provide stronger protections for legal communications, though they said even that remains largely untested.
Some firms said AI legal research is more likely to be protected by attorney-client privilege when it is conducted at the direction of a lawyer. If a lawyer does advise the use of AI, a person should say so in the chatbot prompt, New York-headquartered law firm Debevoise & Plimpton said in a notice on its website.
“I am doing this research at the direction of counsel for X litigation,” the firm suggested people write.
Information about AI use is also becoming common in contracts used by law firms with clients, according to a Reuters review of contracts posted to a US government website.
Sher Tremonte, which often represents white-collar criminal defendants, said in a new contract in March: “Disclosure of privileged communications to a third-party AI platform may constitute a waiver of the attorney-client privilege.”
Justin Ellis of New York-headquartered law firm MoloLamken and other lawyers said they expect that more rulings will eventually clarify when AI chats can be used as evidence.
Until then, attorneys are saying that an age-old assumption still applies: Do not talk to anyone except your lawyer about your case – including AI.