Lawyer In Trouble With Judge After ChatGPT Cited Made-Up Cases In Filing

There’s no shortage of reports of people having saved countless hours at works thanks to ChatGPT, but using ChatGPT in the real world can also lead to some unintended consequences.

A lawyer in the US has been pulled up by the courts after he used ChatGPT for a filing and it cited cases which didn’t exist. In a New York case between Roberto Mata and Avianca, a lawyer used ChatGPT to create an affidavit, but didn’t bother to double-check the cases it had cited. Many of the cases that ChatGPT came up with were completely fake. After the court discovered what had happened, the lawyer said he “regrets” using generative AI for legal purposes, and would never use it in future without absolute verification of its authenticity.

ChatGPT’s made-up cases were caught by the defendants, who wrote to the courts. “On April 25, 2023, Plaintiff’s counsel submitted an affidavit annexing certain documents which he identifies as the cases in question.
Defendant respectfully submits that the authenticity of many of these cases is questionable. For instance, the “Varghese” and “Miller” cases purportedly are federal appellate cases published in the Federal Reporter. [Dkt. 29; 29-1; 29-7]. We could not locate these cases in the Federal Reporter using a Westlaw search. We also searched PACER for the cases using the docket numbers written on the first page of the submissions; those searches resulted in different cases,” the defendants state.

The plantiff then had to explain what happened to the courts. “As the use of generative artificial intelligence has evolved within law firms, your affiant consulted the artificial intelligence website Chat GPT in order to supplement the legal research performed,” the lawyer wrote.

“It was in consultation with the generative artificial intelligence website Chat GPT, that your affiant did locate and cite the following cases in the affirmation in opposition submitted, which this Court has found to be nonexistent:

Varghese v. China Southern Airlines Co Ltd, 925 F.3d 1339 (11th Cir. 2019) Shaboon v. Eciyotair 2013 IL App (1st) 111279-U (III. App. Ct. 2013) Petersen v. Iran Air 905 F. Supp 2d 121 (D.D.C. 2012)

Martinez v. Delta Airlines, Inc., 2019 WL 4639462 (Tex. App. Sept. 25, 2019)

Estate of Durden v. KLM Royal Dutch Airlines 2017 WL 2418825 (Ga. Ct. App. June 5, 2017)

Miller v. United Airlines, Inc., 174 F.3d 366 (2d Cir. 1999)

The lawyer then explained what had gone wrong.

8. That the citations and opinions in question were provided by Chat GPT which also provided its legal source and assured the reliability of its content. Excerpts from the queries presented and responses provided are attached hereto.
9. That your affiant relied on the legal opinions provided to him by a source that has revealed itself to be unreliable.
10. That your affiant has never utilized Chat OPT as a source for conducting legal research prior to this occurrence and therefore was unaware of the possibility that its content could be false.
11. That is the fault of the affiant, in not confirming the sources provided by Chat GPT of the legal opinions it provided.
12. That your affiant had no intent to deceive this Court nor the defendant.
13. That Peter LoDuca, Esq. had no role in performing the research in question, nor did he have any knowledge of how said research was conducted.
14. That neither your affiant nor Mr. LoDuca has ever been cited for any legal misconduct of any kind nor ever been sanctioned by this Court or any Court in over thirty years of practice.
15. That your affiant greatly regrets having utilized generative artificial intelligence to supplement the legal research performed herein and will never do so in the future without absolute verification of its authenticity.

This case is currently going viral on social media, but sadly this might not end up being the first time when LLMs will be found wanting in production environments. Most people using LLMs seem to think that they know and understand what they’re taking about, while they’re just models which are trained to produce realistic-sounding text. As such, these models often hallucinate, and make up entirely fake people, stories and sources. This is often based on probabilities — an economics professor has previously explained why ChatGPT makes up completely fake papers which sound real but are completely made up. ChatGPT now seems to have produced fake court cases, and led to some real-world consequences. It remains to be seen how the courts deal with this particular case, but this fiasco will serve as a cautionary tale for people using currently enthusiastically using ChatGPT in professional settings.