https://simonwillison.net/2023/May/27/lawyer-chatgpt/
A lawyer asked ChatGPT for examples of cases that supported an argument they were trying to make.
ChatGPT, as it often does, hallucinated wildly—it invented several supporting cases out of thin air.
When the lawyer was asked to provide copies of the cases in question, they turned to ChatGPT for help again—and it invented full details of those cases, which they duly screenshotted and copied into their legal filings.
At some point, they asked ChatGPT to confirm that the cases were real… and ChatGPT said that they were. They included screenshots of this in another filing.
The judge is furious. Many of the parties involved are about to have a very bad time.