ChatGPT + Lawyers v Horizon + The Post Office
What does ChatGPT and Lawyers have to do with Horizon and The Post Office Scandal?
This past week the country (UK) has been collectively transfixed, outraged and disillusioned by Mr Bates v The Post Office on ITV. A drama about the real life events known commonly as ‘The Post Office Scandal‘. One of the widest miscarriages of justice in UK legal history. In 1999 The Post Office introduced a computer system from Fujitsu called Horizon which all sub-postmasters had to use to balance their books. Before that point it had all been paper based.
Sub-postmasters were having problems with Horizon not balancing the books. When they called the Fujitsu helpline they were always told “you are the only one”. There were in fact over 700 of them! Those 700+ sub-postmasters were wrongly accused of theft, fraud and false accounting by The Post Office due to a flaw in the computer system Horizon. 555 of them raised a group action which was Bates & others v Post Office (16 December 2019). The sub-postmasters won the case and were awarded £57.75 million but after legal expenses each received only circa £20,000. The independent Post Office Horizon IT Inquiry is ongoing.
Lawyers commenting on Horizon and The Post Office Scandal
Quite rightly, lawyers have been commenting on social media this week on how terrible this was and how there should not have been such an acceptance of the computer over the real life sub-postmasters.
This is where to my mind analogies can be drawn today between ChatGPT, Lawyers, Horizon and The Post Office Scandal.
Lawyers and ChatGPT on the Other hand
Whilst those lawyers are shocked that a computer system (Horizon) was given such credence, on the other hand lawyers are thinking ChatGPT can do no wrong. I created a meme which I posted on social media:
Seeing reactions by lawyers to Horizon and the #postofficescandal compared with their ongoing fascination with ChatGPT. Looks like both unfortunately ‘hallucinate’ (otherwise know as being error-prone). Maybe something to think about?
Here are some of the comments I received in response:
John McCarthy (Supporting Owners of Law Firms to Scale their Practices Increase Revenue & Profit AND Increase The Value of Their Business by Following the Unique P.R.O.F.I.T System):
Do you think AI could make such huge mistakes on the transactions? Possibly.
Would it then lie to try and cover up it’s guilt, and use such disgraceful dirty tactics? Now, that’s an interesting question?
Sounds a lot like Terminator movie to me !
Me:
Yes and Yes. It is well known to double down on its lies. E.g. Where it produced fake legal citations, “Mr. Schwartz, apparently believing the magic of ChatGPT, asked it to verify that the cases were real and it told him that they were!” [The Legal Hallucinatory Detectorist]
-+-+-+-+-+-
Allen Woods (How the hell did that happen?):
On my profile under the “old geek” role, you will find a briefing note on LLM’s. If legals are smiling about ChatGPT, then they really need a sanity pill. The next Horizon will come out of it as sure as eggs are eggs…. Help yourself to the note or not as the case may be.
Pleading, learn something of how this stuff works.
Me:
The legal futurists tell us it will get better and this is just the beginning. No one told the sub-postmasters that! But yes I agree… “as sure as eggs are eggs”. There will be yolks on some faces.
N.B. Allen Wood’s ‘Briefing note on ChatGPT‘ [PDF]
-+-+-+-+-+-
ChatGPT and The Lawyer Scandal
And then I came across an example of the ChatGPT hubris amongst lawyers inexplicably at work. Carolyn Elefant had posted on LinkedIn:
Are courts weaponizing 🏹 use of ChatGPT against pro se litigants?
Sure seems like it.
In preparing for three upcoming presentations on lawyers’ use of generative AI, I searched reported cases for ChatGPT sightings. Of the 7 I located (again, only reported cases), 3 involved either sanctions or suspicion against pro se litigants who employed ChatGPT to assist with pleadings.
Yes, that’s right. Courts are sanctioning folks who can’t afford lawyers for availing themselves of tools to improve the quality of their filings.
Seriously?
We have tools that may finally deliver on the goal and professional obligation to expand access to justice. And yet courts mete out penalties to litigants who try to use them.
Yet another example of why legal can’t have nice things.
Many American lawyers or legal technologists nodded in agreement with Carolyn. With comments such as:
This is gross.
Disturbing but all too believable.
That surprises me, although it really shouldn’t. 🤯
Sad really.
“Yet another example of why legal can’t have nice things.” – Truth 💯
This is horrible and gross.
That’s truly pathetic.
The Opposite is Surely True?
I disagreed:
But surely these are clear examples of the fact that we do not yet have tools that may finally deliver on the goal and professional obligation to expand access to justice?! We are, it would appear, a long way away from that and these tools are arguably making the position worse not better.
Likewise, Thomas Kerner (Wilmington, NC Business Law, Non-Compete, Litigation, Transactional and LLC Formation Attorney), said:
The first one of the three highlighted summaries isn’t clear as to the basis but the second and third are. It looks like the 2nd and 3rd ones have to do specifically with misrepresentation of the law. Doesn’t matter if they cribbed it from an internet forum or copy/pasted out of an outdated formbook. If it’s a misrepresentation, it wastes peoples’ time and money and is sanctionable. Your premise is they’re using it “to improve the quality of their filings” but it sounds like it’s done the opposite.
I would have to read the orders, but it doesn’t look like they’re picking on pro se’s *merely* for using AI, but for submitting false characterizations of the law, then trying to blame the AI.
Using ChatGPT to summarise the legal cases about the use of ChatGPT
There must be something more inquisitive in the genes of Scottish lawyers (by birth or domicile or training) when compared to their American cousins.
Arlene McDaid (Lawyer| Mediator| Tech) raised an interesting point by asking Carolyn:
Are the summaries in your table AI-generated? (the image forming part of your post suggests use of Anthropic).
I’m curious, as on reading the Mescall v Renaissance at Antiquity Order (13 Nov 2023), the brief summary in your table risks the reader drawing the wrong conclusions around the impact the alleged use of AI had in the case.
While the Order refers to the Defendants’ allegation that the Plaintiff’s Response appeared to have been partially written with the aid of AI, there was no suggestion that the “court agrees [that] blind AI reliance [is] no excuse for misrepresenting [the] law.”
Interestingly, a paraphrased variant of that sentence features in the Scott v. Federal National Mortgage Association Order (14 Jun 2023) mentioned in your table.
Over the years, there have been all manner of new technologies and tools heralded as potential A2J solutions. We really need more critical thinking and better objective evaluation of these tools. The wider hype and blindly optimistic coverage and discussion of AI tools as A2J solutions gives undue credence to them and puts people at risk.
Graeme Johnston (Software to map work – before that a lawyer) had similar thoughts:
Interesting cases. Out of interest, is the summary table human-prepared or itself an AI artifact?
On looking up the opinions in the three highlighted cases (links in next comment), there are some serious problems with two of the three summaries:
1. In Middlebrooks, the summary in the table is misleading. The pro se plaintiff wasn’t sanctioned and the court’s only concern was whether some human had helped her, in violation of local UPL rules. On the possibility of AI, the court is actively supportive: ‘Maybe she even used ChatGPT or another artificial intelligence software. All of these explanations are perfectly rational and lawful…’
2. In Mescall, the pro se plaintiff was also not sanctioned. In fact, the court gives a chance to address various problems and guidance on how to do so. The summary is misleading in two respects: (1) the court doesn’t accuse the plaintiff of using ChatGPT, it simply notes the defendant’s assertion and advises the plaintiff not to use AI inappropriately, giving examples of what that means, (2) the ‘blind reliance’ language comes from the ScOtt case, not from Mescall.
3. In ScOtt, the summary in the table is more reasonable, though ‘misrepresentations partly blamed on overreliance on AI tools like ChatGPT’ doesn’t quite communicate how serious the problem was – fictitious citations presumably ‘hallucinated’ 100% (not ‘partly’) by the AI tool. The sanction is simply a requirement to pay the other party’s reasonable costs of the motion, with an opportunity to comment on the amounts claimed before the court determines the matter.
Links:
Middlebrooks: https://casetext.com/case/middlebrooks-v-city-of-macon-bibb-cnty
Mescall: https://storage.courtlistener.com/recap/gov.uscourts.ncwd.112222/gov.uscourts.ncwd.112222.21.0.pdf
ScOtt: https://casetext.com/case/scott-v-fed-natl-mortg-assn
I commented in response to Graeme:
Another great example from those points on why we (party litigants or not) should not be relying on AI for anything that needs legal thought and reasoning.
The Dangers of today’s Horizon (ChatGPT) in the hands of Lawyers
So there we have it. A perfect example of the dangers of ChatGPT not only in the hands of pro se litigants (party litigants as we call them in Scotland or litigants in person as they are called in England & Wales) but also in the hands of lawyers using it to analyse the cases involving those pro se litigants.
This all highlights that another Horizon, this time involving ChatGPT, is sadly very likely just around the corner. It will be a travesty if lawyers take centre stage in that scandal.
ChatGPT, like Horizon, is not a “nice thing” and legal shouldn’t have it if they, like The Post Office, can’t handle it.
