On robot lawyers and predicting the future

The holy grail for some software developers often seems to be something that gets rid of lawyers.

I will be accused of blindness or self-interest, but whatever the marketing pitch du jour, my suspicion is that we will be around long after software developers build systems that replace themselves — the last buggy whip makers, if you will.  See, for example, DoNotPay, which claims to be the world's first "robot lawyer."

(Narrator: it may be useful, but it's not a robot and it's not a lawyer.  There are still no pieces of software that are licensed to practice law in the United States),

Maybe I will be proven wrong and will be replaced in five years. But I doubt it.  That said, the more likely near-term outcome is that, like prior technological developments, new tools will change the profession of law and the need for certain types of support staff.

It may also create a smaller bar in the U.S., along the lines of the Japanese Bengoshi or U.K. Barrister models. Certain types of key functionality will remain with humans for the forseeable future. Experience also suggest that the utility of a legaltech tool will depend on the use to which it's put by a lawyer who actually knows what they're doing.

It's one thing to develop platforms that generate forms or assist with massive document reviews. It's another thing to exercise judgment and discretion and provide advice based on those things.

One thing lawyers are often asked to do — it's been something I've been tasked with since the very early days of my career — is to provide predictions of case outcomes. I saw this in action as a baby lawyer from the partner I worked with, a guy who could take five or 10 pieces of data and, with remarkable accuracy, tell you what a plaintiff was likely to recover if the case went to trial.

Mr. James was an old school Trial Lawyer (deliberately capitalized), a gentleman, and a generous teacher. He tried more than 200 jury trials and resolved 3,000 others. And law was his second career! I still marvel at his predictive skills and have wondered for years if software could make the same sorts of accurate judgments with the same small set of data.

What it means for software to exercise judgment is maybe a topic too weighty (and broad) for this simple essay. My own view, in brief, is that something which exercises judgment is necessarily something more than software. So my answer (for today, anyway) is "no" — software can't make judgments unless the software has become human.  

(If you think this is a science fiction question, incidentally, all Gaius Balthar and no substance, the folks at the Centre For Existential Risk at Cambridge will probably beg to differ.)

But perhaps these are irrelevant semantics. What we want is a something or someone that can make an accurate prediction. So, setting aside weightier question about AI for the moment, how about something simpler? What about a settlement calculator? Something that weighs relevant factors and spits out a settlement value range — it ought to be simple to build something like that, right? How would we do so? And what do tools like that tell us, if anything, about the practice of law or the future of legal technology?

Having pondered some of these questions for years, I tried several years ago to create a very simple settlement value widget, using some newly learned and rudimentary Javascript, and incorporating some of the basic inputs that a lawyer might include in an assessment.

Here's what the widget did: given certain inputs, a user receives a lawsuit's "risk-weighted net present settlement value." Or (in plain English) an answer to the question that we lawyers frequently get and don’t always know how to answer: "what’s it worth?" 

Here is how it worked: you take a range of likely provable damages, weight by (1) chance of success/failure, (2) impact of opposing counsel on the case, (3) impact of court/venue/judge, and (4) add expected legal fees/costs. That will add up to a high/low range which if you could pay today would be a reasonable settlement value.

Think this is simplistic? Maybe. This is how it's done, day in and day out by defense lawyers. Is this the "right" way to "value" a claim? Maybe. Maybe not. It depends — maybe you could make more money playing the horses in the interim. Approaches like this do seem to have some predictive ability, but only if you know what questions to ask and which factors are relevant.

This was not the most brilliant or highly developed piece of software ever. The script was easy enough to put together and deploy. One could pretty easily gussy up the user interface with pretty sliders and dials. But all of that is secondary to getting the right inputs.

How does a lawyer know which inputs are relevant? We often use heuristics, or learned rules of thumb to help. For example, to determine potential value of an injury claim, you might multiply medical special damages by three if you are in X county and by one if you are in Y county. That might provide some rough predictive value (I gather from lawyer friends in St. Louis that this particular rule of thumb may have changed in the last 10 years for the venues where I learned this).

Things like the "impact" of a particular lawyer on a particular case may be harder to assess. Another lawyer I once worked for liked to say "it's harder to dance with a bad dance partner" — and it can be more expensive. So, the fact that the other side has a "great" lawyer (Chambers rated, years of experience, dozens of trials, etc.) doesn't necessarily mean that a case is worth more from a settlement value standpoint. It might make the case less expensive. And expense (at least in this model) has an impact on "settlement value." This oversimplifies something really hard, but that's (sort of) what a good heuristic does, and does well.

Mr. James could generate the right inputs quickly and accurately. He would laugh at me for saying so, perhaps rightly, but he had great heuristics. And that allowed him, generally speaking, to make remarkably accurate predictions. The explanation is simple – right? Years of experience, judgment honed over time. And two hundred jury trials. He knew the lawyers, the judges, had seen every kind of fact pattern. I think he would have looked at my script and guffawed. And not because he was a Luddite, but more likely because he knew how hard the inputs would be for most people.

In short: Mr. James had Really Good Judgment. The widget had No Judgment At All. And I am skeptical that a really fascinatingly complex black box AI would have Really Good Judgment of the James variety, either.

On the one hand, lawyers are not going to be replaced by a widget just yet — at least, not mine.

Without judgment and training, garbage will result in garbage. Someone still has to figure out what the questions are. On the other hand, lawyers face an experience challenge. How many of us will ever get to try 200 jury trials? And if we can't get the experience Mr. James had, can we ever hope to develop the same judgment?

Bottom line — the data, and the questions, have to be selected and weighed by someone who can judge what is important and what isn't. Until software tries cases, I am not sure that the human experience that guides predictions and advises based on this will be easily replaced anytime soon. Predictive software without a Mr. James is not worth the code it's written in, which begs the question of whether this sort of machine makes sense in the first instance.  

My own conclusion is about my widget was that it was fun to make but actually quite worthless as software without my interaction and involvement.