• Predicting litigation outcomes using data science. 0:01
  • Tom Hagee reflects on past attempts at weather prediction and height predictions for babies.
  • Dan Rabinowitz, co-founder and CEO of Predictor, discusses using AI to predict litigation outcomes.
  • Dan’s company conducted a case study on how judges’ political background affects rulings, with fascinating results.
• Predicting judicial outcomes using data analytics. 5:08
  • Dan Rabinowitz shares his professional journey from practicing attorney to predicting judges’ rulings.
  • Rabinowitz seeks to harness big data, analytics for predictive justice.
• Using machine learning and AI to analyze court data for predictive purposes. 9:02
  • Dan Rabinowitz explains how his company uses machine learning and AI to analyze court data and identify predictive factors in cases, such as the parties and attorneys involved.
  • Rabinowitz highlights the limitations of using raw statistical data alone, as it can be misleading and doesn’t provide insight into the underlying rationale for judicial decisions.
  • Dan Rabinowitz explains how he breaks down parties, attorneys, and judges into their “DNA” by analyzing their characteristics and creating an ontology.
  • By analyzing massive datasets of cases and docket entries, Rabinowitz identifies patterns in the data that can help predict outcomes in legal disputes.
• Using behavioral analysis to predict judicial behavior in legal cases. 14:29
  • Attorneys rely on limited data from colleagues to assess judges, despite using more comprehensive research for legal briefs.
  • Rabinowitz explains why attorneys are skeptical of AI-powered prediction models for judicial behavior, despite their high accuracy.
• Estimating litigation time and cost based on case characteristics. 18:27
  • Dan Rabinowitz explains how his company analyzes cases to estimate time to resolution.
  • He highlights the importance of identifying similar cases to create accurate models.
  • Dan Rabinowitz: Attorneys analyze case to predict outcomes, budget costs.
  • Rabinowitz: Timelines, motion models help companies minimize litigation time, cost.
• Using AI to predict court outcomes based on judicial elements. 23:35
  • Dan Rabinowitz explains how his company’s approach to understanding relevant statistics in class certification motions is limited by the judge, but can be expanded by incorporating similar judges’ cases.
  • Rabinowitz discusses how his company’s agnostic approach to the law and facts enables them to determine venue selection before a case is filed, even accounting for the judge’s ruling.
  • Dan Rabinowitz explains how his company’s analysis can help clients determine their likelihood of success in a lawsuit, saving them millions of dollars in legal fees and costs.
  • Rabinowitz provides an example of how his analysis helped a client assess their chances of success in a federal lawsuit in Florida, with potential savings of 10s of millions of dollars.
• Using AI to predict judges’ decisions based on their personal characteristics and past rulings. 28:31
  • Dan Rabinowitz explains how behavioral analytics can be used to predict judges’ decisions by analyzing their genetic makeup, experiences, and characteristics.
  • Rabinowitz discusses how this approach can be applied to appeals, despite the limited availability of data at that level.
  • Dan Rabinowitz discusses using behavioral analytics to predict judicial decisions, acknowledging the challenges and limitations of this approach.
  • Rabinowitz’s company uses a randomized exclusion method to test the accuracy of its predictions, comparing them to real-world outcomes.
• Using data analytics to predict court outcomes, debunking assumptions about political affiliation’s impact on judges’ decisions. 33:49
  • Dan Rabinowitz: Clients want data-driven approach to legal advice, not just billable hours.
  • Law firms can now show clients confidence in case length with data analysis.
  • Dan Rabinowitz and his team analyzed court cases involving corporations and found that relying on a single characteristic, such as political affiliation, does not have predictive value.
  • The team’s multifaceted analysis showed that combining different data points leads to an entirely different conclusion than what people often assume, and that gender is not a determining factor in judges’ decisions.
• Judicial appointments and their impact on court decisions. 39:22
  • Dan Rabinowitz analyzes judicial appointments by Obama and Trump, finding differences in their choices.
  • Trump’s judges are less favorable to corporations than Obama’s, with a 10% difference in the closest grouping.
  • Dan Rabinowitz: Judges’ gender doesn’t affect decisions (13 words)

Study finds no significant differences in business friendliness between Dem, GOP judges

SUMMARY KEYWORDS

judge, case, attorneys, litigation, clients, motion, characteristics, approach, summary judgment, simply, litigating, appointed, dismiss, predict, talk, outcome, law, statistics, behavioral analytics, patterns

SPEAKERS

Dan Rabinowitz, Tom Hagy

Tom Hagy 

Welcome to the emerging litigation podcast. This is a group project driven by HP litigation, now part of critical legal content and V Lex companies fast case and last week media. I’m your host, Tom Hagee, longtime litigation news editor and publisher, and current litigation enthusiast who wish to reach me, please check the appropriate links in the show notes. This podcast is also a companion to the Journal of emerging issues and litigation, for which I serve as editor in chief, published by fastcase. full court press. Now, here’s today’s episode. If you like what you hear, please give us a rating. There of course was a time when we could not predict the weather. But that didn’t stop us from trying. We looked at things like what time of year did the acorn start falling off the trees? Or was there an odd smell in the air? I say we like I was, you know, a weatherman during the Civil War. Apparently, if the frogs are suspiciously quiet, that was some kind of a signal. And today still did a plump Rodin, who was dragged cranky from his den cast a shadow. So yeah, we’ve always tried without science. And at times with disastrous results. Of course, we also could not predict how tall a baby would be when he or she became a full grown adult. But we thought we could. I was born during the Eisenhower administration. When I was three, my pediatrician told my mom, because I was listening, I was probably trying to put my fist in my mouth, that I would grow to be six feet and three inches tall. My mom was thrilled. She was also Italian. And no one in her family was even six feet tall. That would have been a tower. But based on my doctor’s calculations, and given my dimensions 1960 I would grow tall enough to fetch for my mom a carton of cigarettes off the highest shelf in any grocery store or pharmacy or church. six foot three. All I can say is not even close. Maybe it was the secondhand smoke. A couple of episodes ago we talked about career paths litigators take outside of litigation. Certainly make sense. And you have years of immersion in complex business disputes, it’s bound to shine a light on some problems are out there begging for solutions, apart from you know bringing your case to a satisfactory resolution. Our guest is another such person. And what he observed was a laborious and ineffective slog of trying to forecast how long a case would take how much it might cost. Something pesky clients always want to know, or in which jurisdiction a case might have the greatest hope. Or how a judge might rule it’s the Holy Grail. On on summary judgment motions. So this these are among the problems he set out to address. And now in the age of large datasets and artificial intelligence. He believes predicting outcomes across a span of litigation will become as routine as checking the weather. And who knows, maybe I’ll have a growth spurt. He is Dan Rabinowitz. He is co founder and CEO of predictor, a company that provides litigation prediction and forecasting services. Dan practiced as an associate in the Sidley Austin Supreme Court and Appellate Group and in the firm’s mass tort litigation group. Later he served as trial counsel for the US Department of Justice and General Counsel to a Washington DC based data science company. He was also Associate General Counsel, Chief Privacy Officer and the director of fraud analytics for WellPoint military care. Yeah, listen toward the end. You know, we have about 890 federal judges and roughly 30,000, state court judges, Dan and his group did a case study when they set out to test an assumption that how a judge might rule can be determined in part by the president who appointed that judge or that judges political party or that judges gender. I found the outcome fascinating. And as a voracious news consumer, I couldn’t help but notice lately, how often the media when reporting on, you know, which judges overseeing a case or or judges ruled on a case, how they refer refer to their political background, or the president that appointed them. That’s new. So look for a link to that case study in the show notes. I think you’ll you’ll find it interesting. And now here’s my interview with Dan Rubin. Know it’s co founder and CEO of predictor. Opie enjoy it. Dan Rabinowitz, thank you very much for taking the time to talk to me today.

Dan Rabinowitz 

Absolutely. Thank you for inviting me.

Tom Hagy 

We’re talking today about your innovative product that you have called called predict, I’ve already introduced you and and a bit about the product. But just reading your description of what it does, gives a monumental advantage in to lawyers and the ability to strategize for risk exposure, litigation, likelihoods settlement strategy, etc. Not just law firms, but companies too. And so this is about predictions. So first, talk about, if you could talk about your professional journey, I found it interesting.

Dan Rabinowitz 

Absolutely. So I’m a former practicing attorney, I started my career at a large law firm, and eventually did and in the DC area, and eventually did, if you will, the DC Circuit of jobs going into going into the Department of Justice, and then eventually in house had a government contractor. But while I was at the large firm, you know, as large firms, you know, really do the best and cover all the bases, when they’re litigating on behalf of their clients, I was once tasked with a project, whereby I was trying to assess or predict how a judge might rule. And the task was go through all of the judges, motions to dismiss rulings, and see how then the judge would rule it for our motion to dismiss. Now, when I went back and did that, first of all, there were only maybe 20 or 30, motions to dismiss in total that were that were reported or unreported opinions. And none were in the area that we were litigating products liability. And I was just struck by, you know how that might not be the best approach in order to form a prediction. And that is simply because there’s a lack of data, as it relates to opinions, judges write opinions, and fewer than 2% of all motions to dismiss slightly higher when it comes to other types of motions. So with that limited data set, you know, you’re missing either, you know, anywhere from 95 to 98% of all data, needless to say, to extrapolate from that is not the best way to understand statistics. But in the course of my legal career, I was exposed to concepts of big data, data analytics, and harnessing that power. And in particular, trying to look at data in a very different way than I think most attorneys would look at data as it relates to the judicial proceedings. When I was exposed to it, it was much more in the idea of creating linkage between non obvious elements. So rather than simply looking for a direct connection, let’s say, you know, a particular legal question, and then an outcome. There’s so many other factors that we can use in order to form predictions. And of course, we always have to distinguish between outcome and prediction. A prediction is just as it says, it’s how we think, or how we believe or how we’ve done a data analysis in our instances is not based on belief. But how the case where the judge might rule that is, of course different than whether or not you’re, you know, how you’re going to craft your brief, or what legal issues you may raise. So we then in terms of predictions, it enables us to open the aperture significantly as to which characteristics and elements that are involved in any given case, might have predictive value. So that’s, you know, where I got to and then ultimately, decided to really pursue that avenue, rather than continuing to practice law however much. I enjoyed that. Yeah.

Tom Hagy 

Okay. So, so tell me about you talking about the aperture and opening up and then when, what is the, what is this, I guess, algorithm or what is your service trained on?

Dan Rabinowitz 

Yeah, so what we what we wanted to do is rather than looking at that limited data set that attorneys typically use and legal research, again, for the predictive elements, we had to look beyond that. So the largest data source of decisions of outcomes rather than opinions, is, of course, either pacer or an equivalent system within state courts. And that records every judicial event, whether it be you know, an entry of appearance, telephonic conference, but also records, all the decisions that the judge makes, and by decisions, that means the grandson denials, so there’s no context to those. So now that you have a large enough data set, you’re now limited though, because you don’t have any of the rationale as to what what led to that outcome. So we’ve people solve for one problem, but then we need to solve for a different problem with, how do we now use that information. And of course, simply using it from a statistical perspective, from a raw statistical perspective, is non predictive. And in fact, in most instances misleading, and that means that if you would look at a judge, and you go use pacer data, and you say, well, the judge grants motions to dismiss in 80% of all cases, anyone with even a basic understanding of statistics knows that that does not mean you have an 80% chance of your motion to dismiss being granted. Because unless you know what puts you in the 80%, or put you in the 20%, that statistic is meaningless. I mean, it’s nice to know from maybe from an historical perspective, if you’re tracking trends, and you’re an academic, but certainly from a practitioner, or from a client perspective, that number alone doesn’t get you anywhere. So we took those numbers, though. And then what we wanted to do was try to understand if we could see patterns within those numbers. So again, rather than trying to discern which you cannot, the underlying rationale, we said, let’s look at those numbers 80% 70%, and try to see if using machine learning and eventually AI, can we see patterns within them. Now the patterns that we wanted to identify because ultimately, our analysis bore out that the predictive factors in cases are the parties and the attorneys, and I’ll get to the judge in a moment. And by parties and attorneys, we from, you know from from a data perspective, what we do is we break the parties down in the attorneys down into component parts. So rather than simply looking at a company, as just the name of the company, they are how many times this is better when a motion to dismiss, again, you’re faced with the same challenge that unless you know what drove those cases that you know, they won 90% of the time or lost 10% of the time, that’s not helpful. So instead, what we do is we create an ontology or a structure as it relates to every party and every attorney. So Bayer becomes a pharmaceutical company, that’s also a medical device company that’s traded that’s publicly traded, that’s on the s&p that’s located, you know, internationally, it’s internationally headquarters has a certain amount of revenue. Now, once we break the party down, and then we do the exact same for the attorney and the law firm, we now have rather than simply having, you know, Bayer versus Glaxo Smith, on one side, on the other side, we now have, if you will, a whole string of characteristics versus a whole string of characteristics on the other side of the V. Right, so now we’ve taken these four variables, plaintiff and defendant, and then their representative attorneys. And now we’ve created all of these different characteristics and component parts, if you will, the DNA, if you want to look at it that way, of each of those four variables. Now, once you’ve broken it down to those, you know, and you’ve created this DNA structure, or just for purposes of analogy, now, you can start seeing patterns within that, well, if it’s a medical device company, but not a pharmaceutical company, and it’s domestic and it’s privately held. Can we see as we parse through the data, any patterns now, of course, in order to do this type of analysis, you need Massive Datasets. So we looked at, I think the number now is around 15 million. So different cases and docket entries, in order to create and understand these particular patterns. And that’s the only way that you really can do this, because of course, you’d have a very small data set, inevitably, you’re going to run up again, that you’re not going to have sufficient representation of any given type of characteristics. Now, once you do that, and of course, you also have to weigh and determine which of these are signal which of these are noise, you know, when you’re looking at statistics, you always have to understand sometimes it’s the what you’re looking at is irrelevant. And you might see a pattern, but the pattern is not really meaningful. And that would be the noise. And then of course, you have to identify the signal. So once we identify those patterns now what we can now do is incorporate that other key component parts to it, and that is the judge. And we do the exact same process that we did for the parties and the attorneys with the judge. So again, we break the judge down into their DNA and of course, their DNA, you don’t know Anyway, interrupt for a second, of course.

Tom Hagy 

Talk more about the attorneys, the the doing the DNA of attorneys. Sure. So what kind of things do look out with them?

Dan Rabinowitz 

So what we’re generally looking at is association with firms. And we’re trying to understand an attorney associated with the particular firm. What does that structure look like? For any given attorney? There isn’t enough data and this is the problem that you actually have with the current approach that many attorneys take for better for worse, many attorneys will say, you know, and I experienced this certainly where, you know, the firm is, you know, just signed up a new client and that they have A piece of litigation and district about Iowa before a certain judge, and they send out a firm wide email. Does anyone have any experience with Judge so and so, you know, we just, you know, we were we just, you know, our client was just sue there, so on and so forth. And inevitably, they’ll have one or two lawyers that, you know, respond saying, Oh, yes, I appeared before her specially tough on defendants. Now, what they failed to tell you is that they appeared before as an NA, USA 90% of their cases were criminal. This is, of course, a civil contract matter. And even if they appeared before, with any number, you know, with, with any meaningful amount, maybe 1015 20 times, the judge might hear three, four or five cases a day and rule on that many. So their experience is so limited. Again, it goes back to the opinions. But yet attorneys rely on those, you know, very rudimentary assessments of judges. Now, when they’re doing legal research, they would never, you know, rely on that small of a data set, right? They, they wouldn’t find one case, or two cases, right, that they got from their colleagues that feedback that they got and say, Okay, I’m done writing the brief, I found that two cases, I don’t have to do any further research yet, when it comes to sort of the identification of judges, and how they and how they might interact with the particulars of the case. That’s what they do. Now, to be fair to them. The reason that attorneys do that is because there is no effective way to do that, to do that analysis, through legal research. The only way to do that type of analysis is through the approach that we have through this behavioral analysis, but behavioral analytics, more trying to look at different factors, alternative factors, because again, we aren’t interested in writing briefs, we’re interested in predicting judicial behavior. And that’s very, very different. And that in many ways, you know, as you might imagine, when I talked with attorneys about our capability, in some instances to predict with a near 85% rate of accuracy without looking at the facts, and the law, attorneys are incredibly skeptical. And they say no, I can do a better job, or how does that make any sense? I wrote the best brief. Now, of course, that’s true. And we don’t, our approach is not intended, if you will, to enter the realm of legal practice. That is totally apart from what we’re doing. We are not looking for outcomes, we are simply looking for predictions. And in order to get to predictions, you do have to step outside of the traditional legal research, the traditional legal arguments. So that’s where, you know, our the difference in understanding and appreciating discerning human behavior, versus discerning legal arguments, or determining legal arguments is is on is almost like two different tracks that totally apart from one another.

Tom Hagy 

Yeah, makes total sense. So let’s talk about the different applications. I mean, that that you all visual support attorneys, and with motion prediction, litigation timelines, motion models, judicial benchmarking, and venue selection. So you’ve talked a bit about about this, you know, motion prediction. But it also helps you said with litigation timelines, how does that work? Yeah.

Dan Rabinowitz 

So you know, again, let’s go back a bit to the way that this has been done in the past. And that way, we can perhaps see the difference and in our approach, and why our approach is considerably more valuable. Now, when attorneys are trying to determine Well, how long is the case going to take? Right? It’s the question that they’re, of course, gonna get asked by the client, because time is money, right? At the most basic level. So in terms of estimating budgets, terms of estimating resources, whether it be on the client side, right, how long are they going to have to have this distraction of litigation? On the attorney side? You know, how long are they going to have to test and associate or just generally? So, again, to go back to the raw statistics, so if you look at raw statistics, and you say, Well, I can look at, you know, Judge X, and she takes overall in terms of her cases, whether it be for if it’s, you know, summary judgment, and just overall, you know, 350 days for summary judgment. And, you know, if it goes to trial, 700 taste. Now, if you’re not looking at the key characteristics of the case, those numbers can be incredibly skewed. And it’s a very simple, you know, if you just take a very simple example, if, for example, like those 359 or 360 days for summary judgment, let’s just say, if that number includes a lot of individuals litigating against one another, with, you know, lower dollar values with smaller firms potentially, and you’re litigating, you know, it’s a big firm in animal law 50 against another and law 50. And there is, you know, $1.2 billion at issue, and there are multiple plaintiffs and multiple defendants, needless to say, A that that statistic of 360 days because it includes, you know, 95% of those are when, you know, it’s single plaintiffs against single defendants that that has no, that has no bearing on your case. So if you would use that, and you tell your client, well, the average time for summary judgment is little over a year, needless to say, five years in your claim is gonna be what happens here, right. So our approach, again, is to identify similar cases. And again, similar cases means that those five characteristics, the parties, the attorneys, the respective attorneys, and then the judge, so we want to find cases that are like one to one, because if you can find other cases that are almost 100%, similar, or how we create our models, which ways different component parts, so therefore, we can get to a high degree of similarity, even if not every single component matches up. So now you can say with a straight face, oh, it’s actually going to take 720 days to get the summary judgment. If it goes to trial, it’ll take X number. But then beyond that, well we also recognize is, is that there are many different ways that that a piece of litigation can conclude, of course, motions is one motions for dismiss motions for summary judgment. But then there are any number of other ways settlement, there are alternative forms of dismissal. You know, all these different factors, and each of those, depending on and here’s really where the attorneys experience. And, you know, their research about the case, and their analysis about the case really comes in, especially if they’re looking at the very beginning, when they’re looking to provide their clients with serious estimates and budgets, they can then do their research and say, Well, I see this case going two ways. Either it’s going to be, it’s not going to happen on a motion to dismiss too complicated for whatever reason. Instead, we see this as going either to settlement or to summary judgment. So now they look at our timeline, and they say, Okay, well, if it goes to settlement, it’s actually around six months shorter than it would take for us to go to summary judgment. So now, you’re litigating the case. And of course, you gave your estimate at the beginning. But now this is goes even beyond the estimate and budgeting. If you’re litigating your case, and you receive a settlement offer, well, do you take it, say, Well, I have a really strong summary judgment argument. But if that’s going to take you another six months, or a year of time and cost? Well, that’s something that certainly should come into the analysis of how you should approach that settlement offer, maybe you don’t take that one, we take slightly higher one, but at least you now understand, and you’re gonna have an appreciation of what the cost of summary judgment is, because it’s not cost free. Right, you might have to go through any number of additional depositions might have to hire experts if you don’t already have those. And then, of course, just the, you know, the briefing and the argument, and again, every piece of litigation is disruptive for most companies, and you can’t minimize that companies are produced widgets, or whatever it is, they certainly shouldn’t be involved in litigation any longer than needed. So that’s where our timelines are particularly effective. And all of those sorts of use cases. Now, when it comes to when it comes to our motion models, it’s very similar, where again, we’re looking to provide a one to one, rather than simply saying, again, a judge dismisses 80% of the time, we’re looking to find any number of cases and judges that are like your judge in your case. So now we can say, well, summary judgment in New York judge in your case, is 52%. So now you’ve been confident about that. Now, one of the really interesting applications of our approach to understanding how to identify relevant statistics is we aren’t limited by the judge, meaning because we’ve now broken that Judge up into their DNA, there are many other judges that might share enough similarity within that DNA. And that allows us to capture additional cases beyond the judge. And this is especially important for motions that are that are truly significant, and perhaps case changing. But any given judge has a very small number of those, and that is a class certification motion. Motion judges have, I would say, I would say less than five, if you look at you know, all federal judges. So if you want to say, well, the judge last time granted this or two out of two out of five cases, you know, the judge granted is three out of five, right? So 60 or so percent, right? That that is way too small of a dataset to make any sort of, you know, statistical leap. But if we can incorporate, let’s say, 150, because we’ve identified enough judges, or even 75 cases, enough judges that are similar to your judge, that more or less are the same, and we can include now their cases. So now we can actually provide a meaningful statistic that you could use to assess what the likelihood of cert of class sir being granted here. Now, you also mentioned venue selection. You Now, this is another way or another area where the approach of being agnostic to the facts and the law enables us to do something that is nearly impossible otherwise. So prior to a case being filed, even, even after our case was filed, I’ll come back to transferring venue before cases filed, how do you determine where to file that case? You can say, well, there’s you know, the law in this jurisdiction might be better than that jurisdiction, where we have plaintiffs who we think might be, you know, more amenable to class certification than the other. But how do you really assess, you know, that’s very nice, but you’re gonna go before a judge? And how is that judge is going to rule. So we can run our analysis, even before filing. And the reason is, is because we don’t account for the law and the facts, what we do is we simply look at who the parties are, who the attorneys are. Now, of course, when you file in any given jurisdiction, depending on the jurisdiction, there’s three judges, five judges, 20, judges, so how do we account for the fact that you don’t have a particular judge? So we do is we in a way, very simplistic level, we aggregate all the judges, although, depending on any number of factors, each is valued differently. And then we provide an overall score for the entire jurisdiction. So you can now say, well, we file in this jurisdiction, we have a 70% overall chance of surviving a motion to dismiss. And if we file on this jurisdiction, we only have a 30% chance of surviving motion to dismiss. Now, I’ve always heard the complaint, you know, like, this is venues, you know, forum shopping, right? Is it forum shopping? If you look at the law, and you say, well, we the law in this instance, is 30%. You know, it’s not in our favor, and this jurisdiction, it’s 70% in our favor. And if an attorney said, despite the fact that the law in this jurisdiction is really not in our favor, we’re gonna file there anyways, that would probably, I won’t say necessarily be malpractice, but certainly approaches malpractice. So how can you then say that we’re that we’re doing something wrong when we’re simply taking the the judicial elements into account. Now, this is not only for plaintiffs looking to file, we actually had an instance where a client approached us and they had already been sued, the client had already been sued. And they were in one of the Health Center in Florida in federal court. And they wanted to know, their likelihood of success on their motion to dismiss, of course, you know, the litigation had anywhere between, you know, 25 and $40 million at issue, that was the claim. And sure enough, we ran, you know, our analysis, and we said, you actually have a very, very low likelihood of your motion to dismiss being granted. To translate that, that means it’s going to go to discovery, there’s going to be an incredible cost to the client. And maybe you’ll get out with some settlement, hopefully, less than whatever the original demand was. They said, one of the plane, one of the defendants, excuse me, is in California. What’s their likelihood in the Central District of California, right around that, and they had an over 70% likelihood that their motion to dismiss would be granted. So simply by using this, they literally may have saved their client 10s of millions of dollars, by running our analysis, and our analysis is so simple that it just requires them, all we need to do is look at their case. And then it takes us under five seconds, to provide that information, to provide information that can literally save 10s of millions of dollars simply using this and deploying this strategically. You said

Tom Hagy 

you’re looking at behavioral analytics, when it comes to a judge. What what goes into that that’s not available in legal research.

Dan Rabinowitz 

So what we’re doing with the judges, again, we’re not looking at their opinions or decisions, we don’t care about the judicial philosophy or approach to the law. Again, we’re looking at their genetic makeup. So in this instance, rather than, you know, breaking down the party isn’t so publicly traded, etc, we’re looking at the components or characteristics that make the judge right, that make us as human beings. So those are our experiences, who we are right, so we’re going to look at, of course, the obvious ones law school, where they practice wasn’t in public service or private practice, maybe where they state court judge or politician before being elevated to the bench, as well as non obvious ones, let’s say age, or net worth, or any number of other characteristics that have seemingly no relation to the law. And truthfully, they probably don’t have any relation to the law. But in terms of using that for behavioral analytics, those can be very valuable. And the way I like to think about this is when it comes to you know, you’re on your computer and you’re doing a search or you’re, you know, you’re on let’s say, your newspaper or whatever it is, and an ad pops up, and it says, Hey, you can vacation in the Bahamas. And you just say wow, I was just literally talking to my wife about that 20 minutes ago and I hadn’t thought of it before that and you know, immediately we think maybe Google is listening in on our phone or some other device. And maybe maybe not, maybe not. But presumably, the the actual approach is something very similar where they, they understand all of our characteristics. They know where we live, they know who our neighbors are, they know our level of education, they know how much money we make, they know who we’re emailing, right. And with that information, as well as understanding our buying patterns. So rather than looking at our judicial decision, or patterns, they understand our buying patterns. So they can combine those two elements, and then provide those highly targeted ads that predicts seemingly our behavior right, now I am going to take that trip to the Bahamas. So that in a way is what we’re doing. We’re combining the personality elements, the characteristic elements of the judge, and then with their decisions and understanding patterns, and then linking that to how the how, or what they’re made off.

Tom Hagy 

How does this work with appeals? Because I guess the data, the data wouldn’t be as much data but but the judges do have careers. So they have rulings and behaviors and said this is this is apply at the appellate level.

Dan Rabinowitz 

So we have not taken it to that level. But certainly in terms of our product timeline, that is an area that that we are keenly interested in exploring. It is, you know, a little bit more difficult, but not, you know, it has a different challenge that of course, we have to account not for one judge, but for three judges at the same time. That simply means conceptually, that rather than having one generic, if you will, character, we have three. So we would combine all three and try to decide, you know, patterns that are associated with those three collectively or individually, and go ahead and and moves to the appellate round. But certainly that’s an area that that we hope to explore. Because, you know, our approach is, if we can understand behavioral analytics, if we can understand how people behave. That applies, like I said, to the Bahamas trip to judges making decisions and presumably appellate judges, the one area and I, you know, I’ve been asked this, you know, fairly frequently is, what about Supreme Court judges, I would argue that anyone that’s looking to predict how a Supreme Court judges will rule is probably on a fool’s errand because there’s such a small dataset, you think about the number of cases that they hear. And then the number of judges or justices that which one is writing which opinion or decision. And, frankly, while now we have, if you will like the shadow docket, but for the most part, most of the law, you know, the most significant cases, there are written opinions. And you can discern judicial philosophy based upon those. So there’s really no need for behavioral analytics. And frankly, it’s really impossible. I would argue to use them as it relates to Supreme Court judges.

Tom Hagy 

So how’s it going? What’s What’s your track record?

Dan Rabinowitz 

So again, so the way that we look at a track record is not simply by looking at, you know, any handful of cases, you know, if our clients have used them for 510 15 cases, even if you know, we get all 15, right, or we get 13 wrong, what you really have to do, in order to assess accuracy is not simply, you know, hoping the best for, you know, 10 cases, error analysis, in order to determine accuracy, we exclude, we randomly excluded 50,000 motions from our models. And then after we have to rebuild our models, we would then run our model. So in other words, our model had never seen those 50,000 It was it was running those blinds, and then our model would provide its prediction, and then we will compare that against the real world outcome. And that’s how you test accuracy, you know, looking at a handful of cases, obviously, isn’t all that helpful in terms of accuracy. Now, in terms of how our clients are deploying this, they’re certainly deploying it, and it really empowers them in a very, very different way. And it offers them a different approach to how they can advise their clients, you know, for settlement, as I as I described, or which which motions to litigate, where to understand where the, the, you know, how to how, and which motions they should be filing. Additionally, from their clients perspective, you know, I, I, when I went in house, I assumed that you know, pretty much it’s the same as practicing law, you’re no longer on the billable hour, which is kind of nice, and you get to tell other attorneys what to do. But I understood that there’s a fundamental difference where your clients are no longer other attorneys. When When, when you’re in the firm, your clients are attorneys, when you’re in house, your client is someone on the business side, and what they they’re really not interested is in some detailed, you know, nuanced approach to the law that, you know, the attorney spent who knows how many hours writing and then you know, and doing research and coming up well, you know, your case and based on some new legal precedent, and maybe it’ll go this way or that way, the way that most large businesses make this decisions, and they all have significant components, you know, of the organization is all based on data analytics, it would be silly to ignore that. And in fact, you know, nearly every company of any size has recognized, they need to crunch the data. So from our client from the law firm and their clients perspective, they can now go to the their ultimate clients on the business side and show them that they too, are operating with the same approach. They too, are in sync with how the business is operating. And they can give them something very different. And simply saying, Well, you know, here’s a memo, where here’s where our attorneys thinking about this. Now they can say, look, we’ve done deep data analysis, we’ve had, you know, the sophisticated algorithmic models determine how long this case will take. So we can now say in our attorneys have also looked at the facts and the law with confidence, this is going to be a five year slog, it’s very, very different than I think, the way it’s been done before. And that certainly, really puts those attorneys in the firm’s that are using our product on a very different level than those that are still operating just with the more traditional tools of research writing. And then, of course, not even getting into, you know, utilizing statistics that are meaningless or many times wrong, and just lead to incorrect assumptions and advice. Well,

Tom Hagy 

speaking of assumptions, I thought it was very interesting, a very hot topic that you you addressed in a case study was the assumption that whether a judge is considered liberal or appointed by somebody considered liberal, or conservative, whether that has an impact on the outcomes, their decisions with regard to corporations. So the the assumption would be that a liberal judge is going to rule against corporations, conservative judge oral for them, or maybe gender comes into play. did Obama appoint them to Donald Trump point appoint them? So tell me a little bit about what the what you found in that case study? Sure.

Dan Rabinowitz 

So first and foremost, what what our ability to do that analysis is only because we’ve done that classification, and, and and of the various parties and attorneys. In other words, if you go to any database, we can try to discern, well, which cases involve corporations? But just start with that question that, you know, because of the needs for our predictive modeling, we’ve already looked at that aspect of the cases, may those, you know, surfaced those and therefore enables us to go ahead and limit the cases that we’re looking at that involve corporations, that’s first. Second, what we’ve determined is that relying upon any given characteristic, whether it be you know, political affiliation, gender, any number of those, certainly in isolation does not form any does not have any predictive value. Now, of course, in some instances might be borderline, you know, problematic from, you know, any number of discrimination and so on. But people cannot be reduced to one or two or three elements. Instead, if you really want to understand behavior, you have to look at some multifaceted analysis. And what we were trying to do with that case study is tease that out on a very, you know, simple level, and take the assumptions that people jump to all the time, and show that actually, if you start combining different data points, it leads you to an entirely different conclusion. So as you said, Some people assume that, you know, political party, and there’s, you know, the Democrats or liberals are more biased against corporations. And when you look at that, overall, there is a slight difference between judges appointed by Democrat presidents and and and Republicans. But that is, again, too simplistic. What we wanted to dig in and start saying, Well, what about if you incorporate gender and again, there is some notion that, you know, certain genders are more favorable, and certainly when you combine them, again with political affiliation, and what we actually found that that that is an incorrect if you actually crunch the data. So for example, female judges appointed by President Obama are as favorable. In other words, they are the equivalent of Republicans overall. So rather than that being if you will, less favorable to corporations, they’re as favorable as if you will, the the highest favourability towards corporations. Now, on the other hand, the least favorable towards corporations are female judges, appointed by President Trump. Now let’s sort of walk this back and take off our blinders. And recently, unfortunately, one of the greatest behavioral analytics I’m thinkers, Daniel Kahneman passed away. And he’s talked about a lot how, you know, we have these inherent biases, or he calls on heuristics that we just take. And we just make all these different types of assumptions. And this is a perfect example. But when and he’s done this, where, if you take a step back, rather than simply jumping to the conclusion, you actually take a step back. And of course, his book is called thinking fast thinking slow. Try to demonstrate this. You actually are you’re actually the light bulb goes off if you only say, Huh, no. Now I understand that. So this is, I think, a pretty good example about that, if you think about and obviously this is somewhat generalistic. But think, is, is pretty true. When you look at who President Obama appointed. What types of judges did he appoint? Well, he was appointing judges that typically went to top tier law schools that worked in big law that may have left that for some major multinational corporation. Right? And yes, they are, you know, they may have been donors or affiliated with donors or somehow have some political connection. But that’s a very particular type of attorney, and now they’re on the bench. Now, let’s look at President Trump. Now, I think everyone can agree, irrespective of their political views that President Trump is an anomalous president, as it relates to even Republicans, he’s a different type of Republican. And when you think about the different type of Republican and even how, quote unquote pro business he is, or not pro business, the judges that he appointed, were not traditional Republican judges, they didn’t necessarily come from the same pool, they didn’t come from a date, they didn’t necessarily attend the same law schools. They didn’t necessarily work in the same fields that other Republican appointed judges did. And so now, once you take that step back and start thinking slow, you say, Wow, so that is certainly anomalous. Now, once something is anomalous, inevitably, it will present itself in an anomalous way. Now, it could very well be that they were the most favorable by 15 points, but here, it’s actually represented by they’re the least favorable by 10 percentage points, by the closest, closest grouping. And again, once you take that step back, once you start thinking about rather than simply jumping those conclusions or Republican, Democrat, man, male, female, gender, not gender, I mean, none of these that is so reductive, if I walked into, you know, the way like the cocktail party, and someone introduced themselves and said, Well, I’m a Republican, and I say, Well, I know everything about you. Right? That’s essentially what, what you’d be doing, right. But that’s not what you can do. If you have a company like ours that’s looking to provide, you know, highly sophisticated attorneys and their clients with information that they really can rely upon. You can’t simply be that reductive, either in terms of simply by looking for biographical characteristics, or looking on the other flip side of that we’re looking at those raw statistics that don’t have any connection with the particulars of the case at hand.

Tom Hagy 

Okay, well, I think the outcomes or the conclusions from the from that study, were fascinating because it does, it goes against everything. You might think one way or the other. When she said, female judges appointed by President Obama were just as business friendly as GOP appointed judges. You know, in some differences, you saw that really only a couple points in either direction. So anyway, as I said before, I I love actual facts. And so, and this is such an important topic right now that I think people will be interested in, take a look at that. And I’ll flag that paper. So people want to take a look and see what you did there. Perfect. Thank you. Well, Dan Rabinowitz, thank you very much for talking to me today. This is a fascinating topic.

Dan Rabinowitz 

No, Tom, thank you for having me. Really appreciate the discussion.

Tom Hagy 

That concludes this episode of the emerging litigation podcast, a co production of HP litigation, critical legal content, V. Lex fast case, and our friends had lost the media. I’m Tom Hagee, your host which would explain why I’m talking. Please feel free to reach out to me if you have ideas for a future episode. And don’t hesitate to share this with clients, colleagues, friends, animals, you may have left the home teenagers, you irresponsibly, left unsupervised and certain classifications of fruits and vegetables. And if you feel so moved, please give us a rating those always help. Thank you for listening