A 2017 GAO study found that Social Security disability outcomes could swing by forty-six percentage points based solely on which ALJ heard the case. Same claimant, same record, same hearing office, different judge. Individual ALJ approval rates range from under ten percent to over ninety. That's not noise — it's the system.
In this episode — the first of a three-part series on how agencies actually produce outcomes — Gwen and Marc step outside the hearing room and examine the three variables that shape results before a single question gets asked:
- Judge assignment: Random allocation protects against selective routing, but paired with forty-six points of variation, it produces something that looks like a lottery. The system chose neutrality over consistency — and consistency doesn't get recovered downstream.
- Representation: Claimants with attorneys get roughly double the approval rate of those without — not mainly because of what happens in the hearing, but because of record-building beforehand and something more structural: since ALJ decisions aren't published, the only way to know how a specific judge handles specific issues is to have appeared in front of them. Experienced local attorneys hold what amounts to privatized law — accumulated operational knowledge that doesn't exist in any public source.
- Geography: Hearing offices differ by ten to fifteen points in approval rates beyond what the mix of judges explains, driven by local economic conditions, regional medical infrastructure, and office-level practice cultures that develop over time.
A national program, a single statutory standard, and systematically different outcomes depending on which judge you draw, whether you can afford a lawyer who knows that judge, and which office covers your zip code. The binding agency-level law that does exist — Social Security Rulings, HALLEX — covers a narrow band of interpretive questions. Everything outside that band is where the variation lives.
Next episode: if horizontal consistency doesn't exist at the hearing level, does top-down appellate review fix it? Spoiler — it doesn't.
00:00:14 --> 00:00:17 Hello, and welcome to Administrative Remedies,
00:00:17 --> 00:00:19 because you can't fix what you don't understand.
00:00:19 --> 00:00:21 Brought to you in part by the University of Tulsa
00:00:21 --> 00:00:24 College of Law. I'm Gwendolyn Savitz, an associate
00:00:24 --> 00:00:26 professor here at TU and the associate dean of
00:00:26 --> 00:00:28 research and intellectual life. And I'm Mark
00:00:28 --> 00:00:31 Rourke. I'm the dean of the College of Law. We'll
00:00:31 --> 00:00:33 be breaking down complex doctrines with real
00:00:33 --> 00:00:35 -life analogies and examples to demystify the
00:00:35 --> 00:00:37 world of administrative law for everyone trying
00:00:37 --> 00:00:39 to understand how government actually works.
00:00:40 --> 00:00:41 Agencies are the main way the federal government
00:00:41 --> 00:00:44 gets things done. It's not through Congress for
00:00:44 --> 00:00:46 reasons we'll be addressing over the course of
00:00:46 --> 00:00:59 this series. Mark, last episode, you asked the
00:00:59 --> 00:01:01 question that we're going to start with today.
00:01:01 --> 00:01:03 If we have the same claim and the same record,
00:01:04 --> 00:01:06 but a different judge, how different is the likely
00:01:06 --> 00:01:11 outcome? And we can say pretty precisely how
00:01:11 --> 00:01:14 different that is. There's a 2017 Government
00:01:14 --> 00:01:17 Accountability Office study of Social Security
00:01:17 --> 00:01:20 disability decisions. Read this part. The allowance
00:01:20 --> 00:01:23 rate could vary by as much as 46 percentage points
00:01:23 --> 00:01:26 if different judges heard a typical claim, one
00:01:26 --> 00:01:29 that was average in all other factors GAO analyzed.
00:01:30 --> 00:01:33 There's a 46 -point spread just from which judge
00:01:33 --> 00:01:37 hears the case. Same claimant, same record, same
00:01:37 --> 00:01:39 hearing office, different judge, and individual
00:01:39 --> 00:01:42 ALJ approval rates go even wider. The overall
00:01:42 --> 00:01:45 range runs from under 10 % to over 90%. That
00:01:45 --> 00:01:49 didn't sound like just noise. Yes. So last time,
00:01:49 --> 00:01:51 we were inside a single hearing, and we talked
00:01:51 --> 00:01:53 about how credibility gets assessed, how burdens
00:01:53 --> 00:01:56 get applied, how those interact for the claimant
00:01:56 --> 00:01:59 in that room. But last time, everything we talked
00:01:59 --> 00:02:01 about lived inside the four walls of the hearing
00:02:01 --> 00:02:04 room. Now we step outside those walls. If we
00:02:04 --> 00:02:07 have the exact same claimant with the same credibility
00:02:07 --> 00:02:09 profile, the same burden, the same evidence,
00:02:09 --> 00:02:12 but we put that in front of a different ALJ or
00:02:12 --> 00:02:14 a different hearing office with a different representative,
00:02:15 --> 00:02:18 then the outcome can swing by 46 points. None
00:02:18 --> 00:02:20 of that variation is explained by anything that
00:02:20 --> 00:02:22 happens inside the hearing. It's explained by
00:02:22 --> 00:02:25 which hearing you end up in. So credibility and
00:02:25 --> 00:02:27 burden aren't wrong. They're just incomplete.
00:02:27 --> 00:02:29 Right. If they were wrong, we wouldn't have spent
00:02:29 --> 00:02:32 an entire episode talking about them. So they're
00:02:32 --> 00:02:35 definitely a factor, but they're just two inputs
00:02:35 --> 00:02:38 operating inside the hearing. So this episode,
00:02:38 --> 00:02:42 we talk about what happens before that, which
00:02:42 --> 00:02:44 judge you draw, whether you have representation
00:02:44 --> 00:02:47 and where you live. Those determine a huge amount
00:02:47 --> 00:02:48 of what will happen to you, and they're all fixed
00:02:48 --> 00:02:51 before you walk into the room. So the variation
00:02:51 --> 00:02:54 in the number, the input puts that produce it
00:02:54 --> 00:02:57 are what this episode is actually about. Right.
00:02:57 --> 00:02:59 That's the distinction. Which judge, whether
00:02:59 --> 00:03:02 you have a lawyer, where you live, what tools
00:03:02 --> 00:03:04 exist to pull any of those towards the same answer
00:03:04 --> 00:03:06 for similar cases? That's what we're talking
00:03:06 --> 00:03:09 about. OK, let's start with the basic idea that
00:03:09 --> 00:03:12 in courts of stare decisis, that two similar
00:03:12 --> 00:03:15 cases with similar facts should come out similarly
00:03:15 --> 00:03:19 because prior decisions bind later ones. Right.
00:03:19 --> 00:03:22 So we could say, why is it that these ALJ cases
00:03:22 --> 00:03:25 are coming out so differently? Why wouldn't they
00:03:25 --> 00:03:28 do the same thing? ALJs don't bind each other
00:03:28 --> 00:03:32 and just like neither do trial judges. So a district
00:03:32 --> 00:03:35 judge in the Southern District of New York isn't
00:03:35 --> 00:03:37 bound by another Southern District of New York
00:03:37 --> 00:03:39 judge. They can disagree openly and they frequently
00:03:39 --> 00:03:42 do. And it's not just that they disagree. Litigators
00:03:42 --> 00:03:45 actively strategize around it. If you have a
00:03:45 --> 00:03:47 choice about where to file and in federal civil
00:03:47 --> 00:03:50 litigation you often do, you think hard about
00:03:50 --> 00:03:52 which district and sometimes which division your
00:03:52 --> 00:03:55 case ends up in. That's why some districts get
00:03:55 --> 00:03:58 a reputation for being the best place to file,
00:03:58 --> 00:04:01 say, patent troll lawsuits. Hat tip to you, Eastern
00:04:01 --> 00:04:05 District of Texas. But there are multiple judge
00:04:05 --> 00:04:08 divisions where you at least know the pool. Plaintiff
00:04:08 --> 00:04:11 side lawyers forum shop for sympathetic precedent
00:04:11 --> 00:04:13 and sympathetic judges. Defense side lawyers
00:04:13 --> 00:04:15 remove cases to federal court partly for the
00:04:15 --> 00:04:19 same reason. Right. So choosing where you're
00:04:19 --> 00:04:21 going to file this forum shopping strategy, this
00:04:21 --> 00:04:24 is considered normal practice. Completely normal.
00:04:24 --> 00:04:27 It's what anyone experienced would do. Nobody
00:04:27 --> 00:04:29 pretends the variation between trial judges is
00:04:29 --> 00:04:32 small enough to ignore. You pick your venue,
00:04:32 --> 00:04:36 if you can, and when you can't, you prepare for
00:04:36 --> 00:04:38 the specific judge you drew. The variation is
00:04:38 --> 00:04:41 just part of how the system works. Right. So
00:04:41 --> 00:04:44 a claim that... Courts have consistency and agencies
00:04:44 --> 00:04:47 don't would be wrong on its face. Both systems
00:04:47 --> 00:04:49 can have significant judge level variation that
00:04:49 --> 00:04:52 sophisticated parties strategize around. So that's
00:04:52 --> 00:04:54 not the difference between them. OK, so what
00:04:54 --> 00:04:56 is the difference? It's the tools available for
00:04:56 --> 00:04:59 navigating the variation. When a federal litigator
00:04:59 --> 00:05:01 prepares for a judge, they haven't appeared before.
00:05:01 --> 00:05:03 What do they do? Well, they pull the judge's
00:05:03 --> 00:05:06 published opinions. They read their rulings on
00:05:06 --> 00:05:08 similar motions. If there's a recent precedent,
00:05:08 --> 00:05:12 recent ruling on the same kind of issue. that's
00:05:12 --> 00:05:14 the first thing you would pull to read. You'd
00:05:14 --> 00:05:17 pull unpublished opinions. Those are citable
00:05:17 --> 00:05:19 as persuasive authority in most circuits and
00:05:19 --> 00:05:22 in some circuits. Now, even unpublished opinions
00:05:22 --> 00:05:24 can be cited for their reasoning. So you would
00:05:24 --> 00:05:26 look at whether the judge tends to grant or deny
00:05:26 --> 00:05:30 12b -6 motions on a particular theory, and you'd
00:05:30 --> 00:05:32 read their bench rulings in similar cases and
00:05:32 --> 00:05:35 hire local counsel if you really need a deeper
00:05:35 --> 00:05:37 dive on the judge's practices. All right. So
00:05:37 --> 00:05:41 how is it so easy to get all of that? Well, because
00:05:41 --> 00:05:43 the reasoning is accessible. A sophisticated
00:05:43 --> 00:05:47 party can genuinely get up to speed on a judge
00:05:47 --> 00:05:49 they've never appeared before in a week or two
00:05:49 --> 00:05:52 of focused work. All right. That's what's missing
00:05:52 --> 00:05:54 at the ALJ level, not the variation. The variation
00:05:54 --> 00:05:57 exists in both systems. What's missing is the
00:05:57 --> 00:06:00 research path. The ALJ's prior decisions aren't
00:06:00 --> 00:06:02 published. The reasoning isn't accessible. And
00:06:02 --> 00:06:04 a lawyer preparing to appear before a new ALJ.
00:06:05 --> 00:06:07 can pull up the aggregate approval rates SSA
00:06:07 --> 00:06:10 publishes, but that's essentially it. They can't
00:06:10 --> 00:06:12 read prior decisions. They can't pull unpublished
00:06:12 --> 00:06:15 rulings. They can't see how that ALJ has handled
00:06:15 --> 00:06:17 similar medical issues in the past. The research
00:06:17 --> 00:06:19 path that makes federal trial court variation
00:06:19 --> 00:06:22 navigable for sophisticated parties just doesn't
00:06:22 --> 00:06:25 exist here. Okay, so two different parties, the
00:06:25 --> 00:06:27 same kind of variation, and completely different
00:06:27 --> 00:06:30 ability to prepare for it. Yes, and the guidance
00:06:30 --> 00:06:33 from above can also be really sparse. Vertical
00:06:33 --> 00:06:35 binding, like... Some higher pronouncement that
00:06:35 --> 00:06:38 would bind the ALJ is going to be really thin
00:06:38 --> 00:06:40 at agencies like Social Security. The Appeals
00:06:40 --> 00:06:42 Council doesn't issue presidential decisions.
00:06:42 --> 00:06:45 The binding law comes from federal courts, which
00:06:45 --> 00:06:47 see only a tiny slice of the cases, and from
00:06:47 --> 00:06:50 agency -wide interpretive rulings that cover
00:06:50 --> 00:06:53 super narrow questions. We'll come back to those.
00:06:54 --> 00:06:56 It's also an accessibility issue. So individual
00:06:56 --> 00:06:59 ALJ decisions in disability cases generally don't
00:06:59 --> 00:07:01 go anywhere. They're not published. They're not
00:07:01 --> 00:07:03 in a database. They're certainly not systematically
00:07:03 --> 00:07:05 available to lawyers researching how similar
00:07:05 --> 00:07:07 cases have been decided. Wait, so they are completely
00:07:07 --> 00:07:11 not available? Right. Not as a systematic matter.
00:07:11 --> 00:07:14 A disability practitioner can get ALEKS, which
00:07:14 --> 00:07:17 is SSA's Internal Procedural Manual. That's public.
00:07:17 --> 00:07:19 They can get Social Security rulings, which are
00:07:19 --> 00:07:22 agency -wide policy statements that do bind ALJs
00:07:22 --> 00:07:26 on interpretation. Those are also public. They
00:07:26 --> 00:07:28 can get federal court decisions in their circuit.
00:07:28 --> 00:07:31 And the Social Security Administration publishes
00:07:31 --> 00:07:34 aggregate statistics about AJLJ, their approval
00:07:34 --> 00:07:36 rates, their denial rates. But these are aggregate
00:07:36 --> 00:07:39 statistics. They're not decisions. Right. So
00:07:39 --> 00:07:42 the practitioner can know that ALJ Thompson has
00:07:42 --> 00:07:46 a 32 % approval rate and ALJ Martinez has a 64
00:07:46 --> 00:07:48 % approval rate. But what they won't know is
00:07:48 --> 00:07:50 how either of them handle the specific legal
00:07:50 --> 00:07:53 or factual question in a prior case. They can't
00:07:53 --> 00:07:55 pull up their reasoning. They can't research
00:07:55 --> 00:07:57 whether Thompson tends to discount treating physician
00:07:57 --> 00:08:00 opinions when there's been like a long gap in
00:08:00 --> 00:08:02 treatment or whether Martinez is particularly
00:08:02 --> 00:08:04 attentive to functional capacity evaluations.
00:08:05 --> 00:08:07 That's information that exists only in the ALJ's
00:08:07 --> 00:08:10 case files, which are closed. So the federal
00:08:10 --> 00:08:12 litigator can research their judge. The disability
00:08:12 --> 00:08:15 practitioner can research their judge's box score.
00:08:15 --> 00:08:20 Right. And the gravitational pull that that information
00:08:20 --> 00:08:23 produces in the court system, that those prior
00:08:23 --> 00:08:26 opinions, the fact that judges know their colleagues,
00:08:26 --> 00:08:28 the bar can read what they write, that pull is
00:08:28 --> 00:08:30 absent here. There's nothing equivalent pulling
00:08:30 --> 00:08:33 ALJs towards some sort of shared practice. Which
00:08:33 --> 00:08:35 sounds like the straightforward, there is no
00:08:35 --> 00:08:39 precedent to this story. Right. It's similar
00:08:39 --> 00:08:42 to that, but it's not quite that because there's
00:08:42 --> 00:08:43 something that can substitute for this, at least
00:08:43 --> 00:08:45 to some extent. It's just not publicly available.
00:08:46 --> 00:08:50 You mean the lawyers themselves? I do. So the
00:08:50 --> 00:08:52 experienced disability bar in any hearing office
00:08:52 --> 00:08:55 can develop in knowledge of local ALJs. They've
00:08:55 --> 00:08:57 appeared in front of them a bunch of times. They
00:08:57 --> 00:08:59 know how they think. They know what evidence
00:08:59 --> 00:09:01 moves them and what doesn't. They know the specific
00:09:01 --> 00:09:03 ways they ask about different daily activities
00:09:03 --> 00:09:06 and how to prepare a client for it. That knowledge
00:09:06 --> 00:09:08 exists. It's just concentrated in the hands of
00:09:08 --> 00:09:10 attorneys who've been doing this for years, and
00:09:10 --> 00:09:12 it doesn't travel outside that circle. Which
00:09:12 --> 00:09:14 is where the representation starts to matter
00:09:14 --> 00:09:17 in a different way than we've talked about. Right.
00:09:17 --> 00:09:19 And we're going to be there pretty soon. But
00:09:19 --> 00:09:21 first, I want to deal with the other piece of
00:09:21 --> 00:09:22 what determines an outcome at the hearing level.
00:09:23 --> 00:09:25 So before this can even matter, you have to get
00:09:25 --> 00:09:27 assigned to a judge. And how that happens is
00:09:27 --> 00:09:30 a design choice. OK. How does assignment actually
00:09:30 --> 00:09:33 work? The case gets sent to an office that covers
00:09:33 --> 00:09:36 that claimant's whole geographic area. And within
00:09:36 --> 00:09:38 that office, it's assigned to whatever ALJ is
00:09:38 --> 00:09:40 available. There are some exceptions. There are
00:09:40 --> 00:09:42 some specialized dockets at a few offices and
00:09:42 --> 00:09:45 some case management tweaks. But the baseline
00:09:45 --> 00:09:48 is just this randomness. And random assignment
00:09:48 --> 00:09:52 is defended on what grounds? Well, the same way
00:09:52 --> 00:09:54 we defend it in court, neutrality. Nobody picks
00:09:54 --> 00:09:57 your judge. No one can stack your case. You can't
00:09:57 --> 00:09:59 form shop within an office. The agency can't
00:09:59 --> 00:10:01 route sympathetic claimants. Random assignments
00:10:01 --> 00:10:05 eliminate a whole category of abuse. That's fair,
00:10:05 --> 00:10:07 and that's a real virtue. Yeah, it protects against
00:10:07 --> 00:10:09 selective routing. The question is what it produces
00:10:09 --> 00:10:13 when you pair it with that 46 % variation that
00:10:13 --> 00:10:16 we opened with. Sounds like it produces something
00:10:16 --> 00:10:19 that looks like a lottery. So it produces something
00:10:19 --> 00:10:21 that looks like a lottery in the most literal
00:10:21 --> 00:10:23 sense. Two claimants with the same profile file
00:10:23 --> 00:10:25 their applications on the same day. They get
00:10:25 --> 00:10:27 randomly assigned to different judges, and the
00:10:27 --> 00:10:30 outcome swings by 46 points. So this random assignment
00:10:30 --> 00:10:32 means that you're not going to be singled out,
00:10:32 --> 00:10:34 but it doesn't guarantee you're going to be treated
00:10:34 --> 00:10:37 consistently with anyone else. Those are different
00:10:37 --> 00:10:40 forms of fairness, and the system has definitely
00:10:40 --> 00:10:42 chosen the first one. The cost of that choice
00:10:42 --> 00:10:45 is consistency. But what are the alternatives?
00:10:45 --> 00:10:48 Is anyone doing something different? Some offices
00:10:48 --> 00:10:51 have tried using case complexity scoring to get
00:10:51 --> 00:10:53 harder cases to go to more experienced judges.
00:10:54 --> 00:10:56 Some have tried specialized dockets like mental
00:10:56 --> 00:10:58 health claims to one group of judges, physical
00:10:58 --> 00:11:00 impairment claims to another, on the theory that
00:11:00 --> 00:11:03 specialization could help produce consistency
00:11:03 --> 00:11:07 and expertise within a given case type. But the
00:11:07 --> 00:11:09 research on whether those actually narrow the
00:11:09 --> 00:11:13 variation is mixed. But why are they mixed? The
00:11:13 --> 00:11:15 variation isn't coming from case complexity or
00:11:15 --> 00:11:18 subject matter mismatch. It's coming from how
00:11:18 --> 00:11:20 judges weigh evidence in credit testimony, things
00:11:20 --> 00:11:22 that persist even when you control for case type.
00:11:22 --> 00:11:25 So specialization might reduce some variation,
00:11:25 --> 00:11:27 but it won't eliminate it. And any move away
00:11:27 --> 00:11:29 from pure random assignment opens the door to
00:11:29 --> 00:11:32 some version of routing, which reintroduces the
00:11:32 --> 00:11:34 judge selection concern that we were trying to
00:11:34 --> 00:11:36 solve. So you're basically picking your poison.
00:11:37 --> 00:11:40 Yes. The standard story is that random assignment
00:11:40 --> 00:11:43 would be fair. Variations... that result from
00:11:43 --> 00:11:46 this are unfortunate but incidental, and that's
00:11:46 --> 00:11:48 not really quite right. What the system has done
00:11:48 --> 00:11:51 is prioritize neutral allocation over outcome
00:11:51 --> 00:11:54 consistency. These are two totally different
00:11:54 --> 00:11:57 things. Neutral allocation is meaning that nobody
00:11:57 --> 00:12:00 picks your judge. Outcome consistency means that
00:12:00 --> 00:12:02 similar cases should come out similarly. Random
00:12:02 --> 00:12:04 assignment delivers the first, it doesn't deliver
00:12:04 --> 00:12:07 the second. And when you compare this with the
00:12:07 --> 00:12:09 variation between the judges, it can actually
00:12:09 --> 00:12:11 prevent the second. Which, given the variation
00:12:11 --> 00:12:14 you just described, means consistency isn't just
00:12:14 --> 00:12:18 deprioritized at the assignment stage. It's effectively
00:12:18 --> 00:12:20 foreclosed. Right. Not by anyone choosing to
00:12:20 --> 00:12:22 foreclose it, but as a structural consequence
00:12:22 --> 00:12:24 of pairing random assignment with the level of
00:12:24 --> 00:12:26 variation that the system is going to inherently
00:12:26 --> 00:12:30 produce. So this is supposed to be neutral allocation.
00:12:31 --> 00:12:34 Consistency is traded for the ability to get
00:12:34 --> 00:12:36 this neutrality. Whether the trade is worth it
00:12:36 --> 00:12:39 is a separate issue. What we're trying to talk
00:12:39 --> 00:12:41 about today is just that the trade was made and
00:12:41 --> 00:12:43 the consistency that got traded away doesn't
00:12:43 --> 00:12:46 get recovered. OK. And the downstream mechanisms
00:12:46 --> 00:12:49 like appellate review, representation, federal
00:12:49 --> 00:12:52 court review, those are what the system offers
00:12:52 --> 00:12:55 in place of that consistency. Right. And the
00:12:55 --> 00:12:56 next couple episodes, we're going to be looking
00:12:56 --> 00:13:00 at ways that effort is made to try and corral
00:13:00 --> 00:13:05 these decisions. But they still won't produce
00:13:05 --> 00:13:08 the lack of variation that we might. hope for.
00:13:09 --> 00:13:12 Okay. So you get assigned to a judge randomly.
00:13:13 --> 00:13:15 You might have drawn a judge who approves a third
00:13:15 --> 00:13:18 of his cases or one who approves two thirds.
00:13:18 --> 00:13:20 What else shapes the outcome at that point? This
00:13:20 --> 00:13:22 is where we're back to talking about representation.
00:13:23 --> 00:13:26 And this effect is huge. Claimants without representation
00:13:26 --> 00:13:29 get roughly half the approval rate of claimants
00:13:29 --> 00:13:33 with representation. Doubled? Yeah, roughly.
00:13:33 --> 00:13:35 So the exact numbers can vary depending on the
00:13:35 --> 00:13:38 study and how you count it. But this dramatic
00:13:38 --> 00:13:40 increase is consistent. Represented claimants
00:13:40 --> 00:13:42 do substantially better at the hearing level
00:13:42 --> 00:13:46 than unrepresented claimants. And there's a few
00:13:46 --> 00:13:48 different reasons for this. It's not necessarily
00:13:48 --> 00:13:50 what people automatically assume. They might
00:13:50 --> 00:13:53 think that it's just how they perform in the
00:13:53 --> 00:13:55 room. The lawyer shows up, they make a better
00:13:55 --> 00:13:57 argument, and that can matter at the margins,
00:13:57 --> 00:14:00 but it's not the main thing. The main thing happens
00:14:00 --> 00:14:02 before the hearing. Okay, so building a record
00:14:02 --> 00:14:04 like we talked about in the life cycle episode,
00:14:04 --> 00:14:07 getting the treating physician opinion, getting
00:14:07 --> 00:14:10 the functional capacity evaluation, making sure
00:14:10 --> 00:14:12 the medical evidence is complete, anticipating
00:14:12 --> 00:14:15 what the vocational expert will say, and preparing
00:14:15 --> 00:14:17 cross -examination. All of that happens in the
00:14:17 --> 00:14:19 months before the hearing, and it's what mostly
00:14:19 --> 00:14:22 determines what the record looks like when the
00:14:22 --> 00:14:24 hearing starts. Right. It's huge, and it also
00:14:24 --> 00:14:27 plays into something else. If ALJ decisions aren't
00:14:27 --> 00:14:29 accessible, the lawyer can't research how this
00:14:29 --> 00:14:31 judge has previously ruled on similar cases,
00:14:31 --> 00:14:33 then what the lawyer already knows about this
00:14:33 --> 00:14:35 judge is going to be the only substitute for
00:14:35 --> 00:14:38 this kind of precedent. So an experienced local
00:14:38 --> 00:14:40 attorney is going to know how these specific
00:14:40 --> 00:14:43 ALJs in this specific office handle things. They're
00:14:43 --> 00:14:45 more likely to know what's going to convince
00:14:45 --> 00:14:47 this judge. That knowledge makes a huge difference.
00:14:48 --> 00:14:50 Without it, the practitioner is functionally
00:14:50 --> 00:14:53 working blind. The only way to know how an ALJ
00:14:53 --> 00:14:55 actually thinks is to have appeared in front
00:14:55 --> 00:14:58 of them or to know someone who has. So really,
00:14:59 --> 00:15:01 experience in a local hearing office isn't just
00:15:01 --> 00:15:04 the general purpose advantage, it's the specific
00:15:04 --> 00:15:07 substitute for the precedent system that doesn't
00:15:07 --> 00:15:10 exist. Right. So that means that what these experienced
00:15:10 --> 00:15:13 disability attorneys are selling isn't just that
00:15:13 --> 00:15:16 they understand disability law. It's this private
00:15:16 --> 00:15:18 library of knowledge that they've accumulated
00:15:18 --> 00:15:21 over the years that you can't get any other way.
00:15:21 --> 00:15:24 So this informal consistency mechanism, this
00:15:24 --> 00:15:27 trying to pull prior decisions in court practice,
00:15:27 --> 00:15:29 this is only behind the paywall of representation.
00:15:30 --> 00:15:34 Okay. What about unrepresented claimants? They're
00:15:34 --> 00:15:37 not going to have access to this. All they're
00:15:37 --> 00:15:39 doing is working from SSA's public explanations
00:15:39 --> 00:15:42 of the disability standard. There might be some
00:15:42 --> 00:15:44 sort of advocacy organization that's produced
00:15:44 --> 00:15:47 materials they're trying to use, and then whatever
00:15:47 --> 00:15:49 intuitions they developed about how the system
00:15:49 --> 00:15:52 works. And this is why the representation effect
00:15:52 --> 00:15:55 is so large. Yeah, it's part of it. This record
00:15:55 --> 00:15:58 -building piece matters, too. But I think that...
00:15:59 --> 00:16:01 This explains what's otherwise a strange result,
00:16:01 --> 00:16:04 that a system that's supposed to be non -adversarial
00:16:04 --> 00:16:06 with an ALJ who's supposed to be developing the
00:16:06 --> 00:16:09 record nevertheless produces these dramatic differences
00:16:09 --> 00:16:11 between represented and unrepresented claimants.
00:16:11 --> 00:16:14 If the ALJ really were performing the inquisitorial
00:16:14 --> 00:16:16 role we talked about earlier in the season, then
00:16:16 --> 00:16:18 representation really shouldn't matter much.
00:16:18 --> 00:16:20 It matters this much in part because the system
00:16:20 --> 00:16:22 doesn't publish its own reasoning and experienced
00:16:22 --> 00:16:24 attorneys have figured out how to compensate.
00:16:25 --> 00:16:27 Okay, let me push on what that means because
00:16:27 --> 00:16:29 I think it's sharper than what you're saying.
00:16:29 --> 00:16:32 If the substantive law that actually governs
00:16:32 --> 00:16:34 disability adjudication in a given hearing office,
00:16:34 --> 00:16:37 the way credibility gets weighed, the way treating
00:16:37 --> 00:16:40 physician opinions get handled, or the way functional
00:16:40 --> 00:16:43 capacity gets interpreted exists only in the
00:16:43 --> 00:16:46 heads of experienced local attorneys and not
00:16:46 --> 00:16:48 in any public source, then what you're describing
00:16:48 --> 00:16:51 isn't just an access problem. Okay, how would
00:16:51 --> 00:16:53 you describe it? This is basically privatized
00:16:53 --> 00:16:57 law. with the caveat that nobody privatized it.
00:16:57 --> 00:17:00 I mean, this is like the realm of having published
00:17:00 --> 00:17:04 opinions that only a few people can access. There's
00:17:04 --> 00:17:07 a published regulation, but the law as it actually
00:17:07 --> 00:17:10 operates in any specific hearing office is largely
00:17:10 --> 00:17:12 the accumulated private knowledge of the experienced
00:17:12 --> 00:17:15 bar. That's a pretty good way to describe it.
00:17:15 --> 00:17:17 So that doesn't mean that anyone's intentionally
00:17:17 --> 00:17:22 privatizing it. Nobody is. What could in principle
00:17:22 --> 00:17:24 exist as accessible operational knowledge instead
00:17:24 --> 00:17:26 exists as this experience -based knowledge held
00:17:26 --> 00:17:28 by whoever has been in those offices long enough
00:17:28 --> 00:17:32 to accumulate it. Yeah. And I mean, let's just
00:17:32 --> 00:17:35 point out one other thing. I mean, if you want
00:17:35 --> 00:17:39 to get to a level of having something in a written
00:17:39 --> 00:17:42 record that everyone can discover, you're going
00:17:42 --> 00:17:45 to spend a lot more money than just at the basic
00:17:45 --> 00:17:49 ALJ hearing level. And so really, this is going
00:17:49 --> 00:17:51 to come down to a question of who can afford
00:17:51 --> 00:17:54 to retain these attorneys to make their case
00:17:54 --> 00:17:57 both effective at the ALJ level and visible at
00:17:57 --> 00:18:01 the non -ALJ level. Yes. So this is really a
00:18:01 --> 00:18:03 structural problem. It's not just that represented
00:18:03 --> 00:18:05 claimants have a lawyer in their corner. It's
00:18:05 --> 00:18:07 that unrepresented claimants are operating without
00:18:07 --> 00:18:09 this knowledge base that they need to actually
00:18:09 --> 00:18:11 help get the outcome they're looking for. Okay.
00:18:11 --> 00:18:13 There's a third thing, though, that shapes the
00:18:13 --> 00:18:15 outcome of a hearing level that I want to make
00:18:15 --> 00:18:18 sure we cover. You mentioned it earlier, the
00:18:18 --> 00:18:22 hearing office. Right. So geography is separate
00:18:22 --> 00:18:25 from the judge level variation, and there's pretty
00:18:25 --> 00:18:27 strong evidence for it. Hearing offices have
00:18:27 --> 00:18:29 different average approval rates from each other
00:18:29 --> 00:18:31 in patterns that persist even when you control
00:18:31 --> 00:18:34 for the judges in the office. OK, what drives
00:18:34 --> 00:18:36 that then? So we've got a few things that are
00:18:36 --> 00:18:38 stacking on top of each other. We have the local
00:18:38 --> 00:18:41 economic conditions, the vocational expert testimony
00:18:41 --> 00:18:43 about what jobs are available. It's supposed
00:18:43 --> 00:18:45 to be what jobs are available in the national
00:18:45 --> 00:18:48 economy, but it can also get colored by what
00:18:48 --> 00:18:51 the local labor market looks like. We have regional
00:18:51 --> 00:18:53 medical infrastructure. If the claimant in this
00:18:53 --> 00:18:56 office's area tend to have less access to specialists,
00:18:57 --> 00:18:58 their medical records are going to look different
00:18:58 --> 00:19:00 than claimants in a region with a lot of medical
00:19:00 --> 00:19:03 resources. So the same condition, but a thinner
00:19:03 --> 00:19:06 file. Right. So less corroboration, and the ALJ
00:19:06 --> 00:19:08 can only decide based on what's in the record.
00:19:09 --> 00:19:10 There's also the local culture at the hearing
00:19:10 --> 00:19:13 office itself. ALJs who've worked together for
00:19:13 --> 00:19:15 years are potentially going to be developing
00:19:15 --> 00:19:17 shared habits. And a new ALJ coming into the
00:19:17 --> 00:19:20 office is going to drift towards those. office's
00:19:20 --> 00:19:24 norms over time. Not intentionally, not formally,
00:19:24 --> 00:19:28 but that's how people work. And what's the magnitude
00:19:28 --> 00:19:30 of this office level effect then? It's important,
00:19:30 --> 00:19:32 but it's going to be smaller than the judge level
00:19:32 --> 00:19:35 variation. Hearing offices differ from each other
00:19:35 --> 00:19:37 by something like 10 to 15 points in average
00:19:37 --> 00:19:40 approval ratings, beyond just what we would expect
00:19:40 --> 00:19:42 from the mix of judges. So this means that the
00:19:42 --> 00:19:44 claimant's geography, not just which judge they
00:19:44 --> 00:19:48 draw, but the office's territory is another outcome
00:19:48 --> 00:19:50 variable independent of anything else about their
00:19:50 --> 00:19:52 case. Okay, so a federal program with a single
00:19:52 --> 00:19:55 national standard producing systemically different
00:19:55 --> 00:19:59 outcomes depending 10 to 15 points on where you
00:19:59 --> 00:20:02 live? Right. So the standard's national, the
00:20:02 --> 00:20:05 statute's national, the regulations are national,
00:20:05 --> 00:20:08 but this variation is coming from how that national
00:20:08 --> 00:20:10 standard gets applied in the specific hearing
00:20:10 --> 00:20:12 office that happens to cover your zip code. This
00:20:12 --> 00:20:15 isn't a claim about bad offices or good offices.
00:20:15 --> 00:20:18 The variation isn't about anyone doing anything
00:20:18 --> 00:20:22 wrong. It's what national standard produces when
00:20:22 --> 00:20:24 it's administered by all of these different semi
00:20:24 --> 00:20:27 -autonomous offices with no mechanism to pull
00:20:27 --> 00:20:29 their practices toward each other. Right. And
00:20:29 --> 00:20:31 this gets us back to where we started, that 46
00:20:31 --> 00:20:33 -point judge -level variation. If we stack that
00:20:33 --> 00:20:36 on top of a 10 to 15 point... office -level variation,
00:20:36 --> 00:20:39 and we stack that on the representation effect,
00:20:39 --> 00:20:41 these aren't separate problems. A system of independent
00:20:41 --> 00:20:44 adjudicators at scale, without any sort of mechanism
00:20:44 --> 00:20:46 to pull them towards each other, is going to
00:20:46 --> 00:20:48 produce variation along every axis that we could
00:20:48 --> 00:20:51 have variation on. Okay. You've been saying no
00:20:51 --> 00:20:54 precedent in a very broad way, but you mentioned
00:20:54 --> 00:20:56 earlier that there is some agency -level binding
00:20:56 --> 00:20:59 law. So let's get specific about what that is.
00:21:00 --> 00:21:02 All right. We have two categories. The first,
00:21:02 --> 00:21:04 we have social security regulations. These are
00:21:04 --> 00:21:06 agency -wide interpretive policy, how to apply
00:21:06 --> 00:21:09 a particular statutory term, how to weigh a particular
00:21:09 --> 00:21:12 type of evidence. We have SSRs that bind ALJs
00:21:12 --> 00:21:15 on the questions that they address. They're public,
00:21:15 --> 00:21:17 they're citable, they're enforceable. So what
00:21:17 --> 00:21:20 about HALICs? HALICs is procedural. It governs
00:21:20 --> 00:21:22 how the hearings are conducted, what the ALJ
00:21:22 --> 00:21:25 has to do, what the record has to contain. It's
00:21:25 --> 00:21:28 binding on procedure. It's not binding on substantive
00:21:28 --> 00:21:31 interpretations the way SSRs are. So SSA does
00:21:31 --> 00:21:34 have some top -down law. Yes, it does. The question
00:21:34 --> 00:21:37 is just what it covers. These are specific questions
00:21:37 --> 00:21:40 that the agency has chosen to address. How to
00:21:40 --> 00:21:42 evaluate pain testimony, how to treat certain
00:21:42 --> 00:21:44 categories of medical evidence. Those questions
00:21:44 --> 00:21:47 have guidance, but a large number of the interpretive
00:21:47 --> 00:21:48 choices that an ALJ is going to have to make
00:21:48 --> 00:21:50 in any given hearing aren't covered by them.
00:21:51 --> 00:21:54 Okay, so give me an example. Okay, like how we're
00:21:54 --> 00:21:56 going to weigh the treating physician's opinion
00:21:56 --> 00:21:59 against the consultative examiner's opinion if
00:21:59 --> 00:22:01 they disagree. Or what to do when a claimant
00:22:01 --> 00:22:03 has documented physical impairments and also
00:22:03 --> 00:22:05 describes significant mental health symptoms
00:22:05 --> 00:22:08 but hasn't been formally diagnosed? And what
00:22:08 --> 00:22:11 to do if we have a long gap in treatment where
00:22:11 --> 00:22:12 the claimant says that they couldn't afford care?
00:22:13 --> 00:22:15 Those are judgment calls that happen in every
00:22:15 --> 00:22:17 disability hearing, and they're not really governed
00:22:17 --> 00:22:20 by these SSRs. So the binding top -down law covers
00:22:20 --> 00:22:23 a narrow band of the interpretive work, and most
00:22:23 --> 00:22:26 of what ALJs are doing happens outside that band.
00:22:26 --> 00:22:28 Right, outside that, and therefore outside any
00:22:28 --> 00:22:31 direct consistency mechanism. Immigration has
00:22:31 --> 00:22:33 a broader band. The Board of Immigration Appeals
00:22:33 --> 00:22:37 issues precedential decisions that cover a wider
00:22:37 --> 00:22:39 range of substantive questions, and it has mechanisms
00:22:39 --> 00:22:41 we'll talk about next episode that let it reach
00:22:41 --> 00:22:45 cases it wants to decide. But even there... The
00:22:45 --> 00:22:47 designated precedent covers a small fraction
00:22:47 --> 00:22:49 of the questions that immigration judges are
00:22:49 --> 00:22:52 deciding in any given year. And the NLRB, the
00:22:52 --> 00:22:55 SEC, and enforcement agencies? So they have their
00:22:55 --> 00:22:57 own versions, and we'll be looking at those next
00:22:57 --> 00:22:59 week. But the pattern across the agencies is
00:22:59 --> 00:23:02 roughly the same. Where binding agency -level
00:23:02 --> 00:23:04 law exists, it covers the questions the agency
00:23:04 --> 00:23:06 has chosen to address. Where it doesn't, the
00:23:06 --> 00:23:08 ALJ is working from statutes and regulations
00:23:08 --> 00:23:10 and whatever federal court opinions happen to
00:23:10 --> 00:23:14 govern, and from their own judgment. So the answer
00:23:14 --> 00:23:17 to is there no agency level precedent isn't yes.
00:23:17 --> 00:23:20 The answer is there's some. It's narrower than
00:23:20 --> 00:23:23 court level precedent. And the questions it doesn't
00:23:23 --> 00:23:25 cover are where the 46 point variation lives.
00:23:25 --> 00:23:28 Yeah, that's the easiest way to think about this.
00:23:28 --> 00:23:30 Nothing we said today means that there isn't
00:23:30 --> 00:23:32 any law. It means that the law existing at the
00:23:32 --> 00:23:35 top level of these agencies covers specific questions.
00:23:35 --> 00:23:38 And there's a huge space outside this where the
00:23:38 --> 00:23:41 individual ALJ is operating based entirely on
00:23:41 --> 00:23:44 their judgment. And subject to whatever gravitational
00:23:44 --> 00:23:47 pull the informal system produces. And that gravitational
00:23:47 --> 00:23:50 pull is mostly captured by experienced council.
00:23:50 --> 00:23:52 Right. We were mostly local to specific offices
00:23:52 --> 00:23:55 and mostly invisible to anyone outside the ecosystem.
00:23:56 --> 00:24:00 OK, let me try to summarize what we've done so
00:24:00 --> 00:24:03 far. Right. All right. So. At the hearing level,
00:24:03 --> 00:24:05 outcomes depend on which judge you draw, whether
00:24:05 --> 00:24:08 you have a lawyer and where you live. All three
00:24:08 --> 00:24:11 operate independently, all three produce measurable
00:24:11 --> 00:24:13 variation, and none of them is meaningfully controlled
00:24:13 --> 00:24:17 by any top -down mechanism. And yet, the agency
00:24:17 --> 00:24:20 has some binding interpretive law that covers
00:24:20 --> 00:24:23 a narrow slice of what ALJs actually decide,
00:24:23 --> 00:24:26 but everything else is private and in the case
00:24:26 --> 00:24:28 files, in the lawyer's heads, in the local practice
00:24:28 --> 00:24:31 cultures. Yeah. So this is the horizontal picture,
00:24:32 --> 00:24:35 the variation between individual judges at the
00:24:35 --> 00:24:37 hearing level. Which sets up the two obvious
00:24:37 --> 00:24:40 follow -up questions. Right. So if this bottom
00:24:40 --> 00:24:42 -up consistency doesn't work, does top -down
00:24:42 --> 00:24:45 review fix it? Because every agency has some
00:24:45 --> 00:24:47 sort of appellate layer. The Appeals Council,
00:24:47 --> 00:24:50 the BIA, the commissioners at the SEC and FTC,
00:24:50 --> 00:24:53 the board at the NLRB. The theory is even if
00:24:53 --> 00:24:56 ALJs are producing inconsistent decisions. The
00:24:56 --> 00:24:58 review bodies at these agencies correct errors,
00:24:58 --> 00:25:01 resolve conflicts, and pull outcomes towards
00:25:01 --> 00:25:03 consistency. That's the theory, and we're going
00:25:03 --> 00:25:05 to test it. And if the answer is that review
00:25:05 --> 00:25:08 doesn't fix it either. Which, spoiler, doesn't.
00:25:08 --> 00:25:10 Then the third episode asks the question that
00:25:10 --> 00:25:14 follows. If we have no formal mechanism, no horizontal,
00:25:14 --> 00:25:16 no vertical mechanism that's producing consistency,
00:25:16 --> 00:25:19 but agency preferences clearly do translate into
00:25:19 --> 00:25:22 outcomes, something else is doing the work. And
00:25:22 --> 00:25:24 that something else isn't. quite a set of rules.
00:25:24 --> 00:25:27 It's a set of institutional practices like performance
00:25:27 --> 00:25:29 metrics and quality reviews, case completion
00:25:29 --> 00:25:32 targets, how the agency evaluates its own ALJs,
00:25:33 --> 00:25:35 how guidance documents that aren't legally binding
00:25:35 --> 00:25:37 still shape what happens in these hearings. These
00:25:37 --> 00:25:40 are the mechanisms. They're not law, they aren't
00:25:40 --> 00:25:42 review, and they turn out to be how agency preferences
00:25:42 --> 00:25:45 generally reach the hearing room. Even if, as
00:25:45 --> 00:25:48 we've seen, there's a lot they still don't control.
00:25:48 --> 00:25:51 So the trilogy spine is outcomes depend on who
00:25:51 --> 00:25:54 you get, review doesn't fix it, and therefore
00:25:54 --> 00:25:56 control happens through mechanisms that don't
00:25:56 --> 00:25:58 look like law at all. Exactly. So this was the
00:25:58 --> 00:26:01 first. We've got two more episodes to go to talk
00:26:01 --> 00:26:05 about this. So that does it for today's episode
00:26:05 --> 00:26:07 on administrative remedies. Thank you for joining
00:26:07 --> 00:26:10 us today. Please, if you enjoy this podcast and
00:26:10 --> 00:26:13 enjoy this episode, give us a like on Spotify,
00:26:13 --> 00:26:16 iTunes, or whatever platform you're listening
00:26:16 --> 00:26:18 on. And be sure to tune in next time where we'll
00:26:18 --> 00:26:20 continue to dive into the contours of administrative
00:26:20 --> 00:26:23 law, because remember, you can't fix what you
00:26:23 --> 00:26:24 don't understand.