Why the Same Disability Claim Can Win or Lose Depending on Which Judge You Draw

The same claimant, with the same medical record, can have a 46-percentage-point swing in their odds of getting Social Security disability benefits depending on which administrative law judge happens to hear their case. That's not a marginal effect. Individual ALJ approval rates run from under 10% to over 90%. None of that variation is explained by anything that happens inside the hearing room.

This is the horizontal consistency problem in administrative adjudication. Once you step outside the four walls of a single hearing — past the credibility and burden-of-proof analysis that governs what the judge does with the evidence — you find that three things shape outcomes more than anything inside the room: which judge you draw, whether you have a lawyer, and where you live. All three are fixed before the hearing starts. None of them is controlled by anything that looks like top-down law.

This episode is the first of a three-part arc on how administrative agencies handle (or fail to handle) the variation between their own adjudicators. Federal trial courts have their own variation problem — sophisticated litigators forum-shop, strategize around specific judges, and prepare for the bench they drew. The difference isn't that courts are consistent and agencies aren't. The difference is that federal litigators have a research path: published opinions, unpublished rulings cited as persuasive authority, local counsel who know how a judge handles a 12(b)(6) motion. Disability practitioners can pull aggregate ALJ approval rates and that's essentially it. Prior decisions aren't published. Reasoning isn't accessible. The ALJ's case files are closed.

What fills that gap is a kind of privatized law that nobody intentionally privatized. Experienced local attorneys accumulate knowledge of how specific ALJs in specific offices handle treating physician opinions, treatment gaps, mental health symptoms paired with physical impairments — questions that the agency's binding rulings don't reach. That accumulated knowledge sits in the heads of attorneys who've been appearing in those offices for years, and it doesn't travel outside that circle. Unrepresented claimants approve at roughly half the rate of represented ones, and the inquisitorial-model premise that the ALJ develops the record on the claimant's behalf doesn't hold up against that disparity. The system's consistency mechanism, such as it is, exists behind a paywall.

Add a 10-to-15-point hearing office effect on top of judge-level variation, and the result is a federal program with a single national standard producing systematically different outcomes based on the zip code that determined which office covers the claim. The episode closes by setting up the next two: if horizontal consistency doesn't work, does the appellate review layer fix it? And if neither horizontal nor vertical mechanisms produce consistency, what actually does the work of making agency preferences reach the hearing room?

Listen Now

Spotify | Apple Podcasts | Listen on our site

What We Cover

• The 2017 GAO finding that approval rates can swing 46 percentage points across ALJs hearing the same typical claim — and what that variation actually represents
• Why the variation problem isn't unique to agencies, and how federal litigators navigate similar variation in trial courts through forum-shopping, judge research, and local counsel
• What's actually missing at the ALJ level — not the variation itself, but the research path that lets sophisticated parties prepare for the judge they drew
• Why ALJ decisions aren't published, why aggregate approval statistics are a poor substitute for prior reasoning, and what a disability practitioner can and can't research about the judge assigned to their case
• How random case assignment delivers neutral allocation but forecloses outcome consistency, and why those are two different forms of fairness
• Why specialized dockets and case-complexity scoring haven't meaningfully reduced variation, and what it would cost to move away from random assignment
• Why represented claimants approve at roughly double the rate of unrepresented ones — and why most of that effect happens before the hearing, not in the hearing room
• How experienced local attorneys functionally substitute for the precedent system that doesn't exist, and what that means for claimants who can't afford representation
• The 10-to-15-point hearing office effect that persists even after controlling for individual judges, and how local economic conditions, medical infrastructure, and office culture produce it
• What Social Security Rulings and the HALLEX manual actually bind ALJs on, and why most of the interpretive work happens outside what binding agency law covers

Full Rough Transcript

Gwen: Hello, and welcome to Administrative Remedies, because you can’t fix what you don’t understand. Brought to you in part by the University of Tulsa College of Law. I’m Gwendolyn Savitz, an associate professor here at TU and the associate dean of research and intellectual life.

Marc: And I’m Marc Roark. I’m the dean of the College of Law.

Gwen: We’ll be breaking down complex doctrines with real-life analogies and examples to demystify the world of administrative law for everyone trying to understand how government actually works.

Marc: Agencies are the main way the federal government gets things done. It’s not through Congress for reasons we’ll be addressing over the course of this series.

Gwen: Marc, last episode, you asked the question that we’re going to start with today. If we have the same claim and the same record, but a different judge, how different is the likely outcome?

Marc: And we can say pretty precisely how different that is. There’s a 2017 Government Accountability Office study of Social Security disability decisions. Read this part.

Marc: The allowance rate could vary by as much as 46 percentage points if different judges heard a typical claim, one that was average in all other factors GAO analyzed.

Marc: A 46-point spread just from which judge hears the case. Same claimant, same record, same hearing office, different judge, and individual ALJ approval rates go even wider. The overall range runs from under 10% to over 90%.

Gwen: That didn’t sound like just noise.

Marc: Yes. So last time, we were inside a single hearing, and we talked about how credibility gets assessed, how burdens get applied, how those interact for the claimant in that room. But last time, everything we talked about lived inside the four walls of the hearing room. Now we step outside those walls. If we have the exact same claimant with the same credibility profile, the same burden, the same evidence, but we put that in front of a different ALJ or a different hearing office with a different representative, then the outcome can swing by 46 points. None of that variation is explained by anything that happens inside the hearing. It’s explained by which hearing you end up in.

Gwen: So credibility and burden aren’t wrong. They’re just incomplete.

Marc: Right. If they were wrong, we wouldn’t have spent an entire episode talking about them. So they’re definitely a factor, but they’re just two inputs operating inside the hearing.

Gwen: So this episode, we talk about what happens before that. Which judge you draw, whether you have representation, and where you live. Those determine a huge amount of what will happen to you, and they’re all fixed before you walk into the room.

Marc: So the variation in the numbers, the inputs that produce it, are what this episode is actually about.

Gwen: Right. That’s the distinction. Which judge, whether you have a lawyer, where you live, what tools exist to pull any of those towards the same answer for similar cases? That’s what we’re talking about.

Marc: OK, let’s start with the basic idea that in courts, stare decisis means that two similar cases with similar facts should come out similarly because prior decisions bind later ones.

Gwen: Right. So we could say, why is it that these ALJ cases are coming out so differently? Why wouldn’t they do the same thing? ALJs don’t bind each other, and just like neither do trial judges. So a district judge in the Southern District of New York isn’t bound by another Southern District of New York judge. They can disagree openly and they frequently do.

Marc: And it’s not just that they disagree. Litigators actively strategize around it. If you have a choice about where to file, and in federal civil litigation you often do, you think hard about which district and sometimes which division your case ends up in. That’s why some districts get a reputation for being the best place to file, say, patent troll lawsuits. Hat tip to you, Eastern District of Texas.

Gwen: But there are multiple-judge divisions where you at least know the pool. Plaintiff-side lawyers forum shop for sympathetic precedent and sympathetic judges. Defense-side lawyers remove cases to federal court partly for the same reason.

Marc: Right. So choosing where you’re going to file, this forum shopping strategy, this is considered normal practice.

Gwen: Completely normal. It’s what anyone experienced would do. Nobody pretends the variation between trial judges is small enough to ignore. You pick your venue, if you can, and when you can’t, you prepare for the specific judge you drew. The variation is just part of how the system works.

Marc: Right. So a claim that courts have consistency and agencies don’t would be wrong on its face. Both systems can have significant judge-level variation that sophisticated parties strategize around. So that’s not the difference between them.

Marc: OK, so what is the difference?

Gwen: It’s the tools available for navigating the variation. When a federal litigator prepares for a judge they haven’t appeared before, what do they do?

Marc: Well, they pull the judge’s published opinions. They read their rulings on similar motions. If there’s a recent precedent, a recent ruling on the same kind of issue, that’s the first thing you would pull to read. You’d pull unpublished opinions. Those are citable as persuasive authority in most circuits, and in some circuits now even unpublished opinions can be cited for their reasoning. So you would look at whether the judge tends to grant or deny 12(b)(6) motions on a particular theory, and you’d read their bench rulings in similar cases and hire local counsel if you really need a deeper dive on the judge’s practices.

Gwen: All right. So how is it so easy to get all of that?

Marc: Well, because the reasoning is accessible. A sophisticated party can genuinely get up to speed on a judge they’ve never appeared before in a week or two of focused work.

Gwen: All right. That’s what’s missing at the ALJ level, not the variation. The variation exists in both systems. What’s missing is the research path. The ALJ’s prior decisions aren’t published. The reasoning isn’t accessible. And a lawyer preparing to appear before a new ALJ can pull up the aggregate approval rates SSA publishes, but that’s essentially it. They can’t read prior decisions. They can’t pull unpublished rulings. They can’t see how that ALJ has handled similar medical issues in the past. The research path that makes federal trial court variation navigable for sophisticated parties just doesn’t exist here.

Marc: Okay, so two different systems, the same kind of variation, and completely different ability to prepare for it.

Gwen: Yes, and the guidance from above can also be really sparse. Vertical binding, like some higher pronouncement that would bind the ALJ, is going to be really thin at agencies like Social Security. The Appeals Council doesn’t issue precedential decisions. The binding law comes from federal courts, which see only a tiny slice of the cases, and from agency-wide interpretive rulings that cover super narrow questions. We’ll come back to those.

Gwen: It’s also an accessibility issue. So individual ALJ decisions in disability cases generally don’t go anywhere. They’re not published. They’re not in a database. They’re certainly not systematically available to lawyers researching how similar cases have been decided.

Marc: Wait, so they are completely not available?

Gwen: Right. Not as a systematic matter. A disability practitioner can get HALLEX, which is SSA’s internal procedural manual. That’s public. They can get Social Security Rulings, which are agency-wide policy statements that do bind ALJs on interpretation. Those are also public. They can get federal court decisions in their circuit. And the Social Security Administration publishes aggregate statistics about ALJ approval rates, their denial rates. But these are aggregate statistics. They’re not decisions.

Gwen: So the practitioner can know that ALJ Thompson has a 32% approval rate and ALJ Martinez has a 64% approval rate. But what they won’t know is how either of them handle the specific legal or factual question in a prior case. They can’t pull up their reasoning. They can’t research whether Thompson tends to discount treating physician opinions when there’s been a long gap in treatment, or whether Martinez is particularly attentive to functional capacity evaluations.

Marc: That’s information that exists only in the ALJ’s case files, which are closed.

Gwen: So the federal litigator can research their judge. The disability practitioner can research their judge’s box score.

Marc: Right. And the gravitational pull that that information produces in the court system, that those prior opinions, the fact that judges know their colleagues, the bar can read what they write, that pull is absent here. There’s nothing equivalent pulling ALJs towards some sort of shared practice.

Gwen: Which sounds like the straightforward “there is no precedent” story.

Marc: Right. It’s similar to that, but it’s not quite that because there’s something that can substitute for this, at least to some extent. It’s just not publicly available.

Gwen: You mean the lawyers themselves?

Marc: I do.

Gwen: So the experienced disability bar in any hearing office can develop knowledge of local ALJs. They’ve appeared in front of them a bunch of times. They know how they think. They know what evidence moves them and what doesn’t. They know the specific ways they ask about different daily activities and how to prepare a client for it. That knowledge exists. It’s just concentrated in the hands of attorneys who’ve been doing this for years, and it doesn’t travel outside that circle.

Marc: Which is where the representation starts to matter in a different way than we’ve talked about.

Gwen: Right. And we’re going to be there pretty soon. But first, I want to deal with the other piece of what determines an outcome at the hearing level. So before this can even matter, you have to get assigned to a judge. And how that happens is a design choice.

Marc: OK. How does assignment actually work?

Gwen: The case gets sent to an office that covers that claimant’s geographic area. And within that office, it’s assigned to whatever ALJ is available. There are some exceptions. There are some specialized dockets at a few offices and some case management tweaks. But the baseline is just this randomness.

Marc: And random assignment is defended on what grounds?

Gwen: Well, the same way we defend it in court. Neutrality. Nobody picks your judge. No one can stack your case. You can’t forum shop within an office. The agency can’t route sympathetic claimants. Random assignment eliminates a whole category of abuse.

Marc: That’s fair, and that’s a real virtue.

Gwen: Yeah, it protects against selective routing. The question is what it produces when you pair it with that 46% variation that we opened with.

Marc: Sounds like it produces something that looks like a lottery.

Gwen: So it produces something that looks like a lottery in the most literal sense. Two claimants with the same profile file their applications on the same day. They get randomly assigned to different judges, and the outcome swings by 46 points. So this random assignment means that you’re not going to be singled out, but it doesn’t guarantee you’re going to be treated consistently with anyone else. Those are different forms of fairness, and the system has definitely chosen the first one. The cost of that choice is consistency.

Marc: But what are the alternatives? Is anyone doing something different?

Gwen: Some offices have tried using case complexity scoring to get harder cases to go to more experienced judges. Some have tried specialized dockets, like mental health claims to one group of judges, physical impairment claims to another, on the theory that specialization could help produce consistency and expertise within a given case type. But the research on whether those actually narrow the variation is mixed.

Marc: But why are they mixed?

Gwen: The variation isn’t coming from case complexity or subject matter mismatch. It’s coming from how judges weigh evidence and credit testimony, things that persist even when you control for case type. So specialization might reduce some variation, but it won’t eliminate it. And any move away from pure random assignment opens the door to some version of routing, which reintroduces the judge selection concern that we were trying to solve.

Marc: So you’re basically picking your poison.

Gwen: Yes. The standard story is that random assignment is fair. Variations that result from this are unfortunate but incidental, and that’s not really quite right. What the system has done is prioritize neutral allocation over outcome consistency. These are two totally different things. Neutral allocation means that nobody picks your judge. Outcome consistency means that similar cases should come out similarly. Random assignment delivers the first. It doesn’t deliver the second. And when you compare this with the variation between the judges, it can actually prevent the second.

Marc: Which, given the variation you just described, means consistency isn’t just deprioritized at the assignment stage. It’s effectively foreclosed.

Gwen: Right. Not by anyone choosing to foreclose it, but as a structural consequence of pairing random assignment with the level of variation that the system is going to inherently produce.

Marc: So this is supposed to be neutral allocation. Consistency is traded for the ability to get this neutrality. Whether the trade is worth it is a separate issue. What we’re trying to talk about today is just that the trade was made and the consistency that got traded away doesn’t get recovered.

Gwen: OK. And the downstream mechanisms, like appellate review, representation, federal court review, those are what the system offers in place of that consistency.

Marc: Right. And the next couple episodes, we’re going to be looking at ways that effort is made to try and corral these decisions. But they still won’t produce the lack of variation that we might hope for.

Marc: Okay. So you get assigned to a judge randomly. You might have drawn a judge who approves a third of his cases or one who approves two thirds. What else shapes the outcome at that point?

Gwen: This is where we’re back to talking about representation. And this effect is huge. Claimants without representation get roughly half the approval rate of claimants with representation.

Marc: Doubled?

Gwen: Yeah, roughly. So the exact numbers can vary depending on the study and how you count it. But this dramatic increase is consistent. Represented claimants do substantially better at the hearing level than unrepresented claimants.

Marc: And there’s a few different reasons for this. It’s not necessarily what people automatically assume. They might think that it’s just how they perform in the room. The lawyer shows up, they make a better argument, and that can matter at the margins, but it’s not the main thing. The main thing happens before the hearing.

Gwen: Okay, so building a record like we talked about in the lifecycle episode. Getting the treating physician opinion, getting the functional capacity evaluation, making sure the medical evidence is complete, anticipating what the vocational expert will say, and preparing cross-examination. All of that happens in the months before the hearing, and it’s what mostly determines what the record looks like when the hearing starts.

Marc: Right. It’s huge, and it also plays into something else. If ALJ decisions aren’t accessible, the lawyer can’t research how this judge has previously ruled on similar cases, then what the lawyer already knows about this judge is going to be the only substitute for this kind of precedent.

Gwen: So an experienced local attorney is going to know how these specific ALJs in this specific office handle things. They’re more likely to know what’s going to convince this judge. That knowledge makes a huge difference. Without it, the practitioner is functionally working blind. The only way to know how an ALJ actually thinks is to have appeared in front of them or to know someone who has.

Marc: So really, experience in a local hearing office isn’t just the general-purpose advantage, it’s the specific substitute for the precedent system that doesn’t exist.

Gwen: Right. So that means that what these experienced disability attorneys are selling isn’t just that they understand disability law. It’s this private library of knowledge that they’ve accumulated over the years that you can’t get any other way.

Marc: So this informal consistency mechanism, this trying to pull prior decisions into practice, this is only behind the paywall of representation.

Gwen: Okay. What about unrepresented claimants?

Marc: They’re not going to have access to this. All they’re doing is working from SSA’s public explanations of the disability standard. There might be some sort of advocacy organization that’s produced materials they’re trying to use, and then whatever intuitions they developed about how the system works.

Gwen: And this is why the representation effect is so large.

Marc: Yeah, it’s part of it. This record-building piece matters, too. But I think that this explains what’s otherwise a strange result, that a system that’s supposed to be non-adversarial with an ALJ who’s supposed to be developing the record nevertheless produces these dramatic differences between represented and unrepresented claimants. If the ALJ really were performing the inquisitorial role we talked about earlier in the season, then representation really shouldn’t matter much. It matters this much in part because the system doesn’t publish its own reasoning and experienced attorneys have figured out how to compensate.

Marc: Okay, let me push on what that means because I think it’s sharper than what you’re saying. If the substantive law that actually governs disability adjudication in a given hearing office, the way credibility gets weighed, the way treating physician opinions get handled, or the way functional capacity gets interpreted, exists only in the heads of experienced local attorneys and not in any public source, then what you’re describing isn’t just an access problem.

Gwen: Okay, how would you describe it?

Marc: This is basically privatized law, with the caveat that nobody privatized it. I mean, this is like the realm of having published opinions that only a few people can access. There’s a published regulation, but the law as it actually operates in any specific hearing office is largely the accumulated private knowledge of the experienced bar.

Gwen: That’s a pretty good way to describe it. So that doesn’t mean that anyone’s intentionally privatizing it. Nobody is. What could in principle exist as accessible operational knowledge instead exists as this experience-based knowledge held by whoever has been in those offices long enough to accumulate it.

Marc: Yeah. And I mean, let’s just point out one other thing. If you want to get to a level of having something in a written record that everyone can discover, you’re going to spend a lot more money than just at the basic ALJ hearing level. And so really, this is going to come down to a question of who can afford to retain these attorneys to make their case both effective at the ALJ level and visible at the non-ALJ level.

Gwen: Yes. So this is really a structural problem. It’s not just that represented claimants have a lawyer in their corner. It’s that unrepresented claimants are operating without this knowledge base that they need to actually help get the outcome they’re looking for.

Marc: Okay. There’s a third thing, though, that shapes the outcome at the hearing level that I want to make sure we cover. You mentioned it earlier, the hearing office.

Gwen: Right. So geography is separate from the judge-level variation, and there’s pretty strong evidence for it. Hearing offices have different average approval rates from each other in patterns that persist even when you control for the judges in the office.

Marc: OK, what drives that then?

Gwen: So we’ve got a few things that are stacking on top of each other. We have the local economic conditions, the vocational expert testimony about what jobs are available. It’s supposed to be what jobs are available in the national economy, but it can also get colored by what the local labor market looks like. We have regional medical infrastructure. If the claimants in this office’s area tend to have less access to specialists, their medical records are going to look different than claimants in a region with a lot of medical resources.

Marc: So the same condition, but a thinner file.

Gwen: Right. So less corroboration, and the ALJ can only decide based on what’s in the record. There’s also the local culture at the hearing office itself. ALJs who’ve worked together for years are potentially going to be developing shared habits. And a new ALJ coming into the office is going to drift towards that office’s norms over time. Not intentionally, not formally, but that’s how people work.

Marc: And what’s the magnitude of this office-level effect then?

Gwen: It’s important, but it’s going to be smaller than the judge-level variation. Hearing offices differ from each other by something like 10 to 15 points in average approval rates, beyond just what we would expect from the mix of judges. So this means that the claimant’s geography, not just which judge they draw, but the office’s territory, is another outcome variable independent of anything else about their case.

Marc: Okay, so a federal program with a single national standard producing systemically different outcomes depending 10 to 15 points on where you live?

Gwen: Right. So the standard’s national, the statute’s national, the regulations are national, but this variation is coming from how that national standard gets applied in the specific hearing office that happens to cover your zip code.

Marc: This isn’t a claim about bad offices or good offices. The variation isn’t about anyone doing anything wrong. It’s what a national standard produces when it’s administered by all of these different semi-autonomous offices with no mechanism to pull their practices toward each other.

Gwen: Right. And this gets us back to where we started, that 46-point judge-level variation. If we stack that on top of a 10 to 15 point office-level variation, and we stack that on the representation effect, these aren’t separate problems. A system of independent adjudicators at scale, without any sort of mechanism to pull them towards each other, is going to produce variation along every axis that we could have variation on.

Marc: Okay. You’ve been saying no precedent in a very broad way, but you mentioned earlier that there is some agency-level binding law. So let’s get specific about what that is.

Gwen: All right. We have two categories. The first, we have Social Security Rulings. These are agency-wide interpretive policy, how to apply a particular statutory term, how to weigh a particular type of evidence. We have SSRs that bind ALJs on the questions that they address. They’re public, they’re citable, they’re enforceable.

Marc: So what about HALLEX?

Gwen: HALLEX is procedural. It governs how the hearings are conducted, what the ALJ has to do, what the record has to contain. It’s binding on procedure. It’s not binding on substantive interpretations the way SSRs are.

Marc: So SSA does have some top-down law.

Gwen: Yes, it does. The question is just what it covers. These are specific questions that the agency has chosen to address. How to evaluate pain testimony, how to treat certain categories of medical evidence. Those questions have guidance, but a large number of the interpretive choices that an ALJ is going to have to make in any given hearing aren’t covered by them.

Marc: Okay, so give me an example.

Gwen: Okay, like how we’re going to weigh the treating physician’s opinion against the consultative examiner’s opinion if they disagree. Or what to do when a claimant has documented physical impairments and also describes significant mental health symptoms but hasn’t been formally diagnosed. And what to do if we have a long gap in treatment where the claimant says that they couldn’t afford care. Those are judgment calls that happen in every disability hearing, and they’re not really governed by these SSRs.

Marc: So the binding top-down law covers a narrow band of the interpretive work, and most of what ALJs are doing happens outside that band.

Gwen: Right, outside that, and therefore outside any direct consistency mechanism. Immigration has a broader band. The Board of Immigration Appeals issues precedential decisions that cover a wider range of substantive questions, and it has mechanisms we’ll talk about next episode that let it reach cases it wants to decide. But even there, the designated precedent covers a small fraction of the questions that immigration judges are deciding in any given year.

Marc: And the NLRB, the SEC, and enforcement agencies?

Gwen: So they have their own versions, and we’ll be looking at those next week. But the pattern across the agencies is roughly the same. Where binding agency-level law exists, it covers the questions the agency has chosen to address. Where it doesn’t, the ALJ is working from statutes and regulations and whatever federal court opinions happen to govern, and from their own judgment.

Marc: So the answer to “is there no agency-level precedent” isn’t yes. The answer is there’s some. It’s narrower than court-level precedent. And the questions it doesn’t cover are where the 46-point variation lives.

Gwen: Yeah, that’s the easiest way to think about this. Nothing we said today means that there isn’t any law. It means that the law existing at the top level of these agencies covers specific questions. And there’s a huge space outside this where the individual ALJ is operating based entirely on their judgment.

Marc: And subject to whatever gravitational pull the informal system produces. And that gravitational pull is mostly captured by experienced counsel.

Gwen: Right. Mostly local to specific offices and mostly invisible to anyone outside the ecosystem.

Marc: OK, let me try to summarize what we’ve done so far.

Gwen: Right.

Marc: All right. So at the hearing level, outcomes depend on which judge you draw, whether you have a lawyer, and where you live. All three operate independently, all three produce measurable variation, and none of them is meaningfully controlled by any top-down mechanism.

Gwen: And yet, the agency has some binding interpretive law that covers a narrow slice of what ALJs actually decide, but everything else is private. In the case files, in the lawyer’s heads, in the local practice cultures.

Marc: Yeah. So this is the horizontal picture, the variation between individual judges at the hearing level.

Gwen: Which sets up the two obvious follow-up questions.

Marc: Right. So if this bottom-up consistency doesn’t work, does top-down review fix it? Because every agency has some sort of appellate layer. The Appeals Council, the BIA, the commissioners at the SEC and FTC, the board at the NLRB. The theory is even if ALJs are producing inconsistent decisions, the review bodies at these agencies correct errors, resolve conflicts, and pull outcomes towards consistency.

Gwen: That’s the theory, and we’re going to test it.

Marc: And if the answer is that review doesn’t fix it either.

Gwen: Which, spoiler, it doesn’t. Then the third episode asks the question that follows. If we have no formal mechanism, no horizontal, no vertical mechanism that’s producing consistency, but agency preferences clearly do translate into outcomes, something else is doing the work. And that something else isn’t quite a set of rules. It’s a set of institutional practices like performance metrics and quality reviews, case completion targets, how the agency evaluates its own ALJs, how guidance documents that aren’t legally binding still shape what happens in these hearings. These are the mechanisms. They’re not law, they aren’t review, and they turn out to be how agency preferences generally reach the hearing room. Even if, as we’ve seen, there’s a lot they still don’t control.

Marc: So the trilogy spine is: outcomes depend on who you get, review doesn’t fix it, and therefore control happens through mechanisms that don’t look like law at all.

Gwen: Exactly. So this was the first. We’ve got two more episodes to go to talk about this.

Marc: So that does it for today’s episode on Administrative Remedies. Thank you for joining us today. Please, if you enjoy this podcast and enjoy this episode, give us a like on Spotify, iTunes, or whatever platform you’re listening on. And be sure to tune in next time where we’ll continue to dive into the contours of administrative law, because remember, you can’t fix what you don’t understand.

Related Guides

• Why Two Identical Disability Cases Can Have Opposite Outcomes — That episode covered what happens inside the hearing; this one covers what's already determined before you walk in. They're the matched pair on what produces hearing-level outcomes.
• The Lifecycle of an Administrative Case — The record-building work that representation actually does — getting treating physician opinions, anticipating vocational expert testimony — happens during the wait described in the lifecycle episode.
• Who Decides: ALJs vs. Non-ALJs — The variation discussed here is variation between ALJs specifically; this guide explains who they are, how they're appointed, and what makes their role structurally distinct.