In the final episode of a three-part series on how agencies actually produce outcomes, Gwen and Marc trace the mechanisms that did the work — all of them upstream of the hearing room and mostly invisible from outside:
- Case completion targets that measure speed but not thoroughness — and a Seventh Circuit concurrence warning that even well-intentioned production pressure "can alter the essential function of adjudication"
- Quality review rubrics that aren't published, shift with administrations, and can't be appealed — but feed into performance evaluations that have real career consequences
- Social Security Rulings that describe themselves as not having "the force and effect of law" and in the same sentence say they are "binding on all components" — binding policy issued without notice and comment
- The 1984 case that saw it coming: Association of Administrative Law Judges v. Heckler, where the court found outcome-based targeting of ALJs "violated the spirit of the APA, if no specific provision thereof"
- Immigration as the amplified version: 700-case-per-year quotas for judges without APA independence protections, deciding cases where the stakes are deportation
Somewhere out there is a claimant with the same diagnosis as someone approved three years ago by the same ALJ in the same office. That claimant gets denied — not because the case changed, but because the system around the judge changed. And nobody outside the agency can see why.
00:00:14 --> 00:00:17 Hello, and welcome to Administrative Remedies,
00:00:17 --> 00:00:19 because you can't fix what you don't understand.
00:00:19 --> 00:00:21 Brought to you in part by the University of Tulsa
00:00:21 --> 00:00:24 College of Law. I'm Gwendolyn Savitz, an associate
00:00:24 --> 00:00:26 professor here at TU and the associate dean of
00:00:26 --> 00:00:28 research and intellectual life. And I'm Mark
00:00:28 --> 00:00:31 Rourke. I'm the dean of the College of Law. We'll
00:00:31 --> 00:00:33 be breaking down complex doctrines with real
00:00:33 --> 00:00:35 -life analogies and examples to demystify the
00:00:35 --> 00:00:37 world of administrative law for everyone trying
00:00:37 --> 00:00:39 to understand how government actually works.
00:00:40 --> 00:00:41 Agencies are the main way the federal government
00:00:41 --> 00:00:44 gets things done. It's not through Congress for
00:00:44 --> 00:00:46 reasons we'll be addressing over the course of
00:00:46 --> 00:00:51 this series. So, Mark, in 2010, the Social Security
00:00:51 --> 00:00:53 disability allowance rate at the hearing level
00:00:53 --> 00:00:59 was 62%. By 2019, it was 45%. Same statute, same
00:00:59 --> 00:01:01 regulations, same fundamental medical conditions
00:01:01 --> 00:01:04 people apply with. Degenerative disc disease,
00:01:04 --> 00:01:07 depression, congestive heart failure, same diagnoses
00:01:07 --> 00:01:10 year after year. But a 17 -point drop in a single
00:01:10 --> 00:01:13 decade. Right. And nobody issued a directive
00:01:13 --> 00:01:16 telling ALJs to deny more cases. No SSR was published
00:01:16 --> 00:01:19 saying tighten up. No memo went out. Every individual
00:01:19 --> 00:01:21 ALJ will tell you accurately that they decided
00:01:21 --> 00:01:23 each case on the merits as they understood the
00:01:23 --> 00:01:26 merits. And the aggregate still moved 17 points
00:01:26 --> 00:01:29 away. Right. So in the last two episodes, we
00:01:29 --> 00:01:31 talked about what doesn't explain it. That there
00:01:31 --> 00:01:34 isn't a horizontal mechanism. The ALJs don't
00:01:34 --> 00:01:37 bind each other. But the operational law is privatized
00:01:37 --> 00:01:39 behind local counsel, so there's no bottom -up
00:01:39 --> 00:01:42 pull towards consistency. And then in the last
00:01:42 --> 00:01:44 episode, there's no effective vertical mechanism
00:01:44 --> 00:01:47 for it. The appeals counsel reviews almost nothing.
00:01:47 --> 00:01:51 The BIA is selective and political, and the SEC's
00:01:51 --> 00:01:53 review is structurally tilted. None of those
00:01:53 --> 00:01:55 produce consistency, and none of them explains
00:01:55 --> 00:01:58 the 17 -point shift. And still the shift happened.
00:01:59 --> 00:02:01 Right, across thousands of ALJs. across every
00:02:01 --> 00:02:03 hearing office without any of the mechanisms
00:02:03 --> 00:02:05 we've been studying for the last two episodes
00:02:05 --> 00:02:08 producing it. Something else is doing the work.
00:02:08 --> 00:02:10 Something other than just a reversal of the decisions.
00:02:11 --> 00:02:13 Right. Agencies mostly don't control outcomes
00:02:13 --> 00:02:15 by reversing decisions they don't like. They
00:02:15 --> 00:02:18 control outcomes by shaping the conditions under
00:02:18 --> 00:02:20 which the decisions get made, before the hearing,
00:02:20 --> 00:02:22 before the record is even developed, through
00:02:22 --> 00:02:25 training, metrics, guidance documents that aren't
00:02:25 --> 00:02:27 technically law, performance evaluations, and
00:02:27 --> 00:02:30 quality review. None of those things look like
00:02:30 --> 00:02:32 correction. But all of them are how correction
00:02:32 --> 00:02:35 happens. So they happen upstream from the hearing?
00:02:35 --> 00:02:37 Yes, and mostly invisible to everybody outside
00:02:37 --> 00:02:40 the agency. Before we get into the mechanisms,
00:02:40 --> 00:02:42 though, the intuitive picture of how agencies
00:02:42 --> 00:02:45 control adjudication is exactly the one you're
00:02:45 --> 00:02:48 saying isn't the main channel for these changes.
00:02:49 --> 00:02:52 Agency reviews a decision, doesn't like it, reverses
00:02:52 --> 00:02:55 it. That's the model, I would assume. Right.
00:02:55 --> 00:02:58 And that kind of ex -post -correction does happen.
00:02:58 --> 00:03:00 Last episode, we talked about the mechanisms
00:03:00 --> 00:03:03 that do it. They're real. They exist. But they're
00:03:03 --> 00:03:05 not the dominant force for a couple of reasons.
00:03:06 --> 00:03:08 The first is cost. If the agency wants to shift
00:03:08 --> 00:03:11 outcomes by reversing individual decisions, it
00:03:11 --> 00:03:13 has to review every case it wants to change.
00:03:13 --> 00:03:15 That's expensive in a system working at any kind
00:03:15 --> 00:03:18 of scale. If it can change what the average ALJ
00:03:18 --> 00:03:20 thinks is the right answer in a particular kind
00:03:20 --> 00:03:22 of case, it doesn't need to reverse anything.
00:03:23 --> 00:03:25 The decisions come out the way the agency wants
00:03:25 --> 00:03:28 without any individual case being touched. Yeah,
00:03:28 --> 00:03:30 it becomes more efficient. And quieter. So at
00:03:30 --> 00:03:32 agencies where appellate decisions get published,
00:03:32 --> 00:03:37 like the SEC or the NLRB or the BIA's precedential
00:03:37 --> 00:03:39 decisions, a reversal leaves a trail that lawyers
00:03:39 --> 00:03:43 can read and cite. At Social Security, even reversals
00:03:43 --> 00:03:45 don't surface that way. The Appeals Council doesn't
00:03:45 --> 00:03:47 issue precedential decisions. We've talked about
00:03:47 --> 00:03:50 that. But either way, behavioral change leaves
00:03:50 --> 00:03:52 a different kind of trail than reversal. You
00:03:52 --> 00:03:55 can see the 17 -point drop, but you can't necessarily
00:03:55 --> 00:03:57 see the mechanism behind it. Nobody can appeal
00:03:57 --> 00:03:59 a training program. Nobody can sue because their
00:03:59 --> 00:04:02 judge had a quality review meeting. The aggregate
00:04:02 --> 00:04:05 is visible, but the cause is not. So it's cheaper
00:04:05 --> 00:04:07 and quieter. Yes. The agency doesn't necessarily
00:04:07 --> 00:04:10 have to tell people what it wants. It builds
00:04:10 --> 00:04:12 the conditions under which what it wants comes
00:04:12 --> 00:04:14 out anyway. So let me push back on something,
00:04:14 --> 00:04:16 though. Some of what you've just described just
00:04:16 --> 00:04:19 sounds like ordinary management. Every organization
00:04:19 --> 00:04:22 trains its people. Every organization has performance
00:04:22 --> 00:04:26 evaluation. If I'm running anything, a law school,
00:04:27 --> 00:04:29 a company, a hospital, I want there to be consistency.
00:04:30 --> 00:04:33 I also want there to be quality, and I want my
00:04:33 --> 00:04:36 people to be aligned. That doesn't seem sinister.
00:04:36 --> 00:04:38 That just seems like it's running an institution.
00:04:39 --> 00:04:42 Yeah, that's true. So the mechanisms themselves
00:04:42 --> 00:04:45 aren't illegitimate. Training is necessary. It's
00:04:45 --> 00:04:47 a good thing. Performance evaluations make sense.
00:04:48 --> 00:04:50 Some consistency and interpretation is what people
00:04:50 --> 00:04:52 mean when they're asking for the rule of law.
00:04:53 --> 00:04:55 The question isn't whether agencies should manage
00:04:55 --> 00:04:57 their educators. They have to. The question is
00:04:57 --> 00:04:59 what the cumulative effect of these mechanisms
00:04:59 --> 00:05:01 are when they're applied to people whose job
00:05:01 --> 00:05:03 is supposed to be deciding cases independently
00:05:03 --> 00:05:06 and whether the formal protections that we attach
00:05:06 --> 00:05:08 to independents actually cover what's happening.
00:05:09 --> 00:05:10 So you're not saying the existence of training
00:05:10 --> 00:05:13 is the problem? No, not the existence. It's just
00:05:13 --> 00:05:16 trying to acknowledge the cumulative impact and
00:05:16 --> 00:05:19 this lack of transparency, what these actually
00:05:19 --> 00:05:21 do, and the fact that the procedural protections
00:05:21 --> 00:05:24 the APA built for binding agency policy don't
00:05:24 --> 00:05:26 really attach to any of them. The mechanisms
00:05:26 --> 00:05:28 are management practices that, when applied to
00:05:28 --> 00:05:30 adjudicators at scale, do something that looks
00:05:30 --> 00:05:33 an awful lot like setting policy without any
00:05:33 --> 00:05:34 of the procedure that's supposed to attach when
00:05:34 --> 00:05:38 agencies do it. You said this isn't new. How
00:05:38 --> 00:05:40 long has this been going on? It's been documented
00:05:40 --> 00:05:43 for at least 40 years. There's a case that shows
00:05:43 --> 00:05:45 courts have seen this dynamic and struggled with
00:05:45 --> 00:05:47 what to do about it. This is the Association
00:05:47 --> 00:05:49 of Administrative Law Judges v. Heckler. This
00:05:49 --> 00:05:53 is 1984. The Social Security ALJ's union sues
00:05:53 --> 00:05:56 the Secretary of Health and Human Services. Their
00:05:56 --> 00:05:58 claim was that the agency was targeting judges
00:05:58 --> 00:06:00 with high allowance rates, the ones that were
00:06:00 --> 00:06:02 approving too many disability claims, and targeting
00:06:02 --> 00:06:05 them for enhanced quality review. Their decisions
00:06:05 --> 00:06:07 were getting flagged. Their files were under
00:06:07 --> 00:06:09 extra scrutiny. The implicit message, the union
00:06:09 --> 00:06:12 argued, was your approval rates are too high.
00:06:12 --> 00:06:14 Bring it down. And the agency never said that
00:06:14 --> 00:06:17 out loud. Right. Nobody wrote a memo saying approve
00:06:17 --> 00:06:21 fewer claims. What happened was the review apparatus
00:06:21 --> 00:06:23 pointed itself at specific judges based on outcome.
00:06:24 --> 00:06:26 The court ruled against the union on most issues,
00:06:26 --> 00:06:30 but this part didn't. Defendants' unremitting
00:06:30 --> 00:06:32 focus on allowance rates in the individual ALJ
00:06:32 --> 00:06:35 portion of the Bellman Review Program created
00:06:35 --> 00:06:38 an untenable atmosphere of tension and unfairness
00:06:38 --> 00:06:40 which violated the spirit of the Administrative
00:06:40 --> 00:06:44 Procedures Act, if no specific provision thereof.
00:06:44 --> 00:06:46 Defendants' insistentivity to that degree of
00:06:46 --> 00:06:49 decisional independence of the APA affords to
00:06:49 --> 00:06:52 administrative law judges and the injudicious
00:06:52 --> 00:06:56 uses of phrases such as targeting, goals, and
00:06:56 --> 00:06:58 behavior modification could have tended to corrupt
00:06:58 --> 00:07:01 the ability of administrative law judges to exercise
00:07:01 --> 00:07:04 their independence in the vital cases that they
00:07:04 --> 00:07:07 decide. Could have tended to corrupt, the court
00:07:07 --> 00:07:09 is saying that targeting based on outcomes is
00:07:09 --> 00:07:10 the kind of pressure that bends decision -making,
00:07:11 --> 00:07:13 even where you can't trace any single decision
00:07:13 --> 00:07:16 to it. Right. And then the last phrase of that
00:07:16 --> 00:07:18 first sentence is talking about the structural
00:07:18 --> 00:07:21 problem, that it violated the spirit of the APA,
00:07:21 --> 00:07:23 even if it didn't violate any specific provision.
00:07:23 --> 00:07:26 The court's not pointing to a specific. APA section,
00:07:26 --> 00:07:29 it's saying that the APA was written for a particular
00:07:29 --> 00:07:32 kind of agency action, rulemaking, adjudication,
00:07:32 --> 00:07:34 these defined processes with defined procedural
00:07:34 --> 00:07:37 protections. What the agency was doing here,
00:07:37 --> 00:07:39 where it was applying review pressure and patterns
00:07:39 --> 00:07:41 that tracked outcomes, didn't really fit cleanly
00:07:41 --> 00:07:43 into either category. So the court could say
00:07:43 --> 00:07:46 it violated the spirit of the statute, it just
00:07:46 --> 00:07:48 couldn't say which provision, which made it nearly
00:07:48 --> 00:07:51 impossible to enforce. And 40 years later? Different
00:07:51 --> 00:07:54 versions, same dynamic. That 1984 case was a
00:07:54 --> 00:07:56 cruder version. You could see the targeting.
00:07:57 --> 00:07:59 It was legible. They literally used the words
00:07:59 --> 00:08:01 targeting in agency memos, which the court called
00:08:01 --> 00:08:04 injudicious. Modern versions are more sophisticated.
00:08:04 --> 00:08:06 They operate through institutional systems that
00:08:06 --> 00:08:09 don't single out individuals overtly and don't
00:08:09 --> 00:08:11 produce this kind of evidence, the kind that
00:08:11 --> 00:08:13 makes litigation easy. The structure evolved,
00:08:13 --> 00:08:15 but the underlying mechanism didn't. Wong me
00:08:15 --> 00:08:17 through the modern version then. Let's start
00:08:17 --> 00:08:19 with the thing I think I understand the best,
00:08:19 --> 00:08:23 case completion targets. Every major adjudication
00:08:23 --> 00:08:26 agency tracks throughput. Social Security has
00:08:26 --> 00:08:28 targets for how many dispositions an ALJ should
00:08:28 --> 00:08:30 produce in a year. And the official rationale
00:08:30 --> 00:08:32 is entirely defensible. There are millions of
00:08:32 --> 00:08:35 pending claims. There's real human harm from
00:08:35 --> 00:08:37 the backlog. Claimants can wait two to three
00:08:37 --> 00:08:39 years for a hearing. The agency has a responsibility
00:08:39 --> 00:08:42 to move cases. I'm definitely not arguing with
00:08:42 --> 00:08:44 that. And I'm not saying it's illegitimate in
00:08:44 --> 00:08:46 principle. The problem is what productivity pressure
00:08:46 --> 00:08:49 measures and what it doesn't. A case completion
00:08:49 --> 00:08:51 target captures how many cases you've faced.
00:09:00 --> 00:09:03 And thoroughness and speed are in tension here.
00:09:03 --> 00:09:05 They have to be. A more careful hearing is going
00:09:05 --> 00:09:08 to take longer. A more fully developed record
00:09:08 --> 00:09:10 is going to take longer to build. An ALJ who
00:09:10 --> 00:09:13 reliably hits production may be running efficient
00:09:13 --> 00:09:16 hearings of high quality, or they may be cutting
00:09:16 --> 00:09:17 corners, and the metric doesn't tell you which
00:09:17 --> 00:09:19 one. So has there been a case on this directly?
00:09:20 --> 00:09:24 In 2015, the same ALJ union sued, this was the
00:09:24 --> 00:09:27 Association of Administrative Law Judges v. Collin,
00:09:27 --> 00:09:30 over their case production targets. The agency
00:09:30 --> 00:09:33 had directed each ALJ to manage their docket
00:09:33 --> 00:09:35 in such a way that they will be able to issue
00:09:35 --> 00:09:38 500 to 700 legally sufficient decisions each
00:09:38 --> 00:09:40 year. The Seventh Circuit, Judge Posner writing
00:09:40 --> 00:09:42 for the majority, declined to reach the merits.
00:09:43 --> 00:09:45 Posner held that the production quota was a personnel
00:09:45 --> 00:09:48 action and the union didn't have a direct judicial
00:09:48 --> 00:09:51 remedy under the Civil Service Reform Act. But
00:09:51 --> 00:09:53 there is a concurrence that's worth hearing for
00:09:53 --> 00:09:55 what it talks about as the harder problem. serious
00:09:55 --> 00:09:57 impairment of a governmental function can occur
00:09:57 --> 00:10:00 at the hands of officials with the most worthy
00:10:00 --> 00:10:02 of motives. The integrity of the judicial function
00:10:02 --> 00:10:05 at any level of adjudication can be undermined
00:10:05 --> 00:10:08 seriously by even the most benignly motivated
00:10:08 --> 00:10:10 administrative or executive action that alters
00:10:10 --> 00:10:14 the essential function of adjudication. So he's
00:10:14 --> 00:10:17 saying you don't need bad faith to produce the
00:10:17 --> 00:10:18 problem. You just need the production pressure.
00:10:19 --> 00:10:21 Right. And that the production pressure can alter
00:10:21 --> 00:10:23 the essential function of adjudication, even
00:10:23 --> 00:10:26 if nobody means it to, which has become the pattern.
00:10:26 --> 00:10:28 Courts look at this and acknowledge that the
00:10:28 --> 00:10:30 concern is real, and then they can't reach it
00:10:30 --> 00:10:32 because the structural way these pressures operate
00:10:32 --> 00:10:35 through general administrative practice that
00:10:35 --> 00:10:36 doesn't take the form of any particular agency
00:10:36 --> 00:10:39 action doesn't really fit the shape of a concrete
00:10:39 --> 00:10:42 legal claim. So it doesn't get resolved and it
00:10:42 --> 00:10:45 just keeps happening. So what you're saying about
00:10:45 --> 00:10:47 approval rates is productivity is one thing.
00:10:47 --> 00:10:50 The agency tracking how often you say yes is
00:10:50 --> 00:10:52 something else. Right. And they track that, too.
00:10:52 --> 00:10:55 So an ALJ who approves a claim writes a favorable
00:10:55 --> 00:10:57 decision. Relatively short. They don't need to
00:10:57 --> 00:10:59 defeat the evidence. It won't be reviewed at
00:10:59 --> 00:11:01 all in most cases because nobody's going to appeal
00:11:01 --> 00:11:05 a favorable decision. And the ALJ who denies
00:11:05 --> 00:11:06 something has to write a denial that explains
00:11:06 --> 00:11:09 why every favorable piece of evidence wasn't
00:11:09 --> 00:11:12 enough. It's long. careful, or should be long
00:11:12 --> 00:11:14 and careful, because it has to survive whatever
00:11:14 --> 00:11:18 review actually happens. And so that productivity
00:11:18 --> 00:11:21 pressure pushes towards approvals. Right. And
00:11:21 --> 00:11:23 approval rate pressures push the other way. ALJs
00:11:23 --> 00:11:25 are navigating both at the same time, often without
00:11:25 --> 00:11:27 anyone telling them which one is supposed to
00:11:27 --> 00:11:29 win. Okay, let's talk about guidance. You've
00:11:29 --> 00:11:32 said before that agencies issue a lot of it.
00:11:32 --> 00:11:34 I know that's technically not law, but I also
00:11:34 --> 00:11:36 feel like I'm missing something about how it
00:11:36 --> 00:11:38 actually functions. Yes, we'll be spending an
00:11:38 --> 00:11:41 entire episode on it soon. But you're missing
00:11:41 --> 00:11:44 the gap between the formal description and the
00:11:44 --> 00:11:46 reality on the ground. So formally, guidance
00:11:46 --> 00:11:48 definitely isn't law, doesn't go through notice
00:11:48 --> 00:11:50 and comment, doesn't have the force and effect
00:11:50 --> 00:11:53 of a regulation. If you ask an agency lawyer,
00:11:53 --> 00:11:55 they will tell you that guidance documents describe
00:11:55 --> 00:11:57 how the agency is just interpreting the law.
00:11:58 --> 00:12:01 It's descriptive, not prescriptive. But... But
00:12:01 --> 00:12:04 Social Security has a weird category called Social
00:12:04 --> 00:12:07 Security rulings, or SSRs. And those tell a different
00:12:07 --> 00:12:09 story. They're published in the Federal Register.
00:12:09 --> 00:12:11 They're issued under the Commissioner's authority.
00:12:11 --> 00:12:14 They interpret statutory provisions. And their
00:12:14 --> 00:12:16 own description of the status can't really seem
00:12:16 --> 00:12:19 to figure out what they are. This is from SSA's
00:12:19 --> 00:12:21 own published preference to its rulings. Although
00:12:21 --> 00:12:24 Social Security rulings do not have the force
00:12:24 --> 00:12:28 and effect of law or regulations, they are binding
00:12:28 --> 00:12:30 on all components of the Social Security Administration
00:12:30 --> 00:12:33 and are to be relied upon as precedents in adjudicating
00:12:33 --> 00:12:36 other cases. So they don't have the force and
00:12:36 --> 00:12:38 effect of the law or regulations, and they're
00:12:38 --> 00:12:42 binding on all components relied upon as precedents.
00:12:42 --> 00:12:45 That is all in the same system and feels wildly
00:12:45 --> 00:12:48 contradictory. It sure does. It sounds like SSA
00:12:48 --> 00:12:50 doesn't seem to know what they are. Yeah, this
00:12:50 --> 00:12:53 is basically binding guidance issued without
00:12:53 --> 00:12:55 notice and comment. Right. And the APA built
00:12:55 --> 00:12:57 notice and comment as a mechanism for agencies
00:12:57 --> 00:13:00 to make binding policy. Under that, the public
00:13:00 --> 00:13:03 sees the proposal. They have a chance to comment.
00:13:03 --> 00:13:05 The agency has to respond on the record. You
00:13:05 --> 00:13:07 can challenge a rule in court if it wasn't properly
00:13:07 --> 00:13:10 adopted. None of that apparatus attaches to an
00:13:10 --> 00:13:12 SSR. They're issued. They bind every ALJ in the
00:13:12 --> 00:13:14 country on substantive interpretive questions.
00:13:15 --> 00:13:17 And the procedural protections in the APA that
00:13:17 --> 00:13:20 the APA was built for for binding policy don't
00:13:20 --> 00:13:23 apply. Okay, give me a concrete example of what
00:13:23 --> 00:13:25 we're talking about. I want to see what kind
00:13:25 --> 00:13:28 of work these actually do. All right, here's
00:13:28 --> 00:13:32 one. SSR 8213. That means it was issued in 1982.
00:13:32 --> 00:13:35 And officially, it's just procedural. It establishes
00:13:35 --> 00:13:38 the Appeal Council's program of review on its
00:13:38 --> 00:13:41 own motion. They can pull an ALJ's case for review
00:13:41 --> 00:13:43 on their own initiative without anyone having
00:13:43 --> 00:13:46 appealed it. And the ruling tells you in its
00:13:46 --> 00:13:48 opening paragraph which decisions the program
00:13:48 --> 00:14:09 is aimed at. So only looking at decisions particularly
00:14:09 --> 00:14:13 favorable to the claimants? Right. So this program,
00:14:13 --> 00:14:16 which can reach any ALJ decision within 60 days
00:14:16 --> 00:14:18 and reverse it without any party having asked
00:14:18 --> 00:14:22 for review, is directed by this policy to focus
00:14:22 --> 00:14:25 on approvals. That's a one directional review.
00:14:26 --> 00:14:28 Denials don't seem to get the same treatment.
00:14:28 --> 00:14:31 Right. So the agency has substituted a policy
00:14:31 --> 00:14:33 preference, these tighter approval standards,
00:14:33 --> 00:14:35 and that preference is built into the architecture
00:14:35 --> 00:14:38 review. And it's done through a document issued
00:14:38 --> 00:14:40 without public input that functions as binding
00:14:40 --> 00:14:42 procedural policy across the entire adjudicative
00:14:42 --> 00:14:44 system. And this is in the Federal Register.
00:14:45 --> 00:14:47 Yes, it's published, so it's technically public,
00:14:47 --> 00:14:50 but it's practically invisible to someone who
00:14:50 --> 00:14:52 isn't already inside this world, which is part
00:14:52 --> 00:14:55 of how this works. That formal channel is transparent,
00:14:55 --> 00:14:57 but the operational effect is an ALJ who issues
00:14:57 --> 00:15:00 approvals knows or should know that those decisions
00:15:00 --> 00:15:03 face enhanced review. That knowledge can shape
00:15:03 --> 00:15:05 behavior before the next case comes in. And it
00:15:05 --> 00:15:07 could seem like this own motion review, of course
00:15:07 --> 00:15:09 they're going to have to look into the approvals
00:15:09 --> 00:15:11 because the denials would be appealed on their
00:15:11 --> 00:15:14 own, but that's not automatically true. People
00:15:14 --> 00:15:17 won't necessarily know to appeal a denial. And
00:15:17 --> 00:15:19 that means that they're never going to figure
00:15:19 --> 00:15:21 out those were wrongly denied. You keep mentioning
00:15:21 --> 00:15:25 quality review. It assumed it was just monitoring,
00:15:26 --> 00:15:28 but it sounds like it's doing something more
00:15:28 --> 00:15:31 specific. Yes. Quality review is what converts
00:15:31 --> 00:15:34 everything else into behavioral change. The metrics
00:15:34 --> 00:15:37 and guidance and training are inputs. But quality
00:15:37 --> 00:15:39 review is the transmission, the thing that takes
00:15:39 --> 00:15:41 those inputs and produces professional consequences
00:15:41 --> 00:15:44 for individual ALJs. And without it, the inputs
00:15:44 --> 00:15:47 then just float? Right. An ALJ who knows their
00:15:47 --> 00:15:49 case completion rate is tracked but faces no
00:15:49 --> 00:15:51 real consequences from the tracking has no reason
00:15:51 --> 00:15:55 to attend to it, like the efforts that students
00:15:55 --> 00:15:58 put into pass -fail classes. Quality review is
00:15:58 --> 00:16:00 what makes the tracking bite, because quality
00:16:00 --> 00:16:02 review feeds into performance evaluations, and
00:16:02 --> 00:16:04 performance evaluations have career consequences.
00:16:04 --> 00:16:07 So what does that look like at SSA? They have
00:16:07 --> 00:16:10 a dedicated review apparatus inside the Office
00:16:10 --> 00:16:13 of Hearings Operations where ALJ decisions are
00:16:13 --> 00:16:16 reviewed as part of ongoing oversight. Some routine
00:16:16 --> 00:16:19 sampling, some focused review on cases flagged
00:16:19 --> 00:16:21 by particular criteria. The review checks whether
00:16:21 --> 00:16:24 decision followed the applicable SSRs, whether
00:16:24 --> 00:16:26 the analysis of the evidence was adequate, whether
00:16:26 --> 00:16:28 the procedural rules in HALICs were followed.
00:16:28 --> 00:16:31 Okay, so let's make this concrete. What does
00:16:31 --> 00:16:33 this actually look like from the inside then?
00:16:34 --> 00:16:37 All right. Let's think of an ALJ. We can call
00:16:37 --> 00:16:39 her Judge Williams. She's been on the bench for
00:16:39 --> 00:16:41 seven years. She's got a solid record. Her approval
00:16:41 --> 00:16:44 rate is around the office median. On Tuesday,
00:16:44 --> 00:16:47 she hears a case. The claimant's 54, has degenerative
00:16:47 --> 00:16:50 disc disease, an MRI showing disc damage, a treating
00:16:50 --> 00:16:52 physician opinion supporting an inability to
00:16:52 --> 00:16:55 perform sedentary work, consistent testimony
00:16:55 --> 00:16:57 at the hearing. She finds the claimant disabled
00:16:57 --> 00:17:00 and writes a decision. So a routine outcome.
00:17:00 --> 00:17:02 Right. And then three months later, a memo comes
00:17:02 --> 00:17:05 to her. Her decision in that case has been selected
00:17:05 --> 00:17:07 for quality review. The reviewing office found
00:17:07 --> 00:17:09 a concern. They think her analysis didn't sufficiently
00:17:09 --> 00:17:11 engage with the functional capacity evidence
00:17:11 --> 00:17:14 in the file. The memo doesn't reverse her decision.
00:17:14 --> 00:17:16 It's not an appeal. The claimant's award stands,
00:17:17 --> 00:17:19 but the memo goes to her supervisor and it goes
00:17:19 --> 00:17:22 in her file. And so now Judge Williams knows
00:17:22 --> 00:17:25 things. Right. She knows this decision is flagged.
00:17:25 --> 00:17:28 She knows her supervisor knows. She knows that
00:17:28 --> 00:17:31 if another favorable decision of hers gets similarly
00:17:31 --> 00:17:33 flagged, there's a pattern. She can't fully see
00:17:33 --> 00:17:35 the rubric that the reviewing office applied.
00:17:36 --> 00:17:37 That rubric isn't published the way regulation
00:17:37 --> 00:17:40 is. But she can infer from the memo that the
00:17:40 --> 00:17:42 way she handled functional capacity evidence
00:17:42 --> 00:17:45 is now on someone's radar. And so the next time
00:17:45 --> 00:17:47 she has a similar case. Right. So the next time
00:17:47 --> 00:17:49 she's writing for the reviewer as much as for
00:17:49 --> 00:17:51 the claimant. She might spend more time on functional
00:17:51 --> 00:17:54 capacity, maybe a lot more time, or, and this
00:17:54 --> 00:17:57 is the part the system depends on, maybe she's
00:17:57 --> 00:17:58 slightly more willing to find that the evidence
00:17:58 --> 00:18:01 doesn't quite get the person to disabled. Same
00:18:01 --> 00:18:03 medical record, same testimony, but the decision
00:18:03 --> 00:18:06 tilts a degree. And nobody told her to deny anything.
00:18:06 --> 00:18:08 Right. Nobody had to. This information moved
00:18:08 --> 00:18:10 through an institutional feedback loop, and she
00:18:10 --> 00:18:12 learned something about what the agency considers
00:18:12 --> 00:18:15 adequate and what it considers inadequate. That
00:18:15 --> 00:18:17 learning affects the next case and the case after
00:18:17 --> 00:18:20 that. Every favorable decision she writes is
00:18:20 --> 00:18:22 now potentially a draft going to a reviewer whose
00:18:22 --> 00:18:25 rubric she can't fully see. And that rubric isn't
00:18:25 --> 00:18:28 a regulation, so she can't read it. She can only
00:18:28 --> 00:18:31 infer from what gets flagged. Right. But the
00:18:31 --> 00:18:33 criteria can shift with administration changes.
00:18:33 --> 00:18:36 So you never stop being evaluated, but the clock
00:18:36 --> 00:18:38 can reset every time that leadership's priorities
00:18:38 --> 00:18:41 change. And if it keeps happening, three or four
00:18:41 --> 00:18:43 flags over six months, her supervisor starts
00:18:43 --> 00:18:46 having conversations with her, not about specific
00:18:46 --> 00:18:48 cases, about patterns, and whether her approach
00:18:48 --> 00:18:50 is consistent with current agency expectations,
00:18:51 --> 00:18:53 whether she's reading the SSRs the way the agency
00:18:53 --> 00:18:55 reads them. Those conversations can feed into
00:18:55 --> 00:18:57 her annual performance review, which affects
00:18:57 --> 00:19:00 her standing, her assignments, and her professional
00:19:00 --> 00:19:02 life inside the agency. None of it's reversal
00:19:02 --> 00:19:05 of any specific case, but all of it shapes what
00:19:05 --> 00:19:07 she does in every subsequent case. OK, you said
00:19:07 --> 00:19:11 earlier the APA protects ALJs from removal based
00:19:11 --> 00:19:14 on the outcome of any individual case. But this
00:19:14 --> 00:19:19 isn't removal. Right. So that APA removal, that's
00:19:19 --> 00:19:23 that's a little in the air right now. But on
00:19:23 --> 00:19:25 paper, at least, that protection is real. But
00:19:25 --> 00:19:28 it's a narrower thing than freedom from institutional
00:19:28 --> 00:19:30 pressure. The ALJ can't be fired for deciding
00:19:30 --> 00:19:33 the case wrong, but she might be placed under
00:19:33 --> 00:19:35 quality review that correlates strongly with
00:19:35 --> 00:19:37 certain outcomes. She can have a performance
00:19:37 --> 00:19:39 evaluation that reflects patterns in her decision
00:19:39 --> 00:19:41 -making. She can receive feedback that makes
00:19:41 --> 00:19:44 the agency's expectations clear. None of it is
00:19:44 --> 00:19:47 removal for a case outcome, but all of it shapes
00:19:47 --> 00:19:49 how she's handling the next few thousand cases
00:19:49 --> 00:19:52 of her career. Quality review is a permanent
00:19:52 --> 00:19:54 shadow apparatus that's evaluating ALJ work against
00:19:54 --> 00:19:57 standards that aren't fully public and that evolve
00:19:57 --> 00:19:59 over time. So the formal independence protections
00:19:59 --> 00:20:02 are real, assuming that they stay real. They
00:20:02 --> 00:20:04 just protect something narrower than what most
00:20:04 --> 00:20:08 people imagine. So immigration has come up in
00:20:08 --> 00:20:11 basically every episode of this trilogy as the
00:20:11 --> 00:20:14 more aggressive version of whatever we're describing.
00:20:14 --> 00:20:17 Would that be the same as here? Oh, yes. So everything
00:20:17 --> 00:20:19 we've talked about, these metrics, guidance,
00:20:19 --> 00:20:22 performance review, quality review, all of this
00:20:22 --> 00:20:24 exists in immigration, plus structural features
00:20:24 --> 00:20:27 that aren't present with Social Security. Let's
00:20:27 --> 00:20:30 start with the amplification. In March 2018,
00:20:30 --> 00:20:33 the Executive Office for Immigration Review issued
00:20:33 --> 00:20:35 a memo from its director that was implementing
00:20:35 --> 00:20:37 new performance metrics for immigration judges.
00:20:38 --> 00:20:40 It was supposed to be effective October 1st of
00:20:40 --> 00:20:42 that year, and the memo said that to receive
00:20:42 --> 00:20:44 a satisfactory rating, an immigration judge had
00:20:44 --> 00:20:48 to complete 700 cases per year and have fewer
00:20:48 --> 00:20:51 than 15 % of decisions remanded by the BIA or
00:20:51 --> 00:20:55 the circuit courts. 700 cases a year. That seems
00:20:55 --> 00:20:58 like a lot. It sure does. So the pressure to
00:20:58 --> 00:21:00 move fast was built directly into the formal
00:21:00 --> 00:21:03 evaluation structure. The quotas were modified
00:21:03 --> 00:21:05 after litigation and eventually rescinded by
00:21:05 --> 00:21:08 the Biden administration. The underlying pressure
00:21:08 --> 00:21:10 didn't totally disappear. It just took a less
00:21:10 --> 00:21:12 explicit form. And the structural piece beyond
00:21:12 --> 00:21:15 the metrics? So immigration judges are Department
00:21:15 --> 00:21:18 of Justice employees. They are not ALJs under
00:21:18 --> 00:21:21 the APA. They don't have the same statutory tenure
00:21:21 --> 00:21:24 protections. They don't have the same removal
00:21:24 --> 00:21:25 standards or the same independence architecture.
00:21:26 --> 00:21:28 Their professional survival is more directly
00:21:28 --> 00:21:31 tied to keeping leadership satisfied. So the
00:21:31 --> 00:21:33 ex -ante control mechanisms that exist at Social
00:21:33 --> 00:21:35 Security exist in immigration with fewer institutional
00:21:35 --> 00:21:38 buffers between the judge and the pressure. And
00:21:38 --> 00:21:40 we already saw from last episode that the attorney
00:21:40 --> 00:21:43 general can certify cases and rewrite doctrine
00:21:43 --> 00:21:46 from above. Right. So we now have explicit quotas
00:21:46 --> 00:21:50 on the front end, or at least an expectation
00:21:50 --> 00:21:52 that a lot of cases will move through. We have
00:21:52 --> 00:21:54 weaker independence protections throughout, and
00:21:54 --> 00:21:56 we have the attorney general as a trump card
00:21:56 --> 00:21:58 at the top. The cumulative pressure on an immigration
00:21:58 --> 00:22:00 judge is different in kind from the pressure
00:22:00 --> 00:22:03 on a Social Security ALJ. And the stakes in any
00:22:03 --> 00:22:05 individual case, whether someone is returned
00:22:05 --> 00:22:07 to a country where they may be killed, are about
00:22:07 --> 00:22:09 as high as stakes get in administrative law.
00:22:10 --> 00:22:13 So is there a legal line between legitimate oversight
00:22:13 --> 00:22:16 and improper interference? Because we keep talking
00:22:16 --> 00:22:18 about this as a system, and systems, they don't
00:22:18 --> 00:22:21 have villains. But somewhere in there, someone's
00:22:21 --> 00:22:23 making choices. And some of those choices feel
00:22:23 --> 00:22:26 like exactly what the judge earlier called an
00:22:26 --> 00:22:29 atmosphere of tension and unfairness. So there's
00:22:29 --> 00:22:31 no clean legal line. What courts have been willing
00:22:31 --> 00:22:34 to do is police the extremes. A direct command
00:22:34 --> 00:22:37 to decide a specific case a specific way? Everybody
00:22:37 --> 00:22:40 agrees that's improper. Public targeting of individual
00:22:40 --> 00:22:43 judges based on approval rates? That's what the
00:22:43 --> 00:22:45 1984 court found, and courts have sometimes been
00:22:45 --> 00:22:48 willing to reach that. And the structural version
00:22:48 --> 00:22:50 of this discussion, the metrics and guidance
00:22:50 --> 00:22:53 and performance systems? Those are absolutely
00:22:53 --> 00:22:55 tolerated. And the alternative would be courts
00:22:55 --> 00:22:57 monitoring how agencies manage their own internal
00:22:57 --> 00:23:00 operations, their training programs, their quality
00:23:00 --> 00:23:02 review criteria, their performance evaluation
00:23:02 --> 00:23:05 systems. Courts aren't set up to do that. They
00:23:05 --> 00:23:07 don't have the institutional capacity, and they
00:23:07 --> 00:23:08 haven't been willing to claim the authority.
00:23:09 --> 00:23:11 So the line gets drawn at overt, individualized
00:23:11 --> 00:23:13 pressure, and the structural mechanisms that
00:23:13 --> 00:23:16 operate at scale are largely left alone. Has
00:23:16 --> 00:23:18 anyone ever successfully challenged a quality
00:23:18 --> 00:23:22 review rubric? So a FOIA, an APA claim, anything,
00:23:22 --> 00:23:25 or is it just structurally unreachable? Practically
00:23:25 --> 00:23:28 unreachable, but not absolutely. FOIA litigation
00:23:28 --> 00:23:30 has surfaced pieces of internal review criteria
00:23:30 --> 00:23:33 over the years, particularly during that Bellman
00:23:33 --> 00:23:36 era litigation in the 1980s, where Discovery
00:23:36 --> 00:23:38 exposed those targeting memos that became the
00:23:38 --> 00:23:42 basis of that case. And the 2018 memo became
00:23:42 --> 00:23:44 public because journalists obtained it. So things
00:23:44 --> 00:23:48 do leak. But what there isn't is a successful
00:23:48 --> 00:23:50 APA challenge to a quality review rubric itself.
00:23:50 --> 00:23:52 The structural problems we've talked about, there's
00:23:52 --> 00:23:55 no specific APA provision that the practices
00:23:55 --> 00:23:57 violate. It applies to those challenges, too.
00:23:57 --> 00:24:00 The cases that have come closest both ended without
00:24:00 --> 00:24:03 injunctive relief. Plaintiffs win pieces. They
00:24:03 --> 00:24:05 just don't win structural reform. Which means
00:24:05 --> 00:24:07 the mechanisms producing the most behavioral
00:24:07 --> 00:24:10 change are the ones most insulated from challenge.
00:24:10 --> 00:24:12 The quality review rubric isn't published the
00:24:12 --> 00:24:14 way a regulation is. The performance evaluation
00:24:14 --> 00:24:17 is internal. So from outside the agency, we see
00:24:17 --> 00:24:20 outputs. An aggregate approval rate, case completion
00:24:20 --> 00:24:23 targets when they service in litigation, guidance
00:24:23 --> 00:24:25 documents if an ALJ happens to cite one in a
00:24:25 --> 00:24:27 decision. But you don't see the mechanism. The
00:24:27 --> 00:24:30 mechanism is operating in the background. Let's
00:24:30 --> 00:24:34 circle back to Judge Williams' case. The 54 -year
00:24:34 --> 00:24:36 -old with the disc problem, he won, his decision
00:24:36 --> 00:24:39 stood. But you said something earlier I want
00:24:39 --> 00:24:41 to come back to. You said next time she has a
00:24:41 --> 00:24:44 similar case, the decision tilts a degree. Yes.
00:24:45 --> 00:24:47 So. Somewhere out there is the next claimant,
00:24:47 --> 00:24:50 same age, same disc damage, same kind of treating
00:24:50 --> 00:24:53 physician opinion. And that one will come out
00:24:53 --> 00:24:55 differently. Right. That's how we get this 17
00:24:55 --> 00:24:58 point shift across a decade. It's not one big
00:24:58 --> 00:25:00 event. It's hundreds of thousands of cases where
00:25:00 --> 00:25:03 the decision tilted a degree because the decisions
00:25:03 --> 00:25:05 around the decision maker tilted a degree. The
00:25:05 --> 00:25:07 claimant doesn't see any of that. The claimant
00:25:07 --> 00:25:11 sees one hearing, sees their ALJ and gets a decision.
00:25:11 --> 00:25:13 The decision cites the regulations. It walks
00:25:13 --> 00:25:15 through the evidence. It reaches a conclusion
00:25:15 --> 00:25:17 that's formally correct in every visible respect.
00:25:18 --> 00:25:20 But the case that would have been approved in
00:25:20 --> 00:25:23 the same hearing office by the same ALJ on the
00:25:23 --> 00:25:25 same evidence three years earlier? Right. It
00:25:25 --> 00:25:27 can get denied today. Not because anything about
00:25:27 --> 00:25:29 the case changed. Because the system around the
00:25:29 --> 00:25:31 judge changed. The quality review environment
00:25:31 --> 00:25:33 tightened. The feedback signals accumulated.
00:25:34 --> 00:25:37 The metrics pressure shifted. None of which the
00:25:37 --> 00:25:39 claimant can see. None of which the lawyer can
00:25:39 --> 00:25:41 see, even if they're experienced. And none of
00:25:41 --> 00:25:42 which will show up in a federal court appeal.
00:25:43 --> 00:25:46 And when they lose, the natural response is to
00:25:46 --> 00:25:49 ask what they did wrong. Right. What they should
00:25:49 --> 00:25:51 have said differently, what evidence they should
00:25:51 --> 00:25:53 have brought. Sometimes those are exactly the
00:25:53 --> 00:25:56 right questions, but sometimes they're not. Sometimes
00:25:56 --> 00:25:57 the claimant did nothing wrong and the outcome
00:25:57 --> 00:26:00 reflects the system the ALJ is operating in rather
00:26:00 --> 00:26:02 than the merits of the claimant's particular
00:26:02 --> 00:26:05 case. And the lawyer can't tell them that because
00:26:05 --> 00:26:08 the lawyer can't see it either. Right. So if
00:26:08 --> 00:26:10 we look on paper, we can say that the formal
00:26:10 --> 00:26:13 independence of ALJs is real. The APA protections
00:26:13 --> 00:26:16 are meaningful, and formal independence is a
00:26:16 --> 00:26:17 narrower thing than freedom from institutional
00:26:17 --> 00:26:20 pressure. The outcome ALJs produce reflect the
00:26:20 --> 00:26:23 system that they're operating in, not just their
00:26:23 --> 00:26:25 individual judgment. When that system shifts,
00:26:25 --> 00:26:27 the outcomes shift across thousands of cases
00:26:27 --> 00:26:29 without any individual decision being reversed,
00:26:30 --> 00:26:32 without any directive being issued, and without
00:26:32 --> 00:26:35 visible mechanisms anyone can point to. So the
00:26:35 --> 00:26:38 answer to how do agency preferences become hearing
00:26:38 --> 00:26:41 outcomes isn't any single thing. Right. It's
00:26:41 --> 00:26:43 these things together, upstream of the hearing
00:26:43 --> 00:26:46 and mostly invisible from outside and structured
00:26:46 --> 00:26:49 in ways that are hard to challenge. So we spent
00:26:49 --> 00:26:51 three episodes on this, the hearing level variation,
00:26:51 --> 00:26:55 the review layer, and now this. The picture just
00:26:55 --> 00:26:58 keeps getting messier. Much messier. So the textbook
00:26:58 --> 00:27:01 version is independent adjudicators, structured
00:27:01 --> 00:27:03 appellate review, judicial oversight at the back
00:27:03 --> 00:27:05 end. The operational reality is what we've spent
00:27:05 --> 00:27:08 three episodes talking about. The horizontal
00:27:08 --> 00:27:10 variation that gave the system different starting
00:27:10 --> 00:27:12 points across different offices. The narrow internal
00:27:12 --> 00:27:15 review that lets local patents harden. The ex
00:27:15 --> 00:27:17 ante apparatus, the metrics, the guidance, the
00:27:17 --> 00:27:19 quality review that does this work of shifting
00:27:19 --> 00:27:23 where the median ALJ lands. Judge Williams' next
00:27:23 --> 00:27:25 claimant is in that number and her one after
00:27:25 --> 00:27:28 that, too. And neither one knows it. OK, so what
00:27:28 --> 00:27:30 are we doing next week? Next week, we are doing
00:27:30 --> 00:27:33 a deep dive. on social security disability. It
00:27:33 --> 00:27:35 might feel like we've been doing that for so
00:27:35 --> 00:27:38 many of these episodes, but next week we'll be
00:27:38 --> 00:27:40 talking about substantively how the evaluation
00:27:40 --> 00:27:43 occurs. If you want to see what the structures
00:27:43 --> 00:27:45 we've been describing actually do to real people,
00:27:46 --> 00:27:48 that's where you see them. So that does it for
00:27:48 --> 00:27:50 today's episode on administrative remedies. Thank
00:27:50 --> 00:27:53 you for joining us today. Please, if you enjoy
00:27:53 --> 00:27:56 this podcast, enjoy this episode, give us a like
00:27:56 --> 00:27:59 on Spotify, iTunes, or whatever platform you're
00:27:59 --> 00:28:02 listening on. And be sure to tune in next time
00:28:02 --> 00:28:04 where we'll continue to dive into the contours
00:28:04 --> 00:28:06 of administrative law, because remember, you
00:28:06 --> 00:28:08 can't fix what you don't understand.