(I’m ripping off Sasha Chapin and writing a “post” every “day” for 30 days. Kindly adjust quality expectations.)

When should you enforce standards by permission, and when should you enforce them by punishment?

Personally, I’m pretty happy that new aircraft have to be pre-approved by the FAA before I can fly in them, rather than just letting Boeing YOLO out new models and recalling any that end up not working. But it would be infuriating if the FDA demanded to pre-approve every new culinary delight America’s food industry graced us with.

In the life of a scientist, though, nowhere is the pre-approval vs. post-monitoring distinction more salient than when it comes to Institutional Review Boards (IRBs).

An IRB is a group of people who decide whether a research project is ethical. In the U.S. and basically everywhere else, you’re not allowed to do research involving humans unless an IRB first pre-approves your research plan.

IRBs are probably a close 2nd to paywalled journals in the rankings of things scientists hate most. This isn’t because scientists want to do unethical research. It’s because IRBs are a pain in everyone’s butt.

IRB horror stories abound. Scott Alexander’s IRB Nightmare is the best known, with interesting follow up. Andrew Gelman has some good ones, and you can find endless more on reddit. The moral monsters at Give Directly have one from when they once tried to give people free money. A professor I know spent 3 years – 3 years – getting approval for a minutes-long experiment to try and wake up a coma patient using noninvasive, ultra-low intensity ultrasound, and that’s a fairly standard story.

But of course the worst stories always get the most attention. How bad are IRBs really, overall? Pretty bad, is the answer. Even at a top institution like Berkeley, getting permission to email out a social science survey takes at least 3 to 4 weeks. More involved research projects take much longer. IRBs often mandate changes that drastically increase the costs of studies, like requiring scientists to pay a doctor to be present for blood draws or other trivial tasks. IRBs make it challenging for scientists at different institutions to collaborate, since e.g. for medical device studies, if any one of the IRBs at any of the institutions does not approve a study, the study has to go all the way to the FDA to get approval.

How did things get like this?

There’s no question that the creation of IRBs was well-meaning.

The 20th century was full of a lot of awful, unethical science, a shocking amount of which was performed by the U.S. government. In particular, starting in 1932 the U.S. Public Health Service (PHS) and the CDC ran the infamous Tuskegee Syphilis Study. After PHS employee Peter Buxton whistleblew on the study in 1972 there was national outrage, and congress passed the National Research Act in 1974 in response. There followed a series of commissions and reports, and by 1991 the Department of Health and Human Services (HHS) had written a set of regulations for doing human subject research that they were happy with. And the centerpiece of these regulations was the newly invented IRB concept and mandate.

HHS published these regulations1 in 45 CFR part 46, and they were so good that 15 other agencies simultaneously copy-pasted HHS’s regulations into their own regulations. Since then everyone calls these regulations the Common Rule, since they’re common to nearly every government agency.2 There’s more to IRBs than just the Common Rule, since agencies can have their own additional regulations. The FDA for example has lots of extra rules about IRBs for clinical trial research.

All of this is enforced by the Office of Human Research Protections in HHS.

Who actually has to follow these rules?

The Common Rule says you have to use an IRB if your research is supported in any way by HHS or another Common Rule agency.

“Well then,” you might say, “I’ll just get private funding for all my research, and then I don’t have to use an IRB.” That’s absolutely true, and a lot of research is done that way, like political polls (except not the US Census, because that’s funded by the Department of Commerce, a Common Rule agency).

Except that IRBs are also required by basically any agency if you ever want to submit the results of that research to them. For example, if you’re a biotech company, and you’d like to get a drug approved by the FDA, which you have to do in order to sell it, all the research you do to prove the drug is safe and effective has to have been IRB-approved. And even if you don’t care about the FDA, if you’re a researcher and want to publish in basically any journal, just about all of them require IRB approval for anything they publish. And even if you just want to throw your work on arXiv, just about every institution mandates that any research done with their equipment or money go through their IRB. Oh, and in case you were hoping you could at least do some self-experimentation without IRB approval, you can’t.

The Common Rule may have planted the IRB seed, but at this point its roots are everywhere.

Can we fix this by making a better IRB?

To be continued…

Have feedback? Find a mistake? Please let me know!

  1. Quick reminder of how rules work in the US:

    • Congress makes laws, a.k.a. statutes or acts or legislation.
    • Government agencies (FDA, EPA, NSF, etc.) make regulations, which aren’t laws but have the “force of law.”
    • Laws are collected in the United States Code. Regulations are collected in the Code of Federal Regulations.
      • There is about 5x more regulation than law by word count.
    • The courts get to decide whether laws and/or regulations contradict each other. The constitution takes precedence over laws, and laws take precedence over regulations.
    • There are also state and local versions of all the above.
  2. The CIA didn’t want to join in, probably because they were having fun doing stuff like MKUltra, but eventually Reagan forced them to in 1981, along with telling them to stop assassinating people. ↩︎