Accepting Bayesian methods for RCTs is great news and leading biostatisticians like Frank Harrell have been pushing for this change for many years. What I'm most interested to see: will this actually be implemented in practice, or will it be incredibly rare and niche, like Bayesian methods are currently in most biomedical fields?
Regulators are generally really conservative. Spiegelhalter et al. already wrote a fantastic textbook on Bayesian methods for trial analysis back in 2004. It is a great synthesis, and used by statisticians from other fields. I have seen it quoted in e.g. DeepMind presentations.
Bayesian methods enable using prior information and fancy adaptive trial designs, which have the potential to make drug development much cheaper. It's also easier to factor in utility functions and look at cost:benefit. But things move slowly.
They are used in some trials, but not the norm, and require rowing against the stream. This is actually a great niche for a startup. Leveraging prior knowledge to make target discovery, pre-clinical, and clinical trials more adaptive and efficient.
Journals are also conservative. But Bayesian methods are not that niche anymore. Even mainstream journals such as Nature or Nature Genetics include Bayesian-specific items in their standard submission checklists [1]. For example, they require you to indicate prior choice and MCMC parameters.
Bayesian methods are incredibly canonical in most fields I’ve been involved with (cosmology is one of the most beautiful paradises for someone looking for maybe the coolest club of Bayesian applications). I’m surprised there are still holdouts, especially in fields where the stakes are so high. There are also plenty of blog articles and classroom lessons about how frequentist trial designs kill people: if you are not allowed to deviate from your experiment design but you already have enough evidence to form a strong belief about which treatment is better, is that unethical? Maybe the reality is a bit less simplistic but ive seen many instantiations of that argument around.
If choosing a Bayesian approach in a clinical trial can reduce the number of recruited subjects, I would imagine the pharma industry is strongly incentivized to adopt it.
Moreover you can manipulate your results by disingenuous prior choices, and the smaller sample you have the stronger this effect is. I am not sold on the FDA's ability to objectively and carefully review Bayesian research designs, especially given the current administration's wanton disregard for the public good.
I would think there is less opportunity to manipulate your results with bayesian methods than with frequentist ones. Because the frequentist methods don't just require an alternate hypothesis, they depend on the exact set of outcomes possible given your experimental design. You can modify your experimental design afterwards and invisibly make your p-value be whatever you want
BOIN (Bayesian Optimal Interval) trial design is already very common in Phase I studies across therapeutic areas.
Biggest benefit I see from this guidance is support for rare disease trials, where patients are harder to find. Also regulatory bodies will be taking a closer look at stratification groups when it comes time for approval, so sponsors need to keep a super close eye on ensuring even enrollment and preventing misstrats.
The application of Bayesian probabilistic reasoning in general (as described in this video) is not the same thing as "Bayesian statistics" specifically, which usually to modeling and posterior inference using both a likelihood model and a prior model. It's a very different approach to statistical inference both in theory and in practice. This creator himself is either ignorant of this distinction or is trying to mislead his viewers in order to dunk on the FDA. It's obvious from the video comments that many people have indeed been misled as to what Bayesian statistics is and what the implications of its might be in the context of clinical trials.
Indeed, even more broadly online "Bayesian" seems to have taken on the form of "I know Bayes' Rule and think about base rates" as opposed to "Do you prefer jags or stan for MCMC?"
It's an FDA guidance doc so you can expect to see this affecting new filings or supplemental filings for upcoming FDA submissions and clinical trial designs. This is good news, statistical plans are always a point of contention during trial design and the submission process. This guidance lays a line in the sand and will remove some of the reviewer-to-reviewer variance present in the current FDA staff.
Recent comments during JPM don't have me hopeful for smooth sailing at the FDA any time soon.
> He said he was also deeply troubled by agency staff “being trampled on.” He referred to one individual who was “writing inflammatory emails using the F bomb,” telling center directors and deputy center directors that “they will go after them, that they were going to lose their jobs if they did not play ball.”
> He would also not name this person. STAT has reported that employees have been fearful under Vinay Prasad, director of the Center for Biologics Evaluation and Research.
> “It’s terrible to see 25 years of work dismantled,” said Pazdur, who founded the FDA’s oncology center. He later added, “I did not leave because I wanted to leave.”
> “I think I have been consistently critical of parts of the FDA regardless of administration, but what’s emerged over the past few months is just reflective of complete and total disarray and a complete lack of functional leadership,” said Brian Skorney, an analyst at the investment bank Baird.
Horrific stuff seeing the level of expertise leaving the building at FDA. Some of the program managers I’ve dealt with in the past have taken the payout with their roles left vacant. Just absolutely abhorrent levels of leadership, the 30 day response timeframes for submissions is starting to produce some sloppy work on their side.
Trust no one, I say. If a doctor prescribes a drug that's only approved in US not the rest of G-7, then maybe take a pass. And yes, FDA is the global leader, but this disarray has caused others to step up--similar to EU being a bit more self sufficient and responsible when it comes to national defense.
Vinay Prasad is an oncologist who made his career calling out many of the FDA’s brazenly stupid oncology drug approvals, so I am shocked - shocked! - that the creator of that process would be unhappy with his leadership.
“Foxes agree that henhouse security changes will lead to hungry animals.” News at 11.
> This guidance lays a line in the sand and will remove some of the reviewer-to-reviewer variance present in the current FDA staff.
That would be nice, but my experience is there can be quite significant variability between reviewers in different teams/groups, even on topics you'd think were well-established for many years, and for which there is existing FDA guidance.
I’m currently arguing a statistical plan for a neurostim phase two clinical trial IDE submission with the FDA. We got some borderline reckless comments back that are massively out of line with correspondence on previous pivotal submissions. It’s starting to look like some of the reviewers are taking on work outside the scope of their expertise. Really disappointing to see, especially with the new more streamlined MDR reqs coming out of the EU.
Despite working to enforce laws and guidelines --i.e. somewhat similar to a court sitting in judgement on a case-- there's apparently no expectation of consistency, there's no concept of 'case law' (i.e. you applied logic in this way before, so we have the right to expect that you apply it the same next time) which can be relied upon, and maybe worst of all, there's no route to challenge or appeal when bad judgements are handed down.
This is generally happening in the context of RCTs where valid causal inferences are (almost always) guaranteed via the study design, regardless of whether your analysis is frequentist or Bayesian.
Accepting Bayesian methods for RCTs is great news and leading biostatisticians like Frank Harrell have been pushing for this change for many years. What I'm most interested to see: will this actually be implemented in practice, or will it be incredibly rare and niche, like Bayesian methods are currently in most biomedical fields?
Regulators are generally really conservative. Spiegelhalter et al. already wrote a fantastic textbook on Bayesian methods for trial analysis back in 2004. It is a great synthesis, and used by statisticians from other fields. I have seen it quoted in e.g. DeepMind presentations.
Bayesian methods enable using prior information and fancy adaptive trial designs, which have the potential to make drug development much cheaper. It's also easier to factor in utility functions and look at cost:benefit. But things move slowly.
They are used in some trials, but not the norm, and require rowing against the stream. This is actually a great niche for a startup. Leveraging prior knowledge to make target discovery, pre-clinical, and clinical trials more adaptive and efficient.
Journals are also conservative. But Bayesian methods are not that niche anymore. Even mainstream journals such as Nature or Nature Genetics include Bayesian-specific items in their standard submission checklists [1]. For example, they require you to indicate prior choice and MCMC parameters.
[1] https://www.nature.com/documents/nr-reporting-summary-flat.p...
Bayesian methods are incredibly canonical in most fields I’ve been involved with (cosmology is one of the most beautiful paradises for someone looking for maybe the coolest club of Bayesian applications). I’m surprised there are still holdouts, especially in fields where the stakes are so high. There are also plenty of blog articles and classroom lessons about how frequentist trial designs kill people: if you are not allowed to deviate from your experiment design but you already have enough evidence to form a strong belief about which treatment is better, is that unethical? Maybe the reality is a bit less simplistic but ive seen many instantiations of that argument around.
If choosing a Bayesian approach in a clinical trial can reduce the number of recruited subjects, I would imagine the pharma industry is strongly incentivized to adopt it.
Moreover you can manipulate your results by disingenuous prior choices, and the smaller sample you have the stronger this effect is. I am not sold on the FDA's ability to objectively and carefully review Bayesian research designs, especially given the current administration's wanton disregard for the public good.
I would think there is less opportunity to manipulate your results with bayesian methods than with frequentist ones. Because the frequentist methods don't just require an alternate hypothesis, they depend on the exact set of outcomes possible given your experimental design. You can modify your experimental design afterwards and invisibly make your p-value be whatever you want
BOIN (Bayesian Optimal Interval) trial design is already very common in Phase I studies across therapeutic areas.
Biggest benefit I see from this guidance is support for rare disease trials, where patients are harder to find. Also regulatory bodies will be taking a closer look at stratification groups when it comes time for approval, so sponsors need to keep a super close eye on ensuring even enrollment and preventing misstrats.
Please watch https://www.youtube.com/shorts/2TWu_4gd-ik
Been used since the 90s.
The application of Bayesian probabilistic reasoning in general (as described in this video) is not the same thing as "Bayesian statistics" specifically, which usually to modeling and posterior inference using both a likelihood model and a prior model. It's a very different approach to statistical inference both in theory and in practice. This creator himself is either ignorant of this distinction or is trying to mislead his viewers in order to dunk on the FDA. It's obvious from the video comments that many people have indeed been misled as to what Bayesian statistics is and what the implications of its might be in the context of clinical trials.
Indeed, even more broadly online "Bayesian" seems to have taken on the form of "I know Bayes' Rule and think about base rates" as opposed to "Do you prefer jags or stan for MCMC?"
It's an FDA guidance doc so you can expect to see this affecting new filings or supplemental filings for upcoming FDA submissions and clinical trial designs. This is good news, statistical plans are always a point of contention during trial design and the submission process. This guidance lays a line in the sand and will remove some of the reviewer-to-reviewer variance present in the current FDA staff.
> This guidance lays a line in the sand and will remove some of the reviewer-to-reviewer variance present in the current FDA staff.
https://www.statnews.com/2026/01/13/richard-pazdur-jpm-fda-c...
Recent comments during JPM don't have me hopeful for smooth sailing at the FDA any time soon.
> He said he was also deeply troubled by agency staff “being trampled on.” He referred to one individual who was “writing inflammatory emails using the F bomb,” telling center directors and deputy center directors that “they will go after them, that they were going to lose their jobs if they did not play ball.”
> He would also not name this person. STAT has reported that employees have been fearful under Vinay Prasad, director of the Center for Biologics Evaluation and Research.
> “It’s terrible to see 25 years of work dismantled,” said Pazdur, who founded the FDA’s oncology center. He later added, “I did not leave because I wanted to leave.”
> “I think I have been consistently critical of parts of the FDA regardless of administration, but what’s emerged over the past few months is just reflective of complete and total disarray and a complete lack of functional leadership,” said Brian Skorney, an analyst at the investment bank Baird.
Horrific stuff seeing the level of expertise leaving the building at FDA. Some of the program managers I’ve dealt with in the past have taken the payout with their roles left vacant. Just absolutely abhorrent levels of leadership, the 30 day response timeframes for submissions is starting to produce some sloppy work on their side.
Trust no one, I say. If a doctor prescribes a drug that's only approved in US not the rest of G-7, then maybe take a pass. And yes, FDA is the global leader, but this disarray has caused others to step up--similar to EU being a bit more self sufficient and responsible when it comes to national defense.
Vinay Prasad is an oncologist who made his career calling out many of the FDA’s brazenly stupid oncology drug approvals, so I am shocked - shocked! - that the creator of that process would be unhappy with his leadership.
“Foxes agree that henhouse security changes will lead to hungry animals.” News at 11.
Cane you provide a list of the “stupid” drug approvals?
> This guidance lays a line in the sand and will remove some of the reviewer-to-reviewer variance present in the current FDA staff.
That would be nice, but my experience is there can be quite significant variability between reviewers in different teams/groups, even on topics you'd think were well-established for many years, and for which there is existing FDA guidance.
I’m currently arguing a statistical plan for a neurostim phase two clinical trial IDE submission with the FDA. We got some borderline reckless comments back that are massively out of line with correspondence on previous pivotal submissions. It’s starting to look like some of the reviewers are taking on work outside the scope of their expertise. Really disappointing to see, especially with the new more streamlined MDR reqs coming out of the EU.
Sounds familiar.
Despite working to enforce laws and guidelines --i.e. somewhat similar to a court sitting in judgement on a case-- there's apparently no expectation of consistency, there's no concept of 'case law' (i.e. you applied logic in this way before, so we have the right to expect that you apply it the same next time) which can be relied upon, and maybe worst of all, there's no route to challenge or appeal when bad judgements are handed down.
Anything noteworthy here? I noticed they did not mention causal inference.
This is generally happening in the context of RCTs where valid causal inferences are (almost always) guaranteed via the study design, regardless of whether your analysis is frequentist or Bayesian.
[dead]