Interpreting Randomized Controlled Trials

Authors

Pavlos Msaouel
Juhee Lee
Peter F Thall

Publication Date

9-22-2023

Journal

Cancers

Abstract

This article describes rationales and limitations for making inferences based on data from randomized controlled trials (RCTs). We argue that obtaining a representative random sample from a patient population is impossible for a clinical trial because patients are accrued sequentially over time and thus comprise a convenience sample, subject only to protocol entry criteria. Consequently, the trial's sample is unlikely to represent a definable patient population. We use causal diagrams to illustrate the difference between random allocation of interventions within a clinical trial sample and true simple or stratified random sampling, as executed in surveys. We argue that group-specific statistics, such as a median survival time estimate for a treatment arm in an RCT, have limited meaning as estimates of larger patient population parameters. In contrast, random allocation between interventions facilitates comparative causal inferences about between-treatment effects, such as hazard ratios or differences between probabilities of response. Comparative inferences also require the assumption of transportability from a clinical trial's convenience sample to a targeted patient population. We focus on the consequences and limitations of randomization procedures in order to clarify the distinctions between pairs of complementary concepts of fundamental importance to data science and RCT interpretation. These include internal and external validity, generalizability and transportability, uncertainty and variability, representativeness and inclusiveness, blocking and stratification, relevance and robustness, forward and reverse causal inference, intention to treat and per protocol analyses, and potential outcomes and counterfactuals.

Keywords

blocking, confidence intervals, generalizability, hazard ratios, random allocation, random sampling, random treatment assignment, randomized controlled trials, stratification, transportability

DOI

10.3390/cancers15194674

PMID

37835368

PMCID

PMC10571666

PubMedCentral® Posted Date

9-22-2023

PubMedCentral® Full Text Version

Post-print

Published Open-Access

yes