Why is Generalisability Scrutinised in N of 1 Trials but Assumed in Other Study Designs?

Generalisability is one of the most frequently cited concerns in discussions of N of 1 trials. Critics argue that findings from individual-level experiments cannot meaningfully inform practice beyond the person studied, even when trials use standardised protocols and are analysed in aggregate (pooled N of 1 trials). This concern is often treated as self-evident. Yet it prompts a critical question: why is generalisability considered a central limitation of N of 1 designs, while it is frequently assumed rather than demonstrated in randomised controlled trials (RCTs)?

Generalisability is not conferred by design label alone. It depends on design choices, context, measurement, and reporting transparency. This is particularly relevant when examining RCTs, which are routinely regarded as the gold standard and whose findings are often interpreted as broadly applicable by default. However, external validity is an empirical question, not an automatic property of randomisation.

A systematic review by Malmivaara published in 2019 [1] examined RCTs published in four leading medical journals over nine months in 2017. Across 161 trials, substantial gaps were found in reporting information necessary to judge applicability. Descriptions of patient selection prior to randomisation were often incomplete, making it difficult to assess how closely trial participants resembled real-world patients. Reporting of healthcare settings was inconsistent, limiting insight into contextual transferability. Information on comorbidities (despite their prevalence in clinical populations) was frequently absent. Measures of functioning, behavioural factors, environmental context, and inequity-related variables were inconsistently documented. Co-interventions and reasons for dropout were also variably reported, complicating interpretation of both internal validity and real-world relevance.

These deficiencies make it difficult to determine to whom and under what conditions trial findings apply. As Rothwell [2] observed, reporting of determinants of external validity in trial publications is usually inadequate. Subsequent analyses in multiple clinical areas have shown that many real-world patients would not have met eligibility criteria for the trials guiding their care.

The issue is not randomisation itself, but assumptions about external validity that often go unexamined. All designs - parallel-group RCTs, crossover trials, pragmatic trials, and pooled N-of-1 trials - face the same fundamental constraint: evidence can only be generalised to the extent that the study population, context, measurement, and reporting justify such inference.

Reporting standards are therefore central. The CONSORT Extension for Reporting N of 1 Trials (CENT) explicitly requires detailed reporting of participant characteristics, context, outcomes, and analytic methods, facilitating assessment of both methodological rigour and applicability. Generalisability is shaped by inclusion criteria, outcome selection, study setting, and transparency, not by trial design alone.

Claims about generalisability should therefore be evaluated empirically rather than assumed. The relevant question is not which design is inherently superior, but whether a given design is fit for purpose for the decision at hand. Evidence strength is determined not only by internal validity or sample size, but by how applicable it is to decisions beyond the study itself.

References

[1] Malmivaara,A. Generalizability of findings from randomized controlled trials is limited in the leading general medical journals. Journal of clinical epidemiology 107 (2019): 36-41. Link: https://www.sciencedirect.com/science/article/pii/S0895435618306668

[2] Rothwell, P.M. "External validity of randomised controlled trials:“to whom do the results of this trial apply?”." The Lancet 365.9453 (2005): 82-93.https://pubmed.ncbi.nlm.nih.gov/15639683/