Explanation for Anxiety Outcome Discrepancy Between Small Clinical and Large Population Samples

What the evidence says

Sources on this page point to several distinct, non-competing explanations for why anxiety-related findings from small or clinical samples diverge from findings in larger or population-level samples, spanning statistical, sampling, and measurement mechanisms.

The statistical mechanism is the best-documented: small, early, and highly cited studies tend to overestimate effects, and the strength of evidence for a given hypothesis typically declines over time as replication studies present smaller effects than the original studies (Amrhein 2017). This inflation is a direct function of statistical power — at a power of 40%, effects that cross the significance threshold are on average about 1.5 times larger than the true effect size, an inflation that approaches zero as power approaches 100% (Amrhein 2017). Publication bias and selective reporting are described as more prevalent in early-stage studies, with significant results selected via statistical thresholds biased upward while non-significant results are suppressed, so that "most discovered true associations are inflated" (Amrhein 2017). A related but separate statistical mechanism comes from Costello (2022): standard significance testing considers only within-experiment variation and ignores between-experiment variation, which makes it systematically overconfident in labeling results significant. Consistent with this, many statistically significant results in psychology and related fields do not replicate reliably, and when a distributional null hypothesis test that accounts for between-experiment variation is applied to the Many Labs 1 dataset, about 10 percentage points of results significant at p < 0.00001 under standard testing turn out not significant (psig > 0.05) under the distributional test.

A separate, sample-composition mechanism appears in clinical-high-risk (CHR) psychosis research. CHR samples are enriched for transition risk, which increases observed transitions to psychosis but restricts the variance in baseline risk factors relative to population samples, limiting the ability to detect risk-factor associations (Hinton 2026). Within that same literature, comorbid anxiety and mood disorders in CHR youth show no measured association with transition to psychosis, per a narrative synthesis of prior reviews (Hinton 2026). This sits alongside a selection-into-sample effect: most CHR youth seek treatment for anxiety and depression rather than for psychosis-related concerns in the first place (Hinton 2026), which shapes who is included in these clinical cohorts before any statistical analysis occurs.

A third, measurement-level mechanism is documented in a single small clinical validation sample (n = 45; Akhavan-Abiri 2026). There, small sample size specifically decreased the magnitude of convergent-validity correlations relative to larger-sample studies, producing "a consistent, albeit more conservative, pattern of convergence." Small sample size separately reduced generalizability of findings and the stability of the factor structure — though the same source notes that simple one-factor models with adequate factor loadings can still achieve acceptable parameter stability even at small n. That study's sample also carried compounding artifacts: a female-skewed gender ratio (1:4, versus an expected 1:2 in DSM-5-TR panic-disorder epidemiology) that skewed findings toward women and restricted generalizability to men; a temporal mismatch between a past-week-referenced primary instrument and comparator instruments with broader reference periods, which introduced variability into observed correlations; and unestablished cross-cultural and linguistic equivalence of instruments, which may itself change response patterns and factor structure. Working in the opposite direction, use of gold-standard SCID-5 diagnostic interviews rather than convenience or community sampling increased the homogeneity of the clinical group, described as "vital for psychometric accuracy in specialized populations."

Finally, one source raises but does not resolve a meta-analytic artifact: funnel-plot asymmetry in meta-analyses using standardized mean differences might reflect baseline risk or sample size differences rather than publication bias, though the authors explicitly flag this as unconfirmed and note there is currently no reliable method to test for publication bias with this type of data (Bakker-Bons 2026).

Where the evidence splits

The clearest split is over the direction of small-sample bias. Akhavan-Abiri (2026) reports that small sample size (n = 45) produced smaller, more conservative convergent-validity correlations compared to larger-sample studies. Amrhein (2017) and Costello (2022), drawing on the broader psychology literature, report the opposite: small, early, low-power studies tend to produce inflated effect sizes that shrink under replication, with inflation driven by selection on statistical significance. The sources on this page do not reconcile this — none of them cross-reference the other's mechanism or explain why one small clinical sample yielded conservative estimates while the broader significance-filtering literature finds inflation. This split is unexplained by the sources on this page.

Thresholds and dose-response

At 40% statistical power, effects that cross the significance threshold are on average about 1.5 times larger than the true effect size; this inflation approaches zero as power approaches 100% (Amrhein 2017). Separately, among results significant at p < 0.00001 under standard point-form null-hypothesis testing, about 10 percentage points are not significant (psig > 0.05) once between-experiment variation is incorporated via a distributional null hypothesis test (Costello 2022).

Open empirical questions

• Whether the CHR restricted-variance mechanism for baseline risk factors generalizes beyond the specific studies summarized, and how it compares directly to register-based population cohorts.
• Whether small-sample anxiety studies in this literature actually operate at power levels near the ~40% figure used to model effect-size inflation.
• Whether the direction of small-sample bias (conservative underestimate vs. significance-driven inflation) holds consistently across diverse anxiety outcomes and clinical populations, or depends on study-specific factors not identified here.
• Whether cross-cultural or linguistic differences actually produced the one-factor structure found in the Akhavan-Abiri sample, versus the two-factor structures reported in other countries.