Plain Language Summary Effect Sizes on Comprehension

What the evidence says

Two separate experimental literatures speak to plain-language summary effects here. One measures comprehension directly with a test score (Chan 2026); the other measures self-reported reading ease, understanding, and information content across low- versus high-complexity summaries (August 2024); a third source reports readability metrics for AI-transformed medical documents without measuring comprehension directly (Hasani 2025, citing Zaretsky et al). Read together, they show consistent gains in ease-of-processing outcomes, but direct, objective comprehension is measured in only one study.

In the one study that scores comprehension directly, an interactive, personalized plain-language summary increased comprehension test scores compared to a static expert-written summary: 9.83 versus 9.03 on the comprehension measure, a difference of 0.80 points (95% CI [0.21, 1.39], p = .009). The same study found this interactive format also raised perceived quality across all nine rated dimensions by an average of 0.29 points (p < .001), with the largest gains in explanation (+0.45), importance (+0.38), and perceived tailoring (+0.35). But perceived quality did not track the comprehension gain: correlations between quality ratings and comprehension scores were all |r| ≤ .08 (p ≥ .59). This decouples "rated as better" from "understood better" within the same study.

The comprehension gain from personalization did not vary systematically by education, health literacy, age, or sex — no regression model reached significance (all F-test p > .09) and no predictor reached significance (all p ≥ .07), with one uncorrected exception: higher topic familiarity was associated with lower scores on the static summary (β = −0.85, p = .028, uncorrected). Gains were also uneven at the individual level: 27 of 50 participants improved under the interactive format while 13 declined.

Separately, AI-generated plain-language rewriting of medical documents decreased reading level from eleventh grade to sixth grade and increased an understandability score from 13% to 81%, without losing accuracy, though some errors occurred (Hasani 2025, citing Zaretsky et al). This is a readability-metric result, not a measured comprehension outcome.

The August 2024 studies test low- versus high-complexity summaries and consistently find that low-complexity (plain-language) versions increase self-reported reading ease and comprehension confidence for readers with low topic familiarity, replicated with both general-comparison and machine-generated summaries: reading ease increased by 0.893 points (general comparison) and 0.535 points (machine-generated), and comprehension confidence increased by 0.589 points, on 5-point scales. These effects were largest for the least-familiar readers. The same studies found, using automatic alignment metrics and manual annotation, that low-complexity summaries removed details and reported less information than high-complexity summaries — an information-content trade-off alongside the ease-of-reading gain.

Where the evidence splits

No contradictions in the current sources.

Thresholds and dose-response

The August 2024 studies establish a dose-response pattern along self-rated topic familiarity, measured on a 1–5 scale. At the lowest familiarity rating (1 on the scale), the reading-ease and comprehension-confidence benefits of low-complexity summaries are largest: reading ease increased by 1.490 points and comprehension confidence by 1.160 points (general comparison), and reading ease increased by 1.642 points in the machine-generated-summary replication. At the highest familiarity ratings (4 or 5 on the scale), this benefit disappears: readers rated high- and low-complexity summaries as similarly easy to read and understand, and equally interesting and valuable, and instead clicked on significantly more sections of the high-complexity summary than the low-complexity one — a Cohen's d of 0.68 (general comparison) and 0.35 (machine-generated replication), estimated at roughly 0.9 to 1 additional section skipped.

Open empirical questions

• Whether readability-metric gains (reading level, understandability score) correspond to measured comprehension gains, rather than just easier-looking text — flagged as unresolved even within the source reporting them.
• What mechanism explains why perceived quality ratings do not correlate with actual comprehension scores in the one study that measured both.
• Whether the information loss documented in unrestricted-generation low-complexity summaries reduces comprehension of the specific content that was omitted.
• Whether the comprehension and reading-ease effects measured in these controlled studies (N=50 in one case) replicate at larger scale or outside lab settings.