Article Reading Assistant

As a researcher, your instinct to interrogate the sample size is correct. The logical chain typically runs: effect size assumption → required sample size → observed power → statistical significance → inferential claims. First, check whether the authors justify their effect size. If they use a small, clinically trivial effect to inflate feasibility, their later "significant" findings may be statistically real but practically meaningless. Second, verify the attrition rate. A meta-analysis that loses 20% of studies post-screening often weakens the pooled estimate, yet the conclusion may ignore this. Third, examine heterogeneity (I²). If it's high (>75%), the pooled effect is an average of disparate studies; any causal claim is shaky. Finally, trace whether the confidence intervals in the results section actually exclude the null for the primary outcome. If the authors report secondary outcomes as definitive while primary outcomes are borderline, that's a red flag. A critical check: does the sample size calculation match the primary analysis model (e.g., random vs. fixed effects)? If they calculated for fixed effects but reported random effects, the power is overstated. Your conclusion should mirror the precision of the data—not the ambition of the authors.

To avoid cherry-picking, evaluate the report's argument as a whole. Start by identifying the central claim (e.g., "retention improves with onboarding personalization") and then trace every data point they use to support it. Ask: Are the comparison groups matched on baseline characteristics? If they compare cohorts from different time periods without controlling for seasonality, the trend may be spurious. Next, check the denominator. Retention metrics are notoriously sensitive to how you define "active"—daily vs. monthly active users changes the numbers drastically. If the report uses one definition in the headline and another in the footnotes, that's a weak spot. Also, look for survivorship bias: if they only track users who haven't churned yet, the "improvement" may just reflect a shrinking base. Finally, examine the timeline. A 3-month trend is not a 12-month trend. If the report extrapolates long-term conclusions from short-term data, flag it. Your presentation should state the claim, the supporting evidence, and explicitly list the caveats—this demonstrates rigor, not weakness.

Fact-checking clinical trial numbers requires three steps: source verification, arithmetic, and logical fit. First, locate the original trial publications. If the article cites a relative risk reduction (e.g., "40% fewer events") without the absolute risk, that's a red flag—a 40% relative reduction could mean a drop from 5% to 3%, which is modest. Second, check the numbers against each other. If they report 100 patients, 20 events in the control group, and 12 in the treatment group, the relative reduction is 40%, but the absolute reduction is 8%. If the article presents only the relative figure, the conclusion "highly effective" is overstated. Third, verify the statistical significance. A p-value of 0.06 is not significant, yet many health articles report "trend toward benefit" as if it were proof. Finally, check the endpoint. If the trial measured a surrogate endpoint (e.g., blood pressure) but the article concludes a reduction in heart attacks, that's a logical leap. Your fact-check should state: the numbers are accurate as cited, but the interpretation overreaches the data. Cite the absolute risk and the confidence interval to ground your conclusion.