SciPost logo

SciPost Submission Page

Reanalyzing Castner et al. Confirms Most Conclusions, Reveals Additional Findings, and Detects Some Statistical Errors

by Olivia C. Robertson, Erik Parker, Beate Henschel, Anna L. M. Macagno, Daniel E. Kpormegbey, Deependra K. Thapa, David B. Allison

This is not the latest submitted version.

Submission summary

Authors (as registered SciPost users): Olivia Robertson
Submission information
Preprint Link: scipost_202508_00055v1  (pdf)
Date submitted: Aug. 22, 2025, 5:54 p.m.
Submitted by: Olivia Robertson
Submitted to: Journal of Robustness Reports
Ontological classification
Academic field: Multidisciplinary
Specialties:
  • Other
Approach: Computational

Abstract

Patterns of findings generally held with discrepancies in p-values and additional significant findings revealed after reanalyzing with correction of suboptimal statistical practices

Current status:
Has been resubmitted

Reports on this Submission

Report #2 by Anonymous (Referee 2) on 2025-10-22 (Invited Report)

Strengths

(1) The main strength of this report is a clear jargon-free language and concise to-the-point information presentation without unnecessary detours.
(2) The supplementary 'technical report' (Markdown file) is easy to navigate and highlights relevant differences in statistics properly.
(3) The statistical choices are sound alternatives for the ones published, and each decision is conceptually supported.

Weaknesses

(1) The main weakness of the report is a lack of detail that would help the reader to precisely see how the study's conclusions are supported and extended.
(2) The 'technical report' occasionally uses improper language (e.g., Figure 2. Extended 2. Point 2. [detailed in 'Requested changes']).
(3) Although the report fouceses on statistical robustness, the reader would appriciate brief reiterations of the original results to contextualize staitstical decisions and changes.

Report

The report re-analyzed Castner et al. (2023, Nature aging, 10.1038/S43587-023-00441-X) and concluded that the original analyses were sub-optimal and biased towards the null. They found previously uncovered differences, as well as lower p-values in statistical tests.

The report meets the Acceptance criteria of the journal and as a reanalysis fits the 'Journal of Robustness Reports'. The target article is of general interest, the analyses are methodologically sound and clearly reported with a constructive tone. The analysis is fully reproducible, the markdown files are easy to navigate and use and it also enhances open access.

I have minor questions about how the new analyses complement the original results, which could be aided by additional context specifically in the 'Results' section. In addition, some structural and language refinements would further strengthen the paper.

Requested changes

I request the following changes to improve clarity and extensiveness of the report:

(1) At the start of each paragraph in the 'Results' section I suggest to include a very brief reiteration of the original findings to give additional context on how the reanalyses modify these. For example, in the last paragraph, authors could highlight the conceptual neccessity of using a two-tailed t-test (e.g., lack of directional hypothesis).
(2) I suggest explicite statement of specifically what statistically significant differences were uncovered not detected before by the original authors. I also advise higlighting how these results complement the original findings.
(3) I recommend a spell-check and language adjustment of the 'technical report' markdown (e.g., I noticed an unclear sentence 'This figure is just averages of KL levels at each timepoint given (0, 4, 28, and 52).' [Figure 2. Extended data Figue 2. Point 2.])
(4) I suggest to review and correct formatting inconsistencies throughout the report. Specifically, in '1 Goal' the paragraph is indented relative to other sections of the report. Furthermore, in 'Acknowledgments and Disclosures' double linebreaks, single linebreaks and simple whitespaces are inconsistently used as subsection dividers.

Recommendation

Ask for minor revision

  • validity: high
  • significance: good
  • originality: good
  • clarity: high
  • formatting: reasonable
  • grammar: excellent

Report #1 by Anonymous (Referee 1) on 2025-10-15 (Invited Report)

Strengths

The report is well done. The analysis code is well documented and the differences between the original analysis and the re-analysis is fairly described.

Weaknesses

I did not find any weeknesses.

Report

I believe that the journal acceptance criteria are met.

Requested changes

I did not find anything that would require changes.

Recommendation

Publish (easily meets expectations and criteria for this Journal; among top 50%)

  • validity: -
  • significance: -
  • originality: -
  • clarity: -
  • formatting: -
  • grammar: -

Login to report or comment