No evidence that late-sighted individuals rely more on color for object recognition: A Bayesian generalized mixed effects model analysis

1. Reproducible analysis in R markdown format.
2. Clear and short paper.
3. Nice figure.

1. It is unclear why the Bayesian mixed-effects analysis should be preferred to the one reported in the original paper.
2. Code for Figure 1 in report not included in provided R code.
3. The conclusion talks about two different criticisms: The differences in results between t-test and Bayesian-GLMM and learning as a "competing causal explanation". However, the paper only focuses on the differences in analysis so the second issue does not follow from the presented analysis.

I think this is an interesting paper that provides an important addition to the Vogelsang et al. (2024) paper (which appeared in Science and was already cited 14 times). It also provides a clear and reasonable alternative analysis approach (GLMM) and shows that with this alternative approach a different results pattern occurs.

I was also able to reproduce their analysis and confirmed the results myself using a frequentist analysis which shows the same pattern of results (no interaction effect when using a logistic-binomial GLMM, but an interaction effect when using either repeated-measures ANOVA or a linear mixed-effects model assuming a normal response distribution; see: https://gist.github.com/singmann/cd196db4b796c38cfd14160c37c83e9c).

However, the paper currently does not do a good enough job arguing why the alternative analysis approach should be preferred to the original author's approach. What actually is wrong with the original author's approach (t-test)?

To be transparent, I do not think that using a binomial-logistic GLMM is necessarily better in this situation than a paired t-test as done by the original authors. Which analysis approach one ultimately prefers depends on which assumptions one thinks is more believable. For the binomial GLMM, one assumes that participants are normally distributed around their condition means in logistic space. For the t-test, one essentially assumes that participants are normally distributed around their condition means in accuracy space. Which assumption is correct is not an easy question.

Recently, I am becoming more convinced by analyses that do not require a data transformation (such as to logistic space) for accuracy data. One reason for this is a series of papers (e.g., Gomilla, 2021; Jaccard & Brinberg, 2021) arguing for the usage of models on a non-transformed probability space. In this context, models such as paired t-test or RM-ANOVA for accuracy data is also known as the linear probability model.

Taken together, in a revision the authors need to make absolutely clear that the evidence for the interaction hinges on whether one is willing to believe the linear probability model (as done by Vogelsang et al., 2024) or the binomial GLMM with logistic link function. In other words, there is some ambiguity towards the evidence for the interaction. Which side one falls on depends on which statistical model one believes provides a better account of the data generating process.

References

• Gomila, R. (2021). Logistic or linear? Estimating causal effects of experimental treatments on binary outcomes using regression analysis. Journal of Experimental Psychology: General, 150(4), 700–709. https://doi.org/10.1037/xge0000920
• Jaccard, J., & Brinberg, M. (2021). Monte Carlo simulations using extant data to mimic populations: Applications to the modified linear probability model and logistic regression. Psychological Methods, 26(4), 450–465. https://doi.org/10.1037/met0000383

1. Add code for Figure 1 to code repository.
2. Rewrite conclusion and clearly separate the discussion of statistical issues from issues regarding the experimental design (learning as a possible alternative explanation for the pattern).
3. Provide a more balanced discussion of the statistical differences between both analysis approaches. Why do the authors think a binomial GLMM is more justifiable here than the linear probability model employed by Vogelsang et al. (2024).

Ask for minor revision

1-the atuhors should clarify why they suggest using this alternative analysis approach over the original one 2- the atuhors should clarify what is the main reason for why this new analysis results in opposite conclusion compared to the original paper.

Ask for minor revision