SciPost Submission Page
On weak ergodicity breaking in meanfield spin glasses
by Giampaolo Folena, Francesco Zamponi
This is not the latest submitted version.
This Submission thread is now published as
Submission summary
Authors (as registered SciPost users):  Giampaolo Folena 
Submission information  

Preprint Link:  https://arxiv.org/abs/2303.00026v1 (pdf) 
Date submitted:  20230303 23:30 
Submitted by:  Folena, Giampaolo 
Submitted to:  SciPost Physics 
Ontological classification  

Academic field:  Physics 
Specialties: 

Approach:  Theoretical 
Abstract
The weak ergodicity breaking hypothesis postulates that outofequilibrium glassy systems lose memory of their initial state despite being unable to reach an equilibrium stationary state. It is a milestone of glass physics, and has provided a lot of insight on the physical properties of glass aging. Despite its undoubted usefulness as a guiding principle, its general validity remains a subject of debate. Here, we present evidence that this hypothesis does not hold for a class of meanfield spin glass models. While most of the qualitative physical picture of aging remains unaffected, our results suggest that some important technical aspects should be revisited.
Current status:
Reports on this Submission
Report #3 by Anonymous (Referee 3) on 2023427 (Invited Report)
 Cite as: Anonymous, Report on arXiv:2303.00026v1, delivered 20230427, doi: 10.21468/SciPost.Report.7110
Report
The manuscript address the validity of scenario that has become part of the SpinGlass folklore in the last thirty years: the idea that offequilibrium dynamics in meanfield spinglasses with one step of Parisi's ReplicaSymmetryBreaking explores in a thermal fashion the most numerous marginal states. This idea came from observations on pure pspin models by Cugliandolo and Kurchan (CK) and is now challenged by recent work including notably this manuscript.
More precisely the authors address the topic of weak versus strong ergodicity breaking, namely if the correlation with initial condition is lost during aging. Based on their results they favor a strong ergodicity breaking scenario while correctly acknowledging that the time regime explored is limited and thus the question cannot be considered settled: "The possibility that the scenario we propose is only a preasymptotic regime that would crossover to a weak ergodicity regime thus remains open."
The manuscript discusses a number of features and interesting questions
in a very clear fashion and it is carefully written. It deserves publication
in its present form and I only have a few recommendation.
Quite simply the problem is wether the correlation computed at finite time extrapolate to zero or not at infinite time. If one uses powerlaws to extrapolate a finite value is obtained but there is no guarantee that the asymptotic behavior is described by a powerlaw and not by slower logarithmic decays. Two essential open problems are i) the lack of an analytic solution and ii) the lack of a numerical algorithm capable of reaching large times using a time grid with varying spacing.
Considering a different type of microscopic dynamics, the author show that large times dynamics seems to be independent of the short time details and
this indeed gives hope that an analytic solution can indeed be found and that some of the properties of the CK solution remains valid.
As for the second issue, the authors cite Ref. 60, where one such algorithm was used to reach times of order 10^7, and the reader may be puzzled by why they used a fixed spacing algorithm reaching times of order 10^3, a comment on this seems appropriate.
Ref. 22 is misquoted in the introduction as supporting strong ergodicity breaking but it actually takes an agnostic point of view pointing to the difficulties of extrapolating to large times and urging for an analytical solution. It would be interesting to plot the data of fig. 5 and fig. 6 parametrically and see what would be the asymptotic energy if weak ergodicity breaking was correct, as done in fig. 7 of Ref. 22.
After equation (16) and again in Section G it is mentioned that:" In the case of a quench to the critical temperature, exact relations between the αE and αC exponents were found in Ref. [54]." It is true that Ref. 54 is at present the only case beyond the p=2 spherical model where the asymptotic behavior of onetime quantities is computed analytically, nonetheless it should be stated that it deals with systems with continuous RSB transitions and not with the discontinuous 1RSB systems studied here.
Also, the comment of Dr. Theo Nieuwenhuizen should be taken into account.
Author: Giampaolo Folena on 20230509 [id 3659]
(in reply to Report 3 on 20230427)
Referee
The manuscript address the validity of scenario that has become part of the SpinGlass folklore in the last thirty years: the idea that offequilibrium dynamics in meanfield spinglasses with one step of Parisi's ReplicaSymmetryBreaking explores in a thermal fashion the most numerous marginal states. This idea came from observations on pure pspin models by Cugliandolo and Kurchan (CK) and is now challenged by recent work including notably this manuscript.
More precisely the authors address the topic of weak versus strong ergodicity breaking, namely if the correlation with initial condition is lost during aging. Based on their results they favor a strong ergodicity breaking scenario while correctly acknowledging that the time regime explored is limited and thus the question cannot be considered settled: "The possibility that the scenario we propose is only a preasymptotic regime that would crossover to a weak ergodicity regime thus remains open."
The manuscript discusses a number of features and interesting questions in a very clear fashion and it is carefully written. It deserves publication in its present form and I only have a few recommendation.
Response We thank the reviewer for the positive report. We have modified the paper according to the provided recommendations.
Referee
Quite simply the problem is wether the correlation computed at finite time extrapolate to zero or not at infinite time. If one uses powerlaws to extrapolate a finite value is obtained but there is no guarantee that the asymptotic behavior is described by a powerlaw and not by slower logarithmic decays. Two essential open problems are i) the lack of an analytic solution and ii) the lack of a numerical algorithm capable of reaching large times using a time grid with varying spacing. Considering a different type of microscopic dynamics, the author show that large times dynamics seems to be independent of the short time details and this indeed gives hope that an analytic solution can indeed be found and that some of the properties of the CK solution remains valid.
As for the second issue, the authors cite Ref. 60, where one such algorithm was used to reach times of order 10^7, and the reader may be puzzled by why they used a fixed spacing algorithm reaching times of order 10^3, a comment on this seems appropriate.
Response We added a footnote to comment about this point, as we believe that the algorithm of Ref.60 is not reliable at long times.
Referee
Ref. 22 is misquoted in the introduction as supporting strong ergodicity breaking but it actually takes an agnostic point of view pointing to the difficulties of extrapolating to large times and urging for an analytical solution. It would be interesting to plot the data of fig. 5 and fig. 6 parametrically and see what would be the asymptotic energy if weak ergodicity breaking was correct, as done in fig. 7 of Ref. 22.
Response We thank the referee for pointing out this misquotation. We rephrased the sentence.
We have tried the suggested procedure, i.e. plotting the excess energy versus C(t,0) and performing a 3parameters fit (see attached figure CorrEn.pdf: (a) for 2spin and (b) for 3spin). However, the results remain more consistent with the strong ergodicity breaking scenario, without adding further insights. We thus prefer to not add them to the paper, in order to avoid overcharging it.
Referee
After equation (16) and again in Section G it is mentioned that:" In the case of a quench to the critical temperature, exact relations between the αE and αC exponents were found in Ref. [54]." It is true that Ref. 54 is at present the only case beyond the p=2 spherical model where the asymptotic behavior of onetime quantities is computed analytically, nonetheless it should be stated that it deals with systems with continuous RSB transitions and not with the discontinuous 1RSB systems studied here.
Response We have rephrased the two phrases that reference [54] in order to clarify that the considered transition is of continous type.
Referee
Also, the comment of Dr. Theo Nieuwenhuizen should be taken into account.
Response We have added the suggested reference.
Attachment:
Report #1 by Anonymous (Referee 1) on 2023426 (Invited Report)
 Cite as: Anonymous, Report on arXiv:2303.00026v1, delivered 20230426, doi: 10.21468/SciPost.Report.7105
Report
The authors analyze mixed spherical random (p+s)spin glass models (p=2 and 3) undergoing gradient descent dynamics from random initial condition. Numerical integration of the DMFT equations suggests that the weak ergodicity breaking hypothesis does not hold in the mixture models under consideration, at variance with the pure pspin case. This observation is confirmed by a time series expansion of the overlap with the initial condition C(t,0), reaching a nonzero asymptotic value. A similar expansion of the radial reaction shows that the dynamics approaches a marginally stable minimum, while the asymptotic energy remains higher than the threshold value at which typical minima become marginal.
Overall, these results suggest that strong ergodicity breaking is verified in mixed (p+s)spin models at any s>p and even from infinite initial temperature, in contradiction with the result of ref. [25] where strong ergodicity breaking was found only below a finite initial onset temperature. The dynamics appears to find an aging state confined to an initializationdependent manifold sampled in an effectively thermal way, as shown by the presence of an effective FDR.
This work inscribes in the timely effort to understand and elucidate a longstanding picture of the outofequilibrium dynamics of prototypical meanfield models of the glass transition. The previous literature appears to be cited correctly. The paper is of good technical quality and the analytical derivations are well explained and relatively easy to follow. The presentation is generally good, although some aspects could be improved or clarified (see comments below). For these reasons, I recommend publication of the manuscript in SciPost provided the comments below are taken into account.
Comments and questions:
 The analytic derivations are written having an expert reader in mind and frequently referring to other works, sometimes to the detriment of a selfcontained presentation. E.g., the “overlap” and the “characteristic polynomial” are introduced on page 3 without definition, and similarly the “complexity” on page 4.
 The coefficients of the mixture seem to play an important role in discriminating between different asymptotic regimes. It would be useful to clarify how sensitive the discrepancy observed, e.g., in Fig. 2 is to the choice of the coefficient \lambda. It would be useful to report the values of the coefficients used in ref. [25] for comparison.
 It would be interesting if the authors could comment on the implications of their findings for optimization algorithms in planted models, and in relation to the expected discrepancy in performance (if any) between random and “smart” initializations. E.g., should the picture presented in [a,b] be revisited (in particular regarding the (2+4) planted model)? See references below.
 Ref. [25] supports the finding of a finite onset temperature referring to numerical experiments of realistic glassforming liquids, where this threshold is observed. How do the authors reconcile this experimental observation with their findings? Does the experimental T_onset show any finitesize dependence? Moreover, according to the phase diagram in Fig. 1(b) of this paper, the 1RSB dynamical ansatz used in ref. [25] to derive T_onset=0.91 for the (3+4)spin model does not seem unreasonable, in addition to the good agreement with numerical extrapolation. A clarification on why the asymptotic approximation used in [25] is wrong and T_onset instead should be set to infinity would be really appreciated.
 The DMFT equations (14) starting from random initialization do not exhibit any explicit dependence on the initial condition via C(t,0). From the presentation in section C, it is not intuitive to me why this dependence should appear in the asymptotic dynamics and I would appreciate a clarification on this point. Moreover, at first one may wonder why the CK ansatz does not apply in this case. I think this point is further clarified in section F, however it could be hard for nonexpert readers to connect the dots between sections.
[a] Mannelli, S. S., Biroli, G., Cammarota, C., Krzakala, F., Urbani, P., & Zdeborová, L. (2020). Marvels and pitfalls of the langevin algorithm in noisy highdimensional inference. Physical Review X, 10(1), 011057.
[b] Sarao Mannelli, S., Biroli, G., Cammarota, C., Krzakala, F., & Zdeborová, L. (2019). Who is afraid of big bad minima? analysis of gradientflow in spiked matrixtensor models. Advances in Neural Information Processing Systems, 32.
Author: Giampaolo Folena on 20230509 [id 3657]
(in reply to Report 1 on 20230426)
Referee
The authors analyze mixed spherical random (p+s)spin glass models (p=2 and 3) undergoing gradient descent dynamics from random initial condition. Numerical integration of the DMFT equations suggests that the weak ergodicity breaking hypothesis does not hold in the mixture models under consideration, at variance with the pure pspin case. This observation is confirmed by a time series expansion of the overlap with the initial condition C(t,0), reaching a nonzero asymptotic value. A similar expansion of the radial reaction shows that the dynamics approaches a marginally stable minimum, while the asymptotic energy remains higher than the threshold value at which typical minima become marginal. Overall, these results suggest that strong ergodicity breaking is verified in mixed (p+s)spin models at any s>p and even from infinite initial temperature, in contradiction with the result of ref. [25] where strong ergodicity breaking was found only below a finite initial onset temperature. The dynamics appears to find an aging state confined to an initializationdependent manifold sampled in an effectively thermal way, as shown by the presence of an effective FDR.
This work inscribes in the timely effort to understand and elucidate a longstanding picture of the outofequilibrium dynamics of prototypical meanfield models of the glass transition. The previous literature appears to be cited correctly. The paper is of good technical quality and the analytical derivations are well explained and relatively easy to follow. The presentation is generally good, although some aspects could be improved or clarified (see comments below). For these reasons, I recommend publication of the manuscript in SciPost provided the comments below are taken into account.
Response We thank the reviewer for the positive feedback. We have carefully taken into account all provided comments as reported below.
Referee
Comments and questions:
 The analytic derivations are written having an expert reader in mind and frequently referring to other works, sometimes to the detriment of a selfcontained presentation. E.g., the “overlap” and the “characteristic polynomial” are introduced on page 3 without definition, and similarly the “complexity” on page 4.
Response We have modified the text to properly introduce the “overlap”, the “characteristic polynomial” and the “complexity”.
Referee
 The coefficients of the mixture seem to play an important role in discriminating between different asymptotic regimes. It would be useful to clarify how sensitive the discrepancy observed, e.g., in Fig. 2 is to the choice of the coefficient \lambda. It would be useful to report the values of the coefficients used in ref. [25] for comparison.
Response We added a footnote to reiterate on the fact that $\lambda$ is chosen in such a way to maximize the discrepancy, and thus the results of ref. [25] shows an even smaller discrepacy.
Referee
 It would be interesting if the authors could comment on the implications of their findings for optimization algorithms in planted models, and in relation to the expected discrepancy in performance (if any) between random and “smart” initializations. E.g., should the picture presented in [a,b] be revisited (in particular regarding the (2+4) planted model)? See references below.
Response Unfortunately, at present we do not have any clear implication of our results on the suggested optmization questions. However, we thank the referee for the suggestion and we will think about this interesting question.
Referee
 Ref. [25] supports the finding of a finite onset temperature referring to numerical experiments of realistic glassforming liquids, where this threshold is observed. How do the authors reconcile this experimental observation with their findings? Does the experimental T_onset show any finitesize dependence? Moreover, according to the phase diagram in Fig. 1(b) of this paper, the 1RSB dynamical ansatz used in ref. [25] to derive T_onset=0.91 for the (3+4)spin model does not seem unreasonable, in addition to the good agreement with numerical extrapolation. A clarification on why the asymptotic approximation used in [25] is wrong and T_onset instead should be set to infinity would be really appreciated.
Response T_onset is experimentally a crossover temperature difficult to sharply characterize. In Ref.[25] a sharp T_onset is found because the introduced semiphenomenological closure of DMFT equations predicts a sharp transition. However, it is a 'good' approximation for the asymptotic dynamics and not an exact result. This approximation is based on the relative closeness of (3+4)model with pure models. However when the two terms of the mixture (p+s) become more different this approximation gets worse. We have added a phrase to clarify this issue. We feel that numerically, distinguishing between a sharp transition at a finite T_onset or a smoother crossover would be impossible.
Referee
 The DMFT equations (14) starting from random initialization do not exhibit any explicit dependence on the initial condition via C(t,0). From the presentation in section C, it is not intuitive to me why this dependence should appear in the asymptotic dynamics and I would appreciate a clarification on this point. Moreover, at first one may wonder why the CK ansatz does not apply in this case. I think this point is further clarified in section F, however it could be hard for nonexpert readers to connect the dots between sections.
Response We do not have any intuition about why the asymptotic dynamics keeps memory of the initial condition without the direct presence of any explicit correlation term C(t,0). It is one of the major questions raised by this paper, and we hope to have risen the attention of the interested audience on this open problem.
[a] Mannelli, S. S., Biroli, G., Cammarota, C., Krzakala, F., Urbani, P., & Zdeborová, L. (2020). Marvels and pitfalls of the langevin algorithm in noisy highdimensional inference. Physical Review X, 10(1), 011057.
[b] Sarao Mannelli, S., Biroli, G., Cammarota, C., Krzakala, F., & Zdeborová, L. (2019). Who is afraid of big bad minima? analysis of gradientflow in spiked matrixtensor models. Advances in Neural Information Processing Systems, 32.
Theo Nieuwenhuizen on 20230405 [id 3550]
The combination of pair and multiparticle spin glass interactions, including a more general definition of the function f(q), was, to my knowledge, introduced in: Exactly solvable model of a quantum spin glass, PRL 74 4289 (1995).
Anonymous on 20230509 [id 3655]
(in reply to Theo Nieuwenhuizen on 20230405 [id 3550])We completely agree. We have added a reference to the mentioned article.