Volume 5 (2022)
Chen, Hongbin; Mourrat, Jean-Christophe; Xia, Jiaming
Statistical inference of finite-rank tensors
Annales Henri Lebesgue, Volume 5 (2022), pp. 1161-1189.

Metadata

DOI10.5802/ahl.146
Keywords inference problem, Hamilton–Jacobi equation, tensor

Abstract

We consider a general statistical inference model of finite-rank tensor products. For any interaction structure and any order of tensor products, we identify the limit free energy of the model in terms of a variational formula. Our approach consists of showing first that the limit free energy must be the viscosity solution to a certain Hamilton–Jacobi equation.


References

[ACCM21] Alberici, Diego; Camilli, Francesco; Contucci, Pierluigi; Mingione, Emanuele The solution of the deep Boltzmann machine on the Nishimori line, Commun. Math. Phys., Volume 387 (2021) no. 2, pp. 1191-1214 | DOI | MR | Zbl

[BDBG10] Barra, Adriano; Di Biasio, Aldo; Guerra, Francesco Replica symmetry breaking in mean-field spin glasses through the Hamilton–Jacobi technique, J. Stat. Mech. Theory Exp., Volume 2010 (2010) no. 09, P09006 | MR | Zbl

[BDFT13] Barra, Adriano; Dal Ferraro, Gino; Tantari, Daniele Mean field spin glasses treated with PDE techniques, Eur. Phys. J. B, Volume 86 (2013) no. 7, 332 | DOI | MR

[BDM + 16] Barbier, Jean; Dia, Mohamad; Macris, Nicolas; Krzakala, Florent; Lesieur, Thibault; Zdeborová, Lenka, Advances in Neural Information Processing Systems (NIPS) (NeurIPS Proceedings), Volume 29 (2016), pp. 424-432

[BM19a] Barbier, Jean; Macris, Nicolas The adaptive interpolation method: a simple scheme to prove replica formulas in Bayesian inference, Probab. Theory Relat. Fields, Volume 174 (2019) no. 3-4, pp. 1133-1185 | DOI | MR | Zbl

[BM19b] Barbier, Jean; Macris, Nicolas The adaptive interpolation method for proving replica formulas. Applications to the Curie–Weiss and Wigner spike models, J. Phys. A, Math. Theor., Volume 52 (2019) no. 29, 294002 | DOI | MR

[BMM17] Barbier, Jean; Macris, Nicolas; Miolane, Léo, 2017 55 th Annual Allerton Conference on Communication, Control, and Computing (Allerton) (2017), pp. 1056-1063 | DOI

[Che20] Chen, Hong-Bin Hamilton–Jacobi equations for nonsymmetric matrix inference (2020) (https://arxiv.org/abs/2006.05328)

[CX20] Chen, Hong-Bin; Xia, Jiaming Hamilton–Jacobi equations for inference of matrix tensor products (2020) (https://arxiv.org/abs/2009.01678)

[CX22] Chen, Hong-Bin; Xia, Jiaming Hamilton–Jacobi equations from mean-field spin glasses (2022) (https://arxiv.org/abs/2201.12732)

[GB09] Genovese, Giuseppe; Barra, Adriano A mechanical approach to mean field spin models, J. Math. Phys., Volume 50 (2009) no. 5, 053303 | MR | Zbl

[Gue01] Guerra, Francesco Sum rules for the free energy in the mean field spin glass model, Fields Institute Communications, Volume 30 (2001) no. 11 | MR | Zbl

[KG18] Kadmon, Jonathan; Ganguli, Surya, Advances in Neural Information Processing Systems (NeurIPS Proceedings) (2018), pp. 8201-8212

[LBM21] Luneau, Clément; Barbier, Jean; Macris, Nicolas Mutual information for low-rank even-order symmetric tensor estimation, Inf. Inference, Volume 10 (2021) no. 4, pp. 1167-1207 | DOI | MR | Zbl

[LM19] Lelarge, Marc; Miolane, Léo Fundamental limits of symmetric low-rank matrix estimation, Probab. Theory Relat. Fields, Volume 173 (2019) no. 3-4, pp. 859-929 | DOI | MR | Zbl

[LMB20] Luneau, Clément; Macris, Nicolas; Barbier, Jean High-dimensional rank-one nonsymmetric matrix decomposition: the spherical case (2020) (https://arxiv.org/abs/2004.06975)

[LML + 17] Lesieur, Thibault; Miolane, Léo; Lelarge, Marc; Krzakala, Florent; Zdeborová, Lenka, 2017 IEEE International Symposium on Information Theory (ISIT) (2017), pp. 511-515 | DOI

[Mio17] Miolane, Léo Fundamental limits of low-rank matrix estimation: the non-symmetric case (2017) (https://arxiv.org/abs/1702.00473)

[Mou20] Mourrat, Jean-Christophe Hamilton–Jacobi equations for finite-rank matrix inference, Ann. Appl. Probab., Volume 30 (2020) no. 5, pp. 2234-2260 | DOI | MR | Zbl

[Mou21a] Mourrat, Jean-Christophe Free energy upper bound for mean-field vector spin glasses (2021) (https://arxiv.org/abs/2010.09114)

[Mou21b] Mourrat, Jean-Christophe Hamilton–Jacobi equations for mean-field disordered systems, Ann. Henri Lebesgue, Volume 4 (2021), pp. 453-484 | DOI | MR | Zbl

[Mou21c] Mourrat, Jean-Christophe Nonconvex interactions in mean-field spin glasses, Probability and Mathematical Physics, Volume 2 (2021) no. 2, pp. 281-339 | DOI | MR | Zbl

[Mou22] Mourrat, Jean-Christophe The Parisi formula is a Hamilton–Jacobi equation in Wasserstein space, Can. J. Math., Volume 74 (2022) no. 3, pp. 607-629 | DOI | MR | Zbl

[MP20] Mourrat, Jean-Christophe; Panchenko, Dmitry Extending the Parisi formula along a Hamilton–Jacobi equation, Electron. J. Probab., Volume 25 (2020), 23 | DOI | MR | Zbl

[MR19] Mayya, Vaishakhi; Reeves, Galen, 2019 57 th Annual Allerton Conference on Communication, Control, and Computing (Allerton) (2019), pp. 602-607 | DOI

[Ree20] Reeves, Galen Information-Theoretic Limits for the Matrix Tensor Product (2020) (https://arxiv.org/abs/2005.11273)

[RMV19] Reeves, Galen; Mayya, Vaishakhi; Volfovsky, Alexander, 2019 IEEE International Symposium on Information Theory (ISIT) (2019), pp. 400-404 | DOI

[Roc70] Rockafellar, R Tyrrell Convex Analysis, Princeton Mathematical Series, 36, Princeton University Press, 1970 | DOI | Zbl

[Zha11] Zhang, Fuzhen Matrix theory, Universitext, Springer, 2011 | DOI | MR | Zbl