Neural-Network-Based Methods for ARPES Data Processing (Review Article)
DOI:
https://doi.org/10.15407/ujpe69.1.53Keywords:
ARPES, convolutional neural network, machine learningAbstract
In recent years, many developed upgrades of angle-resolved photoemission spectroscopy (ARPES) have significantly increased the amount of the obtained data. In this article, we briefly review the methods of processing of ARPES spectra with the use of convolutional neural networks (CNNs). In addition, we have made a short checkup of the potential application of CNNs that outperforms the existing methods or gives the possibility to achieve previously unachievable results.
References
A.A. Kordyuk. ARPES experiment in fermiology of quasi-2D metals (Review Article). Low Temp. Phys. 40, 286 (2014).
https://doi.org/10.1063/1.4871745
A. Damascelli, Z. Hussain, Z.-X. Shen. Angle-resolved photoemission studies of the cuprate superconductors. Rev. Mod. Phys. 75, 473 (2003).
https://doi.org/10.1103/RevModPhys.75.473
J.A. Sobota, Yu He, Z.-X. Shen. Angle-resolved photoemission studies of quantum materials. Rev. Mod. Phys. 93, 025006 (2021).
https://doi.org/10.1103/RevModPhys.93.025006
Shi Dong, Ping Wang, Khushnood Abbas. A survey on deep learning and its applications. Comp. Sci. Rev. 40, 100379 (2021).
https://doi.org/10.1016/j.cosrev.2021.100379
S. Pouyanfar, S. Sadiq, Y. Yan, H. Tian, Y. Tao, M. Presa Reyes, M.-L. Shyu, S.-C. Chen, S. Iyengar. A survey on deep learning: Algorithms, techniques, and applications. ACM Comput. Surv. 51, 1 (2018).
https://doi.org/10.1145/3234150
A. Mart, P. Barham, J. Chen, Z. Chen, A. Davis, J. Dean, M. Devin, S. Ghemawat, G. Irving, M. Isard, M. Kudlur, J. Levenberg, R. Monga, S. Moore, D. Gordon Murray et al. TensorFlow: A system for large-scale machine learning. arxiv.org/abs/1605.08695.
A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. K¨opf, E. Yang, Z. DeVito, Zach M. Raison et al. PyTorch: An imperative style, High-performance deep learning library. In: Proceedings of the 33rd International Conference on Neural Information Processing Systems, Vancouver, Canada, December 8-14, 2019.
C. Adam-Bourdarios, G. Cowan, C. Germain-Renaud, I. Guyon, B. K'egl, D. Rousseau. The higgs machine learning challenge. J. Phys. Conf. Ser. 664, 072015 (2015).
https://doi.org/10.1088/1742-6596/664/7/072015
J. Dean, M. Scheffler, A.R.P. Thomas, S.V. Barabash, R. Bhowmik, T. Bazhirov. The higgs machine learning challenge. arXiv: 2112.00239 [cond-mat.mtrl-sci].
P. Raccuglia, K.C. Elbert, P.D.F. Adler, C. Falk, M.B. Wenny, A. Mollo, M. Zeller, S.A. Friedler, J. Schrier, A.J. Norquist. Machine-learning-assisted materials discovery using failed experiments. Nature 533, 73 (2016).
https://doi.org/10.1038/nature17439
S.R. Xie, Y. Quan, A.C. Hire, B. Deng, J.M. DeStefano, I. Salinas, U.S. Shah, L. Fanfarillo, J. Lim, J. Kim, G.R. Stewart, J.J. Hamlin, P.J. Hirschfeld, R.G. Hennig. Machine learning of superconducting critical temperature from Eliashberg theory. Npj Comput. Mater. 8, 14 (2022).
https://doi.org/10.1038/s41524-021-00666-7
J. Schmidt, M.R.G. Marques, S. Botti, M.A.L. Marques. Recent advances and applications of machine learning in solid-state materials science. Npj Comput. Mater. 5, 1 (2019).
https://doi.org/10.1038/s41524-019-0221-0
R. Yamashita, M. Nishio, R.K.G. Do, K. Togashi. Convolutional neural networks: An overview and application in radiology. Insights into Imaging 9, 611 (2018).
https://doi.org/10.1007/s13244-018-0639-9
K. O'Shea, R. Nash. An introduction to convolutional neural networks. arXiv:1511.08458.[cs.NE].
I. Guyon, A. Elisseeff. Feature Extraction: Foundations and Applications. Edited by I. Guyon, M. Nikravesh, S. Gunn, L.A. Zadeh (Springer, 2013) [ISBN: 978-3-540-35487-1].
R.C. Gonzalez, R.E. Woods. Digital Image Processing 3rd edition (Prentice Hall, 2008) [ISBN: 978-0131687288].
P. Zhang, P. Richard, T. Qian,Y.-M. Xu,X. Dai, H.Ding. A precise method for visualizing dispersive features in image plots. Rev. Sci. Instrum. 82, 043712 (2011).
https://doi.org/10.1063/1.3585113
Yu He, Y. Wang, Z.-X. Shen.Visualizing dispersive features in 2D image via minimum gradient method. Rev. Sci. Instrum. 88, 073903 (2017).
https://doi.org/10.1063/1.4993919
H. Peng, X. Gao, Yu He, Y. Li, Y. Ji, Ch. Liu, S.A. Ekahana, D. Pei, Z. Liu, Z. Shen, Yu. Chen. Super resolution convolutional neural network for feature extraction in spectroscopic data. Rev. Sci. Instrum. 91, 033905 (2020).
https://doi.org/10.1063/1.5132586
Yo. Kim, D. Oh, S. Huh, D. Song, S. Jeong, Ju. Kwon, M. Kim, D. Kim, H. Ryu, J. Jung, W. Kyung, B. Sohn, S. Lee, J. Hyun, Ye. Lee et al. Deep learning-based statistical noise reduction for multidimensional spectral data. Rev. Sci. Instrum. 92, 073901 (2021).
https://doi.org/10.1063/5.0054920
F. Restrepo, Ju. Zhao, U. Chatterjee. Deep learning-based statistical noise reduction for multidimensional spectral data. Rev. Sci. Instrum. 93, 065106 (2022).
G.E. Hinton, R.R. Salakhutdinov. Reducing the dimensionality of data with neural networks. Science 331, 504 (2006).
https://doi.org/10.1126/science.1127647
P. Vincent, H. Larochelle, I. Lajoie, Y. Bengio, P.-A. Manzagol. Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion. J. Mach. Learn. Res. 11, 3371 (2010).
R. Atienza. Advanced Deep Learning with Keras: Apply Deep Learning Techniques, Autoencoders, GANs, Variational Autoencoders, Deep Reinforcement Learning, Policy Gradients, and More (Packt Publishing, 2018) [ISBN: 978-1788629416].
R. Atienza. Advanced Deep Learning with TensorFlow 2 and Keras: Apply DL, GANs, VAEs, Deep RL, Unsupervised Learning, Object Detection and Segmentation, and More (Packt Publishing, 2020) [ISBN: 978-1838825720].
D. Huang, J. Liu, T. Qian, Y.F. Yang. Spectroscopic data de-noising via training-set-free deep learning method. Sci. China: Phys. Mech. Astron. 66, 267011 (2023).
https://doi.org/10.1007/s11433-022-2075-x
J. Liu,D. Huang, Y.F. Yang, T. Qian. Removing grid structure in angle-resolved photoemission spectra via deep learning method. Phys. Rev. B 107, 165106 (2023).
https://doi.org/10.1103/PhysRevB.107.165106
Y. Huang, J. Xu, Y. Zhou, T. Tong, X. Zhuang. Diagnosis of Alzheimer's disease via multi-modality 3D convolutional neural network. Front. Neurosci. 13, (2019).
https://doi.org/10.3389/fnins.2019.00509
R. Atole, D. Park. A multiclass deep convolutional neural network classifier for detection of common rice plant anomalies. J. Adv. Comput. Sci. App. 9, (2018).
https://doi.org/10.14569/IJACSA.2018.090109
A. Aurisano, A. Radovic, D. Rocco, A. Himmel, M.D. Messier, E. Niner, G. Pawloski, F. Psihas, A. Sousa, P. Vahle. A convolutional neural network neutrino event classifier. J. Instrum. 11, P09001 (2016).
https://doi.org/10.1088/1748-0221/11/09/P09001
A.A. Kordyuk. Pseudogap from ARPES experiment: Three gaps in cuprates and topological superconductivity (Review Article). Low Temp. Phys. 41, 417 (2015).
https://doi.org/10.1063/1.4919371
T. Valla, A.V. Fedorov, P.D. Johnson, B.O. Wells, S.L. Hulbert, Q. Li, G.D. Gu, N. Koshizuka. Evidence for quantum critical behavior in the optimally doped cuprate. Science 285, 2110 (1999).
https://doi.org/10.1126/science.285.5436.2110
Z.-X. Shen, J. R. Schrieffer. Momentum, temperature, and doping dependence of photoemission lineshape and implications for the nature of the pairing potential in high-Tc superconducting materials. Phys. Rev. Lett. 78, 1771 (1997).
https://doi.org/10.1103/PhysRevLett.78.1771
P.A. Casey, J.D. Koralek, N.C. Plumb, D.S. Dessau, P.W. Anderson. Accurate theoretical fits to laser-excited photoemission spectra in the normal phase of high-temperature superconductors. Nat. Phys. 4, 210 (2008).
https://doi.org/10.1038/nphys833
K. Weiss, T,M. Khoshgoftaar, D. Wang. A survey of transfer learning. J. Big. Data 3, 9 (2016).
https://doi.org/10.1186/s40537-016-0043-6
A.F. Kemper, M.A. Sentef, B. Moritz, T.P. Devereaux, J.K. Freericks. Review of the teoretical description of time-resolved angle-resolved photoemission spectroscopy in electron-phonon mediated superconductors. Ann. Phys. 529, 1600235 (2017).
https://doi.org/10.1002/andp.201600235
W. Zhang, C.L. Smallwood, C. Jozwiak, T.L. Miller, Y. Yoshida, H. Eisaki, D.-H. Lee, A. Lanzara. Signatures of superconductivity and pseudogap formation in nonequilibrium nodal quasiparticles revealed by ultrafast angleresolved photoemission. Phys. Rev. B 88, 245132 (2013).
https://doi.org/10.1103/PhysRevB.88.245132
C.L. Smallwood, W. Zhang, T.L. Miller, C. Jozwiak, C.H. Eisaki, D.-H. Lee, A. Lanzara.Time- and momentumresolved gap dynamics in Bi2Sr2CaCu2O8+δ . Phys. Rev. B 89, 115126 (2014).
R. Cort'es, L. Rettig, Y. Yoshida, H. Eisaki, M. Wolf, U. Bovensiepen. Momentum-resolved ultrafast electron dynamics in superconducting Bi2Sr2CaCu2O8+δ. Phys. Rev. Lett. 107, 097002 (2011).
https://doi.org/10.1103/PhysRevLett.107.097002
E. Apostolidis, E. Adamantidou, A.I. Metsai, V. Mezaris, I. Patras. Video summarization using deep neural networks: A survey. arXiv:2101.06072.[cs.CV].
J.Y.-H. Ng, M. Hausknecht, S. Vijayanarasimhan, O. Vinyals, R. Monga, G. Toderici. Beyond short snippets: Deep networks for video classification. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA, June 7-12, 2015.
C. Orozco, E. Xamena, M. Buemi, J. Berlles. Human action recognition in videos using a robust CNN LSTM approach. Ciencia y Tecnolog'ia 21 (2020).
https://doi.org/10.18682/cyt.vi0.3288
A. Agga, A. Abbou, M. Labbadi, Y. El Houm, I. Hammou Ou Ali. CNN-LSTM: An efficient hybrid deep learning architecture for predicting short-term photovoltaic power production. Electric Power Systems Research 208, 107908 (2022).
https://doi.org/10.1016/j.epsr.2022.107908
V. Stanev, C. Oses, A.G. Kusne, E. Rodriguez, J. Paglione, S. Curtarolo, I. Takeuchi. Machine learning modeling of superconducting critical temperature. Npj Comput. Mater. 4, 29 (2018).
https://doi.org/10.1038/s41524-018-0085-8
A.A. Kordyuk. Electronic band structure of optimal superconductors: From cuprates to ferropnictides and back again. Low Temp. Phys. 44, 477 (2018).
https://doi.org/10.1063/1.5037550
Yu.V. Pustovit, A.A. Kordyuk. Metamorphoses of electronic structure of FeSe-based superconductors (Review article). Low Temp. Phys. 42, 995 (2016).
https://doi.org/10.1063/1.4969896
S. Borisenko, A. Fedorov, A. Kuibarov, M. Bianchi, V. Bezguba, P. Majchrzak, P. Hofmann, P. Baumg¨artel, V. Voroshnin, Y. Kushnirenko, J. S'anchez-Barriga, A. Varykhalov, R. Ovsyannikov, I. Morozov, S. Aswartham. Fermi surface tomography. Nat. Commun. 13, 4132 (2022).
https://doi.org/10.1038/s41467-022-31841-z
S. Kirklin, J.E. Saal, B. Meredig, A. Thompson, J.W. Doak, M. Aykol, S. R¨uhl, C. Wolverton. The open quantum materials database (OQMD): Assessing the accuracy of DFT formation energies. Npj Comput. Mater. 1, 15010 (2015).
https://doi.org/10.1038/npjcompumats.2015.10
A. Wang, Y. Wang, Y. Chen. Hyperspectral image classification based on convolutional neural network and random forest. Remote Sens. Lett. 10, 1086 (2019).
https://doi.org/10.1080/2150704X.2019.1649736
G. Xu, M. Liu, Z. Jiang, D. S¨offker, W. Shen. Bearing fault diagnosis method based on deep convolutional neural network and random forest ensemble learning. Sensors 19, 1088 (2019).
https://doi.org/10.3390/s19051088
F.-L. Fan, J. Xiong, M. Li, G. Wang. On interpretability of artificial neural networks: A survey. IEEE trans. Radiat. Plasma Med. Sci. 5, 741 (2020).
https://doi.org/10.1109/TRPMS.2021.3066428
Y. Zhang, P. Tiˇno, A. Leonardis, K. Tang. A survey on neural network interpretability. IEEE Trans. Emerg. Top. Comput. Intell. 5, 726 (2021).
https://doi.org/10.1109/TETCI.2021.3100641
Q. Zhang, Y.N. Wu, C.-H. Zhu. Interpretable convolutional neural networks. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, June 18-23, 2018.
https://doi.org/10.1109/CVPR.2018.00920
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A.N. Gomez, L. Kaiser, I. Polosukhin. Attention is all you need. In: NIPS'17: Proceedings of the 31st International Conference on Neural Information Processing Systems Long Beach, California, USA December 4-9, 2017.
Chunwei Tian, Yong Xu, Zuoyong Li, Wangmeng Zuo, Lunke Fei, Hong Liu. Attention-guided CNN for image denoising. Neural Netw. 124, 117 (2020).
https://doi.org/10.1016/j.neunet.2019.12.024
Z. Baozhou, P. Hofstee, J. Lee, Z. Al-Ars. An attention module for convolutional neural networks. arXiv: 2108.08205.[cs.CV].
M.-I. Georgescu, R.T. Ionescu, Andreea-Iuliana Miron, Olivian Savencu, Nicolae-Catalin Ristea, Nicolae Verga, Fahad Shahbaz Khan. Multimodal multi-head convolutional attention with various kernel sizes for medical image super-resolution. arXiv:2204.04218.[eess.IV].
A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, N. Houlsby. An image is worth 16×16 words: Transformers for image recognition at scale. arXiv:2010.11929.[cs.CV].
J. Devlin, M.-W. Chang, K. Lee, K. Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. arXiv:1810.04805.[cs.CL].
C. Li, C. Zhang. Toward a Deeper understanding: RetNet viewed through convolution. arXiv:2309.05375.[cs.CL].
O. Uparkar, J. Bharti, R.K. Pateriya, R. Kumar Gupta, A. Sharma. Vision transformer outperforms deep convolutional neural network-based model in classifying X-ray images. Procedia Comput. Sci. 218, 2338 (2023).
Downloads
Published
How to Cite
Issue
Section
License
Copyright Agreement
License to Publish the Paper
Kyiv, Ukraine
The corresponding author and the co-authors (hereon referred to as the Author(s)) of the paper being submitted to the Ukrainian Journal of Physics (hereon referred to as the Paper) from one side and the Bogolyubov Institute for Theoretical Physics, National Academy of Sciences of Ukraine, represented by its Director (hereon referred to as the Publisher) from the other side have come to the following Agreement:
1. Subject of the Agreement.
The Author(s) grant(s) the Publisher the free non-exclusive right to use the Paper (of scientific, technical, or any other content) according to the terms and conditions defined by this Agreement.
2. The ways of using the Paper.
2.1. The Author(s) grant(s) the Publisher the right to use the Paper as follows.
2.1.1. To publish the Paper in the Ukrainian Journal of Physics (hereon referred to as the Journal) in original language and translated into English (the copy of the Paper approved by the Author(s) and the Publisher and accepted for publication is a constitutive part of this License Agreement).
2.1.2. To edit, adapt, and correct the Paper by approval of the Author(s).
2.1.3. To translate the Paper in the case when the Paper is written in a language different from that adopted in the Journal.
2.2. If the Author(s) has(ve) an intent to use the Paper in any other way, e.g., to publish the translated version of the Paper (except for the case defined by Section 2.1.3 of this Agreement), to post the full Paper or any its part on the web, to publish the Paper in any other editions, to include the Paper or any its part in other collections, anthologies, encyclopaedias, etc., the Author(s) should get a written permission from the Publisher.
3. License territory.
The Author(s) grant(s) the Publisher the right to use the Paper as regulated by sections 2.1.1–2.1.3 of this Agreement on the territory of Ukraine and to distribute the Paper as indispensable part of the Journal on the territory of Ukraine and other countries by means of subscription, sales, and free transfer to a third party.
4. Duration.
4.1. This Agreement is valid starting from the date of signature and acts for the entire period of the existence of the Journal.
5. Loyalty.
5.1. The Author(s) warrant(s) the Publisher that:
– he/she is the true author (co-author) of the Paper;
– copyright on the Paper was not transferred to any other party;
– the Paper has never been published before and will not be published in any other media before it is published by the Publisher (see also section 2.2);
– the Author(s) do(es) not violate any intellectual property right of other parties. If the Paper includes some materials of other parties, except for citations whose length is regulated by the scientific, informational, or critical character of the Paper, the use of such materials is in compliance with the regulations of the international law and the law of Ukraine.
6. Requisites and signatures of the Parties.
Publisher: Bogolyubov Institute for Theoretical Physics, National Academy of Sciences of Ukraine.
Address: Ukraine, Kyiv, Metrolohichna Str. 14-b.
Author: Electronic signature on behalf and with endorsement of all co-authors.
.png){kind=small}
