Automatic analysis of microscopy images using the DLgram01 cloud service
A.V. Matveev1, M.Y. Mashukov1, A.V. Nartova1,2, N.N. Sankova2,1, A.G. Okunev1,2
1 Novosibirsk State University
2 Boreskov Institute of Catalysis SB RAS
Abstract: The study of materials by microscopy often includes counting the number of observed objects and determining their statistical parameters, for which it is necessary to measure hundreds of objects. The created DLgram01 cloud service allows specialists in the field of materials science who do not have programming skills to perform automated image processing – to determine the number and parameters (area, size) of the objects under study. The service is developed using the latest achievements in the field of deep machine learning. To train a neural network, the user needs to label only several objects. The neural network is trained automatically in a few minutes. Important features of the DLgram01 service are the ability to adjust the results of neural network prediction, as well as obtaining detailed information about all recognized objects. Using the service allows to significantly decrease the time for quantitative image analysis, reduce the influence of the subjective factor, increase the accuracy of the analysis and its ergo-intensity.
Keywords: microscopy, recognition, nanoparticles, deep neural networks, artificial intelligence
- Andrey V. Matveev – Ph. D., Head of the Laboratory of Deep Machine Learning in Physical Methods, Novosibirsk State University
- Mikhail Y. Mashukov – Senior Researcher, Laboratory of Deep Machine Learning in Physical Methods, Novosibirsk State University
- Anna V. Nartova – Ph. D., Docent, Department of Solid State Chemistry, Novosibirsk State University, Senior Researcher of the Laboratory of Surface Science Boreskov Institute of Catalysis SB RAS
- Natalia N. Sankova – Junior Researcher, Department of Nontraditional Catalytic Processes, Boreskov Institute of Catalysis SB RAS, Junior Researcher, Laboratory of Intelligent and Additive Methods of Materials Synthesis Novosibirsk State University
- Alexey G. Okunev – PhD, Director of the Higher College of Informatics, Novosibirsk State University, Senior Researcher, Template Synthesis Group Boreskov Institute of Catalysis SB RAS
Matveev, A.V. Automatic analysis of microscopy images using the DLgram01 cloud service / A.V. Matveev, M.Y. Mashukov, A.V. Nartova, N.N. Sankova, A.G. Okunev // Physical and chemical aspects of the study of clusters, nanostructures and nanomaterials. — 2021. — I. 13. — P. 300-311. DOI: 10.26456/pcascnn/2021.13.300. (In Russian).
Full article (in Russian): download PDF file
1. Horcas I., Fernández R., Gómez-Rodriguez J.M. et al. WSXM: a software for scanning probe microscopy and a tool for nanotechnology, Review of Scientific Instruments, 2002, vol. 78, issue 1, pp. 013705-1-013705-8. DOI: 10.1063/1.2432410.
2. Nečas D, Klapetek P. Gwyddion: an open-source software for SPM data analysis, Central European Journal of Physics, 2012, vol. 10, issue 1, pp. 181-188. DOI: 10.2478/s11534-011-0096-2.
3. Schindelin J., Arganda-Carreras I., Frise E. et al. Fiji: an open-source platform for biological-image analysis, Nature Methods, 2012, vol. 9, issue 7, pp. 676-682. DOI: 10.1038/nmeth.2019.
4. Moen E., Bannon D., Kudo T., Graf W., Covert M., Van Valen D. Deep learning for cellular image analysis, Nature Methods, 2019, vol. 16, issue 12, pp. 1233-1246. DOI: 10.1038/s41592-019-0403-1.
5. Caicedo J., Goodman A., Karhohs K. et al. Nucleus segmentation across imaging experiments: the 2018 Data Science Bowl, Nature Methods, 2019, vol. 16, issue 12, pp. 1247-1253. DOI: 10.1038/s41592-019-0612-7.
6. Yi J., Wu P., Hoeppner D.J., Metaxas D. Pixel-wise neural cell instance segmentation, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), 4-7 April 2018, Washington, DC, USA, 2018, pp. 373-377. DOI: 10.1109/ISBI.2018.8363596.
7. Stringer C., Michaelos M., Pachitariu M. Cellpose: a generalist algorithm for cellular segmentation, Nature Methods, 2021, vol. 18, issue 1, pp. 100-106. DOI: 10.1038/s41592-020-01018-x.
8. Bannon D., Moen E., Schwartz M. et al. DeepCell Kiosk: scaling deep learning-enabled cellular image analysis with Kubernetes, Nat Methods, 2021, vol. 18, issue 1, pp. 43-45. DOI: 10.1038/s41592-020-01023-0.
9. Fu G., Sun P., Zhu W. et al. Deep-learning-based approach for fast and robust steel surface defects classification, Optics and Lasers in Engineering, 2019, vol. 121, pp. 397-405. DOI: 10.1016/j.optlaseng.2019.05.005.
10. Zhu H., Ge W., Liu Z. Deep learning-based classification of weld surface defects, Applied Sciences, 2019, vol. 9, issue 16, art. no 3312, 10 p. DOI: 10.3390/app9163312
11. Liu Y., Xu K., Xu J. Periodic surface defect detection in steel plates based on deep learning, Applied Sciences, 2019, vol. 9, issue 15, art. no. 3127, 14 p. DOI: 10.3390/app9153127.
12. Feng S., Zhou H., Dong, H. Using deep neural network with small dataset to predict material defects, Materials & Design, 2019, vol. 162, pp. 300-310. DOI: 10.1016/j.matdes.2018.11.060.
13. Modarres M.H., Aversa R., Cozzini S. et al. Neural network for nanoscience scanning electron microscope image recognition, Scientific Reports, 2017, vol. 7, art. no 13282, 12 p.. DOI: 10.1038/s41598-017-13565-z.
14. Oktay A.B., Gurses A. Automatic detection, localization and segmentation of nano-particles with deep learning in microscopy images, Micron, 2019, vol. 120, pp. 113-119. DOI: 10.1016/j.micron.2019.02.009.
15. Zhang F., Zhang Q., Xiao Z., Wu J., Liu Y. Spherical nanoparticle parameter measurement method based on Mask R-CNN segmentation and edge fitting, Proceedings of the 8th International Conference on Computing and Pattern Recognition (ICCPR’19), 23-25 October 2019, Beijing, China. New York, Association for Computing Machinery, 2019, pp. 205-212. DOI: 10.1145/3373509.3373590.
16. Okunev A.G., Nartova A.V., Matveev A.V., Recognition of nanoparticles on scanning probe microscopy images using computer vision and deep machine learning, Proceedings of the International Multi-Conference on Engineering, Computer and Information Sciences (SIBIRCON), 21-27 October 2019, Novosibirsk, Russia, Piscataway, NJ, IEEE Publishing, 2019, pp. 0940-0943. DOI: 10.1109/SIBIRCON48586.2019.8958363.
17. Okunev A.G., Mashukov M.Y., Nartova A.V., Matveev A.V. Nanoparticle recognition on scanning probe microscopy images using computer vision and deep learning, Nanomaterials, 2020, vol. 10, issue 7, art. no. 1285, 16 p. DOI: 10.3390/nano10071285.
18. Boiko D.A., Pentsak E.O., Cherepanova V.A., Gordeev E.G., Ananikov V.P. Deep neural network analysis of nanoparticle ordering to identify defects in layered carbon materials, Chemical Science, 2021, vol. 12, issue 21, pp. 7428-7441. DOI: 10.1039/D0SC05696K.
19. Davletshin A., Ko L.T., Milliken K. et al. Detection of framboidal pyrite size distributions using convolutional neural networks, Marine and Petroleum Geology, 2021, vol. 132, art. no 105159, 24 p. DOI: 10.1016/j.marpetgeo.2021.105159.
20. Monchot P., Coquelin L., Guerroudj K. et al. Deep learning based instance segmentation of titanium dioxide particles in the form of agglomerates in scanning electron microscopy, Nanomaterials, 2021, vol. 11, issue 4, art. № 968, 17 p. DOI: 10.3390/nano11040968.
21. Wada K. Labelme: image polygonal annotation with python. – Access mode: www.url: https://lammps.sandia.gov. – 17.08.2021.
22. Telegram channel Nanoparticles. – Access mode: www.url: https://t.me/nanoparticles_nsk. – 17.08.2021.
23. Liz M.F., Nartova A.V., Matveev A.V., Okunev A.G., Using computer vision and deep learning for nanoparticle recognition on scanning probe microscopy images: modified U-net approach, Proceedings of the 2020 Science and Artificial Intelligence conference (S.A.I.ence), 14-15 November 2020, Novosibirsk, Russia, Piscataway, NJ, IEEE Publishing, 2020, pp. 13-16. DOI: 10.1109/S.A.I.ence50533.2020.9303184.