Novel neuromorphic architectures based on crossbar arrays of (Co-Fe-B)x(LiNbO3)100−x nanocomposite memristors
- Authors: Emelyanov A.V.1, Matsukatova A.N.2,3, Iliasov A.I.2,3, Демин V.A.2, Rylkov V.V.2
-
Affiliations:
- National Research Center “Kurchatov Institute”
- National Research Centre “Kurchatov Institute”
- Lomonosov Moscow State University
- Issue: Vol 18, No 4 (2023)
- Pages: 798-801
- Section: Conference proceedings
- URL: https://genescells.ru/2313-1829/article/view/623346
- DOI: https://doi.org/10.17816/gc623346
- ID: 623346
Cite item
Open Access
Access granted
Subscription or Fee Access
Abstract
Memristor-based neuromorphic computing systems (NСSs) provide a fast, high computational and energy efficient approach to neural network (NN) training and solving cognitive problems (pattern recognition, big data processing, prediction, etc.) [1]. Memristors could be organized in large crossbar arrays to perform vector-matrix multiplication (VMM) in a natural one-step way by the weighted electrical current summation (according to the Ohm’s and Kirchhoff’s laws) [1]. In contrast, being the most massively parallel operation in NN learning and inference, VMM is extremely time- and energy-expensive in traditional von Neumann architectures. Owing to this difference, memristor-based NCSs are of high interest. Memristors have already been successfully implemented for diverse NCS realizations, and such schemes as multi-layer perceptron (MLP) [2], long short-term memory and others have been demonstrated. Most of these NCSs are usually trained by various types of gradient descent learning algorithm, the hardware realization of which is challenging due to unreliable cycle-to-cycle (c2c) and device-to-device (d2d) variations of memristive devices. Several approaches have been proposed to partially mitigate these problems, including reservoir computing [3] and fine feature engineering [4]. The general idea of such approaches is to reduce the number of required weights (i.e. memristors) compared with fully connected NNs. In this respect, such novel architectures as convolutional NN (CNN) and MLP-mixer are of high interest as they provide significant weight reduction without classification efficiency drop. Although CNN based on memristors was already demonstrated, different aspects of its realization (such as hybrid hardware-software co-design) have yet to be studied. MLP-mixer was realized only in software. Therefore, in this work we have studied the possibility of hardware realization of CNN and MLP-mixer networks based on crossbar arrays of memristors. For this purpose, we studied (Co-Fe-B)x(LiNbO3)100−x nanocomposite (CFB-LNO NC) memristors, which operate through a multifilamentary resistive switching (RS) mechanism, demonstrate high endurance, long retention and possess multilevel RS [5].
Crossbar array of memristors was fabricated using laser photolithography for patterning electrode buses and ion-beam sputtering on the original facility for active layer deposition (~10 nm thick LiNbO3 and ~290 nm thick CFB-LNO NC with x ≈10–25 at.%). Details of the fabrication process could be found elsewhere [5].
I-V curves of the fabricated memristors showed small c2c and d2d variations, plasticity with 16 different resistive states and endurance of more than 105 cycles. Using the nanocomposite based crossbar arrays, we implemented a hybrid CNN, consisting of a hardware feature extractor with one/two kernels and a software classifier. Additionally, we have demonstrated in simulation that the usage of the memristors under study in the accurately adapted MLP-Mixer architecture results in high classification accuracy that is resilient to memristive variations and stuck devices.
Full Text
Memristor-based neuromorphic computing systems (NCSs) offer a fast, highly computationally efficient, and energy efficient approach to neural network (NN) training and solving cognitive problems, such as pattern recognition, big data processing, and prediction [1]. Memristors can be organized in large crossbar arrays to perform vector-matrix multiplication (VMM) in a natural one-step manner by the weighted electrical current summation, following Ohm’s and Kirchhoff’s laws [1]. In contrast, while VMM is the most massively parallel operation in NN learning and inference, it is extremely time- and energy-expensive in traditional von Neumann architectures. Therefore, memristor-based NCSs are highly desirable. Memristors have already been successfully used for various NCS realizations, including multi-layer perceptron (MLP) [2], long short-term memory, and others. Most NCSs are trained using gradient descent algorithms, but hardware implementation is challenging due to the inconsistent cycle-to-cycle (c2c) and device-to-device (d2d) variations of memristive devices. Several methods have been suggested to alleviate these issues, such as reservoir computing [3] and fine feature engineering [4]. The main concept behind these techniques is to minimize the number of necessary weights (i.e., memristors) in comparison to fully connected NNs. In this regard, convolutional NN (CNN) and MLP-mixer are novel architectures that are immensely intriguing due to their significant weight reduction without compromising classification efficiency. While CNNs based on memristors have already been demonstrated, further investigation of its implementation, such as hybrid hardware-software co-design, remains necessary due to its unrealized aspects. In contrast, only software-based implementation of MLP-mixer has been achieved. Hence, this study explores the potential of crossbar arrays of memristors for the hardware implementation of CNN and MLP-mixer networks. We examined CFB-LNO NC memristors composed of (Co-Fe-B)x(LiNbO3)100−x, which exhibit high endurance, long retention, and multilevel RS through a multifilamentary RS mechanism [5].
A crossbar array of memristors was manufactured by utilizing laser photolithography for the patterning of electrode buses, and ion-beam sputtering at the original facility for deposition of the active layer, which was composed of LiNbO3 (~10 nm thick) and CFB-LNO NC (~290 nm thick) with x ≈10–25 at.%. For more information about the fabrication process, please refer to [5].
The I-V curves of the fabricated memristors indicated negligible c2c and d2d variations, along with 16 distinct resistive states and endurance exceeding 105 cycles. A hybrid CNN was developed using crossbar arrays composed of nanocomposites. The hardware component of this system incorporated a feature extractor with one or two kernels, while the software utilized a classifier. Additionally, we demonstrated through simulations that implementing the studied memristors in the precisely tailored MLP-Mixer framework yields exceptional classification accuracy, impervious to memristive fluctuations and stuck hardware.
ADDITIONAL INFORMATION
Authors’ contribution. All authors made a substantial contribution to the conception of the work, acquisition, analysis, interpretation of data for the work, drafting and revising the work, final approval of the version to be published and agree to be accountable for all aspects of the work.
Funding sources. This work was supported by the Russian Science Foundation (project No. 22-19-00171). Measurements were carried out with the equipment of the Resource Centers (NRC “Kurchatov Institute”). Authors are thankful to Yu.V. Grishchenko, K.Yu. Chernoglazov, and Prof. A.V. Sitnikov for the sample fabrication.
Competing interests. The authors declare that they have no competing interests.
About the authors
A. V. Emelyanov
National Research Center “Kurchatov Institute”
Author for correspondence.
Email: emelyanov.andrey@mail.ru
Russian Federation, Moscow
A. N. Matsukatova
National Research Centre “Kurchatov Institute”; Lomonosov Moscow State University
Email: emelyanov.andrey@mail.ru
Russian Federation, Moscow; Moscow
A. I. Iliasov
National Research Centre “Kurchatov Institute”; Lomonosov Moscow State University
Email: emelyanov.andrey@mail.ru
Russian Federation, Moscow; Moscow
V. A. Демин
National Research Centre “Kurchatov Institute”
Email: emelyanov.andrey@mail.ru
Russian Federation, Moscow
V. V. Rylkov
National Research Centre “Kurchatov Institute”
Email: emelyanov.andrey@mail.ru
Russian Federation, Moscow
References
- Xia Q, Yang JJ. Memristive crossbar arrays for brain-inspired computing. Nat Mater. 2019;18(4):309–323. Corrected and republished from: Nat Mater. 2019;18(5):518. doi: 10.1038/s41563-019-0291-x
- Shchanikov S, Zuev A, Bordanov I, et al. Designing a bidirectional, adaptive neural interface incorporating machine learning capabilities and memristor-enhanced hardware. Chaos, Solitons and Fractals. 2021;142:110504. doi: 10.1016/j.chaos.2020.110504
- Matsukatova AN, Prudnikov NV, Kulagin VA, et al. Combination of organic-based reservoir computing and spiking neuromorphic systems for a robust and efficient pattern classification. Adv Intell Syst. 2023. Vol. 5, N 6. P. 2200407. doi: 10.1002/aisy.202200407
- Matsukatova AN, Vdovichenko AYu, Patsaev TD, et al. Scalable nanocomposite parylene-based memristors: multifilamentary resistive switching and neuromorphic applications. Nano Res. 2023;16:3207–3214. doi: 10.1007/s12274-022-5027-6
- Ilyasov AI, Nikiruy KE, Emelyanov AV, et al. Arrays of nanocomposite crossbar memristors for the implementation of formal and spiking neuromorphic systems. Nanotechnol Russia. 2022;17:118–125. doi: 10.1134/S2635167622010050
Supplementary files
