Efficiently processing broadband signals using convolutional neural networks (CNNs) could enhance the performance of machine learning tools for a wide range of real-time applications, including image recognition, remote sensing and environmental monitoring. However, past studies suggest that performing broadband convolutional processing computations directly in sensors is challenging, particularly when using conventional complementary metal-oxide-semiconductor (CMOS) technology, which underpins the functioning of most existing transistors.
Researchers at Huazhong University of Science and Technology and Nanjing University have recently investigated the possibility of achieving the convolutional processing of broadband signals using an alternative platform, namely van der Waals heterostructures. Their paper, published in Nature Electronicscould ultimately inform the development of better performing image recognition algorithms.
“Our paper was inspired by some of our previous research works,” Tianyou Zhai, Xing Zhou and Feng Miao, three of the researchers who carried out the study, told TechXplore. “In studies published in Advanced Materials and Advanced Functional Materials, we realized type-III and type-II band-alignments in different heterostructures. Furthermore, we published a paper in Science Advanceswhere we realized a reconfigurable neural network vision sensor based on WSetwo.”
Building on their previous research efforts, Zhai, Zhou, Miao and their colleagues decided to fabricate heterostructures that could be used to process broadband signals using convolutional neural networks. They specifically created heterostructures based on PdSetwo/Nicknametwousing a mechanical transfer method.
The heterostructures created by the researchers exhibit gate-tunable positive and negative photoresponses, as well as a broadband linear gate-dependent photoresponsivity. Due to their advantageous characteristics, the team was able to use their heterostructures to create photovoltaic sensors, with which they could implement different types of broadband convolutional processing.
“Under a fixed gate voltage, the photoresponse modulation under different photon energies determines the wavelength-dependent convolutional characteristics,” Zhai, Zhou and Miao explained. “In addition, the photoresponse depends on the gate voltage, and modulation of the gate voltage can be used to realize the configuration of different convolution kernels to achieve different operations on remote sensing images.”
As an initial proof-of-concept, Zhai, Zhou, Miao and their colleagues used a single device based on their heterostructures to receive pixel images individually and then perform broadband convolutional processing on these images. In the future, however, they could also test their proposed system using two or more devices.
The researchers are among the first to perform broadband image recognition directly in-sensor. Their findings are highly promising, as their solution significantly outperformed conventional convolutional networks that are only capable of single-band processing.
Notably, the device created by Zhai, Zhou, Miao and their colleagues simultaneously achieved photodetection and broadband information processing. In the future, it could thus be used to develop more advanced image recognition tools, as well as alternative solutions to monitor remote environments.
“Using our newly devised, special ambipolar vdW heterostructure, we developed a new approach to realize in-sensor computing multifunctional optoelectronic devices,” Zhai, Zhou and Miao added. “We now plan to realize in-sensor-memory computing at hardware level for more complex functional requires. Meanwhile, we also plan to achieve large-scale integration of these devices to implement practical in-sensor convolutional processing.”
LOEN: Lensless opto-electronic neural network empowered machine vision
Lejing Pi et al, Broadband convolutional processing using band-alignment-tunable heterostructures, Nature Electronics (2022). DOI: 10.1038/s41928-022-00747-5
Xing Zhou et al, Tunneling Diode Based on WSe2 /SnS2 Heterostructure Incorporating High Detectivity and Responsivity, Advanced Materials (2018). DOI: 10.1002/adma.201703286
Bao Jin et al, Excellent Excitonic Photovoltaic Effect in 2D CsPbBr 3 /CdS Heterostructures, Advanced Functional Materials (2020). DOI: 10.1002/adfm.202006166
Chen-Yu Wang et al, Gate-tunable van der Waals heterostructure for reconfigurable neural network vision sensor, Science Advances (2020). DOI: 10.1126/sciadv.aba6173
© 2022 Science X Network
citation: Simultaneous broadband image sensing and convolutional processing using van der Waals heterostructures (2022, May 13) retrieved 14 May 2022 from https://techxplore.com/news/2022-05-simultaneous-broadband-image-convolutional-van.html
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.