High-throughput optical neuromorphic graphic processing at millions of images per second

Yibo Dong; Yuchi Bai; Qiming Zhang; Haitao Luan; Min Gu

doi:10.1186/s43593-025-00106-9

Your Location：

Home >

Browse articles >

High-throughput optical neuromorphic graphic processing at millions of images per second

eLight 2025年5卷

Research Article | Updated：2025-12-01

- High-throughput optical neuromorphic graphic processing at millions of images per second
- High-throughput optical neuromorphic graphic processing at millions of images per second
- Affiliations：
- Author bio：
- Funds：
  
  Science and Technology Commission of Shanghai Municipality;Shanghai Municipal Science and Technology Major Project;Frontiers Science Center for Shanghai Municipality;National Key Research and Development program of China;Shanghai Sailing Program
- DOI：10.1186/s43593-025-00106-9
  中图分类号：
- 收稿：2025-06-30，
  
  修回：2025-09-08，
  
  录用：2025-09-17，
  
  网络出版：2025-10-20，
  
  纸质出版：2025-12
- Accepted：
Scan QR Code
Yibo Dong, Yuchi Bai, Qiming Zhang, 等. High-throughput optical neuromorphic graphic processing at millions of images per second[J]. eLight, 2025,5.

Yibo Dong, Yuchi Bai, Qiming Zhang, et al. High-throughput optical neuromorphic graphic processing at millions of images per second[J]. eLight, 2025, 5.
Yibo Dong, Yuchi Bai, Qiming Zhang, 等. High-throughput optical neuromorphic graphic processing at millions of images per second[J]. eLight, 2025,5. DOI： 10.1186/s43593-025-00106-9.

Yibo Dong, Yuchi Bai, Qiming Zhang, et al. High-throughput optical neuromorphic graphic processing at millions of images per second[J]. eLight, 2025, 5. DOI： 10.1186/s43593-025-00106-9.

摘要

Abstract

Optical diffractive neural networks (DNNs) offer superb parallelism and scalability for the direct analogue processing of planar information. However

their complete reliance on coherent light interference constrains the integration and computational frequency

as well as demonstrating low diffraction efficiency and robustness. Here

we present an optical graphics processing unit (OGPU) with a vertically integrated architecture

addressing these challenges through the use of an addressable vertical-cavity surface-emitting laser (VCSEL) array. This array functions as a high-speed planar information fan-in device

with each unit exhibiting individually coherent and mutually incoherent (MI) properties. We develop MI-DNNs that leverage the direct operations of spatially incoherent light while preserving the benefits of coherent computing. Therefore

the entire computing system with free-space architecture can be miniaturized to a handheld form factor. The OGPU operates efficiently under ultralow-light conditions (as low as 3.52 aJ/μm

per frame) and achieves a record image processing speed of 25 million frames per second. The OGPU has a computational power of 77.3 tera-operations per second (TOPS)

and an energy efficiency of 950 TOPS/W. The OPGU achieves competitive image classification accuracy of up to 98.6% and serves as versatile parallel convolutional kernels for image processing tasks

including edge extraction and image denoising.

关键词

Keywords

references

K. Boahen , Dendrocentric learning for synthetic intelligence . Nature 612 , 43 - 50 ( 2022 ). http://doi.org/10.1038/s41586-022-05340-6 http://doi.org/10.1038/s41586-022-05340-6

S. Zhu et al. , Intelligent computing: the latest advances, challenges, and future . Intell. Comput. 2 ,( 2023 ). http://doi.org/10.34133/icomputing.0006 http://doi.org/10.34133/icomputing.0006

G. Wetzstein et al. , Inference in artificial intelligence with deep optics and photonics . Nature 588 , 39 - 47 ( 2020 ). http://doi.org/10.1038/s41586-020-2973-6 http://doi.org/10.1038/s41586-020-2973-6

H. Wu , Q. Dai , Artificial intelligence accelerated by light . Nature 589 , 25 - 26 ( 2021 ). http://doi.org/10.1038/d41586-020-03572-y http://doi.org/10.1038/d41586-020-03572-y

B.J. Shastri et al. , Photonics for artificial intelligence and neuromorphic computing . Nature Photonics 15 , 102 - 114 ( 2021 ). http://doi.org/10.1038/s41566-020-00754-y http://doi.org/10.1038/s41566-020-00754-y

B.J. Shastri, et al., Principles of neuromorphic photonics. Berlin Heidelberg: Springer, 37 (2018).

Q.M. Zhang , H.Y. Yu , M. Barbiero , B.K. Wang , M. Gu , Artificial neural networks enabled by nanophotonics . Light Sci. Appl. 8 ,( 2019 ). http://doi.org/10.1038/s41377-019-0151-0 http://doi.org/10.1038/s41377-019-0151-0

Y. Chen et al. , All-analog photoelectronic chip for high-speed vision tasks . Nature 623 , 48 - 57 ( 2023 ). http://doi.org/10.1038/s41586-023-06558-8 http://doi.org/10.1038/s41586-023-06558-8

B. Dong et al. , Partial coherence enhances parallelized photonic computing . Nature 632 , 55 - 62 ( 2024 ). http://doi.org/10.1038/s41586-024-07590-y http://doi.org/10.1038/s41586-024-07590-y

X. Lin et al. , All-optical machine learning using diffractive deep neural networks . Science 361 , 1004 ( 2018 ). http://doi.org/10.1126/science.aat8084 http://doi.org/10.1126/science.aat8084

J. Feldmann , N. Youngblood , C.D. Wright , H. Bhaskaran , W.H.P. Pernice , All-optical spiking neurosynaptic networks with self-learning capabilities . Nature 569 , 208 - 214 ( 2019 ). http://doi.org/10.1038/s41586-019-1157-8 http://doi.org/10.1038/s41586-019-1157-8

J. Feldmann et al. , Parallel convolutional processing using an integrated photonic tensor core . Nature 589 , 52 - 58 ( 2021 ). http://doi.org/10.1038/s41586-020-03070-1 http://doi.org/10.1038/s41586-020-03070-1

F. Ashtiani , A.J. Geers , F. Aflatouni , An on-chip photonic deep neural network for image classification . Nature 606 , 501 - 506 ( 2022 ). http://doi.org/10.1038/s41586-022-04714-0 http://doi.org/10.1038/s41586-022-04714-0

S. Bandyopadhyay et al. , Single-chip photonic deep neural network with forward-only training . Nature Photonics 18 , 1335 - 1343 ( 2024 ). http://doi.org/10.1038/s41566-024-01567-z http://doi.org/10.1038/s41566-024-01567-z

J. Hu et al. , Diffractive optical computing in free space . Nat. Commun. 15 ,( 2024 ). http://doi.org/10.1038/s41467-024-45982-w http://doi.org/10.1038/s41467-024-45982-w

M. Gu , X. Fang , H. Ren , E. Goi , Optically digitalized holography: a perspective for all-optical machine learning . Engineering 5 , 363 - 365 ( 2019 ). http://doi.org/10.1016/j.eng.2019.04.002 http://doi.org/10.1016/j.eng.2019.04.002

Y. Zhang , S. Zhu , J. Hu , M. Gu , Femtosecond laser direct nanolithography of perovskite hydration for temporally programmable holograms . Nat. Commun. 15 ,( 2024 ). http://doi.org/10.1038/s41467-024-51148-5 http://doi.org/10.1038/s41467-024-51148-5

C. Liu et al. , A programmable diffractive deep neural network based on a digital-coding metasurface array . Nat. Electron. 5 , 113 - 122 ( 2022 ). http://doi.org/10.1038/s41928-022-00719-9 http://doi.org/10.1038/s41928-022-00719-9

T. Zhou et al. , Large-scale neuromorphic optoelectronic computing with a reconfigurable diffractive processing unit . Nat. Photonics 15 , 367 - 373 ( 2021 ). http://doi.org/10.1038/s41566-021-00796-w http://doi.org/10.1038/s41566-021-00796-w

H. Yu, et al., All-optical image transportation through a multimode fiber using a miniaturized diffractive neural network on the distal facet. Nat. Photon. (2025)

T. Wang et al. , An optical neural network using less than 1 photon per multiplication . Nat. Commun. 13 , 123 ( 2022 ). http://doi.org/10.1038/s41467-021-27774-8 http://doi.org/10.1038/s41467-021-27774-8

L. Bernstein et al. , Single-shot optical neural network . Sci. Adv. 9 ,( 2023 ). http://doi.org/10.1126/sciadv.adg7904 http://doi.org/10.1126/sciadv.adg7904

A. Song , S.N. Murty Kottapalli , R. Goyal , B. Schölkopf , P. Fischer , Low-power scalable multilayer optoelectronic neural networks enabled with incoherent light . Nat. Commun. 15 ,( 2024 ). http://doi.org/10.1038/s41467-024-55139-4 http://doi.org/10.1038/s41467-024-55139-4

E. Goi , S. Schoenhardt , M. Gu , Direct retrieval of Zernike-based pupil functions using integrated diffractive deep neural networks . Nat. Commun. 13 ,( 2022 ). http://doi.org/10.1038/s41467-022-35349-4 http://doi.org/10.1038/s41467-022-35349-4

Y. Zhang et al. , Memory-less scattering imaging with ultrafast convolutional optical neural networks . Sci. Adv. 10 ,( 2024 ). http://doi.org/10.1126/sciadv.adn2205 http://doi.org/10.1126/sciadv.adn2205

Y. Dong et al. , Compact eternal diffractive neural network chip for extreme environments . Commun. Eng. 3 ,( 2024 ). http://doi.org/10.1038/s44172-024-00211-6 http://doi.org/10.1038/s44172-024-00211-6

X. Luo et al. , Metasurface-enabled on-chip multiplexed diffractive neural networks in the visible . Light Sci. Appl. 11 , 158 ( 2022 ). http://doi.org/10.1038/s41377-022-00844-2 http://doi.org/10.1038/s41377-022-00844-2

C. Qian et al. , Performing optical logic operations by a diffractive neural network . Light Sci. Appl. 9 , 59 ( 2020 ). http://doi.org/10.1038/s41377-020-0303-2 http://doi.org/10.1038/s41377-020-0303-2

T. Fu et al. , Photonic machine learning with on-chip diffractive optics . Nat. Commun. 14 ,( 2023 ). http://doi.org/10.1038/s41467-022-35772-7 http://doi.org/10.1038/s41467-022-35772-7

H.H. Zhu et al. , Space-efficient optical computing with an integrated chip diffractive neural network . Nat. Commun. 13 ,( 2022 ). http://doi.org/10.1038/s41467-022-28702-0 http://doi.org/10.1038/s41467-022-28702-0

T. Yan et al. , All-optical graph representation learning using integrated diffractive photonic computing units . Sci. Adv. 8 ,( 2022 ). http://doi.org/10.1126/sciadv.abn7630 http://doi.org/10.1126/sciadv.abn7630

D. Mengu , Y. Luo , Y. Rivenson , A. Ozcan , Analysis of diffractive optical neural networks and their integration with electronic neural networks . IEEE J. Sel. Top. Quantum Electron. 26 , 1 - 14 ( 2020 ). http://doi.org/10.1109/JSTQE.2019.2921376 http://doi.org/10.1109/JSTQE.2019.2921376

Y. Cui et al. , Single-mode chirped high-contrast metastructure VCSEL for 106 Gbps PAM4 transmission . Optica 11 , 1567 - 1574 ( 2024 ). http://doi.org/10.1364/OPTICA.539252 http://doi.org/10.1364/OPTICA.539252

G. Pan et al. , Harnessing the capabilities of VCSELs: unlocking the potential for advanced integrated photonic devices and systems . Light Sci. Appl. 13 ,( 2024 ). http://doi.org/10.1038/s41377-024-01561-8 http://doi.org/10.1038/s41377-024-01561-8

M. Gu , Y. Dong , H. Yu , H. Luan , Q. Zhang , Perspective on 3D vertically-integrated photonic neural networks based on VCSEL arrays . Nanophotonics 12 , 827 - 832 ( 2023 ). http://doi.org/10.1515/nanoph-2022-0437 http://doi.org/10.1515/nanoph-2022-0437

D. Owen-Newns , J. Robertson , M. Hejda , A. Hurtado , Ghz rate neuromorphic photonic spiking neural network with a single vertical-cavity surface-emitting laser (VCSEL) . IEEE J. Sel. Top. Quantum Electron. 29 , 1 - 10 ( 2023 ). http://doi.org/10.1109/JSTQE.2022.3205716 http://doi.org/10.1109/JSTQE.2022.3205716

J. Robertson , M. Hejda , J. Bueno , A. Hurtado , Ultrafast optical integration and pattern classification for neuromorphic photonics based on spiking VCSEL neurons . Sci. Rep. 10 ,( 2020 ). http://doi.org/10.1038/s41598-020-62945-5 http://doi.org/10.1038/s41598-020-62945-5

Y. Zhang et al. , All-optical neuromorphic binary convolution with a spiking VCSEL neuron for image gradient magnitudes . Photonics Res. 9 ,( 2021 ). http://doi.org/10.1364/PRJ.412141 http://doi.org/10.1364/PRJ.412141

Z. Chen et al. , Deep learning with coherent VCSEL neural networks . Nat. Photonics 17 , 723 - 730 ( 2023 ). http://doi.org/10.1038/s41566-023-01233-w http://doi.org/10.1038/s41566-023-01233-w

A. Skalli et al. , Photonic neuromorphic computing using vertical cavity semiconductor lasers . Opt. Mater. Express 12 , 2395 - 2414 ( 2022 ). http://doi.org/10.1364/OME.450926 http://doi.org/10.1364/OME.450926

X. Porte et al. , A complete, parallel and autonomous photonic neural network in a semiconductor multimode laser . J. Phys. Photonics 3 ,( 2021 ). http://doi.org/10.1088/2515-7647/abf6bd http://doi.org/10.1088/2515-7647/abf6bd

L. Bernd, et al., Proc. SPIE 240–245 (2000)

B. Lucke , G. Hergenhan , U. Branch , A. Giesen , Phase tuning of injection-locked VCSELs . IEEE Photon. Technol. Lett. 13 , 100 - 102 ( 2001 ). http://doi.org/10.1109/68.910501 http://doi.org/10.1109/68.910501

Y. Lecun , L. Bottou , Y. Bengio , P. Haffner , Gradient-based learning applied to document recognition . Proc. IEEE 86 , 2278 - 2324 ( 1998 ). http://doi.org/10.1109/5.726791 http://doi.org/10.1109/5.726791

G. Cohen, S. Afshar, J. Tapson, A. Van Schaik, 2017 international joint conference on neural networks (IJCNN) 2921–2926 (IEEE, 2017)

B.J. Thompson et al. , Coherence in multielement-phased vertical-cavity surface-emitting laser arrays using resonance tuning . IEEE Photon. J. 9 , 1 - 8 ( 2017 ). http://doi.org/10.1109/JPHOT.2017.2750740 http://doi.org/10.1109/JPHOT.2017.2750740

C. Chih-Hao , L. Chrostowski , C.J. Chang-Hasnain , Injection locking of VCSELs . IEEE J. Sel. Top. Quantum Electron. 9 , 1386 - 1393 ( 2003 ). http://doi.org/10.1109/JSTQE.2003.819510 http://doi.org/10.1109/JSTQE.2003.819510

Z. Liu , R. Slavík , Optical injection locking: from principle to applications . J. Lightwave Technol. 38 , 43 - 59 ( 2020 ). http://doi.org/10.1109/JLT.2019.2945718 http://doi.org/10.1109/JLT.2019.2945718

M. Xun et al. , Phase tuning in two-dimensional coherently coupled vertical-cavity surface-emitting laser array . Appl. Opt. 55 , 5439 - 5443 ( 2016 ). http://doi.org/10.1364/AO.55.005439 http://doi.org/10.1364/AO.55.005439

M. Orenstein , E. Kapon , J.P. Harbison , L.T. Florez , N.G. Stoffel , Large two-dimensional arrays of phase-locked vertical cavity surface emitting lasers . Appl. Phys. Lett. 60 , 1535 - 1537 ( 1992 ). http://doi.org/10.1063/1.107243 http://doi.org/10.1063/1.107243

A. Dikopoltsev et al. , Topological insulator vertical-cavity laser array . Science 373 , 1514 - 1517 ( 2021 ). http://doi.org/10.1126/science.abj2232 http://doi.org/10.1126/science.abj2232

S. Richard, P. Michael, Proc. SPIE, 158–169 (2003)

H. Zhang , G. Mrozynski , A. Wallrabenstein , J. Schrage , Numerical investigation of the effects of the injection current on the SHB-effects of VCSEL’s . Int. J. Infrared Millimeter Waves 24 , 1287 - 1299 ( 2003 ). http://doi.org/10.1023/A:1024857421830 http://doi.org/10.1023/A:1024857421830

X. Zeng et al. , Experimental investigation of LP11 mode to OAM conversion in few mode-polarization maintaining fiber and the usage for all fiber OAM generator . IEEE Photon. J. 8 ,( 2016 ). http://doi.org/10.1109/JPHOT.2016.2594207 http://doi.org/10.1109/JPHOT.2016.2594207

M. Zheng , L. Shi , J. Zi , Optimize performance of a diffractive neural network by controlling the Fresnel number . Photonics Res. 10 , 2667 - 2676 ( 2022 ). http://doi.org/10.1364/PRJ.474535 http://doi.org/10.1364/PRJ.474535

A.V. Krishnamoorthy et al. , 16 x 16 VCSEL array flip-chip bonded to CMOS VLSI circuit . IEEE Photon. Technol. Lett. 12 , 1073 - 1075 ( 2000 ). http://doi.org/10.1109/68.868012 http://doi.org/10.1109/68.868012

Q. Wang et al. , Optically reconfigurable metasurfaces and photonic devices based on phase change materials . Nature Photonics 10 , 60 - 65 ( 2016 ). http://doi.org/10.1038/nphoton.2015.247 http://doi.org/10.1038/nphoton.2015.247

C. Ríos et al. , Integrated all-photonic non-volatile multi-level memory . Nat. Photonics 9 , 725 - 732 ( 2015 ). http://doi.org/10.1038/nphoton.2015.182 http://doi.org/10.1038/nphoton.2015.182

S. Pai et al. , Experimentally realized in situ backpropagation for deep learning in photonic neural networks . Science 380 , 398 - 404 ( 2023 ). http://doi.org/10.1126/science.ade8450 http://doi.org/10.1126/science.ade8450

Z. Xue et al. , Fully forward mode training for optical neural networks . Nature 632 , 280 - 286 ( 2024 ). http://doi.org/10.1038/s41586-024-07687-4 http://doi.org/10.1038/s41586-024-07687-4

K.T.P. Lim , H.L. Liu , Y.J. Liu , J.K.W. Yang , Holographic colour prints for enhanced optical security by combined phase and amplitude control . Nat. Commun. 10 , 25 ( 2019 ). http://doi.org/10.1038/s41467-018-07808-4 http://doi.org/10.1038/s41467-018-07808-4

浏览量

Downloads

CSCD

文章被引用时，请邮件提醒。

Submit

关联资源

暂无数据