VEViD: Vision Enhancement via Virtual diffraction and coherent Detection

Bahram Jalali; Callen MacPhee

doi:10.1186/s43593-022-00034-y

Your Location：

Home >

Browse articles >

VEViD: Vision Enhancement via Virtual diffraction and coherent Detection

eLight Vol. 2, Issue 1, (2022)

RESEARCH ARTICLE | Updated：2024-09-28

- VEViD: Vision Enhancement via Virtual diffraction and coherent Detection
- Affiliations：
  
  Electrical and Computer Engineering Department, UCLA, Los Angeles, USA
- Author bio：
  
  *Bahram Jalalijalali@ucla.edu
- Funds：
- DOI：10.1186/s43593-022-00034-y
  CLC：
- Published：2022-12，
  
  Published Online：08 November 2022，
  
  Received：26 August 2022，
  
  Revised：27 September 2022，
  
  Accepted：29 September 2022
- Accepted：
Scan QR Code
Bahram Jalali, Callen MacPhee. VEViD: Vision Enhancement via Virtual diffraction and coherent Detection. [J]. eLight, 2022,2(1).
DOI：

Bahram Jalali, Callen MacPhee. VEViD: Vision Enhancement via Virtual diffraction and coherent Detection. [J]. eLight, 2022,2(1). DOI： 10.1186/s43593-022-00034-y.

摘要

Abstract

The history of computing started with analog computers consisting of physical devices performing specialized functions such as predicting the position of astronomical bodies and the trajectory of cannon balls. In modern times

this idea has been extended

for example

to ultrafast nonlinear optics serving as a surrogate analog computer to probe the behavior of complex phenomena such as rogue waves. Here we discuss a new paradigm where physical phenomena coded as an algorithm perform computational imaging tasks. Specifically

diffraction followed by coherent detection becomes an image enhancement tool. Vision Enhancement via Virtual diffraction and coherent Detection (VEViD) reimagines a digital image as a spatially varying metaphoric "lightfield" and then subjects the field to the physical processes akin to diffraction and coherent detection. The term "Virtual" captures the deviation from the physical world. The light field is pixelated and the propagation imparts a phase with dependence on frequency which is different from the monotonically-increasing behavior of physical diffraction. Temporal frequencies exist in three bands corresponding to the RGB color channels of a digital image. The phase of the output

not the intensity

represents the output image. VEViD is a high-performance low-light-level and color enhancement tool that emerges from this paradigm. The algorithm is extremely fast

interpretable

and reduces to a compact and intuitively-appealing mathematical expression. We demonstrate image enhancement of 4k video at over 200 frames per second and show the utility of this physical algorithm in improving the accuracy of object detection in low-light conditions by neural networks. The application of VEViD to color enhancement is also demonstrated.

关键词

Keywords

Physics-inspired computer visionNatural algorithmsLow-light image enhancementColor enhancement

references

https://en.wikipedia.org/wiki/Antikythera_mechanismhttps://en.wikipedia.org/wiki/Antikythera_mechanism

Donald Routledge Hill, Mechanical Engineering in the Medieval Near East, Scientific American, 1991, pp. 64-69 (cf. Donald Routledge Hill, Mechanical Engineering)

D.R. Solli, B. Jalali, Analog optical computing. Nat. Photonics 9(11), 704-706 (2015)

D.R. Solli, C. Ropers, P. Koonath, B. Jalali, Optical rogue waves. Nature 450(7172), 1054-1057 (2007)

J.M. Dudley, G. Genty, A. Mussot, A. Chabchoub, F. Dias, Rogue waves and analogies in optics and oceanography. Nat. Rev. Phys. 1 (11), 675-689 (2019)

E.H. Land, J.J. McCann, Lightness and Retinex theory. J. Opt. Soc. Amer. 61(1), 1-11 (1971)

E.H. Land, The retinex theory of color vision. Sci. Am. 237, 108-129 (1977)

M. Li et al., Structure-revealing low-light image enhancement via robust retinex model. IEEE Trans. Image Process. 27(6), 2828-2841 (2018)

X. Guo, Y. Li, H. Ling, 'LIME: low-light image enhancement via illumination map estimation.'IEEE Trans. Image Process. 26(2), 982-993 (2017)

L. Li, S. Sun, C. Xia, Survey of histogram equalization technology. Comput. Syst. Appl. 23(3), 1-8 (2014). (in Chinese)

G. Yadav, S. Maheshwari, A. Agarwal, Contrast limited adaptive histogram equalization based enhancement for real time video system, 2014 International Conference on Advances in Computing, Communications and Informatics (ICACCI), 2014, pp. 2392-2397, https://doi.org/10.1109/ICACCI.2014.6968381https://doi.org/10.1109/ICACCI.2014.6968381.

K.G. Lore, A. Akintayo, S. Sarkar, LLNet: A deep autoencoder approach to natural low-light image enhancement. Pattern Recognit 61, 650-662 (2017)

Lv F, Lu F, Wu J, Lim C. (2022). MBLLEN: Low-light Image/Video Enhancement Using CNNs.

M. Zhu, P. Pan, W. Chen, Y. Yang, EEMEFN: low-light image enhancement via edge-enhanced multi-exposure fusion network. Proc. AAAI Conf. Artif. Intell. 34(07), 13106-13113 (2020). https://doi.org/10.1609/aaai.v34i07.7013https://doi.org/10.1609/aaai.v34i07.7013

K. Lu, L. Zhang, TBEFN: a two-branch exposure-fusion network for low-light image enhancement. IEEE Trans. Multimedia 23, 4093-4105 (2021). https://doi.org/10.1109/TMM.2020.3037526https://doi.org/10.1109/TMM.2020.3037526

Y. Jiang, X. Gong, D. Liu, Y. Cheng, C. Fang, X. Shen, J. Yang, P. Zhou, Z. Wang, Enlightengan: deep light enhancement without paired supervision. IEEE Trans. Image Process. 30, 2340-2349 (2021)

C. Guo, C. Li, J. Guo, C. Loy, J. Hou, S. Kwong, R. Cong (2020). Zeroreference deep curve estimation for low-light image enhancement. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR) (pp. 1780-1789)

M. Asghari, B. Jalali, Edge detection in digital images using dispersive phase stretch transform. Int. J. Biomed. Imaging 2015, 687819 (2015)

M. Suthar, B. Jalali, "Phase-stretch adaptive gradient-field extractor (PAGE)," in Coding Theory, 2020.

C. Neil. Thompson et al. (MIT) 2020https://arxiv.org/pdf/2007.05558.pdfhttps://arxiv.org/pdf/2007.05558.pdf

J Redmon, A. Farhadi (2018). YOLOv3: An Incremental Improvement. ArXiv, abs/1804.02767.

C. Li, C. Guo, L.H. Han, J. Jiang, M.M. Cheng, J. Gu, C.C. Loy, Low-light image and video enhancement using deep learning: A survey. IEEE Trans. Pattern Anal. Mach. Intell. 01, 1-1 (2021)

B. Jalali, Device Physics & Modeling, In: InP HBTs: Growth, Processing, and Applications, Jalali, B., Pearton, S.J. (Eds.), (0-89006-724-4). PP. 229-263 (December 1994) Artech House

W. Yang, Y. Yuan, W. Ren, J. Liu, W. Scheirer, Z. Wang, T. Zhang et al., Advancing image understanding in poor visibility environments: a collective benchmark study. IEEE Trans. Image Process. 29, 5737-5752 (2020)

X. Jia, et al. LLVIP: A visible-infrared paired dataset for low-light vision. Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021.

K. Ma et al., perceptual quality assessment for multi-exposure image fusion. IEEE Trans. Image Process. 24(11), 3345-3356 (2015)

https://www.pexels.com/video/neo-classical-building-in-city-at-night-9935075/.https://www.pexels.com/video/neo-classical-building-in-city-at-night-9935075/.

https://photographylife.com/underexposure-and-overexposure-in-photographyhttps://photographylife.com/underexposure-and-overexposure-in-photography

https://sites.google.com/site/vonikakis/datasets/tm-diedhttps://sites.google.com/site/vonikakis/datasets/tm-died

https://pjreddie.com/media/files/yolov3.weights.https://pjreddie.com/media/files/yolov3.weights.

P.T. DeVore, B.W. Buckley, M.H. Asghari, D.R. Solli, B. Jalali, Coherent timestretch transform for near-field spectroscopy. IEEE Photonics J. 6(2), 1-7 (2014)

D.R. Solli, S. Gupta, B. Jalali, Optical phase recovery in the dispersive Fourier transform. Appl. Phys. Lett. 95(23), 231108 (2009)

K. Goda, D.R. Solli, K.K. Tsia, B. Jalali, Theory of amplified dispersive Fourier transformation. Phys. Rev. A 80(4), 043821 (2009)

Views

Downloads

CSCD

Alert me when the article has been cited

Submit

Publicity Resources

No data

Related Author

No data

Related Institution

No data

⁰