To image, or not to image: class-specific diffractive cameras with all-optical erasure of undesired objects

Bijie Bai; Yi Luo; Tianyi Gan; Jingtian Hu; Yuhang Li; Yifan Zhao; Deniz Mengu; Mona Jarrahi; Aydogan Ozcan

doi:10.1186/s43593-022-00021-3

Your Location：

Home >

Browse articles >

To image, or not to image: class-specific diffractive cameras with all-optical erasure of undesired objects

光：快讯（英文） 2022年2卷第1期

Research Article | Updated：2024-09-28

- To image, or not to image: class-specific diffractive cameras with all-optical erasure of undesired objects
- To image, or not to image: class-specific diffractive cameras with all-optical erasure of undesired objects
- Affiliations：
  
  Electrical and Computer Engineering Department, University of California, Los Angeles, CA 90095, USA
  Bioengineering Department, University of California, Los Angeles 90095, USA
  California NanoSystems Institute (CNSI), University of California, Los Angeles, CA, USA
- Author bio：
  
  *Aydogan Ozcanozcan@ucla.edu
- Funds：
  
  The Ozcan Research Group at UCLA acknowledges the support of ONR(N00014-22-1-2016);The Jarrahi Research Group at UCLA acknowledges the support of the Department of Energy(DE-SC0016925)
- DOI：10.1186/s43593-022-00021-3
  中图分类号：
- 纸质出版日期：2022-12，
  
  网络出版日期：2022-08-15，
  
  收稿日期：2022-05-31，
  
  修回日期：2022-07-16，
  
  录用日期：2022-07-25
- Accepted：
Scan QR Code
To image, or not to image: class-specific diffractive cameras with all-optical erasure of undesired objects[J]. 光：快讯（英文）, 2022,2(1).

Bijie Bai, Yi Luo, Tianyi Gan, et al. To image, or not to image: class-specific diffractive cameras with all-optical erasure of undesired objects[J]. eLight, 2022,2(1).
To image, or not to image: class-specific diffractive cameras with all-optical erasure of undesired objects[J]. 光：快讯（英文）, 2022,2(1). DOI： 10.1186/s43593-022-00021-3.

Bijie Bai, Yi Luo, Tianyi Gan, et al. To image, or not to image: class-specific diffractive cameras with all-optical erasure of undesired objects[J]. eLight, 2022,2(1). DOI： 10.1186/s43593-022-00021-3.

摘要

Abstract

Privacy protection is a growing concern in the digital era

with machine vision techniques widely used throughout public and private settings. Existing methods address this growing problem by

e.g.

encrypting camera images or obscuring/blurring the imaged information through digital algorithms. Here

we demonstrate a camera design that performs class-specific imaging of target objects with instantaneous all-optical erasure of other classes of objects. This diffractive camera consists of transmissive surfaces structured using deep learning to perform selective imaging of target classes of objects positioned at its input field-of-view. After their fabrication

the thin diffractive layers collectively perform optical mode filtering to accurately form images of the objects that belong to a target data class or group of classes

while instantaneously erasing objects of the other data classes at the output field-of-view. Using the same framework

we also demonstrate the design of class-specific permutation and class-specific linear transformation cameras

where the objects of a target data class are pixel-wise permuted or linearly transformed following an arbitrarily selected transformation matrix for all-optical class-specific encryption

while the other classes of objects are irreversibly erased from the output image. The success of class-specific diffractive cameras was experimentally demonstrated using terahertz (THz) waves and 3D-printed diffractive layers that selectively imaged only one class of the MNIST handwritten digit dataset

all-optically erasing the other handwritten digits. This diffractive camera design can be scaled to different parts of the electromagnetic spectrum

including

e.g.

the visible and infrared wavelengths

to provide transformative opportunities for privacy-preserving digital cameras and task-specific data-efficient imaging.

关键词

Keywords

references

J. Scharcanski, Bringing vision-based measurements into our daily life: a grand challenge for computer vision systems. Front. ICT 3, 3 (2016)

X. Feng, Y. Jiang, X. Yang, M. Du, X. Li, Computer vision algorithms and hardware implementations: a survey. Integration 69, 309-320 (2019)

M. Al-Faris, J. Chiverton, D. Ndzi, A.I. Ahmed, A review on computer vision-based methods for human action recognition. J. Imaging 6, 46 (2020)

X. Wang, Intelligent multi-camera video surveillance: a review. Pattern Recogn. Lett. 34, 3-19 (2013)

N. Haering, P.L. Venetianer, A. Lipton, The evolution of video surveillance: an overview. Mach. Vis. Appl. 19, 279-290 (2008)

E. D. Dickmannsin, The development of machine vision for road vehicles in the last decade. in IEEE Intelligent Vehicle Symposium, 2002, vol. 1 (2002), p. 268–281.

J. Janai, F. Güney, A. Behl, A. Geiger, Computer vision for autonomous vehicles: problems, datasets and state of the art. CGV 12, 1-308 (2020)

A. Esteva et al., Deep learning-enabled medical computer vision. NPJ Digit. Med. 4, 1-9 (2021)

M. Tistarelli, M. Bicego, E. Grosso, Dynamic face recognition: from human to machine vision. Image Vis. Comput. 27, 222-232 (2009)

T.B. Moeslund, E. Granum, A survey of computer vision-based human motion capture. Comput. Vis. Image Underst. 81, 231-268 (2001)

G. Singh, G. Bhardwaj, S.V. Singh, V. Garg, Biometric identification system: security and privacy concern, in Artificial intelligence for a sustainable industry 4.0. ed. byS. Awasthi, C.M. Travieso-González, G. Sanyal, D. Kumar Singh (Springer International Publishing, Berlin, 2021), pp. 245-264. https://doi.org/10.1007/978-3-030-77070-9_15https://doi.org/10.1007/978-3-030-77070-9_15

A. Acquisti, L. Brandimarte, G. Loewenstein, Privacy and human behavior in the age of information. Science 347, 509-514 (2015)

A. Acquisti, L. Brandimarte, J. Hancock, How privacy's past may shape its future. Science 375, 270-272 (2022)

W.N. Price, I.G. Cohen, Privacy in the age of medical big data. Nat. Med. 25, 37-43 (2019)

J.R. Padilla-López, A.A. Chaaraoui, F. Flórez-Revuelta, Visual privacy protection methods: a survey. Expert Syst. Appl. 42, 4177-4195 (2015)

C. Neustaedter, S. Greenberg, M. Boyle, Blur filtration fails to preserve privacy for home-based video conferencing. ACM Trans. Comput.-Hum. Interact. 13, 1-36 (2006)

A. Frome, et al., Large-scale privacy protection in Google Street View. in 2009 IEEE 12th International Conference on Computer Vision (2009), pp. 2373–2380. https://doi.org/10.1109/ICCV.2009.5459413https://doi.org/10.1109/ICCV.2009.5459413.

F. Dufaux, T. Ebrahimi, Scrambling for privacy protection in video surveillance systems. IEEE Trans. Circuits Syst. Video Technol. 18, 1168-1174 (2008)

W. Zeng, S. Lei, Efficient frequency domain selective scrambling of digital video. IEEE Trans. Multimed. 5, 118-129 (2003)

A. Criminisi, P. Perez, K. Toyama, Region filling and object removal by exemplar-based image inpainting. IEEE Trans. Image Process. 13, 1200-1212 (2004)

K. Inai, M. Pålsson, V. Frinken, Y. Feng, S. Uchida, Selective concealment of characters for privacy protection. in 2014 22nd International Conference on Pattern Recognition (2014), p. 333–338. https://doi.org/10.1109/ICPR.2014.66https://doi.org/10.1109/ICPR.2014.66.

R. Uittenbogaard et al., Privacy protection in street-view panoramas using depth and multi-view imagery. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (IEEE, 2019), pp. 10573-10582. https://doi.org/10.1109/CVPR.2019.01083https://doi.org/10.1109/CVPR.2019.01083.

K. Brkic, I. Sikiric, T. Hrkac, Z. Kalafatic, I know that person: generative full body and face de-identification of people in images. in 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) (2017), pp. 1319–1328. https://doi.org/10.1109/CVPRW.2017.173https://doi.org/10.1109/CVPRW.2017.173.

F. Pittaluga, S. Koppal, A. Chakrabarti, Learning privacy preserving encodings through adversarial training. in 2019 IEEE Winter Conference on Applications of Computer Vision (WACV), (IEEE, 2019), pp. 791-799. https://doi.org/10.1109/WACV.2019.00089https://doi.org/10.1109/WACV.2019.00089.

A. Chattopadhyay, T. E. Boult, PrivacyCam: a privacy preserving camera using uCLinux on the Blackfin DSP. in 2007 IEEE Conference on Computer Vision and Pattern Recognition (2007), pp. 1–8, https://doi.org/10.1109/CVPR.2007.383413https://doi.org/10.1109/CVPR.2007.383413.

T. Winkler, B. Rinner, TrustCAM: security and privacy-protection for an embedded smart camera based on trusted computing. in 2010 7th IEEE International Conference on Advanced Video and Signal Based Surveillance (2010), pp. 593–600, https://doi.org/10.1109/AVSS.2010.38https://doi.org/10.1109/AVSS.2010.38.

Mrityunjay , P. J. Narayanan, The de-identification camera. in 2011 Third National Conference on Computer Vision, Pattern Recognition, Image Processing and Graphics (2011), pp. 192–195. https://doi.org/10.1109/NCVPRIPG.2011.48https://doi.org/10.1109/NCVPRIPG.2011.48.

53 Important statistics about how much data is created every day. Financesonline.com. (2021). https://financesonline.com/how-much-datais-created-every-day/https://financesonline.com/how-much-datais-created-every-day/.

P. Dhar, The carbon impact of artificial intelligence. Nature Machine Intelligence 2, 423-425 (2020)

S. Thakur, A. Chaurasia, Towards Green Cloud Computing: Impact of carbon footprint on environment. in 2016 6th International Conference—Cloud System and Big Data Engineering (Confluence), (2016), pp. 209–213. https://doi.org/10.1109/CONFLUENCE.2016.7508115https://doi.org/10.1109/CONFLUENCE.2016.7508115.

L. Belkhir, A. Elmeligi, Assessing ICT global emissions footprint: trends to 2040 & recommendations. J. Clean. Prod. 177, 448-463 (2018)

M. Durante, Computational power: the impact of ICT on law, society and knowledge (Routledge, London, 2021). https://doi.org/10.4324/9781003098683https://doi.org/10.4324/9781003098683

Pittaluga, F. & J S. Koppal,. Privacy preserving optics for miniature vision sensors. in 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), (IEEE, 2015), pp. 314-324. https://doi.org/10.1109/CVPR.2015.7298628https://doi.org/10.1109/CVPR.2015.7298628.

F. Pittaluga, A. Zivkovic, S. J. Koppal, Sensor-level privacy for thermal cameras. in 2016 IEEE International Conference on Computational Photography (ICCP) (2016), pp. 1–12. https://doi.org/10.1109/ICCPHOT.2016.7492877https://doi.org/10.1109/ICCPHOT.2016.7492877.

C. Hinojosa, J. C. Niebles, H. Arguello, Learning privacy-preserving Optics for Human Pose Estimation. in 2021 IEEE/CVF International Conference on Computer Vision (ICCV) (IEEE, 2021), pp. 2553-2562. https://doi.org/10.1109/ICCV48922.2021.00257https://doi.org/10.1109/ICCV48922.2021.00257.

Y. LeCun, et al., Handwritten Digit Recognition With A Back-Propagation Network. in Advances in Neural Information Processing Systems vol. 2, (Morgan-Kaufmann, 1989).

H. Xiao, K. Rasul, R. Vollgraf, Fashion-MNIST: a novel image dataset for benchmarking machine learning algorithms. (2017). https://doi.org/10.48550/arXiv.1708.07747https://doi.org/10.48550/arXiv.1708.07747.

J. Benesty, J. Chen, Y. Huang, I. Cohen, Pearson correlation coefficient. in Noise reduction in speech processing (Springer Berlin Heidelberg, 2009).

O. Kulce, D. Mengu, Y. Rivenson, A. Ozcan, All-optical information-processing capacity of diffractive surfaces. Light Sci. Appl. 10, 25 (2021)

O. Kulce, D. Mengu, Y. Rivenson, A. Ozcan, All-optical synthesis of an arbitrary linear transformation using diffractive surfaces. Light Sci. Appl. 10, 196 (2021)

D. Mengu et al., Misalignment resilient diffractive optical networks. Nanophotonics 9, 4207-4219 (2020)

C. Vieu et al., Electron beam lithography: resolution limits and applications. Appl. Surf. Sci. 164, 111-117 (2000)

X. Zhou, Y. Hou, J. Lin, A review on the processing accuracy of two-photon polymerization. AIP Adv. 5, 030701 (2015)

Y. Luo et al., Design of task-specific optical systems using broadband diffractive neural networks. Light Sci. Appl. 8, 112 (2019)

X. Lin et al., All-optical machine learning using diffractive deep neural networks. Science 361, 1004-1008 (2018)

A. Ozcan, E. McLeod, Lensless imaging and sensing. Annu. Rev. Biomed. Eng. 18, 77-102 (2016)

D. P. Kingma, J. Ba, Adam: a method for stochastic optimization. arXiv: 1412. 6980 [cs] (2017).

浏览量

Downloads

CSCD

文章被引用时，请邮件提醒。

Submit

关联资源

暂无数据