2024/10/14 by Md Abdullah-Al Kaiser, Gourav Datta, Kaiser, Md Abdullah-Al +13 · 1 citation
Computer Science · Engineering · #CCD and CMOS Imaging Sensors #FOS: Computer and information sciences #FOS: Electrical engineering #Hardware Architecture (cs.AR) #Image and Video Processing (eess.IV) #Magnetic Field Sensors Techniques #Neural Networks and Reservoir Computing #electronic engineering #information engineering
paper · pdf · doi:10.48550/arxiv.2410.10592
openalex publication_date 2024/10/14 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/01
The vast amount of data generated by camera sensors has prompted the exploration of energy-efficient processing solutions for deploying computer vision tasks on edge devices. Among the various approaches studied, processing-in-pixel integrates massively parallel analog computational capabilities at the extreme-edge, i.e., within the pixel array and exhibits enhanced energy and bandwidth efficiency by generating the output activations of the first neural network layer rather than the raw sensory data. In this article, we propose an energy and bandwidth efficient ADC-less processing-in-pixel architecture. This architecture implements an optimized binary activation neural network trained using Hoyer regularizer for high accuracy on complex vision tasks. In addition, we also introduce a global shutter burst memory read scheme utilizing fast and disturb-free read operation leveraging innovative use of nanoscale voltage-controlled magnetic tunnel junctions (VC-MTJs). Moreover, we develop an algorithmic framework incorporating device and circuit constraints (characteristic device switching behavior and circuit non-linearity) based on state-of-the-art fabricated VC-MTJ characteristics and extensive circuit simulations using commercial GlobalFoundries 22nm FDX technology. Finally, we evaluate the proposed system's performance on two complex datasets - CIFAR10 and ImageNet, showing improvements in front-end and communication energy efficiency by 8.2x and 8.5x respectively and reduction in bandwidth by 6x compared to traditional computer vision systems, without any significant drop in the test accuracy.