CamJ: Enabling System-Level Energy Modeling and Architectural Exploration for In-Sensor Visual Computing

2023/04/06 by Tianrui Ma, Feng Yu, Ma, Tianrui +5 · 1 citation
Engineering · Computer Science · #CCD and CMOS Imaging Sensors #Advanced Memory and Neural Computing #Visual Attention and Saliency Detection

paper · pdf · doi:10.48550/arxiv.2304.03320

Abstract

CMOS Image Sensors (CIS) are fundamental to emerging visual computing applications. While conventional CIS are purely imaging devices for capturing images, increasingly CIS integrate processing capabilities such as Deep Neural Network (DNN). Computational CIS expand the architecture design space, but to date no comprehensive energy model exists. This paper proposes CamJ, a detailed energy modeling framework that provides a component-level energy breakdown for computational CIS and is validated against nine recent CIS chips. We use CamJ to demonstrate three use-cases that explore architectural trade-offs including computing in vs. off CIS, 2D vs. 3D-stacked CIS design, and analog vs. digital processing inside CIS. The code of CamJ is available at: https://github.com/horizon-research/CamJ

Cited by

• A-Graph: A Unified Graph Representation for At-Will Simulation across System Stacks

Related

• OASIS: Optimized Lightweight Autoencoder System for Distributed In-Sensor computing
• J3DAI: A tiny DNN-Based Edge AI Accelerator for 3D-Stacked CMOS Image Sensor
• Energy-Efficient & Real-Time Computer Vision with Intelligent Skipping via Reconfigurable CMOS Image Sensors
• Hardware-Algorithm Co-design Enabling Processing-in-Pixel-in-Memory (P2M) for Neuromorphic Vision Sensors
• P2M: A Processing-in-Pixel-in-Memory Paradigm for Resource-Constrained TinyML Applications