2016/07/12 by Avhishek Chatterjee, Lav R. Varshney, Chatterjee, Avhishek +1
Engineering · #Advanced Memory and Neural Computing #Advancements in Semiconductor Devices and Circuit Design #FOS: Computer and information sciences #Ferroelectric and Negative Capacitance Devices #Information Theory (cs.IT)
paper · pdf · doi:10.48550/arxiv.1607.03572
openalex publication_date 2016/07/12 · openalex created_date 2022/10/07 · openalex updated_date 2026/07/28
Due to energy-efficiency requirements, computational systems are now being\nimplemented using noisy nanoscale semiconductor devices whose reliability\ndepends on energy consumed. We study circuit-level energy-reliability limits\nfor deep feedforward neural networks (multilayer perceptrons) built using such\ndevices, and en route also establish the same limits for formulas (boolean\ntree-structured circuits). To obtain energy lower bounds, we extend Pippenger's\nmutual information propagation technique for characterizing the complexity of\nnoisy circuits, since small circuit complexity need not imply low energy. Many\ndevice technologies require all gates to have the same electrical operating\npoint; in circuits of such uniform gates, we show that the minimum energy\nrequired to achieve any non-trivial reliability scales superlinearly with the\nnumber of inputs. Circuits implemented in emerging device technologies like\nspin electronics can, however, have gates operate at different electrical\npoints; in circuits of such heterogeneous gates, we show energy scaling can be\nlinear in the number of inputs. Building on our extended mutual information\npropagation technique and using crucial insights from convex optimization\ntheory, we develop an algorithm to compute energy lower bounds for any given\nboolean tree under heterogeneous gates. This algorithm runs in linear time in\nnumber of gates, and is therefore practical for modern circuit design. As part\nof our development we find a simple procedure for energy allocation across\ncircuit gates with different operating points and neural networks with\ndifferently-operating layers.\n