2020/05/15 by Yerlan Idelbayev, Idelbayev, Yerlan, Miguel Á. Carreira-Perpiñán +1 · 1 citation
Computer Science · #Advanced Data Compression Techniques #Blind Source Separation Techniques #FOS: Computer and information sciences #FOS: Mathematics #Image and Signal Denoising Methods #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Neural Networks and Applications #Neural and Evolutionary Computing (cs.NE) #Optimization and Control (math.OC)
paper · pdf · doi:10.48550/arxiv.2005.07786
openalex publication_date 2020/05/15 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
We propose a software framework based on the ideas of the\nLearning-Compression (LC) algorithm, that allows a user to compress a neural\nnetwork or other machine learning model using different compression schemes\nwith minimal effort. Currently, the supported compressions include pruning,\nquantization, low-rank methods (including automatically learning the layer\nranks), and combinations of those, and the user can choose different\ncompression types for different parts of a neural network.\n The LC algorithm alternates two types of steps until convergence: a learning\n(L) step, which trains a model on a dataset (using an algorithm such as SGD);\nand a compression (C) step, which compresses the model parameters (using a\ncompression scheme such as low-rank or quantization). This decoupling of the\n"machine learning" aspect from the "signal compression" aspect means that\nchanging the model or the compression type amounts to calling the corresponding\nsubroutine in the L or C step, respectively. The library fully supports this by\ndesign, which makes it flexible and extensible. This does not come at the\nexpense of performance: the runtime needed to compress a model is comparable to\nthat of training the model in the first place; and the compressed model is\ncompetitive in terms of prediction accuracy and compression ratio with other\nalgorithms (which are often specialized for specific models or compression\nschemes). The library is written in Python and PyTorch and available in Github.\n