2014/05/31 by Alon Kipnis, Andrea Goldsmith, Andrea J. Goldsmith +2
Computer Science · Mathematics · #Advanced Data Compression Techniques #Algorithm #Bandwidth (computing) #Blind Source Separation Techniques #Coherent sampling #Computer science #Distortion (music) #Encoder #Filter (signal processing) #Gaussian #Gaussian process #Image and Signal Denoising Methods #Importance sampling #Mathematics #Monte Carlo method #Nyquist rate #Nyquist–Shannon sampling theorem #Oversampling #Sampling (signal processing) #Signal processing #Signal reconstruction #Slice sampling #Statistics #Telecommunications #cs.IT #math.IT
paper · pdf · doi:10.1109/tit.2015.2485271
published as Information Theory, IEEE Transactions on , vol.62, no.1, pp.401-429, Jan. 2016 · Accepted for publication at the IEEE transactions on information theory
openalex publication_date 2015/10/01 · arxiv created 2015/11/06 · arxiv updated 2016/01/26 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05
The amount of information lost in sub-Nyquist sampling of a continuous-time Gaussian stationary process is quantified. We consider a combined source coding and sub-Nyquist reconstruction problem in which the input to the encoder is a noisy sub-Nyquist sampled version of the analog source. We first derive an expression for the mean squared error in the reconstruction of the process from a noisy and information rate-limited version of its samples. This expression is a function of the sampling frequency and the average number of bits describing each sample. It is given as the sum of two terms: minimum mean square error in estimating the source from its noisy but otherwise fully observed sub-Nyquist samples, and a second term obtained by reverse waterfilling over an average of spectral densities associated with the polyphase components of the source. We extend this result to multi-branch uniform sampling, where the samples are available through a set of parallel channels with a uniform sampler and a pre-sampling filter in each branch. Further optimization to reduce distortion is then performed over the pre-sampling filters, and an optimal set of pre-sampling filters associated with the statistics of the input signal and the sampling frequency is found. This results in an expression for the minimal possible distortion achievable under any analog-to-digital conversion scheme involving uniform sampling and linear filtering. These results thus unify the Shannon-Whittaker-Kotelnikov sampling theorem and Shannon rate-distortion theory for Gaussian sources.