Convolutional approximations to linear dimensionality reduction operators

This paper examines the existence of efficiently implementable approximations of a general real linear dimensionality reduction (LDR) operator. The specific focus is on approximating a given LDR operator with a partial circulant structured matrix (a matrix whose rows are related by circular shifts) as these constructions allow for low-memory footprint and computationally efficient implementations. Our main contributions are theoretical: we quantify how well general matrices may be approximated (in a Frobenius sense) by partial circulant structured matrices, and also consider a variation of this problem where the aim is only to accurately approximate the action of a given LDR operator on a restricted set of inputs. For the latter setting, we also propose a sparsity-regularized alternating minimization based algorithm for learning partial circulant approximations from data, and provide experimental evidence demonstrating the potential efficacy of this approach on real-world data.