Complete dictionary recovery over the sphere

We consider the problem of recovering a complete (i.e., square and invertible) dictionary A₀, from Y = A₀X₀ with Y R^n×p. This recovery setting is central to the theoretical understanding of dictionary learning. We give the first efficient algorithm that provably recovers A₀ when X₀ has O (n) nonzeros per column, under suitable probability model for X₀. Prior results provide recovery guarantees when X₀ has only O (√n) nonzeros per column. Our algorithm is based on nonconvex optimization with a spherical constraint, and hence is naturally phrased in the language of manifold optimization. Our proofs give a geometric characterization of the high-dimensional objective landscape, which shows that with high probability there are no spurious local minima. This invited talk summarizes these results, presented in [1]. It also presents numerical experiments demonstrating their implications for practical problems in representation learning and the more general algorithmic problem of recovering matrix decompositions with structured factors.