Qiang Sun (University of Toronto)

Abstract: Statistical optimization has received quite some interests recently. It refers to the case where hidden and local convexity can be discovered in most cases for nonconvex problems, making polynomial algorithms possible. It relies on careful analysis of the geometry near global optima. In this talk, I will explore this direction by focusing on sparse regression problems in high dimensions. A computational framework named iterative local adaptive majorize-minimization (I-LAMM) is proposed to simultaneously control algorithmic complexity and statistical error. I-LAMM effectively turns the nonconvex penalized regression problem into a series of convex programs by utilizing the locally strong convexity of the problem when restricting the solution set in an l1 cone. Computationally, we establish a phase transition phenomenon: it enjoys linear rate of convergence after a sub-linear burn-in. Statistically, it provides solutions with optimal statistical errors. Extensions to robust regression will be discussed.