DMS Applied Mathematics Seminar

Speaker: Prof. Xiaojing Ye, Georgia State University

Title: Decentralized consensus optimization on networks with delayed and stochastic gradients

Abstract: Decentralized consensus optimization has extensive applications in many emerging big data, machine learning, and sensor network problems. In decentralized computing, nodes in a network privately hold parts of the objective function and need to collaboratively solve for the consensual optimal solution of the total objective, while they can only communicate with their immediate neighbors during updates. In real-world networks, it is often difficult and sometimes impossible to synchronize these nodes, and as a result they have to use stale and stochastic gradient information which may steer their iterates away from the optimal solution. In this talk, we focus on a decentralized consensus algorithm by taking the delays of gradients into consideration. We show that, as long as the random delays are bounded in expectation and a proper diminishing step size policy is employed, the iterates generated by this algorithm still converge to a consensual optimal solution. Convergence rates of both objective and consensus are derived. Numerical results on some synthetic optimization problems and on real seismic image reconstruction will also be presented.