Sparse model construction using coordinate descent optimization

Hong, X. ORCID: https://orcid.org/0000-0002-6832-2298, Guo, Y., Chen, S. and Gao, J. (2013) Sparse model construction using coordinate descent optimization. In: Proceedings: 18th International Conference on Digital Signal Processing (DSP2013), 2013-07-01 - 2013-07-03, Santorini - Greece.

Abstract/Summary

We propose a new sparse model construction method aimed at maximizing a model’s generalisation capability for a large class of linear-in-the-parameters models. The coordinate descent optimization algorithm is employed with a modified l1- penalized least squares cost function in order to estimate a single parameter and its regularization parameter simultaneously based on the leave one out mean square error (LOOMSE). Our original contribution is to derive a closed form of optimal LOOMSE regularization parameter for a single term model, for which we show that the LOOMSE can be analytically computed without actually splitting the data set leading to a very simple parameter estimation method. We then integrate the new results within the coordinate descent optimization algorithm to update model parameters one at the time for linear-in-the-parameters models. Consequently a fully automated procedure is achieved without resort to any other validation data set for iterative model evaluation. Illustrative examples are included to demonstrate the effectiveness of the new approaches.