A radial basis function network classifier to maximise leave-one-out mutual information

Hong, X. ORCID: https://orcid.org/0000-0002-6832-2298, Chen, S., Qatawneh, A., Daqrouq, K., Sheikh, M. and Morfeq, A. (2014) A radial basis function network classifier to maximise leave-one-out mutual information. Applied Soft Computing, 23. pp. 9-18. ISSN 1568-4946 doi: 10.1016/j.asoc.2014.06.003

Abstract/Summary

tWe develop an orthogonal forward selection (OFS) approach to construct radial basis function (RBF)network classifiers for two-class problems. Our approach integrates several concepts in probabilisticmodelling, including cross validation, mutual information and Bayesian hyperparameter fitting. At eachstage of the OFS procedure, one model term is selected by maximising the leave-one-out mutual infor-mation (LOOMI) between the classifier’s predicted class labels and the true class labels. We derive theformula of LOOMI within the OFS framework so that the LOOMI can be evaluated efficiently for modelterm selection. Furthermore, a Bayesian procedure of hyperparameter fitting is also integrated into theeach stage of the OFS to infer the l2-norm based local regularisation parameter from the data. Since eachforward stage is effectively fitting of a one-variable model, this task is very fast. The classifier construc-tion procedure is automatically terminated without the need of using additional stopping criterion toyield very sparse RBF classifiers with excellent classification generalisation performance, which is par-ticular useful for the noisy data sets with highly overlapping class distribution. A number of benchmarkexamples are employed to demonstrate the effectiveness of our proposed approach.