| Recursive Least Squares for an Entropy Regularized MSE Cost Function (2008) | |||||||||||||
Abstract | |||||||||||||
| Abstract. Minimum MSE plays an indispensable role in learning and adaptation of neural systems. Nevertheless, the instantaneous value of the modeling error alone does not convey sufficient information about the accuracy of the estimated model in representing the underlying structure of the data. In this paper, we propose an extension to the traditional MSE cost function, a regularization term based on the incremental errors in model output. We demonstrate the stochastic equivalence between the proposed regularization term and the error entropy. Finally, we derive an RLS-type algorithm for the proposed cost function, which we call recursive least squares with entropy regularization (RLSER) algorithm. The performance of RLSER is shown to be better than RLS in supervised training with noisy data. 1. | |||||||||||||
Details der Publikation | |||||||||||||
| |||||||||||||