Analysis of Continuous-Time LMS Adaptive Filter Weights Using Stochastic Calculus and Fokker-Planck Kolmogorov Equation

Abstract

The LMS technique uses a simple approximation to the gradient to update it’s filter weight. This approximation, known as the noisy gradient, introduces a jitter into the LMS weight adaptation process and this jitter is present even at convergence. Consequently, the continuoustime LMS filter weights are stochastic processes, having time varying probability density function during the adaptation phase and a stationary probability density function after the filter has converged. In this paper, the probability density function of the continuous-time LMS adaptive filter weights is obtained. The LMS weight update is formulated as a stochastic differential equation for the system identification problem and the weight probability density function is next derived using a partial differential equation known as the Fokker-Planck Kolmogorov equation. Closed form solution is obtained for the steady state probability density function for the LMS weights. Mean and variance is also obtained in closed form directly from the stochastic differential equation for the LMS weights.