Machine Learning Lesson of the Day – Introduction to Linear Basis Function Models
March 10, 2014 Leave a comment
Given a supervised learning problem of using inputs () to predict a continuous target , the simplest model to use would be linear regression. However, what if we know that the relationship between the inputs and the target is non-linear, but we are unsure of exactly what form this relationship has?
One way to overcome this problem is to use linear basis function models. These models assume that the target is a linear combination of a set of basis functions.
This is a generalization of linear regression that essentially replaces each input with a function of the input. (A linear basis function model that uses the identity function is just linear regression.)
The type of basis functions (i.e. the type of function given by ) is chosen to suitably model the non-linearity in the relationship between the inputs and the target. It also needs to be chosen so that the computation is efficient. I will discuss variations of linear basis function models in a later Machine Learning Lesson of the Day.