Machine Learning Lesson of the Day – Parametric vs. Non-Parametric Models
January 14, 2014 6 Comments
A machine learning algorithm can be classified as either parametric or non-parametric.
A parametric algorithm has a fixed number of parameters. A parametric algorithm is computationally faster, but makes stronger assumptions about the data; the algorithm may work well if the assumptions turn out to be correct, but it may perform badly if the assumptions are wrong. A common example of a parametric algorithm is linear regression.
In contrast, a non-parametric algorithm uses a flexible number of parameters, and the number of parameters often grows as it learns from more data. A non-parametric algorithm is computationally slower, but makes fewer assumptions about the data. A common example of a non-parametric algorithm is K-nearest neighbour.
To summarize, the trade-offs between parametric and non-parametric algorithms are in computational cost and accuracy.