# Machine Learning Lesson of the Day – Parametric vs. Non-Parametric Models

January 14, 2014 7 Comments

A machine learning algorithm can be classified as either parametric or non-parametric.

A **parametric** algorithm has a **fixed number of parameters**. A parametric algorithm is **computationally faster**, but makes **stronger assumptions** about the data; the algorithm may work well if the assumptions turn out to be correct, but it may perform badly if the assumptions are wrong. A common example of a parametric algorithm is **linear regression**.

In contrast, a **non-parametric** algorithm uses a **flexible number of parameters**, and the number of parameters often **grows as it learns from more data**. A non-parametric algorithm is **computationally slower**, but makes **fewer assumptions** about the data. A common example of a non-parametric algorithm is **K-nearest neighbour**.

To summarize, the **trade-offs** between parametric and non-parametric algorithms are in **computational cost** and **accuracy**.

Reblogged this on victoria anugrah and commented:

It clears my mind about non-parametric methods. 🙂

Glad to hear that, Victoria! Thanks for reading!

Pre-exam day and your easy explanation! Kudos.

Glad to hear that, Krupali! Hope your exam went well!

I would like to point out that “linear regression” is not an algorithm, it’s a widely known technique in statistics and not machine learning. We could would be talking about “machine learning” in linear regression if we were to perform parameter estimation with something like a genetic algorithm instead of the common OLS method. But since this is not specified, it’s incorrect to regard linear regression as machine learning. Generally, there are no true parametric algorithms in machine learning, and that’s what differentiates it to statistics. And before someone mentions Artificial Neural Networks, it should be clear that they are NOT parametric in the statistical sense because their parameters do not explain anything about the model and their model is a black box.

I don’t like to engage in deep disagreements about definitions, so I won’t respond in detail. I disagree with your statement, and a counter-example to your denial of parametric models in machine learning is LASSO.

Pingback: Deep dive into Supervised Learning – Catherine Gitau