(This article was originally published at The Chemical Statistician » Statistics, and syndicated at StatsBlogs.)
A machine learning algorithm can be classified as either parametric or non-parametric.
A parametric algorithm has a fixed number of parameters. A parametric algorithm is computationally faster, but makes stronger assumptions about the data; the algorithm may work well if the assumptions turn out to be correct, but it may perform badly if the assumptions are wrong. A common example of a parametric algorithm is linear regression.
In contrast, a non-parametric algorithm uses a flexible number of parameters, and the number of parameters often grows as it learns from more data. A non-parametric algorithm is computationally slower, but makes fewer assumptions about the data. A common example of a non-parametric algorithm is K-nearest neighbour.
To summarize, the trade-offs between parametric and non-parametric algorithms are in computational cost and accuracy.
Filed under: Machine Learning, Machine Learning Lesson of the Day, Statistics Tagged: computational cost, computational speed, K-nearest neighbour, linear regression, machine learning, non-parametric, parametric
Please comment on the article here: The Chemical Statistician » Statistics
