Machine Learning Lesson of the Day – K-Nearest Neighbours Regression

(This article was originally published at The Chemical Statistician » Statistics, and syndicated at StatsBlogs.)

I recently introduced the K-nearest neighbours classifier.  Some slight adjustments to the same algorithm can make it into a regression technique.

Given a training set and a new input , we can predict the target of the new input by

1. identifying the K data (the K “neighbours”) in the training set that are closest to by Euclidean distance
2. build a linear regression model to predict the target for

• the K data are the predictors
• the reciprocals of the predictors’ distances to are their respective regression coefficients (the “weights”)

Validation or cross-validation can be used to determine the best number of “K”.

Please comment on the article here: The Chemical Statistician » Statistics

Tags: cross-validation, Euclidean Distance, K-nearest neighbours regression, KNN, KNN regression, linear regression, Machine learning, Machine Learning Lesson of the Day, statistics, training set, validation