At the start of the year, The Atlantic published a very nice, long article about Netflix's movie recommendation algorithm. You may remember this algorithm (internally known as Cinematch) received a $1 million makeover several years ago (the Netflix Prize), only that the prize-winning entry was deemed too complex--and does not generate sufficient incremental value--to be put into production. The reporter, Alexis Madrigal, noticed that Netflix has shifted attention from the…
Avinash (Web Analytics 2.0) is fond of animal metaphors. I think he's the one who coined HIPPOs (Highest Paid Person's Opinion). Now he's come up with Reporting Squirrels and Analysis Ninjas. See his recent post here. In short, he is shouting about "return on analytics", a really, really important thing. What is being lost in the hype of Big Data is that all the investment in analytics has to generate…
Just a little while ago, I showed an example of imprecise algorithms and how it causes incorrect historical facts to be promulgated. The point is not that algorithms are scary things but that we should not confuse efficiency with accuracy (or truth). So this past week, I have another encounter with imprecise machines, and this time, it's personal. *** If you go to Amazon right now, and search for my…
The mass media continues to gloss over the imprecision of machines/algorithms. Here is another example I came across the other day. In conversation, the name Martin Van Buren popped up. I was curious about this eighth President of the United States. What caught my eye in the following Google search result (right panel) is his height: Mr. Van Buren was very short, only 5 feet tall. I was about to…
The three benchmark algorithms for predicting the location of dark matter halos are, for the most part, all over the map. Most of the test skies look something like this: There are, however, some skies with rather strong halo signals that get a decent amount of agreement: The Lenstool MLE algorithm is the current state
![ISBA towards higher computing goals [yet another new section!!!]](http://xianblog.files.wordpress.com/2012/09/img_1299-e1348083924787.jpg?w=450&h=236)
Surrounding the great and exciting gathering of Bayesian statisticians in Kyoto last June, several ISBA sections have appeared in the past weeks, as already mentioned on the ‘Og. Along with Anto Mira and Nicolas Chopin (who did most of the organisational work while I was wandering down under!), we discussed about a Bayesian computation section [...]
I recently participated in the Kaggle Algorithmic Trading Competition under the username VikP. For those who do not know what Kaggle is, it is a web site where individuals and corporations can host data analysis competitions. This particular competit...
I recently participated in the Kaggle Algorithmic Trading Competition under the username VikP. For those who do not know what Kaggle is, it is a web site where individuals and corporations can host data analysis competitions. This particular competition involved the prediction of how the prices of 50,000 observations of 102 different securities at the tick level recovered after both buyer and seller initiated liquidity shocks. Each competitor was provided…
