Nice article in the New York Times about the "overdiagnosis" problem in cancer screening. The particular case is thyroid cancer in South Korea. There are a number of things about any form of screening tests that one should always bear in mind: Death rate is measured as the number of deaths divided by the number of people with the disease. The latter number increases with better diagnosis techniques. Better diagnosis…
There is now some serious soul-searching in the mainstream media about their (previously) breath-taking coverage of the Big Data revolution. I am collecting some useful links here for those interested in learning more. Here's my Harvard Business Review article in which I discussed the Science paper disclosing that Google Flu Trends, that key exhibit of the Big Data lobby, has systematically over-estimated flu activity for 100 out of the last…
Note: This is part 2 of a three-part response to an important article that appeared in the New York Times in August. See Part 1 here. *** Part 2 deals with a few disagreements I have with Tim Rohan, who is the author of the article. The first sentence of Rohan's article took my breath away: Doping experts have long known that drug tests catch only a tiny fraction of…
Note: This is part 1 of a three-part response to an important article that appeared in the New York Times this month. Doping in elite sports has become a taboo topic in sports circles as no one wants to kill the golden egg. This article will prove to be an important reference. *** Tim Rohan's article is a must-read for anyone who cares about the sanctity of sports (link). I…
Steroids continue to plague professional sports. The latest name to fall is Alex Rodriguez, the shortstop/3rd baseball superstar who currently plays for the Yankees. It wasn't long ago that he was considered a "good guy" of the sport. Now, he's a pariah. In the rush to make Rodriguez the villain, the media continues to miss these two important aspects to the steroids story: Anti-doping tests have a huge false-negative problem.…
When you hear about Big Data, you almost always hear about the supply side: Behold the data in un-pronounceable units of bytes! Admire the new science inspired by all the data! Missing from this narrative is the consumption side. A direct consequence of Big Data will be the explosion of data analyses—there will be more people producing more data analyses more quickly. This will be a world of confusing and…
Mammograms continue to be an emotional and controversial topic. I blogged about it some time ago. (link) Felix Salmon, whose blog should be daily reading, praises an article by Peggy Orenstein called "Our Feel Good War on Breast Cancer", NYT Magazine (link). Salmon's blog provides a quick summary; Orenstein's article is very long. Orenstein's point of view has particular weight because she was diagnosed at an early age, and was…
That is the title of Chapter 4 of Numbers Rule Your World. When the book came out two years ago, lots of steroids scandals have broken out, and yet many people were in deep denial. Recall the saga of Floyd Landis, the U.S. cyclist, who initially claimed the drug test was wrong, and that he was witch-hunted. There was even a website called Trust But Verify where all his supporters…
Readers of Chapter 4 of Numbers Rule Your World would not be surprised about the fall of Lance Armstrong. The big theme of the chapter is that any screening tests has to balance between false positives and false negatives. The anti-doping agencies are so concerned about not falsely accusing anyone that they leave a gigantic hole for dopers to walk through. Strings of negative tests, therefore, do not prove one…
I was a guest on the Analytically Speaking series, organized by JMP. In this webcast (link, registration required), I talk about the coexistence of data science and statistics, why my blog is called "Junk Charts", what I look for in an analy...
