Previously, we looked at the claim that 7 percent of American adults believe that chocolate milk comes from brown cows. The media, including mainstream outlets such as the Washington Post, demonstrated incompetence in reading survey results. That was before I investigated the other survey covered by the Washington Post in that article about brown cows (link). The Post should immediately apologize for their horrible reporting. For this study, the data…
Apparently, the Washington Post decided to assist the dairy industry in its latest advertising campaign by publishing a weird survey result, which claims that some adults believe that chocolate milk comes from brown cows. (link) Just start by thinking about the survey design. In order for people to express this opinion, the survey had to contain a choice of "brown cows." If they had done an open-ended survey, the result…
Kaiser Fung, founder of Principal Analytics Prep, discusses Seth Stephens-Davidowitz's new book, Everybody Lies
If you live in the States, and particularly a blue state, in the last year or two, it has been drilled into your head that Hillary Clinton was the overwhelming favorite to win the Presidential election. On the day before the election, when all the major media outlets finalized their "election forecasting models," they unanimously pronounced Clinton the clear winner, with a probability of winning of 70% to 99%. One…
In class last week, I discussed this New York Times article with the students. One of the claims in the article is that the U.S. News ranking of colleges is under threat by newcomers whose rankings are more relevant because they more directly measure outcomes such as earnings of graduates. This specific claim in the article makes me head hurt: "If nothing else, earnings are objective and, as the database…
Last week, my Columbia students discussed this nice article in the New York Times called "The Most Detailed Map of Gay Marriages in America". (link) The center of the article is this map: I asked the students to identify the problem that this dataviz is supposed to address. Someone responded that it tells us where gay married couples are found geographically. I asked her what is the answer to the…
As word of plummeting sales of Apple Watch spread around last week, an entrepreneur went on Medium to sing a different tune: his survey apparently uncovered a "paradox" - Apple Watch users are "overwhelmingly satisfied, yet not recommending" the product. The "overwhelming" bit comes from this chart: This data portray a hugely successful product in which almost nobody expressed any negative feelings. This next chart is even more impressive. Apparently,…
About half a year ago, when my article on ethics in data science appeared on Harvard Business Review, the second half of the piece was dropped. In the omitted section, I had designed a poll inviting readers to opine on several ethical dilemmas facing real-world data analysts. In April, Manoj Chari read my HBR article and kindly invited me to speak on the subject at the INFORMS Analytics Conference, and…
This piece is part of the StatBusters column written jointly with Andrew Gelman. Hope they fix the labeling soon. In it, we talk about two recent studies on data privacy, which leads to contradictory conclusions. How should the media report such surveys? Is the brand name of the organization enough? In addition, we debunk the notion that consumers will definitely get something valuable out of sharing their data.
This op-ed by Richard Thaler caught my attention because I have a similar experience. In my statistics classes, I have noticed a pattern: if the mid-term exam is hard, with a lower average score (say 75-80%), the students look crestfallen and feel that they did not learn; eventually, when it comes to evaluating the instructor, I receive lower grades, with comments indicating that I have not taught them properly to…
