Infographic gone bad: accuracy v truth (updated)

It landed hard in my Facebook feed.

The headline and graphic were sensational, designed to shock. After all, emotional knee-jerk is what generates shares in digital networks.

And boy did it get shares.

Until Forbes pulled it. (After some prodding, below.)

Let’s talk about the image illustrating confirmed cases of COVID-19.

 

To be clear: this is a Forbes contributor* not a Forbes staffer. Contributors have very little editorial oversight.

This writer specializes in “consumer tech” (which is a long way from epidemiological analyses). His bio states that he is a cognitive psychologist; he should know better.

The image in question is probably accurate; that is, the data are from a reputable source. (It’s not clear how these data came from Stanford because there is no obvious downloadable dataset.)

But it is not truthful: the image implies that most of Georgia is infested, that most counties are being overrun by COVID-19. It does so in two ways: choice of color but, just as importantly, choice of scale.

Ostensibly, it is showing you that much of the state has 100-250 cases per capita (100K). At this time this was written, the US as a whole had about 325 cases/100,000, which would put Georgia below the national average.

Based on that comparison, most of the state should appear light in color.

But the map gives an impression that Georgia is in horrible shape. Why? Choice of color (brown = dirty) and density (dark = bad). Also, the number at the top of the scale is 1000. Doesn’t matter that the next one down is 250. Or the next 100. That’s not how our brains process imagery.

You can see how distorted this map is by looking at the image to its right, the official map.

Although I have some issues with the arbitrary nature of the Georgia state scale as well, the colors much more closely correspond to density and national data.

What about the headline?

The headline … was even more provocative than the graphic.

The author’s analysis appears to be based on calculating the percentage of a ratio. This is not good statistical analysis. Percentages can easily confuse, but are helpful only when comparing two “wholes” (ie, two absolute numbers) with appropriate context. In this case, the author chose to treat a ratio (cases/100,000) as though it were an absolute number.

He is also extrapolating from only five days of data.

There is large day-to-day variation in coronavirus case numbers due to factors other than infections: lab backlog, shortage of swabs, shortage of lab test kits, fewer people conducting tests on Saturday and Sunday … the list is long. There is a reason that statisticians use seven-day moving averages: they are trying to reduce the false variability.

The lesson: when something seems too good or too bad to be true: stop! Don’t share without engaging the “Spock” side of your brain!

Because that thing? It’s probably not true.

Even with COVID-19.

Update: I did not feel qualified to critique the author’s fundamental analysis because I’m not an epidemiologist, but it seems to have been more fatally flawed than the infographic. So here’s another lesson: review credentials! Someone whose writing is branded “consumer tech” is probably not the most credible source of coronavirus opinion.

No. Just no. This guy is just using the total number of cases per 100,000 as measure of risk exposure. This is silly. The cumulative number of cases can't go down. It doesn't measure exposure risk. It's growth has nothing to do with re-opening If you see this, please ignore. pic.twitter.com/0ArrffHHCr

— J.C. Bradbury (@jc_bradbury) May 4, 2020

 

📣About that Forbes brand. A casual review of shares and comments reveals that most (normal) people do not know that a Forbes contributor is not a full-time staffer with an editor. I think this has damaged the brand among media-literate consumers. But we appear to be a minority.

🍑Disclaimer: that hot spot in the southwest corner of Georgia? That’s where I grew up. So I tend to get pretty hot and bothered these days because the area has been so hard hit. And is so very very poor.

* Now former Forbes contributor; @TheInfoMonkey.