As part of AVADO’s digital transformation series, we’re exploring a range of fascinating TED talks which highlight the growing role of technology and data in shaping our future: giving you all you need to know to tackle digital transformation head-on. Today, we look at Mona Chalabi’s TED talk, ‘How to spot a bad statistic’.

How can you tell a good statistic from a bad statistic? And why should you believe in statistics at all, given the wealth of half-truths out there which are seemingly supported by data, albeit poor data?

In her TED talk, data journalist Mona Chalabi outlines why statistics still have value, and how to differentiate between those which are trustworthy and those which aren’t.

What’s the issue?

Nowadays, the US population are questioning statistics that come directly from their government; claims for example which say that the unemployment rate is currently at 5%.

Chelabi tells us how roughly 4 out of 10 American distrust the government’s data; with Trump supporters, this number rises to 7 out of 10.

Why is this a problem?

Data informs how politicians go about creating policies; it measures societal issues like inequality in a fully objective way. However, there’s been a backlash against this data-lead approach; there’s currently a bill in the US Congress which says that government money shouldn’t be used to gather representative data on racial segregation. This draft law would in effect stop racial discrimination being measured at all.

In fact, the name statistics comes from the word state. Without statistics on health and poverty, you can’t change healthcare; without data on how many people are entering and leaving the country, you can’t properly legislate on immigration. Statistics have always been used by governments to help them serve society at large.

How should you assess statistics? Can you see uncertainty?

Polls are fundamentally flawed in a number of ways: you’ll never get an exact breakdown of increasingly-diverse populations, people often lie, and plenty of people don’t even respond to pollsters. Polling companies, however, will produce sleek charts which only serve to iron out any uncertainty, and make the predictions seem wholly objective when in reality they are just educated guesswork.

Chelabi has a potential solution. Instead of charts, statistics really need to come in the form of visualisations. Shaky lines give people an overview of the data rather than a specific number and help reinforce statistical concepts like averages. For example, whilst there’s an average of 6.23 fecal accidents in a US swimming pool, that’s definitely not to say that every pool contains 6.23 pieces of poo!

And how was the data collected?

A frequently-cited poll in 2015 found that 41% of Muslims in the US supported jihad: however, what wasn’t mentioned was that a) most of them defined jihad as a personal, peaceful struggle to be more religious, b) it was an opt-in poll online (so anyone had the chance to answer it), and c) there were 600 respondents, out of a nation-wide three million Muslims. In context, it’s a very different story.

A role we can all play

Statistics are not reliable when taken verbatim – instead, they indicate macro trends. You should question them, insist on checking over everything and interrogate their origin. How did they collect the data? Are there parts of the chart you aren’t seeing, and if so why not?

Chelabi is adamant that dismissing all statistics isn’t constructive. Without objective data we’ll be making key decisions without the context of reality, and that certainly doesn’t bode well for the future.

Ready to improve data literacy in your organisation?

At AVADO’s Data Academy, we work to develop community programmes, so data literacy becomes top-of-mind and seeps into daily consciousness.

Interested in learning more about how the Data Academy can help your organisation and about our free levy-funded programmes? Give us a call on +44 (0)20 3893 5401 or email us at solutions@avadolearning.com.