## Screening: It’s all in the numbers

Bayesian analysis questions how we understand the notion of ‘probability’ and how we update our beliefs in light of new information

You’re a woman in her early fifties. You’re invited to a breast cancer screening unit, and you go along hoping for the all-clear. After all, 99 per cent of women your age do not have breast cancer. But … the scan is positive. The screening process catches 85 per cent of cancers. There is a chance of a false alarm, though: for 10 per cent of healthy women, the screening process wrongly points to cancer. What are the chances that you have breast cancer?

Over 50,000 British women face this awful question each year. I first encountered it – in a less alarming context – as an undergraduate economist. And I was in the audience recently when David Spiegelhalter used it as an example in his Simonyi Lecture, “Working Out the Odds (With the Help of the Reverend Bayes)”. The numbers approximately reflect the odds faced by women who go for breast cancer screening. And the answer – courtesy of the Reverend Bayes in question, who died 250 years ago – is surprising.

Bayes was concerned with how we should understand the notion of “probability”, and how we should update our beliefs in light of new information.

A Bayesian perspective on the apparently grim screening result tells us that things are not as bad as they seem. The two key pieces of information point in different directions. On the one hand, the positive scan substantially worsens the odds that you have cancer. But on the other, the odds are worsening from an extremely favourable starting point: 99 to 1 against. Even after the positive scan, you still probably don’t have cancer.

Imagine 1,000 women in your situation: 990 do not have cancer, which means we can expect 99 false positives, far more than the 10 women who do have cancer. This is why any apparent sign of cancer should be followed up with further tests in the hope of avoiding unnecessary treatments. The chance that you have cancer is 8 per cent ~~9 per cent~~ – up dramatically from 1 per cent, but with plenty of room for optimism.

None of this proves screening is pointless. It can save lives, but it raises dilemmas. The UK’s breast cancer screening programme is currently under review. A systematic analysis published by the Cochrane Collaboration found that for every woman who had her life extended by early detection and treatment, there would be 10 courses of unnecessary treatment in healthy women, and more than 200 women would experience distress as the result of a false positive.

Bayesian reasoning has implications far beyond cancer screening, and we are not natural Bayesians. Daniel Kahneman, a psychologist who won the Nobel memorial prize in economics, discusses the issue in a new book, Thinking, Fast and Slow. I recently had the opportunity to quiz him in front of an audience at the Royal Institution in London. Kahneman argues that we often ignore baseline information unless it can somehow be made emotionally salient. New information – “possible cancer” – tends to monopolise our attention.

Another example: if somebody reads the Financial Times, should you conclude that they are more likely to be a quantitative analyst in an investment bank, or a public sector worker? Before you leap to conclusions, remember that there are six million public sector workers in the country. Base rates matter.

Sometimes there is no objective base rate and we must use our own judgment instead. I think homeopathy is absurd on theoretical grounds; others find it intrinsically plausible. Bayesian analysis tells us how to combine those prior beliefs – or prejudices – with whatever new evidence may come along.

Whenever you receive a piece of news that challenges your expectations, it’s tempting either to conclude that everything has changed – or that nothing has. Bayes taught us that there’s a rational path between those two extremes.

*Also published at ft.com.*

^{th}of December, 2011 • Undercover Economist • Comments off

## 9 Comments

Ian Machan says:Can this approach be explicitly linked to the double egg experiment recently on MOL?

10^{th}of December, 2011Eric A Blair says:Great article Tim. This is the kind of thing that more people should understand and as always you’re doing a great job of ensuring that happens. I do however want to take exception to the penultimate paragraph in this article. The idea that Bayesian analysis can tell us something about whether homeopathy works lies on extremely shaky philosophical ground. Bayesian analysis can be and is applied to historical subjects such as archeology, but it does not follow that it can be applied to scientific questions and in fact most scientists would say that is cannot. In particular here is a video http://research.microsoft.com/apps/tools/tuva/index.HTML (the last one) of the legendary physicist Richard Feynman disagreeing with the notion. In my opinion such ideas as you put forward here are extremely dangerous since they fundamentally undermine the notion of science itself. Please be more careful in the future.

10^{th}of December, 2011Jose Camoes Silva says:Regarding the FT example, you’re getting pretty close to the same problem as Vanity Fair:

Let’s say there are 1000 times more public servants than bankers; all bankers read FT, while among public servants only the upper echelons (Sir Humphrey Appleby and colleagues) do, say 1 in 10,000. This would make the FT an indicator for investment bankers, even taking base rates into consideration.

(I analyzed the Vanity Fair piece mistake here: http://sitacuisses.blogspot.com/2011/11/how-to-misunderstand-technical-material.html )

Cheerio,

10JCS

^{th}of December, 2011bassma says:its like combining the extremes and dividing’em by 2

11^{th}of December, 2011Tom Dawkes says:Tim, I love your More or less programme and the Undercover Economist. But this is just a little quibble about usage: it’s fairly common to use Reverend with only a surname – as in Reverend Bayes – but the traditional, and I’d say, preferable, formula is Reverend with forename and surname, so Reverend Thomas Bayes.

Best wishes

Tom Dawkes

11^{th}of December, 2011Ryan Harding says:A nice example, particularly because it’s subject to an emotional reaction and psychological framing that easily overrides the coldly rational Pascal/Bayes approaches and can lead to bad decision-making. Knowing how to present the information is rather more important in this distressing context than it is in the context of, say, salesmanship. This is probably an even more important place to apply thinking aligned with Robert Cialdini’s ‘Influence’ principles (which draw on Kahneman & Tversky’s work among others’) than in preventing no-shows in medical appointments by inducing voluntary commitment and consistency from the patient (which is also valuable in the NHS, as it is in New York restaurant reservation-taking). I guess the exact approach to presenting results could also be trialled randomly (a nod to Adapt) to find the one that results in the statistically best outcome in terms of encouraging a rational response (not too distressed, but sufficiently motivated to progress towards treatment if necessary)

I guess the best approach in these exact circumstances is to tell the woman who screened positive that the initial screening results put her at about a 9 per cent chance of having breast cancer at this time (compared to a base rate of 1% before screening for women her age) and that further investigation is necessary. The consequences of delaying further investigation should also be spelled out, given that treatment is more effective if done promptly than done late.

It would be helpful to know the statistics that result if all 109 out of 1000 positive-result women were retested to determine the statistical independence of separate tests. How many women would get a false positive then and how many with cancer would get a false negative, or what is the best test for further investigation (e.g. another mammogram or a biopsy)? If a second mammogram were helpful, would it be beneficial to re-screen the cleared women on a shorter schedule to try to catch any false negatives?

The answer (as with all the percentages above) might indeed vary with age and family history, but the Bayesian approach can be applied based on historical data.

With that sort of information it would be possible to paint a picture (possibly even a diagram) of a humanly-understandable number of women who had the same results and how many in that group would be fine and how many would have breast cancer.

Also important is using the false negative rate to press the point that women given the all-clear this time should return for regular screening again when called a few years down the line.

11^{th}of December, 2011Jason says:Small point of difference in the probability calculation. I calculate 7.9% vs 9% mentioned in the article.

Out of 1000 tested, 10 have cancer but only 8.5 on average will test positive. 990 do not have cancer, out of which 99 will test positive. So likelihood of having cancer with a positive test is 8.5/(8.5 + 99)= 7.9%.

Have I missed anything?

—-

Tim H writes: No, you’re correct Jason. Sorry for the error. Now fixed above.

12^{th}of December, 2011Robert Nowak says:What is the harm in an article involving math, such as by Tim H, to show the calculation and carry the first decimal point? Jason shows how he got 7.9%, However Tim H then in his correction rounds up to 8%. Not that rounding up is not Ok, but carrying the first decimal point makes it easier to reproduce the calculation. Specifically, for example, why did Tim H not show how he got 9%?

14^{th}of December, 2011Tony Durham says:Interesting piece! Compare and contrast the ‘Bayesian brain’ neuroscientists like Karl Friston, who believe Bayesian inference is basic to the way our brains work.

Having said that, obviously Daniel Kahneman must be brilliant or he wouldn’t be getting all this media attention. That’s until you realise he’s got a new book out…

16^{th}of December, 2011