Off the rails when the figures don’t add up

‘Three officials at the Department for Transport were suspended on Wednesday after the award of a new contract to run the West Coast mainline rail franchise was cancelled because of “technical flaws” in the bidding process.’

FT.com, October 3

What are these “technical flaws”?

Good question. The way this works is that companies bid for the right to operate train services on particular routes. These rights are lucrative, which is why the government sells them at auction.

How hard can it be to compare two bids and see which one is larger? This is the kind of maths question we ask four-year-olds.

There are complications that put this way beyond the grasp of a four-year-old and into the intellectual realm of, say, a competent first-year undergraduate in economics or accounting. One is that the government can’t sell the rights to the highest bidder, because it needs to consider whether the trains will actually run. The East Coast mainline had to be nationalised three years ago when the winner walked away from the franchise.

The second problem is that the government doesn’t get the money straight away and must consider how to compare cash immediately with cash promised in a decade’s time. In the contest between Virgin Trains and FirstGroup, the latter offered more cash but a lot of that was due well into the future. FirstGroup was awarded the West Coast contract on that basis but the Department for Transport has now thought better of that decision.

It still doesn’t sound that hard.

No. Admittedly there are some profound issues about how to think about uncertain future events. But the department mandarins do not appear to have fallen down any deep philosophical rabbit holes. It sounds as though they either ignored the issue entirely or botched it on a basic level. The transport secretary said the problem was “the way in which inflation and passenger numbers were taken into account”, which if true is fairly elemental.

Aren’t such basic errors unforgivable?

There’s a more worrying question: given how complicated the modern world is, aren’t such basic errors inevitable? This seems to have been a howler, but large spreadsheets are ubiquitous and their size makes it almost impossible to eliminate errors. The Office for National Statistics misreported gross domestic product last year thanks to such an error.

Doesn’t this tell us more about civil servants than spreadsheets?

These are particularly egregious errors, but the private sector is hardly immune. For instance, in 2003 Fannie Mae made a spreadsheet error that led to it misstating its results by more than $1bn.

That’s really not the worst screw-up to affect Fannie Mae.

True, the credit crunch was worse. But there were some mathematical errors involved in that, too: the infamous “Gaussian copula” was an attempt to figure out how to account for correlations between assets, such as different subprime mortgages. Unfortunately, correlations are slippery and if you bet a few hundred billion dollars that they are going to behave themselves, you have a problem.

The whole thing was a slow-motion car crash.

Or a slow-motion air crash? An Air Canada flight ran out of fuel in mid-air in 1983. It was a pounds-or-kilograms thing. Luckily the captain was a glider pilot and managed to land the jet without killing anybody.

You’re starting to make me nervous.

Denial isn’t helpful. We need to accept modern life is full of mathematical, spreadsheet and programming errors. Two famous “man versus machine” matches were affected by such errors. Marion Tinsley, the greatest draughts player in history, was persuaded to accept a draw in one match in 1992 when his silicon foe, Chinook, announced the game was drawn according to its endgame database. It later transpired the database was corrupted and Chinook had no idea what it was doing. Garry Kasparov beat Deep Blue in a game in 1997, but misinterpreted a bizarre move by the computer as an effort to avoid “mate in 20” – suggesting the machine had a terrifying ability to look ahead. In fact, Deep Blue typically looked ahead six to eight moves. The random move was just that: random, the response to a software bug. Mr Kasparov’s confidence was shattered; he should have considered the possibility that even Deep Blue suffers from the occasional “technical flaw”.

Also published at ft.com.