Marginalia

Book of the Week 24 – Dark Data by David Hand

What is Dark Data?

Consider the example of the 1986 Challenger shuttle disaster. Should the shuttle launch, despite fears that low temperatures might weaken the “o-ring” seals? A graph of seven previous occasions when the o-rings had been stressed showed no relationship between temperature and the degree of damage.

Alas, what was missing was the data from all the launches where there had been no damage at all: in each case, air temperatures were higher. On viewing all the data, a clear relationship was visible.

“Dark data are data you do not have,” writes Hand. “They might be data you thought you had, or hoped to have, or wished you had. But they are data you don’t have. You might be aware that you do not have them, or you might be unaware.”

This book, then has some commonalities with Caroline Criado Perez’s Invisible Women, a book about systematic failure to collect data about women or issues that might be of particular relevance to women. Hand’s book is a shade more technical, but that is not the only difference: as the Challenger example makes clear, there are many different kinds of missing data, and many reasons why we might fail to have them. (The most mundane of all: time series data when some of the data points lie in the future…)

I enjoyed this book a lot; it is well-written and accesible, although mostly aimed at practioners.

UK: Blackwells Amazon

US: Powell’s Amazon

My NEW book The Next Fifty Things That Made the Modern Economy is NOW OUT. Details, and to order on Hive, Blackwells, Amazon or Watersones. Bill Bryson comments, “Endlessly insightful and full of surprises — exactly what you would expect from Tim Harford.”

Book of the Week 23 – How To Be An Antiracist by Ibram X Kendi

‘”Racist” and “antiracist” are like peelable name tags that are placed and replaced based on what someone is doing or not doing, supporting or expressing in each moment. These are not permanent tattoos. No one becomes a racist or antiracist. We can only strive to be one or the other.’

Just one of the ideas that stuck with me from Kendi’s thought-provoking polemic. It seemed original (it was certainly new to me) and useful – a way of moving beyond the sterile attempts to peer into each others’ souls and pronounce ourselves and our allies non-racist, while some despicable opponent is, of course, racist. Better to judge by actions.

Kendi begins the book with an account of himself as a young man taking part in a public speaking competition. While he explicitly rejects what he thought and said back then, he certainly hasn’t rejected the toolkit of the debater. He argues cleverly, persuasively, and like a good debater, is happy to take a logical short-cut when it suits him. For example: yes, many racists say “I am not a racist” – but it does not follow, as Kendi repeatedly implies, that “not-racism” is inevitably combination of racism plus lies.

(A personal aside: one of the reasons I picked up How To Be An Antiracist was my sense that I had no idea of the issues involved. I had that generic liberal sense that racism was repugnant and that everyone regardless of skin colour or ethnic heritage should be free to fulfil their potential, free from fear, oppression or discrimination. But it was dawning on me that I really hadn’t a clue what – for example – a young black man in America might have to deal with. Ironic, then, that Kendi’s first anecdote is of him doing pretty much what I was doing when I was the same age: standing on stage and spouting off.)

While I did not agree with everything, I learned a lot from this book – both the personal reflections and the argument. I found it particularly useful to have read Caroline Criado Perez’s excellent Invisible Women; the two books together helped me understand how much damage can be done by the unthinking default to male – or, in Kendi’s book, the unthinking default to white. This made it easier for me to engage with Kendi’s assertion that every policy is either racist or antiracist: what seemed at first a strange exaggeration made sense in the light of Crado Perez’s arguments. Sure, there almost certainly are policies that are neither racist not antiracist, but it’s worth explicitly asking yourself, “what is the likely impact of this policy or idea on different racial groups?”

UK: Amazon Blackwells

US: Amazon Powell’s

My NEW book The Next Fifty Things That Made the Modern Economy is NOW OUT. Details, and to order on Hive, Blackwells, Amazon or Watersones. Bill Bryson comments, “Endlessly insightful and full of surprises — exactly what you would expect from Tim Harford.”

Book of the Week 21: Sleight of Mind by Matt Cook

Sleight of Mind offers “75 ingenious paradoxes in mathematics, physics and philosophy” – and there are many classics here – the Monty Hall problem; the Hilbert hotel; Feynman’s sprinkler; Achilles and the Tortoise.

I was expecting a rather breezy discussion of the puzzles and paradoxes – along the lines of Julian Baggini’s popular book The Pig That Wants To Be Eaten – but it’s actually WAY more technical than I anticipated. I studied Godel’s incompleteness theorem, Arrow’s general possibility theorem and Cantor’s diagonalisation argument at university – none of them in introductory classes. I did not find Matt Cook’s treatment to be dumbed down relative to that level – it was more like being taken back to a classroom surrounded by a small number of very intimidating mathematicians…

Cook’s treatment of the problems is perfectly lucid but for those without a background in mathematics or logic it is going to require some very hard thinking.

All that said: there’s a lot to enjoy here, if you feel up to it. And if you’re studying a relevant degree this is a great resource.

An alternative for the generaly reader is Raymond Smullyan’s wonderful book of puzzles, What Is The Name Of This Book? It starts gently, one brain-teaser at a time, but by the end of the book you’ve gone deep into powerful ideas of logic and mathematics.

UK: Blackwells   Amazon

US: Powells   Amazon

My NEW book The Next Fifty Things That Made the Modern Economy is NOW OUT. Details, and to order on Hive, Blackwells, Amazon or Watersones.

Bill Bryson comments, “Endlessly insightful and full of surprises — exactly what you would expect from Tim Harford.”

Book(s) of the Week 20: The Next Fifty Things That Made The Modern Economy

Okay, this week I’m plugging my own brand new book, The Next Fifty Things That Made The Modern Economy. 

At least, a little bit. But I have some other books to tell you about too.

One of the joys of writing this book was to be able to pick up two or three wonderful books on each topic, learn all about the history and the characters involved, and then try to figure out how to use what I’d learned to tell a story with a particular lesson about how the economy works. In the case of the Langstroth Beehive, for example, I was able to talk about the Nobel laureate James Meade and his discussion of ‘positive externalities’, the long obsession of economists with bees, as well as the long-standing relationship between bees and humans.

Or when it came to the QWERTY keyboard I could discuss the raging controversy over the topic of ‘technological lock-in’, a crucial issue in the debate over how to regulate big tech companies – while at the same time puncturing some myths about the invention of the typewriter.

I loved writing this book and I hope you’ll love reading it. Click here for more information and links to buy from Hive, Amazon, Blackwell’s or Waterstones. (If you are reading in North America, sorry – only the previous book Fifty Inventions That Shaped The Modern Economy is available.)

Do please consider buying, gifting and/or reviewing the book. It’s not an easy time to be publishing a book – or to be a bookseller – and early support makes a big difference.

Now, I promised OTHER books. Here are some of the many, many books that I consulted while writing The Next Fifty Things and which stuck in my mind.

On Bricks: Brick: A World History by James Campbell and Will Pryce – gorgeous coffee-table photographs of brick structures from around the world.

On Beehives: The Hive by Bee Wilson. Quick, accessible history of the long and ever-changing relationship between humans and bees.

On Tulips: Tulipmania by Anne Goldgar (perfectly punctures the tulipmania myths) and Tulipomania by Mike Dash (further great stories and colourful details).

On GPS: PinPoint by Greg Milner, a book that will also be familiar to fans of Cautionary Tales.

On the ChatBot: The Most Human Human by Brian Christian. One of my favourite books of the decade.

On the Bicycle: The Mechanical Horse by Margaret Guroff. Full of telling social observations.

Lots of others that there is no time to discuss here – but the references tell all.

Stay safe, thanks for reading this post – and if you’ve decided to buy my book, thank you for that, too.

Book of the Week 19: Humankind, by Rutger Bregman

Rutger Bregman’s new book, Humankind: A Hopeful History, makes a simple argument: most people, most of the time, are decent. Whether this strikes you as absurd, or obvious, may depend on what side of bed you got out of. Bregman makes a strong case that we’ve been groomed to think the worst of each other by books such as The Lord of The Flies and The Selfish Gene, and a diet of grim stories in the daily news. The book is wide-ranging, and while it is most definitely a polemic – Bregman writes to persuade – it is also full of the most fabulous storytelling. I loved reading it.

Some of the material I knew – for example, the ever-growing question marks over Zimbardo’s prison simulation have become infamous, re-interprerations of Milgram’s shock machine were popularised on RadioLab, and if I recall correctly the urban myth that nobody came to help Kitty Genovese was debunked in Freakonomics. It’s all woven together rather wonderfully here, though.

Other tales, in particular the story of the real-life Lord of the Flies, were completely new to me. The book is spellbindingly well written and you should read it. You’ll learn a lt (I did) and you’ll have good reason to feel better about the human race.

UK: Blackwell’s – Amazon

US: Powell’s – Amazon

While you’re here – my NEW book The Next Fifty Things That Made the Modern Economy is out in two weeks. Please consider ordering online or at your local bookshop, which will be sorely in need of your support.

Book of the Week 18: The Unthinkable, by Amanda Ripley

What is it like to be caught up in the middle of an unthinkable disaster? Why are our responses to these extreme and unexpected events themselves often extreme and unexpected? Amanda Ripley began writing this book after interviewing survivors of the 9/11 attacks. “These people had an agenda,” she writes, “They had thinking they wanted to tell other people before the next terrorist attack.”
One of the strengths of the book – which covers not just 9/11 but Hurricane Katrina and various other catastrophes – is the storytelling, often based on interviews with survivors.
But there are other elements, too. Ripley also relies on historical accounts; her opening tale, well-told, is the explosion of the Mont Blanc munitions ship that devastated Halifax in 1917. One of the survivors Samuel Henry Price, went on to become a sociologist who studied our response to disasters.
Ripley occasionally markets the book as a guide to how to survive if you’re ever in a catastrophic situation, and perhaps it would help. That sells the book short, I think: I was fascinated by the storytelling. The science and psychology of disaster responses is less prominent but there was plenty to interest my inner nerd. I have no doubt you’ll see references to Ripley in the next series of Cautionary Tales.
Recommended!
UK: Amazon Blackwell’s
US: Amazon Powell’s

My NEW book The Next Fifty Things That Made the Modern Economy is out in the UK in May and available to pre-order; please consider doing so online or at your local bookshop – pre-orders help other people find the book and are a huge help.

Book of the Week 17: Rhialto the Marvelous by Jack Vance

Pure escapism this week – Jack Vance’s Rhialto the Marvelous (Kindle only, I am afraid). Rhialto contains three extended short stories, in each case describing the adventures of Rhialto and a small group of other wizards of the 21st aeon. Rhialto is the conceited, narcissistic and amoral anti-hero of these indescribably delightful tales in which a dozen or so powerful, selfish and conceited sorcerors battle strange foes and – chiefly – try to outwit each other in matters of society and petty politics. These are joyful, cruel and wholly original confections. Enjoy!

My NEW book The Next Fifty Things That Made the Modern Economy is out in the UK in May and available to pre-order; please consider doing so online or at your local bookshop – pre-orders help other people find the book and are a huge help.

Book of the Week 16: The Ostrich Paradox

A brief shout-out this week for a brief-but-good book, The Ostrich Paradox: Why We Underprepare For Disasters by Robert Meyer and Howard Kunreuther. Meyer and Kunreuther combine a nice dose of behavioural science with some striking examples: Hurricane Sandy, Hurricane Katrina, the Indian Ocean Tsunami, the 1935 Labor Day storm, the Beverly Hills Supper Club fire, and many others. They explore the cognitive biases that lead us to underprepare, or to abandon protection after a while. Clear writing, good stories, lots of solid academic references.

UK: Amazon – Blackwells

US: Amazon – Powells

 
My NEW book The Next Fifty Things That Made the Modern Economy is out in the UK in May and available to pre-order; please consider doing so online or at your local bookshop – pre-orders help other people find the book and are a BIG help.

Book of the Week 15: The Earthsea Trilogy by Ursula K. Le Guin

A change of pace this week for Easter: Ursula K. le Guin’s Earthsea Trilogy. Last weekend I watched the Studio Ghibli Tales of Earthsea – which has its moments but is not up to the usual stratospheric Ghibli standards. (Le Guin agreed.)

It did, however, prompt me to turn to the trilogy once again. I read it as a teenager, and again on a  long holiday in China in 2003, alongside the fourth book, Tehanu. I picked it up with hazy memories about certain plot points, and was not disappointed by any part of it.

The writing is superbly poetic, the plots are fast-paced and unusual, and the world-building is deft and convincing. A Wizard of Earthsea was originally commissioned as a ‘young adult’ novel, and each of the three novels is told from the point of view of a teenage protagonist, but the themes are mature: ambition and envy, evil done in the name of religion, fear of aging and death, restraint in the use of power.

So many ideas here have been copied – a young boy going to a school for wizards; a wise and powerful order striving to keep the balance against the dark side – but the books still feel fresh and original fifty years on. Yes, there are wizards and dragons, princes destined to be kings and even a damsel needing to be rescued, but Le Guin transcends or subtly subverts each cliché.

Meanwhile I’ve been trying to figure out how to turn Earthsea into a role-playing game. An interesting challenge. An accomplished wizard seems to be able to attempt almost anything, if he is strong enough and is willing to accept the consequences, so part of the fun would be dealing with those consequences. Each success produces the seeds of later trouble. (There is an Earthsea-inspired game, Archipelago, but I have not yet looked at it.)

I know that Le Guin later revisited Earthsea decades later. Tehanu is the fourth book and there are others I’ve not yet read. I found it unsettling to read Tehanu immediately after the original trilogy; not only is it extremely dark, Le Guin so sharply questions some of the implicit perspectives of the previous books that she implicitly criticises herself for having written them, and the reader for having enjoyed them.

Nothing wrong with that – but perhaps leave a breathing space between finishing book three, The Farthest Shore, and picking up the fourth book, Tehanu.

Earthsea: Amazon UK – Blackwells – Amazon US – Powell’s

My NEW book The Next Fifty Things That Made the Modern Economy is out in the UK in May and available to pre-order; please consider doing so online or at your local bookshop – pre-orders help other people find the book and are a BIG help.

Book of the week 14: The Weather Machine by Andrew Blum

I was intrigued by news reports that the Met Office was planning to drop more than a billion pounds on a new supercomputer, and wondered what it was that these clever weather forecasters did with all that silicon. So I picked up Andrew Blum’s recent book, The Weather Machine.
Blum starts with the weather map – and John Ruskin’s metaphor of the “weather machine”, transcending the local observations of an individual forecaster and linking together what James Gleick calls “local surprises” into a larger map. After all, one part of the weather forecasting game is straightforward: if it’s raining to the west of you and the wind is blowing from the west, you can expect rain soon. Weather forecasts begin with weather observations: the more observations, the better.
In the 1850s, the Smithsonian Institution in Washington DC used reports from telegraph operators to patch together those “local surprises” into a national weather map. This map was based purely on observations, but it was still a useful starting point before we had either the scientific understanding or the computational power necessary to make a reliable forecast.
The scientific understanding began to dawn in 1904, when Norwegian mathematician Vilhelm Bjerknes published “The problem of weather prediction”, an academic paper describing the circulation of masses of air. If you knew the density, pressure, temperature, humidity and the velocity of the air in three dimensions, and plugged the results into Bjerknes’s formulas, you would be on the way to a respectable weather forecast – if only you could solve those computationally-demanding equations.
The British mathematician Lewis Fry Richardson attempted just that, attempting to predict the weather of 20th May 1910 given the starting conditions. Alas, it was 1922 before he finished the sums – despite continuing to calculate in the evenings after long days as an ambulance driver during the war. Nor did the equations accurately describe the weather that day, 12 years earlier. Still: one must start somewhere.
Fry Richardson dreamed of a forecasting factory, a stadium filled with 64,000 human computers, conducted by lights and other signals as they furiously calculated the weather equations faster than the weather itself could evolve. It was a remarkable vision: modern weather forecasting works much as Fry anticipated, except that there is no need to perform the calculations by hand – or 128,000 hands. The Met Office’s billion-and-a-half dollars of silicon will do the job nicely.
Blum’s narrative ranges widely and finishes at the European Centre for Medium Range Weather Forecasts HQ in Reading. (My understanding is that this arrangement will survive Brexit, partly because the EMRWF is a separate organisation from the EU. But don’t quote me on that.) The EMRWF, says Blum, are the elite among meteorologists, and he spends some time exploring why they are so admired. Part of the secret is a way of working that can be split into modules and relentlessly tested, experimented with, and improved.
I strongly recommend the book, which is a fascinating glimpse of a mysterious world.

UK: Blackwell’s – Amazon

US: Powell’s – Amazon

My book “Fifty Things That Made the Modern Economy” (UK) / “Fifty Inventions That Shaped The Modern Economy” (US) is out now in paperback – feel free to order online or through your local bookshop.