To tweet or not to tweet?
Economist Justin Wolfers runs a controlled experiment to test how Twitter is affecting his productivity
I don’t normally hold with the traditional New Year’s resolution of quitting some objectionable habit – even though my favourite economist, Thomas Schelling, has written very thoughtfully on the subject. (Schelling, a brilliant game theorist and long-time smoker, used a variety of game theoretic tricks to outwit a formidable opponent – his addicted self.)
But as 2011 drew to a close, I had been wondering about my addiction to Twitter, the service that allows users to publish online short messages – grumbles, aphorisms and most often, links to recommended articles. Other users can choose to whose messages they will subscribe and unlike on fully-fledged social networks, such as Facebook, this is not necessarily a reciprocal relationship. (Facebook users have friends; Twitter users follow and have followers.) My Twitter habit has the pernicious consequence of being rather time-consuming – but it has plenty of benefits too. Should I quit? Cut down? Or should I resolve only to stop feeling guilty?
Part of the problem, I realised, is the difficulty of measuring the costs and benefits of the habit. Imagine my curiosity, then, when I noticed that the economist Justin Wolfers – a self-described Twitter cynic – had joined the club and was running an experiment to test how Twitter was affecting his productivity.
“Every morning I would flip a coin,” he explained to me. “Heads, I would sign on to Twitter, tails, I would simply tweet ‘Tails: goodbye for another day Twitter.’” It might seem strange to run an experiment with a single subject, but that all depends. If the aim is to discover the effect of Twitter on the productivity of Justin Wolfers, the experimental design looks just fine.
The challenge, of course, is to interpret the results. “I tried to be scientific,” said Wolfers, who installed software on his computer to record his use of different programmes and websites, while also using Google alerts to track whether his tweets were having much impact on web chatter. “I’m not sure I succeeded.”
Wolfers rated his productivity levels at the end of each day – revealing, and “also a total bummer” – rarely topping six out of 10. But that’s not unusual: a persistent anxiety that each day has been poorly spent is, I feel, the sign of many a productive person.
Ultimately the formal experiment broke down: “After a while, I got tired of flipping coins.” Wolfers has his data; he has never bothered to analyse it. He has decided that Twitter works for him. The informal experiment of giving Twitter a try to see how it worked out was, it seems, of far more practical use than the formal experiment of randomising days on and days off.
This makes some sense. I’ve become convinced that most of us do not experiment enough with new experiences. (The first 20 years or so of life are an exhausting but stimulating exception to this rule.) Yet few of the experiments we could be trying are conducive to a proper randomised trial.
Somehow this is a great disappointment to my inner nerd. Both Justin and I like the idea of running controlled experiments in everyday life – gut feelings can be so misleading – but he warns that to do it right takes more discipline and time than many of us might want to deploy.
Yet Justin Wolfers’s experiment has inspired an unexpected insight. The toss of a coin might not have generated data that anyone cared to use, but it had the obvious consequence of reducing the days spent on Twitter by about half.
Of course one could simply decide to spend less time on Twitter, but the arbitrary dictates of the coin have a curious power. (Yes, I have read The Dice Man.) So I do not think I’ll be quitting Twitter this year; I will be using the toss of a coin to help me cut back a little.
Also published at ft.com.
12 Comments
James Blight says:
Just got to the bit in The Logic of Life talking about the costs and benefits of living in cities. Is it possible that twitter is a close analog of this? You ‘spend’ productivity to be on twitter, but you increase your chance of a worthwhile interaction with someone from whom you can learn, or who inspires an insightful thought. Like in cities, the net easily measurable result is reduced day to day productivity (c.f. a less valuable dollar) but the positive, albeit more ephemeral benefits are there for the rational tweeter.
7th of January, 2012Colin Lewis says:
Tim from an economic perspective (i.e. money in the bank), Twitter so far fails to pay dividends. At one stage I was spending circa 45 hours per week for 18 months, building relationships and had many word of mouth recommendations, including celebrities and many persons respected in the field, for my online and offline products. The financial return for this was less than US$ 10 k per year. I subsequently quit Twitter and closed my account, but recently re-joined and take a different approach of selecting far fewer people to follow and being far more focused on collecting knowledge than promoting my own products.
7th of January, 2012Tristram Brelstaff says:
“He has decided that Twitter works for him.”
Which is just what an addict would say!
7th of January, 2012k says:
the cost of tweeting is more in terms of coming up with something, or reading something someone else came up with; this experiment doesn’t address that by simply tweeting the same thing everytime
7th of January, 2012Victoria Tomlinson says:
How many of us will completely relate to this article? The worst bit of Twitter time usage (I’m avoiding the word ‘wasting’) is following various conversations through. You learn so much and it is such an interesting life on Twitter – but as you say, how do you measure the benefit?
8th of January, 2012Our advice to clients is to be completely strategic in using Twitter – what do you want to achieve, who do you want to engage with and what will success look like? Then monitor and review. It’s amazing how much focused success you can get for a business.
But as for using it personally…..I wish I followed our advice better!
Ian McCarthy says:
If you are like me and found this posting using Twitter, then how would one measure the value of this?
8th of January, 2012gwenhwyfaer says:
I’m autistic; twitter (through pseudonymity) is the only social life I can cope with. In return, it seems to be effective block on my getting anything else done – but then, I remember not actually being terribly productive in my pre-twitter days either.
I wonder if it’s worth it, but I have no idea how to do the cost/benefit analysis. However, I did close my account for a week recently and I found that for those few days, I was so bored – there was suddenly a gaping hole in my life – that I cast around for things to get done to fill that chasm. So perhaps the secret of Wolfers’ coin-toss experiment was that using twitter on the on days got him just used enough to having a constant time sink to have its conspicuous absence increase his productivity on the off days?
8th of January, 2012Andrew says:
I love the idea that there are people earning a living by advising their clients to be ‘completely strategic in their use of Twitter’. I assume that, before they became ‘completely strategic’ the clients in question spent all their time posting and/or following links to Youtube videos of two legged cats and Onion web pages.
8th of January, 2012David roberts says:
Surely there is a further experiment which is the costs and benefits of paying someon to ghost your twitter account including summarising the most important tweets? But I guess that would not feed the addiction!
8th of January, 2012Alastair Dryburgh says:
I had two thoughts about this.
Firstly, did he choose the most useful research question? Rather than twitter/not twitter I suggest it would have been more useful to ask either:
*how much twitter can I do in a day without adversely affecting productivity?, or;
*assuming that some amount of twitter has a beneift, where is the point of diminishing returns?
Secondly, I wonder if a strictly quantitative approach is the best one here, given the subjectivity of the outcome measure (productivity) and the huge number of possible confounding factors.
An alternative way to approach the question would be to keep a journal recording twitter use and productivity, and then reach an intuitive judgment. I suspect that this might have worked better.
I’d like to think that there are ways in which we can become more enquiring and more experimental about what we do and what benefits it produces, without resorting to mathematics. And this is despite, or perhaps because of, my being a mathematician.
9th of January, 2012Andy says:
Justin’s experiment highlighted the importance of importance of experimental blinding and showed that interpreting experiments is always a challenge.
As a follower of his before the experiment, I can say without a doubt that his tweets increased by several magnitudes during the days he randomly used Twitter. He was clearly putting off some tweets from his ‘off’ days and then posting them during his ‘on’ days. This got so bad I actually had to unfollow him during the experiment because he was overwhelming all the other content on my feed during his ‘on’ days.
Obviously, this makes interpretation of any results rather difficult because the experiment was not pure in the sense that it varied changed his ability to compose tweets even on ‘off’ days. Indeed it might have made those days even less productive because he was effectively tweeting in a less efficient manner.
Although running a blind experiment would be challenging in this environment, at the very least he would have been better served by randomizing over larger blocks of time. If he flipped a coin over whether to use Twitter for a week at a time, then the spillovers across days would be substantially lessened.
10th of January, 2012Prashant P says:
Tim,
What about measuring the benefit to others? I for one find many interesting things to read from your twitter feed every day.
This must have some benefit for you and other tweeters, otherwise you wouldn’t have bothered in the first place.
12th of January, 2012