Bad News: Happily Promoting Bogus Research?

Today, I want to talk to you about Man Bags. Or, rather, I specifically don’t – what I WANT to do is tell you a tale of two bullshitters. You’ll see what I mean as I go. So, from the Telegraph on Feb 14th:

They are the ultimate symbol of a modern metrosexual, sported by David Beckham and Brad Pitt: the man bag. But the grown-up satchel is responsible for causing serious back injuries, according a group of medical experts.

Please note the medical experts bit there, that’s important.

Man bags have come of age in the last decade, replacing the old-fashioned briefcase, and sported by an increasing number of commuters. Unlike a stiff attache case which has a carry handle, a man bag has a strap and is usually made of soft leather or canvas, allowing men to sling it across their backs.

When they first appeared in offices across the country, the owners were often mocked for adopting the distinctly Continental fashion of men having handbags. They were the final nail in the coffin for the era of furled umbrellas, sturdy brief cases and even stiffer upper lips.

But an increasing number of high profile men from David Beckham to Jude Law sporting them meant the trend took off. John Lewis said sales of man bags have increased 21 per cent over the last year, with shoppers buying ever smaller ones thanks to the iPad, the tablet computer made by Apple, being able to squeeze into smaller spaces.

Quite the appeal to celebrity to sell this story so far, with Beckham, Jude Law and Brad Pitt getting a mention. Even the photo was of Beckham and Jude Law. The Mail, similarly, went for a huge Beckham picture, despite the fact that the article is supposedly about back pain caused by heavy man bags – not something we know of Beckham suffering from. Hey ho, this is the news.

Footballer David Beckham has one, as does movie star Robert Downey Jr and model David Gandy (no idea!)– but slinging a man bag over your shoulder could give you a serious back injury, experts warn.

There’s that expert again I wonder who they might be. A medical back specialist? Physiotherapist? Posture expert? Even a humble old GP?

New research from the British Chiropractic Association found that men are carrying too many ‘essentials’ with them on their travels.

That’s right, the BCA – latterly famous for unsuccessfully suing Simon Singh for alluding to the fact that Chiropractic is based on nonsense and is bogus. Red flag ahoy. Continuing:

Laptops, tablet PCs, smart phones and gym kits are now being slipped into ‘man bags’, which are now functional fashion accessories as opposed to objects of scorn.

But the trend has taken a toll on back health with two thirds of men saying they have suffered from back pain.

Two thirds of men are suffering back pain? That seems high, or unremarkable, depending on the definition of back pain. Let’s call that suspicious stat number one and I’ll come back to it.

The BCA study found the average man bag now weighs up to 6.2kgs, the equivalent of more than 12 bags of sugar.

The average bag weighs as much as 12 bags of sugar? The Average? For this to be true, there has to be a LOT of bloody heavy man bags out there, for that to be the average. Dodgy stat two – and likewise, I’ll come back to it.

Gone are the days when a man would leave his home with just his keys and wallet.  According to the study, three in five (60 per cent) men carry some sort of man bag.

60% of men carry ‘some sort’ of man bag? Right, that’s three strikes on the stats, and time to look at the original press release from the BCA, kindly provided on their site.

New research from the British Chiropractic Association (BCA) reveals that two thirds of men (66%)have suffered from back pain.

There’s that 2/3rds suffer from back pain stat again. Interested to see what this meant and where it came from, I called the BCA to clarify this. First of all, the contact I spoke to there couldn’t tell me how big the sample size was – which is always a bit of a concern. To her credit, she said the person who did know was out of office, and she was really very polite and accommodating so perhaps they do indeed have this info and would have shared it if I’d caught the right person, possibly. A technical pass, I think.

But as for the 66% stat, less of a pass there I fear. I was told: ‘We asked if men suffered from, or had ever suffered from, back pain’. So, there was no mention in the question of what actually counts as back pain. I’ve been sat at a computer all day, and my back is slightly sore and tired from it – tick one for me. I’m surprised the stat wasn’t 100%. If they’d asked ‘serious back pain’, ‘chronic back pain’ or ‘back pain requiring painkillers/medical attention’ I imagine the stat would be more reflective.

With back pain on the increase, it seems the humble man bag could have a lot to answer for.

I’ll stop there. Is it? We’ve not established that back pain is on the increase at all – we’ve spuriously shown that two thirds of men apparently have some degree of back pain or have once had some degree of back pain, but is this up or down? What was the percentage 5, 10 or 25 years ago? Where have we demonstrated an increase? We’ve not, we’ve just had it nakedly asserted to us.

Also, note the use of ‘seems’ and ‘could’ – so nothing concrete or useful then, just more speculation? Essentially the whole of the first paragraph has said ‘we have statistics to show that lots of people have bad backs at some point, therefore man bags are to blame, so here’s a lot more about man bags, with a photo of David Beckham to sell papers’. Classic PR bait and switch. And the fact that this is a classic piece of bullshit PR probably makes this next stat come alive too…

According to the study, three in five (60%) men carry some sort of man bag.

Three in five men carry ‘man bags’ – this seemed wildly high to me. So I looked into who conducted the poll. No points to anyone who correctly guessed – fucking OnePoll. They’re my polling nemesis, as you’re probably aware. With that in mind, here’s how I think the stat was generated – as I’ve covered extensively in the past, as a OnePoll user you take the poll and get 10p per survey taken. But you can only cash out at £40, so you need to take every poll you can to even get near making a profit. So you open a poll, and come to a screening question – something like, for example, ‘Do you have a man bag?’ At the same time, you can see that the title of the survey is something like ‘You and your man bag’. Saying no means you lose your 10p. Lie ‘yes’ and you bank it. Easy maths to do.

The above, of course, is just my speculation – the real data itself is still under lock and key until the BCA and OnePoll have generated more (likely spurious) headlines from it. So there’s something for us all to look forward to. Still, this isn’t the only possible scenario – consider an alternative single question, featured in a wider poll covering all sorts of back pain angles, and hinted at in the next line of the press release:

These man bags come in all shapes and sizes – from the traditional over‐the‐shoulder laptop case, to messenger bags and satchels.

Well, what exactly counts as a man bag? How often do you need to carry it for it to qualify? Is a laptop bag really the same thing as a man bag, when it’s designed for a specific purpose? If it is, then how does that differ from the satchel my granda took to the building site to put his sandwiches and fags in 50 years ago? If it doesn’t, then it’s clearly not a recent trend. When we hear man bag, we thing of a male equivalent of a handbag – something for luxuries. We don’t immediately think of a laptop case carrying a laptop, or a bag with a butty in it. So even in a wider poll about trends, this man bag stat can easily be artificially generated.

What is shocking is the sheer amount of items the typical male now carries; in fact the average man bag now weighs up to 6.2kgs the equivalent of over 12 bags of sugar.

Now, let’s not mince words here, 6.2kgs is a FUCKING lot. Twelve bags of sugar? Very suspicious. So, let’s look at how that stat was constructed:

Weight broken down by: 3kg laptop, 2kg gym kit, 0.5kg lunch, 0.5kg book, 100g phone/blackberry and 100g iPod.

Now, here’s the fun part: the average bag is only the average weight if it contains all of those items. The press release provides a breakdown of how many men carry each item:

  • Mobile phone/Blackberry = 56%
  • Lunch = 40%
  • Books = 29%
  • iPod/MP3 player = 23%
  • Laptop/iPad = 19%
  • Gym kit = 10%

So, the heaviest two items, which make up the major bulk of the 6kg, are carried by 19% and 10% respectively (and even within that, the laptop could be an iPad which certainly does not weigh 3kg). What’s more, to get all 6kg you’d need all of those items – so even assuming that everyone who had a gym kit also carried every other item too, that’s a maximum of 10% of all manbag carriers carrying 6kg on a regular basis. So the ‘average’ bag is only carried by, at most, the heaviest 10% of the population, and 90% of all use falls below the supposed average. Terrible, terrible stats. Classic OnePoll.

What I find really quite interesting about all of this is that the advice from the BCA is actually very sensible, and worth following (in this particular case, I’ll add). And I’ll stress the lady I spoke to at the BCA was polite and helpful, and said the full data will most likely be available once the rest of the stories taken from it are already out (which remains to be seen, but let’s be charitable here). But even with that in mind, the problems are still clear to me:

  1. Good advice couched in bullshit fear is still wrong – it’s eroding the quality of research and feeding a PR culture where truth is an ignorable commodity.
  2. People can be dragged into the sensible part of the BCA and find themselves with a pure chiropractor – twisting babies and snapping spines to cure deafness and colic, which we all know plenty about.
  3. Using fear to make headlines and sell papers is always Bad News
  4. The comments below the articles in the Mail and Telegraph show the insidious nature of these kinds of stories – rather than talking back health, chiropractors or bullshit PR, people instinctively start talking about man bags and handbags – distracting from the very real criticisms of the whole piece.

In summary, it now appears the BCA can be said to happily promote bogus research, too…

, , , , , ,

  1. #1 by Oli Conner on March 4, 2011 - 12:46

    I would just like to clear up a couple of points. OnePoll didn’t write the press copy or interpret the statistics, we only asked men if they suffered from backpain and asked them what items they carry.
    Secondly, there is absolutely no reason for us to lead respondents to the correct answer of a survey. We rarely use screening questions, instead opting for profiling data. When we do use screening questions, we ensure that the title of a survey does not in any way reflect the content of the question – this is something that we take very seriously and all of our researchers are trained to do.
    We have no need to lie in the way you state – we have over 100,000 members of our panel. We operate on similar lines to all other online research companies.
    It is a shame that you see us as your nemesis, I hope we can work towards changing that.

  2. #2 by Marsh on March 7, 2011 - 12:02

    Hi Oli – Thanks for the response.

    I’m interested that OnePoll didn’t write the press copy, as the BCA also said they didn’t write it and that the story came to them with the maths and angles already done. I’m happy to believe the spin was put in between the survey coming out of OnePoll and the press release going to the BCA, presumably by the intermediary PR agency, who are as yet un-named.

    As for the second point, I’m not sure this is the case. I’ve taken a large number of surveys on OnePoll, and have seen screening questions used regularly as I’ve described above. I’ve also outlined above that this is one possible way of a number of ways in which the stat could be unreliable, and I’ve outlined another above. Could you share the questioning so we know how the participants were to distinguish between what does and doesn’t count as a man bag, and what level of regular use would count for the overall statistic of the survey?

    Just to be clear, though, I’ve not stated that OnePoll have lied, nor would I like to give the impression any lies or fabrications have taken place – instead I’ve suggested quite how the data could be lead, and I’ve seen many such example of this in the past. Take, for example, these clearly-leading questions that appeared in one of your surveys last year:

    ‘Who do you prefer to work for, men or women?’ –
    ‘If you prefer to work for a man, why do you think women make bad bosses?‘ –

    Clearly, the second question has introduced leading language, and has produced the ‘women make bad bosses’ headline which was the main spin of the end release. Looking at the options of the question, clearly there is significant lead there too.

    I’d be delighted to not have to consider you a nemesis (obviously the word was used for comic hyperbole) but at the moment the track record of consistently dicey articles based on the polls conducted by OnePoll is very poor, and that’s presuming OnePoll/72Point aren’t involved in actively creating those press releases. Taking, for example, the ‘Women Make Bad Bosses’ survey I’ve outlined above – what action was taken to correct the errors made?

    I’m happy to work towards changing my opinion of OnePoll, though – one thing that will go along way towards that will be to make polling questions and options available (online, or even simply upon request) to ensure clarity and honesty in the survey process. Is this something you’re able to do? Without this (which is a standard requirement in psychological polling, for example), it will be very difficult to trust the data produced.

(will not be published)