Post 19: Giving up on surveys

I’m very glad that Media Watch had a go at coverage driven by vendor-created research this week, because I have reached the decision not to use them any more.

For ages I have been uncomfortable with the way surveys are used to generate coverage, largely because the methodologies used are far from transparent and the intent – generating media coverage – is blatantly obvious.

Recently, however, I encountered the worst, most dishonest piece of research I have ever seen. I label it as such because it simply asked the wrong questions and the researcher was ignorant of the right questions. For example, the survey contained assertions that business data stored on computers is not well protected. But it had not asked those surveyed about the use of the most common data storage and protection technologies, so was simply not reporting on the real world. The research also asserted that businesses cannot recover lost or damaged data at acceptable speed, citing 40 hours as disastrous. Yet when I pressed the researcher on the fact that many organisations deliberately set recovery time objectives of more than 40 hours, the researcher was ignorant of the term “recovery time objectives” and admitted it was something it should look at in the next version of the research.

That’s not good enough.

Yet the research concerned has been cobbled together into a glossy brochure and will be pressed into the hands of prospects and suspects for a year, until the next edition of the research is produced.

I sincerely hope that no-one falls for this flawed study which has (to mix cliches) pissed in the well and broken the camel’s back, because it is such an obvious example of research being abused to prove a point that I simply cannot contemplate trying to find out what is wrong with other research I encounter.

Independent research created by dedicated researchers without commission remains something I will consider. But I simply feel I can no longer trust anything initiated by a vendor. And if I cannot trust it, why would I present it to my readers?


Post 18, 2009: Journalists and musicians

A few times a year, I pull my old guitar out of a cupboard and play it for a while.

I can still remember a few tunes and chords and I have fun for a while. Back when I was more interested in playing music, I was a competent if talentless guitarist.

I would never, however, call myself a musician or suggest that anyone should go out of their way to behold me playing music. But I can play a little music.

Of late, I have been wondering about how we consider “journalists” vs. “musicians”. I think the comparison is useful because no-one runs down someone who occasionally picks up a guitar, sits down at a piano or plays any other instrument. It’s cool to have a go at playing music! Heck – it’s normal to encourage kids to do it, even though the sounds they produce are sometimes not much fun to experience!

As I swirl around inside the washing machine of debate about the future of journalism, I think we can usefully apply the attitudes we bring to people who play music for pleasure to those who practice the new forms of journalism that have been made possible by Blogs, Twitter and countless other online tools.

These tools mean that those with the inclination to do so can now pick up a journalism tool (maybe instrument is a better word) whenever they feel like it, and produce some journalism.

It may not be good. It may be the journalistic equivalent a tonally-challenged, lyric-mangling, 40-something playing hits from his teens on a cheap acoustic guitar with 15 year old strings.

But I think that rather than agonising over what sort of journalism is legitimate, we should simply say that it’s cool when people have a go at journalism.

The musical metaphor is also useful, I believe, because music is wonderfully fragmented. We have classical, pop, jazz, rock, about ten different sub-genres of each and about a million other top-line genres I haven’t mentioned.

I think journalism could do with similar diversity.

Music, of course, has some common values. It tries to entertain by making pleasing sounds (experimental music, IMHPO, tries to entertain through the self-conscious creation of un-pleasing sounds, but that’s another story).

Journalism also has common values. It seeks to discover and re-present facts, using the journalist’s experience of events and some rules about objectivity.

I believe that if we agree that journalism has very simple ingredients and apply the same liberal thinking we apply to music, we can get to a place where we revel in the diversity of journalism that we have today, rather than agonising over whether those using the new tools/instruments of journalism are any good at it. The quality, to me, is no longer the point. Understanding the diversity is where it’s at.