Most people would agree that the inter­pret­a­tion and present­a­tion of data is cru­cial to the future of journ­al­ism online and off. And among the world’s biggest gen­er­at­ors of data are democracies.

So how should a demo­cratic gov­ern­ment make its data avail­able to journ­al­ists? And in what form?

Should it take the oppor­tun­ity to bypass the media and provide data dir­ect on the day of a pub­lic announce­ment, for who­so­ever sees fit to sort and parse?

Or should there be priv­ileged access under embargo, to organ­iz­a­tions and indi­vidu­als who can help inter­pret this data for the wider pub­lic? In the ver­nacu­lar — journalists.

In the UK, gov­ern­ment data has already been suc­cess­fully made avail­able through early co-operation with the media. Raw data on schools was released to spe­cial­ist journ­al­ists a week in advance of media announcements.

They could pro­duce detailed online tables, or spe­cial sup­ple­ments, for their vari­ous outlets.

The priv­ileged access, and the time allowed, res­ul­ted in bet­ter inform­a­tion for the pub­lic, that was presen­ted in a num­ber of dif­fer­ent ways and was widely disseminated.

Open gov­ern­ment

It was just the kind of thing that then Cab­inet Office min­is­ter and open gov­ern­ment evan­gel­ist Tom Wat­son MP might have had in mind when he launched his inform­a­tion ini­ti­at­ive Show Us A Bet­ter Way, to soli­cit ideas for data mashups to make gov­ern­ment data more access­ible to the public.

Of course, gov­ern­ments, des­pite attempt­ing to por­tray them­selves as coher­ent and omni­scient insti­tu­tions, tend not to be quite so joined up, or all-knowing.

So whilst, Tom was try­ing to prise open the door for data, worm­ing its way through the legis­lat­ive pro­cess was an order which would have the effect of lim­it­ing access to stat­ist­ics across gov­ern­ment: The Pre-release Access to Offi­cial Stat­ist­ics Order 2008.

It came into force in Decem­ber 2008.

The order had gone through all the bur­eau­cratic “con­sulta­tion” pro­cesses. A year earlier, Britain’s Fin­ance Min­istry (HM Treas­ury), who had ini­ti­ated the meas­ure, released a doc­u­ment with the poten­tial to trans­form com­mu­nic­a­tion between gov­ern­ment and cit­izenry by pop­ping it dis­cretely onto their web­site in the form of a pdf: Lim­it­ing pre-release access to stat­ist­ics: a con­sulta­tion doc­u­ment.

The clue to their inten­tions is the word “lim­it­ing.” As their web­site said: “the Gov­ern­ment is pro­pos­ing to tighten the rules and prin­ciples under which access to offi­cial stat­ist­ics in their final form can be gran­ted ahead of their publication.”

The civil ser­vants were con­fid­ent (the emphasis is theirs) that:

The pro­pos­als in this con­sulta­tion doc­u­ment have an impact only on those giv­ing pre-release access to stat­ist­ics and those receiv­ing it. In vir­tu­ally all cases these people will be work­ing in, or for, the pub­lic sec­tor (and are largely in cent­ral gov­ern­ment). As such there is expec­ted to be no impact on the busi­ness or third sec­tors.

So the con­sulta­tion was drawn to the atten­tion only of who had pre­vi­ously respon­ded to another stat­ist­ical con­sulta­tion — “stake­hold­ers.” Not a journ­al­ist or media organ­iz­a­tion among them.

More con­sulta­tion?

In a par­al­lel con­sulta­tion by the UK Stat­ist­ics Author­ity [pdf] on the release of stat­ist­ics, the only men­tion of journ­al­ists came in an appeal for more media train­ing for stat­ist­i­cians, and in this sub­mis­sion from the Local Gov­ern­ment Asso­ci­ation (LGA):

We sup­port the stricter pro­pos­als on the use of the embargo sys­tem for journ­al­ists. Local coun­cils some­times
have a keen policy interest in stat­ist­ical releases (and indeed con­trib­ute data towards their pro­duc­tion). It can
be, at the least, frus­trat­ing that journ­al­ists are able to form their own inter­pret­a­tion of the stat­ist­ics in a release
before any­body in local gov­ern­ment has seen it.

Else­where on its web­site the LGA notes that it “is involved in a range of pro­grammes aimed at increas­ing local gov­ern­ment account­ab­il­ity and tack­ling corruption.”

The UK Stat­ist­ics Author­ity (UKSA) pub­lished its new code of prac­tice [pdf] in Janu­ary 2009. They too reveal a desire to ensure that every­one gets equal access to data:

Stat­ist­ical reports should be released into the pub­lic domain in an orderly man­ner that pro­motes pub­lic con­fid­ence and gives equal access to all, sub­ject to rel­ev­ant legislation.

The media is left out of the equa­tion. The UKSA code hardly dis­pels the ste­reo­type of civil ser­vice media paranoia:

Ensure that no indic­a­tion of the sub­stance of a stat­ist­ical report is made pub­lic, or given to the media or any other party not recor­ded as eli­gible for access before pub­lic­a­tion. Report to the National Stat­ist­i­cian imme­di­ately any acci­dental or wrong­ful release, and invest­ig­ate the circumstances.

The impact of the order on the appear­ance of school league tables came to light in Janu­ary 2009. The BBC’s Gary Eason told the story. Schools Sec­ret­ary Ed Balls was sym­path­etic, but…

So gov­ern­ment depart­ments reap the “unin­ten­ded con­sequences” of ini­ti­at­ives begun with other agen­das. Let­ters draf­ted by junior bur­eau­crats and sent on behalf of their bosses shuttle between agen­cies and min­is­tries. Con­sulta­tions appear and dis­ap­pear like smoke. Par­lia­ment rubber-stamps legis­la­tion without being made aware of its con­sequences or con­tents. And civil ser­vants use pub­lic access to pre­vent effect­ive dis­sem­in­a­tion throught the media. No need for con­spir­acy the­or­ies in this story.

What to do?

As more data becomes avail­able through gov­ern­ment we need to find ways of deal­ing with it. One way is to divorce it from pub­lic announce­ments and dump it online. Another is to grant priv­ileges and oblig­a­tions to people who have the time and resources to inter­pret, pub­li­cise and dis­sem­in­ate it. They’re called the media.

Per­haps a select com­mit­tee might turn its atten­tion to the mat­ter? Or per­haps another con­sulta­tion, and another review? Or — God for­bid — it might be left to the dis­cre­tion of the people releas­ing such data, with a simple, inde­pend­ent appeals process.