Last week Dan Fox’s proposal on data development loans was voted top f the policies for supporting entrepreneurship at the latest Pragmatic Radicalism event, chaired by Chuka Umunna MP the shadow secretary of state for business.

“Described by pollsters as a weather-vane constituency, it is contemporary information technologies making a difference this week. Jill tapped the side of her Google Glass display and began directing canvassers. “The Harrises. Last time we spoke to them, they complained about the daily commute. Can you get up that chart of rail fares and investment?” Swiping away at his iPad, Jack strolled off, armed with all he needed to connect, both electronically and emotionally, with the voters at number 23. On the opposite kerb, two of his colleagues were not having quite as much luck generating an augmented reality view of the lighting and paving repairs that had taken place on the street since Labour had taken over the local council in 2013.”

Election Sketch, The Times, 9 May 2015

For those who still feel that the biro and clipboard are unnecessary luxuries on the #labourdoorstep, the thought of handheld devices brimming with electronic data being at the centre of elections is at best bewildering. But data is now, as we all know, “Big”. It is all around us and has a permanence in our daily lives akin to a new way of communicating, helping us to understand what we and others are doing and will do. Campaigning is not insulated from this. Last year, Obama For America, set the standard for using Big Data in identifying, motivating and expanding the numbers of your voters. Labour looks set to reflect these techniques.

However, before we get too caught up in a vision of campaignbots traipsing around the marginals like canvassing Terminators, we should also consider thepolicy significance of Big Data. Although it has not been a great couple of weeks for data of any kind – in the news for all the wrong reasons as the full extent of the surveillance of the personal variety has been exposed – this must not distract us from the thousands of positive, world-changing uses of mass data collection and analysis.

In the UK, the release and sharing of millions of cancer treatment records is supporting  better diagnoses and treatments. In Dublin, the Smarter Cities initiative is identifying ways to improve all aspects of urban living. In Uganda, aWater Cost Index is merging data from the complex construction and funding of water projects to bring transparency to the sector and attract ever greater amounts of investment.

There are an estimated 1200 exabytes of data in our world. That is an extraordinary resource and much of it comes from the public sector. In terms of opening it up, huge progress is now likely thanks to both the US government’s Open Data Order, and our own Review of Public Sector Information.

Having the data is not the issue, though. The expertise and resources to process and analyse it is. That is why at last week’s Pragmatic Radicalism Top Of The Policies on supporting entrepreneurship, I proposed a system of data development loans: low cost lending that SMEs could take out with government and use to collaborate with data scientists, technologists and journalists to create new products and services from the exploitation of Big Data. If it is public data being used then this could be combined with a licensing scheme or auction process to ensure government revenues increase alongside the businesses’ growth.

Big data has been described as the new soil and the new oil. We subsidise farmers to grow and energy companies to generate, all with dubious results in terms of efficiency and necessity. Let us at least also give real entrepreneurs and risk-takers the chance to do what they do best with the new, fertile, creative medium of data.

Dan Fox is an intelligence analyst and trainer, Big Data blogger, and member of Walthamstow CLP. He tweets as @campaigner