Let’s finally fix data sharing in London


London boroughs have long recognised the pivotal role that data plays in identifying local needs, delivering better public services, improving policy-making, providing greater accountability and encouraging innovation. 

Given that so many issues that affect their residents transcend borough boundaries, they also know that collaborating with data is key. 

If we want to see what’s happening across London, spot the potential to share resources, intelligently coordinate the actions of different teams, and maybe even predict and prevent, we need the ability to access data from lots of different sources.

Rarely has this been more true than in the face of Covid. Whether it’s to enable a strong recovery or to tackle a second wave, London needs to be able to marshal data from across its public sector quickly and effectively; and do so legally, ethically and securely.

The Challenge

Collaborating with data typically entails sharing data, which is made hard due to a number of well-known technical, process, skills and cultural barriers. I’ve written about them many times before

As a result, arranging for data to be shared between multiple boroughs has been known to take six months or more; a timeline that’s simply not compatible with the urgent needs of Covid Recovery.

The Solution

Over the past year, LOTI has worked with borough CIOs, their Information Governance Leads and the GLA to develop a Joint Statement of Intent on Responsible Data Collaboration and a range of specific measures to tackle barriers to data sharing.

Each measure may feel like a basic step, but collectively they should fundamentally improve the pace and impact of sharing data legally, ethically and securely in London.

Yet they will only make a difference if the majority of boroughs adopt them.

Six measures for boroughs to implement are outlined below, which are endorsed by LOTI, London’s Chief Digital Officer Theo Blackwell, the GLA and the Information Governance Group for London. 

If you work in London local government, this deck outlines the Statement of Intent and the exact things you need to do. The steps are also summarised in the five-minute video below.

Step 1 – Use LOTI’s four-step methodology to design data projects.

It’s very common when trying to tackle a problem to hear someone say: “Let’s get all the data together and see what it tells us.”

It may be tempting, but there are significant downsides to that approach.

First, I’ve learned the hard way that organisations can waste months or even years chasing down datasets that add no extra value to their decision making and enable no new action. Without any specific outcome in mind, every dataset is potentially in scope.

Second, and more seriously, when it comes to sharing data, unless every dataset that we might wish to use happens to be open data (or data that is sufficiently non-personal that it could be open data), we cannot complete the information governance process without being able to state upfront the exact uses to which the data will be put. 

For that reason, LOTI’s four-step methodology focuses on how data leads to action. It starts with real-world outcomes in mind, asking: “Who could do what differently if they had better information?”

This kind of action-orientated approach can enable better conversations between senior officers, service managers and their data teams. It also helps pin down exactly which datasets are actually needed and why.

LOTI's Four step data methodology

LOTI’s Four-step data methodology

Step 2 – Use the London Datastore as the default platform for data sharing.

Early in the Covid crisis, boroughs raised an urgent and important need to be able to identify and support vulnerable children in receipt of free school meals who attend school outside their borough of residence. The information governance arrangements were completed in just two days and… no data was shared.

This was just one example of the flaws of relying on a spider’s web approach to sharing data between boroughs (where each shares data with their peers). To take it to its logical extreme, if all 33 boroughs need to share data with their 32 peers, it requires establishing 528 data sharing relationships. It simply doesn’t work.

The solution is to use the London Datastore. The Datastore is not just for publishing open data. It can be – and is already – used for securely sharing datasets between organisations. 

Using the Datastore for collaboration projects is simpler and more consistently secure than boroughs sharing encrypted spreadsheets with each other. It makes it easier for boroughs to build automation into the data exchange process, reducing the time from analysis to action. It also ensures that metadata (descriptions and limitations) are shared with the data, improves version control (so analysts know they are using the right version of a dataset) and also makes it easier to build and share pan-London datasets.Explanatory diagram of London Datastore acting as a central register of London's datasets

Step 3 – Use IGfL as the primary forum for creating and vetting pan-London data sharing agreements.

While organisations rightly need to assure themselves that they are in compliance with all relevant legislation, pan-London data sharing agreements can be delayed when each wishes to propose their own version.

The steering committee of the Information Governance Group for London (IGfL), which represents IG leads from across London boroughs, has confirmed it is willing and able to create or review data sharing agreements (ISAs) for projects that involve multiple boroughs.

The IGfL stamp of approval can help reduce the number of internal conversations that need to take place within each borough, speeding up the process of creating and agreeing an ISA.

Step 4 – Use LOTI & IGfL’s 7 Step Information Governance process.

Data sharing is held back when each organisation follows a different process for moving from idea to execution. Additionally, Information Governance (IG) staff are often left out of the design stages of data projects. The result is that major IG issues may only be discovered late in the project lifecycle, delaying their launch.

LOTI and IGfL have created a 7 step IG process to specify what steps need to happen and who should be involved in each step. Adopting a common process makes it easier for boroughs to work together on developing data sharing arrangements. The process involves IG Leads in the early stages of data projects, ensuring that legal compliance, ethics and security are built in by design.

LOTI 7 Step IG Framework

LOTI and IGfL’s 7 Step IG Framework

Step 5 – Use the Information Sharing Gateway for creating ISAs.

When creating information sharing agreements (ISAs), it’s common for multiple organisations to end up reviewing and making comments on different copies of the same document. This hinders collaboration and takes up considerable time reconciling changes proposed by each organisation.

Instead, boroughs should use the Information Sharing Gateway, an online platform that enables multiple organisations to collaborate on creating one version of a data sharing agreement. 

The platform is used by thousands of public sector organisations – including the GLA and London Fire Brigade – and is free for all London boroughs.

Screenshot of homepage of Information Sharing Gateway

Step 6 – Use Dapian for creating DPIAs

The creation of Data Privacy Impact Assessments (DPIAs) entails many of the same version-control and collaboration issues as ISAs.

LOTI and IGfL have worked with Looking Local, CC2i and local authorities around the UK to develop Dapian: a digital platform that aids the process of creating DPIAs. (Check out the demo site here.)

The ICO were on the project group involved in the development of the platform, which reduces the time taken to create a DPIA by ~63%, saving around £1,200 per DPIA in saved staff time.

The platform costs £10,000+VAT/year. LOTI members get a 15% discount, and their first year subscription may be funded from LOTI’s budget.

Screenshot of Dapian Homepage

Screenshot of Dapian Homepage

Let’s make it happen!

Readers familiar with my previous blogs will know I feel really passionate about this stuff. The simple truth is that the value we can derive from London’s public sector data will forever be held back until we break down the barriers that keep it siloed.

I firmly believe the measures outlined above can make a real difference. I therefore strongly urge all London boroughs to discuss this deck with their data and Information Governance teams and put a plan in place to implement the measures before the end of 2020.

The LOTI team is willing and able to offer assistance. Let us know what help you need.

So come on, London: let’s get this done.

Find Eddie on Twitter


Eddie Copeland
15 October 2020 ·
Skip to content

Join the LOTI conversation


Sign up for our monthly newsletter to get the latest news and updates