Bigger, better and more data for better social outcomes: what's next for the INDIGO data and learning collaborative?

Introduction

When the Government Outcomes Lab started nurturing INDIGO – the International Network on Data for Impact and Government Outcomes – we had a clear idea of what we wanted to create: a data collaborative where people could use and reuse data on outcomes contracts to make informed decisions and design better programmes in the future.  We envisioned an initiative where different organisations would share their data and the GO Lab would play a catalysing role by collating, quality assuring and hosting a series of open datasets and conversations.

Community events were part of INDIGO from the very beginning: quarterly peer learning sessions and energising Hack and Learn events. We knew that data is only as good as it answers the questions of a community. At the end of the day, what is the value of a great dataset if no one is using it? If there is no community making sense of it?

Over time, I have seen policy makers, practitioners and researchers engaging with our data in a much more critical and meaningful way, and that is something that I celebrate and encourage. The INDIGO community is keen on developing new datasets, but practitioners and government officers have challenged us with questions about the source of our data, the quality of our data model, and the rationale behind publishing particular datasets. We need to answer them if they are going to make use of our data. Being critical about data is one of the main characteristics of the INDIGO community, and I believe that this is something that the initiative should take forward as one of our most valuable learnings. In the spirit of starting a conversation, this blog presents some ideas on the direction that INDIGO should take in the next years.

What have we achieved?

It would be difficult to describe the next steps for INDIGO without taking stock of what we have achieved. We added many more projects to the flagship Impact Bond Dataset, and developed other angles on it, such as the Outcomes Fund and Organisation Directories. We created a Pipeline Dataset, where we collect data on upcoming outcomes contracts. Animated by the spirit of openness and collaboration, other members of the GO Lab decided to share the data that they use for their research as open and interactive datasets. Michael, Elle and Felix designed the Joined-up public services evidence navigator, where they share data on past initiatives where the UK government attempted to join up public services.

Our Systematic Review of Outcomes Contracts – Collaboration tool (SyROCCo) deserves a special mention. Together with the University of Warwick and the Alan Turing Institute, we developed a machine learning prototype tool that helps practitioners and policy makers navigate a large database on evidence around outcomes contracts. We're incredibly excited about SyROCCo, which represents our first attempt to use machine learning to speed up the process of searching for evidence.

Above all else, we are proud of the community that we have nurtured. We are amazed by the quality of the conversation that takes place in our peer learning sessions, the complex questions that this community often asks, and their unbeatable will to find data-driven answers.

Where are we going next? Towards a data & learning collaborative

Our community is very diverse, coming from different countries and addressing complex problem in disparate contexts. Because of this variety of backgrounds, users usually have very different ideas on INDIGO’s next steps. However, there is agreement on three key directions:

1. Not just counting outcomes, but telling a story

We heard practitioners from all parts of the world asking: Do social impact bonds work? How do we know when to choose an outcomes model or a traditional grant? In these cases, we referred practitioners to other papers and evaluation reports, but we recognised that our dataset had very little data on outcome achievements and was not capable of providing an answer. Due to this gap, the community has consistently asked INDIGO to invest more time and effort in collecting data on outcomes achievement. This will be one of our key goals for the coming years, as we believe that having more standardised data on achievement could unlock great value, helping us build a data-driven answer to some of the questions above.

We know that this is a difficult task. Hosting an open dataset comes with the responsibility to strive to avoid misinterpretation and misuse of the data. And we know that data on performance may be challenging to understand, particularly for users with limited experience of outcomes-based projects. As a result, this goal is not just about hosting more data, but actively working with the community to build narratives around what the data tells us.

Of course, this does not erase interpretation by other users, but it aids their interpretation by adding context. The logic behind social impact bonds may seem simple (there are a set of metrics, and the service provider/investor only gets paid if those metrics are achieved), but there is lots of complexity, nuances and layers to every project. We believe that working side-by-side with the community to build the narrative is the only way to ensure that data on outcome achievements is released in a safe and fair way.

2. A bigger dataset on outcomes-based cross-sector partnerships

In recent years, I have encountered many opportunities where brilliant colleagues were showing me outcomes-based projects and asking me if I could include them in the Impact Bond Dataset. I had to reject them, not because they were not valuable, but because they did not meet the definition of a social impact bond. One of the things that I like the least in my job is this one: telling practitioners that they have an amazing project in hand, but they don’t meet a definition.

Our second goal is to create a larger dataset, where we would include any cross-sector partnership that has a focus on outcomes. Of course, this is no easy task. There are many key decisions to be made, and we hope that the community plays an active role in this. For example, we would need to create a definition for ‘cross-sector partnership with an outcomes focus’. When do you know that you are in front of a project with an outcome focus? Is it when you see payment attached to outcomes? When there is a contractual mechanism to achieve that outcomes? Additionally, how do we define partnerships? Do we only include formal partnerships, or do we consider informal partnerships too? The number of inclusion/exclusion decisions that we made for the Impact Bond Dataset will be multiplied if we create a larger dataset on outcomes partnerships. We trust that our community will be involved in the process, helping us understand which projects should be in and out of the new dataset.

3. Finding the sweet spot between “open data” and “no data at all”

In November 2022, we ran a peer learning session where we discussed data sharing practices in outcomes-based partnerships. We concluded that data sharing mechanisms and regulations can differ from project to project. While some projects are clear about sharing their data and learnings from the very beginning, others may struggle to share data during the life of the programme.

INDIGO has developed "sandboxes” for those stakeholders that can’t share their data openly (at least, not yet), but would like to share their data in a closed environment. This will allow the GO Lab to share learnings and insights - without showing all the original data or using it for any other purpose. Our sandboxes are a different database that we can use to store and visualise data without sharing it beyond authorised users, at least until the data owners decide it makes sense to share (some of) it. The development of sandboxes is our way of finding an intermediate point between having open data and having no data at all. We hope that the community will feel confident in using these types of tools to share learnings and insights in the cases where sharing open data is not possible.

We're always keen to connect and learn from INDIGO community (and beyond) as we embark on this mission. As we continue to develop our understanding of the role and value of our data and learning collaborative, why not join us at the Social Outcomes Conference in September, where we'll explore the different uses of data (and all kinds of other innovative practices) to support better social outcomes!