In Their Hands

This case study reflects on the experience of the ‘In their hands’ project, the world’s first Development Impact Bond with a focus on adolescent sexual and reproductive health.

Key stakeholders mentioned different reasons for using an impact bond model.

From the perspective of Triggerise, the implementer, the fact that payments were tied to two concrete outcome metrics generated a shift in the mindsets of all employees. The whole organisation was focused on achieving these outcomes and they aligned all their activities towards them. They describe the process of working under an impact bond as iterative, given that the outcomes focus demanded constant improvement and adaptation from the organisation.

Furthermore, the outcomes focus made them rethink how to run the ITH programme. For instance, Triggerise provided access to a dashboard to closely monitor outcome achievements and as both the investor and funder had access to this dashboard, they were kept up to date with the performance of the bond.

The CIFF team also recognised that the impact bond made them spend more time on analysing performance metrics and focusing on the desired results. Having launched the same programme with a grant approach, CIFF practitioners identified a clear increase in accountability levels. Before the impact bond project, they were only accountable to themselves as they had given a grant to the provider. After transitioning from grant-provider to investor, they still find the usual internal accountabilities and in addition, they become accountable to the outcome funder too.

According to the KOIS team, an impact bond was a timely model for these partners as they were looking for an opportunity to externalise risk to an investor, attract a diversified range of stakeholders, increase visibility of the project, and integrate some innovative elements (such as the use of real-time data). As they knew that the intervention worked well, experimenting with an impact bond model was an opportunity to scale up their project.

The three parties agreed that the ITH DIB had been a positive experience where key lessons should be documented and shared.

The implementer was able to prove the scope of their platform and how much the data behind it could help projects adapt and improve. They also identified the need to re-think the model so that there is more incentive to think of the bigger picture instead of solely focusing on pre-set outcome metrics. This would encourage projects to achieve not only pre-agreed impact but also other positive impacts that were not foreseen in contracts.

The investor raised the issue of value for money analysis and the fact that the sector needs to reflect more on potential ways of reducing the costs of setting new impact bond projects. Even if these projects achieve positive social outcomes, it is still unknown if there is another model to achieve the same outcomes in a more cost-efficient way.

The outcome funder was interested in the opportunities that the use of technology could bring for future projects. In addition, the FCDO signalled that the lessons from this project will be key to improve the design of the next phase of the ITH programme. The United Nations has consulted the stakeholders about additions and improvements to the original design of the service. The FCDO suggested that the programme could be improved by seeking a more active engagement from the government, including HIV services in the basic programme, including mental health services and adding a new branch that could also attract boys to the conversation. The KOIS team recommended adding a bonus for overperformance, so as to incentivise providers to make the most of their time under an impact bond.

As of July 2022, the United Nations Joint SDG fund has invested USD 7 million to scale up the ITH programme. This new tranche will be a three-year programme to improve access and uptake of sexual and reproductive health and HIV services among low-income adolescent girls in 10 counties in Kenya. In addition, this new phase of the DIB will also have more engagement from the public sector as the national government will be a member of the project committee.

Most of the challenges and risks highlighted in previous sections are common to other development impact bonds. For instance, several organisations have highlighted the fact that impact bonds often imply added management cost. This is why it is vital that the different parties are clear about the rationale for using this mechanism and what is the added value that an impact bond could add to the programme. The question that remains unanswered is whether the added costs are worth the added benefits. In the case of the ITH DIB, it would be interesting to commission a study to compare the results of the programme delivered under a grant and the results of the programme under a DIB model to better understand not only how costs differ but also how benefits are achieved in each case.

Discussions around outcome metrics are also common in the field. As impact bonds are focused on predetermined metrics, it is key that the parties agree on the appropriate indicators. In this sense, the GO Lab guide to setting and defining outcomes can be helpful. The general recommendation is to select targets that are reasonable to achieve in the period of time of the specific project and measurable at acceptable cost and effort. When there is no reasonable and measurable outcome target for the period of time of the project, a sound theory of change can justify the selection of a proxy indicator, based on activities rather than outcomes.

Several organisations have highlighted the fact that the focus on a set of metrics can be seen as a strength and weakness of the model at the same time, and have suggested the possibility of introducing more flexibility in the outcome’s verification stage. However, this would also open the possibility for organisations to overbid to win a contract and then use the flexibility to renegotiate targets down.

The fact that the baseline was done 10 months after the start of service delivery affected the payment structure of the DIB. This was related to unforeseen complexities with contracting an evaluator for the DIB. As the main stakeholders agreed that it was not fair to work with a late baseline, the DIB paid outcome payments exclusively on a service delivery basis. In addition to finding ways to reduce costs, it is also necessary to improve the organisation’s capacity to procure and contract key services for the project in a timely manner.

The aim of this case study was to highlight factors that favoured the impact bond model and those that presented challenges or difficulties. The next step of the ITH programme is to scale up one more time under a new impact bond model that will be based in Kenya and funded by United Nations. Many of the difficulties signalled by CIFF, Triggerise and FCDO will be addressed and other organisations will be included in the project, as it is expected to receive more upfront investment, deliver more services and for more service users.

Representatives from the Children’s Investment Fund Foundation (CIFF), the Foreign, Commonwealth & Development Office (FCDO), Triggerise and KOIS were interviewed between February and March 2022. We are thankful to all of them for their time and generosity in sharing their insights and experiences.