How much does ‘it’ cost? (part 2) - Understanding opportunities and barriers to sharing data on Development Impact Bonds (DIB) costs

For the 2023 Spring edition of our Hack and Learn event, hackteam#28 decided to continue the work of a previous Hack and Learn challenge on “transaction costs”. In essence, transaction costs are the costs associated with making a market exchange. They broadly consist of the costs incurred to identify a counterpart (i.e., a supplier or buyer), to negotiate terms, to implement and monitor the agreed exchange, as well as enforce the agreement if needed. In 2022, a group of Hack and Learn-ers decided to create a set of variables to measure the transaction costs that impact bonds incur during the development of the project, delivery and evaluation of results. Our sub-team decided to test how well the cost variables capture the transaction costs of four development impact bonds (DIBs) supported by the FCDO DIB pilot programme, which had recently published some transaction cost information.

What did we “hack”?

Our group focused on the data from the evaluation report of the 2019 UK FCDO DIBs pilot programme. This evaluation provides an assessment of how the DIB model affects the design, delivery, performance and effectiveness of development programmes, and it includes a cost analysis for the initial stages of four DIBs. (Since the 2023 Spring edition of the Hack and Learn, a new report on the FCDO Development Impact Bond Pilot Programme has been published). Participants extracted cost data from the report and populated the set of cost variables in the Impact Bond Dataset spreadsheet.

What did we learn?

The exercise of matching available data to the existing standard model was more challenging than expected! Here we set out three main challenges for standardising data on DIBs’ costs identified by the group, and corresponding ways to move forward.

Data is most useful when it is comparable. To compare results across projects, data needs to be collected in a standardised way, making sure that data definitions are the same for every project and data collection follows the same processes and assumptions. Yet, project teams often work with financial management systems which impose different data standards. In the future, it will be essential to develop a common standard to report cost data in a comparable manner.

The cost categories used in the FCDO DIB Pilot Evaluation evolved over time. By the third iteration, there was some generally agreed categories:

• Service delivery
• Verification
• Investment vehicle
• Trustee fees
• Governance
• Performance management

If a common standard is to be agreed and developed, using data that is readily available is a good place to start. For example, a service provider may already be reporting direct costs (service delivery) and indirect costs (governance and management). Other common costs may be available in service contract agreements (e.g., sub-contractors providing specific services such as verification, performance management, etc.), which may make costs easier to identify than when these services are provided in-house. However, it may still be difficult to determine how much of the cost is specific or additional to the DIB arrangement (in contrast to a more conventional funding arrangement).

Before delivering a service using a DIB model, all the stakeholders invest time and effort in developing and designing the programme. The time and effort invested in this initial stage has a cost, but it is difficult to estimate its monetary value. For instance, legal teams, procurement teams and financial advice teams from various organisations invest several hours in designing and preparing contracts. These teams usually work with other projects too (e.g., procurement officers may be working on several projects at once) and can only provide a very approximate estimate of how much time they spent on each project. Having a rough estimate is a step forward as we begin to get a sense of how many hours went into the preparation stage. However, it is difficult to compare across projects, as estimates can be subject to personal biases, and we would be comparing non-standardised data. Once again, being explicit on the underlying assumptions and working towards a common reporting standard would be beneficial.

Many of our discussions touched on the topic of context. Cost data becomes useful when it is analysed alongside other key data points, such as the maximum potential outcome payments (or the whole size of the contract), the private capital raised, the number of service users that are expected to be served, the length of the intervention, and the challenges of delivering that service in a particular context. In the end, the question that most practitioners and policymakers are trying to answer is one of value-for-money: was that a good use of resources in terms of the results achieved? In addition, it would be helpful to understand how the costs and benefits of using the DIB compare to alternative funding mechanisms – building on the great work of the other sub-team of hackteam#28. To answer these complex questions, better and more standardised data on costs is a step forward, but it is only one piece of a bigger puzzle the community is trying to solve together.

A meaningful analysis of cost-effectiveness, or the transaction costs associated with a DIB, should include a good understanding of the project’s context. All these variables contribute to a broader story around a DIB project that is necessary to understand before making any final judgements. The INDIGO community is making much progress in improving the availability and standardisation of data, including data on costs, by working with pioneers who volunteer to be the first ones to share cost data and build the narrative. The 2024 Spring edition of the Hack and Learn will be the ideal venue to continue this work!