Terhi Kilpi, a physician and paediatrician by training, is the Director of the Department of Public Health Solutions at the National Institute for Health and Welfare, Finland. In her current role, she is responsible for public health evaluation and promotion, including prevention of non-communicable diseases. She previously worked for the same institute in the field of health protection and vaccination as a principal investigator of several clinical vaccine trials and directing the activities of the national immunisation programme.
Prof Kilpi worked as an academic visitor at the GO Lab in October-November 2018 in order to better understand whether social impact bonds could provide a useful tool for responding to the current and emerging public health challenges. In this blog, she summarizes her observations from the perspective of a public health professional and clinical researcher. The views expressed within the blog are hers alone and do not necessarily reflect the opinion of the GO Lab.
Puzzles around social impact bonds
Health, irrespective of what people mean by it, is highly valued, unevenly distributed and widely considered as worth promoting. According to the World Health Organization, the goal of health promotion is to improve equity in health, reduce health risks, promote healthy lifestyles and settings, and respond to the underlying wider determinants of health. Social and environmental factors are thought to affect individual and population health more than the available medical care. Therefore, social and environmental interventions have more potential to improve population health and reduce inequalities than clinical interventions. However, the causal relationship between non-clinical, often complex interventions and outcomes may be difficult to establish.
In recent decades, cardiovascular disease mortality and suicide rates have decreased in many parts of the industrialized world, whereas obesity, an emerging health threat, has increased. Who can tell what effect various campaigns, programs and other efforts of public health authorities and practitioners have had on these trends?
We public health professionals often feel that prevention is underfunded compared to treatment in allocation of public money. However, to make the case for public funding, we need to first demonstrate the impact of preventive interventions. This is often best done by testing them in real-life circumstances, which requires resources, i.e. investment.It was this combination of impact and investment that first raised my interest in social impact bonds (SIBs).
Since SIBs are currently being introduced into Finland with projects on occupational wellbeing and immigrant integration already ongoing and several others in the pipeline, I wanted to understand them better and was fortunate enough to be allowed to spend two months in Oxford learning about outcomes based approaches from the inspiring GO Lab team. Can SIBs help us to provide evidence for investment in prevention? My explorations have given rise to a few puzzles that I try to sort out in the following.
Social impact bonds and competing welfare logics
SIBs are not merely a practical tool that could aid in finding solutions to the most complex social problems of our societies. They appear to elicit strong contradictory opinions. One of the major concerns is that financial logics would prevail over social values. Professor Mildred Warner, a distinguished critic of SIBs argued at the recent SIB Conference co-hosted by the GO Lab that SIBs narrow social values by measuring behaviour change of the vulnerable individuals instead of focusing on such strong market actors as state or private employers. When asked to provide examples of behaviour metrics for state policy or private employers, Prof Warner mentioned paid parental leave and access to high-quality child care. These examples make one wonder whether SIBs might behave differently in different welfare systems.
In the Nordic welfare states, paid parental leave and high-quality child care are widely regarded as part of the extensive social rights of the inhabitants. We are not looking for tools to narrow the established social rights but are interested in finding remedies to the remaining and emerging health and welfare challenges. How should we deal with increasing obesity and type 2 diabetes, addictions or mental health problems, which all contribute to the still prevailing social gradient in life expectancy and health? How should we promote integration of immigrants into the society or support the functional capacity of the ageing population? All these challenges call for collaboration, prevention and innovation, which the recent GO Lab report listed as key elements of the ‘promise’ of SIBs.
SIBs for what kind of problems?
SIBs can hardly be expected to provide a universal solution to the multidisciplinary challenges of prevention. It seems that so far SIBs have been mostly used to reach high-risk populations in need of complex interventions. This does, however, raise questions about the relative impact of SIBs at a population level. If the nature of the financing model is to focus on small at-risk populations, how will SIBs translate to preventive health problems (such as obesity prevention) at a population level? Ideally, the focus on the outcomes of preventive measures should ultimately result in more efficient use of public resources and thereby enable public authorities to expand their activities for the common good.
Puzzling black box
For a former clinical trialist like me, the ‘black box’ approach commonly applied to SIB interventions is much harder to swallow than the other concerns related to SIBs. The black box approach is a novel principle of outcomes based commissioning, in which the commissioner does not specify what the intervention should be but simply agrees to pay for outcomes if they are delivered. In the world of controlled trials, it is generally considered essential to precisely describe the intervention before testing its effectiveness. Since SIBs typically appear to finance complex interventions with multiple ingredients, it becomes impossible to conclude, which of the many services were beneficial and which possibly were more or less useless (or even harmful), even if the overall intervention had the desired impact. How could one be able to repeat and scale up such a non-standardized procedure?
The black-box approach is hoped to create room for innovation, which is one of the great expectations from SIBs. At this point it is too early to say whether or not SIBs will live up to this expectation. Since SIBs mainly operate in the territory of complex problems with no straightforward solutions and since some degree of unorthodoxy probably is an essential element of any successful innovation, I can accept the black box approach as a potential initiator of innovation. Nevertheless, this is where scientific research has to step in. We have to find a way to evaluate the process and components of the SIB interventions with scientific rigour.
The attribution problem
It is not irrelevant to SIB commissioners whether they pay for outcomes that truly resulted from the interventions or just happened anyway. The issue of causal attribution is repeatedly mentioned in the SIB literature and establishment of a counterfactual (what would have happened if the intervention had not taken place) has been recommended. However, causal attribution with experimental or quasi-experimental design appears to be rare in European SIBs, whereas fairly rigorous counterfactual evaluations, including randomised control trials (RCTs) have been almost the rule in the US SIBs (see for examplea post by Prof Chris Fox). Prof Fox correctly points out that if we don’t focus on attribution, it will become hard to demonstrate that SIBs are more than a series of interesting pilots.Considering that RCTs are the most rigorous method to produce the most accurate (i.e. unbiased) estimates of the effectiveness of interventions, this method should not be disregarded when designing SIB evaluations.
Call for transparency
SIBs involve commitments to spend public money and should thus be subject to scrutiny around value for money, risk taking and the effectiveness (and even potential harmfulness) of the services they fund, as Fraser and colleagues state in their Evaluation of the Social Impact Bond Trailblazers. Concerns have been expressed regarding the transparency of SIBs. These concerns are usually related to the financial arrangements and justification of payments. It would also be important to subject the impact of the SIB interventions to peer review of scientific journals. Had the impact of the interventions been scrutinized through peer-review, many of my puzzles would probably have been resolved by now.
One way to increase the transparency and avoid biased reporting of the outcomes could be to have detailed information on SIBs publicly available in databases similar to those currently required for clinical trials (e.g. clinicaltrials.gov)and to an increasing extent also for systematic reviews (e.g. PROSPERO).
The noble concept of promoting public health and welfare means that various actors try to do something good. In addition to good intentions, we need evidence of the effect of the intervention, not least to justify the investment of public money. Both the evidence of impact and the public investment are often hard to achieve. SIBs, if applied appropriately, can address both of these challenges. Good practise of SIBs can best be achieved with the help of scientific research, which investigates both the commissioning structure and the effectiveness of SIB-type interventions. During the past two months, I have witnessed the admirable and outstanding work the GO Lab is doing in this area. As we hopefully make progress in developing outcomes based approaches in Finland, we will closely follow the GO Lab’s research and pay attention to their conclusions on the capacity of SIBs to boost innovation and collaboration and help us to make an evidence-based case for prevention.