Curiously, many behavioural scientists say the biggest contribution behavioural science has made to government is more and better evaluations. Yet few evaluators would say the reverse – that behavioural science has influenced the conduct of evaluations.
Since the ultimate purpose of evaluation is to figure out how to make things better – and every policy, program, and service involves behaviour – integrating behavioural science into evaluation is essential. Behavioural science can improve a Theory of Change, offer new methods for data collection and analysis, and generate more practical recommendations based on evaluation findings.
In my career, half my job titles have been ‘behavioural scientist’ and half have been ‘evaluator’. The evaluation team I currently lead at Kantar Public Australia integrates these approaches to maximise the value we provide to our clients. Let me explain how with an example.
But first, here’s a quick primer on applied behavioural science.
Behavioural science involves designing interventions to change behaviour and then evaluating them. Unlike physics where there are ‘laws’ which predict outcomes, in behavioural science we can’t always predict how someone will behave. This is because behaviour is context-specific, and a wide range of theories and concepts are applied in the literature.
Behaviour is context-specific: A complex mix of subtle influences in the environment shape what we pay attention to, how we interpret that information, the range of behavioural choices we consider, and whether we follow through on our intentions. The famous Texan anti-littering sign “Don’t mess with Texas” is tailored to the behavioural environment by providing timely reminders (displayed on highways where people throw litter out of their car window) that leverage culture (Texas swagger).
A wide range of theories exist: Every day the behavioural science literature grows as new concepts and theories are developed and old ones are contested or revised. For example, a recent study challenged the concept of the ‘mere exposure effect’ which is the idea that we prefer faces we've seen more often. A number of projects exist just to check the reliability and generalisability of such concepts because of the failure of many studies in psychological science to replicate.
If we can’t perfectly predict people’s behaviour, then we need to evaluate behavioural interventions to ensure they will have the desired effect. Therefore, behavioural scientists understand the value of evaluation.
So how should evaluators understand the value of behavioural science?
One of the easiest ways to integrate behavioural science into evaluation is during the development of a Theory of Change.
A Theory of Change (ToC) is a conceptual diagram of how and why a program works. It explicitly displays the ultimate goal, long-term, medium-term and short-term outcomes, outputs, inputs, assumptions and rationales. The process of developing a ToC with stakeholders ensures logical links are drawn between what an organisation does and what it hopes to achieve.
A good ToC should include the psychosocial mechanisms (internal reactions and reasonings) that trigger outcomes, as well as recognise that programs are delivered in social and economic contexts, which can influence how they work. Examining relationships between contexts, mechanisms, and outcomes (Realist evaluation) allows evaluators to say more than just whether a program works or not, but also how, why, for whom, and under what conditions.
Integrating concepts from the behavioural science literature into a ToC therefore opens up a wider range of mechanisms and contexts to explore, which in turn improves our understanding of causal pathways so we can make much more practical recommendations to improve outcomes in the future.
This article was issued under our former global brand name: Kantar Public.