Over the past 5 years, the What Works Team has championed what we call the ‘What Works approach’ as a better way of making policy across government: identifying problems using good data, designing a solution using the best evidence available, and testing it with robust methods like randomised controlled trials.
This way of working was set out in the Cabinet Office paper, ‘Test, Learn, Adapt’, and we stand by it as an ideal model for making policy. Using established methods, it enables us to really understand whether programmes are delivering the desired outcomes, and helps to test different approaches within programmes to deliver maximum impact. The approach is still hugely underused, and much of what we do is centred on advocating its adoption across government.
As a part of this, we also look to improve the ability of policy teams to run and commission trials themselves. In our work, we speak to civil servants from across government who want to adopt the ‘Test, Learn, Adapt’ approach, but lack access to the expertise to design and implement a high-quality trial.
Improving the quality and quantity of impact evaluations in government
It was for this reason that we created the Cross-Government Trial Advice Panel (TAP) in 2015.
The panel connects policy and operational delivery teams to government and academic trialling experts, with the goal of increasing the number and quality of trials in government.
TAP’s experts – around 50 of them from government and universities – are on hand to give free advice to civil servants who are thinking about designing, commissioning, implementing or analysing a trial.
Conversations with TAP members are confidential, and while we take pains to ensure that customers end up with a useful and rigorous trial, there is no obligation to follow the panel’s advice. In short, we’ve tried to make it as easy and appealing as we can for civil servants to make use of TAP’s expertise.
A flexible resource
Over the last 2 years, the panel has helped its clients in a number of ways, from power calculations to peer reviews. Sometimes TAP’s relationship with a policy team lasts the length of a phone call, and in other cases panel members remain engaged throughout the lifecycle of the trial.
Through our work with TAP, we’ve encountered dozens of budding social scientists – analysts, researchers and policy makers from multiple departments who have been looking to conduct a trial but lacked the confidence or encouragement to do so. Some want a quick skim-read of their plans, and others understand the benefits of a trial but need sustained support to help them create a high-quality evaluation.
Whether you're a civil servant looking for in-depth support or a quick Q&A, we’re happy to hear from you. Get in touch if you’d like to know more: TrialAdvicePanel@cabinetoffice.gov.uk.
Leave a comment