This is a guest post from the Justice Data Lab, which uses quasi-experimental methods to help organisations supporting the rehabilitation of offenders to understand and improve their effectiveness.
Across the country, there is a wide range of programmes aimed at supporting the rehabilitation of offenders. Without access to follow-up data about their clients, however, it can be difficult for providers to have quantifiable evidence as to whether they have had a direct impact on recidivism.
The Ministry of Justice’s Justice Data Lab (JDL) was set up to change that. We support rehabilitation organisations by providing vital evidence to understand their effectiveness at reducing reoffending in a secure and transparent manner.
As the first established Data Lab service available to any and all providers, the JDL has become a leader in frontline service evaluation, a key tool to feed into the bigger picture of ‘What Works’ in reducing reoffending behaviour.
How does the Justice Data Lab work?
At the JDL, we overcome the issue of access to sensitive data by providing easy access to aggregate reoffending data about the specific people that rehabilitation organisations work with.
Organisations submit details of the offenders they have worked with to the JDL team along with information about the intervention they have delivered. We then link these offenders to the Ministry of Justice's central data sets and report back on the reoffending rate of the particular cohort, alongside that of a comparison group of offenders with very similar characteristics in order to identify the impact of the organisation’s work.
“Propensity score matching”
We do this using a method called propensity score matching. It allows us to identify a group of similar offenders based on characteristics observed in available data. These characteristics might include gender, ethnicity, age at offence and criminal history. Crucially, members of both the treatment group and the comparison group are judged to have similar propensity to reoffend. The comparison group has not been exposed to the intervention in question.
The provision of a comparison group is extremely important, allowing us to establish whether changes to a cohort’s offending behaviour is attributable to the rehabilitation programme, rather than other variables.
What has worked so far?
Since the launch of the JDL in April 2013, the outcome measures we use have developed from a single reoffending rate to include the frequency of reoffending and time to first reoffence. This gives us a much richer idea of not only which programmes are effective, but also how they are making a difference.
We have now published reports covering over 200 comparison group analyses across a range of intervention types, from prison-based education and post-release accommodation support to the impact of Women’s Centres.
Some programmes have shown clear signs of success, but other results have not crossed the statistical significance threshold, which is most likely due to the size of treatment groups analysed. Because of this, we include a ‘What you can say’ and ‘What you can’t say’ section in all reports so that it is clear what the results actually indicate.
Even statistically insignificant reports add to the evidence base, though. At the moment, meta-analyses are being investigated to bring different sets of results together to demonstrate what overall type of interventions – such as those based on employment or education – work in reducing reoffending.
With this growing number of analyses, we can start to get a better picture of the kinds of programmes that are – or are not – really working, and help organisations build better services based on sound evidence.
Reoffending behaviour is only part of the picture in assessing the impact an intervention can have on an offender. At the JDL, we’re starting to think about how we can capitalise on cross-government data-sharing – rather than simply relying on Ministry of Justice data – to add more context to the picture.
With such rich linked data, we can begin to understand the effect on the offender of exposure to other services and benefits – such as secure accommodation, improved health or steady employment.
We could also investigate the effect of rehabilitation programmes not only on crime rates, but also on outcomes such as employment and income. This is a key area where we plan to enhance our outputs moving forward.
Whilst we could sit back and celebrate the successes and awards we have received so far, at the JDL we want to keep developing the service and lead by example – to be as useful as possible to our customers, continue to attract new clients and strive to make the Data Lab an industry-wide standard in demonstrating effectiveness using good data and robust analysis.