RSVA case study
Gloucestershire colleague adopted RSVA to evaluate the impact of the No Child Left Behind initiative which was led by Cheltenham Borough Council and partners.
“No Child Left Behind (NCLB) is an initiative started by Cheltenham Borough Council and partners. This involved a 12-month programme of activities in 2019, including a campaign on period poverty, a children’s festival, child poverty summit and launching restorative practice in schools. A range of organisations and schools have also signed the NCLB Community Agreement which is a commitment to work in a more holistic and trauma-informed way with children and families.
The Social Return on Investment (RSVA) tool, as adapted by the Public Health team, was chosen to evaluate the impact of the NCLB activities and community agreement because of its broad focus on a range of social, economic and environmental value that results from interventions.”
“No Child Left Behind (NCLB) is an initiative started by Cheltenham Borough Council and partners. This involved a 12-month programme of activities in 2019, including a campaign on period poverty, a children’s festival, child poverty summit and launching restorative practice in schools. A range of organisations and schools have also signed the NCLB Community Agreement which is a commitment to work in a more holistic and trauma-informed way with children and families.
The Social Return on Investment (RSVA) tool, as adapted by the Public Health team, was chosen to evaluate the impact of the NCLB activities and community agreement because of its broad focus on a range of social, economic and environmental value that results from interventions.”
“Initially, we spoke with the NCLB steering group, comprising of representatives from Cheltenham Borough Council, teachers, school governors, and local voluntary and community sector (VCS) organisations. This conversation led to the selection of Oakwood Primary School as a suitable case study for the analysis.
A discussion with the Operational Head of Oakwood School, the district’s Governor and an NCLB coordinator followed to design a suitable stakeholder list. As a result, the evaluation engaged with pupils, parents and carers, staff members and governors.”
“Initially, we spoke with the NCLB steering group, comprising of representatives from Cheltenham Borough Council, teachers, school governors, and local voluntary and community sector (VCS) organisations. This conversation led to the selection of Oakwood Primary School as a suitable case study for the analysis.
A discussion with the Operational Head of Oakwood School, the district’s Governor and an NCLB coordinator followed to design a suitable stakeholder list. As a result, the evaluation engaged with pupils, parents and carers, staff members and governors.”
“Having decided on the stakeholder list, we developed materials to support the evaluation, including consent forms, participant information sheets, a survey, and focus group prompts. The team conducted focus groups with parents and carers, staff members, and pupils on their perceptions of the No child Left Behind initiative and what changes they valued since the school implemented the community agreement. Engaging with a broad range of stakeholders helped to develop a theory of change which showed the relationship between inputs, outputs and outcomes.”
“Having decided on the stakeholder list, we developed materials to support the evaluation, including consent forms, participant information sheets, a survey, and focus group prompts. The team conducted focus groups with parents and carers, staff members, and pupils on their perceptions of the No child Left Behind initiative and what changes they valued since the school implemented the community agreement. Engaging with a broad range of stakeholders helped to develop a theory of change which showed the relationship between inputs, outputs and outcomes.”
The advantages of the tool are:
- It is straightforward to use and captures a broad range of impact and value that results from a project. This was well suited to a project with holistic and wide-reaching aims
- The RSVA tool emphasises the importance of the stakeholder perspective and the things they valued over the intended outcomes. This is useful in cases where traditional cost/benefit analysis would find it difficult to pin down a return on investment.
The disadvantages are:
- It was reasonably resource intensive to complete
- Not all stakeholders were familiar enough with the aims of the initiative to feedback confidently about its impact
- It was not possible to isolate the impact of NCLB from other changes, including other action taken by the school during the Covid-19 pandemic and home schooling.
The advantages of the tool are:
- It is straightforward to use and captures a broad range of impact and value that results from a project. This was well suited to a project with holistic and wide-reaching aims
- The RSVA tool emphasises the importance of the stakeholder perspective and the things they valued over the intended outcomes. This is useful in cases where traditional cost/benefit analysis would find it difficult to pin down a return on investment.
The disadvantages are:
- It was reasonably resource intensive to complete
- Not all stakeholders were familiar enough with the aims of the initiative to feedback confidently about its impact
- It was not possible to isolate the impact of NCLB from other changes, including other action taken by the school during the Covid-19 pandemic and home schooling.
“Although the RSVA methodology did not enable us to isolate the impacts of NCLB from other changes resulting from the pandemic, it was encouraging to see how well the words and phrases used by stakeholders in their feedback aligned to the intended ways of working and actions under the Community Agreement.
Culture change is notoriously difficult to measure and attribute to particular causes. However, it also has the potential to create long-lasting change and is very difficult to implement effectively so any evidence of culture change having taken place should be viewed positively.”
“Although the RSVA methodology did not enable us to isolate the impacts of NCLB from other changes resulting from the pandemic, it was encouraging to see how well the words and phrases used by stakeholders in their feedback aligned to the intended ways of working and actions under the Community Agreement.
Culture change is notoriously difficult to measure and attribute to particular causes. However, it also has the potential to create long-lasting change and is very difficult to implement effectively so any evidence of culture change having taken place should be viewed positively.”
“The resources produced during this case study have been adapted to be useful for others completing an RSVA evaluation. This should hopefully reduce the administration burden.”
“The resources produced during this case study have been adapted to be useful for others completing an RSVA evaluation. This should hopefully reduce the administration burden.”
“It gave me, personally, a greater understanding of the complexity of inequalities and how they manifest in a deprived area of an affluent town. Though the staff, parents and even the children, had an ever-present awareness of the disadvantage faced by members in their community, the way they came together and supported one another above and beyond what a school traditionally does, was inspirational. That narrative was captured well in the social value evaluation tool developed and would be missing from any purely quantitative evaluation.”
“It gave me, personally, a greater understanding of the complexity of inequalities and how they manifest in a deprived area of an affluent town. Though the staff, parents and even the children, had an ever-present awareness of the disadvantage faced by members in their community, the way they came together and supported one another above and beyond what a school traditionally does, was inspirational. That narrative was captured well in the social value evaluation tool developed and would be missing from any purely quantitative evaluation.”
“The input from the community development team that commissioned the evaluation was very important in the initial stages but this engagement would benefit from continuing throughout the process, including making time to thoroughly feedback on the findings.”
“The input from the community development team that commissioned the evaluation was very important in the initial stages but this engagement would benefit from continuing throughout the process, including making time to thoroughly feedback on the findings.”
“Approximately four months, though not continuously or full time.”
“Approximately four months, though not continuously or full time.”
“The most suitable candidates for an RSVA evaluation are those in which the inputs, outputs, outcomes and impacts are easily identified and that recipients of the initiative are aware of it.”
“The most suitable candidates for an RSVA evaluation are those in which the inputs, outputs, outcomes and impacts are easily identified and that recipients of the initiative are aware of it.”
“Publicly available sources of data on health inequalities tend to only go down to the upper tier local authority level, which is not very useful if you want to look at inequalities within a local authority i.e. we can find out how Gloucestershire as a whole performs on a wide range of indicators, and sometimes the districts but this doesn’t say anything about how the burden of disease and health determinants are distributed in our population.”
For more information, please contact maria.arthurs-hartnett@gloucestershire.gov.uk, Gloucestershire County Council.
“Publicly available sources of data on health inequalities tend to only go down to the upper tier local authority level, which is not very useful if you want to look at inequalities within a local authority i.e. we can find out how Gloucestershire as a whole performs on a wide range of indicators, and sometimes the districts but this doesn’t say anything about how the burden of disease and health determinants are distributed in our population.”
For more information, please contact maria.arthurs-hartnett@gloucestershire.gov.uk, Gloucestershire County Council.
“Publicly available sources of data on health inequalities tend to only go down to the upper tier local authority level, which is not very useful if you want to look at inequalities within a local authority i.e. we can find out how Gloucestershire as a whole performs on a wide range of indicators, and sometimes the districts but this doesn’t say anything about how the burden of disease and health determinants are distributed in our population.”
For more information, please contact maria.arthurs-hartnett@gloucestershire.gov.uk, Gloucestershire County Council.
“Publicly available sources of data on health inequalities tend to only go down to the upper tier local authority level, which is not very useful if you want to look at inequalities within a local authority i.e. we can find out how Gloucestershire as a whole performs on a wide range of indicators, and sometimes the districts but this doesn’t say anything about how the burden of disease and health determinants are distributed in our population.”
For more information, please contact maria.arthurs-hartnett@gloucestershire.gov.uk, Gloucestershire County Council.
“Publicly available sources of data on health inequalities tend to only go down to the upper tier local authority level, which is not very useful if you want to look at inequalities within a local authority i.e. we can find out how Gloucestershire as a whole performs on a wide range of indicators, and sometimes the districts but this doesn’t say anything about how the burden of disease and health determinants are distributed in our population.”
For more information, please contact maria.arthurs-hartnett@gloucestershire.gov.uk, Gloucestershire County Council.
“Publicly available sources of data on health inequalities tend to only go down to the upper tier local authority level, which is not very useful if you want to look at inequalities within a local authority i.e. we can find out how Gloucestershire as a whole performs on a wide range of indicators, and sometimes the districts but this doesn’t say anything about how the burden of disease and health determinants are distributed in our population.”
For more information, please contact maria.arthurs-hartnett@gloucestershire.gov.uk, Gloucestershire County Council.
Last reviewed: