In order to assess the value of the reforms and support continuous learning over the reform implementation period, KPMG in Australia was engaged to undertake a three-year evaluation of the reforms.
Taking a holistic view of change
Founded in the Children, Youth and Families Act of 2005, the government recognized that – to ensure the reforms were achieving the stated policy objectives and be accountable to ongoing funding – they would need to put a rigorous evaluation process in place.
To manage such a wide-scale and long-term evaluation and support the capacity for continuous learning throughout the evaluation period, the department required the support of an independent evaluator that could offer a more holistic view of the reforms. A strong understanding of the Child and Family Services System, a strong evaluation capability and expertise in data analysis would be demanded, as would an ability to qualitatively assess the impact of the reforms on the various stakeholders involved: the departmental workforce, its service providers, related professionals who may report to Child Protection and vulnerable children, young people and families themselves.
Leveraging experience to drive success
With a long and successful history of collaboration with the Child and Family Services sector, the department selected KPMG in Australia to develop and conduct the evaluation. The firm had been involved in developing key elements of the programs of reforms, as well as the creation of the performance standards that would ultimately define how the new services would operate.
Based on this hands-on experience and the Australian firm’s ability to draw on a strong bench of professionals with both direct service delivery and policy-based experience, KPMG developed a robust evaluation framework aimed at assessing the implementation of the reforms to improve services going forward.
Painting a clear picture
Central to the evaluation was the Australian firm’s development of a mixed methods approach that combined quantitative and qualitative analysis to deliver a clear vision of the progress and achievements.
For example, the team conducted a series of case studies to evaluate the impact of introducing community based earlier intervention services within a variety of diverse locations across the State; these locations included a spread of metropolitan and rural areas, those that had higher Aboriginal populations and those that had different levels of community need and risk. The team also developed a workforce survey that was deployed at various points during the reform process to ascertain the change and impact of the reforms on Child Protection, Out of Home Care and Family Services workforce. This was further augmented by focus groups and workshops with individuals such as family service workers who were responsible for embedding the change into their practices.
Reaping the benefits of success
As a result of their work, the KPMG team delivered a series of in-depth and practical reports that enabled the government to not only track the progress of the reforms, but also to identify opportunities for further reform.
Furthermore, the evaluation has also contributed to other policy decisions for the Department of Human Services Victoria, particularly in designing the Child Protection workforce reform program.