Learning from evaluations of school-family strengthening programs: lessons for all

For more than 10 years, a program called Families And Schools Together (FAST) has been run in schools across the Northern Territory. These programs have always had an evaluation component built in. However, over time, the evaluations have changed. Initially, they were summative, built around a quantitative psychometric tool (with a positivist research paradigm). The intent of the summative evaluation was to demonstrate the effectiveness of the program. However, as the program was rolled out in remote contexts, the need for adaptation was recognised. Changes were made but it was soon recognised that other outcomes, not captured or explored in the methodology, were emerging. After six years of working together, the evaluator and the program manager felt that it was time to explore effectiveness in a different way with a more qualitative evaluation process (based on naturalistic and pragmatic paradigms). The purpose of this paper is to share learnings from this experience with other program managers and evaluators. The presentation will be an opportunity for participants to engage in a discussion about monitoring and evaluation from a program evaluation perspective, particularly taking account of the complexities of the northern Australian context. This paper explores the six year learning journey through evaluation that the evaluator and program manager have undertaken. It describes tensions between the need for reliable and generalizable objective quantitative data and the need for authentic and credible data based on participant experience.