Project K in Black & White: A theory-driven and randomised controlled trial evaluation of a youth development programme

Abstract

Project K is a New Zealand-based positive youth development programme that combines a three-week wilderness adventure, a five week community exploration and service component, and a year-long mentoring partnership to promote positive growth in 14-15 year olds with low self-efficacy. The primary intention for this research was to produce credible knowledge about Project K in a manner that empowered programme stakeholders to address their information needs. Another intention was to produce knowledge that contributes to academic understanding of youth development programming and programme evaluation. A comprehensive mixed-method evaluation of Project K was guided by a novel theoretical framework, the Thoughtful Evaluation model. The model stresses an approach that balances aspects of scientific credibility and stakeholder empowerment. Findings from eight programme staff focus groups, 361 open-ended comments about the programme from youth participants, four key programme documents, results from six previous Project K research projects and thorough literature reviews of youth development, adventure, service-learning, and mentoring programming were triangulated and integrated into a programme theory of change. A logic model depicting the theory suggests that Project K reinforces participant growth with experiential learning and scaffolded support using an iterative and cyclical process. Characteristics of the individuals involved and the programme environment reportedly influence the success of this process. Hierarchical linear modelling of data from 1092 Project K and Control participants (at baseline) collected as part of a randomised controlled trial revealed that Project K was effective in improving academic and social self-efficacy from pre to post-programme. Effects were sustained one year post-programme. Project K participants reported higher career decision self-efficacy one year post-programme but no difference existed between the groups for achievement on the major national qualification offered in New Zealand secondary schools (NCEA, Level 1). The interaction effects obtained between programme condition and gender, ethnicity, or socioeconomic school status indicated that Project K appears to reduce disparities between different participant subgroups over the follow up period. While limitations were identified with Project K, this research demonstrated that it is a 'model' programme in terms of best practice principles and programme effects. The rigorous, comprehensive and process-focused nature of the evaluation offers a valuable contribution to youth development programme designers, practitioners, policymakers, and researchers interested in enhancing the function of similar programmes. In addition, the Thoughtful Evaluation model is offered as a useful vision for single-evaluator situations.