Browsing by Subject "Programme Evaluation"

Now showing 1 - 20 of 63
  • Thumbnail Image
    Item
    Open Access
    A Design and Short-term Outcome Evaluation of the Commerce Faculty’s First Year Experience Mentoring Programme
    (2018) Moeketsi, Themba; Field, Carren
    This dissertation reports on a design and outcome evaluation of the Commerce First Year Experience (FYEM) programme. The FYEM programme is implemented by the Centre of Higher Education Department at the University of Cape Town. The FYEM programme is an academic and social supportive cooperation between universities faculties, departments, mentors and facilities for first year Commerce students. The collective efforts intend to: i) provide the first year students with a sense of belonging, ii) help students to adapt to university, and iii) offer academic and emotional to the students. A three repeated measure was used for this study. The measures were conducted in May, June and July. The first set of evaluation questions were based on the design of the programme: i) should mentoring be compulsory for first year students? ii) what is the effective duration for tertiary mentoring, iii) what is the most effective way of assigning mentors to mentees? and iv) what is the most effective medium for mentoring relationships? The second set of questions was based on the intended short-term outcomes of the programme: v) did the mentoring programme improve the psychosocial and academic attributes of students at observation time one to observation time two? vi) were these effects sustainable, meaning did the students continue to improve even after the end of the programme between observation time two and observation time three? and vii) was a difference between mentored and non-mentored students over time? The theory design results illustrated that selecting appropriate mentors requires setting up criteria based on certain characteristics. The duration of mentoring should be one year to allow the full transition of first-year students to the university. Another result showed that mentoring should be compulsory for the first year students provided that there is an opt-out option. Mentors and mentees should be given the opportunity to select their preferred individuals to work with and not be randomly assigned. Additionally, the most effective medium for mentoring was group sessions instead of one on one. The short-term outcome results for the mentoring programme were statistically insignificant due to the small sample size. There was, however, a small improvement in the means of the psychosocial and academic outcomes. After three months of the programme, there were no significant differences between the mentored and non-mentored students. The sample size may have contributed to the lack of effect as well as lack of qualitative data. In conclusion, it is evident that FYEM programme results were statistically insignificant. However, positive increases were observed for the mentored students’ sense of belonging, peer interaction, academic adjustment and engagement. The limitations of this study must be addressed so that more reliable and valid data can be gathered. The results from the design evaluation component can be used to improve the programmes’ implementation in 2017.
  • Thumbnail Image
    Item
    Open Access
    A formative evaluation of the development and implementation of the waves for change coach training programme
    Rolfe, Elana; Dawes, Andy
    This dissertation reports on a formative evaluation of the initial implementation of the Waves for Change coach training programme. Waves for Change is a NPO that combines the sport of surfing with youth care work (what the organisation terms surf therapy) to provide psychological support to at-risk children. The individuals who provide the support (Coaches) are young adults who are employed and trained by the organisation to become youth care workers. Coaches attend a training programme designed by Waves for Change. There are three Coaching teams to reflect the three target communities of the organisation. The aim of this evaluation was to provide data on its implementation so as to improve its design and delivery. Due to the vulnerable nature of the surf therapy beneficiaries it was crucial for the evaluation to be flexible and to provide management with accurate and rapid feedback. The evaluator conducted a utilization-focused formative evaluation informed by a collaborative developmental approach. This meant that data were collected through well-suited methods throughout the evaluation term (February 2015 - October 2015) and, where appropriate, fed back to management for the purpose of remedying concerns. The Waves for Change programme theory was elicited by the evaluator and depicted in a usable format before its plausibility as well as key assumptions were investigated. The evaluator found the theory to be high in utility and plausibility. Following Chen's (2005) approach to formative evaluation, and in order to determine areas of the Coach Training programme on which the evaluation should focus, the evaluator worked with management to identify components deemed crucial to its delivery. These Crucial Components were identified as: the Implementing Organisation; the Programme Protocol; the Target Population; and Programme Implementation. The evaluator investigated Areas of Focus within each Crucial Component. For the Implementing Organisation component these were: Organisational Structure; Funding and Equipment. For the Programme Protocol component these were: the Coach Pathway, and Content of the programme. For the Target Population component: Eligibility Criteria and Recruitment Strategies; and Barriers to Retention and Strategies to address them, and finally, for the Programme Implementation component the focus was placed on Monitoring of the Coach programme. By the end of the evaluation the programme had shown development in several Areas of Focus. The organisation had addressed gaps in staff roles by employing people in key positions and in doing so had developed its organisational structure and improved its competence and cultural diversity. Funding had increased which meant that service delivery had been guaranteed for the next three years. The Coach pathway had become performance-based and due to retention strategies that now include incentives for improved performance Coaches' compliance and engagement had increased. The new pathway also created exit points for Coaches who were under-performing. Monitoring of the programme had developed in terms of revisions to the monitoring documentation and overall system which had led to improvements in data quality. Areas of concern were equipment and recruitment. An occasional lack of equipment meant that Coaches struggled to deliver services to child beneficiaries. Lax eligibility criteria and recruitment strategies meant that poor quality coaches were employed. This led to underperformance, misconduct and dissatisfaction among some of the Coaches and ultimately meant that four of the Coaches (all of whom were recruited through word-of-mouth) left the programme during the evaluation term. This led to one of the Coaching teams being all-female. Future programming needs to address these two areas very strongly as both could become a threat to the successful implementation of the Coach programme as well as the surf therapy programme. Ongoing monitoring of data quality, compulsory requirements to work with children (e.g., first aid and checking against the Child Protection Register) and levels of coach satisfaction are strongly recommended.
  • Thumbnail Image
    Item
    Open Access
    A Formative Evaluation of the Dream Toolkit component of the Be the Dream Programme
    (2018) Bhebe, Brilliant; Field, Carren
    The need for positive youth development programmes is necessary in the South African context where youth struggle with many socio-economic challenges including poverty, youth unemployment, alcohol and drug abuse, teenage pregnancies, violent behaviour and high school dropout. These programmes aim to promote personal and interpersonal development outcomes for at-risk youth so that they can lead better purpose-driven lives. The following dissertation presents the findings of a formative evaluation conducted for the Dream Toolkit Component of the Be the Dream Programme, a PYD programme implemented by Dream Factory Foundation in Cape Town. Three evaluations were performed, namely programme theory evaluation, implementation evaluation, and short-term outcome evaluation. A combination of qualitative and quantitative research methods were utilised to answer the evaluation questions posed. Overall, the findings indicate that: a) the programme theory of the Dream Toolkit Programme is consistent with best practices programmes and the causal logic of the programme was deemed to be plausible; b) programme participants were highly satisfied with the programme services; c) the programme was implemented with limited fidelity; and d) majority of the learners demonstrated relatively high self-esteem and career decidedness outcome levels. While the evaluation yielded positive results, the evaluator was able to make a number of recommendations and highlight important considerations for DFF to improve the Dream Toolkit Programme. This evaluation contributes to limited research on implementation and programme theory driven evaluations in the PYD programme context.
  • Thumbnail Image
    Item
    Open Access
    A needs assessment for an intervention on supporting breastfeeding at work in a Cape Town clothing factory
    (2021) Heyns, Hilde; Chapman, Sarah
    The University of Cape Town (UCT), in collaboration with the South African Medical Research Council (SAMRC) and Middlesex University, London, has been requested by the SACTWU Worker Health Programme to design, implement and evaluate an intervention on workplace breastfeeding support for the clothing and textile industry. This study will form part of the bigger project and the results will be utilised in the planning and designing of an intervention on workplace breastfeeding support. Returning to formal employment has been identified as one of the leading barriers to exclusive and continued breastfeeding, especially among women working in low-income jobs. Women working in low-income, low skilled jobs are often easily replaceable and have little bargaining power to request their breastfeeding rights. South Africa's clothing industry employs mostly black women, who hold low-income positions. Clothing factories tend to have rigid working conditions, which could make it difficult for mothers to continue with breastfeeding once they return to work. Few studies have been conducted of breastfeeding support in the workplace in South Africa and no studies were found of breastfeeding support in clothing factories in South Africa. There is a need for more studies to be conducted in workplaces in South Africa, especially in the clothing industry. International studies have found that workplace breastfeeding support interventions can contribute to increased breastfeeding prevalence rates in factories. To increase the rates of breastfeeding in clothing factories, a workplace breastfeeding support programme would need to be developed and implemented. Before designing an intervention, it is important to first conduct a needs assessment to determine whether there is a need for an intervention on workplace support for breastfeeding in the clothing and textile industry. The purpose of the needs assessment in this study was to determine whether there was a need for an intervention on workplace support for breastfeeding in the clothing and textile industry, and if so, what the nature and extent of the need was and the programme activities that were most suitable to address the need. A convergent parallel research design was adopted, which is also known as a mixed methods approach. Quantitative data was collected through a semi-structured survey that was administered to the employees at the factory. In addition, qualitative data was collected through interviews conducted with employees (mothers) and supervisors at the factory. There were various barriers in the factory that made it difficult for women to express breast milk at work, including a lack of time to express, a lack of proper breastfeeding facilities, a lack of information about mothers' breastfeeding and a lack of support from factory management, supervisors and co-workers. There was also a lack of awareness of the law regarding breastfeeding and there was no breastfeeding policy in place. The study found that there was a need for increased support to breastfeeding mothers in the factory, including the support from factory management, supervisors and co-workers, the provision of adequate breastfeeding facilities and breaks and the provision of information about mothers' breastfeeding to all employees. It is also important to develop a breastfeeding policy in the factory which provides clear guidelines on how to support breastfeeding women. According to the findings of the needs assessment, there is a need to design and implement an intervention on workplace support for breastfeeding, to enable women working in a clothing factory to continue breastfeeding when they return to work. The last chapter will include recommendations for the SACTWU Worker Health Programme to design and implement an intervention on workplace breastfeeding support for the clothing and textile industry. Although this study focused only on a single large factory, the basic principles can still be applied and adapted to other clothing factories.
  • Thumbnail Image
    Item
    Open Access
    A programme theory and process evaluation of a youth development programme
    (2020) Swingler, Susan; Boodhoo, Adiilah
    Background The CoolPlay youth development programme (CYDP) is an after-school sport-for development (SFD) programme with an integrated social-emotional learning (SEL) component delivered to schools in low-income communities in Cape Town, South Africa. The CYDP is implemented by CoolPlay, an SFD organisation funded by the Laureus Sport for Good Foundation. The aim is to provide youth with pro-social after-school activities that equip them with the social-emotional competence fundamental to becoming responsible and socially engaged citizens. The main programme stakeholders include the CoolPlay board of trustees, an operations manager, four area managers and 26 sport and SEL coaches (Champions). In 2018, I was appointed as a monitoring and evaluation (M&E) intern within the organisation. In my capacity as an M&E intern, I conducted an evaluability assessment (EA), which determined the scope and approach of the evaluation for this dissertation. Evaluation Focus The objective of the evaluation was to determine: (a) whether or not the CYDP can realistically produce the intended outcomes, and (b) the extent to which the programme is implemented in line with quality parameters identified in the literature. As such, a theory evaluation and a process evaluation were conducted sequentially to address the following evaluation questions: 1. What is the theory and logic underlying the CYDP design? 2. Is the programme theory and logic plausible? 3. What elements of the CYDP theory could be modified to maximise intended outcomes? 4. Do the Champions implement the CoolPlay sport sessions with sufficient quality? 5. What contextual factors may be influencing the implementation of the CoolPlay sport sessions? 6. Are the Champions adequately trained to deliver the CoolPlay sport sessions? 7. Is there adequate organisational support in place to facilitate implementation of CoolPlay sport sessions? Methodology The evaluator used Donaldson's (2007) five-step procedure for constructing and assessing the programme theory. An initial programme theory was constructed using programme documentation and focus group data derived from a purposive sample of seven programme stakeholders (evaluation question 1). The plausibility of the initial programme theory was assessed by means of an extensive literature review (evaluation question 2). The programme theory was critically reconstructed using Brouselle and Champagne's (2011) logic analysis procedure (evaluation question 3). The process evaluation used a combination of quantitative and qualitative data collection, analysis, and integration methods (concurrent mixed methods design). An observational rating scale was administered to a convenience sample of 16 Champions during a routine observation session conducted by area managers to address evaluation question 4. A purposive sample of three area managers and a stratified randomised sample of four Champions participated in interviews to elicit the contextual factors influencing implementation (evaluation question 5), the adequacy of Champion training (evaluation question 6) and the adequacy of programme support functions (evaluation question 7). The operations manager also provided qualitative input on the adequacy of programme support functions through an interview. The qualitative data was analysed using Braun and Clarke (2006)'s procedure for thematic analysis. The interviews were supplemented by quantitative data derived from a mentor survey and an organisational survey completed by four Champions and five members of the programme management team, respectively. Key Findings and Conclusions The programme theory evaluation found that the initial programme theory and logic (evaluation question 1) is plausible, however effect sizes are likely to be low and programme effectiveness is highly dependent on developmentally appropriate content and structure, psychologically and physically safe programme environment, staff characteristics, parent/caregiver involvement, and youth participation and engagement (evaluation question 2 and 3). Preliminary insights derived from the EA suggest that the CYDP does not meet these quality parameters/requirements fully, thus highlighting the need for programme design and implementation improvement, as captured in the critically reconstructed programme theory (evaluation question 3). The process evaluation found deficiencies in implementation (evaluation question 4) linked to schools' commitment and capacity, parent/caregiver involvement, and youths' capacity to engage (evaluation question 5). Certain aspects of the training structure and content were found to be inadequate for developing all relevant competencies/skills, and poor Champion engagement was flagged as a critical issue (evaluation question 6). Implementation of the programme was further constrained by limited organisational capacity (evaluation question 7). These process evaluation findings confirm that the CYDP does not fully meet the programme quality parameters/requirements identified in the literature. Recommendations Key recommendations unpacked in this dissertation include the following: • Alignment of programme content with appropriate developmental milestones and unique interests of programme beneficiaries. It is recommended that the content is structured in a curriculum that follows the SAFE approach to SEL facilitation. • Provision of transport, nutrition, and first aid supplies and training, as well as implementation of strategies to address deviant behaviour to ensure that programme beneficiaries are physically and psychologically safe. • Efforts to hire, train and retain qualified Champions. These include adherence to documented selection criteria, a more comprehensive induction process, ongoing training and support including collaborative planning, debriefing and provision of mental health services. • Implementation of parental and teacher involvement strategies, such as invitation to CoolPlay meetings and workshops, distribution of newsletters, and provision of transport to and from CoolPlay events. • Implementation of M&E systems for participant enrolment, attendance, drop-out and engagement. • Allocation of resources toward securing qualified, full-time staff with experience in positive youth development. • Strengthening of programme quality before the CYDP is rolled out to additional sites, given that the programme has the potential to support both positive and negative developmental outcomes in youth.
  • Thumbnail Image
    Item
    Open Access
    A theory and process evaluation of the parent centre's teen parenting programme in Cape Town, South Africa
    (2020) Kokera, Sandra Betty; Mulenga, Chao Nkhungulu
    Globally, there has been increased momentum in the implementation of parenting programmes. Such programmes have been noted to be effective, with positive outcomes being observed in the parents and their offspring or children they take care of. Most of the evaluations on parenting programmes have been conducted in developed countries, creating a need for further research on implementation processes and impact in low income countries. The current project was a theory and process evaluation on the Teen Parenting Programme (TPP) being offered by the Parent Centre in Cape Town, South Africa. Through group sessions, the programme teaches parenting and life skills to teenagers who are either biological parents or primary caregivers to younger children. The evaluation objective was to develop the programme theory, assess its plausibility and evaluate the programme's implementation fidelity. The study used exploratory and descriptive research designs to address the evaluation questions. The theory evaluation component adopted a theory-based approach, collecting data through workshops with programme staff and review of documents. The process evaluation made use of primary and secondary sources to collect quantitative and qualitative data. A survey questionnaire, focus group discussion guide, key informant interview guide and checklist were used to collect data. Quantitative data were analysed using descriptive statistics and qualitative data was analysed using thematic analysis. The findings indicated that (i) the TPP theory and underlying causal and process assumptions were plausible; (ii) the programme was reaching the target population and there were high attendance levels; (iii) there was implementation fidelity; and (iv) sufficient organisational support was being offered despite the prevailing financial challenges. As a plausible programme theory and high implementation fidelity are key components in ensuring programme success, the TPP was well positioned to be a successful programme. Such implementation fidelity findings indicated that the TPP had a higher likelihood of achieving its expected outcomes. The evaluation made recommendations essential in improving the few noted implementation shortcomings. A key limitation to the evaluation was that the views of the participants were not captured as access to interview them was limited. The study contributes to research on theory evaluation and implementation fidelity of teen parenting programmes in low-resource settings.
  • Thumbnail Image
    Item
    Open Access
    A Theory and Process Evaluation of the Umhlathuze Water Stewardship Programme of the International Water Stewardship Programme
    (2019) Gautier, Estelle; Goodman, Suki
    Water is a scarce resource without which human survival is at risk and major economies would collapse if not carefully managed. Water stewardship involves collaboration between role players from the public- and private sector as well as civil society to effectively mitigate some of the water related risks and improve water security. Good practice examples of water stewardship have been captured in only a limited number of international studies through the work of a few organisations driving adoption of stewardship practices. Most of these are internal documents and remain unpublished. There are a few local, unpublished studies that are specific to water stewardship as an approach to water resources management in South-Africa. A comprehensive literature review on water resource management institutions in South published between 1997 and 2011, shows that research is predominantly focused on catchment management agencies (CMA) than on other entities such as international water management bodies, water user associations or water irrigation boards (Meissner, 2013) The uMhlathuze Water Stewardship Programme (UWaSP) is a South African programme selected to evaluate if the programme has been established against globally recognised good practices of water stewardship and to what extent it has been implemented accordingly. The programme is part of an international water stewardship programme which provided an ideal opportunity to evaluate a local programme against a global good practice model. The international literature on water stewardship as well as literature on adaptive co-management approaches to water resources management enabled the development of a consolidated global good practice evaluation model of water stewardship. The research included consideration of contextual influences that may have enabled or hindered the implementation of the establishment of the water stewardship programme. The South African literature shows that a specific type of integrated water resources management ( IWRM) , catchment management associations (CMA’) share certain adaptive co-management elements with a water stewardship approach to water management. These include three concepts extracted from the literature namely collaborative stakeholder engagement, adaptation through learning (experimentation) as well as a bioregional approach to water resources management which means implementation of management structures at the river basin level (Meisner, 2016). The local contextual factors influencing implementation of the UWaSP and commonalities with CMA’s are considered during the discussion, against the background of the findings of local studies.
  • No Thumbnail Available
    Item
    Open Access
    A theory evaluation of a peer education programme.
    (2012) Naude, Zani; Field, Carren
    Peer education is a popular strategy to promote positive behaviour among youths. Peer education programmes are usually influenced by either the cognitive behaviour or the structural approach. The cognitive behaviour approach views behaviour change as an individualistic decision that is influenced by an individual's knowledge of the causes and consequences of risk behaviour. The structural approach in contrast rejects the idea that behaviour is an individual decision instead arguing that decision-making takes place within the context of socio-economic dynamics and social and cultural norms. The approach that an organisation favours will influence the design of the programme. Mfesane implemented a peer education programme in the Saldanha Bay Municipality in 2008. This programme was rooted in the Cognitive Behaviour Approach and was targeted at Grade 9 to Grade 12 learners in various high schools in the Saldanha bay area. The programme aimed to convey knowledge of HIV/AIDS to the youth in order to change the youth's sexual behaviours and thus reduce the prevalence of HIV/AIDS. An outcome evaluation that was conducted at the end of the programme's 3 year span found that although the programme successfully increased the youth's knowledge of HIV/AIDS, it did not succeed in changing risk behaviour due to the fact that more than half of the youths were already sexually active by age 16. The evaluation findings motivated Mfesane to review its peer education programme. Initially the organisation considered presenting the programme to a younger cohort. However, the current legal framework in South Africa would restrict what messages Mfesane could share with a younger audience. Instead the organisation agreed to continue targeting high school learners, but to reformulate the programme in order to improve the impact of the programme. The evaluator was approached to conduct a theory evaluation that would assist Mfesane to formulate a more effective programme. A theory evaluation extracts programme assumptions from stakeholders in order to assess the plausibility thereof based on social science literature and previous evaluation findings. During the planning phase of a new programme a theory evaluation can be used to ensure that stakeholders have a common view of the programme and the programme can plausibly achieve its desired outcomes. This theory evaluation was guided by three specific questions: 1) what are the underlying assumptions of the programme; 2) are the assumptions based on clear theoretical linkages; and 3) does evidence exist to support the programme assumptions? These questions guided the systematic method that was used to gather the information that informed the recommended programme theory. The programme assumptions were elicited from programme staff through individual interviews and two focus groups' discussions; one with the programme implementers and another with the programme managers. The separate focus group discussions enabled the evaluator to adopt strategies suited to the group's level of experience and previous exposure to programme theory. The programme implementers were systematically introduced to the different concepts used in programme theory before constructing a common programme theory while the programme managers could directly formulate a programme theory. In structuring their programme theories, both groups moved away from the cognitive behaviour approach and adopted a more holistic approach towards behaviour change that resembles the structural approach. An interesting finding was that in both instances the groups also moved away from focussing on risk prevention rather proposing to strengthen youth to become successful adults. However, the two groups proposed significantly different programme theories and in neither instance did these theories have strong causal links. After reviewing both programme theories, the evaluator presented the two groups with the programme outcomes that had been identified uniquely by each group and those that overlapped. This was used by Mfesane as a guide to identify the outcomes it would like to achieve through the new programme. Although the proposed programme theories supplemented the peer education programme with additional interventions such as income generation, in the end, the organisation for practical reasons decided to continue focusing primarily on peer education as an intervention. The evaluator assessed the plausibility of the proposed new peer education programme by interrogating social science literature and previous evaluation findings. A proposed causal chain for behaviour change was identified by integrating two behaviour change theories, the Social Cognitive Theory and the Theory of Planned Behaviour, into a single model. This model was used to measure how effectively the proposed activities could facilitate behaviour change. Based on these findings the evaluator could recommend a plausible programme theory in line with Mfesane's vision for the peer education programme. The differences between the two programme theories illustrated the value of a programme theory evaluation during the planning phases to ensure that stakeholders start the programme off with a common understanding thereof.
  • Thumbnail Image
    Item
    Open Access
    An evaluation of the Qhubeka! Youth Development programme: Using a generic qualitative approach
    (2022) Majola, Sibahle; Chapman, Sarah
    The Qhubeka! Youth Development Programme is the first programme launched by the NonProfit Company (NPC), Just Grace, in 2012. The organisation is implementing five community programmes: (1) School strengthening; (2) Community development; (3) Business support; Community projects; and (5) Youth development programme in the Langa township, Cape Town, South Africa. The Qhubeka! Youth Development Programme is not a typical Positive Youth Development programme (PYD). It offers academic support that is not typical of PYD's. They tend to focus on cultivating proficiencies and other life skills and therefore can be fitted in with After-School Programmes (ASP), which focuses on cultivating learning outcomes. The overall goal of the youth development programme is to enable under-resourced youth to escape the cycle of poverty and equip them to become meaningfully involved in society through further education and employment. The programme aims to develop technical and soft skills in high school learners by focusing on study assistance and psychosocial support. The Qhubeka! Programme provides comprehensive material and guidance over four years. Activities of the programme include (1) academic support; (2) computer training; (3) coding and programming; (4) book club; (5) life skills; (6) career guidance; (7) holiday programme; and (8) psychosocial support. The objective of the evaluation was to assess the performance of the programme in reaching its short-term outcomes and the quality of service delivery. I used primary and secondary data to conduct the evaluation. I collected primary data through semi structured one-on-one interviews with the Qhubeka! Programme learners and programme facilitators. Secondary data, I obtained through the review of the programme documents. I used a thematic inductive content analysis to analyse data. The findings on the evaluation of short term outcomes indicate that the programme achieved its desired short-term outcomes and that the programme was implemented with quality. This was the first evaluation of the Qhubeka! programme and provides programme staff with feedback on how well the programme is reaching its intended outcomes and being implemented with quality, which could guide them in making counteractive changes to improve delivery and impact. This study further contributes to the body of knowledge and evaluations of PYD programmes that incorporate academic support in the South African context of PYD programme.
  • Thumbnail Image
    Item
    Open Access
    An Impact Evaluation of the Chrysalis Academy Programme focusing on the contribution of its Outdoor Component
    (2019) Mkandawire, Sandra; Chapman, Sarah
    This evaluation is an impact evaluation of the Chrysalis Academy (CA) Programme. The CA programme is geared to help youth in the Western Cape develop the skills and resilience needed to transcend the prevailing poverty, inequality, unemployment and crime in their communities. Chrysalis offers a 3-month residential and 5 year After-Care programme for NEET youth aged 18-25 that includes life and vocational skills training as well as counselling and community service. The programme also includes a 2-week outdoor/wilderness component that is deemed crucial to the success of the programme. The evaluation sought to respond to two overarching groups of questions. The first group of questions assessed the long-term impact of the programme with regards to education, employment, crime, drugs as well as family and community relationships. Considering that the outdoor component is regarded as the most crucial component of the programme, the second group of evaluation questions was geared to understand the causal mechanism of the CA outdoor component to assess whether it is consistent with the logic model of successful outdoor programmes and to understand the outcomes that can be expected from this phase. A quasi-experimental impact evaluation design was used to respond to the impact evaluation questions. A database of 14,614 past applicants from the Western Cape formed the basis of a sampling frame from which a random sample of 300 past applicants who received the programme and those not selected into the programme between 2014 and 2016 was drawn. Face-to-face interviews were scheduled with 35 contacted programme recipients and 35 contacted non-selected applicants and interviews were conducted using a structured questionnaire. A final sample of 32 programme applicants and 33 non-selected applicants was analyzed. Differences in the average outcome attainment between past participants versus non-participants were then assessed after Propensity Score Weighting was used to balance the treatment and control group on key variables related to the probability of being selected into the programme. To assess the causal mechanism of the outdoor component, in-depth, semi-structured interviews were conducted with some success and non-success case graduates of the CA programme to assess their experience of the outdoor component and the outcomes that presented as a result. The findings of the evaluation show that the CA programme does not have long-term impact as those who did not receive the programme also attained similar positive outcomes. However, there is a possibility of short-term outcomes attainment from the outdoor component as its causal mechanism is similar to that of successful outdoor programmes. Moreover, participants, regardless of their long-term outcomes, are able to achieve the short-term outcomes expected from the outdoor component. Based on the results, CA should assess how the effect of the outdoor component and possibly other phases of the programme, can be sustained for long periods, possibly by intensifying the After-Care programme. CA should also assess the assumptions around outcomes attainment such as labor market favorability and financial sufficiency and assess how these can be addressed within the scope of the programme.
  • Thumbnail Image
    Item
    Open Access
    An implementation evaluation of light for the world´s inclusive education programme in Ethiopia
    (2021) Muellegger, Birgit; Chapman, Sarah
    Inclusive education programmes aim to include children with disabilities in mainstream schools by providing quality education in a barrier-free environment. These programmes are especially necessary in developing countries, where about one-third of all children have some sort of disability. Sustainable Development Goal 4 of education for all cannot be reached when these children are left out from schooling. The following dissertation presents the findings of an implementation evaluation of an inclusive education programme in Ethiopia. The programme, One Class for All (OCFA), is financed by Light for the World, an international non-government organization (NGO), and implemented by 14 Ethiopian partner organisations. A descriptive evaluation design combining qualitative and quantitative elements was used to examine typical Light for the World programmes as delivered by partners and as they appear on the ground. The data collection mainly involved interviews of partners and participants at selected schools in Ethiopia and a survey of all implementing partners. The evaluation indicated that the composition of implementing partners and their capacity in terms of content knowledge and collaborations are working well. We nevertheless highlight some high-level opportunities to improve the inclusive education programme of Light for the World. More structured development plans for each school are required, and a better connection between Light for the World`s community-based-rehabilitation (CBR) programmes and its inclusive education programmes is necessary to support entry to school for persons with disabilities. Under these overall findings, a more structured teacher training programme, better selection of teachers, and better local networking and targeted awareness raising programmes are also needed. This evaluation contributes to the limited literature on successful inclusive education programmes in developing countries.
  • Thumbnail Image
    Item
    Open Access
    An outcomes evaluation of the Zim-TTECH cervical cancer intervention in Harare Metropolitan Province, Zimbabwe
    (2023) Tapera, Oscar; Duffy, Carren
    This report presents the findings of an outcomes evaluation of the Zimbabwe Technical Assistance, Training and Education for Health's (Zim-TTECH) cervical cancer intervention in Harare and Epworth districts (Harare Metropolitan province). This programme seeks to improve access and uptake of cervical cancer screening and treatment among women living with HIV through recruitment, training and mentorship of nurses, procurement of equipment and commodities, providing support for diagnosis and referral for treatment services, and quality assessment of cervical cancer services. The evaluation interrogated the plausibility of the programme theory of the Zim-TTECH intervention and assessed its short- and medium outcomes. Methods The evaluation used a mixed-methods approach with both qualitative and quantitative data collection methods. This comprised a desk review of programme documents and relevant literature, secondary analysis of programme routine data and collection of primary data. Quantative descriptive analysis was conducted to determine coverage (in percentage) of Zim-TTECH cervical cancer services between 2019- 2021. Qualitative analysis was used to complement quantitative data and aid in the interpretation of programme results. Key Results This assessment showed that the Zim-TTECH cervical cancer intervention contributed to the increase in awareness among women living with HIV and health workers. The programme capacitated health workers to conduct effective health education and awareness-raising campaigns for cervical cancer. The evaluation revealed that the Zim-TTECH cervical cancer intervention improved access to cervical cancer services among women living with HIV. Additionally, the intervention did not deny women who were HIV-negative services though the reach for this group is not routinely documented and reported. Results from the evaluation showed that LEEP was the main treatment type used in Harare and Epworth followed by cryotherapy and thermablation techniques. This assessment also showed that the Zim-TTECH cervical cancer intervention contributed positively to the following areas: increasing access to cervical cancer services, early detection of pre-cancers and invasive disease and increased health education and awareness of cervical cancer among women and health workers. The evaluation also revealed some negative implications of the Zim-TTECH intervention on the health system which include the following: the programme‘s screening resulted in the detection of invasive cervical cancer cases which were referred to weak and ill-capacitated public health facilities, and most screening services are offered in HIV clinics which results in some women who are not HIV positive not wanting to screen for fear of stigma and the programme also contributed to staff attrition in public health facilities as they search for better opportunities in the NGO sector and outside the country. The Zim-TTECH cervical cancer programme rapidly adjusted to the new COVID-19 protocols in 2020 to ensure continuity of services though the demand side was affected by protracted lockdowns. The programme has also been keeping in tandem with technological evolutions including the adoption of thermoablative techniques and planning for the rolling out of highly sensitive human papilloma virus (HPV) DNA testing in line with Ministry of Health strategies and WHO guidelines. However, discussions are still ongoing on the adoption of HPV screening as the primary method as it will negate the ‘see and treat' approach. This evaluation showed that cervical cancer services were likely to continue in Harare and Epworth after the cessation of Zim-TTECH support albeit at a lower scale. The Zim-TTECH intervention is based in public health facilities that have been offering cervical cancer services even before partner support came through and these are likely to continue in their absence. Conclusions Findings from this evaluation showed that Zim-TTECH cervical cancer intervention was relatively effective with regards to short-medium term outcomes, although the COVID-19 pandemic saw disruptions of cervical cancer services which affected service delivery/uptake in 2020-2021. However, the key shortfalls of the intervention revealed in this evaluation showed systemic/structural gaps with the design of health interventions in the country as well as the vertical approaches which are also being perpetuated by donors. Additionally, given the huge resource shortfalls in the MoHCC due to protracted underfunding, there is evidence of limited coordination/leadership capacities to guide partners who come up with piecemeal interventions. Despite the revealed shortcomings of the intervention there are opportunities to improve services including treatment of invasive cervical cancer. Overall, better partnerships and coordination among government, NGOs and donors will be a game changer in ensuring continuity of services for cervical cancer in low-resource settings.
  • Thumbnail Image
    Item
    Open Access
    Animal welfare in South Africa : a multi-component programme design
    (2015) Adams, Leanne; Field, Carren
    The ongoing international atrocity of animal abuse is a persistent and widespread social problem within human society. The maltreatment of animals, particularly in rural impoverished contexts in South Africa continues to be a pertinent challenge facing local animal welfare organisations. Upon identifying the need to improve the conditions of animals living in the Darling community, the Swartland SPCA approached the evaluator in order to design a plausible humane education programme. The evaluator began this research by reviewing the history and underlying principles of humane education, the efficacy of humane education, as well as formalised humane education programmes. Findings from the review revealed that humane education programmes hold diminished capability for the achievement of improved animal welfare when implemented in isolation. Although these programmes aim to affect behavioural change in order to respond to the problem of animal welfare, behavioural change is not easily achieved with a simple intervention such as humane education. Thus, designing a programme for the Swartland SPCA based solely on the logic of humane education with the expectation of behavioural change was deemed problematic. Subsequently, the evaluator endeavoured to develop a novel programme design to target the problem of animal welfare in Darling. Increasingly, research points to the need for a multifaceted approach in dealing with the problem of animal maltreatment due to its complex nature as well as the fact that it is based on a large number of confounding personal, familial, social, cultural and psychological factors. To address this need, the evaluator, in consultation with the SPCA, identified a number of possible reasons as to why the problem of animal welfare was a prevalent issue within the Darling community. This highlighted the need for additional interventions that could be employed holistically to alleviate the problem of animal abuse. Given that this research concerns the welfare of animals in terms of human-directed behaviours, it was decided that humane education as a form of intervention should be investigated further in its ability to affect attitudinal change in conjunction with violence prevention and peer education / modelling interventions as additional programme components. In order to affect interpersonal violence at a behavioural level, a violence prevention component was selected with the hope that reductions in interpersonal violence may extent toward the reduction of violence and abusive behaviour towards animals. Finally, peer education / role modelling was selected as a means of involving community members in the promotion of animal welfare via the concepts developed from the humane education programme component. Since an improvement of animal welfare conditions is essentially dependant on the performance of individual behaviours, grounding the programme in a theory of behaviour change was identified as an important first step in the programme design. As such, Ajzen's theory of planned behaviour was utilised as a behaviour change model in order to guide the programme's design framework and to ensure that the programme components corresponded with theoretically identified mechanisms of behavioural change. It was anticipated that utilising a multidimensional programme design rooted in a theory of behaviour change would increase the likelihood of achieving the link between attitudinal change and behavioural change. Therefore, this evaluation provides the Swartland SPCA with a broad-based programme design which provides a variable-oriented insight into the proposed mechanisms of change and can be utilised as a framework in the adaption of programme components. It is envisioned that this tool be used by the programme stakeholders to decide on intended outcomes, select the programme activities, and further define the design details thereof. A number of recommendations are also made in order to assist with the successful implementation of this programme and its sub-components in the long term. These include lobbying for the inclusion of animal welfare and violence prevention content as part of national school curricula, obtaining a violence prevention partner organisation to assist with certain implementation considerations, and finally the importance of continuous community engagement and involvement prior to and during the programme’s roll out.
  • Thumbnail Image
    Item
    Open Access
    Assessing the usability of the 2006 process evaluation of a grant-making organisation's enhancement of capacity programme
    (2011) Bhikoo, Asgar; Louw-Potgieter, Joha
    Evaluations can only be effective if evaluators provide clients with improvements that are easy to understand, implement and fit their needs. Often evaluation recommendations are not used because these factors are overlooked once the evaluation has been completed. This evaluation assessed whether the client used the evaluation improvements recommended to them by evaluators. It presents a follow-up evaluation to a Process Evaluation conducted in 2006 on a Grant-making Organisation's Capacity Building Programme. It focuses on the Enhancement of Capacity Programme which is a core training programme within the Capacity Building Programme of the Grant-making Organisation. Furthermore this evaluation provides a programme description, a programme theory and a plausibility assessment of the programme theory.
  • Thumbnail Image
    Item
    Open Access
    Breaking cycles of violence, one wave at a time : a formative evaluation of the Waves for Change Surf Therapy programme
    (2016) Snelling, Matthew; Dawes, Andrew
    This dissertation was a formative evaluation of the Waves for Change Surf Therapy Programme, and included both a process evaluation and an outcome evaluation. Waves for Change used surfing as a means of engaging children and adolescents thought to be at risk of long-term social exclusion. This engagement was necessary in order to deliver a psychosocial curriculum. Waves for Change aimed to use this curriculum to enhance psychosocial wellbeing and reduce antisocial behaviour, and association with antisocial peers. Five evaluation questions were generated using programme documents and a rapid evidence assessment. These were concerned with whether the programme was capable of enhancing psychosocial wellbeing, and reducing antisocial behaviour and association with antisocial peers. Further, the evaluation wished to determine whether the programme was correctly targeted, and delivered with fidelity. An intention to treat analysis was conducted within a randomised control trial, using 115 primary school students from Masiphumelele, Khayelitsha, and Lavender Hill. Further 88 interviews were conducted with programme beneficiaries, and 15 coaches were submitted to performance review. The programme was found to be suitably targeted, but delivery of the programme was not achieved with fidelity to the programme design. There were a number of reasons for this, including inadequate completion of programme tasks by coaches, and inadequate attendance by children and adolescents. The result was that children and adolescents received less than half of the psychosocial curriculum, and did not show a change on the outcomes of interest. However, this evaluation suggested that the programme is feasible, pending improvements.
  • Thumbnail Image
    Item
    Open Access
    D-Tree : examining the efficiency of a community case management mobile medical diagnostic tool
    (2016) Schramm, Kai T; Goodman, Suki
    D-Tree International's mission is "to develop and support electronic clinical protocols that enable health workers worldwide to deliver high quality care". They envision a world in which every person has access to high quality healthcare. To achieve this overarching goal D-Tree firstly, develops and validates clinical algorithms for use by health workers, secondly they design software for delivering these algorithms on mobile phones, and thirdly enable the effective use of these algorithms on a scalable basis. D-tree developed the electronic Community Case Management (eCCM) decision support tool for use by Health Surveillance Assistant's (HSAs) in Malawi for the care and treatment of acutely ill children under the age of five. This tool guides the HSAs through a clinical protocol to the correct diagnosis and subsequently treat, as such providing the HSA with a tool that can supplement lack of training, supervision and/or experience. The intervention they provide is supposed to lead to the following outcomes: (1) improved supervision of HSAs in the field, (2) accessible health records for HSAs, their supervisors and the Ministry of Health, (3) better follow up for the patients by HSAs due to accessible health records, (4) improved drug availability, and (5) improved protocol adherence by HSAs. The goals that D-Tree hopes to achieve by creating these outcomes are consistent provision of quality health care, and solidifying the trust between the people and their health care system. The final impact to be achieved are improved health outcomes. This is a formative evaluation aimed at the proximal outcomes of the D-Tree eCCM application.
  • Thumbnail Image
    Item
    Open Access
    Developing a monitoring and evaluation framework for the mentoring component of the Principals Academy Trust
    (2022) Kölzer, Joshua; Goodman, Suki
    The Principals Academy Trust (PAT) is a non-profit organisation, whose programme is focused on improving the leadership competencies of school principals in poor and marginalised communities in South Africa largely through mentoring. While PAT collects quantitative data to monitor the performance of the schools in their programme, currently no data is collected to measure the extent to which PAT's mentoring efforts are positively impacting the schools' culture or climate. This study sought to develop an M&E framework for the mentoring component of the PAT programme. The M&E framework is informed by an extensive programme theory evaluation. The framework is designed to enable PAT to monitor and evaluate the potential changes in school climate and culture as a result of the mentoring component of the PAT programme. For the purposes of this study, Donaldson's step-by-step model for conducting a programme theory evaluation was merged with Markiewicz & Patrick's step-by-step guide on how to develop an M&E framework. The aim of this approach was to ensure that the results of the theory evaluation provide the foundation for the M&E framework. A combination of desk research, focus groups with PAT's programme staff and a number of semi-structured interviews with the head mentor and the fundraising consultant for PAT, was used to create the M&E framework. The results of the theory evaluation indicate that according to recent social science research in the field of education, the causality assumed in PAT's programme theory is plausible. The theory evaluation concludes that it is plausible to assume positive impacts on school culture and climate through systematic mentoring of school principals. The results of the development of the M&E framework are presented as a complete monitoring plan, evaluation plan, data collection and management plan, data analysis and synthesis strategy, learning strategy and implementation plan.
  • Thumbnail Image
    Item
    Open Access
    Developing a monitoring and evaluation system for the ceasefire gang violence programme in Hanover Park, Cape Town
    (2014) Mahamed, Mahamed Rage; Louw, Johann
    This study is a formative evaluation of the Ceasefire gang violence programme in Hanover Park, Cape Town, South Africa. The primary audience of this evaluation is the Ceasefire programme management. The Ceasefire programme is a project of the City of Cape Town's Violence Prevention through Urban Upgrading Unit (VPUU). The Ceasefire programme is run by the First Community Resource Centre (FCRC) in Hanover Park. The main aim of this evaluation is to develop a results-based monitoring and evaluation system for the Ceasefire programme.
  • Thumbnail Image
    Item
    Open Access
    Evaluating a governmental training and development programme
    (2010) Coetzer, Anje; Louw, Johann
    The current study examines the Contractor Development Model (CDM) as used by Working for Water. The overarching goal of the WfW programme is to alleviate poverty by creating short to medium term jobs for unskilled workers through clearing alien vegetation. WfW aims to appoint contractors based on targets from the Expanded Public Works Programme which are 60% women, 20% youth (18 - 36 years) and 2% disabled people. The CDM’s main objectives can be seen as: the employment of youth, women and people with disabilities; skills development through training. The programme aims to to develop contractors so that they become less dependent on WfW. It aims to develop contractors’ alien clearing and business skills to such an extent that contractors eventually exit the CDM in order to pursue more lucrative opportunities outside of WfW.
  • Thumbnail Image
    Item
    Open Access
    Evaluating Mending Mamre: An Animal Welfare Intervention
    (2019) Rabier, Camille; Duffy, Carren
    The need to improve animal welfare and control companion animal populations is necessary for South Africa, especially in impoverished communities lacking resources and education to care for their pets responsibly. These programmes are often multi-dimensional in nature and aim to improve animal welfare in different ways. Sterilisation is often the first step but is not enough. Consequently, education is often used in combination, to teach responsible pet care and ownership to pet owners. By increasing their understanding and knowledge of animals it is hoped that the pets’ overall quality of life and welfare improve. The following dissertation presents the findings of an evaluation conducted for the Mending Mamre Mass Education and Sterilisation Programme. This programme had four components: surgical sterilisation of pets and feral cats, basic veterinary care, education sessions and the rehoming of stray dogs. Three evaluations were performed (as requested by the clients): a programme theory evaluation of the education sessions, a process evaluation to understand why some residents refused sterilisation and an outcome evaluation to measure if the pets’ living conditions and body scores had changed 16 months after the programme. Overall, the results of the programme theory evaluation demonstrated that: the activities and outcomes of the education sessions were consistent with similar programmes but the two causal pathways underlying the programme are not plausible. The results of the process evaluation highlighted that the most common reason why pet owners refused sterilisation was due to fear. Finally, the results of the outcome evaluation indicated mixed results; with an increase observed in the pets’ physical wellbeing but an overall decrease observed in their quality of living conditions. With the results, the evaluator was able to make recommendations to the client and highlight considerations for programme improvement. Overall this study contributes to the paucity of research on evaluations of animal welfare interventions at the community level.