Jobsinghana.com
 

TERMS OF REFERENCE: BASELINE STUDY FOR SAGES SYSTEMS STRENGTHENING ACTIVITY

JOB SUMMARY

Company CARE Internat...
Industry NGO/IGO/INGO
Category Education
Location North East & ...
Job Status Contractor/Co...
Salary GH
Education Advanced Degr...
Experience N/A
Job Expires Sep 30, 2023
Contact ...
 

Company Profile

CARE International is a global network of independent national organizations, which include CARE members, candidates and affiliates, social enterprises, and thousands of community-based partner organizations working together across more than 100 countries to save lives, end poverty and fight social injustice. To learn more, visit www.care-international.org.

CARE International in Ghana: CARE began operations in Ghana in 1994. Since then the Accra office expanded to support programs in Togo and Benin under the umbrella of the CARE Gulf of Guinea country mission. In July 2010, CARE replaced the three-country mission with country offices in Ghana and Benin, the latter of which is also responsible for a small number of activities in Togo.

CARE Ghana’s projects are primarily implemented through partnerships with local and civil society organizations. This enables CARE to further engage with government and the private sector, work effectively in coalition, and to have the greatest possible impact. CARE Ghana prioritizes the rural and vulnerable poor- women and organizes its work around health, governance, sustainable livelihoods and education.

Job Description

 
USAID/GHANA STRENGTHENING ACCOUNTABILITY IN GHANA’S EDUCATION SYSTEM (SAGES): SYSTEMS STRENGTHENING ACTIVITY 
TERMS OF REFERENCE
BASELINE STUDY FOR SAGES SYSTEMS STRENGTHENING ACTIVITY
 
1. Project Background and Context
Ghana has made considerable progress in improving access to primary education. Between 2010 and 2017 primary net enrollment rate has increased from 77.8 percent to 91.5 percent. Despite this remarkable achievement, public perception about the quality of education service delivery has declined over the years. According to the Afrobarometer survey round 8, between 2017 and 2019, public perception about the quality of education service delivery has declined by 8 percentage points. As part of its effort to strengthen the quality of education service delivery, the Ministry of Education has prioritized strengthening accountability as one of the policy goals of the Education Strategic Plan (ESP) 2018-2030. Further to achieving this policy goal, the MOE has developed Education Accountability Framework (EAF) to institutionalize accountability for learning across Ghana’s pre-tertiary education system. As part of efforts to operationalize the EAF, USAID/Ghana has selected Cooperative for Assistance and Relief Everywhere (CARE) and its consortium partners (AfriKids, Community Development Alliance (CDA), Ghana National Education Campaign Coalition (GNECC), and School for Life (SfL)) to implement a 5-year Strengthening Accountability in Ghana’s Education System (SAGES) Systems Strengthening Activity.
 
The Activity supports efforts by the Government of Ghana (GOG) through the Ministry of Education (MOE) and its agencies/affiliates to: 1) Strengthen the enabling environment for education accountability; 2) increase the performance of education system actors; and 3) improve the effectiveness of their interactions with each other in support of education accountability. The Activity supports the operationalization of the MOE’s Education Accountability Framework (EAF) by improving the enabling environment for accountability and holding system actors accountable. USAID/SAGES Systems Strengthening Activity will contribute to the achievement of key priority objectives for basic education in Ghana’s MOE Sector Plan 2018-2030, namely: increased enrolment; gender parity; enhanced instructional practices; improved effectiveness of community-based structures’ supervisory functions; strengthened financial management, coordination, and accountability; and improved planning, monitoring, evaluation, and reporting systems. In doing so, the Activity will contribute to the achievement of key development objectives of USAID/Ghana’s Country Development Cooperation Strategy, specifically its Development Objective (DO) 2, Quality Services Delivered with Increased Accountability, and DO3, Sustainable Development Accelerated in Northern Ghana. 
 
The Activity will target an estimated 1,254 basic schools within the 17 districts that constitute USAID/Ghana’s Zone of Influence (ZOI), situated within the Upper West, Upper East, Northern, and North-East Regions. Overall, the Activity will directly benefit an estimated 668,932 people, comprising 508,269 students, 10,671 headteachers and teachers, 149,183 community members (parents) and SMC members), 380 district officials, 165 regional officials, and 264 national officials. SAGES is being 

[1] NORTHERN REGION (NR): Gushegu Municipal, Karaga District, Mion District, Nanton District, Sagnarigu District and Yendi Municipal

NORTH EAST REGION (NER): Mamprugu Mugduri District, East Mamprusi Municipal

UPPER EAST REGION (UER): Bawku Municipal, Bawku West District, Garu District, and Timpani District

UPPER WEST REGION (UWR): Sissala West District, Sissala East Municipal, Wa East District, Daffiama Bussie Issa District, and Nadowli Kaleo District

implemented by a consortium formed by CARE, Afrikids, Community Development Alliance (CDA), Crown Agents, the Ghana National Education Campaign Coalition (GNECC), and School for Life (SfL), operating in close collaboration with the MOE and Ghana Education Service (GES). 
Through the present Terms of Reference (ToR), CARE seeks to identify a consultancy group/firm to conduct an independent baseline study for USAID/Ghana Strengthening Accountability in Ghana’s Education System (SAGES) Systems Strengthening Activity. The following sections outline the key objectives, research questions, expected methodology, deliverables, and timeline for this study. The ToR includes a summary of the required qualifications for applicants and an outline of the application process. 
 
2. Objectives of the Baseline Study
The key objectives of the baseline study are as follows: 
  • Determine baseline values for key indicators, enabling the Activity to assess progress on routine basis and at the midterm and endline point, and informing the adjustment of annual and life-of-project targets.
  • Assess the current status of implementation of accountability processes in Ghana’s education system as a whole and more specifically, in the schools targeted by the Activity; the main achievements to date in improving accountability processes; and the factors contributing to and hindering improvement in accountability processes in education.
  • Assess the validity of Activity’s Theory of Change and existing assumptions. 
  • Generate evidence to inform adaptations to project design and modalities of implementation, ultimately contributing to maximize impact over time. 
  • Identify contextual factors and trends which may affect implementation and/or results of the Activity and provide recommendations on the responses to those. 
  • Contribute to the Activity’s learning agenda as well as national, regional, and global learning agendas on accountability in education, including the MOE’s, USAID Ghana’s, the Global Partnership for Education’s regional Knowledge and Learning Exchange (GPE-KIX), and USAID’s global learning on system strengthening. 
3. Key Evaluation Questions
The baseline study will generate information to enable the Activity to respond to the following evaluation questions: 
  • What school-level and system-level factors and practices have positively influenced the acquisition of literacy and numeracy skills? 
  • What is the current status of implementation of accountability systems, and what factors are affecting this?
  • To what extent are gender equality and social inclusion gaps affecting student and school-level measures?
  • How is public awareness contributing to accountability for learning and education resource allocation?
  • What are the key behaviors and intrinsic motivation amongst district, regional and national education staff in accountability for learning? 
  • To what extent do system actors use data for decision-making? How does this influence accountability for learning?
  • To what extent do teachers in primary schools in the ZOI use assessment data to inform instructional delivery? 
  • To what extent are education actors at the national, regional and district level making data-driven decisions?
  • To what extent is the B2 National Standardized Test internationally valid in measuring learning outcomes for B2 Ghanaian students?
4. Evaluation Methodology
The USAID/Ghana SAGES Systems Strengthening Activity’s evaluation will use a mixed-methods, pre-post design where project achievements will be measured vis a vis benchmarks set up at the baseline. It is currently anticipated that the project will target all primary schools in USAID’s ZOI, thus precluding the use of a counterfactual (control or comparison group). Furthermore, the project’s approach to improving accountability within the MOE assumes that activities will contribute to impact throughout the education system. The evaluation will assess the project’s contribution through (i) achievement vis a vis benchmarks set at baseline and (ii) determining if the uptake of specific accountability approaches predicts significant change in education outcomes and to what extent this is occurring.
 
The evaluation will: 
a. Measure the learning outcomes of B2 students: The evaluation will assess differences in B2 student learning outcomes using a cross-sectional approach. The project will use the Early Grade Reading Assessment (EGRA) and Early Grade Math Assessment (EGMA) to assess literacy and numeracy outcomes, respectively. If possible, the baseline will conduct the EGRA and EGMA at the same time as the government’s administration of the B2 National Standard Test (NST) to enable the project to conduct an equating study comparing the MOE-administered B2 NST results with EGRA and EGMA results obtained through the baseline. Prior to conducting the baseline, the Activity will request the student names and IDs for the B2 NST that will be conducted at the same time, to enable student-level comparisons.
b. Assess the behavior and perceptions of school community members through surveys: The baseline will conduct surveys with representative samples of community members, School Management Committee (SMC) members, parents of assessed students, teachers, headteachers, district education directorate staff, regional education directorate staff, and MOE officials at the national level. The surveys will assess: i) the application of improved classroom practices using the Inspection Evaluation Framework (IEF); ii) student and teacher attendance (through headcounts vis a vis school records); iii) school environment conditions; iv) School Improvement Support Officer (SISO) support and behavior; iv) SMC and community engagement in holding schools accountable; v) existing school practices in tracking absenteeism and dropout and encouraging out-of-school children to return to class; vi) school practices regarding gender equity and social inclusion (GESI); vi) implementation of Performance and Accountability Support for Schools (PASS) and remedial classes; vii) parental engagement in their child’s education and in accountability processes in their child’s school; viii) district and regional level knowledge, behavior, support, and use of data; ix) national level system actors’ understanding of their roles and responsibilities and the functionality of systems; and x) capacity of civil society organizations (CSOs) to hold system actors accountable. 
c. Conduct an organizational capacity assessment: Given the Activity’s focus on systems strengthening and the fact that the Activity is intended to operationalize the EAF, an organizational capacity assessment (OCA) is particularly pertinent as part of the internal evaluation. An OCA allows for a deeper understanding of the processes, knowledge, behaviors, and capabilities of the organization, and can be aligned with the key functions outlined in the EAF. For the purposes of the baseline, an “organization” will be defined as a District Education Directorate; Regional Education Directorate; the three independent education bodies – National Council for 

Curriculum and Assessment (NaCCA), National Teaching Council (NTC), and National School Inspections Authority (NaSIA); the MOE as the civil service arm of the education system; and the GES at the national level. As part of the baseline study, representatives from each of those organizations will participate in a self-administered OCA. The external evaluator and the project will jointly agree on the areas of inquiry in the OCA, which may include awareness of the EAF; knowledge of their functions documented within the EAF; capability in executing the EAF; GESI in EAF implementation, etc. These domains will also be leveled; for example, awareness of the EAF may constitute the lowest level, followed by knowledge and then capabilities. The self-rating will then be summed into a weighted index for the individual, which will then be aggregated at the organizational level. This will allow measurement of an organization’s performance over time and can be used to measure systems strengthening.

The quantitative tools outlined above will be complemented with qualitative tools, including Focus Group Discussions (FGDs) and Key Informant Interviews (KIIs). The external evaluator may propose alternative approaches to those.  The project plans to use the Most Significant Change (MSC) approach at the midterm and final evaluation, alongside rolling profiles, to assess in-depth changes in attitudes and practices at system and school levels. 

In addition to the above-mentioned quantitative and qualitative approaches, the evaluator will also conduct a comprehensive literature review to assess how contextual and policy-related trends may affect the expected outcomes. The literature review should include but not be limited to most recent system analyses,  education statistics, policy analyses, studies on social and economic trends affecting education outcomes, and studies on cultural aspects affecting accountability and education results.

The external evaluator will be responsible for the design of the data collection tools. To design the EGRA and EGMA tools, the external evaluator will conduct a desk review of existing instruments used in Ghana; determine the minimum set of tasks to be included in each tool, considering potential floor and ceiling effects based on prior performance and the grades assessed; and finalize the design of the literacy tasks through a one-day consultative workshop conducted with MOE and consortium partners. It is expected that the content of the literacy tasks (i.e., letters, non-words, familiar words, vocabulary and syntax for passage reading) will need to be calibrated considering the assessed sample’s linguistic backgrounds and reading levels. This will be done through the consultative workshop. To avoid potential pre-exposure bias, the evaluator is required not to adopt existing tools verbatim; additionally, the evaluator is required to develop a new story for passage reading, not copying textbook content or common folk tales. 

The evaluator will use electronic tools for quantitative data collection. The MEAL Specialist and CARE’s Director of Research / Education will review the electronic tools and provide feedback as necessary. Prior to deployment, the evaluator will pilot-test school-level tools in non-sampled schools with comparable characteristics, adjusting the content as necessary to prevent floor or ceiling effects.

The evaluator will be required to record, transcribe, and translate all qualitative data. The evaluator is expected to use electronic voice recording during qualitative interviews. 

The evaluator will be responsible for the provision of equipment (tablets, phones, voice recorders ) for data collection. The data collection team will not be allowed to use personal equipment to collect data under any circumstances, as this may result in breaches of confidentiality and data protection. 

[1] Education Sector Analysis; Joint Review of the Education Sector; system strengthening activity reports. 
 
5. Sampling
The baseline will assess a representative, randomly drawn sample of schools selected through multi-level stratification. First, out of the 17 districts that comprise the ZOI, the evaluator will select two districts per region, for a total of eight districts.  Within each of these eight districts, 20 SAGES-SSA targeted schools and their communities will be randomly selected, for a total of 160 schools and communities. Within each school, 13 B2 students will be randomly selected, for a total sample size of 2,080 students, which is powered to draw conclusions for both sex disaggregation as well as regional disaggregation. In addition, the headteacher, two teachers and SMC members (as a group) will be selected per school. 
 
In each school, the evaluator will conduct two classroom observations using the IEF, for a total of 320 classroom observations; this sample is appropriately powered considering a 35% attrition / data loss and 30 percentage point increase in classroom practices. The evaluator will randomly select two teachers among those teaching B1, B2, and B3 classes, observing reading (language) and math classes only. In each school, the evaluation team will conduct headcounts to determine student and teacher attendance for all primary classes. 
 
Within each of the 160 communities, five parents of assessed B2 students will be randomly selected, for a total of 800 parents. The parent sample will include the child’s primary caregiver.  The sample size will allow the project to assess a difference of 10 percentage points in reported attendance, with a large standard deviation and 30% attrition. By conducting surveys with parents of assessed students, the evaluator will be able to assess if and to what extent parental engagement / accountability practices predict children’s performance. 
 
In each of the four regions, the evaluator will survey the Director and Budget Officer, and at the district level, at least two SISOs, using quantitative surveys and the OCA. At the national level, the evaluator will conduct surveys with ten individuals within each institution (MOE, national level GES, NaSIA, NACCA, NTC), for a total of 80 individuals surveyed. These individuals will be selectively chosen based on the relevance of their positions as it pertains to rolling out the EAF.
 
6. Approvals
The external evaluator will provide the documentation required for an Institutional Review Board (IRB) approval prior to the onset of data collection. The documentation will include: 
  • Summary of the methodology
  • Tools, including xls forms and reader-friendly PDF format 
  • Summary of the sample
  • Presentation to the IRB
  • Permission letter from GES to conduct the study

[1] The external evaluator may elect to use tablets/ phones only instead of voice recorders, taking advantage of the capture of sound files through Kobo or ONA. The quality of the sound may not be as good as when using voice recorders, however, leading to data loss when operating in field conditions.

[1] Note that in North East region, there only two districts are part of the ZOI, hence both will be selected.

[1] The primary caregiver may be a parent or a relative, in case the child does not live with his/her parents.

[1] May be prepared using Enketo

CARE will be responsible for liaising with the relevant authorities to obtain approvals for the study. 
 
7. Data collection and quality assurance
The evaluator will be responsible for recruiting, vetting, and training enumerators and field supervisors to conduct the study. The data collection team is expected to be gender-balanced and fluent in the local languages spoken in the assessed locations (i.e., Mampruli, Kusaal, Waali, Dagbani). CARE will require a minimum of four days of training and one day of field practice for this assignment. Prior to deployment for data collection, all field supervisors and enumerators must pass a post-test assessing their knowledge of the data collection protocol and quality assurance requirements. The evaluator will be responsible for the design and administration of the post-test, working in collaboration with CARE’s MEAL Specialist.  
 
The training of enumerators is expected to include, at a minimum: 
  • An overview of the project and its evaluation methodology
  • Informed consent and confidentiality
  • GESI-responsive practices in data collection, including minimum standards and potential scenarios
  • Research ethics and safeguarding practices
  • Sampling procedures
  • An overview of each tool
  • Data quality assurance procedures and quality checklists
  • Mock practices with each tool
  • Qualitative data collection practices – facilitation, inclusiveness, probing
  • Field practices with each tool
  • Data storage and transfer
  • Common troubleshooting
  • Daily reporting and oversight
The external evaluator will be required to: 
  • Conduct spot-checks with data collection teams and supervisors.
  • Review uploaded datasets on a regular basis to assess enumerator and supervisor performance and spot errors in a timely manner.
  • Provide weekly data collection reports to CARE.
  • Provide interim datasets to CARE’s Director of Research (following a safe transfer protocol established by CARE) to enable shadow verification of data quality. 
  • Clean and validate datasets prior to data analysis. 
The Consultant will be responsible for all travel and logistics associated with conducting the baseline assessment. 
 
8. Safeguarding, Inclusion, and Research Ethics
CARE has a zero-tolerance policy for any form of sexual harassment, exploitation, and abuse. The selected external evaluator and all supervisors and enumerators will be required to participate in safeguarding training provided by CARE and to sign CARE’s Safeguarding Policy. CARE will inform school communities of our safeguarding policy and reporting mechanisms prior to data collection. Any reports of safeguarding policy violations will be investigated confidentially and reported to local authorities as per local law. If any 
safeguarding incidents are witnessed or reported to supervisors and/or enumerators during data collection, those should be reported to CARE immediately during CARE’s hotline +233 24403340699 and/or CARE’s global Ethicspoint online reporting system
 
Enumerators will only interview students in areas where the respondent can be visible to other people within the school or adjacent community. The student’s answers should not be overheard by other individuals, but the location should allow the student to exit the room/ area and call for adult help easily if necessary. Enumerators and supervisors will not make promises on behalf of the project or use project deliverables as a means of coercing participation. 
 
The external evaluator is required to train enumerators and supervisors on procedures to interview children and adults with disabilities, including both physical, cognitive, and mental health disabilities. While it is anticipated that some individuals with severe visual and/or auditive disabilities may not be included due to the limited use of Braille (in case of learning assessments) and sign language in targeted areas, this study will endeavor to accommodate the needs of those with limited loss of functionality to the extent possible. CARE requires data to be disaggregated by disability status as per the project’s Performance Indicators Reference Sheets (PIRS). The evaluator will be required to collect data on disability status using the Washington Group Short Set on Functioning. 
 
Enumerators will be required to ask for informed consent of all respondents – adults and children – using age-appropriate scripts. Enumerators will also inform respondents of their right to leave the interview at any time or refuse participation without fear of reprisal.
   
The external evaluator is responsible for ensuring that enumerators and supervisors adhere to confidentiality throughout the survey. The data collection team will not be allowed to take photographs or videos of participants, regardless of the circumstances. All data collected will be transferred and stored safely using encrypted systems. Only the main data analyst at the evaluation company and CARE’s Director of Research will have access to non-anonymized datasets. 
 
CARE will train enumerators and supervisors on its Policy on Fraud and Corruption. CARE has zero tolerance for data fraud. The evaluator is required to verify potential instances of fraud. CARE will spot-check data collection and verify trends in uploaded datasets to identify potential cases of false surveys/ assessments, impossible responses, and inconsistencies.  
 
CARE reserves the right to terminate this contract at any time, without prior notice, in case of policy violations.
 
9. Reporting  
The external evaluator will propose an outline of the baseline study and analysis framework to CARE as part of the application package. The analysis framework should include, at a minimum: system, school, and student profiles; analysis of learning outcomes, attendance, and teaching practices; analysis of accountability practices at school and system levels, and perceptions related to those; regression analyses identifying factors contributing to learning and attendance; and baseline values for all relevant indicators as per the project’s MEL Plan. The analysis framework should set the groundwork for the project to respond to its evaluation questions and learning agenda. 
 
Once finalizing the initial analysis, the evaluator will conduct an interpretation workshop, presenting the findings to USAID, MOE, project representatives, and representatives from the ZOI’s schools. The recommendations from the interpretation workshop will inform the final analysis. The evaluator will prepare a draft report for USAID, MOE, and CARE’s review and feedback. Upon receiving the feedback, the evaluator will finalize the report and prepare a reader-friendly summary of the findings for external dissemination. 
 
10. Learning 
The evaluator will present the baseline results and recommendations to the Project Steering Committee; the Education Sector Working Group; and the USAID/SAGES Systems Strengthening Activity technical team. The Activity technical team will use the findings and recommendations to (i) inform adaptations to the design and (ii) prepare learning products for external dissemination with regional and district education directorates, national, regional, and global audiences. The project will upload the report and anonymized datasets to USAID’s Development Experience Clearinghouse (DEC). The project will also upload the report in the CARE Evaluations website. 
 
11. Key Deliverables
The evaluator will provide the following deliverables as a result of this contract: 
Deliverable set 1 – Evaluation setup 
  • Inception report containing an outline of the methodology; tools; school sample; quality assurance protocols; and detailed analysis framework. 
  • Set of training materials, including agenda, presentations, and post-tests. 
  • Data collection training report. 
  • Pilot test report. 
  • Data collection plan.
Deliverable set II – Data collection products
  • Data collection weekly reports and completion report. 
  • Raw quantitative datasets with complete codebooks (in SPSS or STATA files).
  • Cleaned quantitative datasets with complete codebooks and Do-files (in SPSS or STATA files).
  • Qualitative recordings in electronic format. 
  • Complete transcription and translation of qualitative files. 
Deliverable set III – Reporting
  • Draft presentation of findings for interpretation workshop
  • Draft baseline report
  • Final baseline report
  • Reader-friendly summary of baseline findings
  • Baseline Policy Brief (maximum of two pages)
  • Baseline report infographics (one page)
12. Timeline
As part of the application package, potential evaluators are required to complete a Gantt chart including the following tasks and the proposed timeline (see illustrative table below). The draft baseline study report should be submitted no later than midnight on December 31, 2023.
 

Task

Submission date

Inception report

16th October 2023

Tool development, pre-test, and piloting of tools

22nd October 2023

Training of data collectors

28th October 2023

Field Work

30th November 2023

Data cleaning/transcription

5th December 2023

Data analysis

26th December 2023

Submission of draft report and Preliminary findings PPT

31st December 2023

Submission of final report, datasets, and PPTs

15th January 2024

Required Skills or Experience

Qualification and Experience
At a minimum, the evaluation firm should propose three technical team members comprising:
  • Team leader/ quantitative data analyst - with substantial experience in conducting large scale education evaluations of similar scope, including but not limited to the use of early grade assessments.
  • Qualitative data analyst with experience in gender in education, preferably in Ghana.
  • System strengthening specialist with experience in assessing sector development (i.e., decentralization, use of data systems in education supervision, accountability frameworks in education, or equivalent areas).
Any changes to the key personnel above would require CARE’s pre-approval. 
 
In addition to the above, the firm shall demonstrate the following qualifications and experiences in the technical proposal:
  • Advanced qualification in Education (at least one team member) with prior experience of conducting education research. 
  • Demonstrated team experience in conducting studies of similar scope, including but not limited to large-scale learning assessments, institutional capacity analyses/ system analyses, and school surveys. 
  • Demonstrated experience in complex mixed-methods evaluations, including advanced skills in statistical analysis and qualitative analyses using emergent codes.
  • Previous experience in donor-funded evaluations, including but not limited to USAID, FCDO, or Global Partnership for Education-funded projects. 
  • Expertise in GESI in system strengthening in education. 
  • Excellent writing skills in English. 
  • Prior experience in Ghana or West Africa is highly preferred. 
CARE reserves the right to request references and examples of prior experience previously to award.

How To Apply

Sorry, job has expired.

 
Note

Please note, employers receive numerous applications per posting and will only shortlist the most qualified candidates. Also Jobsinghana.com is not involved in any decision made by an employer/recruiter and therefore does not guarantee that applications sent will result in a candidate being shortlisted/selected for that position.
 
 
To Top