implemented by a consortium formed by CARE, Afrikids, Community Development Alliance (CDA), Crown Agents, the Ghana National Education Campaign Coalition (GNECC), and School for Life (SfL), operating in close collaboration with the MOE and Ghana Education Service (GES). 

Through the present Terms of Reference (ToR), CARE seeks to identify a consultancy group/firm to conduct an independent baseline study for USAID/Ghana Strengthening Accountability in Ghana’s Education System (SAGES) Systems Strengthening Activity. The following sections outline the key objectives, research questions, expected methodology, deliverables, and timeline for this study. The ToR includes a summary of the required qualifications for applicants and an outline of the application process. 
 

The key objectives of the baseline study are as follows: 

The baseline study will generate information to enable the Activity to respond to the following evaluation questions: 

The USAID/Ghana SAGES Systems Strengthening Activity’s evaluation will use a mixed-methods, pre-post design where project achievements will be measured vis a vis benchmarks set up at the baseline. It is currently anticipated that the project will target all primary schools in USAID’s ZOI, thus precluding the use of a counterfactual (control or comparison group). Furthermore, the project’s approach to improving accountability within the MOE assumes that activities will contribute to impact throughout the education system. The evaluation will assess the project’s contribution through (i) achievement vis a vis benchmarks set at baseline and (ii) determining if the uptake of specific accountability approaches predicts significant change in education outcomes and to what extent this is occurring.
 

a. Measure the learning outcomes of B2 students: The evaluation will assess differences in B2 student learning outcomes using a cross-sectional approach. The project will use the Early Grade Reading Assessment (EGRA) and Early Grade Math Assessment (EGMA) to assess literacy and numeracy outcomes, respectively. If possible, the baseline will conduct the EGRA and EGMA at the same time as the government’s administration of the B2 National Standard Test (NST) to enable the project to conduct an equating study comparing the MOE-administered B2 NST results with EGRA and EGMA results obtained through the baseline. Prior to conducting the baseline, the Activity will request the student names and IDs for the B2 NST that will be conducted at the same time, to enable student-level comparisons.

b. Assess the behavior and perceptions of school community members through surveys: The baseline will conduct surveys with representative samples of community members, School Management Committee (SMC) members, parents of assessed students, teachers, headteachers, district education directorate staff, regional education directorate staff, and MOE officials at the national level. The surveys will assess: i) the application of improved classroom practices using the Inspection Evaluation Framework (IEF); ii) student and teacher attendance (through headcounts vis a vis school records); iii) school environment conditions; iv) School Improvement Support Officer (SISO) support and behavior; iv) SMC and community engagement in holding schools accountable; v) existing school practices in tracking absenteeism and dropout and encouraging out-of-school children to return to class; vi) school practices regarding gender equity and social inclusion (GESI); vi) implementation of Performance and Accountability Support for Schools (PASS) and remedial classes; vii) parental engagement in their child’s education and in accountability processes in their child’s school; viii) district and regional level knowledge, behavior, support, and use of data; ix) national level system actors’ understanding of their roles and responsibilities and the functionality of systems; and x) capacity of civil society organizations (CSOs) to hold system actors accountable. 

c. Conduct an organizational capacity assessment: Given the Activity’s focus on systems strengthening and the fact that the Activity is intended to operationalize the EAF, an organizational capacity assessment (OCA) is particularly pertinent as part of the internal evaluation. An OCA allows for a deeper understanding of the processes, knowledge, behaviors, and capabilities of the organization, and can be aligned with the key functions outlined in the EAF. For the purposes of the baseline, an “organization” will be defined as a District Education Directorate; Regional Education Directorate; the three independent education bodies – National Council for 

Curriculum and Assessment (NaCCA), National Teaching Council (NTC), and National School Inspections Authority (NaSIA); the MOE as the civil service arm of the education system; and the GES at the national level. As part of the baseline study, representatives from each of those organizations will participate in a self-administered OCA. The external evaluator and the project will jointly agree on the areas of inquiry in the OCA, which may include awareness of the EAF; knowledge of their functions documented within the EAF; capability in executing the EAF; GESI in EAF implementation, etc. These domains will also be leveled; for example, awareness of the EAF may constitute the lowest level, followed by knowledge and then capabilities. The self-rating will then be summed into a weighted index for the individual, which will then be aggregated at the organizational level. This will allow measurement of an organization’s performance over time and can be used to measure systems strengthening.

The quantitative tools outlined above will be complemented with qualitative tools, including Focus Group Discussions (FGDs) and Key Informant Interviews (KIIs). The external evaluator may propose alternative approaches to those.  The project plans to use the Most Significant Change (MSC) approach at the midterm and final evaluation, alongside rolling profiles, to assess in-depth changes in attitudes and practices at system and school levels. 

In addition to the above-mentioned quantitative and qualitative approaches, the evaluator will also conduct a comprehensive literature review to assess how contextual and policy-related trends may affect the expected outcomes. The literature review should include but not be limited to most recent system analyses,  education statistics, policy analyses, studies on social and economic trends affecting education outcomes, and studies on cultural aspects affecting accountability and education results.

The external evaluator will be responsible for the design of the data collection tools. To design the EGRA and EGMA tools, the external evaluator will conduct a desk review of existing instruments used in Ghana; determine the minimum set of tasks to be included in each tool, considering potential floor and ceiling effects based on prior performance and the grades assessed; and finalize the design of the literacy tasks through a one-day consultative workshop conducted with MOE and consortium partners. It is expected that the content of the literacy tasks (i.e., letters, non-words, familiar words, vocabulary and syntax for passage reading) will need to be calibrated considering the assessed sample’s linguistic backgrounds and reading levels. This will be done through the consultative workshop. To avoid potential pre-exposure bias, the evaluator is required not to adopt existing tools verbatim; additionally, the evaluator is required to develop a new story for passage reading, not copying textbook content or common folk tales. 

The evaluator will use electronic tools for quantitative data collection. The MEAL Specialist and CARE’s Director of Research / Education will review the electronic tools and provide feedback as necessary. Prior to deployment, the evaluator will pilot-test school-level tools in non-sampled schools with comparable characteristics, adjusting the content as necessary to prevent floor or ceiling effects.

The evaluator will be required to record, transcribe, and translate all qualitative data. The evaluator is expected to use electronic voice recording during qualitative interviews. 

• Summary of the methodology
• Tools, including xls forms and reader-friendly PDF format 
• Summary of the sample
• Presentation to the IRB
• Permission letter from GES to conduct the study

[1] The external evaluator may elect to use tablets/ phones only instead of voice recorders, taking advantage of the capture of sound files through Kobo or ONA. The quality of the sound may not be as good as when using voice recorders, however, leading to data loss when operating in field conditions.

[1] Note that in North East region, there only two districts are part of the ZOI, hence both will be selected.

[1] The primary caregiver may be a parent or a relative, in case the child does not live with his/her parents.

[1] May be prepared using Enketo

• An overview of the project and its evaluation methodology
• Informed consent and confidentiality
• GESI-responsive practices in data collection, including minimum standards and potential scenarios
• Research ethics and safeguarding practices
• Sampling procedures
• An overview of each tool
• Data quality assurance procedures and quality checklists
• Mock practices with each tool
• Qualitative data collection practices – facilitation, inclusiveness, probing
• Field practices with each tool
• Data storage and transfer
• Common troubleshooting
• Daily reporting and oversight

• Conduct spot-checks with data collection teams and supervisors.
• Review uploaded datasets on a regular basis to assess enumerator and supervisor performance and spot errors in a timely manner.
• Provide weekly data collection reports to CARE.
• Provide interim datasets to CARE’s Director of Research (following a safe transfer protocol established by CARE) to enable shadow verification of data quality. 
• Clean and validate datasets prior to data analysis.