Job Description
Part A: Cover Page
Issuance Date: 03/08/2020
Questions Due Date/Time: Friday 7th August 2020, 12:00 Noon GMT/ poliveira@righttoplay.com
Proposal Due Date/Time: Monday 17th August 2020, 12:00 Noon GMT, 5:00PM GMT/ RTP UK Country Office
The Partners in Play (P3) Project implemented by Right To Play and funded by the LEGO Foundation, is soliciting proposals for the services of an external evaluator.
Please submit your most competitive proposal in accordance with the terms of reference and instructions to offerors. Any award issued as a result of this RFP will be subject to all instructions, terms of reference/ specifications, certifications, terms and conditions and funder required clauses. This RFP document includes the following parts:
PART A: Cover Page
PART B: Terms of Reference
PART C: Instructions to Offerors
All proposals, inquiries, and correspondence pertaining to this solicitation are to be directed to the attention of:
PARTNERS IN PLAY (P3) PROJECT
Attn: Patricia Oliveira, Global Monitoring, Evaluation and Learning Manager
Email : poliveira@righttoplay.com
with Kwarteng Frimpong, Monitoring, Evaluation and Learning Specialist in copy at: KFrimpong@righttoplay.com
Part B: Terms of Reference
1.0 RIGHT TO PLAY
Right To Play is a global organization committed to improving the lives of children and youth affected by conflict, disease and poverty. Established in 2000, Right To Play has pioneered a unique play-based approach to learning and development which focuses on quality education, life skills, health, gender equality, child protection and building peaceful communities. With programming in over 20 countries, Right To Play transforms the lives of more than one million children each week using play and sports, both inside and outside of the classroom. In addition to our work with children, Right To Play advocates with parents, local communities, and governments to advance the fundamental rights of all children. Right To Play is headquartered in Toronto, Canada and has operations in North America, Europe, the Middle East, Africa and Asia. Visit us at www.righttoplay.com
As such, Right To Play is seeking a consultancy firm/group to undertake the baseline study of the Partners in Play (P3) project in Ghana including the development of an evaluation framework design; the review, refinement and/or development of instruments; fieldwork administration and logistics plan; data collection, review and cleaning; data analysis; reporting; and dissemination documents. This will include the writing of all accompanying reports as specified in the key deliverables.
2.0 PROGRAM OVERVIEW
The Partners in Play Project (P3) aims to improve the quality education for Ghanaian girls and boys aged 4-12 through a scalable and replicable Learn Through Play (LtP) model.
Target Area and Reach
With the support of RTP, the project will reach 689,920 children, representing three levels of intervention in three regions (Accra Region, Northern Region, Volta Region):
- 32,000 children (14,720 F); from 100 schools that will receive direct and intense RTP support (includes training of 800 teachers)
- 81,920 children (37,683); from 257 schools that will receive support from district support teams (RTP project staff will train district teams and provide support in the training, coaching and mentoring of additional 2,048 teachers)
- 576,000 children (264,960 F) from 1800 schools, part of the GALOP project (Ghana Accountability for Learning Outcomes Project) that will receive support from district support teams. District support teams will train, coach and mentor 14,400 GALOP teachers
An additional 2.55M children in Ghana will benefit from LtP over the next four years as part of the RTP’s involvement in the GALOP initiative. This refers specifically to the integration of LtP into the national in-service training framework. This intervention/phase will not be covered by this baseline evaluation.
This will be achieved through a series of Key Activities, organized by target group, which include:
i. Government:
- Validation workshop of the teachers´ coaching and Mentoring guides and framework,
- RTP participation in the working group on the development of training framework and training plans,
- Develop resource manuals for the integration of play-based learning in the existing subjects in teacher's education colleges,
- Partner with NaCCA, The National Council for Curriculum and Assessment in the development of the play-based teachers' guide to support KG and primary schools in the use of PBL to deliver the curriculum,
- Update of National INSET guidelines and development of content to include LtP approach,
ii. Professionals
- Purchase of Teaching & Learning materials to support play-based learning in 100 implementing schools,
- Training for teachers in Continuum of Teacher Training (CoTT)
- Facilitate a 3-day training on cascading model training framework aligned to the district INSET Plans,
- Workshop for selected education officials and teacher champions to develop plans for INSET/CPD per school and cluster level,
- Hold a workshop with teacher's champions and district support teams to develop the guidelines and content of the Learning Communities,
- Training for teacher champions on coaching and mentoring,
iii. Caregivers
- Conduct a 2-day training for CBOs/CSO on sports for development and LtP approaches and supplemental learning for children,
- Awareness raising events on the importance of learning through play, supplemental learning and child protection issues,
- Training provided to Parent Councils on LtP, gender equality, child protection, school management and supporting school action plans,
iv. Systems
- Develop District-level coaching and mentoring plan in collaboration with district education officials,
- Training of 765 district officials on coaching and mentoring and teacher portfolio building,
- Support partner districts on regular field support to schools/ coaching and mentoring trained teachers,
- Training of 24 National Trainers on PBL,
- Training of 765 Teachers District Support Teams in LtP
- Training of 200 National level Master
- Hold series of workshops with different faculty members to identify integration process of LtP into existing subjects/course and establish working groups for the development of LtP teaching guides
- Conduct quarterly meetings with education colleges on the integration of LtP in teaching and learning practices including practical experiences in schools for student teachers
- Training of teachers on play-based learning from supported teaching schools
P3 project defines success as improving the quality of education in Ghana by increasing literacy outcomes, attendance rates and improved holistic/core skills development among children.
Additionally, the project aims to achieve the following key interrelated Outcomes organized by target group, which will lead to improved quality education at four levels:
i. Government:
- Lego Foundation Outcome 4.2. Ghana adopts policies that include holistic/core skills that enable Learn through Play (LtP)
- Immediate Outcome 1.1 Improved capacity of education sector to integrate LtP to deliver the curriculum.
ii. Professionals
- Lego Foundation Immediate Outcome 2.1. Professionals have improved or increased awareness and understanding to implement LtP in their learning environment
- Lego Foundation Outcome 2.2. Professionals have improved skills to implement LtP within their learning environments
- Lego Foundation Outcome 2.3. Professionals implement the LtP approach in their learning environments
- Immediate Outcome 2.1 Improved knowledge and skills & practices of teachers to implement LtP
- Immediate Outcome 2.2 Improved capacity of Communities of Practice (CoPs) to provide peer support to teachers to integrate LtP approaches.
iii. Caregivers
- Lego Foundation Outcome 1.1. Caregivers have increased awareness and understanding of the importance of holistic/core skills for their children and how LtP develops these skills
- Lego Foundation Outcome 1.4. Caregivers actively support LtP
- Immediate Outcome 3.1 Improved awareness of learning needs of girls and boys among caregivers.
iv. Systems
- Lego Foundation Outcome 3.2. Institutions implement curricula and development plans that include LtP
- Immediate Outcome 4.1 Improved ability of district administration to support teachers to integrate LtP into their teaching practice.
- Immediate Outcome 4.2 Improved capacity of national, regional and district level education officials to plan and monitor the integration integrate LtP into INSET (in service training)
- Immediate Outcome 4.3 Improved capacity of pre-service teacher training institutions to integrate LtP into training for student teachers.
In addition to following outcomes indicators will also be measured: %/total of children that demonstrate life skills (M/F); % of children who perceive their learning climate to be supportive (M/F); % of lessons which engage learners in child-centered learning
Please refer to Outcomes Logframe indicators in Annex I
For background information, please refer to The LEGO Foundation report Learning through play: a review of the evidence´(2017) (https://www.legofoundation.com/media/1063/learning-through-play_web.pdf)
3.0 OBJECTIVES
The primary objectives of the baseline study are as follows:
- To produce a rigorous evaluation framework design, including an analytical framework which can be used for the midline and endline evaluations;
- To review, refine and/or develop reliable and valid survey and other measurement instruments;
- To report the baseline starting points for the P3 project’s intended outcomes;
- To inform realistic and achievable targets that are grounded within the local context;
- To produce evidence which facilitate discussion and learning regarding program best practices among beneficiaries, community members, Right To Play staff, partners and institutional stakeholders;
- To produce evidence which inform organizational strategic learning and continuous improvement to strengthen capacity to carry out quality program design, implementation, and monitoring and evaluation in the future.
4.0 EVALUATION OVERVIEW
The objective of these ToR is to procure the services of an independent external evaluator to conduct a mixed-method base line evaluation of the project P3.
The evaluation design will allow to assess the coherence, delivery, effectiveness, sustainability and impact of the project and report the findings and lessons learnt throughout the process.
Evaluation questions:
The Evaluation Team will be required to develop an evaluation approach that answers the following overarching questions:
- Process – Was the project successfully designed and implemented?
- Impact – What impact did the project have on the main outcomes of interest and specifically its contribution to increasing literacy outcomes, attendance rates and improved holistic/core skills among children. How and why was this impact achieved?
- Value for Money (VfM) – Did the project demonstrate a good VfM approach?
- Effectiveness – What worked (and did not work) to increase the main outcomes of interest and specially literacy, attendance and holistic/core skills among girls as defined by the project?
- Sustainability – How sustainable were the learn through Play activities, and was the project successful in leveraging additional interest and investment?
Specific project level evaluation questions will be outlined as part of the MEL Framework. These questions will help define the scope and focus of the project evaluation process. The successful bidder will be expected to work with the Project Management Team to review and revise these questions as appropriate at the outset of the project. Project specific context is important in this respect
Activities/Tasks (Services) or Specifications (Goods)
- The initial part of the baseline study will involve a comprehensive review of the P3 program, monitoring, and evaluation plan including a desk research and literature review which include the following key project documents:
- logic model (RTP’s ToC),
- performance measurement framework,
- MEL Plan,
- review of existing tools,
- Propose measurement tools (including revision of existing tools).
- Development of a robust evaluation framework (using the OECD/DAC criteria for evaluating development assistance). This will comprise relevant research questions, a rigorous sampling methodology; and review, and/or draft measurement instruments to be piloted during the training of data collectors and enumerators.
- Sample design. The sampling methodology should include baseline sample size calculations while considering the need to assure meaningful statistical testing at the endline.
- Data collection: A fieldwork/data collection administration and logistics plan (includes Data quality assurance detailed process) will be developed and implemented, ensuring coordination with Right To Play country team. Travel will be required and the consultancy team will need to develop a robust data collection strategy while ensuring ability to train (as needed) enumerators, to collect the requested qualitative and quantitative data. Right To Play Country Office is expected to be involved in the coordination with national counterpart for approval in accessing beneficiaries, a consultancy firm/group should be mindful of this during planning.
- Data analysis and report writing. The final part of the baseline study process will consist of the review, cleaning and analysis of the data collected and drafting of the country-level report with inputs from the validation workshop. The consultancy team must also produce key findings that will be used for dissemination.
5.0 TIMELINE
The baseline study for the P3 program will be conducted within the anticipated timeline of September 2020 to March 2021, depending on COVID19 circumstances and related school opening.
6.0 LOCATION
The evaluation framework design process is expected to be carried out remotely at the consultants’ base, with significant collaboration with the Global MEL Right to Play Monitoring, Evaluation and Learning team and project Management Team. For the field work and data collection, extensive in-country travel is expected in Ghana as per the proposed evaluation approach.
7. 0 METHODOLOGY
Overall evaluation approach and design
The overall evaluation approach requires the Evaluation Team to design, plan and conduct a mixed-methods evaluation.
Research design
Comparison groups: bidders are required to outline their approach to evaluating the impact of the project. This should include consideration of the most rigorous approach to establishing a counterfactual. This should enable comparison of the outcomes achieved by a target group who were affected by a project intervention with the outcomes achieved by a group who are similar in every way to the target group, except that they have not in any way been exposed to or affected by the project intervention i.e. a comparison group. Careful consideration should be given to the use of quasi-experimental methods for this purpose.
Measuring outcomes: bidders are expected to understand the project’s key and immediate outcomes and suggest the most appropriate data collection approach to evaluate each outcome indicators. This should include a mixture of quantitative and qualitative approaches.
The Evaluator will be expected to pilot tools that will be used for data collection and refine as necessary.
Project sampling framework: The Evaluation Team will be required to propose a sampling framework for both qualitative and quantitative samples. These should be of a sufficient size and representativeness to allow reasonable:
- Levels of certainty that the findings are representative for the target population.
- Ability to generalise the intervention’s effectiveness to similar contexts.
- Ability to generalise the insights into what works and why for similar contexts. Minimum sample sizes required to determine whether the intervention is effective with the degree of precision required. Determining the proper sample size requires (1) calculating a minimum sample size necessary, (2) accounting for sampling design, and (3) accounting for attrition. The sampling size should reflect the three levels of intervention described above (100 schools direct and intense RTP support; 257 schools who will receive support from district support teams with additional RTP support; 1800 GALOP schools that will receive support only from district support teams under RTP´s cascade model).
Research questions may include:
- Is the P3 programme relevant to the contexts where it operates and whether it is reaching the most vulnerable children,
- Does the intervention address the barriers that most significantly affect educational outcomes
- Is the theory of change consistent with the outcomes it aims to achieve and likely to lead to the effectiveness of the intervention,
- What are the current levels of children´s life skills proficiency? What is the impact of Right to Play´s child centered learning in the development of such skills across time?
- Does Right To Play´s particular approach on training, coaching and mentoring for teachers and other educational personnel lead to successful application of learning through play? In which ways?
- What specific Right To Play child centered practices are associated with increased learn through play methods used in the classroom?
- What are the current levels of teacher´s awareness, knowledge and skills to facilitate learn through in their learning environment with their children?
- What is the impact of RTP´s teaching, coaching and mentoring approach to teacher´s instruction and management practices in the classroom (teacher directed/child-centered)?
- In which ways does Right To Play´s particular approach engages parents and caregivers in being aware of and support the learn through play approach and gender-specific learning needs of girls and boys?
- What aspects of RTP´s Cascade Model are adopted into national, regional and district level systems that operationalise Learn through Play?
Data Collection Tools
The evaluation will include a toolkit with relevant, appropriate tools to measure progress on the outcome indicators selected and to adequately answer the research questions posed. It is required that the consultancy firm refine the questions and propose additional ones to answer each of the outcomes identified above.
Where necessary as per indicator needs, existing tools will undergo a critical review and revision while new ones will be developed. The following is a (non-exhaustive) list of tools that will need to be revised/developed:
- Teacher Coaching and Mentoring Assessment (existing)
- Teacher Survey
- Life Skills Assessment (existing ISELA tool)
- Interview Guides (stakeholders and beneficiaries)
- Focus Group Discussions (FDG, children, parents, community members, other stakeholders)
- Classroom Observation (Child centered learning- existing)
- Parents’ Knowledge Attitudes and Practices (KAP)
- School/Classroom Environment Review Guide (existing)
- School Records Guide (for attendance, etc)
Data collection plan/ quality assurance process
The research planning process will entail an in-depth training of enumerators and data collectors which will include piloting of the instruments to ensure satisfactory reliability and validity. Tools will be subsequently refined after training and prior to going to the field. Firms will need to collect data using tablets and other softwares.
Data Analysis
Data on outcome indicators will be analyzed both quantitatively and qualitatively. Moreover, an analytical framework is required to be submitted as part of the evaluation design process which will detail the specific analytical methodology to produce the result for each indicator and each research question posed. The consultant will engage significantly with the Right To Play team to determine and agree on these.
The consultant will require to utilize analytical software to analyze both the quantitative (e.g. Excel, SPSS, STATA, R), and qualitative (e.g. Nvivo) data. Outputs from data analysis will be submitted as part of the deliverables, as will the scripts (or list of commands) with clear notes/guidance, particularly for quantitative data analyzed (in SPSS, STATA, R).
8.0 GENERAL CONDITIONS OF THE CONSULTANCY
Steering Committee
A steering committee of key implementation stakeholders (i.e. Right To Play country and global staff) will be formed to guide and inform the research process. They will help to inform the relevance and appropriateness of the evaluation design framework, the data collection tools and the analytical framework. They will also help to ensure that the research planning and data collection processes are sound, culturally appropriate and contextually relevant to Right To Play’s programmatic needs and to the needs of all relevant stakeholders (i.e. beneficiaries, community members and partners).
Measurement Plan
The measurement plan adheres to the P3 program Performance Measurement Framework (PMF) and MEL Plan and uses a results-based management approach for tracking the progress of outputs and outcomes against targets.
Consultancy Expectations
- The consultancy firm/group will have an orientation to Right To Play’s program delivery model;
- The consultancy firm/group will submit a detailed work plan and time frame for the completion of the baseline research;
- The data analysis and draft reports will be shared with Right To Play with sufficient time to allow for Right To Play to review and provide feedback, which will be incorporated into subsequent work;
- After the completion of any research reports, a presentation will be given to Right To Play staff and appropriate stakeholders to share both results and recommendations;
- Throughout each phase, the consultancy firm/group will be expected to maintain regular communications with Right To Play regarding progress;
- Travel and accommodations for visits within the country are to be included in the budget
- The consultancy firm/group must follow OECD-DAC principles for evaluation;
- All materials, processes, methodologies, reports, plans and other works provided to the consultancy firm/group or developed by the consultancy firm/group on behalf of Right To Play remain the property of Right To Play;
- All data must be stored in a safe and secure location, allowing full access to Right To Play staff during the evaluation process;
- Upon completion of the research, all raw data must be submitted to Right To Play.
9.0 ROLES & RESPONSIBILITIES (SUMMARY)
The consultancy firm/group will report directly to RTP´s Global, Monitoring, Evaluation and Learning Manager at Right To Play International and all deliverables should be submitted to her according to the schedule outlined in the “Key Deliverables and Timeline” section below.
The consultancy firm/group’s roles and responsibilities include, but are not limited to, the following:
- Background research to familiarize themselves with the P3 program and Right To Play’s approach (i.e. program documents provided by Right To Play);
- Document review of program, monitoring and evaluation plans, MEL Plan, including logical model, performance measurement framework, and existing tools;
- Development of the evaluation design/framework in collaboration with the Right To Play International Steering Committee;
- Development and implementation of a fieldwork/data collection administration and logistics plan of implementation;
- Data collection, entry & cleaning, and analysis;
- Data validation and interpretation through internal review of findings (with inputs from validation workshop);
- Completion of final report
- Preparation and presentation of findings and recommendations.
10. KEY DELIVERABLES AND TIMELINE (NEGOTIABLE AND TO BE FINALIZED WITH RTP STAFF)
The above timeline is negotiable and subject to change based on consultation with Right To Play International, and Country staff.
