Consultancies: Early Childhood Development (ECD) and Early Childhood Education (ECE), Two Team Members, 9 months, Evaluation Office, NYHQ

• Country: USA
• Organization: UNICEF 
• Deadline: 2023-11-22
• No of Vacancies: N/A

The evaluation will pursue three specific objectives, namely:

1. Assess UNICEF work in support of national policy reform, financing strategies, and budgeting to expand access to opportunities for early stimulation and learning for children from 0-8 years old, including children in the pre-school grade, and for strengthening the ECE sub-sector in countries of different typologies.
2. Examine the efficacy of UNICEF integrated ECD approach and the adequacy of available capacities (core skills, tools, processes, partnerships, coordination mechanisms, and institutional arrangements) to support countries of different typologies in the adaption, acceleration, and scale-up efforts for providing early stimulation and learning support and services for different profiles of children from 0-8 years old, including children in the pre-school grade.
3. Evaluate UNICEF inputs and outcomes in supporting countries of different typologies to develop effective programmes to increase access to opportunities for early stimulation and learning for children from 0-8 years old, including children in the pre-school grade (the grade before Primary 1) to achieve universal access to pre-primary (SDG 4.2), and to promote school readiness among children and caregivers.

The evaluation will occur in four phases (i) inception; (ii) desk-based document review and analysis; (iii) field-based data collection; and, (iv) data analysis, reporting and communication of evaluation results. Each of the stages is described below.

Phase 1-Inception: The inception face will feature five activities, as described below:

1. Search, compile and conduct a preliminary review of UNICEF documents and academic and grey literature: This activity is meant to deepen the understanding of the context of UNICEF ECD/ECE programming, and to develop a succinct synthesis of the global context and UNICEF role in early childhood care and education.
2. Conduct a systematic stakeholder analysis: Concurrent with compilation of documents, key groups of stakeholders will be identified (i.e., UNICEF staff members, consultants, international NGO partners, cluster members/partners, donors, researchers/consultancies and other independents subject matter experts, etc.), and a stakeholder analysis will be conducted to determine the role of each key stakeholder and inputs in ECD/ECE, and in the evaluation process. Groups of stakeholders will be judged on three parameters: (i) interest and involvement in ECD/ECE work; (ii) influence/power that they exercise in major decisions about UNICEF work and, (iii) involvement of the stakeholder in the evaluation and expected impact of their involvement. Based on the stakeholder analysis, the evaluation team will articulate an approach for engagement and management of the different stakeholder groups, to be used by the evaluation manager and the evaluation team. The stakeholder engagement strategy should be included as an annex of the inception report.
3. Compilation of all elements of the inception report and submission: The inception report will include (i) revised evaluation questions (improved from the ToR version and informed by users as necessary) presented in an evaluation matrix; (ii) expand on the methodology featuring a confirmation of countries to be included in the desk-based review (approximately 15), based on a well-articulated sampling strategy;  (iii) a proposal for the case study approach (thematic, country-based, or both), avoiding undue duplication with on-going and/or planned evaluations and/or research studies; and, (iv) draft evaluation tools for piloting. A more detailed outline for the inception report is provided in Section X of the attached terms of reference.
4. Convene a 1-day inception workshop: After circulating the draft inception report to the primary users, the evaluation manager will convene a virtual workshop to provide an opportunity for interaction between the evaluation team and the primary users, namely ECD and education teams in New York, and to the extent possible, the counterpart in country offices sampled for in-depth case studies. The focus of the workshop will be to validate evaluation questions, discuss the scope of the evaluation and key technical elements, and to communicate/manage expectations of the primary users. The inception workshop will also be used to familiarize the evaluation team with UNICEF standards, processes, and tools, clarify UNICEF expectations and contractual provisions, and to refine the work plan.
5. Presentation to the meeting of the Evaluation Reference Group, and revision of the inception report: The final activity of the inception phase will be to submit and present the inception report to the reference group. The evaluation team will produce a revised inception report, after consideration of the advice of the reference group.

Phase 2-Desk-based document review and country self-assessments: This phase will be data collection from secondary sources, featuring three activities:

Comprehensive search and compilation of documents from 15 country offices selected for the desk-based document review: Continuing from the search described in the inception phase, relevant UNICEF documentation from the global and regional levels, will be collected. These will include work plans, corporate policies, strategies, programme guidance, position papers, inter-agency and global education cluster documents, studies, reviews and evaluations from UNICEF and other organizations. From the country offices which will be a focus of the desk-based review, the team will collect planning and programming documents; situation reports (SitRep); needs assessments; surveys and statistics; donor proposals and reports; field monitoring reports from UNICEF and partners; country audits and financial data; data and reports. A significant amount of data is readily available from existing UNICEF and non-UNICEF databases. These must also be reviewed before any requests for data can be made to UNICEF country office staff, government counterparts, or any other key informants.

Conduct a detailed desk-based document review and analysis and light-touch country self- assessments: The focus of this activity is to review and analyse secondary data and other documentary evidence, and data from 15 selected countries. Task for this activity are: (i) to pre-populate the evaluation matrix for the 15 desk review countries; (ii) execute self-assessments by the ECD/Education teams in the countries that will selected for field-based data collection (using and existing/adapted self-assessment tools) to ensure that the evaluators understand the context and the ECD/ECE programmes fully and  refine the methodology for the theme-based case studies accordingly; (iii) refine data collection and data analysis tools for Phase 3; and, (iv) to orient and inform countries that will be asked to participate in the evaluative case studies and host primary data collection missions.

Drafting and submission of the document review report: To be reviewed only by the evaluation manager, the desk review report should present an updated chapter of the global context and UNICEF role in ECD/ECE, with an updated methodology section, and a 3-page summary on the context of each of the 15 countries featured in the desk-based review, confirm the themes for the evaluative case studies (2-3 themes). The format for this report is discussed in Section X of the attached terms of reference.

Phase 3: Primary data collection: The activities proposed for this phase are presented below. However, evaluation consultants are free to augment or propose alternatives data collection approaches.

In depth interviews with key informants at the global and regional level: Using the stakeholder analysis generated in Phase 1, a list of key informants will be updated by the evaluation team, in conjunction with the evaluation manager, UNICEF staff, and members of the Evaluation Reference Group (ERG). Primary data will then be collected through key informant interviews, in part to answer a subset of questions for which these informants will be the primary sources, and also to triangulate and/or corroborate some of the data from the desk-based review.

In-country visits: Field visits will be undertaken in a subset of countries that were designated for theme-based evaluative case studies (3 or 4 of the 15 countries, depending on the budget), also for the purpose of collecting primary data. The duration for each visit will be up to 10 working days. Each field visit will commence with a briefing meeting with UNICEF country office education teams, where the 3-page summary about the country context can be shared and subjected for review by the country office. Field-based data collection methods will include validation of the self-assessment, a country level stakeholder analysis, field observations, key informant interviews for UNICEF staff and implementing partners, as well as interviews and/or focus groups for beneficiaries. It is recommended to pilot the methodology and data collection tools during the first field mission before proceeding to subsequent missions. Two debriefing meetings will be conducted, one with the UNICEF country office, and another with interested evaluation participants to present key observations from the field missions, and where feasible, preliminary findings. The debriefing meeting will also be used as an opportunity to fill data gaps, and/or correct the record on factual information, were necessary.

Phase 4 – Data analysis and reporting: The activities proposed for this phase are presented below. However, evaluation consultants are free to augment or propose alternatives data analysis and reporting approaches.

Updating the data analysis plan: Accuracy, triangulation and disaggregation of data will be of crucial importance for the comprehensiveness, usefulness and credibility of the evaluation findings. hence data analysis and interpretation methods proffered in the earlier stages will be updated accordingly. Also, the evaluation will seek to identify contextual and structural/systemic factors explaining the evaluation findings – supporting or impeding performance of UNICEF. Some of the bottlenecks may relate to corporate characteristics, system-wide processes and cross-sectoral issues that lie beyond the education sector. These need to be listed but not examined in great depth.

Drafting and submission extended outline (zero draft): The data phase will conclude with the submission of an extended outline as described in Section X of the attached terms of reference, which will be a mock-up of the final report, and if possible, a collection of insights and/or preliminary findings from the field visits, which will be subjected to more systematic and comprehensive analysis.

Confirmatory online survey of findings: At the reporting stage, a single round of an online survey using the Delphi methodology, or a comparable technique will be executed to validate preliminary findings, and to establish the extent to which there is consensus and/or generalizability of findings across UNICEF offices. All UNICEF country and regional offices will be included (approximately 150 offices, which makes this a census approach). Delivered online, the survey will be addressed to the Education/ECD Specialist who will be requested to complete it, ideally with inputs from other programme leads. Each country office will be asked to return only one completed questionnaire.

Drafting and submission final report: Based on the outline in Section X of the attached terms of reference, the report will be presented in four drafts (completed first draft, revised draft, penultimate draft, and final report). The first draft will be reviewed by the evaluation manager and second reader within the Goal Area 2/5 team, while the revised draft will be presented to ERG for review. To be submitted for clearance by the Director, the penultimate draft should incorporate feedback from the ERG. The final draft will be submitted for copyediting and publishing.

To qualify as an advocate for every child you will have…

Expertise in early childhood development and/or early childhood education. We are seeking individuals with the following key qualifications:

1. Master’s degree in education or social sciences, with a specialization in early childhood development or early childhood education, or a related field (e.g., developmental psychology).
   1. Practitioner as a subject matter expert in early child development or early childhood education, with a minimum of 8 years post-qualification work experience
   2. Technical expertise in high-level and high-quality programme and/or significant experience in conducting evaluations of education or ECD programmes implemented in various contexts, including countries implementing humanitarian programmes, or countries characterized by fragility
   3. Good knowledge of UNICEF programming strategies, field work, procedures and organizational culture (prior UNICEF employment experience should be declared to enable assessment of possible conflicts of interest)
   4. Strong qualifications in quantitative and qualitative data collection methods and analysis
   5. Readiness to travel to the countries selected for primary data collection, and to undertake domestic travel in the country
   7. Fluency in English (oral communication, reading complex documents, writing reports)
   8. Functional proficiency in Arabic, French, and/or Spanish

APPLY NOW