Organization: UNICEF – United Nations Children’s Fund
Location: Remote | Nairobi
Grade: Consultancy – Consultant – Contractors Agreement
Occupational Groups:
Political Affairs
Development Cooperation and Sustainable Development Goals
Democratic Governance
Monitoring and Evaluation
Children’s rights (health and protection)
Project and Programme Management
Closing Date: 2024-09-19
As per UNICEF’s Evaluation Policy, every UNICEF Country Program must be evaluated at least once every two country Program cycles. The Evaluation Section of the respective UNICEF Regional Office conducts such country program evaluations (CPEs). CPEs intend to capture and demonstrate evaluative evidence of UNICEF’s contributions to development results at the country level and the effectiveness of UNICEF’s strategy in facilitating and leveraging national efforts for achieving development results. Their purpose is to: • Assess results from the ongoing Country Program to inform the development of the next UNICEF Country Program Document (CPD) • Strengthen the accountability of UNICEF to national and international stakeholders and the Executive Board The CPE will be managed by the Evaluation Section of UNICEF’s East and Southern Africa Regional Office, under the overall oversight of the ESARO Regional Director, and in close collaboration with the UNICEF Eswatini Country Office (ECO), Government of Eswatini and development partners. Quality assurance will be provided by UNICEF’s Evaluation Office, which reports directly to UNICEF’s Executive Director and is functionally independent within the Organization. The CPE will be conducted in accordance with the provisions of UNICEF’s Evaluation Policy and the norms and standards of the United Nations Development Group (UNEG).
UNICEF works in some of the world’s toughest places, to reach the world’s most disadvantaged children. To save their lives. To defend their rights. To help them fulfill their potential.
Across 190 countries and territories, we work for every child, everywhere, every day, to build a better world for everyone.
And we never give up.
For every child, reimagine the future.
Evaluation Objectives
The overall objectives of the CPE are:
To assess the relevance, effectiveness, efficiency, coherence, and sustainability of the Eswatini Country Program from its inception to the present, with a particular focus on equity, gender equality, disability, and UNICEF’s strategic positioning in relation to its child rights mandate.
To identify and document key successes, challenges, and lessons in implementing the current (2021-2025) Country Program, which can inform and support advocacy efforts for scale-up and replication.
To provide a set of forward-looking and actionable recommendations to strengthen programmatic strategies in the design of the next Eswatini Country Program, taking into consideration national development priorities and plans and the 2030 Agenda for Sustainable Development in the country.
Scope of the Evaluation
The proposed CPE will cover the Eswatini Country Program (ECP) from 2021 to 2025, capturing and demonstrating evaluative evidence of the effectiveness of UNICEF in leveraging national efforts and the organisation’s direct contributions to achieving development results for children at the country level. It is anticipated to begin in late September 2024 with the inception phase and take 60 days over seven months to complete. As a country-level evaluation of UNICEF, the CPE will focus on the formal UNICEF CPD approved by the Executive Board but also consider any changes/revisions from the initial CPD during the period under review. Subject to specific areas of focus identified below, the scope of the CPE includes the entirety of UNICEF’s engagement in the country and, therefore, covers interventions funded by all sources. The CPE will also cover any humanitarian or emergency response supported by UNICEF Eswatini during the period under evaluation.
The CPE should focus particularly on UNICEF’s positioning within the development community and national partners regarding its child rights mandate. It will also examine equity, gender equality, and disability.
Click here to access the Terms of Reference for more details.
Evaluation Criteria and Questions
The CPE will be guided by key evaluation criteria and aim to answer the following evaluation questions:
Relevance: The CPE will seek to assess the extent to which the Country Program’s objectives and design were and remain appropriate within the country context, as well as whether UNICEF’s approach to addressing them was the most suitable considering its mandate, resource base, comparative advantages, and operational structures.
1. To what extent is the Country Program guided by national priorities, clear Program theories, and relevant Program strategies appropriate to the changing context and emerging issues, and responded and adjusted, as necessary?
2. To what extent has UNICEF been able to position itself as a strategic partner in the country context? What are UNICEF’s comparative strengths in the country – particularly in comparison to other UN agencies and development partners – and how were these harnessed to help achieve the results?
Coherence: The CPE will assess policy consistency with key UNICEF strategies and international commitments, including gender equality and women’s empowerment, disability inclusiveness, equity for children, and the human rights-based approach; and UNICEF’s coordination and convening role within the UN, with government sectors and donors in Eswatini.
3. To what extent have ECP strategies to address gender equality, disability, and equity, particularly the alignment of the ECP with the Core Commitments for Children in Humanitarian Action, UNICEF’s Gender Action Plans (2014-2017 and 2018–2021) and United Nations Disability Inclusion Strategy (UNDIS), been consistently integrated into all aspects of programming and implementation, including policy and advocacy?
4. Did the Country Office’s strategic approach to addressing the challenges of equity, gender equality, and disability play a complementary role to that of the Government and other development actors?
5. To what extent is the Country Program linked to and achieving synergies and coordination with other UN agencies, particularly in response to emergencies such as COVID-19?
Effectiveness: The CPE will assess the extent to which the Country Program results were achieved and whether UNICEF’s adopted strategies were gender responsive/transformative, disability-inclusive, and demonstrated a reasonable contribution at the outcome level, including any differential results across groups.
6. To what extent has the Country Program achieved or is likely to achieve its outcomes, including any differential results across gender, disability status, income, ethnicities, etc.? What results have been achieved through convergence, and what are the other major factors influencing the achievement (or not) of Country Program outcomes?
7. Did the country Program contribute to reducing inequities and exclusion and progressing towards greater gender equality and disability inclusion? To what extent are programs, communications, and advocacy efforts gender-responsive/transformative and disability-inclusive, and, relatedly, are UNICEF ECO staff capacitated to integrate and implement gender-responsive/transformative and disability-inclusive programs?
Efficiency: The CPE will measure how resources/inputs (funds, expertise, time, etc.) were converted into and affected results. It is also understood as how UNICEF manages its partnerships, operationalizes its strategies, implements activities, and delivers outputs.
8. Were resources (funds, human resources, time, expertise, etc.) allocated and utilized strategically to track and achieve results, including equity, gender, and disability-related objectives?
9. To what extent have the convergence strategy, the Program structure, and the office structure supported the delivery of the Country Program? Were the chosen strategies and approaches the most cost-effective and efficient? Were there alternatives that would have worked better, and what are those?
Sustainability: The CPE will assess the extent to which continuation of benefits from Country Program interventions was ensured, including the likelihood of and risks to continued long-term benefits and its potential for scale-up and/or replication.
10. To what extent are the positive changes and effects of the Country Program sustainable at the relevant levels (e.g., community, provincial/state, national)? To what extent have the Program strategies adopted by UNICEF contributed to or were designed in a way that will contribute to the sustainability of results, especially equity, gender, and disability-related results?
11. To what extent have the Program strategies, plans, and tools, particularly those with an equity, gender, and disability focus, been institutionalized in systems, policies, mechanisms, and strategies among government, NGO/civil society, and other partners and stakeholders? Will the strategies/plans/tools be more widely replicated or adapted? What’s the scalability of models introduced by UNICEF?
To answer these overarching questions, the evaluator will be expected to develop sub-questions as part of the evaluation matrix during the inception phase to further focus the evaluation, not expand its scope. These questions will be reviewed with all stakeholders during the inception period.
Evaluation Approach and Methods
The evaluation methodology will adhere to the United Nations Evaluation Group (UNEG) Norms & Standards[1] and will be further refined by the external evaluators during the inception phase, in close consultation with the ESARO Evaluation Section and key evaluation stakeholders. The evaluation design will emphasize the analysis of UNICEF’s contributions to outcome-level and successes in influencing upstream changes, such as policy shifts, system strengthening, and high-level decision-making processes that contribute to the broader goals of the Country Program. The design should specify how data collection and analysis methods integrate gender and disability considerations throughout the evaluation process.
Theory of Change (ToC) Approach
Click here to access the Terms of Reference for more details.
Where possible and appropriate, the evaluation should seek evidence of what may or may not have occurred in the absence of UNICEF’s Program.
As part of the inception phase, the evaluator will assess the available information, identify data constraints, and determine the data collection needs and methods. The methodology should be aware of and prepared to take advantage of the accumulated and in-process evidence generated through research, studies, surveys, and evaluations conducted within the UNICEF Eswatini Country Program.
With respect to indicators, the CPD results framework is updated annually during the annual reporting to show progress toward outputs. A Country Office Annual Report (COAR) is produced annually, covering all the Program components and implementation strategies and their status. The evaluator will be provided with other relevant programmatic surveys and studies.
The following secondary data will be reviewed, among others: background documents on the national context, documents prepared by international partners during the period under review and documents prepared by UN system agencies; Program plans and frameworks; progress reports; monitoring self-assessments such as the UNICEF Country Office Annual Reports; national surveys (e.g. Multiple Indicator Cluster Surveys (MICS); national reports; and evaluations conducted by the country office and partners. Sex-disaggregated data will be collected, where available, and assessed against Program outcomes.
Data collection methods
The evaluation will use data from primary and secondary sources, including a desk review of documentation and information and interviews with key informants. It will follow a multi-stakeholder approach, and interviews will include Government representatives, civil society organizations, private-sector representatives, UN agencies, multilateral organizations, bilateral donors, and rights holders under the Program. Focus group discussions may be used to consult different groups of rights holders and duty bearers as appropriate.
Click here to access the Terms of Reference for more details.
Management Arrangements and Quality Assurance
The evaluator/s will be recruited by and report to the ESARO Evaluation Section under the overall oversight of the ESARO Regional Director. For the day-to-day management of the CPE, the Evaluation Section will appoint an Evaluation Manager who will be accountable to the Regional Evaluation Adviser. UNICEF’s Evaluation Office will provide quality assurance.
Eswatini Country Office will appoint an Evaluation Focal Point who will act as the primary liaison with the Evaluation Manager and will facilitate the data collection and evaluation process at the country level.
The Evaluation Manager will work with the Eswatini Country Office to constitute an Evaluation Reference Group (ERG) comprising critical stakeholders of the CPE, including Government counterparts; several senior Eswatini Country Office and ESARO staff members; select development, civil society organizations, and private sector partners. The ERG’s responsibilities are described in the ERG terms of reference. The ERG has an advisory capacity whose primary role is to review evaluation milestones (terms of reference, inception report, draft evaluation report) and to provide comments. The ERG Secretariat will maintain a written record, as part of an audit trail, of all ERG comments, which the evaluator is expected to respond to in writing (agree – actions taken; disagree – justification).
Specific Tasks, Deliverables, and Timeline
The CPE is anticipated to begin in September 2024 with the inception phase and be completed in 60 days over seven months.
Timeline
Activity
Deliverable
Other elements to be aware of or link to
5 days
Preparatory phase
Secondary data collection and desk review
Preliminary stakeholder analysis
Preparation for the inception phase
1. Plan for the inception phase
Recipients: members of the evaluation reference group
An evaluation Reference Group is formed.
UNICEF and other stakeholders are informed to secure cooperation for the effort.
UNICEF assembles documentation and data for use by the evaluator.
15 days
Inception phase
May include an inception mission to Mbabane, Eswatini
(Re)Construction of Theory of Change
Stakeholder Analysis
Preparation of draft inception report and data collection tools
Engagement with stakeholders on the inception report
Ethical approval process
Finalization of inception report
2. Draft inception report
Recipients: members of the evaluation reference group
3. The draft inception report and instruments will be presented to the Evaluation Reference Group in person or via video link.
4. Final inception report (plus completed audit trail addressing all comments)
Recipients: members of the evaluation reference group
20 days
Data collection phase
Preparation for data collection, including piloting of instruments
Mission in Eswatini to collect data and meet with stakeholders
Preparation of interview reports
Population of evaluation matrix
Preparation and delivery of Country Mission Debrief
5. Country Mission Debrief with key ECO staff at the end of the in-country mission
20 days
Drafting, validation, and completion phase
Data analysis and drafting
Preparation of a PowerPoint presentation on emerging findings, conclusions, and recommendations
Engagement with stakeholders on the draft report
Finalization of report and summary PowerPoint presentation
6. Presentation of Preliminary Findings – in person or via video link – on emerging findings, conclusions, and recommendations with key evaluation stakeholders, including the Evaluation Reference Group.
Recipients: members of the evaluation reference group
7. A complete first draft evaluation report
Recipients: members of the evaluation reference group
8. Presentation of the findings, conclusions, and recommendations at the ECO Strategic Moment of Reflection;
9. A final evaluation report (plus a completed audit trail addressing all comments). The final report should be illustrated with data and infographics. Detailed recommendations on each theme should be presented in a separate concluding chapter. Equity, gender, and disability should also be included as cross-cutting themes throughout the findings.
Recipients: members of the evaluation reference group
10. Final PowerPoint presentation that summarizes the evaluation findings.
Recipients: members of the evaluation reference group
Presentations of key findings and recommendations need to be ready for the UNICEF Eswatini Strategic Moment of Reflection (preparation of the next CP) and other identified events in Q1 of 2025.
The report structure, format, and quality should adhere to the UNICEF Evaluation Report standards and the GEROS Quality Assessment System.
Dissemination and use strategy commences as deliverables are received.
60 days
TOTAL
Desired competencies, technical background, and experience
The Country Program Evaluation (CPE) is envisioned to be completed by one experienced individual. However, if the consultant proposes a team, the number of team members shall be determined by the consultant based on the requirements for the successful completion of the assignment. Each team member’s involvement level must be specified in the proposal. UNICEF will not be involved in the contractual and financial agreements between the team leader and team members.
Minimum Requirements
The consultant must meet the following minimum requirements:
Education: Advanced university degree in disciplines relevant to evaluation (e.g., social policy, economics, demography, anthropology, public health).
Experience: At least ten years of experience conducting strategic program evaluations or evaluative reviews, with a proven track record of evaluating large multisectoral and multi-stakeholder country programs supported by UN or UNICEF. Experience in conducting country program/portfolio evaluations is preferred.
Expertise: Demonstrated expertise in evaluating institutional support systems, including operations and human resource functions.
Knowledge: Familiarity with programming theories and strategies employed in each program outcome component.
Regional Experience: Work record in a Middle-Income Country in Africa.
Language Skills: Excellent command of English, with proven ability to prepare high-quality reports.
Analytical Skills: Strong quantitative and qualitative analytical skills.
Capacity: Demonstrated ability to carry out the CPE and complete deliverables.
Significant Advantages
Communication Skills: Proven ability to develop attractive evidence products that present complex information through infographics and other means of communication.
Contextual Knowledge: Knowledge of the social, economic, and political context of Eswatini.
Evaluation Reports: Record of top-ranked evaluation reports by GEROS.
Administrative issues
Relevant administrative arrangements include the following:
The consultancy will be conducted remotely, with a preference for candidates based in the Eastern and Southern Africa region.
The consultant will be required to travel to Eswatini for data collection.
The consultant must participate in weekly 30-minute progress meetings with UNICEF throughout the duration of the contract.
How to apply
Qualified candidates are requested to submit a cover letter, CV, or P11 form, samples of similar work, and their technical proposals to the online recruitment portal (Talent Management System).
Interested candidates are to indicate their ability, availability, and rate (daily/monthly) expressed in US$ to undertake the terms of reference. The fees should include all other costs incurred, such as travel, VISA, and subsistence allowances.
Please click here to access the Terms of Reference.
Applications submitted without a fee/ rate will not be considered.
[1] http://www.unevaluation.org/document/detail/1914
For every Child, you demonstrate…
To view our competency framework, please visit here.
UNICEF is here to serve the world’s most disadvantaged children and our global workforce must reflect the diversity of those children. The UNICEF family is committed to include everyone, irrespective of their race/ethnicity, age, disability, gender identity, sexual orientation, religion, nationality, socio-economic background, or any other personal characteristic.
UNICEF offers reasonable accommodation for consultants/individual contractors with disabilities. This may include, for example, accessible software, travel assistance for missions or personal attendants. We encourage you to disclose your disability during your application in case you need reasonable accommodation during the selection process and afterwards in your assignment.
UNICEF has a zero-tolerance policy on conduct that is incompatible with the aims and objectives of the United Nations and UNICEF, including sexual exploitation and abuse, sexual harassment, abuse of authority and discrimination. UNICEF also adheres to strict child safeguarding principles. All selected candidates will be expected to adhere to these standards and principles and will therefore undergo rigorous reference and background checks. Background checks will include the verification of academic credential(s) and employment history. Selected candidates may be required to provide additional information to conduct a background check.
Remarks:
Only shortlisted candidates will be contacted and advance to the next stage of the selection process.