UN WOMEN Jobs – 109826

Evaluation objectives, criteria, and key questions

The objective of this assignment is to conduct an end-term independent evaluation of the programme to assess its effectiveness, build evidence of results, and highlight the strategies that have contributed to, or hindered, their achievement in one of the three implementing countries: Senegal, and specifically in the Saint Louis region. In addition, the evaluation will assess the project’s contribution to increased national capacity, ownership and transparency.

The evaluation will provide highly relevant recommendations and lessons learned for future and ongoing programmes on unpaid care work.

Within the broader and country specific context, the evaluation is expected to:

* Assess effectiveness and a potential measurable impact of the Program intervention on the target group on the recognition, reduction and redistribution of unpaid care work in Senegal.

*   Identify and document lessons learned, best practices, success stories and document and analyze challenges and possible weaknesses to inform future work of UN Women in the area of unpaid care work in Senegal.

* Analyze the relevance of the Program objectives, strategy and approach at the local and national levels in Senegal.

* Analyse and assess the strategies in place and contribute to identify additional strategies for replication and up-scaling of the programme’s best practices in Senegal.

* Identify, assess, and validate innovation in all aspects of the program in Senegal.

Criteria to be used for this evaluation includes OECD/DAC evaluation criteria[1]: relevance, coherence, effectiveness, efficiency, sustainability. Human Rights, Gender Equality and Disability Inclusion will be considered as an additional criterion. In line with the quest to incorporate human rights, gender equality and disability inclusion in all UN work, and acknowledging UN Women’s Evaluation Policy, which promotes the integration of women’s rights and gender equality principles into evaluation, these dimensions will require special attention for this evaluation and will be considered under each evaluation criterion.

A number of key evaluation questions are reported below for each of the criteria.

Relevance and Design

• Do the project expected results address the needs of the target groups and how? 
• Are the activities, outputs and outcomes of the project consistent with the overall global, regional and national gender priorities?
• To what extent to the local population, beneficiaries and external observers perceive the intervention as relevant?
• To what extent has the project been catalytic in addressing some of the root causes of unpaid care work and gender inequalities? To what extend have the proposed approaches been transformative and able to contribute to the shifting of structural inequalities that drive care inequalities?

Effectiveness

• What has been the progress made towards achievement of the two expected outcomes? What specific results were achieved, both positive and negative? To what extent are beneficiaries satisfied with the results?
• What major factors contributed to the achievement or non-achievement of expected programme objectives?
• Have the programme’s organizational structures, managerial support and coordination mechanisms (such as the Steering Committees) effectively supported the delivery of the programme?  What are the recommendations for improvement?
• To what extent are the programme approaches and strategies innovative for recognizing, reducing and redistributing unpaid care work?  What -if any- types of innovative good practices have been introduced in the programme to achieve results in this field?
• What impact has been registered on women from the use of time saving climate resilient technologies/equipment provided by the programme (in terms of wellbeing, women’s economic autonomy and time poverty)? What else women are now doing with their time (i.e. are they doing more paid work, study, community work, rest/leisure, or different tasks), and are men taking on a greater share of the responsibility as a result?

Efficiency

• Have resources (financial, human, technical support, etc.) been allocated strategically to achieve the programme outcomes?
• Are the programme and its components cost-effective? Could the activities and outputs have been delivered with fewer resources or within a reduced timeframe, without reducing their quality and quantity?
• To what extent can the partnership between UN Women and Implementing partners be regarded as efficient?
• Has the programme build synergies with previous and/or ongoing UN Women interventions?

Sustainability

• Is the programme supported by national/local institutions? Do these institutions, including Government and Civil Society, beneficiaries, other implementing partners demonstrate leadership commitment and technical capacity to continue to work with the programme or replicate it to ensure continuity in pursuing the recognition, reduction and redistribution of unpaid care work after the programme ends?
• What capacity of national partners, both technical and operational, has been strengthened?
• Did the intervention design include an appropriate sustainability strategy (including promoting national/ local ownership, use of local capacity, etc.) to support positive changes in Gender Equality and Human Rights after the end of the intervention?
• Is there evidence of sustainable transformations at the local and national level that can be associated with the intervention?

Stakeholder engagement

In line with UN Women evaluation standards, this exercise will be gender sensitive, consultative, inclusive and participatory and will ensure the participation of women and CSOs representing various groups of women in the target municipalities of Senegal. Special attention will be given to representativeness of all target groups of women including rural women, women and girls with disabilities, single parents, illiterate women, ethnic minority women and women with other identities, as applicable. A diverse group of women who have participated in the programme, and other community members who were direct and indirect programme beneficiaries, will benefit from this evaluation findings and should be included as the key actors in the design of the main recommendations.

Similarly, direct implementing partners will be involved in the evaluation process as key informants and sources of information.

Scope of Evaluation

The evaluation will be conducted by an evaluation team composed by a Senior International Evaluation Consultant who will lead the evaluation process and collect data and information in Rwanda and South Africa, and a National Consultant who will collect data and information in Senegal and share the findings with the Senior Evaluator.

The evaluation will cover the whole duration of the project, between 1st April 2021 and 31st December 2022. The geographic scope of this evaluation is represented by the three implementing countries (Rwanda, Senegal, South Africa). The National Consultant is expected to conduct the evaluation in the target municipalities in the Saint Louis Region in Senegal.

 Evaluation process and methodology                                                             

The external end-term evaluation will be conducted in accordance with United Nations Evaluation Group (UNEG) standards, and more specifically, UN Women’s GERAAS (see Annex 1) will be used to assess and ensure the quality of evaluation products. All deliverables will be reviewed against the GERAAS criteria by the Evaluation Management Group for approval.

This includes subjecting the Evaluation Report to UN System-wide Action Plan on Gender Equality and the Empowerment of Women (UN SWAP) quality scoring, requiring that evidence of gender integration is demonstrated throughout the evaluation process and in the report. The evaluation methodology should clearly focus on highlighting gender issues in the implementation of the programme, in line with the following criteria:

–          GEWE is integrated in the evaluation scope of analysis and evaluation indicators are designed in a way that ensures GEWE related data will be collected.

–          GEWE is integrated in evaluation criteria and evaluation questions are included that specifically address how GEWE has been integrated into the design, planning, implementation of the intervention and the results achieved;

–          A gender-responsive methodology, methods and tools, and data analysis techniques are selected.

–          Evaluation findings, conclusions and recommendations reflect a gender analysis.

The design should promote inclusion and participation by employing gender equality and human rights responsive methodological approaches such as those with a focus on utilization,  empowerment  or feminist approaches.

Materials that the evaluator will be expected to consult are: programme documents and tools at global and country level, country offices’ records (meetings, monitoring visits), documents related to the Steering Committees, etc. Data sources that the evaluator will be expected to consult are: UN Women teams, beneficiaries, partner organizations, implementing partners, governmental partners, etc.

The evaluator will have to detail a data management plan on how confidentiality will be guaranteed, that the nature of informed consent must be included in the inception report, that the evaluator develops a sampling frame and to take measure to ensure data quality, reliability and validity of data.

The evaluation process is divided in six phases:

1. Inception phase
2. Remote data collection phase
3. Data analyses and syntheses phase
4. Validation
5. Dissemination and Management Response

A full methodology will be developed by the consultant during the inception phase and included in the inception report. This will include the identification of a variety of key informants to consult, and specifically outline the instruments and tools (interviews, observations, focus groups, literary journal, survey, site visits, etc.) that will be used to collect relevant information and data. The list of sources of information collected will be then attached to the evaluation report

Under the coordination and supervision of UN Women, in close collaboration with the Evaluation Reference Group, the Junior Consultant will conduct the evaluation of the part of the 3R Programme implemented in Senegal under the supervision of a Senior Consultant who will lead the final evaluation process.

The Junior Consultant is expected to perform the following activities as part of conducting the independent evaluation:

Review relevant documents related to Senegal and consult with the Senegal coordinator.
Support senior consultant with inception report drafting.
Carry out field visits and interviews in Senegal based on an interview protocol reviewed by the programme teams (individual and focus groups, as relevant) with key stakeholders, beneficiaries, implementing partners and members of the programme management teams.
Conduct a comprehensive analysis of data collected in Senegal and share with the senior consultant.
Support senior consultant with online debriefing session.
Support senior consultant with report drafting.
Support report finalization.

 Deliverables

The following deliverables[2] are expected:

An inception report to be provided. The inception report should capture relevant information such as proposed methods; proposed sources of data; data collection procedures and tools, including an interview protocol, which will be reviewed and approved by the programme team. The inception report should also include an evaluation matrix, proposed schedule of tasks, activities and deliverables and should also contain background information. (payment 40%)
A PPT presentation with interim findings.
An interim evaluation report
A final evaluation report. The final report should address comments from the Evaluation Reference Group. The final version, accepted as such following final approval by the Evaluation Management Group, will be delivered latest on the 12th week of the assignment (payment 60%).

Both the interim and the final report will be structured as follows:

I. Table of Contents
II. List of abbreviations and acronyms
III. Executive summary
IV. Background and context
V. Evaluation purpose
VI. Evaluation objectives and scope
VII. Evaluation methodology and limitations
VIII. Evaluation findings

      a. Relevance and Design
      b. Efficiency
      c. Effectiveness
      d. Sustainability
      e. Gender, Equity and Human Rights

XI. Conclusions
X. Recommendations
XI. Lessons learned
XII. Annexes

     a. Terms of Reference
     b. Documents consulted
     c. List of institutions interviewed, and sites visited
     d. Evaluation tools (questionnaires, interview guides, etc.)
     e. Summary matrix of findings, evidence, and recommendations
     f. Evaluation brief

N.B.: All payments are subject to the submission of deliverables and following the Evaluation Management Group’s approval of satisfactory performance. Satisfactory performance will be assessed based on GERAAS quality assessment standards (see annex 1 for more

information)Suggested agenda (subject to possible evolutions):

Evaluation Management, reporting and supervision

The Evaluation Management Group:

The Management Group, under UN Women’s leadership, is in charge of the management of the evaluation. With this regard, it organises the consultant recruitment process, and shares the necessary information with the Evaluator. It will support the Evaluator with the needed technical assistance to ensure an in-depth understanding of the project under evaluation. It quality assures the different deliverables and takes key decisions concerning the evaluation, including approving deliverables.  

The Evaluation Reference Group :

The Reference group, which comprises a selection of project stakeholders, strengthens the independent character of the evaluation and bring technical contributions at all stages of the evaluation, from its planning, to the submission of the final report. It contributes to identify any factual errors or errors of omission or interpretation at key stages of the evaluation process.

 Duration of the assignment

The assignment must be conducted during 20 working days within the span of maximum three months.

[1] OECD/DAC Evaluation criteria available at: http://www.oecd.org/dac/evaluation/revised-evaluation-criteria-dec-2019.pdf

[2] All deliverables, including the final evaluation report, are considered as ‘final’ following approval by the Evaluation Management Group. 

Source