TE Aproach and Methodology:
The TE report must provide evidence-based information that is credible, reliable and useful. The TE team will review all relevant sources of information including documents prepared during the preparation phase (i.e. PIF, UNDP Initiation Plan, UNDP Social and Environmental Screening Procedure/SESP) the Project Document, project reports including annual PIRs, project budget revisions, lessons learned reports, national strategic and legal documents, and any other materials that the team considers useful for this evidence-based evaluation. The TE team will review the baseline and midterm GEF focal area Core Indicators/Tracking Tools submitted to the GEF at the CEO endorsement and midterm stages and the terminal Core Indicators/Tracking Tools that must be completed before the TE field missionor online interviews with relevant stakeholders and counterparts begin.
The TE team is expected to follow a participatory and consultative approach ensuring close engagement with the Project Team, government counterparts (the GEF Operational Focal Point), Implementing Partners, the UNDP Country Office(s), the Chief Technical Advisor (CTA), the Regional Technical Advisor (RTA), direct beneficiaries and other stakeholders. Engagement of stakeholders is vital to a successful TE. For this reason, in case if travel restrictions due to COVID-19 pandemic are eased, it is absolutely essential that shortly after the start of the assignment the international consultant travels to Turkmenistan for a period of 2 weeks (10 working days, not including weekends) to meet with all relevant stakeholders. Stakeholder involvement should include interviews with stakeholders who have project responsibilities, including but not limited to (State Committee of Water Economy of Turkmenistan (SCWE), Ministry of Agriculture and Environmental Protection of Turkmenistan (MAEP), State Agricultural University, Municipality of Ahal region, Municipality of Kaahka district and Municipality of Geokdepe district); executing agencies, senior officials and task team/component leaders, key experts and consultants in the subject area, Project Board, project beneficiaries, academia, local government, etc. Additionally, the TE team is expected to conduct field missions to (Kaahka and Geokdepe project sites), including the following project sites (Kaahka water pipeline and Green Polygon) as the travel conditions due to COVID-10 permit.
The specific design and methodology for the TE should emerge from consultations between the TE team and the above-mentioned parties regarding what is appropriate and feasible for meeting the TE purpose and objectives and answering the evaluation questions, given limitations of budget, time and data. The TE team, however, must use gender-responsive methodologies and tools and ensure that gender equality and women’s empowerment, as well as other cross-cutting issues and SDGs are incorporated into the TE report.
The final methodological approach including interview schedule, field visits and data to be used in the evaluation must be clearly outlined in the TE Inception Report and be fully discussed and agreed between UNDP, stakeholders and the TE team.
The final TE report must describe the full TE approach taken and the rationale for the approach making explicit the underlying assumptions, challenges, strengths and weaknesses about the methods and approach of the evaluation.
Detailed Scope of the TE:
The TE will assess project performance against expectations set out in the project’s Logical Framework/Results Framework (see ToR Annex A). The TE will assess results according to the criteria outlined in the Guidance for TEs of UNDP-supported GEF-financed Projects http://web.undp.org/evaluation/guideline/documents/GEF/TE_GuidanceforUNDP-supportedGEFfinancedProjects.pdf
The Findings section of the TE report will cover the topics listed below. A full outline of the TE report’s content is provided in ToR Annex C. The asterisk “(*)” indicates criteria for which a rating is required.
Findings:
i. Project Design/Formulation
- National priorities and country drivenness;
- Project Strategy and objectives;
- Gender equality and women’s empowerment;
- Social and Environmental Safeguards;
- Analysis of Results Framework: project logic and strategy, indicators;
- Assumptions and Risks;
- Lessons from other relevant projects (e.g. same focal area) incorporated into project design;
- Planned stakeholder participation;
- Linkages between project and other interventions within the sector;
- Management arrangements;
ii. Project Implementation:
- Adaptive management (changes to the project design and project outputs during implementation);
- Actual stakeholder participation and partnership arrangements;
- Project Finance and Co-finance;
- Monitoring & Evaluation: design at entry (*), implementation (*), and overall assessment of M&E (*);
- Implementing Agency (UNDP) (*) and Executing Agency (*), overall project oversight/implementation and execution (*);
- Risk Management, including Social and Environmental Standards;
iii. Project Results:
- Assess the achievement of outcomes against indicators by reporting on the level of progress for each objective and outcome indicator at the time of the TE and noting final achievements;
- Relevance (*), Effectiveness (*), Efficiency (*) and overall project outcome (*);
- Sustainability: financial (*), socio-political (*), institutional framework and governance (*), environmental (*), overall likelihood of sustainability (*);
- Country ownership;
- Gender equality and women’s empowerment;
- Cross-cutting issues (improved governance, climate change mitigation and adaptation, capacity development, knowledge management, etc., as relevant);
- GEF Additionality;
- Catalytic Role / Replication Effect;
- Progress to impact;
iv. Main Findings, Conclusions, Recommendations and Lessons Learned:
- The TE team will include a summary of the main findings of the TE report. Findings should be presented as statements of fact that are based on analysis of the data;
- The section on conclusions will be written in light of the findings. Conclusions should be comprehensive and balanced statements that are well substantiated by evidence and logically connected to the TE findings. They should highlight the strengths, weaknesses and results of the project, respond to key evaluation questions and provide insights into the identification of and/or solutions to important problems or issues pertinent to project beneficiaries, UNDP and the GEF, including issues in relation to gender equality and women’s empowerment;
- Recommendations should provide concrete, practical, feasible and targeted recommendations directed to the intended users of the evaluation about what actions to take and decisions to make;
- The recommendations should be specifically supported by the evidence and linked to the findings and conclusions around key questions addressed by the evaluation;
- The TE report should also include lessons that can be taken from the evaluation, including best practices in addressing issues relating to relevance, performance and success that can provide knowledge gained from the particular circumstance (programmatic and evaluation methods used, partnerships, financial leveraging, etc.) that are applicable to other GEF and UNDP interventions. When possible, the TE team should include examples of good practices in project design and implementation.
- It is important for the conclusions, recommendations and lessons learned of the TE report to include results related to gender equality and empowerment of women;
The TE report will include an Evaluation Ratings Table, as shown in the ToR Annex F.
Expected Outputs and Deliverables:
The TE Consultant shall prepare and submit:
- TE Inception Report: TE team clarifies objectives and methods of the TE no later than 2 weeks before the TE mission or online interviews with relevant stakeholders. TE team submits the Inception Report to the Commissioning Unit and project management. Approximate due date: (19/10/2021);
- Presentation: TE team presents initial findings to project management and the Commissioning Unit at the end of the TE mission online interviews with relevant stakeholders and counterparts. Approximate due date: (09/11/2021);
- Draft TE Report: TE team submits full draft report with annexes within 3 weeks of the end of the TE mission or online interviews with relevant stakeholders. Approximate due date: (01/12/2021);
- Final TE Report* and Audit Trail: TE team submits revised report, with Audit Trail detailing how all received comments have (and have not) been addressed in the final TE report, to the Commissioning Unit within 1 week of receiving UNDP comments on draft. Approximate due date: (10/12/2021).
*The final TE report must be in English. If applicable, the Commissioning Unit may choose to arrange for a translation of the report into a language more widely shared by national stakeholders. All final TE reports will be quality assessed by the UNDP Independent Evaluation Office (IEO). Details of the IEO’s quality assessment of decentralized evaluations can be found in Section 6 of the UNDP Evaluation Guidelines. http://web.undp.org/evaluation/guideline/.
TE Arrangements:
The principal responsibility for managing the TE resides with the Commissioning Unit. The Commissioning Unit for this project’s TE is the UNDP Country Office in Turkmenistan. The Commissioning Unit will contract the evaluators and ensure the timely provision of per diems and travel arrangements within the country (if applicable) for the TE team. The Project Team will be responsible for liaising with the TE team to provide all relevant documents, set up stakeholder interviews, and arrange field visits.
Duration of the Work:
The total duration of the TE will be approximately 26 working days over a time period of 9 weeks starting on 10/10/2021. The tentative TE timeframe is as follows:
- 20/08/2021: Application closes;
- 05/10/2021: Selection of TE Team;
- 10/10/2021: Preparation of the TE team (handover of project documents);
- 10/10/2021: 4 days: Document review and preparing TE Inception Report;
- 19/10/2021: 1 day: Finalization and Validation of TE Inception Report- latest start of TE mission;
- 28/10/2021: 10 days: TE mission or online consultations: stakeholder meetings, interviews, field visits or online interviews with relevant stakeholders and counterparts;
- 09/11/2021: 1 day: Mission wrap-up meeting & presentation of initial findings- earliest end of TE mission;
- 15/11/2021: 8 days: Preparation of draft TE report;
- 01/12/2021: 1 day: Circulation of draft TE report for comments;
- 10/12/2021: 1 day: Incorporation of comments on draft TE report into Audit Trail & finalization of TE report;
- 15/12/2021: Preparation & Issue of Management Response;
- 06/01/2022: Expected date of full TE completion.
The expected date start date of contract is 10/10/2021.
Duty Station:
The TE assignment is expected to be home-based in case if current travel restrictions due to COVID-19 pandemic are not eased. However, in case if travel restrictions are eased it is absolutely essential that shortly after the start of the assignment the international consultant travels to Turkmenistan for a period of 2 weeks (10 working days, not including weekends) to meet with all relevant stakeholders and conducts field missions to Kaahka and Geokdepe project sites.
Travel:
- International travel might be required to Turkmenistan during the TE mission;
- The BSAFE course must be successfully completed prior to commencement of travel;
- Individual Consultants are responsible for ensuring they have vaccinations/inoculations when travelling to certain countries, as designated by the UN Medical Director;
- Consultants are required to comply with the UN security directives set forth under: https://dss.un.org/dssweb/ ;
- All related travel expenses should be included to the Letter of Confirmation of Interest and Availability.
Evaluator Ethics:
The TE team will be held to the highest ethical standards and is required to sign a code of conduct upon acceptance of the assignment. This evaluation will be conducted in accordance with the principles outlined in the UNEG ‘Ethical Guidelines for Evaluation’. The evaluator must safeguard the rights and confidentiality of information providers, interviewees and stakeholders through measures to ensure compliance with legal and other relevant codes governing collection of data and reporting on data. The evaluator must also ensure security of collected information before and after the evaluation and protocols to ensure anonymity and confidentiality of sources of information where that is expected. The information knowledge and data gathered in the evaluation process must also be solely used for the evaluation and not for other uses without the express authorization of UNDP and partners.
Payment Schedule:
- 20% payment upon satisfactory delivery of the final TE Inception Report and approval by the Commissioning Unit;
- 40% payment upon satisfactory delivery of the draft TE report to the Commissioning Unit;
- 40% payment upon satisfactory delivery of the final TE report and approval by the Commissioning Unit and RTA (via signatures on the TE Report Clearance Form) and delivery of completed TE Audit Trail;
Criteria for issuing the final payment of 40%:
- The final TE report includes all requirements outlined in the TE TOR and is in accordance with the TE guidance;
- The final TE report is clearly written, logically organized, and is specific for this project (i.e. text has not been cut & pasted from other TE reports);
- The Audit Trail includes responses to and justification for each comment listed.