UNDP and ILO jointly implement the ADA funded sub-regional project Promoting Inclusive Labour Market Solutions in the Western Balkans – Phase II. The project aims to strengthen capacities of national institutions and local service providers to design and implement inclusive policies and programmes for individuals at risk of social and labour market exclusion. To achieve these objectives, the project envisages a set of technical assistance and capacity development activities that assist labour market intermediaries in introducing innovative practices of service provision by public employment offices (PES) and centres for social work (CSW). Moreover, the project fosters peer-learning and in-country and regional dialogue on the topic of inclusive employment. In this respect, the project forges partnership with private sector and civil society organizations. This three-and-a-half-year project (December 2018 – June 2022) is being implemented in all six Western Balkan countries/territory i.e., Albania, Bosnia and Herzegovina, Kosovo*, Montenegro, North Macedonia and Serbia. The Total value of the project is 2,268,109.20 EUR (Annex 5).
The project builds on the results of the Promoting Inclusive Labour Market Solutions in the Western Balkans – Phase I project, also implemented jointly by UNDP and ILO in the period 1 May 2016 – 31 December 2018 and it also incorporates the recommendations provided in the final project evaluation.
The project’s theory of change (Annex 4) is built on four major pillars:
- Collaborative service provision coordinated by central PES and CSW and implemented by local branches is a method to tackle the multiple forms of exclusion of those who are most distant from the labour market,
- Multi-stakeholder territorial employment partnerships, which mobilize private sector, local governments, academia are best positioned to identify the potentials of local economy and to identify the specific vulnerabilities as well as to co-design suitable solutions and financing models for accelerated jobs creation and employment.
- Mutual learning and the availability of an inclusiveness facility are effective methods of convergence, as well as design and replication of effective solutions for more inclusive service delivery and practices among public employment services.
- Mutual learning and the availability of an inclusiveness facility prepare the target countries/territory to adopt the same institutional set up that will be subsumed under the OMC on employment and social inclusion policies of the EU.
Under this structure, the project consists of four components (outputs):
Output 1: PES and CSWs have institutionalized models of integrated and user-centred provision of services for marginalized groups; (UNDP)
Output 2: A mutual learning programme among public employment services (modelled on the EU’s MLP and focused on inclusiveness approaches) is delivered; (ILO)
Output 3: Local multi-stakeholder partnerships have capacities and resources to develop and implement inclusive employment initiatives. (UNDP)
Output 4: An Inclusiveness Facility for public employment agencies to implement inclusiveness adjustments to their practices (services and programmes) is established. (ILO)
As a result of these outputs, this project expects to:
- Increase in capacity for local and national stakeholders to provide customised services for disadvantaged people.
- Increase in capacity for local stakeholders to engage in partnerships aiming to enhance vulnerable groups’ access to formal employment opportunities.
- Institutionalization of pilot innovative models of partnerships for inclusive employment that involve public and business sector and the civil society
- An intensification of the mutual learning opportunities on inclusiveness for practitioners of the public employment services and ministries in charge of employment, as a sustainable modality to improve policies.
- Implementation of a number of short- and medium-term changes and adjustments in the practices of public employment services with respect to making them more inclusive.
Following the outbreak of COVID-19 pandemic, both UNDP and ILO ought to adjust the focus and the modalities of implementation of project activities. While the overall focus of the project has been kept a new logframe and targets have been developed and approved by ADA.
Evaluation Scope:
This final project evaluation of Promoting Inclusive Labour Market Solutions in the Western Balkans – Phase II (January 2019 – June 2022) project will be conducted at the end of project implementation and will cover the entire duration of the project from 1 December 2018 until 30 June 2022.
The evaluation is scheduled between May 2022 and June 2022.
The evaluation includes a data collection, with a field mission to two selected countries out of six that have been involved in project implementation. The evaluation shall cover all aspects of the project including inter relations between the various project components but it will focus on in depth analysis of two out of four project components (outputs) – Output 1: PES and CSWs have institutionalized models of integrated and user-centred provision of services for marginalized groups, implemented by UNDP and Output 4: An Inclusiveness Facility for public employment agencies to implement inclusiveness adjustments to their practices (services and programmes) is established, implemented by ILO.
Evaluation Purpose:
The main purpose of the evaluation is to assess the programmatic progress and performance of the above-described intervention from the point of view of relevance, effectiveness, and sustainability.
The evaluation will assess the project performance and address the following questions with the results and evidence that is available to date.
The findings of the evaluation will contribute to effective programming, organizational learning and accountability. The findings of the evaluation will moreover be used to engage policy makers and other stakeholders at national and local levels in evidence-based dialogues and to advocate for inclusive employment strategies to promote inclusive local and national labour market development with a particular focus on disadvantaged individuals. The evaluation should also provide specific recommendations as to the priority areas that should be considered in next projects implemented by UNDP and ILO, including interventions that require continued support, successful interventions for expansion, and recommendations on prioritizing interventions to maximize impact. It should also define recommendations to improve project management structure.
The evaluation will follow a participatory approach that will include both implementing agencies management structure, key partners will be represented, and additional consultation conducted with key stakeholders, governmental representatives from relevant ministries and national institutions, with civil society representatives as well as key donor partners.
Evaluation Objectives:
The overall objective of the project evaluation will be to assess the extent, to which the project achieved its overall objectives and outputs as identified in the project document and annual working plans. More specifically, the evaluation will aim to:
- Review effectiveness of the overall Project interventions, its main achievements and overall impact of the poject in improving the inclusivness of the labour market in the Western Balkan;
- Review and evaluate the extent to which project outputs have reached the intended beneficiaries;
- Assess the sustainability of project outputs and benefits – analyze how far has the project ensured the sustainability of the project benefits;
- Asses how the intervention and its results relate and contribute to the achievement of 2030 Agenda for Sustainable Development, EU’s “Credible enlargement perspective for and en-hanced EU engagement with the Western Balkans”, and RCC’s South East Europe Strategy 2020.
- Identify lessons learnt from the project implementation and the interventions in the area;
- Identify future similar needs.
Evaluation Approach and Methodology:
The evaluator is expected to consult all relevant sources of information, such as the project document, project reports, project budget, progress reports, project files and any other material that may be consider useful for evidence-based assessment, including interviews and focus groups with stakeholders and beneficiaries.
The evaluation should provide as much gender disaggregated data as possible.
The evaluators are expected to follow a participatory and consultative approach ensuring close engagement with the project counterparts at the national and local level, Project Board, project beneficiaries and key stakeholders. All relevant project documentation will be made available by the implementing agencies, UNDP and ILO, regional and country/territory.
Considering the mandates to incorporate human rights and gender equality in all UN work these dimensions will require special attention for this evaluation and will be considered under each evaluation criterion. When collecting data, evaluators need to ensure that women and disadvantaged groups are adequately represented.
It is expected that the evaluation consultant will develop an evaluation matrix, which will relate to the below questions (and refine them as needed), the areas they refer to, the criteria for evaluating them, the indicators and the means for verification as a tool for the evaluation. Final evaluation matrix will be approved in the evaluation inception report.
The Consultant is expected to explore evidence and where possible validate the achievement against set targets. The Project’s Logframe Matrix will be used as a benchmark for the evaluation.
The results of this evaluation will be shared with the Project Board, ADA, UNDP and ILO offices and national stakeholders. Information specifically targeting the successes and failures of the project is especially sought after.
The project evaluation will be carried out in accordance with UNDP’s[1] and ILO’s [2] evaluation policies and ADA’s[3] guidelines for project evaluations.
Evaluation Criteria and Questions:
Due to the limited scope of the project, the evaluation should refer and use relevance, effectiveness, and sustainability as the evaluation criteria, mainstreaming gender equality and social inclusion perspective as a cross cutting issues.
More specifically, the evaluation will address the following evaluation questions that will be further refined once the evaluation team is recruited[4]:
Relevance:
- To what extent was the design of the intervention and its results relevant to the needs and priorities of the beneficiaries? Was the choice of interventions relevant to the situation of the target group?
- To what extent key national partners were involved in programme’s conceptualization and design process?
- Review how the project addressed countries’ priorities. Review country ownership. Was the project concept in line with the national sector development priorities and plans of the country?
- To what extent has gender and human rights principles and strategies been integrated into the project design and implementation?
- To what extent is the intervention aligned with Agenda for Sustainable Development, EU’s “Credible enlargement perspective for and en-hanced EU engagement with the Western Balkans”, and RCC’s South East Europe Strategy 2020?
- To what extent was the design of the intervention relevant to the inclusiveness agenda in the Western Balkan?
Effectiveness:
- To what extent have the expected results of the project been achieved on both outcome and output levels?
- What are the reasons for the achievement or non-achievement of the project results? Has project achieved any unforeseen results, either positive or negative? For whom? What are the good practices and the obstacles or shortcomings encountered? How were they overcome?
- How effective have the selected programme strategies and approaches been in achieving programme results?
- How does the project fit in and align with other relevant initiatives supported by bilateral and multilateral donors and organizations?
- To what extent the project interventions have ensured coherency in the approach with relevant interventions implemented in the Western Balkans?
- How well did the intervention succeed in involving and building the capacities of rights-holders, duty-bearers, as well as the project partners?
- Have the addressed beneficiaries been reached? What are the achievements in terms of employment and job creation for single individuals and facilitating access to more efficient and effective public services – quantitative and qualitative?
- How the intervention impacted employment outcomes of particularly disadvantaged individuals?
- To what extent are the programme approaches and strategies are innovative for achieving labour market inclusion and gender equality in project countries? What -if any- types of innovative good practices have been introduced in the programme?
- What contribution are participating UN agencies making to implementing global norms and standards for economic empowerment and labour market inclusion?
- To what extent the project improved communication, coordination and information exchange within the national labour markets at all level?
- Is there a clear understanding of roles and responsibilities by all parties involved?
Sustainability:
- What is the likelihood that the benefits from the project will be maintained for a reasonably long period of time after the project phase out?
- To what extent the intervention succeeded in building individual and institutional capacities of rights-holders and duty-bearers to ensure sustainability of benefits and more inclusive practices to local development and good governance?
- How effectively has the project generated national ownership of the results achieved, the establishment of partnerships with relevant stakeholders and the development of national capacities to ensure sustainability of efforts and benefits?
- What steps were taken to develop and/or reinforce the operating capacities of national partners during the implementation of the programme?
- To what extent has the project been able to promote replication and/or up-scaling of successful practices from the Inclusive Labour Market Phase I?
- To what extent has the exit strategy been well planned and successfully implemented, including potentials and directions for conituation of support?
- How effectively has project contributed to the establishment of effective partnerships and development of national capacities?
[1] http://web.undp.org/evaluation/guidance.shtml#outcome
[2] http://www.ilo.org/eval/Evaluationguidance/WCMS_168289/lang–en/index.htm
[3]https://www.entwicklung.at/fileadmin/user_upload/Dokumente/Zivilgesellschaft/GL_ExpenditureVerification_Dec2021.pdf