1. Introduction
This document sets out the requirements relating to the external evaluation mandate, the selection process, and criteria for the Weather-Water-Climate Services (WWCS) project implemented in Tajikistan and funded jointly and equally by Caritas Switzerland and the Swiss Confederation (acting through the Swiss Agency for Development and Cooperation, SDC, based in Dushanbe).
The Terms of Reference (ToR) describe the purpose, context, objectives (including guiding indicative evaluation questions), scope and a proposed methodology of the evaluation. They further describe the evaluation process and the expected deliverables. The ToR will form an integral component of the contract for this evaluation.
2. Background information and context of the evaluation
2.1. Project rationale and context
With close to 70% of the population of Tajikistan residing within rural communities, agriculture plays a significant role in rural livelihoods and will likely remain for some time. The inextricable links between weather, water, land use management, food security and disaster risk fundamentally characterize rural livelihoods, but equally the country as a whole and its economy. Whereas availability of water is in principle plentiful, though depleting, and essential for the local economy, farmers lack qualified and robust information on precipitation and weather in general. They therefore cannot easily adjust their agricultural practices to changing weather and climate, find challenges to inform crop mix choices (linked to national cropping plans), and rely on irrigation practices that are based on generally outdated norms. Natural hazards (flooding, mudflow, landslides) are of equal concern, impacting agricultural production systems as well as risk to human life and capital assets. Farmers also face limited market access. Most can only sell their produce at local markets. For strategic crops such as wheat, potato and cotton, official quotas are paired with unregulated prices – basically shifting all economic risks to the producer. Availability of high vigour and drought-tolerant seed is limited – particularly for forage related varieties and species – further compounded by varietal release and introduction systems that suffer from limited public resources. Vulnerability to weather and climate change therefore remains in so far as accessibility to contextually relevant approaches for adaptation. Food security and malnutrition, together with questions of sustainability of rural livelihoods within rural communities are therefore persistent concerns.
Addressing these concerns requires, as a first step, a dense network of weather observations that can support more precise and reliable weather forecasts within microclimates that characterize the mountainous topography. Supporting this need, the project develops, tests and rolls out a network of low-cost weather stations together with the national agency for hydrometeorology. This naturally then leads to a secondary and necessary effort in the development of accessible, affordable and demanded weather-water-climate services. In the first phase, these services include, among others, timely agronomic decision support, weather-based irrigation advisory services, early warnings/alerts for heat and frost, dynamic monitoring of livestock movements and productivity enhancements within mid to high mountain pastures.
Institutionalizing these services into a national innovation system and (economically) sustainable modality for provision, the project endeavours to uncover and test contextually relevant approaches for weather-water-climate service provision through effective public-private partnerships. Assembling a range of national partners to come on board, and to take ownership of process and outcomes, together with required need for international provision of technical support for capacity strengthening is a core component of this first project phase.
2.2. Project lead and partners
Caritas Switzerland (www.caritas.ch) is a Swiss NGO founded in 1901 with headquarters in Lucerne, Switzerland. The regional programme in Tajikistan (since 1996) and Uzbekistan (since 2022) maintains a focus on enhanced resilience and well-being, through the employment of contextually relevant weather-water-climate services. The country programme in Tajikistan addresses contemporary issues related to climate change adaptation within rural communities that remain vulnerable to natural hazards (flooding, mudflow, landslides) and persistent challenges for food and nutritional security.
This project is implemented by Caritas Switzerland in collaboration with reputable international partners including MeteoSwiss, the Swiss WSL institute for snow and avalanche research (SLF), International Center for Agricultural Research in the Dry Areas (ICARDA), BOKU University (Vienna), and Oregon State University. Ownership is taken by relevant Tajik governmental agencies (Ministry of Agriculture, Agency for Hydrometeorology, Committee for Emergency Situations and Civil Defense, Food Safety Committee, Forest Agency) as well as by district level governments and rural communities.
2.3. Project objectives
Overall Goal (Impact)
Leveraging weather, water and climate services for sustainable rural livelihoods and well-being.
Desired outcomes
Outcome 1: Tajik Hydromet provides timely and effective WWCS.
Outcome 2: Farmers utilize WWCS to inform their decisions for effective agronomic practices, diversified crop mix associations and Sustainable Land Management practices.
Outcome 3: Communities and authorities in target areas employ WWCS to manage risks from natural hazards.
Outcome 4: Governmental structures at all levels (national, oblast, district, jamoat) incorporate WWCS into their development planning processes.
Longer term vision of the project
- Through incentive based (innovative) approaches and systems, governmental partners are transformed into service-oriented agencies that provide needed WWCS to a broad range of sectors and clients (public and private).
- WWCS is integrated into district development plans that are developed participatorily with relevant stakeholders.
- Rural Advisory Service providers employ WWCS in their advice to farmers to support more effective decision-making processes at farm level to increase resilience to climate change, improve livelihoods and protection of productive assets.
- Timely and location specific alerts for inclement weather are utilized by national agencies to support public health and safety through advice and recommendation to targeted populations.
3. Objective, scope and focus of the evaluation
3.1. Evaluation objectives
- To provide Caritas Switzerland and SDC with an independent, evidence-based assessment of how the approaches developed by the project address relevant and contemporary challenges; and how they contributed to the outcomes envisioned within the life of the project and towards attaining the longer-term goals.
- To contribute to the understanding of the performance of the project against the planned project objectives, expected results and targets as per the logical framework. Furthermore, the evaluation will generate lessons learned for the future phase II of the project and identify, where possible best practices and innovations.
- To provide recommendations for potential improvements in the approaches and systems developed and implemented to foster broad uptake and scaling at the level of national policy.
- To validate positive impacts of the project in terms of national ownership and impact at the level of rural households and communities.
3.2. Purpose and objectives
The objective of this evaluation is to evaluate overall implementation of the project as well to inform the implementation efforts within Phase 1, to present opportunities for Phase 2, as well the planning of the WWCS project exit phase. Linkages and potential synergies and uptake of the WWCS objective by other SDC and non-SDC funded projects, more specifically, the evaluation shall
- Evaluate the delivered outcomes and outputs and assess their contribution to the project’s overall objective;
- Assess the effectiveness and efficiency of the project implementation;
- Assess the long-term sustainability of the project interventions;
- Identify key lessons learnt to date, particularly with regards to its chosen strategic processes, mechanisms and partnerships to achieve the project’s objectives to date;
- Assess the role, input and project ownership by the implementing government partners (TJHM, CoES and MoA);
- Analyze and validate the methodologies and approaches applied;
- Develop recommendations for each aspect to further improve efficiency, effectiveness, and sustainability of the WWCS (Phase-1). Make recommendations on how to best achieve all project objectives in the remaining time of the ongoing phase and the design for next phase;
- To assess on the levels of satisfaction of all the beneficiaries involved in the initiative and collect their recommendations to adjust eventual drifts.
3.3. Scope
The breadth and depth of the evaluation will be informed by the indicative evaluation questions that the evaluation seeks to answer (see section 3.4). The evaluation will assess the intervention area of the WWCS project and thereby prepare the support the planning of follow-up phases (providing a baseline for this planning). The period for evaluation is from April 2021 until the start of the review.
3.4. Indicative evaluation questions / key focus area
During the inception phase, the evaluator(s), in consultation with Caritas Switzerland and the Swiss Cooperation Office in Tajikistan, should further refine and prioritise the questions that are structured according to the OECD DAC criteria, but they are not only limited to these. The bidder is also expected to address these questions within the technical bid.
| Relevance | · How relevant are the project’s thematic and geographic focus in Tajikistan? · How are the results and ongoing approaches of WWCS project in line with the project partners (TJHM, CoES and MoA) mandate? · How the applied approaches and achieved outcomes are suited to the priorities and policies of the target groups, stakeholders and project funding partner organization? · What are the opportunities for addressing the climate related issues of the country? · Has the project been based on the correct assumptions regarding risks, opportunities and challenges? · Are the planned outcomes and outputs objectives and their respective indicators still relevant from the view of the project impact objective and the latest legislative development? · What recommendations can be drawn for the end of Phase I, and for the potential phase II? |
| Coherence | · To what extent is the project compatible with other Swiss development cooperation interventions in Tajikistan and thematic field (consistency, complementarity and synergies)? · How is the project positioned among donors’ and government’s interventions in the climate change and DRR agenda? · How the project supports the government strategies and programmes’ implementation related to CC and DRR, as well improvement of the agricultural systems, · What are the roles and contributions of the state agencies (mainly MoA, CoES, CoEP/TJHM) in the implementation process, in terms of taking the ownership, policy advice, staff allocation and etc.) · What is the chance of climate service mainstreaming and synergizing in/with other projects? To what extent is the intervention compatible with interventions of other actors (bilateral and multilateral donors, private sector, UN, NGOs, etc) in the country and thematic fields (complementarity and synergies)? · Are there any unintended results (outcomes/outputs)? If so, which are their effects? · How is the impact of National Steering Committee meetings’ influence on cooperation and coordination among relevant state institutions? |
| Effectiveness | · What is the status of the results achieved from the project start until the start of the review? · How the WWCS intervention has been effectively applied at all levels, from national to district levels? · Were the outcomes achieved as stated (or on target to achievement)? · To what extent did capacity building activities meet the needs ? Did these activities address capacity gaps of all beneficiaries of the project? · Should any other government sectors be involved to reach the project outcomes? · Were the outputs delivered as planned? · Can the outcomes attained be directly attributed to project activities implemented? What major (external & internal) factors contributed to, or challenged the successful attainment of outcomes? · Were there any positive or negative effects as a result of the intervention? · Does the project strengthen community participation, notably women and vulnerable groups at the local level? · To what extent were the intended results of the intervention achieved (or are likely to be achieved) at the level of output, outcome and overall goal of the intervention? · What are the recommendations for the remaining phase? |
| Efficiency | · Were the human and financial resources used as planned and appropriately and fully utilised? If not, how were these adjusted? Why and how? A. Support of National partners? B. Involvement of International partners? C. Project Trust Fund? D. Local staff allocation? · Was the timeline for achieving the outcomes realistic? What should be extended? What is delayed? · What have been unforeseen challenges/opportunities in terms of resource allocation (technical, human and financial)? How well were these handled? · How do project management as well as steering and decision-making processes function? Are problems identified in good time and are practical, feasible solutions proposed? · Was the project implemented on the basis of a result-oriented approach? |
| Impact | · How did the intervention cause higher-level effects (such as changes in norms or systems)? such as water use, rural advisory systems… · How will the intervention contribute to changing society for the better? Or is any improvement made in communities’ livelihood? Were there unintended (positive or negative) effects for the target groups and in society? · What environmental and economic impacts have been achieved (or likely to be achieved)? · Which positive, lasting effects and behavioural changes can be identified? · Are there any identified good/best practices? If yes how these can be replicated in other regions of Tajikistan that have similar interventions? · Which unexpected and unintended positive and negative (side) effects have occurred?· Did the specific part of the intervention have a greater impact than another? · How did the project contribute towards innovation, capacity building, policy processes and catalazing implementation? · What is the impact of the project towards resilience building of communities in the project areas (lasting effects and behavioural change)? · What is the potential (today and/or expected by the end of the phase) for upscaling of the project experience in Tajikistan and beyond? |
| Sustainability | · The extent of partners involvement in project decision making? · Have the partner institutions and addressed population sections embraced the aims and activities originally promoted by the project (ownership)? · How sustainable is the RAS mechanism fostered through the project initiative? Should they be supported and/or expanded further? · What potential exists, based on experience from the first phase, to sustain the WWCS system beyond the project duration? · What are the recommendations to make the project results sustainable until the end of the phase and beyond? · Are there any unintended consequences from the project initiative (positive or negative)? · Are there important synergies or complementarities to be considered with relevant/related initiatives (by SDC or other) in view of future engagement? · How the project has contributed towards the achievement of national and international frameworks and strategies? (National DRR Strategy, Sendai Framework, SDGs…) |
4. Evaluation process and methods
4.1. Evaluation methodology
At a minimum, the evaluation should be based on the following:
- Desk review of relevant documents (project proposal, budgets, project period reports, submitted to SDC including the partners’ reports, communication materials developed, et cetera);
- Pre-engagement briefing (Dushanbe) prior to field visits and engagement with the project team;
- Field visits that include focus group discussions, key informant interviews, interviews with selected beneficiaries (including villagers as well as dekhkan farms, individual farmers, PTF recipients and women production groups), interviews with governmental partners; other discussions as relevant and requested. The consultant/evaluator (s) will propose an agenda for discussions with relevant stakeholders inclusive of type of participant as well as number of participants.
In addition, it is expected that the applicant for evaluation can propose methodology (s) for conducting the survey.
4.2. Roles and responsibilities of the evaluator(s)
The evaluation will be conducted by a team composed of an international consultant and a local consultant. The overall responsibility will lie with the international consultant that is awarded the contract. The international consultant will report to the regional office of Caritas Switzerland based in Dushanbe, Tajikistan.
4.3. Evaluation process and timeframe
The following work plan provides suggested dates, responsibilities and resources needed for the various activities of the evaluation process. This work plan will eventually be adapted by the evaluation team, in consultation with Caritas Switzerland and the SCO. The bidder may request amendments or additions to the activity table below for consideration.
| Activity | Date | Responsibilities |
| Pre-evaluation meeting with evaluator(s) with SDC and CACH separately | June 21, 2024 | Caritas CACH, SDC; Consultant/s |
| Field visit: Interviews with stakeholders, partners and, if relevant, focus group and workshops, desk study including field visits. | June 23-28, 2024 | Consultant/s |
| Meeting with the National partners | July 01-02, 2024 | Consultant/s (CACH will assist in setting up the meetings) |
| Meeting with International partners (online) | July 03, 2024 | Consultant/s |
| Debriefing session on collected information for the report writeup and report outlines presentation (in Dushanbe); separate sessions with CACH and SDC | July 04, 2024 | Consultant/s -SDC/Caritas CACH |
| Draft Inception Report | July 12, 2024 | Consultant/s |
| Feedback on the Evaluation Report by Caritas Switzerland and the SCO | July 13, 2024 | Caritas CACH/SDC |
| Finalisation of the Evaluation Report (incorporation of Caritas CaCH/SDC comments) | July 18, 2024 | Consultant/s |
| Final Evaluation Report | July 18, 2024 | Consultant/s |
| Activity | Number of days |
| Preparatory work, data collection, meeting with partners/stakeholders | 4 days |
| Field days | 5 days |
| Report writing | 11 days |
| International travel | 2 days |
| Total | 22 days |
Timeframe to be discussed with consultant(s), but the work will be undertaken over a timeline of approximately 22 days.
5. Deliverables
The following deliverables are expected from the evaluator(s):
- Debriefing session in Dushanbe, Tajikistan, with Caritas Switzerland, the SDC in Tajikistan, and other relevant partners with presentation of findings.
- Submission of an initial draft report to Caritas Switzerland and SDC within 10 days after the undertaking the evaluation.
- Final Report (within 2 weeks after receiving feedback from Caritas Switzerland and SDC).
- Analysis of the intervention logic (logframe): extent to which objectives have been achieved.
- The SDC’s Assessment Grid of the DAC Criteria completed by the evaluator(s) and attached to the final evaluation report.
- List of interviewed persons.
The final report will have a maximum of 20 pages, excl. annexes. Standard structure (can be expanded): Table of content, executive summary, methodology, findings and lessons learnt, conclusion and recommendations for phase II.
Note: Reporting language is English. CACH and SDC may use the (translated) reports or parts thereof for informing national partners of the project.
The evaluator (s) will report to Regional Director – Central Asia of Caritas Switzerland for the entire duration of the assignment.
6. Reference Documents
After signing the contract, the evaluation manager (Caritas Switzerland) will share the following documents with the evaluator(s) for the evaluator’s first desk review:
- WWCS project proposal, interim and annual reports, including finances, policy briefs, project partners reports.
- Project Steering Committee meeting minutes.
- Contact list of key people for the interview and meeting
- SDC guidance documents and policies for the Climate Change and DRR sector.
- Swiss international cooperation strategy.
- An open list of key people to interview/meet
7. Competency profile of the evaluator(s)
The evaluation team composed of an international (lead) and a national expert, is expected to bring along the following evaluation and thematic expertise, experience and qualities.
Required qualities are:
- Thematic up-to date knowledge and experience in projects/programs/initiatives in Climate Change Adaptation, DRR, agricultural sciences, natural resource management, social sciences with substantive experience in related fields of endeavour.
- Minimum of 5-10 years of experience in evaluations comparable in scope, team size and composition; Central Asian experience would be considered an asset.
- Strong analytical and editorial skills, ability to synthesize and write intelligibly for different audiences.
- Excellent interviewing and social skills, including ability to conduct interviews with both community representatives and high-level officials.
- Substantive experience in project implementation and project monitoring is preferred;
- Team leading competencies (applies to team leader)
- Proficiency in written and spoken English, knowledge of Tajik and Russian languages (for local consultant).
8. Reporting
The evaluator(s) will report to the Regional Director of Caritas Switzerland in Central Asia. Operational support will be provided by the WWCS Project Manager based in Dushanbe.
9. Suitability and award criteria
The required suitability and award criteria as well as their order of precedence and weighting must be defined when the ToR are prepared. For this engagement, the technical offer will be evaluated according to formal quality of the offer; experience of the tenderer; link to the ToR / Technical specifications; and qualification of foreseen personnel. Each of these categories contributes equally to the overall assessment of the technical offer. The assessment of the technical offer will have twice as much weight in the final assessment as the financial offer, which will be assessed according to the quality and form of the offer as well as the price.
One important suitability criteria is that the evaluator(s) are independent[1] of the FDFA and, in particular, the SDC and were not involved in activities covered by this evaluation.
10. Application procedure
Technical and financial proposals are to be submitted to the Caritas Switzerland Regional office by email to skassam@caritas.ch copy to borlowsky@caritas.ch, anasibov@caritas.ch or by hand to the Caritas Switzerland Regional office in Dushanbe, Tajikistan, 59 Shukufon street, by 17:00 local time on May 30, 2024 COB.
The technical proposal should not exceed 5-6 pages and should outline the service provider’s:
- Presenting the methodology and approaches will be applied for the assignment.
- Experience with similar assignments (incl. CVs);
- A detail financial proposal
The financial proposal should be no more than one page and should clearly outline the daily rates in Swiss Francs (CHF).
In the subject line, please indicate “External Evaluation: Weather, Water and Climate Services.”
11. Contracting
The contract will be awarded by Caritas Switzerland following an analysis of technical and financial proposals received in response to these terms of reference and chosen jointly by Caritas Switzerland and the Swiss Cooperation Office in Tajikistan.
[1] ‘Independent’ means that the members of the evaluation team must not have worked for the FDFA and, in particular, the SDC, in the past five years (except in the capacity of conducting other external evaluation mandates) or have any other strong links or dependencies on the FDFA.
