Seasonal Assessment and Mid Term Evaluation Report For "Increase the resilience of chronically vulnerable Somali people, households, communities and systems to recurrent climatic shocks" Implemented in Afgoye and Baidoa districts and funded by the BMZ Transitional Assistance The Gu Season (April to June)
SomReP Seasonal Assessment and BMZ Evaluation Report ⎯ Oct 2022 2 Table of Contents I. ACKNOWLEDGEMENTS.....................................................................................................................4 II. GLOSSARY/ACRONYMS AND ABBREVIATIONS ..........................................................................5 III. INTRODUCTION ...................................................................................................................................7 1.0 EXECUTIVE SUMMARY..................................................................................................................9 2.0 ASSESSMENT &EVALUATION INTRODUCTION/BACKGROUND ........................................17 3.0 METHODOLOGY............................................................................................................................20 3.1 SAMPLING........................................................................................................................................... 20 3.2 DATA MANAGEMENT .......................................................................................................................... 21 4.0 LIMITATIONS..................................................................................................................................21 5.0 FINDINGS........................................................................................................................................22 5.1 SEASONAL ASSESSMENT FINDINGS .................................................................................................. 22 5.1.1 Program Intervention Intensity .................................................................................................. 22 5.1.2 Income.......................................................................................................................................... 24 5.1.3 Expenditure.................................................................................................................................. 26 5.1.4 Shocks .......................................................................................................................................... 27 5.1.5 Shock Exposure Index ............................................................................................................... 28 5.1.6 Access to Water.......................................................................................................................... 30 5.1.7 TVET............................................................................................................................................. 31 5.1.8 Agriculture .................................................................................................................................... 31 5.1.9 Livestock ...................................................................................................................................... 33 5.1.10 Markets .................................................................................................................................... 34 5.1.11 Food Security.......................................................................................................................... 35 5.1.12 Household Hunger Scale ...................................................................................................... 36 5.1.13 Reduced Coping Strategy..................................................................................................... 38 5.2 THE BMZ FUNDED PROJECT MIDTERM EVALUATION FINDINGS ...................................................... 40 5.2.1 Relevance .................................................................................................................................... 41 5.2.2 Effectiveness ............................................................................................................................... 44 5.2.3 Sustainability................................................................................................................................ 52 6.0 DISCUSSION ..................................................................................................................................56 6.1 SEASONAL ASSESSMENT................................................................................................................... 56 6.2 THE BMZ FUNDED PROJECT IN SOUTH WEST STATE...................................................................... 59 6.3 A NOTE ON SUSTAINABILITY ............................................................................................................... 63 7.0 CONCLUSIONS AND RECOMMENDATIONS............................................................................64 8.0 REFERENCES................................................................................................................................68 9.0 ANNEXES .......................................................................................................................................70
SomReP Seasonal Assessment and BMZ Evaluation Report ⎯ Oct 2022 3 Tables Table 1: Sampling Table .......................................................................................................20 Table 2: Gender Disability and Livelihood on Program Intensity....................................23 Table 3: Program Intensity in Districts ................................................................................23 Table 4: Mean incomes between Seasons Gender and Disability.................................25 Table 5: Mean Incomes across Districts ............................................................................25 Table 6: Expenditure by livelihoods ....................................................................................26 Table 7: Expenditure by the project districts......................................................................26 Table 8: Proportion of Households exposed to shocks. ..................................................28 Table 9: Access to Water......................................................................................................30 Table 10: Landownership by Livelihood Zones.................................................................33 Table 11: Markets by distance .............................................................................................34 Table 12: Food Security Overall ..........................................................................................35 Table 13: Food Consumption score – Gender Disability and Livelihoods ....................35 Table 14: Food Consumption and Livelihood zones - Differentials................................36 Table 15: Gender disability and livelihood dimensions of HHS......................................37 Table 16: Program District and Household Hunger Category – Differentials...............38 Table 17: Household Coping Measurements ....................................................................39 Table 18: Household Coping by gender, disability and livelihood..................................39 Table 19: Household Coping Measurements by district – Differentials.........................40 Table 20: Traffic Light Ratings .............................................................................................44 Table 21 BMZ Funded project result 1 outcomes.............................................................45 Table 22: Output delivery under Outcome 1......................................................................46 Table 23: Indicator traffic light scores .................................................................................46 Table 24: BMZ Funded project result 2 outcomes............................................................47 Table 25: Indicator Traffic light score..................................................................................47 Table 26: Result area #2 Outputs Scores ..........................................................................48 Table 27: Result area # 3 outcome indictor scores ..........................................................49 Table 28: The Traffic Light for Outcome Areas #3............................................................49 Table 29: Result #3 Outputs Indicator scores ...................................................................49 Table 30: Result 4 outcome indicator scores ....................................................................50 Table 31: outcome #4 traffic light scores............................................................................50 Table 32: Outcome #4 Output Scores ................................................................................51 Table 33: Outcome #5 Indicator Scores.............................................................................51 Table 34: Outcome #5 Traffic Light Score .........................................................................52 Table 35: Outcome #5 output indicator Score...................................................................52 Figures Figure 1: Variances in Program Intensity between Seasons ..........................................22 Figure 2: Incomes between Seasons..................................................................................24 Figure 3: Shock Exposure during The Seasons................................................................27 Figure 4: Shock Exposure Index..........................................................................................29 Figure 5: Shock Exposure Index..........................................................................................29 Figure 6: TVET Outcomes ....................................................................................................31 Figure 7: Agricultural Outcomes ..........................................................................................32 Figure 8: Agricultural Outcomes Comparisons..................................................................32 Figure 9: Livestock ownership..............................................................................................33 Figure 10: Mean Earnings from Livestock..........................................................................34 Figure 11: Markets commodity price...................................................................................34 Figure 12: Household Hunger Scale...................................................................................37
SomReP Seasonal Assessment and BMZ Evaluation Report ⎯ Oct 2022 4 i. Acknowledgements This report has been prepared with the help of key Monitoring and Evaluation staff from world Vision Somalia led by Jerome Galagade and Sandra Ndungu. Technical inputs have been made by Dr. Peter Hayombe, Mr. Abdihamat Abdullahi and Mr. Chris Smoot. The overall review has been done by Gideon Odhiambo.
SomReP Seasonal Assessment and BMZ Evaluation Report ⎯ Oct 2022 5 ii. Glossary/Acronyms and Abbreviations AfRACS Alliance For Research and Consulting Service ARM Annual Resilience Measurement C4FC Community-Led Capacity Strengthening for Fragile Contexts CAAP Community Action and Adaptation Plan CAHWS Community Animal Health Workers CAP Community Action Plans CARE Cooperative for Assistance and Relief Everywhere CBDRM Community Based Disaster Risk Management CBDRM-A Community Based Disaster Risk Management Advisor CCT Conditional Cash Transfer CF Contingency Plans CFW Cash For Work COOPI Cooperazione Internazionale COVID-19 Corona Virus Disease of 2019 CSI Coping Strategy Index DAG Donor Advisory Group DIP Detailed Implementation Plan DRC Danish Refugee Council DRR Disaster Risk Reduction EU European Union EURO European Union Currency EWC Early Warning Committee EWS Early Warning System FCS Food Consumption Score FFS Farmer Field School FGD Focus Group Discussion FSL Food Security and Livelihood GAP Good Agricultural Practices GCVCA Gender Climate Vulnerability Capability Assessment GDPR General Data Protection Regulation GEEL Growth, Enterprise, Employment and Livelihoods HADMA Humanitarian Affairs and Disaster Management Agency HHs Households HHS Household Hunger Score IBLI Index Based Livestock Insurance IDP Internally Displaced Person IOM International Organisation for Migration IPM Integrated Pest Management ITT Indicator Tracking Table KII Key Informant Interview L-NGOS Local Non Governmental Organisation M&E Monitoring and Evaluation MEAL Monitoring Evaluation and Learning
SomReP Seasonal Assessment and BMZ Evaluation Report ⎯ Oct 2022 6 MFI Microfinance Institution MoA Ministry of Agriculture MoLFR Ministry of Livestock, Forestry and Rangeland MOPIED Ministry of Planning, Investment and Economic Development NEMHEWC National Emergency and Multi-Hazard Early Warning Centre NRM Natural Resources Management P/FFS Pastoralist /Farmers Field Schools PMREL Participatory Monitoring Reflection, Evaluation and Learning PPP Public Private Partnership QRM Quarterly Review Meetings rCSI Reduced Coping Strategy Index SAC Social Affairs Committee SHQ Shaqodoon Organisation SomReP Somalia Resilience Programme SWS South West States ToR Terms of Reference ToT Trainer of Trainers TU Technical Unit TVET Technical Vocational Education and Training TWG Technical Working Group UCT Unconditional Cash Transfer UI User Interface USAID The United States Agency for International Development USD United States Dollars -Currency VDC Village Development Committee VSLA Village Savings and Loans Association WASH Water, Sanitation and Health
SomReP Seasonal Assessment and BMZ Evaluation Report ⎯ Oct 2022 7 iii. Introduction The Seasonal Assessment and midterm Evaluation Report for the three year project “increase the resilience of chronically vulnerable Somali people, households, communities, and systems to recurrent climatic shocks in the Afgoye and Baidoa districts and funded by the BMZ transitional assistance, has been prepared by Agency for Development Research Innovation and Training (ADRIT) LTD in collaboration with the SomRep Monitoring and Evaluation Team. The SomRep team provided ADRIT with necessary qualitative and quantitative data, relevant project documents for review and analysis and the template for reporting. The narratives used in report writing have been derived from mixed methods data analysis and framed to respond to the evaluation questions and specific objective of this Seasonal Assessment and Midterm Evaluation based on the Terms of References provided. The development challenges in Somalia have for more than 30 years been eclipsed by complex and protracted states of crisis fuelled by political instability and frequent conflicts on the one hand and a mixture of shock exposures on the other hand. Among the shocks are environmental, economic and health related including the recent COVID-19 pandemic. The consequences of these have been severe on populations with many suffering widespread displacement, food insecurity and high levels of poverty. Overtime, the capacities of individuals, communities and systems to effectively respond and recover from the humanitarian and human development setbacks have been weakened. SomRep is one of the significant interventions in Somalia seeking to not only cushion individuals, communities and systems from these adversities but also; more importantly, stimulate and enable them acquire adaptive, absorptive and transformative capacities for resilience. SomRep has continued to intervene in Somalia, Puntland and Somaliland, in resilience programming which bridges the humanitarian and development continuum1 . The character of the SomRep interventions in this humanitarian and development continuum for resilience building is constructed around multi-sector-multi-actor approaches, evidenced based (learning systems and research) action, flexibility, rigorous monitoring system, holistic approach to building resilience, community based early warning Early action system, increased capacity of local responders, conflict sensitivity, ecosystem strengthening, robust financial controls and risk mitigation systems, collaborative relationships with key institutions and initiatives, donor engagement, established consortium structure.These priority issues are echoed by other actors in Somalia who seek to contribute to a more self-sustaining Somalia; one that is a net contributor to regional peace and prosperity through supporting inclusive and resilient economic growth as well as stable, effective governance2 . These strategies align with Somalia’s development planning designed around four pillars (the ninth National Development Plan (NDP-9), 2020-2024)3 which are pillar(1) Inclusive politics, pillar (2): Security and the Rule of Law, pillar (3): Economic development and pillar (4): Social development. These country specific priorities are critical for creating an enabling framework for development actors to collaborate more effectively with federal and local governments to deliver resilience development action and obtain sustainable results. In this context of contribution to resilience actions and outcomes in Somalia, SomRep program initiated and has been implementing a three years project titled “increase the resilience of chronically vulnerable Somali people, households, communities, and systems to recurrent climatic shocks in the Afgoye and Baidoa districts” in South West State of Somalia funded by the BMZ transitional assistance. The project runs from 1 st October 2020 to 30thSeptember 2023. More specifically the project seeks to contribute to the resilience of chronically vulnerable Somali people, households, communities and systems in 43 villages in South West State of Somaliathrough enhanced food security, sustainable livelihoods and 1 SomReP Strategy Phase II 2018-2023 2 The U.S. Mission to Somalia: Integrated Country Strategy accessed at https://www.state.gov/wpcontent/uploads/2022/04/ICS_AF_Somalia_Public.pdf 3MPIED – FRS Somalia National Development Plan 2020 to 2024 (NDP-9) accessed at https://mop.gov.so/national-development-plan /
SomReP Seasonal Assessment and BMZ Evaluation Report ⎯ Oct 2022 8 strengthened disaster risk management capacities in target areas in Southwest State by 2023. The project is being led by WVS working with partrners who include CARE (and a local partner WARDI Relief and Development Initiatives in Afgoye), Cooperazione Internazionale (COOPI) and Danish Refugee Council (DRC) both in Baidoa and Shaqodoon. Shaqodoon is also a local NGO supporting the three SomRep partners through online crowd funding for community projects selected from community action and adaptation plans (CAAPs) developed together with target communities. The project structure is made up of five outcome areas twelve (12) outputs in total and a total of 65 activities. The project results are also measured using several indicators at output at outcome levels during the baseline and expected to be measured at midterm and endline Periods. The full project structure is found in the results framework and its M&E system which is incorporated here by reference. This report is measuring progress and changes at midterm.
SomReP Seasonal Assessment and BMZ Evaluation Report ⎯ Oct 2022 9 1.0 Executive Summary This executive summary has been prepared to show and explain the results related to outputs and outcomes from the activities of the project to give better understanding of performance and inform implementation of the remainder project. The summary focuses on the main analytical points and indicates the main conclusions, lessons learned, and specific recommendations. The Integrated Seasonal Assessment and BMZ project Midterm Evaluation study and report writing was commissioned and overseen by The Somali Resilience Programme (SomRep). The Somali Resilience Programme (SomRep) is a consortium of seven international NGOs namely Action Against Hunger (AAH), the Adventist Development and Relief Agency International (ADRA), Cooperative Assistance for Relief Everywhere (CARE), Cooperazione Internazionale (COOPI), Danish Refugee Council (DRC), Oxfam and World Vision Somalia. The program aims to increase the resilience of chronically vulnerable Somali people, households, communities and systems to climatic shocks and other related risks in targeted pastoral, agro-pastoral and peri-urban livelihood zones by 2023. More information about SomRep can be accessed at https://www.somrep.org/. While this seasonal assessment forms part of the monitoring approach to improve sensitivity to more immediate changes (seasonal or otherwise) as well as capture household responses in the face of shocks closer to the time they happen, the BMZ mid-term evaluation exercise is to provide the SomReP and its stakeholders with an assessment of the performance of the BMZ project to date, based on the agreement, logical framework and activities. The complete TORs for the Integrated Seasonal Assessment and BMZ Midterm Evaluation study and report writing is attached as an annex to this report. A mixed methods study approach was used to design and collect mixed methods data from 960 respondents distributed in 12 districts of project implementation using a combination of probability proportion to size sampling and Purposive sampling techniques. Twenty-four (24) Research Assistants worked for ten (10) days and collected quantitative data from households while 28 Research Assistants collected qualitative data (30 FGDs and 6 KIIs). The results from the data for the period between 2021 and 2022shows a reversal in fundamental wellbeing indicators. Severe hunger figures were up, shock exposure index shot up across all categories of livelihoods as well as the number of those exposed to shocks. To mitigate the situation the target households resorted to negative coping habits such as borrowing proven by increase in the high debts among the households. Medium coping was up under indicator RCSI. Similarly, the project reacted to the situation by increasing interventions on target areas with data showing increase in the number of households with two and three interventions Paradoxically the FCS indicator showed otherwise. Food was more available to the households as those with moderate and acceptable food scores increased and poor category declined. The Results of the data analysis are tabulated and highlighted below through standalone tables illustrating trends and milestones in Seasonal Assessment reporting (SAR) in both 2021 and 2022 as well as the critical variances on indicator performance between the two seasonal assessment reporting. The result areas are on Program Intervention Intensity, Income, Expenditure, Shock Exposure, Shock Exposure Index, Access to Water, TVET, Agriculture, Livestock, Markets, Food Security, Household Hunger Scale and Coping Strategies Index (CSI) : Program Intervention Intensity The evaluation found that program intensity outcomes showed a better trajectory between the 2021 and 2022 seasons. While households participating in one intervention dropped from 72.5% to 70.3% those participating in two interventions increased from 13.5% to 20.5%
SomReP Seasonal Assessment and BMZ Evaluation Report ⎯ Oct 2022 10 and those participating in three interventions increased from 5.7% to 9.2%. The largest increase in program intensity is 6.9% of those participating in two interventions. Although gender and disability differences on program intensity between seasons could not be computed for lack of corresponding data outputs in 2021, it was found that in 2022 females showed better outcomes than males in both medium (22.1% to 16.7%) and high intensity (9.5% to 8.6%) and people with without disability were also better off than those with disability in the same categories of medium (20.5% to 14.3%) and high (9.9% to 8.6%). While gender outcomes are marginally better for females than males and for people with without disability than those with disability they nonetheless signal a continued need for gender parity and disability programming in resilience responses to improve program intensity outcomes. High program intensity scores across livelihood types show agro pastorals to be better off (13.2%) followed by IDPs (5.9%), Peri urban (4.9%) and Pastorals (2.6%). While high program intensity among agro pastorals increased by 4.2% between the two seasons from 9.0% to 13.2% this declined by about 1% among pastoralists. The pastoralists experienced a significant rise of 11.6% in medium intensity between the seasons indicating intention towards high intensity but suffering from the pull to low intensity. But for peri urbans the pull back to low intensity by 16.3% was a very strong negative outcome. Program intensity across districts showed a general trend of regression back to low intensity with the exception of CeelAfweyne and Xudur districts which posted better scores on high intensity between seasons increasing by 22.7% (2.0% to 24.7%) and 16.5% (6.0% to 22.5%) respectively. The largest increase in low intensity was witnessed in Dollow by 49.9% (From 51.10% in SAR 2021 to 1.20% in SAR 2022). Income While in general incomes have reduced between the seasons in the SomRep program areas by USD 17.65 it has increased by $20.30in the BMZ funded project areas. Those who have been most affected by income reductions include pastorals by USD 36.25 from USD 102.0 to USD 65.75 and peri urbans by USD 5.12 from USD 80 to USD 74.88. Only agropastorals saw marginal increase of USD 1.10 from USD 59 to USD 60.1. This speaks to the importance of livelihood diversification and multi capital investments which has not been a strong component of livelihoods of pastorals. Gender and disability results on mean incomes are negative. Mean incomes dropped by USD 96.6 for females and 122.90 for males. The largest drop was among people without disability by USD 19.50 compared to a mere 2.20 drop among the disabled. These differences signal a possible impact of intentional targeting and crisis modifiers in that the vulnerable groups here are faring better. The reductions in mean incomes were also witnessed in the districts. Dollow saw a decline of USD 85.50 from USD 131 in SAR 2021 to USD 45.5 in SAR 2022 followed by Xudur the mean income also declined by USD 24.20 from USD 44.00 in SAR 2021 to USD 19.80 in SAR 2022. Only Burco district saw a small increment in mean income by USD 6.40. Expenditure Spending ability of the households and community declined significantly. With reduced incomes, beneficiaries were forced to borrow to sustain life. The evaluation found that indebtedness increased by 11.42% between the seasons from 55.4% last season to 66.8% this season. Agro pastorals suffered more by 19.3% (from 42.2% to 61.5%) followed by peri urban beneficiaries at 18.6% (from 56% to 74.6%) and pastoral at 7.8% (from 65.7% to 73.5%). Most of the debts have been incurred for basic sustenance on food and still most of them have been unable to repay debts. Indebtedness increased by 76.5% in Togdheer, 40.3% in Sanaag and 55.3 in Laas Caanood. More females (68.0%) incurred debts compared to males (64.6%) Shock Exposure The evaluation found that shock exposure has increased between the seasons by 18.3% (from 55.2% to 73.5%). More households in 2022 (beneficiaries in the project areas have been highly exposed to shocks. More agro pastorals experienced shock exposure by 34.6%
SomReP Seasonal Assessment and BMZ Evaluation Report ⎯ Oct 2022 11 (from 40.9% to 75.5%) followed by Peri-urban households at 28.6% (from 43.7% to 72.3%) and pastorals by 0.4% (from 71.2% to 71.6%). Qualitative data shows that pastorals saw decrease in livestock sales due to drought. The shock exposure index also increased by 8.7 in the program season from 1.5 (SAR 2021) to 10.4 (SAR 2022). The increase was highest among agro pastoral beneficiaries by 8.2 (from 1.8 (SAR 2021) to 10.0) (SAR 2022 and lowest among peri urban livelihood zones at 5.9 (from 4.1(SAR 2021) to 10.0). For the peri urban group it appears that a small exposure to shock results in significant inability to cope as has been shown in food security outcomes. This could signal that peri urban livelihoods are highly sensitive to shocks. The gender differences on shock exposure index were 10.6 for females and 10.3 for males in SAR 2022 showing near similar degree of exposure for both genders. The shock exposure index was 9.3 for persons living with disability slightly lower than for males and females. It suggests a possible impact of crisis modifiers in targeting of this group of beneficiaries. Access to Water Overall, the proportion of household reporting water point to be less than 30 minutes away reduced by 7.0% (from 59.9% to 52.9%). Those reporting water point to be 30 minutes to 1 hour away increased by 5.2% from 25.5% to 30.7% while the ones spending more than 1 hour to a water point increased by2.2% from 14.2% to 16.4%. Over the reporting period between SAR 2021 and SAR 2022 water has become more inaccessible. Water inaccessibility affected females more than males with females (60.3%) more than males (39.7%) spending more than one hour to fetch water. However, the BMZ project areas witnessed better scores on this indicator. As at midterm the project had registered a 6.5% increase with some beneficiaries accessing loans to procure water tanks to store water for irrigation and domestic use. These BMZ funded project beneficries owned the development outcome and spent their own resources to deliver it for themselves. Beneficiary ownership of development priorities is crucial for better outcomes. TVET The results on TVET outcomes are encouraging. More TVET graduates were registered in 2022 (71.0%) than in 2021 (62.3%). An increase by 8.7%. This is a good thing in that a skill once gained remains with the person. The negative outcomes on TVET outcomes on those employed reducing by 4.7% from 30.1% in SAR 2021 to 25.4% in SAR 2022 as well as reduction by 9.1% (from 30.2% to 21.1%) of those who had started a business are part of the effects of shock exposure to market economies in the region. While this calls for creativity and innovation on the part of the graduates (to be provided through lifeskills training) it also challenges TVET curriculum development and delivery to be more sensitive to markets faced by shocks of the likes of Somalia. These outcomes target the youth in Somalia and it seems possible that a component of crisis modification such as a conditional cash transfer specific to TVET training, gradation and transition to employment or self employment could improve outcomes. Agriculture In agriculture there were interesting outcomes. While average land holding had increased by 24.3% between the assessment seasons rising from 37.9% in 2021 to 62.2% in 2022 the average land under cultivation reduced by 35.0% from 2 hectares in 2021 to 1.3 Ha in 2022. Reduction was also witnessed in the proportion of households trained in Good Agriculture Practices; by 34.6%% from 69.1% to 34.5%. It is clear here that acquiring more land does not necessarily translate to its cultivation. Even when agro pastoralists cultivated on average more land (1.1 Ha) followed by pastoralists (0.8 Ha) and peri urban (0.8 Ha) it is clear that land cultivated between the groups do not vary much between the two seasons. While in some cases lack of appropriate tools and inputs were the reasons given for not cultivating in others it was the cost of cultivation being prohibitively higher than returns due to market systems exposure to shocks. On ownership of livestock the results oft of the evaluation are negative. The proportion of households owning livestock dropped by 6.7% from 66.5% in
SomReP Seasonal Assessment and BMZ Evaluation Report ⎯ Oct 2022 12 2021 to 59.8% in 2022 mainly due to disease, death as a result of drought and mass sales to avoid looses. The mean income earned from livestock was also eroded by USD 64 declining from USD 79 to USD 15 Markets The market outcomes were a reversal of project expectations. This is would be expected in the overall scheme of outcomes in this situation of severe exposure to shocks. The proportion of households reporting on market indicators showed mixed results. While those within a 30 minutes walking access to markets decreased by 5.5% (from 49.9% to 44.4%) households within 30 minutes to one hour increased by 2.3% from 30.4% to 32.7%) and those who took more than one hour increased by 2.7% (from 19.6% to 22.3%). Furthermore commodity price perception was dissatisfactory. More beneficiaries perceived them as poor by 41.1% up from 27.0% to 68.1% while those who perceived the prices as good declined by 6.3% (from 37.1% to 30.8%). Beneficiary Wellbeing Outcomes In the evaluation and in general measurement, beneficiary wellbeing is consolidated around food security indicators including Household Hunger Scale and Coping Strategies Index (CSI).While overall acceptable food consumption increased by a marginal 1.3% from 55.6% in SAR 2021 to 56.9% in SAR 2022 Households with Little to no hunger reduced by 7.5% between SAR2021 and SAR2022 and severe hunger increased by 9.6% from 1.7% in SAR 2021 to 11.3% in SAR 2022, The reducing coping strategy results show that households experienced huge decline of 18.2% from 43.5% in SAR 2021 to 25.3% in SAR 2022. Where acceptable food consumption marginally increases by 1.3%, sever hunger increases by 9.6% and reducing coping strategy declines by 18.2% then the wellbeing situation of beneficiaries is improving. As has been shown in this evaluation tghe single most critical difference on wellbeing outcomes is crisis modifiers in the strategic implementation plan in which beneficiaries participate with government institutions and SomRep consortium support to development response strategies to shock exposure and measure progress together in an accountable and transparent manner. Relevance The BMZ funded project has been relevant to a great extent. The project design and components as well as its structure and implementation strategy have been framed based on a theory of change which is derived from the resilience community of practice and responds to the Needs of the people, communities and systems exposed to shocks through activities and outputs designed to build resilience capacities among beneficiaries. The people, communities and systems exposed to shock have needed crisis modifiers, resilience capacity building, participatory planning and governance and collective decision making as well as policy innovation advocacy for government uptake and adoption of participatory planning and governance skills and commitments after project ends. These strategies and approaches have responded to the needs of the community. The creation of community structures including Village Development Committees (VDC), Early Warning Committees (EWC), Natural Resource Management Committee (NRM), Water Management Committees (WMC), and Social Affairs Committee to work with existing community groups such as elders, religious leaders, youth and also government institutions and development partners has been a very strategic project design issue. These community structures have mobilized synergies and community level reserves as local resources to contribute to collective responses to shocks. Effectiveness The project has been effective to a large extent. More than 75% of activities have been executed and the outputs obtained. Out of the nine (9) outcome indicators required by the midterm evaluation the project has scored five (5) greens signaling the project indicator rates well on the evaluation criteria, one (1) orange signaling the project indicator rates relatively
SomReP Seasonal Assessment and BMZ Evaluation Report ⎯ Oct 2022 13 well on the evaluation criteria and three (3) reds signaling the project rates poorly overall on the evaluation criteria. The performance of the BMZ Funded project has been facilitated by flexible agile program implementation decisions on strategy and activities, participation of beneficiaries in conditional cash transfers (crisis modifiers linked to activities which contribute directly to designed outputs and outcomes such as rehabilitation of water sources), participatory planning and governance through PMERL as well as collective decision making within transparent and accountable structures. Sustainability The sustainability of the BMZ funded project is well thought out derived from the SomRep Program second strategy 2018 – 2023. The thinking has been framed adequately into the BMZ funded project design, structure and components but its measurement is indeterminate at this point. The thinking on sustainable water provision, inclusive market systems development, sustainable ecosystem management and governance and policy is excellent but does not find its way into indicator matrices specific to sustainability post project period in 2023 or shortly after. The evaluation sees possible leverages for sustainability of the BMZ funded project benefits and capacity of structures post 2023. They (1) Linkages with SomRep strategy programming direction, approaches and scope post 2023, (2) continuity of participatory planning and governance as well as transparency and accountability in collective decision making on program priorities among community structures and (3) advocacy for and securing government policy innovation to accommodate the CAAPs into its budgetary and oversight functions, Conclusions and Recommendations The following conclusions are made: General (1) Households in the project areas are highly sensitive to shocks from season to season. Household responses in the face of shocks have largely been negative as shown by mixed food security outcomes as well as selling of productive assets without reciprocal productivity activities. The positive thing is that households are progressively transitioning to positive responses with the introduction of crisis modifiers and participatory development approaches. While on the one hand crisis modifiers have improved food security outcomes in some areas (especially with persons with disability, women and agro pastoralists) and cushioned others from sinking deeper into negative coping participatory response planning and governance resilience programming through PMERL has inculcated more ownership of capital development for community action and elevated transparency and accountability in collective decision making on development proprieties. (2) The resilience interventions implemented under the BMZ project have delivered on relevance to a great extent (75% - 100%), and on effectiveness to large extent (50% - 74%), and sustainability to a somewhat (25% - 49%) extent with a pathway to a large extent (75% - 100%). The pathway for sustainability from somewhat to a large extent has been proposed in the recommendation section. (3) The Seasonal Assessment reporting is not consistent from season to season and the data sets are not consolidated into a single registry for measurement, trend analysis and learning. Even where data could be available outputs for indicator status reporting are not derived for actual reporting such as with water outcome indicators and TVET between SAR 2021 and SAR 2022. This affects standardized (at least at program level)
SomReP Seasonal Assessment and BMZ Evaluation Report ⎯ Oct 2022 14 indicator definitions and value determination for comparisons and contrasts between seasons, livelihood types, demographics and program performance measurements over time. Seasonal Assessment (4) Food security in the households within the project areas as well as market prices of goods and services, access to water for households and livestock and irrigation across locations are all highly sensitive to seasons especially to females, persons with disability and peri urbans. Households and livelihood types that have contingency reserves, access to crisis modifiers and have program intensity fare much better than those who do not. (5) The nature and frequency of idiosyncratic and covariate shocks in SomReP operational areas in the last several seasons has been severe, largely overwhelming and widespread in scope beyond project areas of Afgoye and Baidoa. Their effect on food consumption and ability to recover from shocks has been devastating and only moderated using crisis modifiers and hope (encouragement) as a psychosocial and Spritual 4capital built from being involved in participatory response planning and governance of resilience programming activities through PMERL approaches. (6) The markets in the project areas are relatively functional even during shocks but the prices of goods and services are prohibitively high for women, person with disability and peri urban livelihoods making access to goods and services more difficult in times of crisis. (7) While pastoralists and peri urbans have been affected more on most project outcomes in the project areas the agro pastoralists and IDPs have shown better resilience outcomes. When pastoralists sell land and livestock to survive the shock exposures the agro pastoralists who buy them do not cultivates them as much owing to the severity of the shocks to farm inputs, prohibitive and inaccessible credit facilities as well as limited water for irrigation and domestic use. However, some of the beneficiaries who accessed loans built water reservoirs for irrigation of vegetables and fruits for both consumption and incomes from market. Midterm Evaluation (8) The implementation progress made in activities towards achieving the planned results is highly commendable. The BMZ funded project has been diligent and implemented most of activities which it had planned by more than 75%. The implementation is also flexible and agile in nature in that some activities which were delayed in one quarter have still been implemented in another quarter. For instance there was change of strategy to include local partnerships to access inaccessible areas such as in Afgoye. There was also change of activity types to replace important activities with more urgent and important ones such as from roads to water projects in Afgoye. (9) The continued relevance of the BMZ funded project is strong and its effectiveness very positive. And while sustainability of the project benefits, capacity of community structures is well thought out and is being implemented in its strategic plans, its realisation hinges on critical programming factors which include (1) the programming direction, approaches and scope SomRep as a consortium and strategy takes post 2023, (2) continuity of participatory planning and governance as well as transparency and accountability in 4 Community structures Being able to believe that they and their communities can indeed build capacity to overcome crisis when they happen through participatory planning and governance. The hope created and goals set become rally points for collective energy to respond effectively
SomReP Seasonal Assessment and BMZ Evaluation Report ⎯ Oct 2022 15 collective decision making on program priorities among community structures and (3) government policy innovation to accommodate the CAAPs into its budgetary and oversight functions. (10) The evaluation has made strategic, programmatic recommendations for implementing the remaining part of the project. These are in the aspects of boosting crisis modifiers, strengthening participatory planning and governance as well as collective decision making through PMERL approaches within a continued ideological and implementation cover of the SomRep Consortium. Recommendations (1) The BMZ funded project with the help of SomRep program needs to review the seasonal assessments data sets for 2020 and 2021 and 2022 and align each dataset with the indicators of resilience programming results framework as design in the second SomRep strategy 2018 – 2022 with room for modification to include future indicators (on the dataset technology platform) based on continuous learning on resilience programming within WVS and SomRep program. Once the three datasets are aligned to SomRep strategy 2, a single data registry of Seasonal assessment datasets (SRD) should be created to consolidate all the seasonal assessment data between 2020, 2021 and 2022. The single registry dataset becomes the parent dataset in 2022/2023 upon which subsequent seasonal assessments are built and data outputs for reporting derived. This parent dataset will enable SomRep program and the BMZ funded project to frame its data outputs for DME learning through comparisons and contrasts as well as milestones, cycles, patterns, associations between and across various indicators, timelines, variables and theories of change on specific outcome areas and measures. A detailed parent TOR should be written to guide this recommendation from which future TORs on seasonal assessments will be derived. (2) In view of the struggles of Youth in Livehoods during crisis as shown in the TVET outcomes, it is recommended that the project introduces Crisis Modification into youth livelihoods interventions on a conditional basis (Membership into a Youth Livelihood Group, transparency and accountability on enterprise management, savings on incomes, productivity and attending economic capacity development training sessions) through the Economic Development Technical Advisory arm of SomRep program. (3) The BMZ funded project needs to consider and include a male food security vulnerability assessment criteria into its targeting mechanism which can be used in some cases during beneficiary targeting for crisis modification. The evaluation has found that males have shown worse performance on some food security indicators than females. This could signal the beginning of change of power dynamics in which women have traditionally been a marginalised demographic. Targeting marginalized males among pastorals, agro pastorals, peri urbans and IDPs could be crucial to reducing vulnerability and building resilience. Resilience policy-making, programming and funding instruments should embrace the ‘leave no one behind’ principle explicitly, prioritizing actions to support the poorest and most marginalized with the aim of ending extreme poverty and reducing inequalities. (4) Pastoralists should be linked to livestock insurance programs in the project areas and those who are program beneficiaries connected to NRM committees whose scope should be expanded to include activities which help convert lands owned by pastoralists into pastures for livestock through range land management processes including fodder development and water harvesting and conservation technologies for pasture development. This way pastoralists will reduce selling their lands while also making them productive assets for livestock food during shocks. With this modification in project design the project will focus on diversifying livelihoods and help pastoralists re-consider
SomReP Seasonal Assessment and BMZ Evaluation Report ⎯ Oct 2022 16 or minimise migration and transitions from rural to informal urban livelihoods as possible adaptive strategies for smallholder farming. It will also enable resilience strategies to target both host communities and displaced people through access to and employment opportunities, to prevent conflicts over resources including land. (5) The BMZ funded project should maintain the design of its programming and implementation strategies as they are working effectively including its flexible and agile nature in activity implementation. The only modifications proposed to design and strategy is for a 35% downward adjustments on set indicator targets on numbers planned for trainings and contingency funds (budget adjustments difficult midpoint) as well as creating an end of project transition plan for post 2023 sustainability which includes (1) Linkages with SomRep strategy programming direction, approaches and scope post 2023, (2) continuity of participatory planning and governance as well as transparency and accountability in collective decision making on program priorities among community structures and (3) advocacy for and securing government policy innovation to accommodate the CAAPs into its budgetary and oversight functions. A linkage with government policy innovation is a long-term vision as part of ‘durable solutions’ for sustainable management of an enabling environment which is fundamental to fostering resilience. This requires integrated policies and strategies to address regional, district and local development priorities that are climate-smart, environmentally friendly and gender-sensitive, and that address the drivers of vulnerability. (6) SomRep should create a theory of change in a tripartite arrangement of (1) crisis modifiers connected to (2) participatory planning and governance as well as (3) collective decision-making in transparent and accountable structures is applied as instruments for transformative transition from humanitarian to development programming in resilience capacity building. While crisis modifiers have been shown to improve resilience outcomes, participatory planning and governance as well as collective decision-making in transparent and accountable structures have increased ownership of ideology and processes leading to greater adoption of responsibilities for funding and management at community levels. A governance and process oriented approach focused on building and strengthening partnerships between a broad range of local public and private actors is essential to achieving goals in fragile settings at all levels (households, communities, systems). (7) The BMZ funded project has been surprisingly effective in delivering its activities and outputs under severe beneficiary shock exposure. This is a feature of nexus programming that needs further understanding with better evidence determination. It is therefore recommended that the BMZ funded end of project evaluation in 2023 should require that the data bases of SAR 2020 and SAR 2021 and SAR 2022, having been consolidated in 2022 -2023 period (or could be consolidated at that time by the successful consultant) be used to inform trend analysis on resilience outcomes compared between other SomRep projects and districts (as one whole program area) and the two BMZ districts. The end of project evaluation should be based on a quasi experimental evaluation design with strong qualitative data collection in 2023 based on village sample sizes with acceptable scientific validity – at least 30 cases (most district sample sizes for SomRep districts in this evaluation were below this threshold being very low for statistical analysis). These should be complimented with a detailed documents review (activity contribution to and logic connection with outputs and outcomes) combined with analytical observation methods for pictorial analysis. These should provide a robust and more significant understanding of the BMZ funded project design, performance and results against other projects and districts within SomRep programming strategy 2018 - 2023. (8) The BMZ Funded Project in South West State has been surprisingly effective given the severity of the shocks. A possible factor could be the component of Crisis Modification. Given this possibility, it may add value to WVS project design, implementation, results
SomReP Seasonal Assessment and BMZ Evaluation Report ⎯ Oct 2022 17 and learning if the end of project evaluation in 2023 explores the relationship between Resilience Capacities and Crisis Modification across project districts, gender, type of community structure (VSLA, NRM, VDC, TVET e.t.c) and livelihood zones as well as disability to confirm if this indeed as a factor of programming has contributed in resilience capacities in the BMZ funded project compared to other projects in the same districts. This relationship could be qualified with further analysis of how conditionality (conditional or unconditional), trainings (capacity building), contingency reserves, household size, education (literacy), program intensity has impacted on its strength, quality and effects. This could potentially be monumental discovery. 2.0 Assessment &Evaluation Introduction/Background This assessment and evaluation report covers the efforts of The Somali Resilience Programme (SomRep) in Somalia, Puntland and Somaliland and the BMZ Funded project in Afgoye and Baidoa. The Somali Resilience Programme SomRep is a consortium of seven international NGOs namely Action Against Hunger (AAH), the Adventist Development and Relief Agency International (ADRA), Cooperative Assistance for Relief Everywhere (CARE), Cooperazione Internazionale (COOPI), Danish Refugee Council (DRC), Oxfam and World Vision Somalia. The consortium aims to increase the resilience of chronically vulnerable Somali people, households, communities and systems to climatic shocks and other related risks in targeted pastoral, agro-pastoral and peri-urban livelihood zones by 2023. To achieve this, SomReP programming supports resilience through: a) Livelihoods & food security: HHs in targeted communities have improved access to productive livelihoods for enhanced food access and diversity; b) Social Safety Nets: HHs in target communities have their livelihoods and assets protected during shocks and stressors through the establishment and strengthening of social safety nets, including the use of crisis modifier mechanisms such as Savings Group Schemes; c) Natural resource management: Ecosystem health improved through promotion of equitable and sustainable natural resource management; d) Local governance capacity building: Communities, civil society and local institutions are better equipped with resilience strategies and response capacities to cope with recurrent shocks and stressors. Through these interventions, SomRep contributes to the resilience of vulnerable communities in Somalia by bringing communities at the center of cross-border policy and investment discourse and actions; not only as beneficiaries but also as key stakeholders defining the agenda of their future. The BMZ Transitional Assistance funded Project BMZ Transitional Assistance is funding a three year project titled "Increase the resilience of chronically vulnerable Somali people, households, communities and systems to recurrent climatic shocks". It is a 36 months intervention being implemented in 43 villages in South West State of Somalia within the districts of Afgooye and Baidoa between October 2020 to September 2023. The intended outcomes of this project are: 1. Improved capacity of households to implement effective disaster risk management and positive coping strategies to mitigate the immediate effect of exposure of shock. 2. Improved capacity of individuals, households and communities to adhere to positive development trajectories; despite exposures to shocks and utilize strategies designed to allow adaptation to rapid and slow on-set hazards.
SomReP Seasonal Assessment and BMZ Evaluation Report ⎯ Oct 2022 18 3. Enhanced food security and resilience through improved capacity to engage in strategies for sustainable livelihoods and economic growth. 4. Transparent and accountable governance structures at community, district and national levels to ensure an enabling policy and regulatory environment for sustainable livelihoods and economic growth. 5. Local and international stakeholders use Monitoring, Evaluation, Accountability and Learning systems to measure resilience data and use it for evidence based programming and strategic investments in transformative capacities. The BMZ Transitional Assistance funded project seeks to achieve these outcomes through enhanced food security, sustainable livelihoods and strengthened disaster risk management capacities in affected areas in Southwest State by 2023. While the BMZ funded Project is being implemented in 43 villages in South West State of Somalia within the districts of Afgooye and Baidoa, the Seasonal Assessment measures seasonality of resilience covering the twelve (12) SomRep districts of Afgoye, Badhan, Baidoa, Bossaso, Burco, CeelAfweyne, Dollow, Eyl, Hargeisa, Las Canood, Lughaya and Xudur(all found in the three countries of Somaliland , Puntland and Somalia)spread across Somalia, Somaliland and Puntland regions. The map shows the intervention regions as at July 2021 which also include the BMZ funded project locations. Objectives of The Seasonal Assessment and Project Midterm Evaluation The assessment is seasonal occurring during the GU season (April – June) while the BMZ evaluation is midterm coming after (twenty two 22) months of implementation. The objectives of both the Seasonal Assessment and Project Midterm Evaluation are listed below: The Seasonal Assessment The Seasonal assessment is a measurement activity conducted seasonally as part of the monitoring approach to improve sensitivity to more immediate changes (seasonal or otherwise) as well as capture household responses in the face of shocks closer to the time they happen. There are four seasons in Somalia: Jilaal, Gu, Hagaa and Deyr each with its own possible shock possibilities and scenarios. The overall objective is Assess seasonal impacts and household sensitivity to more immediate changes (seasonal or otherwise) as well as capture household responses in the face of shocks closer to the time they happen. The specific objectives of the assessment include: 1. Understand the status of food security and how seasonality affects household income, household expenditures, access to water for households, livestock and irrigation across locations, livelihood zones and other household vulnerabilities. 2. Understand the nature and frequency of idiosyncratic and covariate shocks in SomReP operational areas motivated by season and its effects on food consumption and ability to recover from shocks. 3. Track the completion, employment, access to capital and business start-up of TVET graduates seasonally.
SomReP Seasonal Assessment and BMZ Evaluation Report ⎯ Oct 2022 19 4. Understand the efficiency of markets and track prices of basic commodities in SomReP operational areas. 5. Understand agricultural activities carried out by beneficiaries across SomReP locations seasonally. The assessment helps SomReP understand how individuals, households and communities draw on resources and employ strategies to respond to shocks and stresses and how this affects their well-being trajectories in the short-term from season to season. The BMZ Transitional Assistance funded project Mid Term Evaluation The midterm evaluation seeks to evaluate the extent to which resilience interventions implemented under the BMZ project have delivered on relevance, effectiveness, and sustainability in order to develop a body of evidence that will enable SomReP and its partners to enhance programme effectiveness, increase accountability, and inform any necessary programmatic changes. The specific objectives of this midterm evaluation include: 1. Assess the implementation progress made in activities towards achieving the planned results. 2. Assess the continued relevance, effectiveness and sustainability of the project in supporting local government in enhancing resilience and livelihood of communities in Baidoa and Afgoye and engagement of stakeholder groups in implementation actions. 3. Recommend strategic, programmatic and management considerations for implementing the remaining part of the project. The BMZ funded project contributes to a variety of resources and strategies (as highlighted above) to respond to shocks and stresses through the seasons as part of the wider SomRep response strategy to obtain key well-being outcomes based on the three core areas of change; Absorptive Capacity, adaptive capacity and Transformative Capacities. The SomRep consortium partners have each made contributions to support the seasonal assessment and BMZ midterm evaluation activities in their districts of operation. Qualitative and quantitative data collection with project beneficiaries and partners was led by ADRA in Las Canood, ACF in Eyl and Xudur, COOPI in Dollow and Baidoa, CARE in Badhan, Afgoye, Burco and Bossaso, Oxfam in CeelAfweyne, DRC in Hargeisa and Lughaya. The partners organized FGDs with the existing community structures, such as VLSAs groups, NRM committees, TVETs, Livestock & farmer associations, among others and KIIs with government stakeholders. The whole evaluation team was guided into the assessment and evaluation activities by the SomReP Technical Unit M&E team, Partner M&E staff, Partner Program Managers and field staff. The specific Seasonal Assessment and Midterm Evaluation Team Roles and Responsibilities are detailed in the TOR annexed to this report. 2.1 The Report Structure The Seasonal Assessment and BMZ funded project Midterm Evaluation report follows the structure provided by SomRep TU M&E. In summary the structure is made up of a title page, acknowledgments, glossary, introduction, executive summary, assessment and evaluation introduction, background, methodology, limitations, findings, discussions, conclusions and recommendations as well as annexes: Maps, tools used, plausibility check, and row data. The full structure is annexed to this report. The structure is designed to respond to the Seasonal Assessment and Midterm Evaluation objectives and answer evaluation questions to provide SomRep with two key opportunities: first is to understand how individuals, households and communities draw on resources and employ strategies to respond to shocks and stresses and how this affects their well-being trajectories in the short-term from season to season and secondly for key stakeholders to provide feedback, as well as identify lessons learned and recommendations for the remainder of the project's implementation.
SomReP Seasonal Assessment and BMZ Evaluation Report ⎯ Oct 2022 20 3.0 Methodology The seasonal assessment and BMZ midterm evaluation has been conducted using a mixed methods approach combining quantitative & qualitative data collection tools. Mixed methods data was collected using a household survey, focus group discussions and Key informant interviews between July 1st 2022 and July 21st 2022.Secondary data was collected using an in-depth document review of existing data from secondary sources, primarily SomRep program as well as BMZ funded project reports and project documents. Household survey data was collected from primary project beneficiaries in the twelve (12) districts of SomRep implementation using household questionnaire. The questionnaires asked for household demographics, household’s exposure to shocks (both covariate and idiosyncratic), household resilience and wellbeing indicators for example household productive assets, household expenditure/income, household food security and coping strategies –i.e., Food Consumption Score (FCS), Household Hunger Scale (HHS) and reduced Consumption Strategy Index (rCSI). The FGDs and KII data was obtained from project beneficiaries and partners in order on issues pertinent to the assignment's scope and requirements including existing community structures, such as VLSAs groups, NRM committees, TVETs, Livestock & farmer associations. Both the FGDs and KIIs maximized analytical power and triangulation with other data sources to report on the season under review and the duration of BMZ implementation. The list of FGDs and KIIs and documents are annexed to the report. 3.1 Sampling Purposive sampling was used for the qualitative survey to sample direct beneficiaries (pastoral, agro-pastoral, peri-urban, and internally displaced persons (IDPs)), community members (men, women, boys, and girls, including people with disabilities), government stakeholders, and strategic partners for key informants and focus group discussions. At the community level, focus group discussions ensured that all gender groups (men, women, boys, and girls) and marginalized groups (lesser-clans, people with disabilities, landless farmers, and single-headed households) had an equal opportunity to participate in the study Twenty four (24) Research Assistants worked for ten (10) days and collected quantitative data from households while 28 Research Assistants collected qualitative data which included 30 FGDs and 6 KIIs. A probability proportion to size sampling technique was used to sample beneficiaries for the quantitative survey. Interview was conducted with household heads in each household. Spouses were considered where the household heads were absent. Fisher et al.; (1998) formula was adopted in computing the total sample size for the study which was computed to yield a sample size of 960. The sample was distributed proportionally in SomReP intervention villages per district (clusters) as shown in table 1 below: Table 1: Sampling Table Partner District Population Sample ADRA Las Canood 2530 44 ACF Eyl 1960 34 COOPI Dollow 9680 169 CARE Badhan 3270 57 DRC and COOPI Baidoa 12962 226 CARE Afgoye 10612 185 ACF Xudur 3000 52 DRC Hargeisa 650 11 DRC Lughaya 1446 25 CARE Burco 2730 48 CARE Bossaso 1130 20
SomReP Seasonal Assessment and BMZ Evaluation Report ⎯ Oct 2022 21 OXFAM CeelAfweyne 5012 88 Total 959 The final sample size achieved during data collection was 889 respondents. 3.2 Data Management There are three sets of data outputs being used to report findings in this report. The first set of data outputs is ARM 2021 whose data was collected in November – December 2021. The second set of data outputs is the SAR2021 whose data was collected in May – June 2021. The final one is the SAR2022 whose data was collected in June – July 2022. November – December (Short rains) falls under Dyer season while May – June is in the Guu season (Rainy) and June – July is Hagaa season (cool and Cloudy). Data quality was checked for all data types: quantitative and qualitative. Screening, diagnosing, and editing of suspected data abnormalities was conducted and missing data omitted from analysis. Data analysis has also been conducted using mixed methods. Quantitative data has been analyzed using STATA software exported as Comma Separated Values (CSV) from the online platform. The most critical analysis techniques included frequency distributions with percentages to determine indicator values for the seasonal assessment and BMZ results matrices. Data has been disaggregated by district, gender, livelihood zones and disability. The data outputs are in the form of tables, charts, frequencies and cross tabulations. Qualitative data has been analyzed using the six steps recommended by Braun and Clarke’s approach5 ; familiarization with the data through reading and rereading transcripts and field notes; coding of the transcripts; searching for themes and grouping codes into potential themes; reviewed themes to create a thematic ‘map’; defining and naming themes and subthemes, following our study’s theoretical framework and analysis matrix and finally narrating the themes and sub-themes, using a selection of participant voices as exemplars of each theme. 4.0 Limitations The narratives of qualitative data indicated that minimal probing was done at the point of interviews. This meant that in-depth data analysis and appreciation of the respondents meaning’s attach to their experiences from exposure to project activities become skimmed. However, detailed report reviews have been used to argue out the changes which have been reported. Appreciation of indicator phrasing and calculation took longer than planned. While some indicators in both seasonal assessment results matrix and that of the BMZ Project required only basic frequency runs to obtain their values others required complex calculations as they were composite in nature; indices. The consultant and the SomRep project manager held several consultations for clarifications and were able to harmonize understanding out of which appropriate indicator values were derived from primary data collected from SomRep project beneficiaries. 5Braun, V. and V. Clarke, Using thematic analysis in psychology. Qualitative research in psychology, 2006. 3(2): p. 77-101.
SomReP Seasonal Assessment and BMZ Evaluation Report ⎯ Oct 2022 22 5.0 Findings The findings in this report have been presented based on comparisons and contrasts between the previous seasonal assessment and the current one as well as the baseline indicators values against values at midterm for the BMZ project. The comparisons and contrasts are district wide, gendered, livelihood zone specific and also disaggregated by disability. The focus is on trends, patterns, milestones, and cycles of occurrences illustrated by the meaning of indicator variances over time. 5.1 Seasonal Assessment Findings The seasonal assessment findings have been presented following the domains as per the results matrix. 5.1.1 Program Intervention Intensity Program intensity is the extent to which layering and sequencing occurs in SomReP program areas. SomRep applies layering and sequencing of interventions to build effective response capacity (absorptive, adaptive and transformative) of beneficiaries who face shocks of various kinds. High program intensity is desirable. This means more beneficiaries participating in more interventions within one or multiple partners and across sectors. The theory is that high program intensity is likely to lead to better resilience response to shocks. The comparison of indicator values for program intensity between Seasonal Assessment Reporting (SAR) 2021 and SAR 2022 shows that the proportion of households participating in one intervention has dropped from 72.5% to 70.3% and those participating in two interventions have increased to 20.5% from 13.5% while those participating in three interventions have increased to 9.2% from 5.7%. The largest increase in program intensity is 6.9% of those participating in two interventions as shown in figure 1 below. Figure 1: Variances in Program Intensity between Seasons Gender and disability differences on program intensity over time could not be computed for SAR 2022 since there were no comparison Values (Value Not Included - VNI) in SAR 2021. However, in SAR 2022 males were more among low intensity category (74.7%) compared to females (68.4%) but less in Medium (16.7% to 22.1%) and High (8.6% to 9.5%) categories. The 0.9% difference between males and Female on high intensity in SAR 2022 suggests that there is minimal gender disparity on high program intensity. This is a positive thing as the program seeks to achieve high program intensity for both males and females. Major differences are in the medium and low categories. Table 2 below shows that among the Agro pastoral program low intensity scores declined by 7.8% from 71.0% in SAR 2021 to 63.22% in SAR 2022 while rising by 9.6% in medium intensity from 14.0% in SAR 2021 to 23.6% in SAR 2022. On high intensity agro pastoral livelihoods posted an increase from 9.0% in SAR 2021 to 13.2% in SAR 2022. This is an increase of 4.2% The low intensity category of the 5.70% 15.80% 9.20% 0.00% 10.00% 20.00% 30.00% 40.00% 50.00% 60.00% 70.00% 80.00% SA-21 ARM-21 SA22 One Intervention Two Intervetions Three interventions
SomReP Seasonal Assessment and BMZ Evaluation Report ⎯ Oct 2022 23 Pastoral livelihoods was characterized by a rise of 0.8% from 77.0% in SAR 2021 to 77.8% in SAR 2022 and the medium intensity rose by 11.6% from 8.0% in SAR 2021 to 19.6% in SAR 2022 while the high intensity scores registered a decline of 0.4% from 3.0% in SAR 2021 to 2.6% in SAR 2022. The Peri urban livelihoods registered relatively large decline in the low intensity category posting a rise of 16.3% from 69.0% in SAR 2021 to 85.3% in SAR 2022 while experiencing a decline in the medium intensity scores by 9.2% from 19.0% in SAR 2021 to 9.8% in SAR 2022. The high intensity scores for Peri urban population marginally rose by 0.9% coming from 4.0% in SAR 2021 to 4.9% in SAR 2022. The agropastoral group showed more positive outcomes on program intensity with an increase of 4,2% in the high category followed by Peri Urban. The pastoral group registered a slight decline. In fact a lot more of them (16.3%) dropped to the low intensity category which is worrisome. Table 2: Gender Disability and Livelihood on Program Intensity Low Medium High Year of Reporting SAR21 SAR22 SAR21 SAR22 SAR21 SAR22 Gender Female VNI 68.4% VNI 22.1% VNI 9.5% Male VNI 74.7% VNI 16.7% VNI 8.6% Disability No Disability VNI 69.7% VNI 20.5% VNI 9.9% Yes Disability VNI 77.1% VNI 14.3% VNI 8.6% Livelihood Zones Fisher Folk VNI 100.0% VNI 0.0% VNI 0.0% Agro Pastoral 71.0% 63.2% 14.0% 23.6% 9.0% 13.2% IDP VNI 70.6% VNI 23.5% VNI 5.9% Pastoral 77.0% 77.8% 8.0% 19.6% 3.0% 2.6% Peri Urban 69.0% 85.3% 19.0% 9.8% 4.0% 4.9% Table 3 below provides program intensity across districts. It shows mixed results. The largest decline among the Low intensity segment was registered in Hargeisa at 66.7% dropping from 100% in SAR 2021 to 33.3% in SAR 2022. The largest decline in medium intensity was in Burcoat 28.2% (from 28.20% in SAR 2021 to 0.00% in SAR 2022) while the largest decline in High Intensity scores was in Dollow at 49.9% (From 51.10% in SAR 2021 to 1.20% in SAR 2022). The largest increase in low intensity was registered in Dollow of 42.9% (from 35.9% in SAR 2021 to 78.8% in SAR 2022) while the largest increase in medium intensity was seen in Hargeisa of 66.7% (from 0.0% in SAR 2021 to 66.7% in SAR 2022). The highest rise in High intensity was posted in CeelAfweyne of 22.7% (from 2.0% in SAR 2021 to 24.7% in SAR 2022). The movement toward better outcomes in program intensity is seen in CeelAfweyne followed by Xudur while negative outcomes are seen in Dollow and Eyle. The reasons for these outcomes were not immediately discernible but could be retrieved from further from additional qualitative data. Table 3: Program Intensity in Districts District Region Low Medium High SAR21 SAR22 SAR21 SAR22 SAR21 SAR22 Afgooye Lower Shabelle 87.30% 100.00% 6.30% 0.00% 6.30% 0.00% Baydhaba Bay 65.80% 65.50% 13.30% 18.80% 20.80% 15.70% Bossaso Bari 100.00% 0.00% 0.10% 0.00% Burco Togdheer 69.40% 100.00% 28.20% 0.00% 2.40% 0.00% CeelAfweyne Sanaag 83.70% 19.40% 14.30% 55.90% 2.00% 24.70%
SomReP Seasonal Assessment and BMZ Evaluation Report ⎯ Oct 2022 24 Doolow Gedo 35.90% 78.80% 13.00% 20.00% 51.10% 1.20% Eyl Nugaal 61.40% 82.40% 22.80% 17.70% 15.80% 0.00% Hargeysa WaqooyiGalbeed 100.00% 33.30% 0.00% 66.70% 0.00% 0.00% Laas_Caanood Sool 98.90% 95.80% 1.10% 4.20% 0.00% 0.00% Laasqoray Sanaag Lughaye Awdal 88.50% 72.00% 11.50% 28.00% 0.00% 0.00% Salahley Waqooyi_Galbeed 80.00% 20.00% 0.00% Xudur Bakool 82.10% 24.50% 11.90% 53.10% 6.00% 22.50% 5.1.2 Income The average incomes per month for beneficiaries have reduced by USD 17.65 between the last season (USD 80) and this one (USD 62.35). The reduction is highest among pastoral beneficiaries at USD 36.25 (from USD 102.0 to USD 65.75) followed by peri urban at USD 5.12 (from USD 80 to USD 74.88). Only agro pastoralists registered a marginal increase of USD 1.10 (from USD 59 to USD 60.1). Figure 2: Incomes between Seasons At baseline the reported average incomes had increased by 22% compared to the previous year (ARM report 2021) but for pastoral groups there was a reduction on incomes at that time from 47.3% to 22%. There has been a further drop this time also as shown above. Agro pastoral groups which had experienced a 28% increase of incomes at baselines have registered a 1.1% increase at midterm. The gender impact on incomes continues to favour men against women. In 2021 there was huge variation in the average incomes between men and women. The average income for women was USD156.9 while that of men was USD188.2. Men had far higher incomes in Crop USD102.4), livestock (USD40.3), and agriculture related business (USD10.7) compared to women. Women on the other side had higher incomes in Non-farm-related business (USD7.9) and Cash transfers (USD23.1). Table 4 below shows that gender, disability and mean incomes performed worse among females and not disabled persons between the seasons. For Females it dropped by USD 96.6 and for males by 122.90. The largest drop was among people without disability by USD 19.50 compared to a mere 2.20 drop among the disabled.( This could be attributed to the disabled being categorized as more vulnerable hence priority in cash transfers which cushions them in times of shocks compared to people with Disability. In this season more women have joined VSLAs compared to men. A quick comparison of VSLA gender composition in the seasonal assessment FGDs shows a male to female proportion of 1: 3 (24.2% to 75.8%). Males in one of the districts desired to join an existing all female VSLA $80 $91.40 $62.35 $0 $20 $40 $60 $80 $100 $120 $140 SA-21 ARM-21 SA-22 AV Mo income - Primary Source AV Mo income - Pastoral AV Mo income - Peri Urban AV Mo income - Agro Pastoral
SomReP Seasonal Assessment and BMZ Evaluation Report ⎯ Oct 2022 25 and were denied access to membership on grounds that males “do not abide by rules of groups and may bring losses”. It has been argued elsewhere that “supporting women directly through interventions such as the VSLAs is helping to reduce the differences and give women economic power.”(ARM 2021) This remains to be seen in the SomRep intervention model. Incomes were also registered in this season through TVET graduates who got employed after graduation, “some of us got work. In the government sector and the SO company. One TVET trainee said he earns $150 a month. Yes, the salary is commensurate with our skills in terms of location and where I live.”Salhley (TVET FGD - DRC). An all female FGD in Bossaso; decried the severe drought on incomes saying, “Business has been affected by drought and high inflation in the country, i needed to learn customer satisfaction, income management, how to saving.” In an effort to diversify opportunity and possibly income sources. This was also echoed in Salhley, “As drought is severe in our areas it is hard to earn some incomes. But We should be given more training in the areas of cooking and tailor to increase the skills” Table 4: Mean incomes between Seasons Gender and Disability Income Variances Reporting Year SAR21 SAR22 Gender Female $156.9 $60.0 ($96.90) Male $188.2 $66.0 ($122.20) Disability No Disability $82 $62.5 ($19.50) Yes Disability $77 $74.8 ($2.20) Mean income across the districts posted mixed results. Table 5 below shows that largest increase in mean income was in Bosasso at USD 188 rising from USD 22 in SAR 2021 to USD 210.00 in SAR 2022. The largest loss in mean income was seen in Dollow of USD 85.50 declining from USD 131 in SAR 2021 to USD 45.5 in SAR 2022. In Burco there was a small increment in mean income by USD 6.40. Other losses in mean income values were registered in Hargeisa at USD 11.70 coming from a high of USD 150 in SAR 2021 to USD 138.30 in SAR 2022. In Xudur the mean income also declined by USD 24.20 coming from USD 44.00 in SAR 2021 to USD 19.80 in SAR 2022. Table 5: Mean Incomes across Districts District Region Mean income SAR 2021 SAR 2022 Afgooye Lower Shabelle VNI $57.1 Baydhaba Bay VNI $39.5 Bossaso Bari $22 $210.8 Burco Togdheer $71 $77.4 Ceel-Afweyne Sanaag VNI $66.1 Doolow Gedo $131 $45.5 Eyl Nugaal $105 $106.5 Hargeysa WaqooyiGalbeed $150 $138.3 LaasCaanood Sool VNI $122.3 Laasqoray Sanaag VNI $71 Lughaye Awdal VNI $77.8 Salahley Waqooyi_Galbeed VNI $65.4 Xudur Bakool $44 $19.8
SomReP Seasonal Assessment and BMZ Evaluation Report ⎯ Oct 2022 26 Total Average $XXX $62.4 5.1.3 Expenditure Table 6 below shows that indebtedness increased by 11.42% over the seasons from 55.4% last season to 66.8% this season. This was highest for agro pastoral beneficiaries at 19.3% (from 42.2% to 61.5%) followed by peri urban beneficiaries at 18.6% (from 56% to 74.6%) and lastly the pastoral at 7.8% (from 65.7% to 73.5%). Most of the debts have been incurred for basic sustenance on food. In between seasons after baseline, most beneficiaries have been unable to repay their debts. The general feeling among VSLA FGDs such as the one in BIYO CADDO was that saving has been made difficult by the triple shocks and inflation. They therefore have opted for indebtedness for survival. Those who took debts for livestock farming have sunk deeper into debts. They have been unable to repay existing debts as their livestock fortunes have plummeted with increasing drought situation negatively affecting the markets and the demand to meet their basic needs becoming even more burdensome. The BMZ funded project has moderated this situation through contingency and social funds under SAC and VDC which manage crisis as modifiers at community level. These funds have helped the beneficiaries during severe crisis impacts. Additionally the project incorporates a crisis modifier contingency fund where SomRep members can request external resources when the shock overwhelms the communities in the project proposal. Table 6: Expenditure by livelihoods Indicators SA-2021 ARM-2021 SA 2022 Variance % Incurred a debt 55.4% 23.8% 66.8% 11.4% % Incurred a debt (Pastoral) 65.7% 24.3% 73.5% 7.8% % Incurred a debt (Peri-urban) 56% 26.2% 74.6% 18.6% % Incurred a debt (Agro-pastoral) 42.2% 22.7% 61.5% 19.3% The SAR 2021 recorded debts incurred was highest in Sool (Laas Caanood), Sanaag (CeelAfweyne) and Togdheer (Burco) with 18%, 17% and 12.9% respectively. The proportions for the rest of the districts (regions in SAR 2021) are not included. In SAR 2022 the general trajectory of the debt indicators based on available SAR 2021 data shows a significant rise. Incurring debt increased by 76.5% in Togdheer, 40.3% in Sanaag and 55.3 in Laas Caanood. In SAR 2022 64.6% of Males incurred debt while 68.0% of Females did so. For PLWDs debt incurred was 65.4%. The SAR 2021 did not indicate these values (VNI) by gender and disability. Therefore no comparison was made on indebtedness by these variables. Table 7: Expenditure by the project districts District Region Incurring Debt SAR 2021 SAR 2022 Afgooye Lower Shabelle VNI 69.2% Baydhaba Bay VNI 62.9% Bossaso Bari VNI 60.0% Burco Togdheer 12.9% 89.4% Ceel-Afweyne Sanaag 17.0% 72.3% Doolow Gedo VNI 82.4%
SomReP Seasonal Assessment and BMZ Evaluation Report ⎯ Oct 2022 27 55.2% 71.2% 43.7% 40.9% 73.5% 71.6% 72.3% 75.5% 18.3% 0.4% 28.6% 34.6% HHs exposed to shocks and hazards Pastoral HH Exposed Peri-urban HHs Exposed Agro-pastoral HH Exposed Shock Exposure SA-2021 SA 2022 Variance Eyl Nugaal VNI 85.3% Hargeysa WaqooyiGalbeed VNI 16.7% LaasCaanood Sool 18.0% 58.3% Laasqoray Sanaag VNI 66.1% Lughaye Awdal VNI 56.0% Salahley Waqooyi_Galbeed VNI 40.0% Xudur Bakool VNI 30.6% Total VNI 62.4% 5.1.4 Shocks The number of households that have been exposed to shocks and hazards have increased between the seasons by 18.3% (from 55.2% to 73.5%). More agro pastoral households experienced this increase by 34.6% (from 40.9% to 75.5%) followed by Peri-urban households at 28.6% (from 43.7% to 72.3%). The least exposed households were Pastoral at merely 0.4% (from 71.2% to 71.6%) yet they too felt the impact of the continuing drought conditions as was observed in one of the FGDs, “During these days due to drought we have witnessed decrease in selling animals” (Livestock FGD in Dayah (COOPI) Figure 3: Shock Exposure during The Seasons Recent reports on worsening drought places the seasonal scenarios at an emergency level with an increased possibility of famine to affect about 2.1 million people mostly pastoral and agro pastoral livelihoods6 . The BMZ project areas are particularly most concerning in this regard as critical thresholds of child wellbeing outcomes have been breached chiefly the acute malnutrition threshold for famine. It was reported that in pastoral areas, water and pasture shortages have forced pastoralists to migrate to distant grazing areas. Poor pastoralists in many areas were unable to cope with the rising costs of water and food, especially when they were already experiencing a significant decline in saleable animals due to distressed sales, weak/emaciated body conditions, and excess deaths. With fewer livestock births expected, reduced income from livestock sales, and low availability of milk for both adults and children, poor pastoral households faced moderate to large food consumption gaps through June 2022. By the time of the last study, Somalia was 6 Somalia: Acute Food Insecurity Situation May 2022 and Projection June to September 2022
SomReP Seasonal Assessment and BMZ Evaluation Report ⎯ Oct 2022 28 experiencing a total failure of 2021 Gu rains, and drought conditions were at the peak with populations across the agriculturalist zone of central moving to IDP camps in Baidoa. This situation has persisted to this day. At district levels beneficiary households exposed to shocks between SAR 2021 and SAR 2022 showed a spike in the figures. While those exposed to shock in Afgooye was at 17.5% at the time of reporting last season, this had shot to 71.9% in SAR 2022; a 54.2% rise. Those exposed to Shock in Doolow reduced by 21.3% from 93.1% in SAR 2021 to 71.8% in SAR 2022. There was another spike in proportion of households exposed to of 36.9% in Lughaye district as the indicator value rose from 23.1% in SAR 2021 to 60.0% in SAR 2022. Households exposed to Shocks in the other districts were not indicated in SAR 2021 and no comparison therefore. The general trend however was that most households in project areas had been significantly exposed to shocks since SAR 2021. Gender differences were not recorded in SAR 2021 but were 69.9% for males and 75.2% for females in SAR 2022. Disability headed household suffered a 79.5% proportion of those exposed to shock in SAR 2022. In SAR 2021 this was not included in reporting and no comparisons have been made in this regard. Table 8: Proportion of Households exposed to shocks. District Region Shock Exposure SAR 2021 SAR 2022 Afgooye Lower Shabelle 17.7% 71.9% Baydhaba Bay VNI 60.7% Bossaso Bari VNI 90.0% Burco Togdheer VNI 100.0% Ceel-Afweyne Sanaag VNI 97.9% Doolow Gedo 93.1% 71.8% Eyl Nugaal VNI 47.1% Hargeysa WaqooyiGalbeed VNI 100.0% LaasCaanood Sool 100% 91.7% Laasqoray Sanaag VNI 55.4% Lughaye Awdal 23.1% 60.0% Salahley Waqooyi_Galbeed VNI 100.0% Xudur Bakool VNI 83.7% 5.1.5 Shock Exposure Index In the SomReP program resilience means the ability of people, households, communities, and systems to mitigate, adapt, and recover from shocks and stresses in a manner that reduces chronic vulnerability and facilitates inclusive growth. The shock exposure index is one of the composite measures SomRep uses to determine the overall degree of shock exposure for each household and the perceived severity of the shock on household income and food consumption. The index is a weighted average of the incidence of each shock between a value of one (1) if the shock was experienced and zero (0) if none, weighted by the perceived severity of the shock measured based on a Likert scale of 1 to 4 where 1=no impact, 2 =moderate impact, 3=strong impact and 4=worse-ever impact. A higher mean shock exposure index means more people experienced shocks in the season of assessment with debilitating impacts on their resilience capacities. The figure 4 below shows that overall, the shock exposure index increased by 8.7 in the program season from 1.5 (SAR 2021)to 10.4.(SAR 2022) The increase was highest among agro pastoral beneficiaries by 8.2 (from 1.8(SAR 2021) to 10.0) (SAR 2022 and lowest among peri urban livelihood zones at 5.9 (from 4.1(SAR 2021) to 10.0). (SAR 2022) The pastoral community registered an increased mean of 7 (from 2.8 (SAR 2021) to 9.8).(SAR 2022) An FGD with NRM in Qoryaale stated
SomReP Seasonal Assessment and BMZ Evaluation Report ⎯ Oct 2022 29 “We don’t feel that we have matured because of the droughts and lack of economics.” Suggesting need for more time for community to develop capacity to respond to shocks. Even with these shocks most of the BMZ funded activities continued and the crisis modifiers helped cushion beneficries in the two districts of Afgoye and Baidoa. Figure 4: Shock Exposure Index Figure 5 below provides the results of the variances of shock exposure indices for each district. The highest variance is in Xudur at 17.6%and the least in Eyl at -1.2%. While the shock exposure index in Xudur is a spike, in Eyl it is a drop. Half of the project locations have indices of above 10. Figure 5: Shock Exposure Index The gender differences on shock exposure index were 10.6 for females and 10.3 for males in SAR 2022 showing near similar degree of exposure for both genders. The shock exposure index was 9.3 for persons living with disability slightly lower than for males and females and 1.5 9.1 8.3 9.8 9.1 10.4 9.8 10 10 0 2 4 6 8 10 12 Overall Mean Shock Exposure (MSE) index MSE index - Pastoral Overall mean Shock exposure index (Periurban) Overall mean Shock exposure index (Agropastoral) SA2021 ARM-21 SA2022 0 2 4 6 8 10 12 14 16 18 20 SAR 2021 SAR 2022
SomReP Seasonal Assessment and BMZ Evaluation Report ⎯ Oct 2022 30 suggesting a possible impact of crisis modifiers in targeting of this group of beneficiaries. The values of these scores were not included (VNI) in the SAR 2021. 5.1.6 Access to Water The Water indicator values were computed on time taken on a round trip to a water source measured between a time less than 30 minutes and more than an hour. Table 6 below shows that overall, the proportion of household reporting water point to be less than 30 minutes away reduced by 7.0% (from 59.9% to 52.9%). Those reporting water point to be 30 minutes to 1 hour away increased by 5.2% from 25.5% to 30.7% while the ones spending more than 1 hour to a water point increased by2.2% from 14.2% to 16.4%. Over the reporting period between SAR 2021 and SAR 2022 water has become more inaccessible. Table 9: Access to Water Accessed at SA2021 SA2022 Variance Less that 30 minutes away 59.9% 52.9% -7.0% 30 minutes to 1 hour away 25.5% 30.7% 5.2% More than 1 hour away 14.2% 16.4% 2.2% The gender dimension of access to water has not been compared for lack of baseline reference values from SAR 2021. In SAR 2022 females were more prominent in all the three aspects than males. In the “less than 30 Minutes” access females were 66.6% while Males 33.4%. Within 30 Minutes to one-hour females were 69.2% while males were 30.8%. More than an hour saw females register a score of 60.3% and males 39.7%. Whereas females continue to bear a heavy burden in regards to accessing water, the most burdensome in SAR 2022 was between 30 minutes to one hour time cycle. In the BMZ project areas a lot of water tracking is done alongside bore holes and fetching from river Shebelle (which incidentally is drying up).The BMZ funded project targeted a 20% increase in proportion of targeted households who report all year access to water (for irrigation agricultural production & domestic use) as a result of investment in large scale multi-use water infrastructure and small scale water infrastructure by 2023. As at midterm the project had registered a 5% increase. Out of the 9 water infrastructure to be rehabilitated/upgraded only 3 had been completed. The result is due to delays in implementation caused largely by an emergent need on the ground in Afgoye for a local partner to be involved in the large scale multi-use water infrastructure and small scale water infrastructure Rehabilitation, upgrade and establishment. Finally WARDI; a local partner has been identified and included in the deliverable and work is scheduled to commence as designed. In other areas within Baidoa, three producer groups invested in construction of elevated water tanks for irrigation, irrigation pumps, and motor cycle to ease in transportation of produce to the main market. These farmer groups reported that the water availability has increased and they are now irrigating their farms with the hope to increase production of fruits and vegetables. In Afgoye, the project rehabilitated a total of four secondary irrigation canals with a combined total length of 9.5 kilometers to channel water to the farms from the river. The SPHERE standards recommend that for a household to have improved access to water, the round trip should be not more than half an hour7 , thus there are still regions that have gaps in improved access to water considering this standard. But the significant thing about outputs on access to water is that the SAR2022 data was collected just after the rainy season and yet access to water is 7 https://handbook.spherestandards.org/en/sphere/#ch006_004
SomReP Seasonal Assessment and BMZ Evaluation Report ⎯ Oct 2022 31 still a challenge. The implication for natural resources management (drought), climate action (failed rains) as well as water infrastructure delivery (delayed) cannot be overstated here. 5.1.7 TVET The proportion of households reporting to have been recently trained in a skill fell by 0.8% from 12% in SAR 2021 to 11.2% in SAR 2022. While the proportion of trained TVET beneficiaries who graduated had increased by 8.7% (from 62.3% to 71.0%) those who reported having been employed fell by 4.7% from 30.1% in SAR 2021 to 25.4% in SAR 2022. The proportion of those who had started a business also declined and was by 9.1% (from 30.2% to 21.1%). In SAR 2021 tailoring emerged as the most accessed skill with 38.4% followed by Henna (25.3%), electrician (19.2%) and Barber at 13.1% (page 30 Jilaal SAR 21). In SAR 2022 some of the courses taught in TVET included effective marketing, cooking, tailoring and computer skills, including hardware and software. Although the proportion of TVET graduates who had graduated and been formally employed or started a business dropped between the seasons, there was promise for self-employment and formal employment over time. In ARM 2021 it reported that one of the graduates; RA (name withheld for anonymity) begun tailoring business and is earning USD 10 per day and has been able to enroll her daughter to school and pay her sustenance bills (see page 59, ARM 2021). In a 2022 FGD a TVET graduate started his own welding business in oog village after failing to gain employment. “I have graduated, certificate was given and its recognized certificate, also I was given the start -up kit which I still use whenever I am working on welding. I started welding business in my village oog, unfortunately I have not found a job, but i work independently and my personal income is not fixed but I can say I make around $100 monthly. I would like SomRep to expand their training and include electricity trainings, health trainings, and also plumbing training which would increase self-employment in this area.” Figure 6: TVET Outcomes 5.1.8 Agriculture The proportion of beneficiaries owning land increased by 24.3% between the assessment seasons rising from 37.9% in 2021 to 62.2% in 2022. There was a marked reduction of average land under cultivation from 2 hectares in 2021 to 1.3 Ha in 2022; a 35.0% value. Reduction was further witnessed in the proportion of households trained in Good Agriculture Practices. This reduced by 34.6%% from 69.1% to 34.5%. It was observed during FGDs that skills training needed to be put into practice and this requires tools, “Some of the skills received in the GAPS training do not work without tools and facilities. The practices need Tools and we do not have enough tools like wheel barrow, Axes, weeding tools, seeds (cabbage seed) and fence of predators.” (Farmer Group/Association FGD in Badhan). In BMZ funded districts farmer groups who received training accessed loans which they used 12.0% 62.3% 30.1% 11.3% 71.0% 25.4% 0.0% 10.0% 20.0% 30.0% 40.0% 50.0% 60.0% 70.0% 80.0% Recently trained in a skill TVET attendees graduated TVET attendees formally employed SA2021 SA2022
SomReP Seasonal Assessment and BMZ Evaluation Report ⎯ Oct 2022 32 24.3% -70.0% -34.6% % of households owning land Average land under cultivation (hectares) % of households trained in Good Agriculture Practices Agricultural Results Variance 2022 SA-21 for water tanks construction and irrigated their farms for vegetable and fruit farming. This difference is significant in that the BMZ funded program design links beneficiary participation in training to application of knowledge received. Figure 7: Agricultural Outcomes The proportions of households with average farmland holding of three and above hectares per household declined by 5.1% from 20.1% to 15.0%. The average farmland holding per household with less than one hectare dropped by 1.1% from 46.7% to 45.6% while that of households with one to two hectares increased by 6.2% from 33.2% to 39.4%. Of the 2 hectares average land under cultivation in SAR 2021, pastoralists were 2.3%, agro pastoralists 82.5% and peri urban were 31.3%. The rest of the SAR 2021 reporting on agricultural practices was not aligned to the indicator definitions, descriptions and values required in SAR 2022. In SAR 2022 the gender variable in average land cultivation showed females cultivated more land (1.1 Ha) than males (1.0 Ha). The livelihood variable on the other hand showed agro pastoralists cultivated on average more land (1.1 Ha) followed by pastoralists (0.8 Ha) and peri urban (0.8 Ha). Figure 8: Agricultural Outcomes Comparisons Gender and disability variables on land ownership were not reported in SAR 2021. However in ARM 2021 (pg 70) it was reported that land ownership among the communities of project implementation is culturally defined and managed by males more than females. Land ownership within livelihood zones indicates shifts between SAR 2021 and SAR 2022. Table 10 below shows that among pastoralists land ownership had declined from 43.3% to 19.1% 46.7% 33.20% 20.10% 45.60% 39.44% 14.96% 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% Average farmland holding per household (less than one hectare ) Average farmland holding per household (One to two hectares ) Average farmland holding per household (Three and above hectares ) SA21 SA22
SomReP Seasonal Assessment and BMZ Evaluation Report ⎯ Oct 2022 33 which is a loss by 24.20% while it had increased among agro pastoralist from 42.4% to 60.5%. The increase is by 18.10%. The Peri urban livelihood has also lost land ownership by 5.70% reducing from 25.4% in SAR 2021 to 19.7% in SAR 2022. It is interesting that owning more land does not necessarily translate into cultivation. At baseline it was reported land use is more pastoralist dominated than rain fed agriculture as the livestock industry employs over 60% of the population and livestock earnings account for over 80% of Somalia’s foreign exchange earnings (ARM report 2021, pg 38). Other land uses include irrigated agriculture and forestry. It is therefore possible that while pastoralists sell off their livestock and land to fend off looses due to devastating drought and increase incomes to survive the shocks, the agro pastoralists who buy land are unable to quickly transform the traditional livestock economy into other agricultural economies for lack of capital, skills, tools and low market demand. Table 10: Landownership by Livelihood Zones Livelihood Zone Land Ownership SAR 2021 SAR 2022 Variance Pastoral 43.3% 19.1% -24.20% Agro Pastoral 42.4% 60.5% 18.10% Peri Urban 25.4% 19.7% -5.70% 5.1.9 Livestock Figure 9 below shows that the proportion of households owning livestock dropped by 6.7% from 66.5% in 2021 to 59.8% in 2022. From discussions with beneficiaries it was noted that the severe drought had affected most livestock farmers in dropped sales and loss to diseases as was noted in the livestock FGD in BIYO CADDO (ACF), “The Main challenge is drought. For the beginning about 30 farmers have witnessed increase in sales but as for now with the loss of large animals due to drought there are no sales or production. We would like to support us in terms of improving the awareness animal treatment and assisting us the safety and feeding the animal amid this severe drought. We need practical training to understand animal diseases.” Figure 9: Livestock ownership The mean income earned from livestock was eroded by USD 64 declining from USD 79 to USD 15. In the ARM report 2021 (pg 68), it was reported that only 22.0% of the respondents 66.5% 59.8% 56.0% 58.0% 60.0% 62.0% 64.0% 66.0% 68.0% SA2021 SA2022 HH owning livestock
SomReP Seasonal Assessment and BMZ Evaluation Report ⎯ Oct 2022 34 earned income from livestock compared to 47.3% in 2021 suggesting that the devastation of the shocks were more significant on livestock farming. Figure 10: Mean Earnings from Livestock 5.1.10 Markets The proportion of households reporting on market indicators showed mixed results. While those within a 30 minutes walking access to markets decreased by 5.5% (from 49.9% to 44.4%) households within 30 minutes to one hour increased by 2.3% from 30.4% to 32.7%) and those who took more than one hour increased by 2.7% (from 19.6% to 22.3%). Table 11: Markets by distance Indicators SA2021 SA 2022 Variance 30 minutes Distance to market 49.9% 44.4% -5.5% 30 minutes to 1 hour Distance to market 30.4% 32.7% 2.3% More than an hour Distance to market 19.6% 22.3% 2.7% Commodity prices in the markets received highly dissatisfactory evaluation as more beneficiaries perceived them as poor by 41.1% up from 27.0% to 68.1%. Those who felt the prices were good declined by 6.3% (from 37.1% to 30.8%). The most significant drop was at 34.8% by those who indicated fair pricing (down from 35.9% to 1.1%). In general beneficiary populations in the project areas perceive commodity prices as high and in some cases prohibitively so. Figure 11: Markets commodity price $15.00 $79 $0.00 $20.00 $40.00 $60.00 $80.00 $100.00 SA2022 SA2021 Mean income earned from livestock
SomReP Seasonal Assessment and BMZ Evaluation Report ⎯ Oct 2022 35 5.1.11 Food Security Households with poor consumption score reduced by 4.2% from 18.7% to 14.5% between the SAR 2021 and SAR 2022. Borderline food consumption marginally increased by 3.0% from 25.7% to 28.7%. Acceptable Food Consumption Score saw the smallest increment at 1.3% up from 55.6% to 56.9%. A registration of 1.3% on acceptable food consumption score indicates better outcome on the indicator even though at a minimal level. Given the severity of the shocks in the project areas this is a positive thing. The 3.0% increase in borderline consumption against the 4.2% decline in poor consumption is a significant indication to the positive aspect of project action on food consumption among beneficiaries. Table 12: Food Security Overall Indicators SAR2021 SAR2022 Variance Poor Food Consumption Score 18.7% 14.5% -4.2% Borderline Food Consumption Score 25.7% 28.7% 3.0% Acceptable Food Consumption Score 55.6% 56.9% 1.3% Table 13 below shows results on FCS by gender, disability and livelihood zones. The female gender performed well compared to males in all three levels of food consumption in SAR 2022. The SAR 2021 did not include values for gender, disability and livelihood for this indicator except only on acceptable consumption for disability (from 53.6% in SAR 2021 to 62.3% in SAR 2022) and agropastoral (from 67.8% in SAR 2021 to 56.7% in SAR 2022) and also pastoral (from 55.2% in SAR 2021 to 64.2% in SAR 2022). The agropastoral beneficiaries experienced an 11.1% decline in acceptable food consumption while pastoral experienced a 9.0% improvement. People with Disability also experienced better food consumption that those without as shown in the table 13 below. Table 13: Food Consumption score – Gender Disability and Livelihoods Poor Borderline Acceptable Reporting Year SAR21 SAR22 SAR21 SAR22 SAR21 SAR22 Gender Female VNI 13.0% VNI 29.5% VNI 57.5% Male VNI 17.3% VNI 27.1% VNI 55.6% Disability No Disability VNI 15.7% VNI 30.4% VNI 53.9% 37.1% 35.9% 27.0% 30.8% 1.1% 68.1% Commodity price evaluation (Good) Commodity price evaluation (Fair) Commodity price evaluation (Poor) SA2021 SA2022
SomReP Seasonal Assessment and BMZ Evaluation Report ⎯ Oct 2022 36 Yes Disability VNI 14.3% VNI 23.4% 53.6% 62.3% Livelihood Zone Fisher Folk VNI 0.0% VNI 0.0% VNI 100.0% Agro Pastoral VNI 12.9% VNI 30.5% 67.8% 56.7% IDP VNI 35.3% VNI 23.5% VNI 41.2% Pastoral VNI 7.8% VNI 27.9% 55.2% 64.2% Peri Urban VNI 23.1% VNI 26.2% VNI 50.8% The FCS within the specific districts has shown mixed results. Table 11 below presents the variances in proportions for each category of FCS between SAR 2021 and SAR 2022. The values in the negative indicate that over time the district has experienced a decline among beneficiaries in that category of score. Positive values indicate an increase. Hargeisa experienced no change in its FCS across all the categories. The largest negative change in poor consumption score was found in Dollow (-23.8%) followed by Lughaye(-23.1%)and Laas_Caanood(-10.3%).The largest negative change in borderline FCS was witnessed in Lughaye(-41.8%) followed by Xudur(-39.5%) and Bossaso (-33.3%). Negative change in Acceptable FCS was largest in Eyl (-55.4%)followed by Baidoa(-23.8%) and Burco(-17.8%). The largest positive change in FCS was experienced in Salahley (100%) and Laasqoray (71.4%)and Lughaye (64.8%). Table 14: Food Consumption and Livelihood zones - Differentials District Poor Borderline Acceptable Afgooye -0.20% 3.50% -3.4% Baydhaba -1.00% 24.90% -23.8% Bossaso 21.10% -33.30% 12.2% Burco -8.40% 26.20% -17.8% CeelAfweyne -4.80% -7.10% 11.8% Doolow -23.80% 6.70% 17.2% Eyl 15.90% 39.50% -55.4% Hargeysa 0.00% 0.00% 0.0% Laas_Caanood -10.30% 6.90% 3.4% Laasqoray 1.80% 26.80% 71.4% Lughaye -23.10% -41.80% 64.8% Salahley 0.00% 0.00% 100.0% Xudur 32.70% -39.50% 6.80% 5.1.12 Household Hunger Scale The Household Hunger Scores (HHS) show declining performance overall between the baseline and midline. Figure 12 below gives interesting scores on Household Hunger. Households with Little to no hunger have reduced by 7.5% between SAR2021 and SAR2022. This is a concern for beneficiary wellbeing and programming. It is however very important to note also that the ARM2021 which comes later than SAR2021 shows better outcomes on this with a 3.8% increment on households with little no huger. These are likely effects of project interventions showing a cushioning element to household hunger. The trend on moderate hunger also shows this better outcome between SAR2021 and ARM2021 with a dip of 11.5% of households experiencing moderate hunger. And even though overall
SomReP Seasonal Assessment and BMZ Evaluation Report ⎯ Oct 2022 37 severe hunger scores show an increase of 9.6% between SAR2-021 and SAR 2022 this is light in scope and intensity at programming levels and is moderated by crisis modifiers as has been noted elsewhere. At the program inception (ARM 2021) it seems a lot more efforts were injected to respond to wellbeing status of beneficiaries. This can be felt in the better scores on household hunger just after project baseline. Figure 12: Household Hunger Scale The gender, disability and livelihood dimensions of HHS are shown in table 14 below. Although comparable data between the seasons on gender is unavailable results of the SAR2022 shows that females are worse off than males on this indicator for wellbeing. While less females (56.0%) have little to no hunger compared to males (60.5%) more of them (31.7%) experienced moderate hunger compared to males (28.1%). This is a differential of 4.5% and 3.6%. Also more females (12.3%) compared to Males (11.4%) had severe hunger; a 0.9% difference. The story of the differentials from 4.5% to 3.6% to 0.9% signify that females are closing the gap on males on household hunger score. It affirms the efforts of project to target and invest in females among vulnerable populations. On disability comparable data existed only for the No to Little Hunger category which was 55.8% in SAR 2021 which changed to 50.0% in SAR 2022 (this is a positive change). Comparable data also existed for livelihood dimension only for No to Little Hunger category with the Agropastoral changing (from 57.9% in SAR 2021 to 55.0% in SAR 2022). This is a negative change for agro pastoralist. Pastoralist also experienced a negative change on littler to no hunger (from 72.2% in SAR 2021 to 64.2% in SAR 2022). The Peri Urban households also suffered the same experience with little to no hunger scores of 68.2% in SAR 2021 to 56.9% in SAR 2022). This elevated hunger situation on livelihoods has affected the pastoralists more followed by IDPs and Peri urban. For the livelihood groups, sensitivity to shocks has meant a more negative experience. Table 15: Gender disability and livelihood dimensions of HHS No or Little Hunger Moderate Hunger Severe Hunger Year of Reporting SAR21 SAR22 SAR21 SAR22 SAR21 SAR22 Gender Female VNI 56.0% VNI 31.7% VNI 12.3% Male VNI 60.5% VNI 28.1% VNI 11.4% Disability No Disability VNI 58.5% VNI 30.5% VNI 11.1% Yes Disability 55.8% 50.0% VNI 26.9% VNI 23.1% 65.8% 69.6% 58.3% 32.5% 21.0% 29.9% 1.7% 9.4% 11.3% SA21 ARM21 SA22 Little to no hunger Moderate hunger Severe hunger
SomReP Seasonal Assessment and BMZ Evaluation Report ⎯ Oct 2022 38 Livelihood Zone Fisher Folk VNI 100.0% VNI 0.0% VNI 0.0% Agro Pastoral 57.9% 55.0% VNI 32.7% VNI 12.4% IDP VNI 52.9% VNI 37.3% VNI 9.8% Pastoral 72.2% 64.2% VNI 26.0% VNI 9.8% Peri Urban 68.2% 56.9% VNI 27.7% VNI 15.4% The HHS across the districts showed mixed results in all the categories of measurement. The proportions of those with Little to No Hunger declined most in Eyl (-47.8%), Xudur (- 23.5%) and Ceel Afweyne (-22.3%) indicating increased hunger situation in these districts. The highest increase in those with Little to No Hunger was in Burco (71.8%), Laasqoray (64.3%) and Salahley(40.0%). Among those with Moderate Hunger the largest drop was experienced in Burco (-68.5%) and Doolow (-26.9%) and also in Bossaso (-10.6%). Severe Hunger scores reduced only Burco (-3.2%) and Laas_Caanood (-1.3%) but rose in most of the districts with Eyl leading at 29.4% followed by CeelAfweyne at 19.2% and Baidoa at 17.0%.. The general hunger situation across the districts had been sensitive and affected negatively by the shocks as has been shown in the findings on expenditure in which most debt is incurred on food and agro pastoralists acquiring more land but unable to cultivate it. From the SAR21 and SAR22 results it appears that the household hunger situation is more sensitive to seasonality more than programme intervention intensity. But this can be ascertained more clearly if data seasonal data collection is conducted consistently in each season. Table 16: Program District and Household Hunger Category – Differentials District No or Little Hunger Moderate Hunger Severe Hunger Afgooye -0.9% -2.9% 3.8% Baydhaba -19.3% 2.3% 17.0% Bossaso 0.6% -10.6% 10.0% Burco 71.8% -68.5% -3.2% Ceel Afweyne -22.3% 3.1% 19.2% Doolow 21.7% -26.9% 5.3% Eyl -47.8% 18.4% 29.4% Hargeysa -16.7% 16.7% 0.0% Laas Caanood -22.1% 23.3% -1.3% Laasqoray 64.3% 19.6% 16.1% Lughaye -12.0% 12.0% 0.0% Salahley 40.0% 60.0% 0.0% Xudur -23.5% 5.8% 17.7% 5.1.13 Reduced Coping Strategy Reduced coping Strategy scores show mixed results in specific indicators but a positive trend in general. The proportion of households with Low or No Coping declined by 5.5%from 21.1% in SAR 2021 to 15.6% in SAR 2022. This is a negative thing. The numbers under this category should be increasing. The proportion of those in medium coping category also declined by 7.2% from 36.4% in SAR 2021 to 29.2% in SAR 2022. This also is tending towards a negative situation. The High Coping category of households experienced the highest decline of 18.2% down from 43.5% to 25.3%. This reduction within the indicator is highly positive. The magnitude of change at 18.2% is larger than that of Low to no coping (5.5%) and medium coping (7.2%) combined. These are shown in table 17 below.
SomReP Seasonal Assessment and BMZ Evaluation Report ⎯ Oct 2022 39 Table 17: Household Coping Measurements Indicators SAR2021 SAR2022 Variance Low or no coping 21.1% 15.6% -5.5% Medium Coping 36.4% 29.2% -7.2% High Coping 43.5% 25.3% -18.2% The household coping scores disaggregated by gender, disability and livelihood are shown in table 17 below. As with a number of indicator values for comparison in this study, the relevant comparable data between the seasons on gender and livelihoods as designed in the assessment measurement was unavailable. In SAR2022 there were less females (13.7%) compared to males (18.7%) experiencing little to no coping. For females this is negative. However fewer females (23.3%) compared to Males (30.1%) experienced high coping. This is much better for females compared to males. High coping is undesirable and less females are in that category. On disability comparable data existed only for the low coping category which reduced from 23.1% in SAR 2021 to 17.5% in SAR 2022; a 5.6% reduction. This is a negative outcome among persons with disability between the seasons. However compared to persons without disability under this category they were better off at 17.5% to 13.8%. Fewer persons without disability are experiencing low to no coping. Generally, most households were in the Medium Coping category in SAR 2022. This shows a movement from high coping towards low coping resting more at medium coping during the season. This trajectory is a positive thing and its taking time to mature into low or no coping. The IDP (31.4%) and Agropastoral (26.8%) peri Urban (23.9%) households showed elevated levels of high coping compared to the other livelihoods. The Pastoral livelihood group showed better coping than others (20.1%). Table 18: Household Coping by gender, disability and livelihood Reporting Season Low Coping Medium Coping High Coping SAR21 SAR22 SAR21 SAR22 SAR21 SAR22 Gender Female VNI 13.7% VNI 62.8% VNI 23.3 % Male VNI 18.7% VNI 51.2% VNI 30.1 % Disability No Disability VNI 13.8% VNI 59.3% VNI 26.9 % Yes Disability 23.1% 17.5% VNI 65.0% VNI 17.5 % Livelihood Zones Fisher Folk VNI 0.0% VNI 40.0% VNI 60.0 % Agro Pastoral VNI 14.3% VNI 58.9% VNI 26.8 % IDP VNI 15.7% VNI 52.9% VNI 31.4 % Pastoral VNI 17.2% VNI 62.8% VNI 20.1 % Peri Urban VNI 18.5% VNI 57.7% VNI 23.9 % Coping measurements by district is presented in table 20 below. In the No or Low Coping category the highest decline in proportions was experienced in Eyl (-63.2%) and Laas_Caanood (-10.7%) and Afgooye (-9.9%) while the largest increase was in Laasqoray
SomReP Seasonal Assessment and BMZ Evaluation Report ⎯ Oct 2022 40 (16.1%) and Lughaye (9.2%) districts. Reduction in coping strategy score is a positive thing in Eyl, Las Caanood and Afgoye. For the remainder of the districts the scores in the No or low coping category are in decline. In the Medium coping category there was an increase of proportions of households in Salahley (60.0%) followed by Laasqoray (48.2%) and Afgooye (40.4%) while the largest decline was in Hargeysa (-83.3%) followed by Bossaso (-52.3%) and Lughaye(-18.0%) districts. Ideally these should be better coping outcomes as seen in these districts but the sample sizes are too low to accept them as such. Generally, most districts in the program areas witnessed an increment in proportions of Medium coping between the seasons. Under high coping there were increased proportions of households in Hargeysa (83.3%) followed by Bosasso (57.4%) and Eyl (51.3%). The findings on Hargeisa (83.3%), Salahley (40.0%) and Bossaso (57.9%) are invalid on account of inadmissible sample size of six (6) respondents, five (5) and 19 respectively. A minimum sample of 30 cases is needed to validate a statistical output from analysis. Revised high coping scores are Eyl (51.3%) followed by Laasqoray (35.7%). Households in these districts experienced better coping outcomes. Table 19: Household Coping Measurements by district – Differentials District No or low coping Medium coping High coping Afgooye -9.9% 40.4% -30.5% Baydhaba -3.8% 27.2% -23.4% Bossaso -5.1% -52.3% 57.4% Burco -7.3% 33.7% -26.5% CeelAfweyne 4.3% 20.7% -25.0% Doolow -4.1% 22.6% -18.3% Eyl -63.2% 11.9% 51.3% Hargeysa 0.0% -83.3% 83.3% Laas_Caanood -10.7% 16.4% -5.8% Laasqoray 16.1% 48.2% 35.7% Lughaye 9.2% -18.0% 8.8% Salahley 0.1% 60.0% 40.0% Xudur -3.0% 37.0% -34.0% 5.2 The BMZ Funded Project Midterm Evaluation Findings The TOR required findings and discussions to be structured according to Results area as well as OECD thematic areas of relevance, effectiveness and Sustainability. Once the content was generated along these lines it became clear that this requirement would not be feasible as expected. While the OECD thematic areas are distinct so also are the BMZ funded project results areas. It was therefore necessary to find a compromise and not sacrifice both. We therefore structured the findings around OECD criteria and the discussions around Results areas. The TOR required an evaluation of the extent to which resilience interventions implemented under the BMZ funded project in Afgoye and Baidoa have delivered on relevance, effectiveness, and sustainability in order to develop a body of evidence that will enable SomReP and its partners to enhance programme effectiveness, increase accountability, and inform any necessary programmatic changes. The evaluation questions have been answered under three OECD evaluation criteria of relevance, effectiveness, and
SomReP Seasonal Assessment and BMZ Evaluation Report ⎯ Oct 2022 41 “The VSLA investments empowered members in borrowing loans for business operations and the benefit gained was used in buying their buying food (improvement of food and nutrition’s, paying of school fees and other domestic use during this time of drought and economic problems”. (Combined Community FGD in Afgoye) sustainability8 . The OECD evaluation criteria provide a normative framework used to determine the merit or worth of an intervention (policy, strategy, programme, project or activity). They serve as the basis upon which evaluative judgments are made. Under each of these criteria three specific summaries are provided as follows: 1. Assess the implementation progress made in activities towards achieving the planned results. 2. Assess the continued relevance, effectiveness and sustainability of the project in supporting local government in enhancing resilience and livelihood of communities in Baidoa and Afgoye and engagement of stakeholder groups in implementation actions. 3. Recommend strategic, programmatic and management considerations for implementing the remaining part of the project. 5.2.1 Relevance The overarching OECD question on relevance is “is the intervention doing the right things?” The BMZ Funded question on relevance is “To what extent are the project’s adopted strategies/approaches pertaining to each Result and overall objective still valid?” The Needs of The People Communities and Systems: The people, communities and systems in Somalia have undergone severe shock exposures which threatened their very existence. Individuals have included men and women as well as boys and girls living in the project areas in different livelihood types including pastoralists, agro pastoralists, peri urbanites, fisher folk and IDPs. Their resilience capacities have been extremely eroded. The community groups such as elders, religious leaders, TVETs, VSLAs, natural resource management committees, water committees among others are overburdened by the severity of shocks and have needed external support to exist and continue their task. The systems affected include water provision, market inclusivity systems, governance and policy development and implementation as well as ecosystem management. These are the needs among individuals, communities and systems in project areas to which the BMZ funded project has responded. Some of the voices that echoed this response are illustrated below. The need for intervention was also echoed by another FGD in this season as shown below in which a comparison is made between need and response in a BMZ funded village and a village that is not BMZ funded. When asked whether they had taken a loan to start/boost their business after training the VSLA group in oog village (not BMZ funded project area) stated “No, its difficult and almost impossible to get loan to start or boost business this area , since there is no financial institutions in this area and also there is drought and inflation which makes it very difficult to get loan.” (VSLA group in oog village) An Agro-pastoralist FGD in Afgoye revealed how relevant the BMZ funded project had been to them saying, A few voices also raised concern on targeting of beneficiaries, relevance of the TVET training courses for woman as shown below, “Some communities still have some concerns not to address their needs and concerns” – Men FGD Baidoa. Also some women in Baidoa said that “carpentry and electricians are less productive because many people do not fancy luxury items or services in the homes”. These may be useful for community infrastructure projects like schools, shops and hand carts and most women are 8 https://www.oecd.org/dac/evaluation/evaluation-criteria-flyer-2020.pdf
SomReP Seasonal Assessment and BMZ Evaluation Report ⎯ Oct 2022 42 “All the community and government ministries participated in discussions and preparations of the project to get and understand the requirements well” (Roble Abdirahin, Representative from The Ministry of Livestock Baidoa) selective with regards to TVET courses. A young boys FGD in the project area also voiced concerns of exclusion saying, “A negative impact for some community children is that they are forgotten when outreaches are conducted to recruit beneficiaries. The unreached children are unable to access wash facilitating and services. This is creating huge gap for them and those who are reached.” Project Design Structure and Components/ Strategies or Approaches for Result and Overall Objective: First, the BMZ funded project has been designed with one goal, five outcomes, 14 outputs, 74 indicators and 66 activities to be implemented in two districts of Afgoye and Baidoa over 36 months. This can be overwhelming in situations of shock exposure whose severity becomes an existential threat. It is in this context that the SomRep overall goal and Consortium as a network provides the anchorage for implementation relevance of the BMZ funded project. For a start, the SomRep overall goal and the BMZ funded goal are one and the same. This consolidates focus and energy of design into a singular value. Secondly, the five outcome areas of the BMZ funded project are results which are connected to SomRep program’s five strategies and approaches of Livelihoods & food security, Social Safety Nets, Natural resource management and Local governance capacity building and Monitoring, Evaluation, Accountability & Learning systems. There is a seamless flow of strate4y to results which aligns inputs to outcomes much more readily. More fundamental is that, SomRep as a consortium has a continued presence in the project areas and has established relationships with each other and other development actors in the same areas in which the BMZ funded project is operating. This is a huge asset to the project design, implementation and achievement of results. The Project builds on lessons leant and recommendations adopted and implemented, active relationships created, leverages of other projects and funds and intellectual energies generated and flowing towards the same direction for resilience ahead of 2023. The approach of participatory planning, implementation and measurement of action has been highly appropriate and appreciated in project locations. Alignments of project and program cycles and districts of implementation: The SomRep Strategy Phase 2 runs between 2018-2023 in 12 districts while the BMZ Funded project runs between October 2020 and October 2023 in 2 districts. This alignment of program-Project focus and cycle makes it easier for comparison and contrasts of contributions of both interventions to beneficiary resilience capacities at the end of project and end of program evaluations. Comparisons and contributions can be made on duration of exposure to “treatment”, patterns and association of variables between project and program, the disaggregated results on gender, district, disability and livelihood zones among others. “The project is addressing community needs and helping us manage disasters such as droughts. The entire strategies used by SomRep program have given priority to our community needs. For example, the cash program has improved our livelihoods as a community (pastoralists) affected by droughts. We are resisting shocks. Some SomRep strategies have contributed to job opportunities and improvements of infrastructures such as canals and the availability of water for irrigation. Also the cash we have received from cash for work we use to buy food for our families. The support provided such as VSLA investments has contributed to our incomes and access to loans for business operation. These help us as a community to increased community adaptation on challenges of climate change that has affected our livelihoods” (Agro-pastoralist FGD in Afgoye)
SomReP Seasonal Assessment and BMZ Evaluation Report ⎯ Oct 2022 43 “The formation of community early warning committees by SomRep program has contributed to capacity improvements on how to prepare for droughts and early warning in plantations. Also SomRep provided GAP training that equipped small scale farmers with modern way of farm preparations that increased crop productions. The NRM training was also facilitated by SomRep in national resource managements and has helped us think better about managing our natural resources to improve our ability to respond to climate shocks”. The Community of Practice: Resilience programming is being shaped by ideas, attitudes and practices from a community of practice on the subject matter which is linked to other communities of practice in related subject matters such as the humanitarian - development nexus. The evaluation finds that the BMZ funded project in South West State is highly aligned in its theory and practice to the OECD thinking and implementation framework for actions that strengthen the capacities of sustainable systems in different contexts9 including Somalia. Specifically, the structure and components of the project have created a variety of capitals which include financial capital, human capital, natural capital, physical capital, political capital and social capital. In this context, ideas on resilience knowledge, adoption, funding, measurement, practices and learning are created, funded and tested for uptake by beneficiary communities. The knowledge and practice is first tested in a pilot setting; usually a smaller scope, and then scaled up as a larger intervention informed by evidence to respond more effectively to the needs of the communities. The SomRIL and lab of ideas is one component of relevant support to the BMZ funded project design as well as learning which is connected to its activities on PMERL. There is a continuous need for sustainable resilience knowledge transfer and practice. Existing resilience Response Instruments: The response instruments cover program intellectual products, program implementation executive documents, government policy and strategic plan documents, funding types, community level groups governance and executable plans. These instruments are the creation of SomRep consortium working with Community-level groups and Community-level institutions supported by the Ministry of Livestock, Rangeland and Forestry SWS, the Ministry of Humanitarian Affairs SWS, the Ministry of Planning Investment and Economic Development SWS, and the Private Sector. The evaluation found that the resilience response instruments applied by the BMZ funded project are relevant to the needs of the individuals, communities and systems in the project area. The funding type as a “Transitional Assistance” matches the theory of change of increasing resilience to respond to shocks and maintain positive development trajectory. The SomRep strategy 2 document itself is a strong intellectual product which transforms theory into practice in ways that build strong actions with individuals, communities and systems. Rethinking diaspora remittances for development in addition to household consumption needs opens up more funding for resilience action from the Somali community itself connected globally. The deliberations and plans created by Community-level institutions including Village Development Committees (VDC), Early Warning Committees (EWC), Natural Resource Management Committee (NRM), Water Management Committees (WMC), and Social Affairs Committee are resilience response instruments born out of participatory governance and carry the corporate power and synergy of action. This is especially true of the Community Action & Adaptation Plans and Contingency Plans and the Contingency/Development Funds. An FGD in Baidoa with community committees echoed the appropriateness of some resilience response instruments Based on the above findings, and in the context of current practice in humanitarian – development nexus, the evaluation establishes that the BMZ funded project in Afgoye and Baidoa is highly relevant. It has responded to the right needs of the people, communities and systems. The project has 9 OECD (2014) Guidelines for resilience systems analysis, OECD Publishing; accessed at www.oecd.org/publishing/corrigenda
SomReP Seasonal Assessment and BMZ Evaluation Report ⎯ Oct 2022 44 applied itself using carefully structured intervention design and logic, project components and instruments as well as appropriate partners and funding mechanisms. The BMZ funded project is doing the right things in Afgoye and Baidoa and its approaches and strategies pertaining to each Result and overall objective are still valid. The summary score for relevance is to a great extent. 5.2.2 Effectiveness The effectiveness question was “To what extent are the delivery of outputs and outcomes contributing to the achievement of the Results and overall objective?” From outputs and outcomes to Results and overall objective The first issue in the findings on effectiveness is the need to clarify and sharpen the evaluation question itself between its focus on implementation and results. In common practice outputs and outcomes form part of results alongside impact. While impact is not presently a focus of this midterm evaluation, activities, outputs and outcomes are part of it. Given this, the evaluation question seeking to determine “To what extent are the delivery of outputs and outcomes contributing to the achievement of the Results and overall objective?” has needed to be reframed without losing the intended meaning of the question. The question has been rephrased as follows, “To what extent has the conduct of activities delivered expected results (outputs and outcomes) and contributed to the achievement of the overall objective?.” This evaluation analysis has applied a five point extent scale to examine the conduct of activities and derive outputs scores so as to determine appropriate responses to the evaluation question as a whole. The measures are as follows, “To a great extent “is given a score of 5, “To a large extent 4”, “Somewhat 3”,“Little 2” and “To no extent” 0. The scores are scaled to a performance score of 100% for full progress towards results and 0% for expected but not achieved. The rest are in between. The evaluation scale is then aggregated and converted into a simple “traffic light” rating system format based on the approach used by the Independent Commission on Aid Impact (ICAI). This midterm evaluation has modified this approach to benefit the BMZ funded midterm evaluation assessment of the extent to which the conduct of activities delivered expected results (outputs and outcomes) and contributed to the achievement of the overall objective. An explanation of the modified traffic light ratings is presented in table below: Table 20: Traffic Light Ratings Rating Symbol Rating Description Rating Vale Green - the project rates well against the criteria in the Evaluation Framework. Limited improvements could be made to strengthen the ratings against these criteria. 75% - 100% Orange the project rates relatively well against the criteria in the Evaluation Framework. Some improvements could be made to strengthen the ratings against these criteria. 50% - 74% Yellow the project rates relatively poorly against the criteria in the Evaluation Framework. Significant improvements could be made to strengthen the ratings against these criteria. 25% - 49% Red the project rates poorly overall against the criteria in the Evaluation Framework. Immediate and major changes need to be made to strengthen the ratings against these criteria. 0% - 24%
SomReP Seasonal Assessment and BMZ Evaluation Report ⎯ Oct 2022 45 These scores are indicative of performance and results at midterm. Their meaning for programming and future strategies is argued under the discussions section of this report. The evaluation findings for each result area are shown below based on the rephrasing of the effectiveness evaluation question as described above. Result 1 - Improved capacity of households to implement effective disaster risk management and positive coping strategies to mitigate the immediate effect of exposure to shock Table 21 BMZ Funded project result 1 outcomes Indicator Baseline ARM20 Change Measure MidtermEval22 Change % Point Degree % point increase in proportion of targeted households who are practicing positive coping strategies as measured by the reduced coping strategy index (rCSI) by 2023 (disaggregated by household head gender, vulnerability type, and livelihood zone). 30.3% 20.0% 15.6% -14.7% -48.7% % point increase in proportion of targeted households with little to no hunger as measured by Household Hunger by 2023. 73.2% 20.0% 58.3% -14.9% -20.4 % point increase in proportion of households who report to have contingency reserves e.g. livestock fodder, grain or finances reserved for emergency use (disaggregated by livelihood zone) 37.0% 20.0% 42.0% 5.0% 13.5% The planned results from the progress made in the conduct of activities under result area 1 of the BMZ funded project show that the outcome on capacity improvement for positive coping in the face of exposure to shocks has lagged behind its target at midterm. On outcome indicator #1 a 20.0% point increase at maturity (in 2023) results into a 50.3% value. At least by midterm this should be 40.3% assuming an increment of 10.0% point which is half the value of expected final % point increment. The midterm evaluation value of this outcome indicator is 15.6%. This is a 14.7% point decrease at midterm rather than an increase. The magnitude of decrease from the baseline is 48.7%. For outcome indicator #2 the midterm value of 58.3% against a baseline of 73.2% is a decrease by 20.4% while a midterm value of 42.0% for outcome indicator #3 is an increase of 13.5% from a baseline of 37.0%. While the first two outcome indictors signal a reversal of expected outcomes the third one shows a positive result at midterm. Positive coping in the face of shocks as well as little to no hunger are key outcomes desired by SomRep program as well as the BMZ funded project. The most affected indicator value is positive coping which declines by 48.7%. The positive result on reports of contingency reserves is viewed in the context of crisis modifiers as shall be noted under outputs analysis below. In assessing the implementation progress made in activities towards achieving the planned results we link the outcomes to outputs below and provide reasons based on activities conducted.
SomReP Seasonal Assessment and BMZ Evaluation Report ⎯ Oct 2022 46 Table 22: Output delivery under Outcome 1 Output Planned Achieved @ Midterm Level Participatory development and implementation of DRR/Resilience Plans 112 80 71.4% Early Warning/Action System Strengthened 19 9 47.4% Private Shock Responsive Safety Nets established and strengthened Crisis Modifiers: 1860 1993 107.2% Trainings: 116 34 29.3% There is a sense of flexibility and agility within the project such that an activity planned for a specific quarter can easily be shifted to another quarter within the project cycle and still be conducted. The Project implementation design is not rigid to break but can accommodate some level of changes (adjustable) and still be able to achieve its intended objectives. For example activity A1.1.4.2 Establish National/International Mobilizer/Goodwill Ambassadors SHQ Mobilizations which in quarter 2 of 2021 targeted 3 ambassadors achieved none (0). This is 0%. However in quarter 3 of 2021 the project targeted 3 ambassadors and achieved 3 (100%) but in quarter 4 of 2021 the project targeted 3 but achieved 10 which is 333% (BMZ funded project Indicator Tracking Table). This is flexibility and agility of implementation for results. Summary of progress The extent to which the conduct of activities has delivered the expected results (outputs and outcomes) on “Improved capacity of households to implement effective disaster risk management and positive coping strategies to mitigate the immediate effect of exposure to shock” and contributed to the achievement of the overall objective is red for outcome indicator #1 and #2 but Orange for outcome indicator #3 as shown in the table below. Table 23: Indicator traffic light scores Indicator Scale Light % point increase in proportion of targeted households who are practicing positive coping strategies as measured by the reduced coping strategy index (rCSI) by 2023 (disaggregated by household head gender, vulnerability type, and livelihood zone). -242.6% % point increase in proportion of targeted households with little to no hunger as measured by Household Hunger by 2023. -101.8% % point increase in proportion of households who report to have contingency reserves e.g. livestock fodder, grain or finances reserved for emergency use (disaggregated by livelihood zone) 67.6%10 The improvements that need to be considered are in capacity building planning and delivery. The capacity building of VSLAs in good hygiene practices, life skills & gender empowerment as well as the training of TU on BDS, Life skills, Hygiene and Sanitation, Nutrition, PSP model had not yet taken place s at the time of report writing. Additionally, the group level revolving loan fund has not yet been delivered to the groups and follow up of groups has not yet be done. It appears as if efforts to deliver crisis modifiers have overtaken capacity building activities under results areas 1. Crisis response takes centre stage more than 10 Computed by applying the magnitude of change (13.5%) against expected increment value of 20.0% at the scale of 100%.
SomReP Seasonal Assessment and BMZ Evaluation Report ⎯ Oct 2022 47 capacity building. This speaks also to program design when consolidating activities under d and output. Lumping crisis modifier activities with capacity building could compromise efforts to deliver one category against another in cases of emergency response. Usually capacity building is sacrificed in favour of crisis response. Result 2: Improved capacity of individuals, households and communities to adhere to positive development trajectories; despite exposures to shocks and utilize strategies designed to allow adaptation to rapid and slow on-set hazards. Table 24: BMZ Funded project result 2 outcomes Indicator Baseline ARM20 Change Measure MidtermEval22 Change % Point Degree % point reduction in households who are in need of humanitarian assistance during shocks and stress in the project impact area by 2023 26.8% 15% 73.5% -46.70% -174.3% % point increase in average household income in the project area by 2023 (data disaggregated by household head gender, vulnerability type and livelihood zone). (Mean in USD) $140.5 10% $160.8 $20.30 14.4% % point increase in proportion of targeted households who report all year access to water (for irrigation agricultural production & domestic use) as a result of investment in large scale multi-use water infrastructure and small scale water infrastructure by 2023 (disaggregated by household head gender, vulnerability type, and livelihood zones). 77.5% 10% 84.1% 6.57% 8.5% The evaluation found that the conduct of activities under result area 2 of the BMZ funded project delivered the following results. Adherence to positive development trajectory and adaptation to rapid and slow on-set hazards has a mixed outcome. Whereas outcome indicator #1 shows severe negative outcome at -174.3%, outcome indicators #2 and #3 post very favorable results at 14.4% and 8.5% respectively. At -174.3% outcome indicator #1 on the % point reduction of households who are in need of humanitarian assistance during shocks and stress in the project impact area tells the story of reversed humanitarian fortunes and dire need at a magnitude of 1161.7%. This is a crisis situation. The % point increases of 14.4% in average household income in the project area as well as the 8.5% point increase in proportion of targeted households who report all year access to water are change measures on incomes at a magnitude of 144.5% and 84.8% respectively. These are positive outcome and are to be understood in the context of a humanitarian crisis modified by interventions to restore balance and normalcy (crisis modification). Table 25: Indicator Traffic light score Indicator Scale Light % point reduction in households who are in need of humanitarian assistance during shocks and stress in the project impact area by 2023 - 1161.7% % point increase in average household income in the project area by 2023 (data disaggregated by household head gender, vulnerability type and livelihood zone). (Mean in USD) 144.5%
SomReP Seasonal Assessment and BMZ Evaluation Report ⎯ Oct 2022 48 % point increase in proportion of targeted households who report all year access to water (for irrigation agricultural production & domestic use) as a result of investment in large scale multi-use water infrastructure and small scale water infrastructure by 2023 (disaggregated by household head gender, vulnerability type, and livelihood zones). 84.5% Table 26: Result area #2 Outputs Scores Output Planned Achieved @Midterm Level 2.1 Access to sustainable water improved for agricultural & domestic use 9 (+ % point increase of 5) 3 38.3% 2.2 Livestock & agricultural production systems strengthened 2976 1783 59.9% The activities under output #2.1 and #2.2 that saw exponential participation included GAP training for 1440 farmers out of expected 1550 (92.9% achievement) and establishment of kitchen 33 gardens out of expected 20 (1565.0% achievement). Although only 3 of the 9 water infrastructure were rehabilitated/upgraded the score of 84.5% on proportion of targeted households who report all year access to water (for irrigation agricultural production & domestic use) at the outcome level is most likely as a result of significant participation of beneficries in cash for work activities which included rehabilitation of water resources such as canals. The other factor is that farmers in the project areas accessed loans to buy water tanks and conduct irrigation for fruits and vegetables. It is not clear from program design at what point do “water Tanks” or “water harvesting reservoirs” become “large scale” water infrastructure yet these could have contributed to respondent’s perception of access to water and informed the magnitude of the outcome indicator score. A third output under outcome area #2 is “Youth and women equipped with skills and resources for livelihood diversification”. Activities under this output included on vocational education training, Labour market surveys, revolving funds for business development at group level, provision of enabling inputs for disabled persons to participate in livelihood activities, provision of grants to start own business and /or receive trainings, provision of startup awards and or kits to graduates for self-employment. None of this had been conducted at the time of reporting. It is not yet known how this component of program design under outcome area would have impacted the results had it been executed as planned. Output 2.1 combines measurements in proportion and absolutes; # of water infrastructure rehabilitated/upgraded (disaggregated by livelihood zones) and 20 % Point increase in proportion of targeted households that report to be using improved soil and water conservation measures. Combining proportion and absolute numbers in activity design and measurement could be problematic for computations and learning about performance and contribution to results. Possible design misfit affects results achievement and measurement as well as learning. Activities are best designed in absolutes rather than proportions. One more point on unintended outcome on water access. The men FGD in Baidoa perceived that “even after the rehabilitation and fixing of more water points there were individuals still using unprotected water source such as ponds, stagnant water and hand dug well water rather than piped water hence creating room for waterborne disease like diarrhoea and bilharzias” . While this could speak to messaging of resilience knowledge it could also speak to knowledge adoption and motivation for behaviour change.
SomReP Seasonal Assessment and BMZ Evaluation Report ⎯ Oct 2022 49 Result 3: Improved capacity to engage in strategies for sustainable livelihoods and economic growth to enhance food security and resilience Table 27: Result area # 3 outcome indictor scores Indicator Baseline ARM20 Change Measure MidtermEval22 Change % Point Degree % point increase in proportion of households with acceptable food consumption scores by 2023 (disaggregated by household head gender, and vulnerability type) 44.90% 20% 56.90% 12.00% 26.7% There are two other outcome indicators under this result area which the evaluation has not required to be measured but which have been implemented and have a bearing on the results. They include outcome indicator # 3.2. 25% point increase in proportion of HHs engaging in multiple income generating activities by 2023 as measured by participation index. (Disaggregated by household head gender, vulnerability type, and livelihood zone) and outcome indicator # 3.3. 15% point increase in proportion of women that engage in jobs in sectors of sustainable livelihoods (diversified production related incomes) and economic growth (business development services like transport, milling, bulking, and input supply). A complete indicator assessment of this outcome at midterm would contextualize “acceptable food consumption score” to “engaging in multiple income generating activities” as well as “proportion of women that engage in jobs in sectors of sustainable livelihoods” (enterprise development and Job creation). As it is now the acceptable food consumption score of 12.0% point increase at midterm is a 133.6% increase measured from expected magnitude of increase overall and a light of green a shown in the table below. Table 28: The Traffic Light for Outcome Areas #3 Indicator Score Light % point increase in proportion of households with acceptable food consumption scores by 2023 (disaggregated by household head gender, and vulnerability type) 133.6% While this is very positive results it has to be understood in the context of specific activities conducted among pastoralist, peri urbans, agro pastoralists and IDPs in the project area to lead to such an outcome. Activity focus has been on improving their capacity to engage in strategies for sustainable livelihoods and economic growth to enhance food security and resilience. The table below shows findings on three outputs under which several activities were conducted to contribute to this result. Table 29: Result #3 Outputs Indicator scores Output Planned Achieved @Midterm Level Capacity for Production and competitiveness enhanced for Pastoralist and Farmers 2 0 0.0% Formal financial linkages strengthened 98 73 73.6% IDPs supported with Durable Solutions to improve their livelihoods 35 25 71.4% Although the activities under the output on Capacity for Production and competitiveness enhanced for Pastoralist and Farmers had not been conducted the ones on Formal financial linkages and support with durable solutions to improve their livelihoods were successfully executed. These included linking 16 out of planned 11 producer groups to informal microfinance bodies such as savings groups; linking 56 out of planned 86 VSLA groups to
SomReP Seasonal Assessment and BMZ Evaluation Report ⎯ Oct 2022 50 formal MFIs; establishing 10 out of planned 20 permaculture sites in IDP settings/urban settlements to support h/hold level water management and household gardens and then also establishing 15 out of planned 15 Savings groups in IDP settings and training them in VSLA methodology. It is interesting to compare the opportunity for resilience building with regards to “Crisis modification only” as was seen under outcome areas ….and “Crisis Modification with Capacity building in enterprise and employment creation” inferred under this outcome area. Whereas under outcome area #1 the outputs on cash transfers as crisis modifiers had a 107.2% achievement, those on capacity building scored a 29.3% achievement. In the outcome area #3 the crisis modification aspect IS maintained while at the same time executing capacity building at 73.6% and 71.4%. The outcome ratings in both outcome areas are different. While outcome area #1 scores two reds and one orange outcome area # scores a green. It is interesting in that crisis modification could well be more impactful if conducted concurrently with capacity building in enterprise and employment. Result 4: Governance structures at community, district and national levels are strengthened to enhance participation, transparency and accountability Table 30: Result 4 outcome indicator scores Indicator Baseline ARM20 Change Measure MidtermEval22 Change % Point Degree % point increase in proportion of women and other disadvantaged groups who report to have been involved in decision-making processes in the determination of development priorities and CAAP financing at the community and district level 46.20% 15% 66.78% 20.58% 44.5% The outcome #4 on strong participatory, transparent and accountable governance has two outcome indicators; one on community satisfaction with service delivery and inclusion in decision making. The midterm evaluation has only requested for an evaluations on inclusion in decision making for development priorities. As at midterm the BMZ funded project had registered a very huge positive outcome (44.5% from baseline) on inclusive decision making for development priorities. This is a change magnitude of 297.0% considered against its planned change level of 15%. It is huge and positive. It scores a strong green light as shown in table below: Table 31: outcome #4 traffic light scores Indicator Score Light % point increase in proportion of women and other disadvantaged groups who report to have been involved in decision-making processes in the determination of development priorities and CAAP financing at the community and district level 297.0% One of the examples of inclusive participation of vulnerable persons especially women in decisions for development was witnessed in Afgoye. There was a change of development priority from the initially proposed feeder road rehabilitation in Balgure village, school construction in Awgoye School and health post in Jaran village to water availability which is top priority during the time due to drought conditions. In the development committees women are specifically included by design so that their participation is guaranteed And Voices Heard. Other instances where women voices have been heard in decision making include periodic community and local government review and adaptation of CAAPs for 11 villages in