Skip to main content

Sustainability evaluation of community-based, solar photovoltaic projects in Malawi

Abstract

Background

A novel project sustainability framework is used to evaluate 65 off-grid solar photovoltaic (PV) energy system projects in Malawi. This study addresses PV projects serving rural public facilities, a solution known to have had historical issues with poor sustainability. A recent countrywide program targeting such facilities was evaluated against existing projects to determine whether this latest iteration offered better results.

Method

Sustainability is defined at the project-level with metrics justified under the main themes of technical, economic, social, and organizational. Data captured for each project is based on a custom survey and interview of key stakeholders. Projects are grouped according to age, project implementer, income level, and PV system size to compare relative results.

Results

The sustainability outlook for each project is evaluated. We find sustainability for most projects to be low. Social sustainability was weak with many projects due to low local community stakeholder engagement in terms of numbers of stakeholders, meeting frequency, and community contributions. Organizational challenges included a lack of key management positions in place and extremely limited training provisions. Furthermore, the evaluation highlights the economic health of the system to be key to sustainability, with the “healthiest” projects affording only 37% of expected operations and maintenance costs. Relative to expected demand, systems were found to be undersized for both panels (53% of required) and batteries (57%). Users reported achieving only 60% of their desired consumption. Poor sizing standards related to the lack of load profiles for first-time electricity users as well as poor quality components reduced the overall technical sustainability.

Conclusion

Rural public facilities with solar PV in Malawi are not well served by isolated community management. Improved sustainability requires the establishment of a robust financial framework prior to project development that includes formal local government support. This paper discusses ideal management frameworks and their ultimate implications for project developers, policy makers, and the research community.

Background

Globally, an estimated 992 million people lack access to electricity, which is widely recognized as a key enabler for development [1]. Lack of electricity serves as a barrier for economic empowerment and poverty eradication in developing nations. This has led to the United Nations defining Sustainable Development Goal 7 - “Ensure access to affordable, reliable, sustainable and modern energy for all” as one of their 17 global goals [2].

Malawi ranks close to the bottom of the United Nations Human Development Index league [3], and its extreme poverty levels are exacerbated by one of the lowest electricity access rates in the world. An estimated 16 million Malawians, or 91% of the country’s population, currently live without access to electricity [4]. Rural households and public services such as primary schools and health centers often lack electricity access for basic, but critical, lighting and power services leading to sub-standard delivery of education and health-care and further constricting socio-economic welfare and development of these communities.

Solar photovoltaic (PV) technology has become a key element for improving electricity access for rural communities in Malawi due to abundant solar resources and relatively low capital and operating expenses associated with PV systems [5]. Poor rural electricity access is compounded by low coverage, high losses, and poor reliability of the chronically under supplied Malawian main grid [6]. As such, many non-governmental organizations (NGOs), Malawian government, and international initiatives have focused on off-grid PV deployment, including the recently concluded Malawi Renewable Energy Acceleration Programme (MREAP) [5, 79].

The sustainability of off-grid projects and systems in wider developing country contexts remains an ongoing challenge for practitioners and is often responsible for limiting the potential impact of these projects. For new electricity access interventions to achieve their intended impact, project designs and deployment methods must pay close attention to sustainability issues from past deployments and ideally use a systematic framework to evaluate project sustainability.

This paper is aimed at evaluating the sustainability of off-grid solar PV projects in developing countries and offers two contributions to the energy for development literature. Firstly, a novel framework for evaluating the sustainability of such projects is proposed. The framework focuses on the implications of project design choices on the sustainability of the project and, as such, provides a valuable tool for the systematic design, monitoring, and evaluation of sustainable off-grid energy projects. Secondly, the framework is applied to evaluate 65 off-grid PV projects already installed in Malawi with results from a recent tranche of projects drawn out for detailed discussion. The evaluation has immediate importance to practitioners in Malawi, as issues affecting the sustainability of off-grid projects are widespread and still not fully understood.

The paper begins with a brief review of the landscape of off-grid electrification in Malawi in the “Malawian electricity landscape” section. The methodology is described in the “Methodology” section. The definition of sustainability, as it is used within this paper, is defined in the “A project-based definition of sustainability” section, while useful and relevant data sources are described in the “Data sources” section. The scoring and ranking methodology are covered in detail in the “Sustainability scoring methodology” section along with justifications for the inclusion of each sustainability evaluation metric. The “Results” section presents the results of the evaluation by each sustainability factor: technical, economic, social, and organizational. The discussion in the “Discussion” section focuses on implications for project developers in 1 and implications for policy in the “Implications for policy makers” subsection, and finally, a methodological discussion is presented in the “Methodological implications” section. Conclusions are given in the “Conclusions” section.

Malawian electricity landscape

The national grid in Malawi operates on centralized hydro-generation, with a reported installed capacity of 361 MW serving 11.9% of the population (mainly in urban locations) [4]. Since the early 1980s, the Government of Malawi has focused almost exclusively on extension and modernization of the national grid through the Malawi Rural Electrification Program (MAREP) currently in its 8th phase [10]. Despite steady progress in electrifying trading centers across the country, the limit on generation capacity ultimately restricts access to electricity and the grid’s capacity to accommodate increasing demand from connected customers. Consequently, customers connected to the national grid experience regular and prolonged blackouts, often on a daily basis, due to planned load shedding. Furthermore, the grid generally serves the urban region of Malawi, with only 4.7% of the rural population being connected despite accounting for 83% of the population. [4]. This disparity has led key international energy stakeholders to identify off-grid solar PV as a key solution for rural energy access in Malawi [11, 12].

Despite some notable rural electrification project successes [11, 13, 14], off-grid energy projects in Malawi have had poor sustainability overall. In 2012, an estimated 7000 PV systems were installed in Malawi, with the majority identified by practitioners as being in varying states of disrepair or not fully functional with expected shortened lifespans [5]. An evaluation of 12 case studies of community renewable energy projects in 2012 cited insufficient financial models, unclear management structures, and limited local technical capacity [15] as being responsible for their poor sustainability. Similarly, a nationwide evaluation of 43 projects in 2015 found many projects at various states of failure due to poor technical design, inadequate revenue, and low operator skill levels [16].

The Government of Malawi’s Energy Policy in 2018 targets rapid deployment of renewable energy to 2030 and outlines plans to expand support to both grid extension and off-grid renewable electricity projects [10] [17]. In addition, the Malawi Renewable Energy Strategy [17] targets 100% of schools to have modern energy solutions by 2025. If these aspirations are to be delivered in a fashion that delivers long-term impact, evidence-based sustainability strategies are essential.

Methodology

This section develops the sustainability framework used throughout this research. A short review of recent project sustainability documentation is provided to establish the breadth of challenges currently facing projects. Since the definition of sustainability is both ubiquitous and ambiguous, the section first discusses important recent efforts at providing a useful definition specifically for off-grid electrification projects. Building on this work, this paper offers a new criteria for inclusion of project-centric metrics for sustainability evaluations, with motivations for each metric provided. The effectiveness of the metrics is discussed following the results.

Review of sustainability challenges

Sustainability challenges found in off-grid projects outside Malawi have been widely documented in literature [1821]. The present study focuses on Malawi, yet it is worthwhile to establish that the overall context of challenges are widely applicable. A broad range of issues have been cited including unsuitable implementation approaches for unique cultural contexts, failure to setup clear managerial structures, and non-viable financial models. Additional issues for failure stemmed from low user operation and maintenance skills, low awareness of technology, lack of availability of quality components, and insufficient spare parts.

Although comprehensive studies that capture the prevalence of specific sustainability issues are often not detailed enough to provide conclusive evidence, reviews of multiple projects over many countries do provide a sense of the general challenges that are not necessarily site specific. Ikejemba et al. reviewed 29 projects in 9 Sub-Saharan African countries and found roughly 90% had experienced some degree of failure [22, 23]. Contributing factors to these failures included poor political support, corrupt awarding bodies, poor project ownership, and poor coordination between stakeholders, insufficient project planning, lack of maintenance systems, and public exclusion, resulting in low levels of support and engagement. The projects reviewed by Urmee et al. highlighted the need for socio-cultural understanding and knowledge of policy issues during off-grid electrification and ensuring solutions are designed to meet the actual (and not perceived) community need [24]. An earlier study by Urmee and Harries captures a wide range of sustainability factors, including system design that avoids subsidization, ensuring readily available spare parts, providing training for users and technicians, and ensuring components are high quality [25]. It was found that nearly 21% of projects failed or partially failed and only 48% were fully functional in a review of 23 renewable energy development projects in 17 countries [26]. Factors contributing to the poor sustainability of these projects were cited as lack of user ownership, low user satisfaction with the technology, external influences (political, institutional, environmental), and problematic logistics. A related study found that grid competition undermined around half of the evaluated projects [27]. The authors find that geographic location could not be considered a direct influence on project sustainability though local factors such as transportation and institutional infrastructure was linked to project sustainability.

Gender has also been considered with respect to social sustainability. Ilskog and Kjellström use share of women in staff and management as indicator of social sustainability in their framework while evaluating seven organizations operating in Tanzania, Kenya, and Zambia [28]. However, in other works, the authors argue that the impact of the gender dimension on sustainability of projects lacked sufficient evidence to draw causal links but was nonetheless expected. [27]

A performance and impact evaluation for solar home lighting systems in Assam, India, in 2017 found only 28.9% of the systems were found to be totally functional [29]. The study implemented a comprehensive questionnaire to 544 households in the area where the overall number of installed systems is estimated at 40,035. A wide range of technical faults were documented: compact fluorescent light (CFL) blow outs and other minor issues (62% of households), major issues such as charge controller failure or partial battery failure (1.6%), and some units completely non-functional (7.2%). Overloading due to insufficient solar resource and heavy system usage was found in 9.8% of households. Additionally, users aiming to get more out of the batteries had bypassed charge controllers.

Individual project experiences often provide more detailed descriptions of the sustainability challenges, sequence of events, and better understanding of the inter-relationships between sustainability issues. Hong and Abe investigated the sustainability of an off-grid project in the Philippines facing challenges with long-term financial and human capacity [30]. The project had been in operation for 12 years at the time of publication. The island system included a 45 kW PV system, inverter, and battery storage totaling 423 kWh and used by 236 households. Sustainability issues faced by the project included a total failure of the battery storage systems which resulted in a reduction of plant availability to only daylight hours or in some cases when there was cloud cover, total unavailability of the system. Poor components were cited as the primary cause, with equipment failing much sooner than expected. Funds for replacement were not available so the system operated with outage or very limited service for a 1 year period. It was found that the majority of usage was for welfare and lifestyle improvements with very few economically significant industries being provided power.

Bhandari et al. conducted a sustainability assessment on a 26 kW micro-hydro plant, established in 2008 in Nepal, which involved the development of locally validated sustainability indicators [31]. Although not solar PV-based, the sustainability challenges are relevant more generally. Indicators were organized among the main themes of social, economic, environmental, and technical with 54 sub-indicators that are weighted in consultation with the community. Data was collected primarily through interviews with 15 households, project management committee, the operator, and several external experts in alternative energy. While the authors find strong evidence that the project has had major local benefits and that many users are ‘satisfied” with the power supply, several challenges were also noted. Technical issues faced were a low plant capacity factor (~ 40%), no plans of plant upgrades, and no schedule for a maintenance program. Economic sustainability was hampered by project revenue generation not being sufficient to cover costs, limited economic benefits to local enterprises, and low levels of entrepreneurial conscientiousness (local ambitions to utilize power for economic gain). Additionally, the authors cited insufficient training for community members and potentially dangerous working conditions for the operator.

Elusakin et al’s review of the off-grid experience in Nigeria in 2014 described it as a “monumental failure” [32, p. 53] and challenged whether further investment is warranted. Specific issues raised were planning design inadequacies, corruption, and poor quality equipment in the supply chain. Design issues such as targeting communities which cannot afford the technology or do not have the capacity for management of the projects resulted in abandonment. In Nigeria, many government led off-grid projects include a diesel generator; while upfront costs are relatively low, the ongoing costs and logistical challenge of providing fuel supply has resulted in many failures as the economic model was unfeasible. The cases of government corruption where off-grid project funds are captured by officials and projects are announced as an election tool have decayed the reputation of the projects. The proliferation of low quality components to off-grid systems and price gouging, taking advantage of consumer ignorance, has undermined consumer trust in the industry. Local ownership was considered a key element and should include proper buy-in and an early feasibility assessment to determine the optimal choice between off-grid and grid-extension.

The sustainability of rural electrification efforts in Ecuador is captured in a 2016 study by Feron, Henrichs, and Cordero [33]. In Ecuador, grid extension has been government-led while off-grid efforts have been relatively underfunded—only 1.86% of funding went to off-grid from 1998–2009. Additional international funded efforts targeted off-grid communities, though by 2009, only around 10% of these systems were still in use. Another 3270 standalone PV systems installed since 2009 by other organizations were subsidised and required outside financial support to cover operations and maintenance. The projects were found to have a wide array of sustainability challenges. Uncertainty from the adoption of seven different constitutions since 1938 (with the latest in 2006) has disrupted the availability of information on national programs and application of policies and resulted in lack of project ownership. Furthermore, a poor supply chain in rural areas and limited skills to conduct repairs resulted in compromised system reliability. Higher long-term costs and low revenue generation undercut the economic viability of many projects that had no outside financial support. A fundamental issue was the limited ability to pay for energy by users and oversizing of systems. More recently, PV micro-grids have become a strategy for the government but have been criticized for being overly sophisticated for local management and potentially creating a sense of inequality for those who can immediately access the system. Environmentally, poor battery regulations for standalone PV systems has led to inconsistent disposal—in some cases, they were buried. Finally, efforts throughout the country have taken different approaches to gaining social acceptance. In some cases, there has been rejection of the program by indigenous leaders and other cases where acceptance has been much higher.

This review of the wider sustainability experience with off-grid projects has shown that sustainability issues and obstacles to development are felt by many and are varied in their nature. The reviewed case studies do not represent an exhaustive coverage of varied projects throughout the world, but they do reflect recent experiences and a wide geographical coverage. Furthermore, when considering indicator frameworks to measure sustainability, it is clearly critical that the framework is able to express both the breadth and details of challenges that commonly effect projects.

A project-based definition of sustainability

In order to learn from and adapt to these challenges, a revised definition of sustainability is necessary that is compatible with project-level issues that are faced. A common starting point to evaluating project sustainability is through the use of sustainability indicator frameworks. These select indicators that are combined to track sustainability levels of a project. A high score among the combination of indicators represents a “sustainable” project.

Historically, sustainability indicator frameworks have been targeted to the country-level and have been developed by the United Nations (UN) and later by the International Atomic Energy Agency (IAEA) [34, 35]. The themes of “technical,” “economic,” “social/ethical,” “organizational,” and “environmental” have been extended from the IAEA initial classification, and despite their ambiguity and lack of practical consensus to their specific meaning, they remain popular for thematic classification [36].

Perhaps the most prominent recent work that extends this tradition is an indicator framework developed by Ilskog [37]. The framework identifies 39 indicators over the sustainability themes. Since its publication, it has been adopted by a number of other authors to conduct evaluations of energy projects and programs [3840]. However, the application of indicators raises an important question as to their suitability for sustainability evaluations. The relevance of some indicators appears tied up with high-level, countrywide goals or what might be desirable development outcomes rather than true sustainability issues facing projects. For example, indicators such as carbon dioxide emissions and share of population with primary school education may have no direct relevance toward an individual project’s ability to sustain itself. Ilsog and Kjellström implement the framework in Tanzania, Kenya, and Zambia for a series of off-grid projects [28]. Considering carbon dioxide emissions, the evaluation identified two projects that did not have renewable energy sources and thus were considered less sustainable. However, it is unclear how this connection is drawn—do the emissions literally prevent customers from consuming electricity? In some cases, this may actually be the case; however, the example demonstrates how indicator frameworks, in many cases, are the manifestation of the authors conceptualization of sustainability but may not have a tangible link to the project’s survival.

To differentiate the intention of such indicators, one can establish “development outcome-centric” indicators that retain the UN and IAEA heritage versus “project-centric” indicators that are used strictly for relating to the project’s survival. It is argued that this differentiation is useful for applications such as sustainability evaluations, although this does not reduce the importance of indicators being applied to impact evaluations [41]. In project-centric indicators, one would expect that the wide range of sustainability challenges, as documented previously, would produce a sufficient signal to detect potential problems and respond accordingly.

One of the most recent iterations of sustainability evaluations by Katre and Tozzi considers 40 off-grid projects in India [42]. The method is unique in that it clearly classifies and defines dimensions, measures, and a data gathering structure. Although several other studies provide a justification for the included indicators, Katre and Tozzi also define a criteria with a rubric for the scores and have a strong emphasis toward project-centric indicators [43, 44]. With most of the projects considered younger than 32 months, and with only two observations per location, it is a convincing, if emergent, method for measuring project sustainability. Although the framework proposed is theoretically adaptable to any off-grid energy context, it has been developed, and subsequently applied [45], specifically to the mini-grid context.

This paper uses the common thematic classification and proposes several new metrics that are project-centric. Developed in parallel, this work is complementary to that of Katre and Tozzi, providing metrics developed for application to public institutions in a sub-Saharan African context. For further studies of off-grid projects in sub-Saharan Africa, the metrics proposed here could be adopted as is or used to adapt an existing framework such as that developed by Katre and Tozzi. To ensure the terminology of sustainability retains a project-centric focus, this study adopts the use of the terminology “factors” (versus theme) and “metrics” (versus indicator). The following definition of project sustainability is proposed and is further defined by a set of metrics in “Sustainability scoring methodology” section:

  • Social sustainability. Considers all forms of community and stakeholder engagement, representation, participation, and ultimately continuing acceptance of a project within the local social structure.

  • Organizational sustainability. Refers to maintenance, retention, and endurance of human and business assets as well as decision-making capacities to ensure efficient undertaking of project operational routines.

  • Technical sustainability. Considers the technological performance and ability of projects to operate as designed and to provide the expected level of energy service throughout the project lifespan.

  • Economic sustainability. Considers the continued financial well-being throughout the project lifespan.

  • Environmental sustainability. Considers local ecological impacts and negative externalized effects due to the installation and continued operation of the project.Footnote 1

Data sources

Community electricity project data

The project data used in this study consists of project-level information captured via a structured questionnaire from site visits to 65 individual solar electricity projects in Malawi. A solar electricity project is defined as a group of solar electricity assets (one or more individual systems) for which a distinct management team is responsible for its ongoing operation and management. In all, the projects had 246 systems and 642 separate rooms.

Data gathering took place in two phases. The first phase, undertaken in 2014, surveyed 43 projects and evaluated the sustainability of existing PV projects [16]. The original study design intended to have additional sites, but this was limited due to implementation program scope and reporting needs. In 2016, an opportunity to extend the data set became available through follow-up MREAP funding. Thus, in 2016, the second phase data gathering added 22 new projects to this data set. These focused on non-MREAP projects as a wider set of projects for comparison was desired. For both phases, the entire survey data was captured at each project—in other words, there was no follow-up on previously visited projects between phase 1 and 2.

Data handling

Data gathering was facilitated by an enumerator using a questionnaire to interview project managers and their support staff (typically 1 to 3 persons). Although rare, records such as log books and financial reporting were used to assist respondents when available. Technical data was taken from each system on a room by room basis, and electrical appliances were visually inspected by the enumerator where possible. Economic, social, and organizational data was captured for the project as a whole.

Minor changes for clarity adjusted the survey between phase 1 and phase 2, though the content was identical. Phase 1 used a paper-based questionnaire while phase 2 uses digital format through the KoboCollect Platform [46, 47]. Projects that did not have full data available on all systems were removed from the analysis. Considering the number of projects surveyed, the analysis presented is the most comprehensive and current information available for PV sustainability in Malawi and is inclusive of most major off-grid PV projects in Malawi.

Sustainability scoring methodology

The methodology presented here employs a novel scoring method to evaluate project sustainability, both in terms of the proposed sustainability metrics and as a whole. Several metrics are combined under each thematic area or sustainability factor. Each metric is selected based on relevance, availability, and comprehensive coverage of the sustainability factor. Metrics under a particular factor are averaged (unweighted) to provide a normalized sustainability factor score between 0 and 1. Scores are calculated at the project level for all factors except technical sustainability, which is analyzed on a per system basis and then reduced to a project level by summing or averaging as appropriate.

Grouping criteria

The surveyed projects are grouped for comparison in the analysis. First, projects are grouped by project implementer: through the MREAP Community Energy Development Program (CEDP), under the MREAP Strategic Energy Project (SEP) led by the University of Malawi Polytechnic, and all other projects (OTHER). The CEDP and SEP project groups had different implementation strategies and designs from the OTHER systems such that comparison of these interventions is of particular interest.

Second, projects were grouped according to age, where young corresponds to projects started from 2014 to 2016, mid refers to projects started between 2010 and 2013, and old projects started before 2010. Figure 1 shows the projects against year of installation. An intuitive assumption is that older projects are more likely to have experienced sustainability issues than younger projects. The increments chosen correspond to periods where there will be a higher likelihood of major component failure (usually batteries). mid projects are expected to face their first instance of battery failure. By the time a project is old, several technical breakdowns may have occurred.

Fig. 1
figure1

Year of installation by project group

Third, the income grouping differentiates between projects with a positive net income, negative net income, and those which have no income records. The establishment of a sufficient revenue generating scheme is widely accepted as a requirement for positive project net income. Negative net income projects persist precariously due to sporadic and inconsistent incomes (such as gifting), and in this case, savings are unlikely. Projects that lack revenues and expenses altogether, but still exist to provide a benefit, represent the simplest project implementation format but are expected to have only short-lived benefit.

Fourth, projects are grouped according to size: small includes projects with PV capacity less than 500 watts (W), medium is between 500 W and 2000 W, and large is greater than 2000 W. System sizing corresponds to expense, complexity, and potential impact, suggesting project size impacts sustainability in a non-linear way.

Social sustainability factor

The social sustainability factor relates to the community acceptance of the project. Integrating community needs and priorities in the project design stage and development of community ownership are well acknowledged as key aspects of any community development project [24].

In Malawi, many government functions are decentralized at the district level where Area Development Committees (ADCs) feed community priorities to the District Council (DC). Therefore, the DC is highly influential for community development initiatives, both in terms of funding support and public support. As the energy sector is not decentralized, historically, DC’s provide little ongoing financial support for PV projects. DC project involvement in the project is a social sustainability indicator and is important for achieving village support.

Once the stakeholders of a PV project are identified, meeting frequency and the number of stakeholders provides a measure of the active involvement of the stakeholders supporting the project.

Yadoo and Cruickshank note that local ownership at rural cooperatives were more effective at reducing energy theft due in part to peer pressure from the entire community [38]. Theft is therefore included as an indicator of community acceptance and ownership. In addition, the presence of community contributions to the development and/or ongoing operation of the project provides a good indicator of community ownership.

The following metrics, their notation, and type (binary or otherwise), for the social sustainability factor are described below:

  1. 1

    Needs assessment, Na, binary. Did a needs assessment take place?

  2. 2

    District involvement, D, binary. Is the district involved in the project in any way?

  3. 3

    Meeting frequency, λ. How often does the community meet? Six potential values representing more or less frequent meeting took place. 0=never, 1=once every year, 2=once every 6 months, 3=once every 2 months, 4= once every month, 5=once every 2 weeks, 6=once every week. This was re-scaled to range from 0 to 1.

  4. 4

    Theft, T, binary. Has theft occurred in the history of the project? For this metric, a higher score means a lower rate of theft.

  5. 5

    Number of stakeholders, Ns. How many stakeholders are there? The highest recorded number for all projects was 6. It is assumed in general for off-grid projects that more stakeholders have a positive effect on the social sustainability, representing a higher level of local engagement and wider support network. The score is scaled from 0 to 1 where 1 is given to projects with 3 stakeholders or more. Stakeholders beyond 3 were assumed to have a negligible effect on social sustainability. Scores for 2 stakeholders = 0.66, 1 = 0.33, and no stakeholders = 0.

  6. 6

    Community contribution, CC. Three types of community contribution were captured: money, materialsFootnote 2, or labor. An equal score was given for each type of contribution and scaled from 0 to 1.

The metrics are averaged with equal weighting to obtain an overall social sustainability factor, Fs.

Organizational sustainability factor

The motivation of the organizational sustainability is the capacity of the managing organization to effectively operate the project and maintain necessary human capital. Strong organizational personnel surrounding an off grid PV project implies local capacity to deal with issues that arise. As such, adequate initial or ongoing training provided to the organizational team is important [48]. Organizational sustainability metrics are defined as:

  1. 1

    Role filled, 3 metrics, binary (roles: technical, RT, managerial, RM, and financial RF). Is there an individual responsible for this role in the project? Yes =1, no =0.

  2. 2

    Initial role training, 3 metrics, binary (tT, tM, tF for each role respectively). Was training provided prior to installation? Yes =1, no =0.

  3. 3

    Ongoing role training, 3 metrics, binary (oT, oM, oF for each role respectively). Is ongoing training provided? Yes =1, no =0.

  4. 4

    Maintenance, binary, m. Is there a maintenance arrangement in place currently? Yes =1, no =0.

  5. 5

    Bank account, binary, β. Does the project have an formal bank account? Yes =1, no =0.

The metrics are averaged with equal weighting to obtain an overall organizational sustainability score, FO.

Technical sustainability factor

Technical sustainability refers to the quality of technical system design as well as the ongoing functionality of the sub-components. A set of technical field questions is designed to assess the apparent instantaneous health of components. In addition, technical sustainability is widely regarded as effected by system use and the appropriateness of the initial system design [49].

For each system, the quantified technical sustainability metrics include:

  1. 1

    Panel quality, QP. The online visibility of panel manufacturer and make is an indirect indicator of panel quality, assuming a more well established company with a good online presence is more likely to be offering quality panels. Under this assumption, a score of 1 indicates the panel manufacturer and make has online presence and a score of 0 if not.

  2. 2

    Battery quality, QB. A similar approach was taken for the quality of battery manufacturer where A score of 1 indicates the battery manufacturer and make has online presence and a score of 0 if not.

  3. 3

    Battery health, HB.; The specific gravity of the batteries are often available with an on-board hydrometer common on many battery models in Malawi. These typically have a red, orange, and green ‘’traffic light” indicator on the battery. Scoring was given as follows: 1= green, 0.5= orange, and 0= red.

  4. 4

    Ratio of actual usage against expected usage, Ruse. Respondent provided values for current actual usage (by appliance, daily usage, and rating) to the expected usage. If the system is not used at all, it receives a score of 0. The score is bounded at a maximum of 1 which occurs if actual usage fully meets, or exceeds, expected usage.

  5. 5

    Ratio of installed PV panel array size to optimal PV panel array size, RP. A score of 1 in this component refers to an optimally sized panel array (using an intuitive sizing methodology), with less than 1 considered undersized. A score of 1 is given for an optimally sized or oversized system. From a technical perspective, oversizing is usually positive, though it has economic implications.

  6. 6

    Installed battery bank size against optimal battery bank size rB. Calculated similar to the panel design, this is the ratio of designed battery sizing to an optimal battery sizing.

The technical indicators are averaged with equal weighting to obtain an overall technical sustainability score, FT.

It should be noted that scores among project groupings are the unweighted average of systems within each project where it occurs. The variable N continues to refer the number of projects that are included. Furthermore, projects that do not have full data available on all systems are censored from the analysis.

Economic sustainability factor

The economic sustainability score, FE, is compiled differently to the other sustainability factors. It consists of only one metric, which is the comparison between project income levels (collected from questionnaires) and the system costs. System costs are estimated from system components. Therefore, this factor relies on project cost modeling and the assumptions made during the modeling are described in this section.

Although the systems serve primarily health posts and schools (rather than a commercial operation), income is achieved through a number of routes, such as a maintenance budget provided by local government, community fund-raising, and creation of various income generation activities associated with the project. These monthly sources are summed and extrapolated to give an estimated project yearly income, incomeYR.

Project cost records are sparse within the data-set. Therefore, project costs are estimated as a function of panel and battery sizes. Capital expenditure (Cap) includes the panel costs which were calculated as $1.555/Wp and battery costs as $0.168/Wh. This was determined by a linear regression from a field survey of 20 local company quotes of each taken in 2015 [50]. Additional capital costs such as labor, installation, inverter, charge controller, and other costs are calculated as a proportion, 24.2%, of panel and battery costs. Equation 1 gives the Captot breakdown in $USD.

$$ \begin{aligned} Cap_{pan,bat} &= P_{Wp} * 1.555 + B_{Wh} * 0.168 \\ Cap_{tot} &= Cap_{pan,bat}*0.242 + Cap_{pan,bat} \end{aligned} $$
(1)

Operating expenditures (Opx) is assumed a standard 10% of the Cap per year. This is a conservative estimation, as a literature review shows between 1 and 6% of Cap as a standard OPEX yearly value [51, 52] for developed countries.

A simple depreciation model assumes a replacement for panels (every 20 years), batteries (every 5 years), and other capital equipment (every 10 years). In line with the authors experience, it is assumed that projects have been granted the initial capital equipment, so there it is assumed that there is no interest and debt repayment required. A 20-year total cost consisting of depreciation and total Opx costs is levelized yearly as shown in Eq. 2.

$$ \begin{aligned} Opx &= (Cap_{tot} * 0.1) * 20 \\ Deprec &= Cap_{pan} * 1 + Cap_{bat} * 4 + Cap_{other} * 2 \\ Costs_{Y} &= (Opx + Deprec) / 20 \end{aligned} $$
(2)

The economic sustainability factor FE is therefore the ratio of reported income to required income to cover the Opx calculated costs, Rincexp, and is capped at 1, as shown in Eq. 3.

$$ F_{E} = R_{incexp} = Income_{YR} / Costs_{YR} $$
(3)

Results

Social sustainability factor analysis

Table 1 shows how the metrics of ownership structure (D, Ns), involvement of stakeholders, and community contributions (Cc) varies significantly by the grouping categories of age, size, income, and project implementer.

Table 1 Average metric values (described in the “Social sustainability factor” section) contributing to social sustainability factor

The majority of projects conduct a needs assessment (Na), with a whole data set mean score of 0.646. CEDP and SEP projects almost always had a needs assessment (13 of 14), compared to 29 of 51 for OTHER projects. This weighting is reflected in the finding that younger projects are more likely to have had a needs assessment (mean score 0.783). Projects with some kind of income record were likely to have had a needs assessment (29 of 33) compared to 13 of 32 projects with no income records (mean score 0.406).

District involvement (D) was present for 28 of the 65 projects. District involvement was most prevalent for CEDP projects and for SEP projects while other projects scored 0.314. The meeting frequency, λ, mean score for all projects was relatively low, at 0.315, corresponding to meeting frequency of slightly less than once per 6 months. Most noticeably, the grouping of young projects or CEDP projects have more regular stakeholder meetings. The low scores for λ strongly shows that stakeholders do not meet to discuss the PV system for a significant number of projects. Within the project implementer groupings, the stakeholders for the majority of OTHER projects never meet.

Theft, T, was relatively uncommon throughout the data set with the average score of 0.738 among all projects. A higher score for this metric corresponds to less theft. Theft was reported in 17 of the 65 projects and was most likely in projects that were either CEDP or with a positive income (both scoring 0.500). Practices to reduce incidence of theft in Malawi have depended on a combination of practical preventive measures (locked spaces, hiring a security guard, avoiding the citing of equipment in places where it could be easily stolen, and transparent accounting processes) and ensuring community support is sufficient (project has local ownership, support from traditional authorities, project addresses a clearly identified need).

The mean score for the Ns metric was 0.410, corresponding to between 1 and 2 stakeholders. This can be considered to be low (a score of 1 on this metric corresponds to 3 or more stakeholders). As the CEDP and SEP projects scored 0.933 and 0.750 respectively, the mean overall value for Ns was pulled down by the OTHER projects with an average value of 0.281. Low scores are observed for projects that are old (0.206), small (0.250), and with no income (0.208). Types of stakeholders involved in the projects included community-based organizations, non-governmental organizations, government entities, traditional authorities, school representation, student representation, faith based, and a number of other types. The proportions of stakeholder involved in projects is shown Fig. 2. For community-based projects, multiple stakeholders were associated with higher project transparency and additional (human) resources for managing the project.

Fig. 2
figure2

Percentage of projects indicating the presence of specific stakeholders, by project grouping

An interesting result from Table 1 is the consistently low values for community contributions, Cc, with the lowest scoring metric of the social sustainability factor at 0.179. Lack of community contributions is a signal of lower community stake in the outcome of a project. Contributions are reported most often in the CEDP projects, corresponding to a local contribution of 2 of 3 items (money, labor, or materials). To some extent, this reflects the changing approach from the development community in Malawi more consistently insisting on higher community involvement, with specific contribution requirements, to proceed toward completion.

Considering overall social sustainability factor, the CEDP and SEP projects show an improved result compared to OTHER projects. However, this result is reported alongside the age grouping. Noting that the CEDP and SEP projects are relatively young, the result is therefore unsurprising as meeting frequency, theft, and the number of stakeholders engaged in the project will be affected by project age. The results show that the young projects are reporting more theft, which is unexpected. This highlights a need for future strategies to prevent and mitigate against theft, beyond setting up community ownership structures, such as physical locks, transparent book-keeping, and employment of security personnel.

Curiously, projects with negative income records outperform those with positive incomes in the social factor. Although having negative income is not intuitively an encouraging sign for overall sustainability, it appears that these projects are still being supported financially in some way, either by the community themselves or other concerned stakeholders. Additionally, they have higher district involvement, a lower rate of theft, and a high number of stakeholders in comparison the positive income group. This result points toward a recognition of the value and positive impact delivered by the project such that subsidy is seen as worthwhile and more generally indicates the possibility that projects with a high social sustainability may be able to compensate for deficiency in other factors.

Organizational sustainability factor results

Overall organizational sustainability factor, FO, is low among the dataset with a mean FO of 0.329. The metrics are summarized in Tables 2 and 3.

Table 2 Metric values for organizational sustainability factor, part 1
Table 3 Metric values for organizational sustainability factor, part 2

High organizational sustainability is achieved firstly through the filling of technical, managerial, and financial roles (Rt, Rm, and Rf respectively). The post-installation and ongoing training (T and O respectively) may be in place for each role type. As additional organizational sustainability metrics, existence of a maintenance arrangement m and a formal bank account β are included.

Unsurprisingly, the managerial role is most likely filled, with a mean metric value of 0.738. The technical role, Rt, was filled for less than half of all projects, with a mean score of 0.462, and the financial role was even less likely to be filled. The managerial role metric is consistently high for all project types and groupings. In contrast, technical and financial role metrics are low. Dedicated local technical expertise, with appropriate training, encourages local troubleshooting and minor fixes.

Considering project types, the SEP projects technical role metrics are comparatively high; however, the SEP projects managerial role metrics is relatively low. As the SEP projects were setup by an engineering group from the University of Malawi Polytechnic, it was logical that these projects placed relatively more emphasis on technical training. In addition, small projects were served with more technical staff when compared to mid or large projects.

The CEDP group shows the lowest score for the technical role; however, all CEDP have some sort of financial and managerial role locally. This reflects the initial and ongoing emphasis of the CEDP program to train business skills.

Although the CEDP and SEP projects focused specifically on capacity building locally, significant gaps in training are identified through this study. CEDP projects were mostly provided managerial training, with little focus on technical and financial roles. SEP projects were all provided technical training, with less focus on managerial and financial roles. Metrics for OTHER projects show little initial training, in particular for financial roles. Unsurprisingly, positive income projects tended to have relatively more consistent initial financial training. The lack of key positions at many projects and sparsity of training for key functions is common throughout the dataset and agrees with literature findings, both in Malawi and internationally for PV systems in developing countries. This study highlights the limited ongoing training affecting total project sustainability. To explore the extent of investment in local roles and skills development, initial training and on-going training metrics are plotted by project grouping in Fig. 3. On average, ongoing training metrics are low for all role types (technical, managerial, and financial) with values Ot=0.138, Om=0.231, and Of=0.138 respectively. Figure 3 shows how CEDP project metrics are higher with Ot=0.400, Om=0.700, and Of=0.700.

Fig. 3
figure3

Percentage of projects providing training after installation and prior to installation, within the three project groups, CEDP, SEP, and OTHER

CEDP systems have the highest overall organizational factor score, FO, at 0.582; however, all project groups score low for organizational sustainability. It is most noticeable that SEP and OTHER projects provided little or no initial and ongoing training, with the roles filled sporadically, whereas the CEDP projects are more consistent with filling and training financial and managerial positions.

For all other project differentiators (age, size, income), there is little difference between the organizational metrics and the total factor scores. Metrics are similar across the table (apart from project grouping and roles filled) ranging between 0.3 and 0.5. The metric for initial financial training, Tf, is low across all categories, with a mean score of 0.123 for all projects.

Basic maintenance arrangements are lacking, impacting both CEDP and OTHER projects. Projects with no income are particularly vulnerable to low organizational sustainability, with an overall organizational sustainability score of 0.224.

Finally, formal bank accounts, β, were uncommon for most projects with a mean metric of 0.260. The formal financial sector is absent from rural communities in Malawi, so owning a bank account for is a major undertaking. In addition to the practical value of having a bank account, it reflects community’s perceived importance of the project. CEDP projects routinely insisted on a community bank account, leading to an average value of 0.800. This is a significant change to usual project deployment (an SEP projects metric of β=0.000 and for OTHER projects β=0.176). As projects reporting a positive income were three times more likely to have a community bank account, the existence of a community bank account is expected to improve sustainability of CEDP systems.

Technical sustainability factor results

The average technical sustainability factor, FT, is 0.396. The individual metrics combine in FT and are summarized in Table 4 for each grouping.

Table 4 Average metrics values for technical sustainability factor

The metrics for panel quality, QP, and battery quality, QB, are relatively high, with average values of 0.620 and 0.717 respectively. Projects implemented by OTHER are more likely to have poorer quality panels. It is surprising that young projects have lower QP and QB compared to mid/old projects. This may be an indication of the continuing prevalence of low component standards despite general awareness of the issue in the country. Although quality components are important for projects to meet design specification, installation quality, which could not be captured fully in this study, is also a critical concern—the industry has had some notoriety for cutting corners (mislabeled PV capacity, reduced cable sizing, “knock-off” charging equipment, etc.) to the detriment of projects that lack sufficient technical support during procurement.

The metric for battery health, HB, has an average score of 0.787 meaning that most batteries were either found to be orange or green on the on-board hydrometer. As expected, older projects tended to have lower battery health indicators 0.558 compared to middle aged 0.812 and young 0.907 projects. The ratio of actual usage to expected usage, Ruse, shows a mean value of 0.599, meaning the majority of projects fail to meet user energy demands by roughly 40%. Small projects standout as the least able to meet expectations with a score of 0.400. One explanation is that users do not adjust usage expectations for smaller systems either because they have a minimum (expected) usage requirement or were unaware of appropriate system limits. There was a gradual trend that younger projects scoring 0.733 meet expectations better than middle aged 0.580 or old projects 0.515 suggesting that either expectations increase over time, system capacity drops, or both.

The Ruse metric was analyzed for different room types in Fig. 4, specifically for daily room lighting use. Usage expectations at staff houses were nearly met, though overall usage was relatively low. Maternity wards, outpatient departments (OPD), and classrooms had relatively high usage expectations, much larger than the actual usage. Security, staff houses, and other usage types were closest to meeting their expectations. Classrooms and health facilities had less predictable use patterns, shown by the flatter distribution, and had larger gaps between actual usage and expectation.

Fig. 4
figure4

Density plot of daily lighting consumption in watt-hours by room type. Actual use (red) is plotted against expected use (blue), with the averages represented by a dashed line

Figure 5 compares the actual size of the system against an optimal size for the expected system loads. To determine the optimal size, an intuitive sizing methodology is used, based on the “ad hoc” methodology used in Malawi [50]. This is an important metric for technical sustainability as undersizing can produce low reliability whereas oversizing may represent low value for money. Panel sizing, RP, and battery sizing, RB, had mean scores of 0.525 and 0.574 respectively. While most groupings have broadly similar scores, the exception is the implementer grouping. SEP projects had high metric values at RP of 0.890 and RB of 0.967, CEDP with RP of 0.611 and RB of 0.715, and OTHER with RP of 0.472 and RB of 0.522.

Fig. 5
figure5

Ratio of optimal to actual sizing of panels and batteries. The box center line is the data median with the 25% and 75% percentiles at the box edges. The whiskers equal 1.5 times the interquartile range. Solid points indicate outliers outside the whisker range. The dashed line indicates the optimal sizing of the system, which is normalized to 100, based on the reported desired consumption for each system

Figure 5 shows that while there are a number of oversized systems based on the outliers, the majority of systems were undersized for both battery and panels. The median battery and panel sizes were 54% and 79% of the optimal sizing. Old projects were particularly undersized in panels (by about half) as well as those without an income.

The consumption changes in the systems, many of which are have been operating for 3 years or more, may have invalidated a design which was originally “optimal”. This observation points to the need for dedicated study of consumption over time and, when designing systems, more sophisticated demand modeling or use of comparable datasets especially considering the high potential error involved with energy use surveys [53].

Economic factor results

The economic factor, FE, is determined by one metric which is the ratio of reported annual income to yearly estimated costs. The overall FE average is the lowest average of all the sustainability factors, at 0.162 (as shown in Table 5). This is heavily affected by 32 projects, which have no reported income, forcing Rincexp and FE to 0. Only 9 of the 65 projects scored over 0.800—which could cover most of the long-term project costs. The highest FE score of 0.623 unsurprisingly correlates to positive income project grouping. Both CEDP (0.374) and small (0.375) projects outperformed the other comparands in their respective groupings. Age of project seems to have very little effect on FE.

Table 5 Economic Factor

The extremely low average score indicates that most projects have insufficient revenue generation to support their current and expected future costs, with nearly half of all projects having no reported income at all. CEDP projects, the best performing implementer group, were designed to include one or more income generating activities (IGAs) in support of the project. SEP projects did not emphasize income generation, leading to negative net yearly income and dependence on community financial support. Small projects scored relatively highly, likely due to the lowered capital replacement costs required. Conversely, large projects scored close to 0, indicating that higher capacity systems were not typically used for economic activities. Generally, low scores suggest that the number of IGAs should be expanded and a higher proportion of energy produced should be sold commercially. The scoring assumed that all projects were funded by grants and thus had no capital repayment costs, a fair assumption in Malawi. However, it is worth noting that economic sustainability with a more stringent model involving capital repayment would be impossible with included projects.

Further analysis of the main types of income sources found within the projects is useful for learning more about success cases. Formal documentation of revenue generation for specific business activities in rural projects has been virtually non-existent and, when present, is usually limited to mobile phone charging. Figure 6 provides a breakdown of the mean monthly income from a range of income generating activities. Nearly all income generation activities are within CEDP projects. The largest revenue generating sources were barbershops, MWK 66,000 per month, and mobile phone charging at MWK 54,000 per month. CEDP had the highest mean monthly income of MWK 191,458, from all income sources.

Fig. 6
figure6

Monthly income in Malawi Kwacha (MWK) by types. Data is shown for all three implementation groups, CEDP, SEP and OTHER

The reported cost structure of the projects are not usually available but have been captured in this study for the projects in the data set. Figure 7 plots the average yearly income and yearly OPEX as bar charts with net income (dark blue bar) as the far right bar. The result is given for the following: all projects (top-left panel), CEDP projects (top-right), SEP projects (bottom-left), and OTHER projects (bottom-right). The All Projects graph shows how projects are usually financially balanced, as the far right (dark blue) bar is close to 0 (the yearly expenses meet the yearly income). In each graph, the first bar is the yearly income, the following four bars are the breakdown of expenses and the final bar is net income.

Fig. 7
figure7

Yearly cash flow for each project grouping and all projects. Positive bars indicate project income, negative bars represent expenditure, and the far right bar (dark blue) on each plot is the net yearly financial balance

Figure 7 shows that the CEDP projects (with average yearly net income of MWK 399,041) cancel against the negative net income from other projects, leading to a total All_projects yearly net income of MWK 61,819. OTHER projects have both low costs and income with a negligible net income. In the SEP projects, the expensive replacement of a key power system component in one of the projects occurred in the year the survey was completed. Such costs are inevitable to most off-grid projects and demonstrate the need to save sufficiency for when equipment must be replaced. As a proportion of average yearly income, equipment expense was 22.4%, OPEX was 39.5%, cost of free goods and services (for example free phone charging for school staff members or specific people from the community) was 6.7%, and other non-system expenditures was 12.4%.

Overall sustainability FSustainability

Distributions of sustainability factor scores are plotted by project in Fig. 8. In previous sections, the sustainability factor values (and their metrics) are averaged within project groupings. In contrast, Fig. 8 plots a point for each individual project (color coded based on implementer SEP, CEDP, or OTHER) for economic, organizational, social, technical, and finally total sustainability factor, FSustainability. For each sustainability factor, the average is given as a solid black line. The spread of the data points around the average are shown.

Fig. 8
figure8

FE,FO,FS,FT, and overall FSustainability. All projects are plotted and color coded in project implementer grouping. The mean is given as a solid black line for each factor and the spread around the mean for the projects surveyed is shown

In Fig. 8, organizational, social, and technical factors are shown to be distributed evenly around the factor mean. However, the economic factor scores are polarized at the minimum and maximum points. The bunching of scores at discrete levels for organizational and social factors is an artifact of the binary or discrete step metric scores for these factors. The aggregate, total score for each project is represented in the total distribution, which has a mean of 0.318. The economic factor has a significant negative effect on total project sustainability for these projects.

All of the factors display a large spread, nearly spanning the possible range of 0 to 1. Although there are individual examples of high performance for each factor, only two projects (both CEDP) have sustainability values higher than 0.5 for every factor. Overall, there are some relatively high FSustainability, with 14 projects over 0.5.

Discussion

Overall, the results for the 65 surveyed projects present a picture of poor sustainability. Many projects score exceedingly low in several factors; this is reflected in the mean scores for each factor as well as low mean score for overall sustainability.

Despite this conclusion, a focus on community capacity building and a high level of technical input at the design stage has been shown to partially improve sustainability. The results show that, for all sustainability factors, the CEDP projects generally score higher and have a mean score that is consistently above the total mean for all projects.

There were three specific aspects of community development from the CEDP projects that were successful. First, the selection of community groups (CBOs) that had a track record of successful local development projects. Second, the process for building project business plans were co-developed by the community and project coordinator and was coupled with educational activities (such as financial management and technical skills) where gaps in knowledge were identified. Third, there was a clear hand over of project ownership to the community and recognition early on by the external partners to the project development to avoid creating dependencies and phase out their involvement when appropriate.

SEP projects tend to out perform other project groups in the technical factor but generally have lower scores in other factors. Technical performance has been improved by SEP and CEDP when compared to OTHER, as systems appear to be sized better for current consumption expectations and are more often meeting those expectations.

Efforts to improve the quality of inputs, in particular better load and resource modeling, directly improve system sizing calculation. Intuitive PV system sizing methodologies, used by all the projects within the study, can be tailored to be more or less conservative and hence cannot be relied on to produce identical component sizes [50]. Although these have the appeal of simplicity, they are better suited for pre-feasibility stages with computer aided modeling and simulation serving as a more robust methods for PV system sizing [54]. Changing user expectations over the lifetime of a project have been difficult to predict as the users leave the project and are replaced with new users, behaviors change with more experience using electricity, and long-term load growth materializes (or not). In contrast to undersizing a system, oversizing creates an financial burden for the project that undermines its economic feasibility [55]. With a system design only being optimized for a window of electricity usage, this suggests aiming for design flexibility where possible and to scale the system with the revealed load while minimizing financial risks.

The most significant finding is the obvious need for further development of models for economic sustainability. It is apparent that even where significant focus has been placed on community capacity building and establishing income generation activities, insufficient revenues are being collected. This analysis indicates that community based management of PV systems located on public education and health institutions requires an external revenue stream. Without an appropriately sized operations and maintenance budget provided by the relevant local authority, or an equivalent public body, economic sustainability is unlikely to be achieved. Potential options that could be integrated into future projects have been documented in the Dedza district. A recent case study has found commercial viability of refrigeration, barber shop, shop lighting, irrigation, and entertainment (TV shows) [56].

Implications for project developers

The methodology presented in this paper gives practitioners and project developers a framework that is useful when planning for sustainable off grid PV projects in Malawi. Each of the factors (and their metrics) outlined can be utilized as a checklist when planning a project and actions taken to maximize the sustainability score in each category.

Specific examples of good practice leading to higher scoring in the social factor include engaging with a variety of stakeholders, engaging with district councils, formalizing stakeholder roles, and establishing a system support structure for the lifespan of the project. Taking steps to mitigate against theft should be seriously considered. Although younger, CEDP projects scored highly in community engagement and ownership, they were more prone to theft, implying that simply focusing on generating a sense of community ownership is not enough and new projects require specific security policies.

Given the low organizational scores, practitioners should focus on utilizing a delivery model with sufficient local capacity to manage the project and organize system maintenance, specifically fulfilling technical and financial roles as these were missing from most of the projects surveyed. Increasing the level of training provision throughout the project life cycle is needed, potentially through engagement of local educational institutions or appropriate technical professional bodies.

Regarding technical metrics, the fact that on average only 60% of the consumption was met by the systems implies that better understanding of load profiles is required to prevent mis-sizing of systems. The majority of systems were undersized for both battery and panels, with high variation in sizing error in general, suggesting that no clear sizing standard has been applied. Investing in standardized sizing methodologies and training for system designers are clear steps practitioners can take to increase system sustainability. Absence of high-quality load profile data sets of users first gaining access to electricity hampers energy use prediction and thus sizing efforts. There is also an issue with standard of components, particularly with newer projects.

The biggest implication for practitioners comes from the economic factor scores being the lowest of all categories, which highlights a gap in economic planning at the design and installation stage. Establishing an appropriate maintenance and operation budget should be a key test for the feasibility of a PV project. Without this, the project should not progress to installation. Furthermore, cash flow estimations from this data can be used to validate business modeling expectations.

Implications for policy makers

In current modes of deployment, community-based solar PV systems in Malawi have been shown to suffer from shortfalls in every sustainability category. Without improvement, the model is arguably an inviable mechanism to deliver off-grid electricity access at rural public facilities leading to a high risk of failure. Centralized support for energy projects and more widely off-grid energy infrastructure has historically been virtually non-existent. All projects included in the data set involved public infrastructure, yet there was no governmental income recorded, a finding which is striking as ostensibly electricity bills are paid for public facilities which have grid access.

Typical stakeholders in communities generally do not have sufficient skills for effective management of all project aspects. Policy makers should investigate models where district authorities can partner and support community energy projects for education and health infrastructure, taking into account the cost structure and technical support requirements of deployed PV systems [10]. Another option is direct management and responsibility by the district over the electrical assets.

A robust financial framework under which the government could afford to pay a regular monthly electricity bill for public facilities in rural areas would significantly impact and change the landscape, ultimately attracting further non-government investment into PV system development.

Methodological implications

Several challenges with the methodology were found and are discussed in this section, primarily for the benefit of scholars conducting similar studies.

The coverage of metrics of the major aspects affecting sustainability of a project is critical for the analysis to be relevant. For example, coverage of only one aspect of organizational sustainability, say the provision of financial training, is insufficient alone to conclude on sustainability. Each factor in this study draws from multiple metrics that together have been supported by the literature or from the author’s experience. The one exception which has only one metric, the economic factor (FE), was due to limited or poor quality data. Although the financial structure of the project is perhaps the most important aspect, additional metrics were not possible to add such as level of competition and level of (re-)investment in the project. Consideration of temporal nature of the metrics themselves is also important as, using the same example, training conducted at the start of a project may no longer be relevant if there is a regular turnover of staff. The inclusion of specific metrics in the study was, in a practical sense, limited by the availability of data. Quality components of many metrics in many cases could not be meaningfully captured, for example, the level of social and gender inclusion in the needs assessment and community engagement stages of a project, skill level of personnel assigned to specific project roles, and severity of theft.

The option of weighting metrics individually and factors as a whole was found to be problematic. Since robust longitudinal studies that quantify the degree to which each sustainability factor score and metric actually influences sustainability have not been published, we argue that assigning an arbitrary weighting is not justified. The use of metrics developed through consultation from the bottom-up do not guarantee that those chosen meet the criteria of comprehensive coverage, relevance, and practicality, though may be valuable for impact measurement. The use of sustainability metrics which overlap a specific subjectFootnote 3 is another form of weighting. Instead, we recommend striving for a comprehensive coverage of each factor with distinct non-overlapping metrics.

Since the study depended on respondents, biases such as interviewer and respondent biases are likely present. This was mitigated as much as possible by interviewer training, use of pilots to refine the survey, survey design that structured the approach, and using verifiable and quantifiable data when available. As many projects lacked accurate and consistent record-keeping, the data was dependent on respondent recollection.

The results and analysis presented here provide a basis against which future data can be collected to validate the accuracy of this method (and future developed methods) as a continued analysis of long-term sustainability. A major inherent weakness in sustainability evaluations, presented here and elsewhere, is the lack of time-series data, as evaluators must often resort to data gathered during a single field visit which limits evidence to support the final analysis. It is expected that data collection and analysis of these sustainability factors, over several years, is required to obtain a sufficient evidence base to enable a deep understanding of the relative influence of the different sustainability factors for community energy projects in a variety of contexts.

Conclusions

This paper presents a sustainability analysis of 65 off-grid, community solar PV projects in Malawi, utilizing a novel scoring methodology comprising of social, organizational, technical, and economic factors. Each project was associated with the provision of electricity to a rural public facility such as a primary school or health center. The research found that the sustainability of these projects is lacking in many key areas and overall the outlook for long-term sustainability is poor. On a scale normalized from 0 to 1, overall sustainability averaged 0.318. The analysis found that community-based management of PV systems at public education and health institutions have serious limitations on their sustainability. In the context of overall poor performance, economic sustainability is the most challenging factor where 32 of the 65 projects had no income whatsoever and only 9 were covering at least 80% of their long-term costs. Key recommendations for practitioners and policy makers focus on the need to establish maintenance and operations revenue streams prior to deployment. In particular, policy options for support for such systems at a district government level need to be further developed.

This paper provides a basis against which future data and methods can be compared, to ultimately tackle the persistent and significant issue of poor PV system sustainability in developing countries. A major inherent weakness in past sustainability evaluations in literature is the lack of sustained time series data. While we have provided some contributions to methodology, further experience documenting experiences with the metrics in this paper, as well as others, must be addressed in order to further refine sustainability analysis methods.

Availability of data and materials

Anonymized data and instruments (in digital format) from this research can be made available upon request to the corresponding author.

Notes

  1. 1.

    As data gathered from this study found no negative environmental issues, due to the inherently negligible impacts of solar PV deployments, the remainder of the study removes the environmental sustainability factor.

  2. 2.

    The “materials” metric refers specifically to PV system or civil/structural capital, provided in-kind, and does not include, for example, land or local permissions (such as letter of support from local traditional authorities), as the community provision of this would be a requirement for project commissioning.

  3. 3.

    For example, two conceivable metrics could be (A) “ratio of actual versus expected consumption” and (B) “hours of access per day.” If the ratio of (A) is low, one would also expect to find similarly low (B) number of hours of access.

Abbreviations

ADC:

Area Development Committees

CBO:

Community based organization

CEDP:

Community Energy Development Program

DC:

District Council

IAEA:

International Atomic Energy Agency

IGA:

Income generating activities

MAREP:

Malawi Rural Electrification Program

MREAP:

Malawi Renewable Energy Acceleration Programme

NGO:

Non-governmental organizations

OPD:

Outpatient departments

PV:

Photovoltaic

SEP:

(MREAP) Strategic Energy Project

UN:

United Nations

W:

Watts

References

  1. 1

    International Energy Agency (2018) World Energy Outlook 2018. https://doi.org/10.1787/weo-2018-en. https://www.oecd-ilibrary.org/content/publication/weo-2018-en.

  2. 2

    United Nations (2016) Sustainable Development Goal 7: Ensure access to affordable, reliable, sustainable and modern energy for all. https://sustainabledevelopment.un.org/sdg7. Accessed 15 Mar 2019.

  3. 3

    UNDP (2017) Human development reports. Technical report, UNDP.

  4. 4

    World Bank IEA ESMAP (2017) Sustainable Energy for All (SE4ALL) database from the SE4ALL Global Tracking Framework. https://data.worldbank.org/indicator/EG.ELC.ACCS.ZS?end=2014&locations=MW&start=1990&view=chart. Accessed 15 Feb 18.

  5. 5

    Zalengera C, Blanchard RE, Eames PC, Juma AM, Chitawo ML, Gondwe KT (2014) Overview of the Malawi energy situation and A PESTLE analysis for sustainable development of renewable energy. Renew Sust Energ Rev 38:335–347. https://doi.org/10.1016/j.rser.2014.05.050.

    Article  Google Scholar 

  6. 6

    Taulo JL, Gondwe KJ, Sebitosi AB (2015) Energy supply in Malawi: options and issues. J Energy Southern Afr 26(2). https://doi.org/10.17159/2413-3051/2015/v26i2a2192.

    Article  Google Scholar 

  7. 7

    Girdis D, Hoskote M (2005) Malawi: rural energy and institutional development. 069. Technical paper, ESMAP. World Bank, Washington, DC.

  8. 8

    Adkins E, Eapen S, Kaluwile F, Nair G, Modi V (2010) Off-grid energy services for the poor: introducing LED lighting in the Millennium Villages Project in Malawi. Energy Policy 38(2):1087–1097. https://doi.org/10.1016/j.enpol.2009.10.061.

    Article  Google Scholar 

  9. 9

    Frame D, Dauenhauer P, Tembo K, Currie C, Ault G (2013) An integrated approach to energy for development: case study of the Malawi Renewable Energy Acceleration Programme In: 2013 IEEE Global Humanitarian Technology Conference (GHTC), 72–77. https://doi.org/10.1109/GHTC.2013.6713657.

  10. 10

    Buckland H, Eales A, Brown E, Cloke J, Blanchard R, Yona L, Zalengera C, Batchelor S, Sieff R, Nyirenda E, Bayani E (2017) Malawi District Energy Officer Blueprint: Recommendations Paper. University of Strathclyde, Glasgow.

    Google Scholar 

  11. 11

    Sumanik-Leary J, Org M, Little M, Persson J, Kalonga C, Eales A, Yona L, Kaunda M, Kamwendo M, Bayani E (2016) Market Assessment for locally manufactured PV-wind hybrid systems in Malawi. University of Strathclyde, Glasgow.

    Google Scholar 

  12. 12

    Banks DI, Gondwe K (2005) Barrier removal to renewable energy in Malawi - draft mid term review report. Technical report. United Nations Development Programme, Lilongwe.

    Google Scholar 

  13. 13

    UNDP (2015) Increasing access to clean and affordable decentralised energy services in selected vulnerable areas of Malawi. United Nations Development Programme (UNDP), Lilongwe.

    Google Scholar 

  14. 14

    Way C, Upadhya S (2011) Assessment of development results: Malawi. Technical report. UNDP Evaluation Office.

  15. 15

    Currie C, Banda G, Dinsmore H, Henderson S, Kafakoma R, O’Reilly S, Watson S (2012) Evaluation of off-grid community managed renewable energy projects in malawi: dedicated study. In: Dauenhauer P (ed)Evaluation of Off-grid Community Managed Renewable Energy Projects in Malawi.. University of Strathclyde. Summary Report.

  16. 16

    Dauenhauer PM, Frame DF (2016) Sustainability analysis of off-grid community solar PV projects in malawi In: 2016 IEEE Global Humanitarian Technology Conference (GHTC), 113–120. https://doi.org/10.1109/GHTC.2016.7857268.

  17. 17

    Government of Malawi (2018) Malawi Renewable Energy Strategy. Malawi Energy Regulatory Authority, Lilongwe, Malawi. https://www.meramalawi.mw/index.php/resource-center/other-regulatory-tools/send/20-other-regulatory-tools/61-malawi-renewable-energy-strategy.

  18. 18

    Sovacool BK, D’Agostino AL, Bambawale MJ (2011) The socio-technical barriers to solar home systems (SHS) in Papua New Guinea: choosing pigs, prostitutes, and poker chips over panels. Energy Policy 39(3):1532–1542. https://doi.org/10.1016/j.enpol.2010.12.027.

    Article  Google Scholar 

  19. 19

    Palit D, Sovacool BK, Cooper C, Zoppo D, Eidsness J, Crafton M, Johnson K, Clarke S (2013) The trials and tribulations of the Village Energy Security Programme (VESP) in India. Energy Policy 57:407–417. https://doi.org/10.1016/j.enpol.2013.02.006.

    Article  Google Scholar 

  20. 20

    Mabuza LO, Brent AC, Mapako M (2007) The transfer of energy technologies in a developing country context - towards improved practice from past successes and failures In: Proceedings of World Academy of Science, Engineering, and Technology (2007), 237–241. http://hdl.handle.net/2263/4854.

  21. 21

    Brent AC, Rogers DE (2010) Renewable rural electrification: sustainability assessment of mini-hybrid off-grid technological systems in the African context. Renew Energy 35(1):257–265. https://doi.org/10.1016/j.renene.2009.03.028.

    Article  Google Scholar 

  22. 22

    Ikejemba EC, Schuur PC, Van Hillegersberg J, Mpuan PB (2017) Failures & generic recommendations towards the sustainable management of renewable energy projects in Sub-Saharan Africa (Part 2 of 2). Renew Energy 113:639–647. https://doi.org/10.1016/j.renene.2017.06.002.

    Article  Google Scholar 

  23. 23

    Ikejemba EC, Mpuan PB, Schuur PC, Van Hillegersberg J (2017) The empirical reality & sustainable management failures of renewable energy projects in Sub-Saharan Africa (part 1 of 2). Renew Energy 102:234–240. https://doi.org/10.1016/j.renene.2016.10.037.

    Article  Google Scholar 

  24. 24

    Urmee T, Md A (2016) Social, cultural and political dimensions of off-grid renewable energy programs in developing countries. Renew Energy 93:159–167. https://doi.org/10.1016/j.renene.2016.02.040.

    Article  Google Scholar 

  25. 25

    Urmee T, Harries D (2011) Determinants of the success and sustainability of Bangladesh’s SHS program. Renew Energy 36(11):2822–2830. https://doi.org/10.1016/j.renene.2011.04.021.

    Article  Google Scholar 

  26. 26

    Terrapon-Pfaff J, Dienst C, König J, Ortiz W (2014) A cross-sectional review: Impacts and sustainability of small-scale renewable energy projects in developing countries. Renew Sust Energ Rev 40:1–10. https://doi.org/10.1016/j.rser.2014.07.161.

    Article  Google Scholar 

  27. 27

    Terrapon-Pfaff J, Dienst C, König J, Ortiz W (2014) How effective are small-scale energy interventions in developing countries? Results from a post-evaluation on project-level. Appl Energy 135:809–814. https://doi.org/10.1016/j.apenergy.2014.05.032.

    Article  Google Scholar 

  28. 28

    Ilskog E, Kjellström B (2008) And then they lived sustainably ever after? Assessment of rural electrification cases by means of indicators. Energy Policy 36(7):2674–2684. https://doi.org/10.1016/j.enpol.2008.03.022.

    Article  Google Scholar 

  29. 29

    Barman M, Mahapatra S, Palit D, Chaudhury MK (2017) Performance and impact evaluation of solar home lighting systems on the rural livelihood in assam, india. Energy Sustain Dev 38:10–20. https://doi.org/10.1016/j.esd.2017.02.004.

    Article  Google Scholar 

  30. 30

    Hong GW, Abe N (2012) Sustainability assessment of renewable energy projects for off-grid rural electrification: The Pangan-an Island case in the Philippines. Renew Sust Energ Rev 16(1):54–64. https://doi.org/10.1016/j.rser.2011.07.136.

    Article  Google Scholar 

  31. 31

    Bhandari R, Septalena LG, Kusch W (2018) Sustainability assessment of a micro hydropower plant in Nepal. Energy Sustain Soc 8(1). https://doi.org/10.1186/s13705-018-0147-2.

  32. 32

    Elusakin Julius E, Olufemi AO, Chuks DJ (2014) Challenges of sustaining off-grid power generation in Nigeria rural communities. Afr J Eng Res 2(2):51–57.

    Google Scholar 

  33. 33

    Feron S, Heinrichs H, Cordero RR (2016) Are the rural electrification efforts in the Ecuadorian Amazon sustainable?Sustainability 8(5). https://doi.org/10.3390/su8050443.

    Article  Google Scholar 

  34. 34

    International Atomic Energy Agency (IAEA) (2005) Energy indicators for sustainable development: methodologies and guidelines. Report:1–161.

  35. 35

    United Nations (2007) Indicators of sustainable development: guidelines and methodologies. Technical Report. United Nations.

  36. 36

    Purvis B, Mao Y, Robinson D (2018) Three pillars of sustainability: in search of conceptual origins. Sustain Sci. https://doi.org/10.1007/s11625-018-0627-5.

    Article  Google Scholar 

  37. 37

    Ilskog E (2008) Indicators for assessment of rural electrification - an approach for the comparison of apples and pears. Energy Policy 36:2665–2673. https://doi.org/10.1016/j.enpol.2008.03.023.

    Article  Google Scholar 

  38. 38

    Yadoo A, Cruickshank H (2012) The role for low carbon electrification technologies in poverty reduction and climate change strategies: a focus on renewable energy mini-grids with case studies in Nepal, Peru and Kenya. Energy Policy 42:591–602. https://doi.org/10.1016/j.enpol.2011.12.029.

    Article  Google Scholar 

  39. 39

    Bhattacharyya SC (2012) Energy access programmes and sustainable development: a critical review and analysis. Energy Sustain Dev 16(3):260–271. https://doi.org/10.1016/j.esd.2012.05.002.

    Article  Google Scholar 

  40. 40

    Lillo P, Ferrer-Martì L, Àlvaro Fernàndez-Baldor, Ramìrez B (2015) A new integral management model and evaluation method to enhance sustainability of renewable energy projects for energy and sanitation services. Energy Sustain Dev 29:1–12. https://doi.org/10.1016/j.esd.2015.08.003.

    Article  Google Scholar 

  41. 41

    Eales A, Walley L, Buckland H, Frame D, Strachan S (2018) Social Impacts of mini-grids: towards an evaluation methodology In: 2018 IEEE PES/IAS PowerAfrica, 354–359. https://doi.org/10.1109/PowerAfrica.2018.8521049.

  42. 42

    Katre A, Tozzi A (2018) Assessing the Sustainability of decentralized renewable energy systems: a comprehensive framework with analytical methods. Sustainability 10(4). https://doi.org/10.3390/su10041058.

    Article  Google Scholar 

  43. 43

    Purwanto WW, Afifah N (2016) Assessing the impact of techno socioeconomic factors on sustainability indicators of microhydro power projects in Indonesia: A comparative study. Renew Energy 93:312–322. https://doi.org/10.1016/j.renene.2016.02.071.

    Article  Google Scholar 

  44. 44

    Rahman MM, Paatero JV, Lahdelma R (2013) Evaluation of choices for sustainable rural electrification in developing countries: a multicriteria approach. Energy Policy 59:589–599. https://doi.org/10.1016/j.enpol.2013.04.017.

    Article  Google Scholar 

  45. 45

    Katre A, Tozzi A, Bhattacharyya S (2019) Sustainability of community-owned mini-grids: evidence from india. Energy Sustain Soc 9(1):2. https://doi.org/10.1186/s13705-018-0185-9.

    Article  Google Scholar 

  46. 46

    Hartung C, Lerer A, Anokwa Y, Tseng C, Brunette W, Borriello G (2010) Open data kit: tools to build information services for developing regions In: Proceedings of the 4th ACM/IEEE International Conference on Information and Communication Technologies and Development. ICTD ’10, 18–11812.. ACM, New York. https://doi.org/10.1145/2369220.2369236.

    Google Scholar 

  47. 47

    Belliveau J (2016) Humanitarian access and technology: opportunities and applications. Procedia Eng 159:300–306. https://doi.org/10.1016/j.proeng.2016.08.182. Humanitarian Technology: Science, Systems and Global Impact 2016, HumTech2016.

    Article  Google Scholar 

  48. 48

    Terrado E, Cabraal A, Mukherjee I (2008) Operational guidance for World Bank group staff: designing sustainable off-grid rural electrification projects: principles and practices. World Bank, Energy and Mining Sector Board, Washington, DC.

    Google Scholar 

  49. 49

    Feron S, Cordero R, Labbe F (2017) Rural electrification efforts based on off-grid photovoltaic systems in the Andean Region: comparative assessment of their sustainability. Sustainability 9(1825). https://doi.org/10.3390/su9101825.

    Article  Google Scholar 

  50. 50

    Louie H, Dauenhauer P (2016) Effects of load estimation error on small-scale off-grid photovoltaic system design, cost and reliability. Energy Sustain Dev 34:30–43. https://doi.org/10.1016/j.esd.2016.08.002.

    Article  Google Scholar 

  51. 51

    Committee on Climate Change MottMacDonald (2011) Costs of low-carbon generation technologies. https://www.theccc.org.uk. Accessed 9 Apr 2017.

  52. 52

    National Renewable Energy Laboratory, Energy Analysis (2016) Distributed Generation Renewable Energy Estimate of Costs. https://www.nrel.gov/analysis/tech-lcoe-re-cost-est.html. Accessed 1 Nov 2019.

  53. 53

    Blodgett C, Dauenhauer P, Louie H, Kickham L (2017) Accuracy of energy-use surveys in predicting rural mini-grid user consumption. Energy Sustain Dev 41:88–105. https://doi.org/10.1016/j.esd.2017.08.002.

    Article  Google Scholar 

  54. 54

    Boait P, Advani V, Gammon R (2015) Estimation of demand diversity and daily demand profile for off-grid electrification in developing countries. Energy Sustain Dev 29:135–141. https://doi.org/10.1016/j.esd.2015.10.009.

    Article  Google Scholar 

  55. 55

    Williams NJ, Jaramillo P, Taneja J, Ustun TS (2015) Enabling private sector investment in microgrid-based rural electrification in developing countries: a review. Renew Sustain Energy Rev 52:1268–1281. https://doi.org/10.1016/j.rser.2015.07.153.

    Article  Google Scholar 

  56. 56

    Smith K, Eales A, Frame D, Galloway S (2019) Techno-economic analysis of PAYG productive uses of energy in Malawi.. IEEE Global Humanitarian Technology Conference 2019. https://strathprints.strath.ac.uk/70490/.

Download references

Acknowledgements

This work would not have been possible without the support of the Malawi Polytechnic and Mzuzu University. Additionally, we would like to thank the organization Renama, who assisted in the identification of some locations.

Funding

This research was funded under the Malawi Renewable Acceleration Programme by the Scottish Government.

Author information

Affiliations

Authors

Contributions

Writing of this article was a collaborative effort between the named authors. PMD and DF were primarily responsible for the original design of the sustainability survey. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Peter M. Dauenhauer.

Ethics declarations

Ethics approval and consent to participate

The research team has concluded that there are no significant ethical issues involved in this research. The data was primarily technical, related to projects or systems, not humans, and did not capture any personal details. The survey was piloted by each enumerator who received training from the research team. The preamble of the survey noted that data would be used for research purposes only and that there were no direct benefits to participation nor would any specific decision be made as a result of any answers given. Participation was voluntary and confidential; respondents were given opportunities to stop the survey or skip questions at any time if they felt uncomfortable or unable answering any questions.

Consent for publication

All participants consented the confidential publication of their contributions in this study.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Dauenhauer, P.M., Frame, D., Eales, A. et al. Sustainability evaluation of community-based, solar photovoltaic projects in Malawi. Energ Sustain Soc 10, 12 (2020). https://doi.org/10.1186/s13705-020-0241-0

Download citation

Keywords

  • Sustainability
  • Solar photovoltaics
  • Community energy
  • Evaluation
  • Rural public facilities
  • Indicator frameworks