Skip to main content
  • Original article
  • Open access
  • Published:

Uncertainty analysis of the environmental sustainability of biofuels

Abstract

Background

Faced with a changing climate, bioenergy has been promoted as a sustainable resource. However, while it is a renewable energy source, biofuel cultivation comes with several environmental problems such as land use change and water consumption, the environmental impacts of which often counterbalance the reduced global warming potential compared to fossil alternatives.

Methods

This paper presents life cycle assessment (LCA) results for greenhouse gas (GHG) emissions (carbon footprints) and water consumption (water scarcity footprint) for different first-generation biofuels in comparison with their fossil alternatives. We employed standard (secondary) LCA data including uncertainties of process activities, resource consumption, and emissions throughout the value chain in order to compare different options under the influence of uncertainties and in order to identify the contribution to variance (CTV) of input data, giving insight into which environmental flows need to be better assessed. Furthermore, by also introducing uncertainties in the impact assessment for GHG emissions and water consumption, we were able to determine which LCA stage is more influenced by uncertainties, the accounting or impact assessment stage. Additionally, we analyzed the effect of choosing different time horizons GHGs (typically set to 100 years).

Results

The analyzed fuels perform differently depending on the choice of impact category considered. For liquid fuels, we observed a tradeoff between resource depletion and water footprint, while biogas options have lower impacts in most categories. Biogas from waste has significantly lower carbon footprints than natural gas for long time horizons and a similar water footprint. However, with the 20-year Intergovernmental Panel on Climate Change (IPCC) factors, methane emissions from biogas largely compensate the fossil CO2 emissions in our case and no robust difference is observed under uncertainty considerations. Both impact assessment and inventory uncertainties are important. Due to the very high number of parameters, the CTV analysis was not robust for assessing GHG emissions.

Conclusions

This study shows that uncertainty is important in LCA and carbon or water footprint assessment of agricultural feedstock production. Integration of parameter uncertainties helps to evaluate the significance of the difference from two product options. For biogas, the choice of the time horizon in carbon footprint assessment is decisive and needs a strong justification.

Background

Biofuels have gained attention in the last decade in the context of sustainable resource use and global warming. Many governments or regulating bodies, such as the US and the EU, have formulated goals regarding biofuel production or use. The US regulation has placed a target of annual biofuel production of 36 billion US gallons (136 million m3) by 2022, of which 79 million m3 should be second generation biofuels [1]. The Renewable Energy Directive (RED) [2] of the European Commission demands 10 % renewable fuels for road transport in each member state by 2020. While renewable fuels include electricity and hydrogen produced from renewable sources, biofuels seem to be the most relevant fuels to reach this goal [3]. Other countries such as Switzerland grant tax exemptions for sustainable biofuels [4].

However, bioenergy requires biomass cultivation, which has a variety of environmental impacts, especially if additional crops are harvested to produce biofuels. The main rationale behind the increased use of biofuels is the reduction of total greenhouse gas (GHG) emissions, typically accounted in carbon footprints [5]. For a more holistic view of environmental impacts, life cycle assessment (LCA) [6] is applied, which is a standardized method to assess a comprehensive set of environmental impacts including, among others, GHG emissions, land and water use, as well as toxic emissions. LCA consist of four stages: (1) goal and scope definition, (2) inventory analysis, (3) impact assessment, and (4) interpretation. In the goal and scope stage, the goal of the study (e.g., in this paper comparing transport service) as well as the system boundaries and assessed impacts are defined. The inventory analysis collects all processes (technosphere) as well as all emissions and resource uses (environmental exchanges) that the system includes (e.g., car production and disposal, fuel production, as well as all associated emissions and resource uses related to it). It principally encompasses the whole life cycle, but in some cases, it can be restricted to, e.g., production for comparison purposes. The overall emissions and resource consumption are then characterized in terms of environmental damage in the impact assessment phase. Typically, a system contains several hundred emissions that are characterized within ~10–15 impact categories such as “global warming,” “ecotoxicity,” or “resource depletion.” The final stage of interpretation involves an iterative process, since the analysis should be adjusted, e.g., if there is potential for improved data quality of important processes.

Swiss law [4] requires an LCA to show the overall environmental impact of biofuels compared to fossil options and requires specific assessments of GHG emissions and water consumption, since these are important impacts related to bioenergy production. The RED imposes measures, such as the ban of cultivation on ecologically sensitive lands, and requirements, such as a 35 % reduction in GHG emissions, the latter of which is often not met, if indirect land use effects are taken into consideration [7]. However, the calculation of indirect effects due to such consequential land use changes is scientifically debated, so it is not required in international standards [8].

A comprehensive study calculating and analyzing the LCA results of a variety of biofuels found that while the carbon footprint can be reduced by up to 50 % for biodiesel and 65 % for bioethanol, the total environmental impact might actually be even higher in some cases, rendering it impossible to characterize biofuels as favorable in general [9].

Intrinsically, LCA results have large uncertainties, and these must be understood in order to allow for sound decision-making [10]. For carbon footprinting, one recognized source of high uncertainties lies in N2O emissions in crop production [11, 12] and in highly variable methane emissions from the storage of digested residues (as a function of plant design) [13]. Unlike in water footprint studies, the assessment of water consumption, which shows that biofuels have significantly higher impacts on water resources than their fossil counterparts [14, 15], is not included in standard LCA methods. Still, the uncertainty of the results for water consumption in crop cultivation and the variability within countries are very high [15].

LCA results that neglect uncertainties have the potential to be highly misleading, since decision-makers are often not qualified to judge when results, especially comparative one, are significantly different from one another. For instance, they might consider a difference of, e.g., 10 % significant, even though such differences in LCA results do not typically indicate a significant difference in the environmental impact of products or services. In most cases, uncertainties are not reported at all and if they are, the focus is on inventory uncertainty. However, for the comparison of fossil fuels and biofuels, it is not only the uncertainty of the different emissions or land and water use that is important but also the uncertainty of the characterization factors (CFs) in impact assessment [16]. CFs quantify impacts caused by an emission or resource use (e.g., kg CO2-equivalent per kg N2O emission) and obviously are uncertain. CF uncertainty is typically neglected in carbon footprint or LCA studies so this work serves to fill this gap, providing a full uncertainty assessment for the impact categories water consumption (water scarcity footprint) and global warming potential (carbon footprint).

The goals of this work are to (1) analyze the environmental impact of different biofuel options in different impact categories within the framework of LCA using secondary data, (2) analyze the relevance of inventory and characterization factor uncertainty in the results of carbon and water footprinting, and (3) discuss the robustness of life cycle-based comparisons of fuels.

Methods

A complete list of abbreviations is available in the supporting information (Additional file 1).

Life cycle inventory

In order to account for all environmental flows (emissions and resource consumption) over the entire life cycle of a product, we employ the largest transparent process-based life cycle database ecoinvent v2.2 [17], which includes >4000 processes and >1000 environmental flows and reports uncertainty of process use and environmental flows in an LCA system. The inventory analysis is done on a unit process level, which means we know how much each of the >4000 processes in the database is used for the production of the demanded output (functional unit), and we can not only compare the total emission of each product system but also the emissions of the difference in the activity of each process within the compared product systems. This is very important for the uncertainty assessment in order to account for covariance as described below.

We analyze the inventories for the following ecoinvent v2.2 processes in order to compare the life cycle impacts of bioenergy used as a fuel in a passenger car per vehicle-kilometer (vkm) transport service. The abbreviations of the service used in this paper and corresponding ecoinvent processes are presented in Table 1.

Table 1 Abbreviations and processes used to model different passenger transport services

For water consumption in crop production, we use the data by Pfister et al. [18] and apply uncertainty estimates based on comparison of expected and deficit irrigation in each region. Other water flows are disregarded as they are considered irrelevant for biofuels [15].

Typically, LCA results are calculated based on the “technosphere” matrix A, which lists all the process inputs i (reported in rows) required for the output j (reported in columns) of a process and the “biosphere” matrix B, which lists the environmental flows k (resource inputs and emissions; rows) caused by the output of each process j (columns). The demand vector f identifies the process output of the system (functional unit) and by using the Leontief inverse of A, we get the overall process activity vector x, which shows the total life cycle process activity for the provision of the functional unit:

$$ \mathbf{x}={\left(\mathbf{I}-\mathbf{A}\right)}^{-1}\mathbf{f} $$
(1)

By multiplying B with x, we get the overall emissions, and as such, the inventory result y listing all environmental flows:

$$ \mathbf{y}=\mathbf{B}\ \mathbf{x} $$
(2)

In order to know which emissions or resource consumption occurred in which process, we calculate the entrywise product (Hadamard product) of the overall activity (x) and B as

$$ \mathbf{Y}=\mathbf{B}\kern0.5em {}^{\circ}\kern0.5em \left(\mathbf{t}\ {\mathbf{x}}^{\mathrm{T}}\right) $$
(3)

where Y is the resulting matrix with all emissions k (row) per process j (column) and t is a vector with k rows and all elements being equal to one. Inventory is consequently compared on two levels: the overall process activity (x) and overall emissions and resource consumptions (y).

Impact assessment

Impact assessment quantifies the relevance of resource consumption or emissions in terms of environmental damage within each impact category. Different characterization methods exist, among which the global warming potential (GWP) reported by Intergovernmental Panel on Climate Change (IPCC) is the most prominent (for the impact category radiative forcing). We use the IPCC 2007 factors [19] for the 20-, 100-, and 500-year global warming potential (GWP20, GWP100, GWP500, respectively). For a full LCA, a comprehensive set of impact categories need to be considered such as the set available by the ReCiPe method [20]. We used the fully aggregated impacts on endpoint level of ReCiPe version H/A for overall environmental assessment (grouped by impacts on human health, ecosystem quality, and resource depletion). Since water consumption characterization factors are not yet integrated with other methods, we use the water stress index (WSI, [21]) to quantify impacts of water consumption, which is also referred to as water scarcity footprint (WSFP). The impacts are calculated in vector z as follows:

$$ \mathbf{z}=\mathbf{C}\ \mathbf{y} $$
(4)

where C has the dimension of l impact categories (rows) and k environmental flows (columns).

In order to assess the impact per flow and process, we can multiply a single method l from C (c l ; a row vector, e.g., for GWP100) entrywise to Y:

$$ \mathbf{Z}=\mathbf{Y}\kern0.5em {}^{\circ}\kern0.5em \left({{\mathbf{c}}_{\mathbf{l}}}^{\mathbf{T}}\mathbf{u}\right) $$
(5)

where Z is the resulting matrix with all impacts of emissions k (row) per process j (column) and u is a row vector with j columns and all elements being equal to one.

Uncertainty analysis for the final LCA results

Uncertainty can generally be assessed in two different ways: analytically (e.g., by Taylor series expansion) and numerically (e.g., by a Monte Carlo simulation). We propagated uncertainties numerically through the equations in Sections Life cycle inventory and Impact assessment as it offers the advantage of not being restricted by validity assumptions (such as small uncertainties) and of providing more information on the output distribution [22, 23]. The numerical approach is, however, much more computationally intensive. To lessen this computational burden, we replaced the simple random sampling commonly performed in Monte Carlo simulations (primitive Monte Carlo simulations (PMC)) by Latin hypercube sampling (LMC) where results converge faster and require less iterations to archive robust results [22, 24]. Latin hypercube sampling is a stratified sampling procedure where the probability distribution is subdivided into as many equally probable intervals as simulations will be run and one sample is drawn randomly from each such interval. The sequence of these samples is again randomized [24]. Typically, 10,000 iterations are used for PMC [23], whereas Steen [25] recommends as little as 1000 iterations for PMC. Based on the estimation that LMC are five times more efficient than PMC [26], we used 1000 iterations. The uncertainty of the final LCA results was expressed by k values (approximately squared geometric standard deviation, which can be estimated by the square root of the ratio: 97.5th percentile/2.5th percentile [27]). For the comparisons between two products, the likelihood of one product having higher or lower impact than its alternative was additionally provided for each product and impact category, by pairwise comparisons of the results from the 1000 runs.

When two product systems are compared, they share some processes and therefore the uncertainties are not independent from each other (co-variant). The overall process activity might have a large share of common processes, such as in our case the infrastructure and car, which are identical for fossil and biofuels and therefore co-variant. If we assume, e.g., high emissions for power consumption in biogas car production, we should assume that the same power consumption in natural gas car production would also have high emissions. In order to avoid any potential dependencies when comparing two product options, we analyze uncertainty not only based on the final results but also by analyzing the difference between the two options for process activity (Δx) and inventory results (Δy).

To investigate the influence of uncertainty in different LCA stages (parameters in A, B, and C) on the uncertainty in impacts (z), we performed partial uncertainty analyses. This means that some parts were modeled stochastically (considering uncertainty) and other parts were modeled deterministically (neglecting uncertainty). Case A for instance includes uncertainty of A but not of B and C. The uncertainties of the final results were compared based on their k values.

Uncertainty information for characterization factors were derived from the newest IPCC report [28] for GHG emissions and from Pfister and Hellweg [29] for WSI. WSI uncertainty varies greatly per watershed as presented in Fig. 1. The uncertainties of GHG emissions vary by substance and time horizon as provided by the IPCC for the six most important GHGs and are presented as coefficient of variations (CV) in Table 2. For all other GHGs, we assumed a CV of 50 % based on uncertainty estimates of the two last IPCC reports [28, 29].

Fig. 1
figure 1

Dispersion factor (k value) of WSI on watershed level. Based on [40]

Table 2 Coefficient of variation (CV) of the 6 major GHG based on IPCC [28]

Contribution to variance (CTV) of input parameters

For the reduction of overall uncertainty, it is helpful to identify the most influential parameters and set priorities for improved data collection accordingly [10]. This sensitivity testing can be done via contribution to variance (CTV) analysis as previously done in LCA studies [30, 31]. Due to the non-normality of the impact scores, CTV is derived from Spearman’s rank correlation coefficients (R p , [32]) between sampled input parameters (p) and resulting impact scores:

$$ {\mathrm{CTV}}_p=\frac{R_p^2}{{\displaystyle {\sum}_{p=1}^n}{R}_p^2} $$
(6)

We compared the contributions to variance among all input parameters but also those within one single LCA stage as summarized in Table 3. In the latter case, the denominator will be lower as fewer parameters will be considered.

Table 3 Description of the different CTV analysis settings

All statistical analyses were performed with the statistical programming language R and specifically the packages “foreach” [33] and “doSNOW” [34] for parallelization of the Monte Carlo simulation and CTV analysis. CTV analysis is very intensive in terms of computational power for large systems as the one present here. Mutel et al. [31] suggest using 100 Monte Carlo iterations per parameter, which would result in more than 106 runs.

Results and discussion

LCA results

The standard LCA results show that different impact assessment methods result in different trends (Table 4). With complete LCA assessments using the ReCiPe (H/A) method, the overall impacts (R_TT) for diesel, biodiesel, petrol, ethanol, natural gas, and biogas from grass are very close and roughly a factor 2 higher than for the other natural gas options assessed based on ecoinvent data. The pattern is similar for impacts on ecosystem quality (R-EQ, except for bg_grass due to high land use) and resource depletion (R-RS), with higher differences for resources. Biogas and biodiesel do not perform better than fossil alternatives due to the fact that crops are specifically grown for biofuels in the selected datasets and therefore land use, machinery, and agrochemicals largely compensate the reduced GHG emissions. For impacts on human health (R-HH), all options have similar results, since there is less difference in the various production systems. For global warming potential, the time horizon is very relevant, since biogas options have a higher carbon footprint under a 20-year time horizon (GWP20) and perform better for longer time horizons (GWP100, GWP500). This is explained through the fact that biogas options in our data have higher methane emissions, which have a short atmospheric lifetime and therefore a high impact in the short term but a much lower in the long term compared to fossil CO2 emissions (the main problem of fossil fuels). For the water footprint assessment, biodiesel and bioethanol score by far the worst, since they have cultivated crops as feedstock, which include irrigation, while biogas production uses waste and co-products for its production, except for grass, which is generally not irrigated. Since LCA includes a variety of different indicators, without a scientific proper way for aggregating them, we also calculate the average rank of all considered results. Although not a common procedure in LCA, it is an alternative to fully aggregating methods (R-TT). The results are quite consistent between average rank and R-TT, indicating that biogas based on waste is the best option of the cases analyzed in this paper.

Table 4 LCA impact scores by deterministic modeling per vehicle-kilometer (vkm)

Uncertainties in carbon and water footprinting

Table 4 reports the LCA results without uncertainties and highlights the influence on the results of the impact assessment method choice. The results are very similar for different options for many indicators. In Table 5, we present the results including uncertainties of the stochastic modeling. The expected value is described by the geometric mean (GM) and is consistent with the deterministic value for carbon footprints but less so for water footprints due to the higher uncertainty: the upper and lower bound of the 95 % confidence interval for WSFP differ by a factor of 27–64. For carbon footprints, the results are less uncertain but still the 95 % confidence interval spans a factor ~2 for liquid fuels and a factor of ~3–4 for the biogas options. Typically, liquid fuels have lower uncertainties in the short-term GWP, since CF uncertainty is lower. Biogas has lower uncertainty in the long term since the importance of the uncertainty of methane emissions diminishes in the GWP results.

Table 5 Carbon and water footprint results by stochastic modeling (geometric mean (GM)) and k values

Analyzing the uncertainties for the different steps in calculating GWP100, we see that for the liquid fuels, characterization uncertainty had higher contribution to uncertainty than the impact assessment part because CO2 emissions have lower uncertainty than CH4 emissions, the latter being more relevant for comparing gas fuels (Table 6). Also, we see that the uncertainty is mainly caused by the methane emissions per process and not by the process activity uncertainty (higher k value). This is mainly explained by the high variability of uncontrolled methane emissions. While typically 2 % uncontrolled loss is assumed, the emission can be up to a factor 10 higher or lower [35], which makes a huge difference in the GWP performance of biogas as a function of the plant design and management.

Table 6 GWP100 impacts by stochastic modeling (LMC with 1000 runs) for individual stages within LCA

For the water footprint (Table 7), we mainly focus on biodiesel and ethanol as the other fuels had no significant water consumption. The inventory and impact assessment cause a similar, high uncertainty (k ~ 4), and in the inventory, this is mainly caused by the consumption factors and not the uncertainty of the process activity. Water consumption estimates have high uncertainty, since each farm has individual irrigation activities and water consumption of crops can only be derived from models for average products. For the other fuels, process activity is much more relevant, since water consumption occurs deeper in the supply chain and therefore accumulates uncertainties within the technosphere matrix (A).

Table 7 WSFP impacts by stochastic modeling (LMC with 1000 runs) for individual stages within LCA

Comparing biofuels to fossil fuels while accounting for uncertainties

To identify the robustness of the difference when comparing two fuel options, we can analyze the LCA results presented above and assess their relative difference. However from a stochastic point of view, we want to know the probability of one option being better than another for a specific impact category. Table 8 shows that the biogas mix has a higher relative difference for water consumption than for GWP100, while the chance that biogas is really better is much higher for GWP100 due to the lower uncertainty in inventory and impact assessment. This shows that the robustness of the LCA or footprint result cannot be estimated based on the relative difference of the results without accounting for uncertainty too: a high difference with high uncertainty might reflect the same level of decision uncertainty as a low difference with low uncertainty. This notice is a crucial point for decision-makers, and therefore, it might be better to report chances of one option being better than another for comparative LCA and footprint assessments, since results including uncertainties might be too complex for reporting.

Table 8 Impact comparison of bioenergy and its conventional alternative with regard to GWP and WSFP

Interestingly, the probability that biogas options have a lower carbon footprint compared to natural gas is very high for the 500-year time horizon and very low for the 20-year time horizon. In this case, the model choice is much more important than the other uncertainties. When using ecoinvent data, preferring biogas over natural gas is mainly a question of having a short-term or long-term perspective and not so much a question of higher or lower uncertainties for one option compared to the other. It can be argued that improving inventory data is not the most crucial task when comparing gas to liquid fuels, and a rationale for the chosen time horizon is more relevant, since no proper scientific argumentation is available.

Additional file 1: Table S2 additionally shows the probabilities if only part of the uncertainty is analyzed. For the liquid fuels, decision uncertainty is mainly caused by emission factors for GWP, and essentially no uncertainty exists for WSFP. For gas alternatives, the decision uncertainty in the result is affected by all three uncertainties for the GWP20, while it is robust (>90 % chance) for the GWP500.

Since the GWP100 carbon footprint comparing two alternatives is not robust in most cases, additional analysis of the most important flows in relevant processes can help improve the inventory of the system and hence the overall result. To identify which processes need higher attention to improve the comparison, we present the most relevant rows and columns of the result matrix Z in Table 9, showing the difference in impacts of the respective flow and process for a vehicle-kilometer using natural gas compared to biogas for GWP100 and WSFP. For a natural gas powered car, the fossil CO2-emissions from combustion in the car are the most important, while for a biogas car, the biogenic CH4 emissions from biogas at storage and at purification are the most crucial. Water consumption is less relevant and the main contributions for WSFP are found in soybean (in Switzerland) and husked coconut (Philippines) production. However, since these values are very low and represent water consumption in oil production for soaps in supply chain processes, other water flows such as process water, which are not accounted for, might be more important in this biogas case. On the other hand, process water can be neglected compared to irrigation water demands and if no irrigation is applied, water footprints are typically low [36].

Table 9 Flows and processes contributing most to the impact comparison of natgas and biogas (per vkm)

Contribution to variance (CTV)

A more structured way to identify important flows and processes causing uncertainty is the CTV analysis. Due to the limited numbers of Monte Carlo runs, we do not get a highly robust result for CTV of GWP100 when analyzed over all input parameters for the comparison of a biogas and natural gas car. A selection of the CTV results is presented in Table 10. For CTV_all of GWP100, the highest value is 0.38 % with the CTV of several hundred parameters being >0.1 % (comprehensive list in Additional file 1: Table S3 in the supporting information). Based on Mutel et al. [31], the CTV of uncorrelated parameters would still be ~0.10 % for 1000 runs, which means the results have very low discriminating power and therefore, we cannot select the most important parameters from the total of >3000 contributing parameters. For water footprint, only 51 parameters affect the impact and therefore quite robust results are attained (Additional file 1: Table S4 in the supporting information). The uncertainty is mainly driven by the process activities for crop production (x is responsible for >97 % CTV). As for the contribution to impacts (Table 9), the most relevant parameters for uncertainty is palm oil and coconut production uncertainty with >25 % CTV each.

Table 10 Strongest parameter contributions to variance of WSFP and GWP100 results

Since CTV for a full LCA system is not practical due to the very high calculation demands, we suggest another procedure for application by practitioners. For carbon footprints, we split the CTV analysis into subsections to get more robust results. However, for the uncertainties of the environmental flow factors (CTV_B) and for process uncertainty (CTV_x), we get only slightly higher values of up to 0.57 (for N2O, CH4, CO2 flow factors) and 0.40 (for biogas processes), respectively (Additional file 1: Table S3). For the uncertainty of the inventory results (CTV_y) and of the characterization factors (CTV_C), we get CTV values of 19 % for fossil CO2 emissions in the inventory results and 14 % for the minor GHG “Halon 1001” in the CFs, respectively (Table 11 and Additional file 1: Table S3). The discrepancy of CTV in y and C might suggest that the results are not robust. However, the relative uncertainties of GWP100 for the six major GHGs are a factor 2 lower than for the remaining GHGs (see Table 2). While further research is needed to improve CTV analyses on such complex systems, we suggest a structured analysis of CTV in separate steps in the assessment for practical applications, as presented here. In combination with uncertainty assessment of the respective calculation steps (Tables 6 and 7), we can identify some hotspots for uncertainty. The resulting flows and processes should be revisited in order to improve the results.

Table 11 Strongest CTV of inventory (CTV_y) and CF (CTV_C) parameters to GWP100 results

Conclusions

This work shows that consideration of uncertainties is essential for informed decision-making and can help to improve the results. For biofuel and carbon footprints, we suggest that, for purposes of good practice, footprint studies report results for all three time horizons including uncertainties, as presented here. This also involves assessment of offsetting activities, which is clearly illustrated by the example of methane flaring, one of the cheapest options for offsetting GHG emissions, that has more than a factor 3 lower CO2-equivalents when analyzed by GWP500 compared to the normally used GWP100 (or, equivalently, almost a factor 3 higher using the GWP20). Due to the significant effects of such choices, it is very difficult to select best fuel options, as illustrated by the comparison of fossil and biofuels in this work based on secondary data.

Often, LCA or carbon and water footprints just report a number without the detailed background information. However, if product-labeling and CO2 compensation are based on just a number without uncertainty ranges, the results might be misinterpreted by non-experts. As an example, rapeseed biofuel production assessed with uncertainty analysis showed that life cycle GHG savings compared to diesel are not significant, even though results neglecting uncertainties would suggest they are [37]. The European Union is encouraging the establishment of product environmental footprints (PEF, [38]) in order to make the environmental impacts related to a product visible. However, as shown for the case of biofuel, the 95 % confidence interval of these numbers might vary by a factor 2–4 for carbon footprints and even by 1–2 orders of magnitude for water footprints. Stating numbers without uncertainties is even more questionable if we consider the importance of the choice of impact assessment methods on the results. The highest uncertainty is caused by the time horizon choice for carbon footprints, which determines the choice for or against biogas compared to other fuels, yet such time horizons are inconsistent with other LCA impact categories. As a minimum, background information must be available to decision makers and the public in order to allow an open and transparent discussion of footprint results by the stakeholders [39]. Furthermore, sensitivity to methodological choices should be tested and reported. As an argument against the reporting of uncertainty information, communication issues are often cited. While best way of communication is definitely open for debate, the results presented in Table 8 indicate that reporting a probability that one option is better than another could be a viable way forward for best practice reporting of single numbers. The remaining issue, however, is that this only works for comparison studies and it requires access to the full inventory including uncertainties of the compared product systems, which can be a problem if one compares the results for a product with those in an already published study.

The second equally important advantage of proper uncertainty assessment is the identification of inventory flows that require additional attention to improve the quality of the results. While contribution to variance analysis could help with this problem, indicating which parameters are the most crucial for overall uncertainty, it is not a practical for practitioners due to the extreme computational power it demands to find robust results. Therefore, this goal of the study has not been fully met. Instead, our suggestion is to split the different calculation steps in the LCA structure and after identifying the most important steps, to perform CTV analysis within each calculation step, giving more robust results with a given computational power and time.

Finally, high uncertainty in assessing environmental performance also opens the room for more comprehensive sustainability assessment, including social aspects: LCA and footprint results should not be taken at face value, but they should be interpreted as indicators that help to identify hotspots for further assessment and to better understand the studied product system and related drivers for environmental issues. Additional research on how to better include uncertainty evaluation in LCA and footprints for practical applications is required to face the many challenges for robust assessment of different fuel options.

References

  1. Energy independence and security act of 2007, P.L. 110–140, available online http://www.gpo.gov/fdsys/pkg/PLAW-110publ140/pdf/PLAW-110publ140.pdf Accessed March 27, 2015.

  2. Directive 2009/28/EC. Directive 2009/28/ec of the european parliament and of the council of 23 April 2009 on the promotion of the use of energy from renewable sources. Available online http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=uriserv:l27035 February 06, 2015.

  3. European Commission (EC) (2012) New commission proposal to minimise the climate impacts of biofuel production. European Commission, Brussels, Retrieved from http://ec.europa.eu/clima/policies/transport/fuel/docs/com_2012_595_en.pdf Accessed March 27, 2015.

  4. Swiss Federal Law on fuel taxes 641.61 Mineralölsteuergesetz (MinöStG); available online http://www.admin.ch/opc/de/classified-compilation/19960320/index.html Accessed March 27, 2015.

  5. ISO 14064. Iso 14064–1 greenhouse gases—part 1—specification for the quantification, monitoring and reporting of entity emissions and removals; iso 14064–2 greenhouse gases—part 2—specification for the quantification, monitoring and reporting of project emissions and removals; iso 14064–3 greenhouse gases—part 3—specification and guidance for validation, verification and certification. International Organization for Standardization, Ed. 2006.

  6. ISO 14044 (2006) Environmental management—life cycle assessment—requirements and guidelines (iso 14044:2006). International Organisation for Standardisation, Geneva

    Google Scholar 

  7. Sanchez ST, Woods J, Akhurst M, Brander M, O’Hare M, Dawson TP, Edwards R, Liska AJ, Malpas R (2012) Accounting for indirect land-use change in the life cycle assessment of biofuel supply chains. J R Soc Interface 9:1105–1119

    Article  Google Scholar 

  8. Finkbeiner M (2014) Indirect land use change—help beyond the hype? Biomass and Bioenergy 62:218–221

    Article  Google Scholar 

  9. Zah R, Böni H, Gauch M, Hischier R, Lehmann M, Wäger P (2007) Ökobilanz von energieprodukten: Ökologische bewertung von biotreibstoffen. Empa, Technology and Society Lab, St. Gallen

    Google Scholar 

  10. Lloyd SM, Ries R (2007) Characterizing, propagating, and analyzing uncertainty in life-cycle assessment: a survey of quantitative approaches. J Ind Ecol 11(1):161–179. doi:10.1162/jiec.2007.1136

    Article  Google Scholar 

  11. Bachmaier H, Effenberger M, Gronauer A, Boxberger J (2014) Changes in greenhouse gas balance and resource demand of biogas plants in southern Germany after a period of three years. Waste Manag Res 31(4):368–375

    Article  Google Scholar 

  12. Eric AD, David K (2014) Inventories and scenarios of nitrous oxide emissions. Environ Res Lett 9(10):105012

    Article  Google Scholar 

  13. Bachmaier H (2012) Treibhausgasemissionen und fossiler Energieverbrauch landwirtschaftlicher Biogasanlagen – Eine Bewertung auf Basis von Messdaten mit Evaluierung der Ergebnisunsicherheit mittels Monte-Carlo-Simulation. Universität für Bodenkultur, Wien: Dissertation

  14. Gerbens-Leenes W, Hoekstra AY, Van der Meer TH (2009) The water footprint of bioenergy. Proc Natl Acad Sci U S A 106:10219–10223

    Article  Google Scholar 

  15. Pfister S, Bayer P, Koehler A, Hellweg S (2011) Environmental impacts of water use in global crop production: hotspots and trade-offs with land use. Environ Sci Technol 45(13):5761–5768

  16. Huijbregts MAJ, Thissen U, Jager T, van de Meent D, Ragas AMJ (2000) Priority assessment of toxic substances in life cycle assessment. Part II: assessing parameter uncertainty and human variability in the calculation of toxicity potentials. Chemosphere 41(4):575–588

  17. Ecoinvent Centre. (2010). ecoinvent data v2.2. Available at http://www.ecoinvent.org accessed November 21, 2014.

  18. Pfister S, Bayer P (2014) Monthly water stress: spatially and temporally explicit consumptive water footprint of global crop production. J Clean Prod 73:52–62

    Article  Google Scholar 

  19. IPCC (2007). IPCC Fourth assessment report. The physical science basis. Available at: http://www.ipcc.ch/ipccreports/ar4-wg1.htm Accessed November 21, 2014.

  20. Goedkoop M, Heijungs R, Huijbregts M, De Schryver A, Struijs J, van Zelm R (2009) ReCiPe 2008—a life cycle impact assessment method which comprises harmonised category indicators at the midpoint and the endpoint level. Available at: http://www.lcia-recipe.net/ Accessed November 21, 2014.

  21. Pfister S, Koehler A, Hellweg S (2009) Assessing the environmental impacts of freshwater consumption in LCA. Environ Sci Technol 43(11):4098–4104

    Article  Google Scholar 

  22. Groen EA, Heijungs R, Bokkers EAM, de Boer IJM (2014) Methods for uncertainty propagation in life cycle assessment. Environ Model Software 62:316–325. doi:10.1016/j.envsoft.2014.10.006

  23. Heijungs R, Lenzen M (2014) Error propagation methods for LCA—a comparison. Int J Life Cycle Ass 19(7):1445–1461. doi:10.1007/s11367-014-0751-0

    Article  Google Scholar 

  24. Helton JC, Johnson JD, Sallaberry CJ, Storlie CB (2006) Survey of sampling-based methods for uncertainty and sensitivity analysis. The fourth international conference on sensitivity analysis of model output (SAMO 2004) SAMO 2004 91(10–11): 1175–1209. doi:10.1016/j.ress.2005.11.017

  25. Steen B (1997) On uncertainty and sensitivity of LCA-based priority setting. J Clean Prod 5(4):255–262. doi:10.1016/S0959-6526(97)00039-5

    Article  Google Scholar 

  26. Swidzinski JF, Chang K (eds) (2000) A novel nonlinear statistical modeling technique for microwave devices. Microwave Symposium Digest. 2000 IEEE MTT-S International, vol 2 Boston, Massachusetts

  27. Slob W (1994) Uncertainty analysis in multiplicative models. Risk Anal 14(4):571–576. doi:10.1111/j.1539-6924.1994.tb00271.x

    Article  Google Scholar 

  28. Myhre G, Shindell D, Bréon F-M, Collins W, Fuglestvedt J, Huang J, Koch D, Lamarque J-F, Lee D, Mendoza B, Nakajima T, Robock A, Stephens G, Takemura T, Zhang H (2013) Anthropogenic and natural radiative forcing: supplementary material. In: Stocker TF, Qin D, Plattner G-K, Tignor M, Allen SK, Boschung J, Nauels A, Xia Y, Bex V, Midgley PM (eds) Climate change 2013: the physical science basis. Contribution of working group I to the fifth assessment report of the intergovernmental panel on climate change. Available from www.climatechange2013.org and www.ipcc.ch Accessed November 21, 2014.

  29. Solomon S, Qin D, Manning M, Chen Z, Marquis M, Averyt KB, Tignor M, Miller HL (eds) (2007) Climate change 2007: the physical science basis. Contribution of working group I to the fourth assessment. Report of the intergovernmental panel on climate change. Cambridge University Press, Cambridge, United Kingdom and New York, NY, USA, 996 pp

    Google Scholar 

  30. Geisler G, Hellweg S, Hungerbühler K (2005) Uncertainty analysis in life cycle assessment (LCA): case study on plant-protection products and implications for decision making (9 pp + 3 pp). Int J Life Cycle Ass 10(3):184–192. doi:10.1065/lca2004.09.178

    Article  Google Scholar 

  31. Mutel CL, de Baan L, Hellweg S (2013) Two-step sensitivity testing of parametrized and regionalized life cycle assessments: methodology and case study. Environ Sci Technol 47(11):5660–5667. doi:10.1021/es3050949

    Article  Google Scholar 

  32. Spearman C (1910) Correlation calculated from faulty data. Brit J Psychol 3(3):271–295. doi:10.1111/j.2044-8295.1910.tb00206.x

    Google Scholar 

  33. Revolution analytics, Weston S (2014) foreach: Foreach looping construct for R. R package version 1.4.2. http://CRAN.R-project.org/package=foreach Accessed February 27, 2015.

  34. Revolution analytics, Weston S (2014) doSNOW: Foreach parallel adaptor for the snow package. R package version 1.0.12. http://CRAN.R-project.org/package=doSNOW. Accessed February 27, 2015.

  35. Börjesson P, Berglund M (2006) Environmental systems analysis of biogas systems—part I: fuel-cycle emissions. Biomass and Bioenergy 30(5):469–485. doi:10.1016/j.biombioe.2005.11.014

    Article  Google Scholar 

  36. Berger M, Pfister S, Bach V, Finkbeiner M (2015) Saving the planet’s climate or water resources? The trade-off between carbon and water footprints of European biofuels. Sustainability 7(6):6665–6683

    Article  Google Scholar 

  37. Malça J, Freire F (2010) Uncertainty analysis in biofuel systems. J Indus Ecol 14(2):322–334. doi:10.1111/j.1530-9290.2010.00227.x

    Article  Google Scholar 

  38. European Comission (2015). The development of PEF. http://ec.europa.eu/environment/eussd/smgp/dev_pef.htm accessed March 27, 2015

  39. Ridoutt B, Fantke P, Pfister S, Bare J, Boulay A-M, Cherubini F, Frischknecht R, Hauschild M, Hellweg S, Henderson A, Jolliet O, Levasseur A, Margni M, McKone T, Michelsen O, Milà i Canals L, Page G, Pant R, Raugei M, Sala S, Saouter E, Verones F, Wiedmann T (2015) Making sense of the minefield of footprint indicators. Environ Sci Technol 49:2601–2603

  40. Pfister S, Hellweg S (2011) Surface water use—human health impacts. Report of the LC-IMPACT project (EC: FP7). Available online at: http://www.ifu.ethz.ch/ESD/downloads/Uncertainty_water_LCIA.pdf Accessed November 21, 2014.

Download references

Acknowledgements

We acknowledge Dominik Saner for the matlab data of ecoinvent v 2.2 and Catherine Raptis for proof-reading.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Stephan Pfister.

Additional information

Competing interests

The authors declare that they have no competing interests.

Authors’ contributions

SP designed the study, compiled the information about uncertainty in impact assessment methods, and structured the manuscript. LS developed the scripts and calculated the uncertainty and contribution to variance results. Both authors interpreted and discussed the results, drafted the paper, and approved the final manuscript.

Additional file

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pfister, S., Scherer, L. Uncertainty analysis of the environmental sustainability of biofuels. Energ Sustain Soc 5, 30 (2015). https://doi.org/10.1186/s13705-015-0058-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13705-015-0058-4

Keywords