Linking qualitative scenarios with quantitative energy models: knowledge integration in different methodological designs

Abstract

Background

Linking qualitative scenarios with quantitative models is a common approach to integrate assumptions on possible future societal contexts into modeling. But reflection on how and to what degree knowledge is effectively integrated during this endeavor does not generally take place. In this paper, we reflect on the performance of a specific hybrid scenario approach (qualitative Cross-Impact Balance analysis, CIB, linked with quantitative energy models) concerning knowledge integration through 11 different process steps. In order to guide the scenario community in applying this approach, we reflect on general methodological features as well as different design options. We conceptualize different forms of interdisciplinary knowledge integration (compiling, combining and synthesizing) and analyze how and to what degree knowledge about society and uncertainty are integrated into scenario process and products. In addition, we discuss trade-offs regarding design choices and forms of knowledge integration.

Results

On the basis of three case studies, we identify two general designs of linking which build on each other (basic and extended design) and which differ in essence regarding the balance of power between the CIB and the energy modeling. Ex post assessment of the form of interdisciplinary knowledge integration in each step revealed that specific method properties of CIB as well as the interaction with additional quantitative as well as specific qualitative methods foster distinct forms of knowledge integration. The specific roles assigned to CIB in the hybrid scenario process can also influence the form of knowledge integration.

Conclusions

In this study, we use a joint process scheme linking qualitative context scenarios with energy modeling. By applying our conceptualization of different forms of knowledge integration we analyze the designs’ respective potential for and respective effects on knowledge integration. Consequently, our findings can give guidance to those who are designing their own hybrid scenario processes. As this is an explorative study, it would be useful to further test our hypotheses in different hybrid scenario designs. Finally, we note that at some points in the process a more precise differentiation of three forms of knowledge integration would have been useful and propose to further differentiate and detail them in future research.

Background

Hybrid scenario construction and its challenges

The awareness within the energy research community of the importance of also integrating more explicitly social, political and cultural dimensions in the analysis of energy systems of the future has increased over the years [1,2,3]. During energy scenario construction, energy models need data input about future societal developments and their impact on energy demand and supply. Such information relates for example to population, lifestyles, economy, innovation and other factors and must be defined on the basis of so-called framework assumptions [4]. Such assumptions implicitly draw on the modelers’ perceptions concerning the future developments of the society into which the modeled system is embedded [5]. These future assumptions can be integrated intuitively or more formally using various available sources. But not every combination of framework assumptions builds a meaningful picture of societal contexts, even if their sources are highly credible.

In response to this difficulty, hybrid approaches have been developed, e.g., by Gallopin et al. [6], Alcamo et al. [7] and Raskin et al. [8] and have become prominent in environmental research under the label of “Story And Simulation” (SAS) [7, 9]. Hybrid (also named combined or integrated) scenario approaches are methodologies combining qualitative and model-based scenario approaches (cf. also in the following [10, 11]. Their aim is to realize more complete system representations, which combine and/or integrate qualitative as well as quantitative information. Hybrid scenario approaches have been used in various fields such as water (e.g., [12]), climate change (e.g., [13], biodiversity [14]), sustainability and development [15] as well as energy (e.g., [16, 17]. In the traditional approach, the qualitative part of hybrid scenarios is developed using the Intuitive Logics method [18, 19]. Intuitive Logics develops scenarios based on reflection and discussion. While this approach has enabled impressive results in climate, environmental, and energy research (cf. above), it has also drawn concern from scholars about the imbalance between the rather simple storyline procedures in comparison to the sophistication of the models [20]. To counter this imbalance, scholars recommend the use of formalized storyline construction methods and point to Cross-Impact Balances (CIB, [21] as a promising alternative [22,23,24,25]. The advantages of CIB are seen in a more systematic, consistent, complete, transparent and objective scenario construction process. CIB conceptualizes systems as qualitative networks [21]. The main drivers of future developments (descriptors) are set as the nodes of the network and a small set of qualitatively and/or quantitatively defined alternative futures of the drivers are assigned to the nodes as discrete states. Qualitative information about the promoting and hindering influences between the nodes is collected by literature review or expert elicitation. The consistent configurations (consistent scenarios) of the network are obtained by calculating the Nash equilibria [26] of the cross-impact data.¹

CIB-based qualitative scenarios forming hybrid scenario construction are called ‘context scenarios’ [5]. Examples of recent applications of CIB context scenarios in hybrid scenario exercises are Ruth et al. [27], Vögele et al. [28], Brodecki et al. [29], Schütze et al. [30], Vögele et al. [31], and Pregger et al. [32]. The role of CIB context scenarios, their strengths and challenges are outlined by Weimer-Jehle et al. [33] who reflect on the approach in comparison to other current hybrid approaches in energy research, as, e.g., Ault et al. [34], Stocker et al. [35]/Spangenberg et al. [36], O’Mahony et al. [37], McDowell [38], Trutnevyte et al. [39]/Foxon [40] or Fortes et al. [41].

Hybrid scenarios share the fundamental challenge of needing to bring together the different worlds of separate disciplines from engineering, natural and social sciences as well as the humanities. These often have contrasting, or even conflicting, ontologies, epistemologies and methodologies (e.g., [10]. On a pragmatic level, regarding the concrete scenario construction processes, hybrid scenarios show as a minimal condition, a “need for shared understanding amongst researches in cross-disciplinary collaboration” [42]: 583).

Aim and outline

For several years, qualitative scenarios have functioned as boundary objects for knowledge integration in energy modeling and various hybrid approaches have been used and proposed. However, hybrid scenario processes still pose a challenge to successful interdisciplinary cooperation and, more precisely, to interdisciplinary knowledge integration. As we understand it, knowledge integration does not function by itself. Effective knowledge integration needs a “unifying framework for integrative research” [43]: 324). Regarding SAS, the ways in which scenario processes can be structured, are already well researched and documented (e.g., [9]. The context scenario approach, on the other hand, is relatively young and its methodological options yet not systematically reflected. We would like to provide scenario builders and energy system modelers with an overview of design options and therefore explore (i) the different forms in which the linking of context scenarios and energy models can be structured and designed and what roles CIB plays with regard to that linking (research question 1).

Although qualitative scenarios function as knowledge integration methods [20], and hybrid scenario construction is seen as an enabler of interdisciplinary knowledge integration in energy scenarios, we still lack knowledge of whether, how and to what degree knowledge integration in hybrid scenario approaches has actually been implemented. We want to fill these research gaps by (ii) presenting a framework for the analysis of interdisciplinary knowledge integration (see ‘Forms of knowledge integration’ in the next subsection) and by exemplarily (iii) applying this framework to three methodological designs linking qualitative CIB-based context scenarios with quantitative (energy) models. We furthermore consider the effects of (iv) the scenario method CIB itself and (v) the interplay of different methods on the form of interdisciplinary knowledge integration (research question 2). Our approach is explorative and thus generates hypotheses.

With our paper, we aim to report on what we learned from our first experiences from three projects applied in ENERGY-TRANS.² These methodological analyses and reflections target the energy scenario community as well as the entire field of environmental modeling concerned with using and advancing hybrid scenario approaches with regard to context scenarios and other hybrid approaches.

The next subsection provides a review of existing scientific work that addresses knowledge integration in interdisciplinary and transdisciplinary research. This is the basis for our conceptual framework on ‘forms of interdisciplinary knowledge integration’, which we developed for the analysis. Then, we present the three empirical cases. In the next section, we introduce our methods of data collection and analysis. In the following, we show and interpret our results concerning different designs and the resulting form of interdisciplinary knowledge integration. Finally, we discuss our results and draw conclusions.

Literature review and derived forms of interdisciplinary knowledge integration

To link our analysis to prior work on interdisciplinary knowledge integration (e.g., [43,44,45], we consider the distinction often made between data, information and knowledge [46]. Data refers to symbols, such as letters and numbers; information refers to “data that are processed to be useful and provides answers to "who", "what", "where", and "when" questions; knowledge refers to the application of data and information and is necessary to answer "how" questions” [47].

The setting of the type of research we are presenting is interdisciplinary³ and requires “a synthesis of ideas, data and information, methods, tools, concepts, and/or theories from two or more disciplines aimed at answering a complex question, solving a complex problem, or producing new knowledge or a product of knowledge” [48]: 286). Although interdisciplinary research can also be realized by individuals [49, 50], we focus on the research of a team, which is considerably different regarding the operationalization of a project and also the challenges an interdisciplinary team has to handle [51]. The members of an interdisciplinary project team can belong to different scientific realms as well as to different schools of thought, different regions, etc. [50, 51]. We work with a pragmatic understanding of scientific knowledge defined as scientifically valid representation(s) of the world.

The integration of knowledge is seen by many scholars as being the key to interdisciplinary research (see also [44, 50, 52]. It can be defined as the blending of concepts and perspectives “to create innovative new worldviews” [53]: 299). Hinkel [44]:19 distinguishes two phases of knowledge integration: “the elaboration of a shared language” (see also [42], also defined as “common ground” by Edelenbos [54]: 454) and “the design of a methodology”. A methodology consists of a combination of instruments and methods: in our case, this is the specific empirical application of the qualitative scenario method Cross-Impact Balance analysis and its link to quantitative (energy) models, including individual additional methods and techniques, researchers and data.

To compare transdisciplinary research projects with regard to knowledge integration, different scholars have provided analytical frameworks (e.g., [55, 56]. These frameworks include typologies of actors, of actor involvement and knowledge, and the differentiation of synthesis stages. As these analytical frameworks are developed for transdisciplinary research, they do not exactly fit our purpose, but have inspired the development of our own analytical framework for a comparative analysis of interdisciplinary research projects linking CIB-based scenarios with quantitative (energy) models.

For our comparative analysis, we analytically distinguish between three different forms of how scientific knowledge is integrated in interdisciplinary research processes. We have conceptualized knowledge integration in a way that is tailored to hybrid scenario construction in interdisciplinary teams. To meet this aim, we propose to distinguish between compiling, combining and synthesizing knowledge. These three forms of integration are described in more detail in the following section and visualized in Fig. 1.

• Compiling knowledge refers to activities when separate bodies of knowledge (i.e., representations of the world, such as the energy system) are produced by distinct disciplines, which might use separate methods and concepts and, once the analysis is achieved, compile the products. This is the case if, for instance, a joint question or issue is dealt with separately by different mono-disciplinary methods or concepts, and the results of each disciplinary perspective are finally brought together. An example of this could be a report presenting the different perspectives on the issue. Such a compilation may contain for instance one chapter on the technical aspects of the energy system, one on economic aspects and another one on legal issues. Furthermore, compiling can refer to the adding up of knowledge about “issues of concern” [45]. This can be knowledge which needs to be newly produced in disciplinary ways as described above or that already exists, e.g., in publications, and which needs to be consolidated.

• Combining knowledge refers to activities that go beyond the accumulation of different disciplinary approaches and elements of knowledge, resulting in the production of jointly usable content. To achieve this aim, combining often requires the use of translation methods. Translation methods translate content (representations of the world) produced by one discipline, making them understandable and allowing them to be potentially further processed by another disciplinary approach. One example is the quantification of qualitative expert assessments by fuzzy logics (e.g., [9] or Bayesian Networks (e.g., [57]. Also, in order to allow the mutual understanding of approaches and contents of initially separate disciplines, conceptual bridges are used, such as joint meta-languages, i.e., “language[s] for speaking about knowledge to be integrated and a specification of the integration process” [44]: 15).

• Synthesizing knowledge refers to interdisciplinary activities in which researches are using joint, or (rather) closely coupled, interdisciplinary concepts and/or methods to jointly develop “a new kind of knowledge” ([58]: 409). This new knowledge is characterized by novel connections distilled by the integration of results [59, 60] and it is “a synergy of contributing parts that are not visible any more” (Angyal 1939 in: [58]: 409). Interdisciplinary concepts and methods can (but need not) be the starting point for developing new disciplinary (sub-)fields.

Fig. 1

Forms of interdisciplinary knowledge integration

Please note that, first of all, this distinction of three forms of interdisciplinary knowledge integration is an analytical one, and that they do not automatically exclude each other. Empirically, gradual transitions between forms might be found. And secondly, we do not intend to imply that any normative judgment, or claim that any form of knowledge integration is ‘better’ than any other. On the contrary, according to each specific research question and setting in a particular project different forms of integration might be appropriate. Nevertheless, the forms indicate to a certain degree whether integration was ‘on the surface’ (compiling), ‘deeper’ (combining) or ‘very deep’ (synthesizing).

Methods

Case study approach

Our research position is based on the concept of the ‘reflective practitioner’ (in the sense of [61]. We aim to go beyond textbook descriptions of methods and designs by taking a step back in order to critically consider our own practice as scenario builders (cf. [62]). To gain the necessary distance, we chose to use the case study approach [63] to explore, analyze and finally compare different approaches linking CIB with energy models. The three projects linking qualitative context scenarios (based on CIB) with energy modeling, which are described in the previous chapter, are considered as our three case studies. Overall, due to the low number of case studies sampled, our case study approach encourages qualitative in-depth analysis and theoretical generalization, but no statistical generalization of results (cf. [63]. But “a knowledge base not only expands by generalising experience, but also by becoming more and more experienced in identifying the specifics of a new case and in gaining the ability to take features of the cases to be typical.” (Krohn 2008 in: [55]: 113). The approach used here is exploratory and generates hypotheses.

Overview of the three exemplary case studies

We applied our framework to three exemplary case studies on context scenarios. First of all, the sampling of the case studies was pragmatic: these were, to our knowledge, among the very first current case studies linking qualitative, CIB-generated, context scenarios with energy modeling. They started simultaneously within the same research alliance. At that time, only two other case studies existed, namely the pioneer application by Förster [64], as well as a study published by Wachsmuth [65] and Ruth et al. [27]. We did not select these case studies due to their limited (ex post) accessibility, especially in comparison to the insights on methodological considerations that were possible through our role as insiders in the three selected case studies. Secondly, the three selected case studies seemed to be good examples of applications of the methodology in different domains, on different scales, and for different purposes of energy modeling, with regard to different designs used to develop hybrid scenarios. The choice of the three specific designs within the three cases was not a systematic or supervised decision, but was taken individually in each team determined by the function of the scope and preferences of each project. However, comparing the three cases now enables researchers who want to apply the context scenario approach themselves, to derive advice regarding their own methodological design decision. Table 1 provides a brief overview of the three case studies⁴ concerning their overall objectives, researcher constellation, a short characterization of the models used and some details on how they linked CIB-based context scenarios and quantitative energy models.

Table 1 Overview on case studies: objectives, researcher constellation, models, specific information concerning linking

The objective of C1 was to analyze the future energy demands of private households. The core team consisted of one CIB expert and one energy modeler and was further supported by eight scientific experts from the same institution in order to construct the context scenarios. Taking different policies and frameworks into account, a technology-based simulation model (JEMS-BTS) was thereby used to create different energy scenarios. This model is based on a scenario approach. By using a comprehensive typology for residential buildings, heating, and hot water technologies in Germany [28], it calculates the effects of various energy efficiency measures until 2050.

The application of CIB was planned from the outset and used for the purpose of (a) creating internally consistent storylines, (b) checking the consistency of pre-existing energy scenarios, and (c) extending the general scope of the study towards (societal) issues beyond the realm modeled. To integrate different regional scales, three separate CIB matrices were constructed and linked to each other, at the global, national, and sectoral levels.

The main objective of C2 was to translate the idea of ‘socio-technical’ scenarios into the field of national energy transition scenarios, by developing a new methodology which combines conventional energy systems analysis with societal context scenarios. The core team consisted of three CIB experts and two energy modelers and for the construction of the context scenarios, 65 further researchers with different professional backgrounds and from 32 different institutions became involved. The energy system model primarily applied in the project was based on the accounting framework Mesap/PlaNet, which has been used for years in the building of target-oriented energy scenarios relating to Germany [67], as well as other countries and world regions [68, 69]. The model uses a detailed and transparent data structure to represent the energy system. Science-based premises are key to the methodology applied when defining and modeling development paths in all sectors of the energy system. Infrastructure expansion is therefore determined exogenously for the energy system model, and is not the result of a cost-optimizing objective function.

CIB was used in this project to construct context scenarios that (a) provide the energy model with internally consistent sets of socio-technical assumptions (i.e., avoiding explicit and implicit contradictions between the different assumptions) and (b) represent the future uncertainty of such socio-technical assumptions. Finally, (c) context scenarios and model-based energy scenarios were planned to be merged into ‘socio-technical energy-scenarios’. To integrate different regional scales, C2 created two descriptors which constitute already existing scenarios in a condensed way in an international [8] and a European context [70].

The objective of C3 was to shed light on the regional idiosyncrasies of the German energy transition, particularly at the level of the regional planning region (English for “Raumordnungsregion”), which normally consists of several municipalities. The core team consisted of modelers only, with two of them taking over the responsibility of constructing the context scenarios as CIB novices; they were supported in doing so by four scientific experts. The approach was to model energy scenarios for selected regional planning regions, from the current state of electricity, heat and mobility supply and demand, to possible future states (up to 2030, as well as providing a prognosis on further development). Multiple sub-models were combined in order to quantify the regional impact of the energy transition on the environment, economy and mobility and vice versa.

The application of CIB was only decided upon later during the course of the modeling process. Its aim was (a) to promote the harmonization of factors and trends, which are used equally in the individual sub-models; for example, the future trend of regional population development, and (b) to gain insights into whether the context is adequate, in order to be able to better reflect the completeness of the set of models. C3 decided to select one national scenario from C1 with favorable basic conditions for a successful energy transition and embed the regional CIB analysis into it. This selection was done through discussion.

Structure of the cross-case comparison

To structure the cross-case comparison, we adapted the 8-step process of linking context scenarios with energy models described in Weimer-Jehle et al. [5]. We omitted the last step described there because it did not affect any of the case studies. On the other hand, we have also added steps if they are relevant for knowledge integration from the literature point of view. This includes, e.g., step 1, where it is about defining the design of the whole approach and what common goals should be achieved by the different project partners [44]. The question of how to deal with dissent between the participants (step 6) also plays an important role when considering knowledge integration [44]. The translation of qualitative data into quantitative equivalents on the other hand plays an important role especially in hybrid scenario approaches [9]. This knowledge transformation is also taken into account in this analysis by defining a further process step (step 4). In systematic scenario approaches such as cross-impact balance analysis, the selection of qualitative scenarios for the further process represents an opportunity, but also a challenge for knowledge integration [33] and was included as a process step for the comparative analysis. Finally, we specified 11 steps, which are briefly described in Table 2.

Table 2 Process scheme of linking context scenarios with energy models in 11 steps

Ex post questionnaire

To collect (ex post) evidence on knowledge integration in our case studies, we developed a detailed questionnaire, which was filled out by the participants of the three projects. Data collection was supported by a virtual workshop to assure a joint understanding of this questionnaire. This questionnaire included general questions on the objectives of the project, the applied model(s), the use of CIB and the descriptors included in the context scenario construction (‘Questionnaire and responses from the case studies' [see Additional file 1: Table S1]). Furthermore, specific questions on the methodological procedure in different process steps were asked. In addition to the questionnaire, interviews or written requests served to obtain specific information from individual case study participants as well as observation and participatory observation.

Cross-case comparison and qualitative content analysis

To answer our research question, we carried out a cross-case comparison to learn about similarities and differences in the case studies concerning the aims, process and individual methods. First of all, this allowed us to characterize different design options linking qualitative CIB-based scenarios with energy models as well as roles of CIB within the linking process.

We then applied our analytical framework for the comparative analysis of interdisciplinary research projects and assessed the form of knowledge integration for each process step and case study, respectively.

Finally, we analyzed the effects of different methods and designs on the form of knowledge integration (identified beforehand). For this aim, we used qualitative content analysis [71, 72]. Data were summarized and then structured in a “topic matrix” (English for “Themenmatrix” [72]: 73), which means that they were organized in form of qualitative cross tables (see [72]). For instance, to analyze whether forms of knowledge integration varied with individual methods such as interviews or workshops applied in the different process steps, we juxtaposed these categories in a cross table and then interpreted similarities and differences, and finally derived effects therefrom. In this paper, we analyzed data from a case-oriented as well as from a topic-oriented perspective and juxtaposed categories in different constellations.

To understand the effects of the CIB method, we first examined the CIB for its specific properties (such as the evaluation of interactions by discrete numbers or the underlying consistency principle in the form of the balance algorithm). These may be fundamentally different from other scenario methods and thus might make a difference regarding the ‘predefined’ forms of knowledge integration. Then we checked in the empirical material to see whether these methodological properties determine one or another form of knowledge integration. If this was the case, we declared these as hints for CIB-specific effects on the form of knowledge integration.

Results and interpretation

We subdivided the following section into two parts to generate hypotheses regarding our two main research questions. First, we asked how the linking of CIB-based context scenarios and energy modeling could be structured and which roles the method CIB could play within those designs. Second, we wanted to know what effects the linking design, the CIB method itself and the interplay of different methods have on interdisciplinary knowledge integration.

Linking designs in CIB-based hybrid scenario construction and roles of CIB

To answer our first research question, we compared the processes of the three case studies. We described similarities and differences concerning their methodological design and presented different design options for linking CIB-based context scenarios with quantitative energy models. In addition, we wanted to build a bridge between the purposes of the use of context scenario approach in the respective case studies and the potential roles CIB could play with regard to the linking.

Differences and similarities in the methodologies of the three case studies

Generally, integrating societal contexts requires the integration of perspectives of experts of multidisciplinary scientific domains. Thus, in addition to CIB as systematic qualitative scenario technique and a quantitative model, further qualitative and/or quantitative methods need to be applied to integrate this knowledge and to perform the different process steps described previously. Depending on the model or set of models, more or less complex contexts must be considered. For instance, the application of a set of models makes it more complicated to find joint descriptors which fit or are linkable to all models. Other models might be more or less flexible with regard to required changes due to the linking of qualitative scenarios with quantitative models. Furthermore, there might be less data or knowledge available for some contexts, so that knowledge integration is somewhat “restricted” to limited sources. We show the effects of the design on knowledge integration and that the applications can be manifold and need to be decided on the basis of specific project requirements. Table 3 shows the detailed methodologies of the three case studies.

Table 3 Detailed methodological designs of the three case studies

The table shows that there are many similar methods applied in specific process steps (in more than one project). For instance, in step 1, the methods desk research, workshops and interviews were each applied by two case studies. To implement step 3, all case studies chose the method literature review. However, literature review was carried out in different intensities; it ranged from reviewing only their own prior work (C1) to reviewing a whole scientific discourse (C2). Step 9 is also supported by similar methods in all three case studies, namely by discussions, literature review and the parameterizing of the linking factors.

The main differences between the case studies in the use of qualitative and quantitative methods can be found, for example, in step 6. Complex model contexts require integrating the perspectives of multidisciplinary scientific domain experts. The possibilities of integrating the perspectives of different domain experts and handling dissent between them can be realized in very different ways. For example, regarding impact assessments, C1 intended to reach consent between experts (to define a joint impact judgement) and reached this within a workshop. C2, in most cases, only reached an approximation concerning the impact assessments with the applied method (Delphi-style written 'discussions')⁵. Most non-consensual impact assessments were resolved by averaging the assessments statistically. Important differences can also be identified during step 7. Discussions between energy modelers and CIB experts are an integral part of this step in all cases. But further qualitative methods such as interviews as well as further quantitative methods like correspondence analysis were applied by the different case studies. Furthermore, methodical enhancements of CIB were reached in this step like the linking of three matrices of different scales (cf. [28]. Steps 2 and 4 also revealed evident differences in the application of qualitative and quantitative methods between the cases. Despite these differences, all individual combinations of methods and techniques used in Steps 2–8 were effective in creating context scenarios. And the methods (modeling approaches) in step 9 led to model runs and the effective calculation of energy scenarios (step 10), see footnote 5. What effects the application of the individual methodologies had on knowledge integration is discussed later in this paper.

Different designs linking context scenarios and energy modeling and roles of CIB

In order to answer research question one, we identified different designs of linking CIB with energy modeling, and defined the roles of CIB within this relationship.

Overall, two designs of linking context scenarios based on CIB with energy models emerged from our empirical study. These are shown in Fig. 2: CIB as provider (basic design) and CIB as equal partner (extended design).

Fig. 2

Design alternatives observed in the three case studies and design steps of linking context scenarios and energy modeling

CIB as provider means that energy models use consistent context scenarios that have been derived by a CIB analysis. The results of both the CIB and the energy modeling are interpreted separately; the CIB provides a service for the energy model(s) and the final products are energy scenarios with (consistent) socio-technical context assumptions. This linking design was performed in all three case studies. The extended design, CIB as equal partner, is an optional extension: CIB-based context scenarios and energy model(s) are of equal value for a joint final product, the socio-technical energy scenarios. For a specific energy scenario, the results are reflected and explained on the basis of its underlying societal implications. This procedure leads to a more balanced integration of techno-economic and societal aspects in the hybrid scenario construction. C2 is the only case study which realized this extended type of linking.

Within design alternative CIB as provider, we furthermore found two alternatives concerning the position (a) of the context scenarios within the approach (Design I) and (b) of quantification of descriptors definitions within the process (Design II), respectively. The position of context scenarios can be distinguished as Design Ia ‘Energy model first’ (C1 and C2) and Design Ib ‘Context scenarios first’ (C3). In Design Ia the choice of energy model(s) enables the definition of input requirements for the subsequent context scenario construction. The context is designed depending on the model scope and can serve specific model needs. In Design Ib—if the decision which energy model(s) are to be chosen is still open—the context scenarios are constructed with CIB in a first step and then serve to adapt the final model set (i.e., later and accordingly). The context scenarios are intended to define the context under consideration first (to make mental models and context assumptions explicit) and the models then serve to examine the consequences of the explicit context assumptions. Context scenarios are instead constructed independently of model requirement as consistent framework scenarios for various (potential) models. One or the other design alternative can then be combined with Design II. In Design IIa ‘Quantification as part of context scenario construction’ (C2 and C3), the descriptors and variants are already defined qualitatively and quantitatively within the process of context scenario construction. In Design IIb ‘Quantification as part of energy modeling’ (C1), on the other hand, the quantification takes place immediately before energy modeling which means that the expert judgements of the impact assessments within the context scenario construction process are based on relative classifications, e.g., high/medium/low share instead of quantitative descriptions like 80%/50%/20%.

In sum, linking context scenarios and energy models can be realized in two designs: the basic design, CIB as provider (C1, C2 and C3); and the extended design, CIB as equal partner (C2). The latter needs the preliminaries of the basic linking. Within the basic linking design, the main differences in design between the case studies are the position of CIB in the process (Design Ia—C1 and C2 vs. Design Ib—C3), as well as the position of the quantification (Design IIa—C2 and C3 vs. Design IIb—C1).

In addition to these two general roles of CIB in the linking design, CIB can resume more specific roles within the relationship of context scenarios and models. These roles were not appointed as such by the project members, but are derived through interpreting project objectives, experiences and results, which are partly collected through the questionnaire and partly through personal information.

Firstly, “knowledge representation requires a language to represent the knowledge in” [44]: 15) so CIB can effectively function as such a ‘meta-language provider for interdisciplinary groups’. The language expresses itself through descriptors and variants describing system elements as well as their reciprocal promoting or hindering impacts to discuss their interactions. All types of knowledge can be linked by applying this meta-language. Explicit ‘linking factors’ can be integrated into the context scenarios, and context scenarios and energy models thus have, due to this direct interface, the ability to mutually inform each other during the process. Furthermore, CIB can function as an ‘input-data provider’, meaning that the context scenarios provide combinations of input factors to function as (scenario) frameworks for the energy model(s). Additionally, the inclusion of qualitative (context) factors and the explicit consideration of the socio-technical system results in enhanced system understanding and in a substantiation of the choice of input assumptions made in the context scenarios. The role of an input-data provider in combination with more than one model (it is not restricted to energy models) can moreover result into the role of a ‘manager of context assumptions for a multi-model exercise’, meaning that the scenarios reflect the context of different models and therefore can be applied by all of them (see [10, 39, 74]).

Because of the natural effect of CIB providing various consistent scenarios and enabling the calculation of energy scenarios in the light of different contexts, reflecting alternative future possibilities—future openness—in a changing society, CIB can also function as a ‘context-uncertainty dealer’. Another role of CIB in combination with energy models is the ‘conceptual modeling of the social system or of the socio-technical system’. CIB can enable the adaptation of scenario premises, the linking of further qualitative factors (if only slightly) with the energy model through plausible arguments (cf. [5, 32]. As a model is never static, we assume that the role of conceptual modeling can also result in (deep) structural adaptations of the energy models in the form, that, for example, the model can be further developed depending on the requirements⁶ resulting from the linking with the context scenarios.

Lastly, CIB can also play the role of a “knowledge container” [33], a provider (and storage) of argumentations for socio-technical pathways: within CIB, additional explicit information on contexts and their (assumed) internal structure are stored so that it can be accessed if necessary.

Effects on interdisciplinary knowledge integration

To answer our second research question we first analyze which form of knowledge integration could be assigned to the different process steps conducted by the case studies and then developed hypotheses on how similarities and differences in knowledge integration could be explained by different methodological factors.

Interdisciplinary knowledge integration in the different designs of the case studies

We conceptualized the forms of knowledge integration before the analysis, deriving them from the current literature. As can be seen in the following section, the analysis then showed that differences between the cases in some steps could not be depicted, although the methodological design was different. We found that, for example no literature review against profound literature review cannot be depicted in the form of knowledge integration other than showing that there also was compiling in the latter beforehand. This is how we dealt with such differences in this paper. Furthermore, we are not able to show all the results of the case studies within the limits of a paper, but ‘Examples of interdisciplinary knowledge integration visible in process and results' [see Additional file 2: Table S2] are given.

Knowledge integration starts in step 1 with the definition of the design of the context scenario approach. The linking of the context scenarios and energy modeling is prepared and planned and the roles CIB should play as well as the ambition of integrating methods are decided upon. This step requires bringing knowledge (from different perspectives) together and the research partners need to develop strategies of how to link and to integrate knowledge with CIB and energy modeling. Thus, the knowledge integration form combining is reached through this step (for an overview see Table 4).

Table 4 Final forms of knowledge integration per process step per case study

Then the process of constructing the context scenarios begin. We assessed all processes leading to products and important interim products separately in regard to their forms of knowledge integration.

The steps ‘definition of context, descriptors and variants’ (Step 2 and 3) could be split into two processes each, namely identification and definition, as each process results in its own forms of knowledge integration. While the identification of descriptors and variants simply serves as a collector of ideas of different perspectives (compiling interdisciplinary knowledge, see example A, Additional file 2: Table S2), during the definition, the descriptors and variants are described in-depth and need to be mutually understood and supported by experts of the same domain or by domain experts and energy modelers. The experts need to define descriptors and variants in a way that they can be used by scientific domain experts and energy modelers. Therefore, the overall form of interdisciplinary knowledge integration is combining (see example B and C, Additional file 2: Table S2).

During the quantification of descriptor definitions (step 4), C1 compiled quantifications of their own prior research, while C2 compiled knowledge from various energy scenario studies to identify the range of quantifications discussed in the literature (see example D, Additional file 2: Table S2). The latter approach increased the diversity of knowledge sources that were integrated in the following and thus, the level of interdisciplinarity. Nevertheless, this cannot be depicted in the forms of knowledge integration. C3 integrated new knowledge by calculating population data for a specific region and compiled data from existing sources. As the quantification took place early in C2 and C3, jointly usable content was produced, which means translated, by finding quantitative equivalents to the qualitative descriptor definitions. Thus, in this step the final integration form combining was achieved in C2 and C3, as joint sense-making took place and was then applicable by experts constructing the context scenarios as well as later by the experts running the energy models (see example E, Additional file 2: Table S2). C1 has quantified much later in the process, just before energy modeling. Thus, the knowledge was translated into data. As no joint sense-making took place between the CIB expert and the modeler, and the quantified knowledge is no longer usable for the construction of the context scenarios, it could therefore be assigned to the knowledge integration form compiling. This design had the advantage that the scenarios “could be interpreted, in principle, as frameworks” [28]:942) which allowed C1 to analyze the consistency of other scenarios, which was one of the initial aims of the project (see example F, Additional file 2: Table S2).

Another difference regarding the forms of knowledge integration between the cases could be found in step 5, the assessment of descriptor interdependencies and step 6, the handling of dissent of assessments of interdependencies between descriptors. C1 and C3 performed group discussions to assess interdependencies between descriptors and decided on one assessment per interaction between descriptor developments. They handled dissent directly during a workshop by discussing and finding one solution. The result was joint assessments, where dissent was not visible (any more) (type combining). C2 carried out several interviews per descriptor assessment and thus compiled assessments on interactions between descriptor developments (step 5). Dissent could not be dealt with (step 6) during the survey. Thus, in the aftermath, dissent was dealt with in a written Delphi-style process, as an offer to agree and change one’s own arguments, to disagree and to stay with one’s own assessment or to approximate one’s prior assessments to others. The aim was to get a “cross-checked” CIB assessment for each interaction, to validate assessments by asking more than one expert and also to represent the legitimate dissent within the scientific discourse. Performing step 6 with this multiple interview and Delphi-style technique showed some aspects of combining knowledge, but not fully achieves it, no matter whether this was the aim or not. Thus, knowledge was mostly compiled using this method. Compiling diverging assessments allowed them to maintain the ambivalence of equally valid, but different, arguments (vs. forcing everything into one matrix).⁷ If the arguments for the diverging assessments are traceable, which is the case if CIB is applied, a wider spectrum of diverging scenarios can be interpreted later (see example G, Additional file 2: Table S2). However, the CIB-specific impact scale to decide on assessments was the joint language used by all experts. Thus, to further process data, joint content was possible to be brought about methodically by averaging the impact assessments with the CIB-specific evaluation tool ‘ensemble evaluation’ (type combining).

During the analysis of the interdependencies (step 7), knowledge was synthesized by applying the CIB with its balance algorithm as a bridging method. Single assessments were balanced against each other and produced new content, namely raw scenarios (see example H, Additional file 2: Table S2). But the following understanding of those raw scenarios and their interpretation require more (knowledge) than applying CIB and had to be implemented in a subsequent step (step 8).

Further differences in the three case studies as to forms of knowledge integration can be found in step 8, scenario selection. C3 combined different contents: researchers used one of the consistent national scenarios of C1 reflecting a positive development of the energy transition and matched it qualitatively (verbally) with one of their own regional scenarios. They mirrored the national scenario of C1 and discussed which of the regional scenarios would best fit it. Also, C1 selected the final scenarios regarding their regional compatibility, albeit in a much more complex approach: they constructed individual context scenarios on the sectoral, national and global scale and developed their own approach to tie the individual CIB matrices to each other. They selected scenarios out of the spectrum of scenarios that were mutually consistent regarding main drivers (tie-descriptors) on the three levels [28] and thus, further integrated knowledge to a joint product (type synthesizing, see example B, Additional file 2: Table S2). In contrast, researchers in C2 applied various additional methods and further processed their results together with interim and final products of the CIB analysis (e.g., by using the CIB coding of the scenario constellations as input for statistical analysis) and created new content to assess the CIB output (type synthesizing). For example, knowledge was condensed by entering the datasets of all consistent context scenarios into correspondence analysis, which is able to identify latent structures in the datasets. By doing so, a new understanding of the scenario space was reached and it was possible to visualize scenario landscapes (see example I, Additional file 2: Table S2). By also linking it with the simplified energy model of C2, energy-related knowledge can be assigned to the different scenarios (see example J, Additional file 2: Table S2).

During the energy scenario construction, a set of framework assumptions needed to be derived from the selected consistent scenarios (step 9). This was realized by directly linking either the already quantified descriptors in the specific variant constellation as the basis for model parameter translation (directly integrated or with only minor adaptations, see example E, Additional file 2: Table S2) (C2 and C3), or by parameterizing the quantifiable qualitative descriptors in this step (see example F, Additional file 2: Table S2) (C1). Additionally, some qualitative descriptors were used as model parameter by integrating them as plausibility arguments (defined as soft linking). This therefore resulted in the knowledge integration form combining in all three case studies.

The prepared sets of framework assumptions are the basis for the model calculations (step 10). This is where a direct link between context scenarios and energy models takes place. Applying different context scenarios implies that the resulting energy scenarios reflect future uncertainty (which in turn is reflected in different sets of framework assumptions). In this way, the energy scenarios reflect the spirit of the different selected context scenarios. Not only is this an application of the context scenarios as framework assumptions for energy modeling, but new knowledge can be derived from the explorative approach of confronting energy technologies with uncertain societal conditions. Therefore, knowledge integration in this step can be characterized as synthesizing in C1 and C2 (see example K, L and M, Additional file 2: Table S2). No such exploration occurred in C3, as it only applied one of the consistent scenarios which fulfilled a normative aim. In this approach, no new knowledge could be derived regarding context uncertainty and therefore knowledge was only combined.

Finally, step 11, an integration of context and energy scenarios as socio-technical energy scenarios was implemented by C2 only. CIB and an energy model were combined as equal methodical partners and created a new product, a socio-technical energy scenario (an example is given in [32], SM-(7)). In this step, a predefined energy scenario from the preceding analysis was chosen and examined for its hidden societal implications and reveales new knowledge. The result was a joint product, a socio-technical energy scenario, containing not only quantitative model results, but also “qualitative impact diagrams show[ing] how societal drivers act on the techno-economic factors in the sector, explaining the connection between societal and technical dynamics” [32], SM-(7):22). Therefore, we classified the knowledge integration that occurred at this moment as synthesizing (see example N, Additional file 2: Table S2).

In sum, the forms of knowledge integration found during the same process steps (steps 1–10) in the three different case studies were either partly equal or very similar (steps 1–3, 7 and 9) and partly differ (steps 4–6, 8, and 10). As step 11 was only applied by C2 no comparison was possible.

Effects of CIB on interdisciplinary knowledge integration

CIB as a knowledge integration method which, in different roles, shapes and, at times even predefines, the forms of knowledge integration that are achieved through certain steps. This holds true mainly for steps 2, 3, 5 and 7.

The definition of context, descriptors and variants (step 2 and 3) is strongly shaped by the requirements of CIB, as ‘meta-language provider’ to codify knowledge from different (disciplinary) perspectives in the joint form of distinct descriptors and their variants (see [21]. The CIB method thus supports and, at the same time, requires that knowledge is combined into descriptors and variants to be further processable by CIB. Regarding this step, the integration form of combining knowledge is predefined from the outset, when the CIB method is applied and when at the same time multidisciplinary experts are participating (C1, C2 and C3). The same applies for CIB as ‘manager of context assumptions for a multi-model exercise’, if context assumptions need to be defined for various models which reflect the same problem from a different perspective (C3).

Analyzing interdependencies in process step 7 turns CIB into an ‘input-data provider’ and reveals a synthesizing effect of CIB: the CIB balance algorithm creates new, joint knowledge that integrates the individual (pairwise) impact assessments into an impact network and identifies internally consistent constellations of this network.

Despite those roles of CIB predetermining the form of knowledge integration from the outset, there are other roles showing trade-offs with regard to the form of knowledge integration. This applies to the assessment of mutual impacts between descriptors (step 5). On the one hand, this step is also strongly shaped by the requirements of CIB to use a joint meta-language to communicate interdependencies. This is namely a joint (and semi formalized) impact scale that evaluates interrelations as hindering and fostering impacts. The CIB method at this moment leads to combining knowledge. On the other hand, we found that this CIB-specific effect can be outweighed in this step by the interplay of CIB with certain qualitative methods and in doing so lead to compiling knowledge (see section after next).

A trade-off can also be identified to the role of CIB as ‘knowledge container’. If a project aims for socio-technical scenarios, accessible knowledge (in form of the impact network stored) is a precondition for step 11 being able to further synthesize knowledge by systematically linking model calculations to their specific societal implications in socio-technical energy scenarios. On the other hand, as described in the former section, dissent between experts concerning the impact assessment can be solved (by deciding on one joint assessment) or kept (by storing the different assessments). This has content-specific implications, but also has effects on the form of knowledge integration as knowledge can be compiled (if dissent is kept) or combined (if dissent is solved).

Choosing CIB as ‘context-uncertainty dealer’ implies that distinct context scenarios are selected to be linked with the energy model. Thus, new (synthesized) knowledge develops through dealing with different future perspectives. However, if CIB is not applied to deal with uncertainty, but to provide one consistent scenario only (no matter for which reason), knowledge is not synthesized, but combined.

Lastly, applying CIB as ‘conceptual modeling for the social system or socio-technical system’ also shows a trade-off with regard to the form of knowledge integration. Depending on how the context scenarios are reflected in energy modeling, the form of knowledge integration can change. Combined knowledge integration can be found if the context scenarios are linked through exogenous parameters with models which directly (linking descriptors), softly (plausibility arguments) or indirectly (through impact network) affect the model. If the context scenarios also include endogenous model parameter (which was not the case in the three case studies analyzed), in addition to the direct, soft and indirect links, this could lead to an adaption of model structures, if the impact network required it, and would show synthesized knowledge integration. The same holds true if the model is extended with specifically developed sub-models.

Effects of the linking design on interdisciplinary knowledge integration

In our three case studies, depending on time and financial resources, the researchers defined the purpose of the endeavor and agreed on method(s) and their interplay to fulfill these purposes. Additional competencies in the form of other researchers and/or methods were integrated, if they were judged necessary to reach the project goals and if financial and time resources allowed for them.

CIB is the joint method used by all case studies. We have shown that, in the approach of linking context scenarios based on CIB with energy models, the context scenario approach can follow two design alternatives (see Fig. 2) depending on the interplay of CIB and the model (CIB as provider for the energy model(s) vs. CIB as equal partner). Both designs can lead to a high level of knowledge integration (synthesizing). Within the first design, which is a prerequisite for the second, we can, furthermore, distinguish two different design options depending on the position of CIB and the timing of the quantification within the process. We found out that the order of application (CIB first or model first) does not matter with regard to knowledge integration in the three case studies. C1 and C2 applied the energy model first option (Design Ia), which means that an energy model already existed to be used for the energy scenario construction and context scenarios were constructed in the aftermath. The advantage of this option was that the context scenarios could be constructed in a more target-oriented way, considering specific model premises and model input needs. C3 applied the CIB first option (Design Ib). In all three case studies, the overall form of knowledge integration was not affected by the chosen approach.

The timing of the quantification of qualitative descriptors had an effect on the form of knowledge integration and entailed an important trade-off. If the quantification took place early in the process (quantify during context scenario construction Design IIa as in C2) before the impact judgements were assessed (step 7), the quantifications made were thus the basis for the impact judgements as well as for the energy modeling and therefore were the bridge and joint content between them (combining). Furthermore, the impact judgments made on the basis of qualitative and quantitative definitions could be more consistent. On the other hand, this led to a kind of restriction of the modeling, because model results were more or less fixed to the specified values of the quantified descriptor variants. If, on the other hand, the quantification took place later in the process, i.e., after the impact assessment (quantify later during energy modeling, Design IIb as in C1 and C3), the impact judgements were based only on relative classifications, which might be interpreted differently by each expert and therefore might lead to unclear impact assessments. The quantifiable descriptors in this case had a more indicative character. The advantage is that the quantifications can be adapted more easily to modeling requirements. With regard to knowledge integration of the design option IIb (quantify later) reveals the form of compiling as the quantification is made without joint sense-making.

Effects of the interplay of different methods on interdisciplinary knowledge integration

CIB is a method which can be applied for the integration of knowledge. Furthermore, its specific storage function in form of discrete assessments within a matrix, enables final and interim products of the CIB process to be further processed by linking them with additional (disciplinary) methods. Also, additional methods can build a bridge between context scenario products and energy models. An example of this is the application of correspondence analysis (C2): firstly, to further condense results by considering the whole context scenario set in the future uncertainty space and thus revealing new knowledge about the scenario set (type synthesizing) and secondly, to integrate the context and energy system knowledge (type synthesizing).

The three case studies differ little in their overall methodological approach. But when we go further into the detail of their individual methodologies, rather important differences become visible. C2 made the most innovative choices by trying out new/additional methods and C1 also developed methodical enhancements. The reasons for this might be that knowledge integration and the application of CIB were part of the project agendas and adequate resources were planned for this. C3 did include the application of context scenarios only in the aftermath and, in addition, without having planned adequate resources.

In sum, we found out that the individual (qualitative) methods used to operationalize different process steps mattered only slightly for the forms of knowledge integration that were reached. The only major differences could be found in step 5 (assessment of interdependencies) and step 6 (handling of dissent). C2 has decided to conduct the assessments with individual interviews and has obtained several expert opinions for each. Even if these experts used the same “language” to assess the interdependencies, it is not possible to reach consensus in interviews conducted separately within the same step. To deal with dissent C2 implemented a written Delphi-style process. A Delphi process is generally designed to generate consensus or to establish consent on dissent [73] on a specific topic among a group of experts. The written Delphi applied in C2 via expert interviews did not strive for consensus on impact assessments at all costs, but rather were used to clarify where justified dissent was present. This decision thus had an effect on the form of knowledge integration in addition to the choice of the interview method. The methodological choice to carry out interviews to assess the impact assessments (step 5) with more than one expert from one domain (C2) leads to less integration (type compiled), at the very moment when interviews are carried out. But, integration could be reached methodically. In terms of content, this approach has led to the fact that the richer background information could be synthesized in a later process step, which also reflects the ambivalence and dissent of the scientific discourse. However, in our conceptualization of forms of interdisciplinary knowledge integration this cannot be made visible. Other methods which do matter (directly) concerning the form of knowledge integration are additional (disciplinary and quantitative) methods as well as CIB method enhancements which have the potential to produce new knowledge such as correspondence analysis (C2) or Multi-level-CIB (C1).

Discussion and conclusion

Scenario constructors apply hybrid scenario methodologies to benefit from advantages of both the qualitative as well as the quantitative scenarios (cf. [9]. In such hybrid scenario approaches interdisciplinary knowledge integration is the overarching aim, but also its central challenge (cf. [11]. However, we still lack knowledge of whether, how and to what degree knowledge integration in hybrid scenario approaches actually is achieved. To better analyze the knowledge integration performance of hybrid scenario methodologies, we propose to distinguish three forms of interdisciplinary knowledge integration (compiling, combining, synthesizing). This distinction is a conceptual contribution to interdisciplinarity research (cf. e.g., [50]. In this paper, we applied these concepts to better understand interdisciplinary knowledge integration in processes using a relatively new hybrid scenario approach, the linking of qualitative scenarios based on Cross-Impact Balance analysis (CIB) with quantitative energy models and draw on three different case studies applying this approach. In an ex post analysis, we compared and analyzed the different case studies concerning their linking design and the role of CIB. We were able to identify forms of knowledge integration within the scenario process, and reflected the effects of the interplay of respective methods with regard to interdisciplinary knowledge integration step by step throughout the process.

Our exploratory cross-case analysis was based on three case studies: therefore, statistically we are not able to generalize results. Still, we assume that the process scheme presented in the method section and the findings concerning the linking designs are useful to guide scenario constructors and energy modelers to construct hybrid scenarios in general. Furthermore, we assume that our results on forms of interdisciplinary knowledge integration resulting from the different designs are generally valid. The same applies to the roles of CIB and their (predetermined and trade-off) effects on knowledge integration. Clearly, there might be further roles of CIB, and further forms of linking methods, which were not seen in the cases presented here. Apart from that, there are also hints at aspects, one should pay attention to when linking CIB with quantitative models. The first hint is that, in one case (C3), the rationale of the model (Input–Output) did not fit one future perspective (here: shrinking GDP) and, therefore, the scenarios including this future perspective had to be excluded from further research. It could also be that other models have rationales which cannot be linked with context scenarios properly, this should be clarified at forehand. Secondly, the design option CIB first showed no particular effect on the form of interdisciplinary knowledge integration in our case study. But experiences from other studies indicate that there could be synthesizing effects, e.g., by adapting model structures with regard to the context scenarios [10]. And thirdly, the case studies presented here show that the choice of qualitative methods to realize the different process steps do, with two exceptions, not play a decisive role for the form of interdisciplinary knowledge integration as we operationalized it. But there could be different method constellations that might show further effects of qualitative methods on interdisciplinary knowledge integration. Overall, these hints require further empirical analysis.

In sum, the overall methodological design (meaning the technical and empirical realization of each of the 11-step process scheme), should depend on the specific roles researchers want CIB to play and on the form of knowledge integration they want or need to achieve. If interdisciplinary perspectives need to be brought together, CIB can function as a meta-language and can be used to combine knowledge through a joint language. Or, by applying CIB as context-uncertainty dealer, knowledge can be synthesized if context uncertainty and its consequences on energy systems are reflected in the energy scenarios. It seems that further qualitative methods can mainly be selected according to aims, preferences, time and financial resources.

This was an explorative and qualitative study, thus many roads for further research were indicated: one is to test the hypotheses generated by our study through further empirical analysis. Furthermore, the conceptualization of forms of interdisciplinary knowledge integration for hybrid scenario approaches was tested with our exemplary case studies linking CIB-based context scenarios with energy modeling only. We assume that some of the (predefined or trade-off) effects CIB generates through its specific method features do differ from effects on knowledge integration of hybrid approaches that apply intuitive methods for the qualitative scenario construction as SAS (e.g., [9]. Still, it seems promising to compare the effects of different methodologies of the ‘context scenario approach’ on knowledge integration with the effects of other hybrid scenario methodologies for a better design of hybrid scenario processes. Last, but not least, our conceptualization of the forms of knowledge integration has shown limitations with regard to a more precise differentiation. Thus, we propose to further differentiate and detail the three forms of interdisciplinary knowledge integration developed here by conceptualizing and operationalizing 'interdisciplinarity or multidisciplinarity' in more detail. This would allow a better inclusion of corresponding aspects such as disciplinary multitude and bandwidth into the analysis. We assume that further differences of the cases, e.g., concerning different forms of researcher involvement, could be analyzed in more detail and might reveal additional effects on interdisciplinary knowledge integration.