Determining Essential Components of a College-level Bioenergy Curriculum in the United States Using the Delphi Technique
Table of Contents: February 2018 — Arts, Social Change, and a Vision for Sustainability
Introduction
Bioenergy could play a vital part in a successful global bioeconomy (Smáradóttir, 2014), defined as “an economy where the basic building blocks for materials, chemicals and energy are derived from renewable biological resources” (McCormick & Kautto, 2013, p.2589). A prosperous bioeconomy has the potential to lead to healthier lives, lower healthcare costs, transformed manufacturing practices, and improved agricultural productivity to meet the nutritional needs of the population (White House, 2012). An important, additional benefit of a successful bioenergy industry is the numerous employment opportunities it will create, especially in rural areas (Becker, Skog, Hellman, Halvorsen, & Mace, 2009; Mathews, 2008). Over 40 nations have promoted the bioeconomy as a way to combat the challenges of sustainable global development (Communiqué, 2015).
Worldwide energy consumption is predicted to nearly double between 1990-2030 (Taylor, Govindarajalu, Levin, Meyer, & Ward, 2008), and there is a need to find feasible solutions to meet this demand. Heavy reliance on fossil fuels for energy needs has resulted in accelerated climate change, compromising global environmental health and national independence and security. Much research is currently being conducted to improve the process of converting the solar energy stored in biomass to a form that can be utilized efficiently and cost-effectively to the benefit of society and the environment (Devarpalli & Atiyeh, 2015). New technologies that advance conversions of biomass to biofuels, coupled with the instability of oil prices, strongly suggest that the importance, applicability and utilization of bioenergy will continue to escalate (Popp, Lakner, Harangi-Rákos, & Fári, 2014). In fact, from 2005 to 2011, global demand for biofuels increased four-fold to approximately €60 billion (Scarlat, Dallemand, Monforti-Ferrario, & Nita, 2015).
As the bioenergy industry and job market continue to grow (Marra, Antonelli, Dell’Anna, & Pozzi, 2015), a well-trained workforce capable of implementing innovations and meeting impending challenges will be required. Yet both worldwide and in the US, there is a severe deficiency in training programs and courses dedicated to bioenergy (Maclean & Panth, 2018). The lack of designated training programs for bioenergy suggests that many of the graduates currently entering this growing field are not being specifically educated about bioenergy prior to joining the workforce. Because “a vibrant bioeconomy depends on the education and skills of its workers” (White House, 2012, p.33), the US National Bioeconomy Blueprint (2012) includes updating training programs as one of its five strategic objectives. Internationally, interdisciplinary collaboration among educators has been identified as one of the five bioeconomy cornerstones (El-Chichakli, von Braun, Lang, Barben, & Philp, 2016).
However, the bioenergy field requires a unique set of skills that is not typically covered in traditional undergraduate education. It requires knowledge of energy as a foundation, but it also requires general knowledge in a variety of other subjects such as chemistry, biology, engineering, policy, and economics. This highly interdisciplinary field necessitates the integration of multiple disciplines, highlighting the need to organize the many aspects into a structured educational framework.
Because bioenergy is a nascent industry and limited college-level bioenergy education programs have been established, little research has been done to determine what a curriculum should include. In the past, curricula for other sectors of environmental education, such as conservation education, have been ineffective, because curricular components were chosen via “intuition” (Hungerford, Peyton, & Wilke, 1980). When there is a lack of research available to inform the development of a curriculum, it “results in curriculum development which is largely intuitive in nature without a cohesive strategy to guide developers towards goals which would facilitate the production of a citizenry competent to cope successfully with environmental issues” (Hungerford et al., 1980, p.42). Bioenergy has many implications for the environment (e.g. Suttles, Tyner, Shively, Sands, & Sohngen, 2014), particularly by providing alternatives to fossil fuels and mitigating climate change. Therefore, it is crucial to establish a research-based educational framework for appropriate bioenergy employment preparation.
The Delphi research methodology has been successfully employed in a variety of curriculum development and workforce training investigations (e.g. Kloser, 2014; Rossouw, Hacker, & Vries, 2011; Osborne, Collins, Ratcliffe, Millar & Duschl, 2003). This recognized, mixed-methods procedure is popularly defined as a technique used to attain consensus from experts using survey iterations and selective feedback (Dalkey & Helmer, 1963). Combining both quantitative and qualitative approaches, the Delphi procedure assists in forecasting upcoming needs in order to make suitable plans for the future (Ludwig, 1997). Delphi’s sustained attractiveness as a planning and decision making tool stems from its ability to efficiently elicit opinions from experts who bring knowledge, authority, and insight to the problem (Gupta & Clark, 1996). Busy schedules and travel costs can prevent face-to-face gatherings of people from multiple locations, but Delphi provides a convenient, lower cost alternative to traditional brainstorming (Dalkey & Helmer, 1963). Additionally, anonymity among panel members, combined with the iterative format, reduces the bandwagon effect, facilitates the expression of honest opinions, and allows participants the freedom to change their minds between rounds without judgment (Rowe & Wright, 1999).
The Delphi technique is useful and applicable when new trends and subsequent occupational needs are emerging (Toohey, 1999). The method is applied when traditional statistical methods are not appropriate or possible due to inadequate historical or technical data, therefore necessitating judgment via human input (Rowe & Wright, 1996). Given the current developmental stage of the bioenergy field, Delphi provides an appropriate forecasting technique to cultivate a consensus among experts to improve workforce education efforts.
This study aimed to determine the essential components of a college-level bioenergy curriculum. The present study emphasized the understanding of content knowledge to provide a foundation for the development of bioenergy training programs. Results can be used to bolster existing bioenergy education initiatives and provide guidance to administrators interested in developing new programs. The study took place in the US, but using this framework, international institutions and training programs can adapt the learning content based on geographical location and technologies and industries growing in their region.
Materials and Methods
A three-round Delphi study (Okoli & Pawlowski, 2004) was implemented to establish consensus among a panel of US experts regarding necessary components of a college-level bioenergy curriculum. For more established areas of study, a review of the literature is often utilized to inform the first round of Delphi questionnaires (Franklin & Hart, 2007). For example, the broader and more general energy literacy principles (US Department of Energy, 2017) could have been used as the initial starting point for the current study. However, these principles do not address the bioenergy-specific knowledge that was the goal of the present study. Therefore, due to the exploratory nature of this research and lack of literature available, the traditional open-ended questionnaire was used (Murry & Hammonds, 1995). The study was conducted in accordance with Oregon State University’s Institutional Review Board requirements. The nature of the research was described in Round One, and only those participants that selected “agree” to the online informed consent information were allowed to complete the survey. Participants were given approximately two weeks to complete each round, and reminders were sent after one week and again one day before the deadline for each round.
Panel Selection
The expert panel in this Delphi study was composed of 12 diverse professionals with extensive experience in the areas of bioenergy research, education, and industry. Initially, in May 2014, 169 potential participants were invited based on their affiliation with the United States Department of Agriculture (USDA) consortium, Advanced Hardwood Biofuels Northwest (www.hardwoodbiofuels.org). They were asked to extend the invitation to additional suitable experts, resulting in a purposive, snowball sampling of participants. To be eligible as an expert for this study, participants had either (a) a publication in a peer-reviewed, bioenergy-related journal; (b) at least two years of experience teaching bioenergy classes at the college level; (c) three years of bioenergy industry experience; and/or (d) a PhD in a related field.
Recruitment resulted in 42 interested experts who were then randomly assigned to either an elementary through high school (K12) (20 participants) or college-level (22 participants) curriculum Delphi panel. Only the college-level results are reported in this paper. Of the 12 experts who joined the college-level panel in Round One, eleven members (92%) remained through Round Three for this study. While there is no agreement on minimum or maximum panel sizes in Delphi studies (Atkins, Tolson, & Cole, 2005), many studies select between 10 and 100 experts from a sample of convenience. Of the initial employers, educators, researchers, administrators, and government agents that made up the expert panel, one was female and seven hold PhDs in associated fields. They also represent states from across the US: Oregon, California, Colorado, New York, and Maryland.
Delphi Rounds
Round One
In May 2014, panelists were asked to respond to a single open-ended question: “Keeping in mind the future of a commercial bioenergy industry, what content knowledge should a student have upon completion of a college-level bioenergy curriculum?” The questionnaires for all rounds were delivered electronically using the survey software package Qualtrics (Qualtrics, Provo, UT). Responses from Round One were qualitatively coded by two members of the research team, using NVivo (QSR International Pty Ltd, Victoria, Australia), until 100% inter-rater reliability was reached. The resulting 14 themes from Round One became the questionnaire items for subsequent rounds.
Round Two
In June 2014, participants were presented with the 14 themes and asked to rate the importance of including each theme in an undergraduate bioenergy curriculum, using a five-point Likert-type scale (1=Non-essential to 5=Essential). Included with each theme title were a theme summary and anonymous example panel responses (Figure 2.1) to increase transparency regarding how the themes were determined. For each theme, experts were also given the opportunity to provide additions, modifications, clarifications, and/or justifications.
Figure 1. Example of theme title, summary, example responses, and rating scale presented to experts in Round Two. Source: Author’s own.
Round Three
In August 2014, experts were shown the panel’s mean rating and standard deviation from Round Two, as well as the title and summary for each theme, and again asked to rate them on the five-point scale. To encourage consensus (e.g. Rossouw et al., 2011), participants were asked to provide an explanation if they deviated more than one point from the panel’s mean score. Again, panel members were asked to provide comments, so this final round also served as a type of member-check that ensured the researchers had correctly interpreted and categorized the experts’ responses (Engels & Kennedy, 2007). Eleven of the 12 experts responded.
Analysis
Due to the non-random selection of experts and non-independence and non-reproducibility of the study, statistical inference cannot be drawn, and therefore analysis was limited to descriptive statistics (Quendler & Lamb, 2016). Mean ratings were used to determine the importance of each item, while standard deviation was used to represent consensus. Stability was also calculated in the final round and was defined as a difference of 33% (one third) or less in the expert panel’s mean ratings between Rounds Two and Three (Osborne et al., 2003). Stability is referred to as “the consistency of answers between successive rounds of the study”, and all stability values fell within an acceptable range (Dajani, Sincoff & Talley, 1979).
Results
Delphi Rounds
Round One
This round served as a brainstorming session. Response length ranged from a few sentences to over 1,100 words; there was no limit to the number of items an expert was allowed to list. Participants took between three and sixty-three minutes to complete the survey. After coding, the themes resulting from Round One were: Types of Bioenergy, Logistics, Societal Issues, Bioenergy Market, Environmental Impacts, Biomass Production, Biomass Composition, Conversions, Energy Basics, Current Technologies, Life Cycle Analysis, Policy, Business-Related Knowledge, and Non-Bioenergy-Specific Fundamentals.
Round Two
All twelve of the experts from Round One completed Round Two. There were multiple panel comments from this round about the interrelatedness of two of the themes, Life Cycle Analysis (LCA) and Environmental Impacts. For example, experts stated that “LCA and the environmental theme overlap significantly”, and “LCA goes hand-in-hand with Environmental Impacts…it’s essentially the same concept, but more in depth.” Based on these comments, the researchers included Life Cycle Analysis within the theme of Environmental Impacts, resulting in 13 themes to be rated in Round Three. Upon completion of Round Two, twelve of the original fourteen themes had a mean of ≥4 on the five-point scale (Table 2.1), signifying that the experts considered them important (Osborne et al., 2003). Of these twelve essential themes, nine had standard deviations of <1.0, which indicates a high level of consensus (Osborne et al., 2003) for these themes. Interestingly, 2 of the 3 themes with standard deviations >1.0, Energy Basics and Environmental Impacts, were among the three highest rated themes, receiving means of 4.5 and 4.33, respectively. Although the Delphi method lacks consensus in the literature regarding cut off scores (Kloser, 2014), this study used a mean of ≥3.6 out of 5, as done by Osborne et al. (2003), so all themes were included to be rated in Round Three (albeit two were combined into one).
Table 1: Bioenergy Curriculum Framework Themes and Summaries with Ratings from Rounds Two and Three
| Round Two | Round Three | |||
| Theme Title/Summary | M1 | SD | M1 | SD |
| Energy Basics: Students should understand the fundamental principles of energy | 4.50 | 1.24 | 4.73 | 0.47 |
| Types of Bioenergy: Students should be familiar with a broad range of available and emerging types of bioenergy | 4.50 | 0.90 | 4.64 | 0.67 |
| Environmental Impacts (including Life Cycle Analysis): Students should be familiar with positive and negative environmental impacts related to bioenergy production and evaluate inputs and outputs to make informed decisions | 4.33 | 1.23 | 4.45 | 0.52 |
| Current Technologies: Students should be familiar with current energy production | 4.17 | 0.94 | 4.27 | 0.47 |
| Societal Issues: Students should recognize the societal consequences (pros and cons) resulting from a bioenergy industry | 4.08 | 0.90 | 4.27 | 0.47 |
| Logistics: Students should understand the planning, implementation, and coordination required for the bioenergy supply-chain | 4.08 | 0.67 | 4.18 | 0.40 |
| Policy: Students should be familiar with existing and proposed policies that influence the growth of the industry | 4.00 | 0.95 | 4.09 | 0.54 |
| Biomass Composition: Students should know the basic biomass components | 4.33 | 0.89 | 4.00 | 1.00 |
| Non-Bioenergy-Specific Fundamentals: In addition to the above-mentioned topics, students should also have fundamental coursework and skills (Biology, Chemistry, Math, Physics, Writing Skills, Ecosystems, Communication, Data Analysis/Statistics, Process Modeling) | 4.23 | n/a | 4.00 | n/a |
| Biomass Production: Students should understand the methods involved with producing commercial quantities of biomass | 4.42 | 0.79 | 3.91 | 0.94 |
| Conversions: Students should have scientific knowledge of converting biomass to intermediates and end products | 4.08 | 0.90 | 3.91 | 0.54 |
| Bioenergy Market: Students should be familiar with the current and projected bioenergy market | 3.92 | 1.00 | 3.73 | 0.65 |
| Business-Related Knowledge: Students should have a basic understanding of business management and strategy (Finance, Economics, Risk/SWOT Analysis, Return on Investment Calculations) | 3.75 | n/a | 3.65 | n/a |
Round Three
After Round Three, all themes had a mean rating of ≥3.6 and a standard deviation of ≤1.0, indicating consensus regarding the importance of including all themes in a college-level bioenergy curriculum. Although experts were given the opportunity, no new themes were added during Round Three.
Energy Basics consistently ranked (solely or tied) as the most important item to include in a college-level bioenergy curriculum (Table 2.1). One expert stated “The greatest source of wasted effort and outright foolishness in the bioenergy field arises from a poor understanding of energy production…”, and another wrote “I don’t know how you can teach bioenergy without assuring they know this stuff.” Conversely, one participant was concerned that this topic was one that students should be familiar with before attending a bioenergy course, and thus should not be included in a bioenergy curriculum, because “having to start with this basic information…will result in many important topics not being covered”. This may explain the large standard deviation in Round Two. However, in the end, this theme (mean=4.73) outranked Types of Bioenergy (mean=4.64), with which it was formerly tied (means=4.50) during Round Two, and the standard deviation dropped from 1.24 to 0.47, indicating overall consensus regarding its importance.
Similarly, Types of Bioenergy maintained a high rating throughout all rounds (Table 2.1). A panelist mentioned that “students will tend to focus on a narrow set or single type, so understanding the whole range of types is useful mainly for context”. However, it was also stated that “technologies are changing rapidly, and what seems hot in their first year of study may have failed, either in demonstration or commercial, before they even finish college”. Because these themes are general enough to be applied to many types of bioenergy, and “each existing and emerging type of bioenergy has a unique market and environmental impact” (expert panelist), another expert suggested that “students should know where to look for and how to evaluate emerging types of bioenergy”.
Based on panelist comments, Environmental Impacts was combined with LCA to form the theme titled Environmental Impacts (including Life Cycle Analysis). Additionally, the following two comments, “the general assumption is that bioenergy only has positive environmental impacts. That is not even close to being true. Students have to understand the possible environmental costs”, and “particular attention should be made to give a balanced perspective”, led the researchers to reword the theme summary from “Students should be familiar with environmental impacts related to bioenergy production” to “Students should be familiar with positive and negative environmental impacts related to bioenergy production and evaluate inputs and outputs to make informed decisions”.
General understanding of Current Technologies was deemed important to provide “bioenergy’s role in context” (expert panelist). Comments suggested that enough background knowledge was needed in order for graduates to make “a critical comparison of these to Bioenergy options”, but specifics, such as “refining and petrochemical information is not that essential”. An expert also claimed that “this comparison should be THE CENTRAL OBJECTIVE[sic] of the curriculum”, expressing that all of the themes are sub-topics that are encompassed in this comparison and should support the primary goal of producing graduates that can propose and assess bioenergy alternatives to current technologies.
The theme Societal Issues was developed from comments about the changes (positive and negative) that will occur at local, national, and global levels. Some experts were concerned about how implementation of bioenergy plants in rural areas could affect the daily lives of residents in these areas, with increased “noise”, “traffic”, and “food and land prices”. Although it was not mentioned specifically by experts, food security has been deemed “top priority” (El-Chichakli, von Braun, Lang, Barben, & Philp, 2016) in regards to a global bioeconomy, and could be included in this theme. Experts also mentioned “clarifying misconceptions” and “convincing people to choose bioenergy options”. Increased awareness and education can increase bioenergy usage (Domac et al., 2004), which could lead to “increased national security” (expert panelist). Because of the range of local, regional, and national effects, it was suggested by an expert that “broad survey of these issues, perhaps with a few case studies where they are investigated in greater details should suffice.”
Items listed during Round One, such as “a holistic approach to the supply chain” and infrastructure requirements such as transportation and storage of biomass, led to the creation of the theme Logistics. This was considered an important theme because “Logistics are key to the Value Chain as a whole, and also key to understanding the limitations of biomass vs. more logistically favorable resources” and “Without the complete supply chain biofuels do not enter the market.”
Experts believed that the theme of Policy should be covered at the “overview level” and “Students should learn where to find this information as policies will probably change during their college study period”. One expert had a particularly strong response about this theme:
Policy has a huge effect on Bioenergy markets as a whole, as well as profound impacts all along the Value Chain. Policy needs to be emphasized not only because it is important, but also to make sure that no one graduates thinking that science, engineering, and logical thinking in general are even close to sufficient to develop a Bioenergy economy. The greedy, corrupt, ignorant, science-denying fools who inhabit political office at all levels, and the vested interests that buy and own them will have more to do with the development of this field than any underlying truth.
Biomass Composition encompassed cellulose, hemicellulose, lignin, ash, and moisture content. This was considered essential because “Understanding the chemistry of the feedstocks is important for understanding fuel conversion technologies”. One expert expressed that this theme should be covered “As long as it is limited to the BASICS [sic] as stated in the Summary… Students will only need to get into detail as they pursue specific cases, either in the course of their education or later in their professional careers.”
Experts listed a variety of Non-bioenergy Specific Fundamentals, such as chemistry, math, physics, ecology, writing and other communication skills, and data analysis capabilities. When rated individually, the topics considered most important by the experts were chemistry, communications skills, and data analysis. Although some of these topics can be practiced simultaneously within a bioenergy education context (e.g. writing proposals, giving presentations), much of the hard science would be likely to be covered in courses taken prior to a bioenergy course.
Biomass Production was a theme that emerged based on the example responses in Figure 2.1. During Round Two, more specific comments about the importance of this theme were communicated. For example, “The whole value chain is important, but nothing happens without BM production, and no value chain can be sustainable if the BM [biomass] production is not sustainable, nor economical if the BM production is not economical, etc., etc.; BM Production is “First among equals” in the Value Chain [sic].” Additionally, the “grade” or “quality” of feedstocks “can be impacted by storage, transport and seasonality as well”, and “This needs to [be] amplified as it will have profound impacts on conversion processes.”
Panel responses such as “biological, thermal, and chemical conversions”; “residence time and recycle/waste streams”; and “biomass processing” resulted in the theme, Conversions. Although it was considered important enough to be included in the framework (mean=3.91), one respondent was concerned that the “technical details may be overwhelming”. Similarly, another expert stated “All students should have a basic understanding of conversion technologies and how different biomass resources are best suited for certain conversion pathways. However, this can quickly get too complex for students lacking a strong technical foundation.” This shared concern about the intricate details of these processes may explain why this item was considered a lower priority.
Despite the notions that “Policy and natural gas availability drive the market [in the United States]”, and “The market ultimately will drive renewable bioenergy implementation, and students need a sense of past, current and ongoing changes to that energy market picture,” Bioenergy Market as a theme received a mean rating of <4 during both Round Two and Round Three (Table 2.1). The low rating of this item may be justified by the following expert quote: “These are going to vary greatly by geography and type of bioenergy…There is too much variety and variability to cover in any depth; if the students get a firm understanding of EXISTING [sic] energy markets under the “Energy Basics” topic, then they will have the background to dig into Bioenergy Markets as needed after graduation.”
Although it fell within the predetermined cutoff values to be included in the framework, Business-related knowledge consistently ranked as the lowest priority theme throughout the study (Table 2.1). The general consensus was that risk, return on investment, and micro- and macroeconomics all play crucial roles in a bioenergy industry, particularly when projects are scaled up for commercial production. However, multiple experts expressed that this content knowledge should be learned in separate business courses. For instance, a unit “designed specifically to look at Bioenergy market economics would be reasonable”, according to one expert, but general business-related information should be learned outside of a bioenergy curriculum.
Bioenergy Education Structure
Although the initial open-ended question specifically asked about content knowledge, a variety of general comments were made about the structure of bioenergy education. These comments were not coded as curriculum content themes; however, they may be useful when establishing new educational programs. For instance,
A college level bioenergy curriculum should have two broad learning objectives: 1. Become conversant in the terminology, technology, and issues associated with the advancement of bioenergy. 2. Develop a deeper expertise in one or more of the aspects of the bioenergy arena. The overall goal of a bioenergy curriculum should be to prepare students to be knowledgeable critical thinkers.
Similarly, it was also suggested that there should not be a traditional college degree in bioenergy, but, rather, that bioenergy should be offered as a specialization option that can be earned alongside a conventional major. These comments suggest that students should have both breadth and depth in their knowledge about bioenergy. Lastly, it was also noted that to apply the content knowledge to practice, “Some sort of multi-disciplinary ‘Capstone’ project would also be great, bringing together students from two or more disciplines on teams that would conduct a significant Design or Analysis Project [sic] in a Bioenergy field”.
Discussion
Bioenergy is gaining increased attention as a potentially sustainable solution to energy needs (Sagar & Kartha, 2007). As a result, a number of new university courses are being developed in this field (Malone, Harmon, Dyer, Maxwell, & Perillo, 2014). The results of this study demonstrate that bioenergy could be a formidable platform for interdisciplinary education due to the numerous subjects involved and the manner in which these disciplines must merge to create viable solutions and innovations to advance the field. As Malone et al. (2014) state, “The effectiveness of interdisciplinary courses is enhanced when diverse faculty and stakeholders are intimately involved in curriculum development…”, and interdisciplinary education “encourages students to adopt a broad world view, facilitates a richer understanding of individual disciplines, enhances critical thinking, and provides students with the tools to develop solution-focused problem-solving skills” (2014, p.3). Moreover, McArthur and Sachs (2009) note the important role that university-level interdisciplinary programs play in producing workers who are able to innovate in order to solve problems related to sustainable energy. For these reasons, strong efforts were made to recruit a variety of expert bioenergy researchers, educators, and employers to participate in this study to determine the essential components of a college-level bioenergy curriculum.
Although noteworthy research exists pertaining to energy education, and even renewable energy (i.e. solar, wind) as a whole, to date there is inadequate bioenergy-explicit curricula literature (Malone et al., 2014). Many countries are aiming to boost the bioeconomy (El-Chichakli, von Braun, Lang, Barben, & Philp, 2016), including the US, which has funded bioenergy projects with a substantial amount of public funds (e.g. through US Department of Agriculture and Department of Energy). Thus it is vital that these efforts be augmented by providing current students, who will become future leaders, with suitable, rigorous curricula to ensure they are supported for success. However, due to the complex and interdisciplinary nature of the field of bioenergy, attempting to create a focused, effective, college-level curriculum may prove to be intimidating and difficult for individual instructors. Similar to curriculum-oriented Delphi studies done by Osborne et al. (2003) for the nature of science and Rossouw et al. (2011) for engineering and technology education, this study assists in establishing a common language and curricular priorities. By utilizing expert knowledge from a variety of bioenergy-related fields (e.g. feedstocks, conversions, policy, business) from across the country, we have developed a general bioenergy curriculum framework that provides a starting point for institutions looking to launch educational programs.
Limitations and Implications for Future Research
Initially, only individuals affiliated with the Advanced Hardwood Biofuels NW consortium were invited to participate in this study. Although additional experts were invited as a result of snowball sampling, more than half of the expert panel was comprised of researchers from the Pacific Northwest of the US, and only one female participated. The size of our expert panel was relatively small. Studies have found that this size panel is sufficient when experts represent heterogeneous areas of expertise; smaller heterogeneous Delphi panels have been equally effective in generating ideas as larger, homogeneous groups of experts (Linstone & Turoff, 1975). However, to obtain the most inclusive insight, best efforts should be made to obtain a larger panel for future research, including more females, as well as more stakeholders from industry and government agencies from diverse locales, including international sites.
Since the bioenergy arena is continually changing, the results are dependent on the nature of the problem at the present time, and opinions may change with technological advancements and new research developments (McKenna, 1994). The findings presented here represent the viewpoints of experts involved in the field in 2014. Because the market is highly influenced by policies and petroleum prices, expert perspectives and priorities may change as more bioenergy technologies attain commercial status.
While the present study emphases the understanding of foundational content knowledge for bioenergy education, it will be important for future research to focus on the skills and competencies needed in this growing field. Knowledge in a particular field is only valuable to the extent that it can be utilized by those leaving higher education. A deeper understanding of the skills needed by students in the bioenergy field will complement the results from the present study and provide a broader framework for future curriculum developers.
Conclusion
The purpose of this study was to generate a list of basic and broad, yet essential, bioenergy concepts to be used in the establishment of college-level curricula. The results of this study can be used as a baseline to guide the future development of bioenergy curricula and to develop instruments to measure bioenergy learning in the US, and may be adapted for region-specific technologies in the US and internationally. The themes represent the foundational knowledge students should have in order to begin to contribute effectively to the advancement of this expanding discipline and support them for success in the forthcoming bioeconomy.
Acknowledgements
This project is part of the Advanced Hardwood Biofuels Northwest consortium and is supported by Agriculture and Food Research Initiative Competitive Grant No. 2011-68005-30407 from the USDA National Institute of Food and Agriculture. The funding source had no involvement in study design; in the collection, analysis and interpretation of data; in the writing of the report; nor in the decision to submit the article for publication. The authors would also like to thank the panel experts for volunteering their time and knowledge to this study.
References
Akins, R. B., Tolson, H., & Cole, B. R. (2005). Stability of response characteristics of a Delphi panel: Application of bootstrap data expansion. BMC Medical Research Methodology, 5, 1–12. https://doi.org/10.1186/1471-2288-5-37
Becker, D. R., Skog, K., Hellman, A., Halvorsen, K. E., & Mace, T. (2009). An outlook for sustainable forest bioenergy production in the Lake States. Energy Policy, 37(12), 5687- 5693.
Communiqué: Making bioeconomy work for sustainable development, Global Bioeconomy Summit, Berlin, Germany, November 2015. Retrieved 5/4/2017 from http://gbs2015.com/fileadmin/gbs2015/Downloads/Communique_final.pdf
Dajani, J. S., Sincoff, M. Z., & Talley, W. K. (1979). Stability and agreement criteria for the termination of Delphi studies. Technological Forecasting and Social Change, 13(1), 83-90.
Dalkey, N., & Helmer, O. (1963). An Experimental Application of the DELPHI Method to the Use of Experts. Management Science, 9(3), 458-467.
Devarapalli, M., & Atiyeh, H. K. (2015). A review of conversion processes for bioethanol production with a focus on syngas fermentation. Biofuel Research Journal, 2(3), 268–280. https://doi.org/10.18331/BRJ2015.2.3.5
Domac, J., Healion, K., Lunnan, A., Madlener, R., Nilsson, S., Richards, K., … & Segon, V. (2004, January). Educational Work of Iea Bioenergy Task 29: Socio-Economic Drivers in Implementing Bioenergy Projects. In 2nd World Biomass Conference-Biomass for Energy, Industry and Climate Protection (pp. 10-14).
El-Chichakli, B., von Braun, J., Lang, C., Barben, D., & Philp, J. (2016). Five cornerstones of a global bioeconomy. Nature, 535(7611), 221–223.
Engels, T. C., & Kennedy, H. P. (2007). Enhancing a Delphi study on family-focused prevention. Technological forecasting and social change, 74(4), 433-451.
Franklin, K. K., & Hart, J. K. (2007). Idea generation and exploration: benefits and limitations of the policy Delphi research method. Innovative Higher Education, 31(4), 237-246.
Gupta, U.G., & Clark, R.E. (1996). Theory and applications of the Delphi technique: A bibliography (1975-1994). Technological Forecasting and Social Change, 58(3), 185-211.
Hungerford, H., Peyton, R. B., & Wilke, R. J. (1980). Goals for curriculum development in environmental education. The Journal of Environmental Education, 11(3), 42-47.
Kloser, M. (2014). Identifying a core set of science teaching practices: A Delphi expert panel approach. Journal of Research in Science Teaching, 51(9), 1185-1217.
Linstone, H. & Turoff, M. (1975). The Delphi Method: Techniques and Applications. Reading, MA: Addison-Wesley Pub. Co.
Ludwig, B. (1997). Predicting the future: Have you considered using the Delphi methodology. Journal of Extension, 35(5), 1-4.
Malone, K., Harmon, A. H., Dyer, W. E., Maxwell, B. D., & Perillo, C. A. (2014). Development and evaluation of an introductory course in sustainable food and bioenergy systems. Journal of Agriculture, Food Systems, and Community Development, 4, 1-13.
McKenna, H. P. (1994). The Delphi technique: a worthwhile research approach for nursing?. Journal of Advanced Nursing, 19(6), 1221-1225.
Maclean R., Jagannathan S., Panth B. (2018) Overview. In: Education and Skills for Inclusive Growth, Green Jobs and the Greening of Economies in Asia. Technical and Vocational Education and Training: Issues, Concerns and Prospects, vol 27. Springer, Singapore
Marra, A., Antonelli, P., Dell’Anna, L., & Pozzi, C. (2015). A network analysis using metadata to investigate innovation in clean-tech – Implications for energy policy. Energy Policy, 86, 17–26. https://doi.org/10.1016/j.enpol.2015.06.025
Mathews, J. A. (2008). Energizing industrial development. Transnational Corporations, 17(3), 59-84.
McArthur, J. W., & Sachs, J. (2009). Needed: A new generation of problem solvers. Chronicle of Higher Education, 55(40), 1-4.
McCormick, K., & Kautto, N. (2013). The bioeconomy in Europe: An overview. Sustainability, 5(6), 2589-2608.
Murry, J. W., & Hammons, J. O. (1995). Delphi: A versatile methodology for conducting qualitative research. The Review of Higher Education, 18(4), 423.
Okoli, C., & Pawlowski, S. D. (2004). The Delphi method as a research tool: an example, design considerations and applications. Information & Management, 42(1), 15-29.
Osborne, J., Collins, S., Ratcliffe, M., Millar, R., & Duschl, R. (2003). What “ideas‐about‐science” should be taught in school science? A Delphi study of the expert community. Journal of Research in Science Teaching, 40(7), 692-720.
Popp, J., Lakner, Z., Harangi-Rákos, M., & Fári, M. (2014). The effect of bioenergy expansion: food, energy, and environment. Renewable and Sustainable Energy Reviews, 32, 559-578.
Quendler, E., & Lamb, M. (2016). Learning as a lifelong process-meeting the challenges of the changing employability landscape: competences, skills and knowledge for sustainable development. International Journal of Continuing Engineering Education and Life Long Learning, 26(3), 273-293.
Rowe, G., & Wright, G. (1996). The impact of task characteristics on the performance of structured group forecasting techniques. International Journal of Forecasting, 12(1), 73-89.
Rowe, G., & Wright, G. (1999). The Delphi technique as a forecasting tool: issues and analysis. International Journal of Forecasting, 15(4), 353-375.
Rossouw, A., Hacker, M., & de Vries, M. J. (2011). Concepts and contexts in engineering and technology education: An international and interdisciplinary Delphi study. International Journal of Technology and Design Education, 21(4), 409-424.
Sagar, A. D., & Kartha, S. (2007). Bioenergy and sustainable development?. Annual Review Environ. Resources, 32, 131-167.
Scarlat, N., Dallemand, J. F., Monforti-Ferrario, F., & Nita, V. (2015). The role of biomass and bioenergy in a future bioeconomy: policies and facts. Environmental Development, 15, 3-34.
Smáradóttir, S. E. 2014. Future Opportunities for Bioeconomy in the West Nordic Countries. Retrieved 6/1/2015 from http://www.matis.is/media/matis/utgafa/Bioeconomy-in-the-West-Nordic-countries-37-14.pdf.
Suttles, S. A., Tyner, W. E., Shively, G., Sands, R. D., & Sohngen, B. (2014). Economic effects of bioenergy policy in the United States and Europe: A general equilibrium approach focusing on forest biomass. Renewable Energy, 69, 428-436.
Taylor, R., Govindarajalu, C., Levin, J., Meyer, A., & Ward, W. (Eds.). (2008). Financing energy efficiency: Lessons from Brazil, China. World Bank: India and Beyond. Washington, D.C.
Toohey, S. (1999). Designing courses for higher education. Buckingham: The Society for Research into Higher Education and Open University Press.
US Department of Energy (2017). Energy Literacy. Washington, DC: US Department of Energy.
White House. (2012). National Bioeconomy Blueprint. Washington, DC, The White House,
April.Images
Kimi Grzyb is a recent PhD graduate of the Environmental Sciences (Education Track) program at Oregon State University. She is interested in research pertaining to the development of a future bioeconomy workforce. 
Brian Hartman is an Assistant Professor of K-12 STEM education at Walla Walla University. He worked as an engineer for over 10 years prior to completing his PhD in Science and Engineering Education at Oregon State University. His research interests include energy education, K-12 engineering, and the nature of engineering. 
Katharine G. Field is a Professor of Microbiology at Oregon State University, and also directs an undergraduate interdisciplinary research-based major, BioResource Research. She directed the education part of the Advanced Hardwoods Biofuels Northwest consortium. 
