Abstract: Introductory urban planning courses provide an effective platform for delivering education for sustainable development (ESD) competencies. As general education courses, they constitute a unique niche for conveying sustainability concepts, theories, and applications to undergraduates. Learning outcomes include new skill sets, such as transdisciplinarity. One vexing question, however, is how to ascertain if ESD is actually being delivered. This paper suggests that instructors can answer this question by building an understanding of their classrooms, students, and objectives over time using simple techniques. We illustrate our course design considerations and our attempts to gain insight by inspecting class assignments and student survey data. We hope to engage readers in a conversation that develops an outcomes assessment paradigm which recognizes the oftentimes hard-to-quantify nature of affective learning outcomes inherent to ESD, and embraces the values of exploration, diversity, and emergence intrinsic to sustainability.
Keywords: planning pedagogy, assessment, Elaboration Theory, learning outcomes, education for sustainable development
Reza Banai describes a powerful intersection between urban planning pedagogy and sustainability (Banai, 2010 , 2013 ). He suggests that the many synergies between urban planning and sustainability make planning education an effective way to introduce sustainability concepts to undergraduate learners from various disciplinary backgrounds. Arizona State University’s (ASU) Sustainable Cities, an introductory undergraduate planning course, explores urban sustainability issues using an education for sustainable development (ESD) framework. These courses are relatively new and their presence raises questions regarding course design and assessment, i.e., do such courses deliver ESD? If yes, how? This paper hopes to open a discussion around introductory urban sustainability course construction by presenting an “exemplifying case” based on urban planning pedagogy (Bryman, 2010 ).
There are at least three major concerns with designing sustainability planning courses. The first is how to facilitate the personally transformative experience called for by ESD (Sterling & Thomas, 2006 ; Thomas, 2009 ). ESD elicits questions regarding the relationship between sustainability and the disciplinary subject matter. For example, is it an ESD course that interrogates urban planning topics, or is it an urban planning course that explores sustainability topics? Thomas (2009 ), Sterling (2001 ) and others regard the central consideration to be where sustainability is situated in the curriculum.
We agree with our colleagues and believe that because ESD is paradigmatic, sustainability precedes planning. Sustainability involves a problem-solving approach that incorporates disciplinary content and expresses itself in transdisciplinarity. Thus, it has implications for course design and learning objectives. How this is addressed is a matter for individual instructors to determine.
Second, assuming the course delivers a meaningful introduction to urban planning, how can we ensure the course also delivers the affective and cognitive ESD outcomes? What are the appropriate ESD objectives at the introductory level, and how might we measure them? Third, there is the question of what to teach and how to teach it in order to achieve the appropriate objectives. There are an incalculable number of potential lesson topics in urban sustainability, and a baffling array of potential pedagogic tools, methods, and frameworks in learning and instructional design theory. Which topics and methods work best together to support the learning outcomes, and in which order should they be delivered?
These questions and concerns shape our “introductory urban sustainability course design and assessment in ESD” study. A literature review informs course objectives by exploring learning outcomes in ESD linked to introductory urban planning. ASU’s Sustainable Cities course construction will be matched against Elaboration Theory as a framework for instructional design. The methods section utilizes statistical analysis and simple analytic techniques to explore student responses to a series of survey instruments on learning outcomes in an assessment of course design. Finally, interpretations of our analysis will be offered, followed by a discussion of potential implications.
Course Design – Elaboration Theory
This article focuses on Elaboration Theory (ET) as an instructional design for achieving learning outcomes (Reigeluth, 1999, pp. 433-434 ). Sustainable Cities utilizes ET fundamentals for its course design, such as focusing on concepts as opposed to procedures or theories. The course also makes frequent use of “epitomizing examples” that are later “unpacked,” uncovering their complexity (Reigeluth, 1992 ).
ET identifies two general directions for course design: macro and micro (Van Patten, Chao, & Reigeluth, 1986 ). These are defined by the scope and depth of course content running from the general (macro) to the specific (micro). Sustainable Cities favors the macro strategy for three reasons. First, introductory courses necessitate a broad survey of issues. Second, the course is large (e.g. over 400 seats) and is open to students from all academic majors. Third, macro strategies tend to establish long-term memorization appropriate for introductory courses (Van Patten et al., 1986 ), which supports advanced coursework in sustainability.
Exploring new concepts with a diversity of students seems best established by using exemplary problematics that are regularly accessed by most students, what Barr (2003 ) might call “situational factors.” In Phoenix, Az., water consumption, food deserts, and suburbanization are perfect examples of wicked problems that most ASU students are familiar with. These concrete issues tend to encourage individual reflection through Dole and Sinatra’s (1998 ) notion of “personal relevance” as opposed to abstractions, which may not (Schahn & Holzer, 1990 ). Once personal relevance is established and a level of interest produced, students can meaningfully investigate particulars and uncover barriers to resolution. By paying careful attention to critical and counter-intuitive interrelationships along the way (Pijawka, Yabes, Frederick, & White, 2013 ), the complexity of wicked problems are illustrated in a personally relevant manner that facilitates achieving outcomes in ESD.
ET’s notions of “sequencing and synthesizing” are also important for macro strategies and are central to the Sustainable Cities’ course structure. Sequencing “… refers to decisions about the order in which to present different topics of a subject,” while synthesizing represents “…ways of showing the interrelationships among these topics” (Reigeluth, Merrill, Wilson, & Spiller, 1980 ). The research discussed by Van Patten et al. (1986 ) suggests that urban planning content may not suffer from randomly sequencing “learning episodes,” whereas “perhaps some highly structured content, such as mathematics, might benefit more from sequence,” even to the level of ordering components within a single learning episode (Reigeluth, 1999, pp. 428-429 ).
Unlike mathematics and other natural sciences, social sciences have no “inherently dependent form.” However, complex affective ESD outcomes—especially critical reflection, ethical reasoning, and behavioural change—may require a more nuanced approach. While many instructional designs exist for cognitive outcomes, far fewer support affective outcomes. Only a handful take an integrative approach for both cognitive and affective (Reigeluth, 1999, p. 483 ), although these tend to be unwieldy. ET provides a framework with which to organize the sequence of instructional elements in a course. Flexible enough to provide structure for courses engaging both affective and cognitive outcomes in a variety of disciplines, its strength is its simplicity, enabling a wide range of instructors to incorporate it without radically altering their pedagogic approach.
Course Objectives – Education for Sustainable Development
Even a brief review of the literature reveals an extensive list of key ESD competencies (Brundiers & Wiek, 2011 ; de Haan, 2010 ; Frisk & Larson, 2011 ; Sterling & Thomas, 2006 ) that are difficult to quantify and assess (Buissink-Smith, Mann, & Shephard, 2011 ; Ramaswami et al., 2012 ). Furthermore, many conceptualize ESD learning outcomes as “graduate attributes” or “toolkits”—skills and competencies students develop over time and master by graduation (Borrego & Newswander, 2010 ; Sibbel, 2009 ; Wiek, Withycombe, & Redman, 2011 ). While consistent with undergraduate programs’ goals, it leaves open the question of a sustainability learning outcomes taxonomy where foundational outcomes are linked to introductory courses and advanced outcomes with upper-level coursework (Shephard, 2008 ).
Taxonomies suggesting distinct progressions of cognitive and affective competencies are well established in the fields of educational assessment and developmental psychology (Anderson, 2001 ; Bloom, Krathwohl, & Masia, 1969 , 1970 ; Flavell, 1979 ). However, disagreement on their interrelationship has limited research into combined learning outcomes assessment (Forgas, 2000, p. 3 ; Krathwohl, Bloom, & Masia, 1971, pp. 45-49 ; Martin & Briggs, 1986, pp. 9-13 ). Not surprisingly, theories suggesting progressions of ESD skills and competencies remain unexplored (Svanström, Lozano-García, & Rowe, 2008 ; Wiek et al., 2011 ).
Whether course designers opt for behavioural change or skills, the time commitments required for the list of key graduate competencies far exceeds the scope of most programs, much less a single class. Indeed, Pintrich, Marx, and Boyle (1993 ) hold that “[i]nstruction designed to foster conceptual change is likely to take place over larger units of time than more conventional didactic instruction….” Krathwohl et al. (1971 ) similarly argues that:
… complex [cognitive] abilities may be learned in a one-semester or one-year course, and the evidences of the learning may be seen in the examination … In contrast, interests, attitudes and personality characteristics are assumed to develop relatively slowly and to be visible in appraisal techniques only over long periods of time, perhaps even years. (Krathwohl et al., 1971, p. 19 )
Nevertheless, Krathwohl goes on to say, “… certain objectives in the cognitive and affective domain may be quickly learned or developed, whereas other objectives in both domains may be developed only over long periods of time” (Krathwohl et al., 1971, p. 20). Furthermore, in his taxonomy, lower-ordered skills are those most quickly adopted. Identifying how specific ESD outcomes interrelate and build on each other would seem to be central to attaining graduate competencies. While Barth et al. (2007 ) and others offer some heuristics for establishing affective and cognitive ESD competencies, literature regarding the role of the introductory course is limited.
Essential Outcomes and Assessments in Sustainable Cities
One of Sustainable Cities’ fundamental and stated goals is increased environmental conscientiousness, an affective outcome. What ties the various learning episodes together for advancing this affective outcome? We return to ET’s expansion on synthesizers. Explicating a synthesizer varies depending on the content and disciplinary matter. While some courses call for concept maps or charts outlining procedural synthesizers, for others, such as Sustainable Cities, an explicit prose structure is sufficient (Van Patten et al., 1986 ). For Sustainable Cities, the synthesizer or “conceptual thread” is intergenerational ethics, which is introduced to students as a vocabulary term.
To avoid framing urban sustainability merely as a technical exercise in the course, we sequence reflective learning episodes through a discourse in ethical understanding. This approach informs a personal definition of sustainability necessary for articulating an environmental conscientiousness. The general sequence of learning episodes is as follows:
Concept vocabulary. In its capacity as an urban planning course, Sustainable Cities acquaints students with a representative selection of the myriad physical and social systems sufficient for developing an understanding of the discipline, but not so broad as to become incoherent. It also introduces a similarly representative suite of evaluative and analytic tools for solving urban problematics, including but not limited to carbon footprints, environmental justice, triple-bottom line, full-cost accounting, and lifecycle analysis. The term “introduce” is purposeful as we are instructing for an awareness of, not a complete understanding of, the major tools, underlying principles, and concepts. ET suggests beginning with such macro concepts. Vocabulary as an early central goal is also supported by educational taxonomies (Bloom et al., 1969, pp. 63-65 ), and principles of ecological literacy (Orr, 1989, pp. 53-60 ; 2002, p. 137 ).
Interdisciplinary understanding. Next, the course conveys the centrality of interdisciplinarity for addressing sustainability issues in practice (Barth et al., 2007 ). The notion of specialists as members of an intellectual problem-solving community is illustrated early in the course (McKeown, Hopkins, Rizi, & Chrystalbridge, 2002 ). As many as 12 key guest lecturers, from across campus and the community, are introduced throughout the semester. Students have myriad opportunities to recognize sustainability’s interdisciplinary nature by analyzing linkages in guest lecturers’ topics as well as the ethical considerations associated with those topics.
Ethical conceptualization. In order to address complex problems facing society, we need citizens, technicians, and consumers with the capacity for ethical reasoning from a sustainability perspective. This requires course content devoted to such philosophical concepts as intergenerational equity, rights to the city, and land-use ethics, among others. Introductory planning courses are ideal for rehearsing the necessary dialogue regarding sustainability and ethics (Martin & Beatley, 1993 ; Warburton, 2003 ). Incorporating this dialogue in the course results in students who are better prepared for self-reflection on ethical positions when exploring a personal definition of sustainability.
Definition of sustainability. We need to move students beyond the assumption of sustainability as primarily an environmental concern. Only after interdisciplinarity and intergenerational ethics have been considered can a more critical examination of the various definitions of sustainability be undertaken (Wheeler & Beatley, 2004, pp. 460-461 ). This goal is fourth in the sequence for the very reason that the preceding three allow for a comprehensive and personally relevant definition of sustainability (sensu Shearman, 1990 ). This approach provides a meaningful basis for environmental conscientiousness.
Environmental conscientiousness. Finally, the intent of the course is to allow for the possibility of behavioural transformation. How this manifests in a particular student, if at all, is complex and depends on the intersection of their particular identity, the presentation of sustainability, and their appraisal of the importance of sustainability (Sinatra, 2005 ). Thus, Sustainable Cities final lectures explore multi-scalar interactions from the regional to the local to the individual, intending to “bring it home” to the student by tying their actions and dispositions to applied sustainable solutions.
Classroom assessment techniques have been shown to be of dubious value (Simpson-Beck, 2011 ). Our position is that assessments should follow the nature of the particular learning outcomes (LO) and recognize the context within which the LO is to be achieved (Shephard, 2008 ). Although each of the five sequences in the elaboration builds upon the previous one, they require separate approaches towards assessment. Therefore, Sustainable Cities employs two basic strategies.
First, we design class assignments to measure student comprehension and demonstrate course outcomes by integrating a reflective component. Second, student surveys provide sufficient information necessary to measure learning outcomes (LO) and evaluate as well as update course content (Accordino, 1991 ; Buissink-Smith et al., 2011 ). Typically, we distribute two or three surveys—one before the first class, another before the final examination, and one during midterms as needed. The surveys are presented to students as opportunities to take an active role in shaping the course and their education. Response rates are typically above 80%. Questions are updated regularly to improve construct validity. The first survey begins with an open-ended inquiry into students’ definition of sustainability to ensure that the survey content does not influence the description.
Instructors typically have limited time to commit towards outcomes assessment. Navigating the available assessments can be as time-consuming as employing techniques themselves. Additionally, for instructors who engage in self-reflection and use student feedback to inform course construction, issues that pique their interest are emergent and ongoing. New students and new social events bring new concerns, which bring new assessments. The point is that reflexive instructors build an understanding of the efficacy and subtleties of their course.
Instructors mostly gather discrete information that fill a gap generated from previous semesters and create ad hoc collections of assessments. For example, the particular iteration of LO assessments studied in this paper made use of an assignment that explores peoples’ rights of consumption to focus more closely on students’ positions regarding ethics. Therefore, in the interest of keeping the survey short, we did not include questions related to ethics. While this means the students’ ethical development during the course cannot be used as an independent predictor of behavioural change, we can still look at the effect of the course as a whole regarding behavioural change and gain a detailed understanding of students’ ethical positions.
Methods and Findings
Survey and Assignment Analysis
Examining the complete survey instrument is beyond the scope of this paper. We will instead describe only the elements used in assessing each of the five LO’s—Concept Vocabulary, Interdisciplinary Understanding, Ethical Conceptualization, Definition of Sustainability, and Environmental Conscientiousness. The core of the surveys resides in student perceptions of the 10 central lecture topics discussed in class, measured in three broad domains: students’ ‘knowledge of’, ‘interest in’, and ‘importance of’ each of the following:
- Personal values and lifestyle
- Ancient civilizations and collapse
- History of environmentalism
- Increasing urbanization
- Energy: production and consumption
- Climate change
- Environmental justice
- Architectural design
- Water consumption
Students quantify the topics in each domain on a Likert-type scale with seven levels—1 as the lowest value, 4 as average, and 7 as the highest. Students’ self-perceived level of environmental conscientiousness is rated along a similar scale. Student responses to the ten topics in each of the three domains are aggregated in both the before- and after-course surveys. The first survey scores are simply subtracted from the second to get a measure of change. Ten topics on a scale from 1 to 7 allows for a possible change score from -60 to 60 in each domain. A similar method is applied to the environmental conscientious score, giving a range of change from -6 to 6. The survey also queries students’ familiarity with key concepts in sustainability and the perceived benefit of course design elements for learning about sustainability. Statistical tests were generated using SPSS 21.0. Qualitative analyses of open-ended essay questions were hand-coded.
We wanted a measure of concept mastery independent of the exams. Therefore, the survey queried students on their understanding (see de Haan, 2010 ) of six concepts in introductory urban sustainability coursework—ecological footprint, food desert, food-miles, ecosystem assets, natural capital, and resilience. Concepts were selected based on their complexity and their coverage in readings and lectures (see Figure 1).
In order to identify the course’s effect on concept acquisition, we analyzed key student characteristics that predict mastery of sustainability terms and concepts. The number of terms or concepts a student identified as having mastery over constitutes a dependent variable, so a simple negative binomial regression method was used. Using course survey data, we created a model consisting of eight independent variables that suggest influence in predicting outcomes (concept understanding), removing variables from the model one at a time to find the most parsimonious model.
First, we asked if a positive change in any of the three domains predicts greater vocabulary acquisition. To capture this effect, we used the aggregated change score variables for each of the three domains: knowledge, interest, and importance. Next, we asked if student perceptions of the importance of sustainability had predictive value for concept mastery. Other predicative variables were a) how important they thought sustainability was in general, b) students’ environmental conscientiousness c) changes in students’ conception of the importance of sustainability, d) students’ prior coursework in sustainability and e) the initial number of vocabulary terms that the student had already mastered.
Other categories included “Academic Year,” which comprised a set of dummy variables with freshmen as the reference and assuming upperclassmen have greater abilities for concept mastery. “Academic Major” also comprised a set of dummy variables, including Art and Architecture, Hard Sciences, Social Sciences, Business, Urban Planning, and Sustainability, with “Undecided” and “Other” majors combined to form the reference group. These categories were developed using Chatman’s factor analysis of the 60,000 students who completed the University of California Undergraduate Experience Survey (2009 ). The full model consisting of all eight independent variables was statistically significant, χ2 (11, n=349) = 39.021, p <0.000, indicating that the model identifies characteristics which predict an increase in concept mastery (Table 1).
As illustrated in Table 1, four variables made noteworthy contributions to the model—academic year, academic major, change in interest in aggregated topics, and environmental conscientiousness. The strongest predictor was Academic Year with seniors having an 18.7% greater likelihood of mastering an additional concept over juniors, and sophomores having close to 9% probability for mastering an additional concept over freshmen. Juniors were not more likely than sophomores to master additional concepts.
Academic major also played a role, with Art and Architecture students having 15.7% greater odds than our reference variable (“undecided and other” academic majors) to master an additional concept. A student’s environmental conscientiousness had an effect with each additional recorded rank on the raw score equating to a 6% greater likelihood for mastering an additional term. Finally, each increment of change in the interest in topics marked nearly a half-per cent increase in the odds for mastering an additional term on an effective range of 98 levels.
How to measure the course’s impact for conveying the value of interdisciplinarity to students presented a problem. This is different from measuring a student’s skills in interdisciplinary thinking or teamwork (Jones, Selby, & Sterling, 2010, pp. 25-31 ). A literature search came up empty regarding a fitting survey instrument. Attempts to measure this construct in the before-course survey would therefore have no value. For that reason, we used an exploratory method to help us design a future instrument with good construct validity.
The course uses multiple, interdisciplinary instructional mechanisms. Primary instructors are drawn from ASU’s School of Geographical Sciences and Urban Planning, and the School of Sustainability. In-class guest lecturers come from diverse departments as well as community practitioners. These face-to-face mechanisms are supplemented with online lectures and documentaries. We queried students’ perceptions regarding the relative benefit from each mechanism for learning about sustainability. While this construct is not ideal, given the limited time and resources for assessment testing, we opted to first explore student responses.
In Table 2, we see significant changes in the perceived beneficence of different types of lecturing for delivering an ESD. The survey posited a question measured on a 7-point Likert-type scale, with one being “not beneficial at all,” and seven being “extremely beneficial.” Not surprisingly, the before-course survey revealed that students assumed course faculty lectures as being most essential for learning about sustainability.
For comparing before and after survey responses, a paired-samples t-test was used. There was a statistically significant decrease in the perceived benefits of faculty lectures from before the course (M = 6.17, SD = 1.02) to after the course (M = 5.92, SD = 1.23), t (348) = 2.75, p < .01 (two-tailed). The mean decrease for faculty lecture benefit was .24 with a 95% confidence interval ranging from .070 to .418. The eta-squared statistic (.02) indicated a negligible effect. There was also a statistically significant decrease in the benefits of online lecture from before the course (M = 4.40, SD = 1.76) to after the course (M = 4.11, SD = 1.80), t (348) = 2.75, p < .01 (two-tailed). The mean decrease in online lecture benefit was .29 with a 95% confidence interval ranging from .024 to .561. The eta-squared statistic (.01) indicated a negligible effect.
A Wilcoxon signed-rank test was also applied revealing a statistically significant decrease in the professed benefit of faculty lectures, z = -2.82, p < .05 with a small effect size (r = .11), and online lectures, z = 2.09, p < .05 with a small effect size (r = .08). The results support the earlier t-test results that students reported lower benefits from in-class faculty lectures and online lectures for learning about sustainability after the course. As with concept vocabulary, we asked which student characteristics influenced the perceived beneficence of delivery types. However, a Kruskal-Wallis test revealed no statistically significant differences in student preference before or after the course for any lecture type based on major or academic year.
Another insight can be gleaned from the results of a question on the after-course survey. Students were asked if taking Sustainable Cities encouraged them to consider adding a minor to their academic plan (see Figure 2). While almost half of students did not consider altering their plan, nearly 15% considered switching their major, and 25% considered adding a minor. An additional 15% would have switched or added a minor, but they were too far along in their current studies.
This data derives from a question on our Ecological Footprint essay assignment. The homework marked the conclusion of a discussion on human rights and material consumption that illustrated the differences in resource consumption and waste between different societies. Note that the students answered this question immediately after completing a comparative analysis of their personal carbon footprint personal inventory. Therefore, it provides an interesting window into students’ ethical landscapes. The last question for the essay asks students if people have a right to consume. Teaching assistants evaluated answers for content and uncovered five basic clusters. Over two-thirds (67%) of the students say “no” and most of them say this unequivocally (see Figure 3). Only 4% of the students said “yes” unequivocally. Just over a fourth of the total responded that people do have a right to consume as they please, but that such actions are unconstructive or harmful, and should not apply that right.
Definition of Sustainability
In this study, 349 students answered an open-ended question on what they supposed the definition of “sustainability” to be. The before-course survey responses contained 9,095 words, while the after-course survey word count dropped to 7,997. This shortening of their definitions equals a 12% drop in average words per response, from over 26 words to fewer than 23 words. In addition, the types of words students used to define sustainability changed dramatically (see Table 3).
The words and the roots of words reflecting an ecology-based definition (e.g. resource, natur-, enviro-) dropped considerably, while words and roots corresponding to social or temporal concepts (e.g. future, generation, social, socie-) increased substantially. The frequencies of change, consci-, action, and lifestyle also showed considerable increase, suggesting a definition based in personal responsibility. Other interesting shifts included a drop in words and roots reflecting design and production (e.g. energy, renew, produc-, material, tech-, and effici-), while roots suggesting social strategies and consumption (e.g. plan, econo-, social and preserve.) increased.
One ESD learning outcome is for students to become more environmentally conscientious. Surveying student conscientiousness is relatively straightforward. The survey prompts are in terms of thought and action (e.g. thinking about the environment, consuming in a particular manner, shopping in particular places, being active in environmental organizations). The before-course distribution of student responses had a slight rightward skew. After the course, it shifts significantly to the more environmentally aware (see Figure 4).
A Wilcoxon Signed Rank Test revealed a statistically significant increase in perceived environmental conscientiousness, z = -3.286, p < .01 from before the course to the course conclusion (see Table 4).
While urban sustainability content may be what it is regardless of whom is sitting in the class, the course structure and methods for delivering content can be both well thought out and reflected upon. Thus, instructors should consider a range of content that respects the disciplinary diversity and stages of learning of myriad students taking the course. Here we suggest aspects for consideration based upon the results of this study.
Course Structure and Impacts
Results suggest that student perceptions of the benefits of a lecture-type course are consistent across year and major. It is important to note that no subgroup of students is driving the results from the course overall regarding lecture preference. If instructors find reason to investigate this further, they can use the information from such survey responses to help shape a better survey instrument.
In addition, while there is no control group to measure against, the idea that a substantial number of students considered altering their academic program seems important. What is troubling is that nearly 15% of students would have altered their plan, but they are too far along towards graduation. This underscores the value of such courses being accessible early in the students’ academic careers. Of particular interest for further research would be which majors students would switch to and which minors would they add after such a course.
Although the Sustainable Cities course is an introductory one directed at lower-division students, it attracts large numbers of upper-division students especially as it is required for a minor degree. As the data show, it is no surprise that upperclassmen have an advantage in mastering concepts. Students’ interest in sustainability topics and their environmental conscientiousness are also easy to imagine having some effect on their ability to master concepts. Less intuitive, but not far-fetched, is the idea that some majors may also have an advantage. Architecture students, for example, may better grasp spatial concepts earlier. The central message of these findings is that instructors should know who is sitting in their classroom. This knowledge will help them build curriculum that stimulates seniors as well as challenges freshmen.
One reason why students initially rated faculty lectures relatively high as a mode of content delivery may be that they did not have a sense of the interdisciplinary nature of sustainability. For example, at the start of the course most students might not have realized that professors from different disciplines teach the class. They might have assumed that since sustainability is an academic program at ASU, a single departmental faculty would be able to adequately address sustainability problems. After the course, they realize that a sustainability department itself is interdisciplinary. Once students understood that approaching sustainable development issues necessitates the involvement of multiple disciplines, they attributed less importance to the faculty lecturer. This finding can be taken as a validation for the inclusion of guest lecturers and contributes to the assertion that the course conveys an understanding to the students that sustainable development is a topic that requires a holistic, interdisciplinary sensibility.
We need to be careful interpreting these results, as student expectations of the correct answer may have coloured their true opinions despite our efforts to encourage them to think freely. Nevertheless, the mere act of writing the arguments regarding limited consumption is an exercise worth undertaking. When we look at the small fraction of students (4%) who take an exclusively “might makes right” approach, we are satisfied to simply add questions to the surveys and follow changes in responses. After all, training teaching assistants to interpret essay results and the qualitative coding that comes with it is extremely time-consuming. For this LO, the survey approach is likely to be sufficient. Our experience serves as example that assessments should follow the emergent questions that instructors want to ask.
Definition of Sustainability
Our analysis of the survey data found a decrease in word count when defining “sustainability” between pre- and post-survey responses. The decrease suggests that students articulated their definition with more precision after taking the course. Admittedly, the decrease may also be an influence from the position in the survey mechanism and/or process. For instance, in the before-course survey, the first question asked students to define sustainability. This was intentional as the survey was designed so that the survey instrument itself did not influence the student’s definition. However, the same question was second to last in the after-course survey. While the survey was brief, answering an open-ended question near the end of the survey may have hastened their responses. In either case, these results show a broadening of student definitions of sustainability to include more nuanced, specific, or meaningful definitions. For example, the words future and generation overtook resource and enviro- for the second and third most commonly used words, indicating a shift away from environmental concerns specifically, and towards definitions with an intergenerational component. Additionally, phrases utilizing greater disciplinary linkages or interdisciplinarity were found as well. Such phrases as ‘socio-ecological’ were seen in the post-content coverage of ‘resilience thinking’ and ‘systems.’
The shift towards greater environmental conscientiousness suggests the course met its goal of fostering change agency. However, it is likely that a number of students initially over-estimated or under-estimated their environmental conscientiousness. Having discovered over the course of the semester that they were less environmentally conscientious as they had previously thought, they might have chosen to record a lower score on the after-course survey. With such a large class, we might conceive of these two possibilities as potentially balancing each other out. With smaller classes, instructors may want to create more precise assessment instruments, although these may be more time-consuming to develop and analyze.
Introductory planning courses in urban sustainability are newcomers to the academic world. At the 2013 Association for the Advancement of Sustainability in Higher Education (AASHE) conference in Nashville, a major topic of discussion among graduate students was the idea that nearly every instructor of urban sustainability currently teaching today was not formally instructed in that subject. With no history to reference, and scant literature to review on course construction, a frank discussion about how to teach these courses is overdue. Furthermore, ESD is much more than a new discipline. It opens the possibility of paradigm-shattering changes to teaching in higher education and even the very purpose of education in general. Nevertheless, instructors of introductory urban sustainability courses must begin somewhere. We have offered a modest overview of our attempt to apply theory to course construction and suggest undemanding yet effective tools to reflect on one simple question that every instructor should ask, “Does this course teach an education for sustainable development and how well does it do it?”
Of course, we are not totally in the dark. First, we have ample theories to draw upon from related disciplines. Elaboration Theory (ET) offers a flexible framework with which to structure a course. Instructors who are unable to spend a great deal of time uncovering the myriad approaches to course design may find ET’s adaptability useful. ET is responsive to the notion that courses may or may not be restricted to delivering a particular subset of skills from within a larger list of graduate competencies. While many course design theories seem abstract, we have found ET to be extremely approachable.
Second, while measuring affective learning outcomes is contested terrain, instructors have a wealth of quantitative and qualitative analytic tools at their disposal. Student surveys offer reflexive instructors the valuable information necessary to test their course content as relevant and timely. Furthermore, instructors can use surveys to keep apprised of subtle changes in the demographics of the course and to keep relevant in the face of a proliferation in K-12 education for sustainable development. A course that proactively responds to the changes around it offers students a tangible example of adaptability in action. Surveys do not have to be cumbersome or comprehensive. Instead, they can help instructors build an understanding of the interrelationships between course content and design, students, and learning outcomes over time. Analyses of the elements in the classroom can be problem-based. Questions about how and if the course achieves its objectives can drive the assessments. While measurements of affective learning outcomes are still elusive, planning instructors should not be dismissive of gathering what bits of information about their classes as they may for assembling an intricate understanding of their composition.
If conceived as a foundation upon which upper division courses and graduate outcomes are grounded, departments offering courses in urban sustainability should be able to discern the effectiveness of the introductory course for delivering ESD by the quality of students taking advanced coursework. However, when offered as general education courses, the introductory course must also be conceived as perhaps the only exposure to sustainability many students may encounter. In these cases, the course should be rich enough conceptually and broad enough topically to situate sustainability as a positive factor in the students’ overall academic and ethical development.
Accordino, J. (1991). Student Outcomes Assessment: Challenge and Opportunity For Planning Educators. Journal of Planning Education and Research Journal of Planning Education and Research, 10(3), 233-237.
Anderson, L. W., & Krathwohl, D. R. (2001). A taxonomy for learning, teaching, and assessing: a revision of Bloom’s taxonomy of educational objectives. New York: Longman.
Banai, R. (2010). A Note on Urban Sustainability-Education Nexus. Education, 2010.
Banai, R. (2013). Cities and Regions: The Urban Sustainability, Planning, Pedagogy, and Technology Nexus. Journal of Sustainability Education, 5.
Barr, S. (2003). Strategies for sustainability: citizens and responsible environmental behaviour. Area Area, 35(3), 227-240.
Barth, M., Godemann, J., Rieckmann, M., & Stoltenberg, U. (2007). Developing key competencies for sustainable development in higher education. International Journal of Sustainability in Higher Education, 8(4), 416-430.
Bloom, B. S., Krathwohl, D. R., & Masia, B. B. (1969). Taxonomy of educational objectives: the classification of educational goals; handbook. 1, 1. New York: McKay.
Bloom, B. S., Krathwohl, D. R., & Masia, B. B. (1970). Taxonomy of educational objectives: the classification of educational goals; handbook. 2, 2. New York: McKay.
Borrego, M., & Newswander, L. K. (2010). Definitions of interdisciplinary research: Toward graduate-level interdisciplinary learning outcomes. Rev. High. Educ. Review of Higher Education, 34(1), 61-84.
Brundiers, K., & Wiek, A. (2011). Educating Students in Real-world Sustainability Research: Vision and Implementation. Innovative Higher Education, 36(2), 107-124.
Bryman, A. (2010). Case Study Research w/ Alan Bryman Pt. 1-6. Education for Advancement, June 9 – June 12, 2010 [Lecture]: University of Leicester Regional Summer School, Ft. Lauderdale, Florida.
Buissink-Smith, N., Mann, S., & Shephard, K. (2011). How Do We Measure Affective Learning in Higher Education? Journal of Education for Sustainable Development, 5(1), 101-114.
Chatman, S. (2009). Recognizing and then Using Disciplinary Patterns of the Undergraduate Experience: Getting Past Institutional Standards. from http://www.escholarship.org/uc/item/2f53363n 
de Haan, G. (2010). The Development of ESD-Related Competencies in Supportive Institutional Frameworks. International Review of Education, 56(2-3), 315-328.
Dole, J. A., & Sinatra, G. M. (1998). Reconceptualizing Change in the Cognitive Construction of Knowledge. Educational Psychologist, 33(2/3), 109-128.
Flavell, J. H. (1979). Metacognition and cognitive monitoring. American psychologist, 34(10), 906-911.
Forgas, J. P. (2000). Feeling and thinking: the role of affect in social cognition. Cambridge; New York; Paris: Cambridge University Press; Editions de la Maison des Sciences de lHomme.
Frisk, E., & Larson, K. L. (2011). Educating for sustainability: Competencies & practices for transformative action. Journal of Sustainability Education, 2.
Jones, P., Selby, D., & Sterling, S. R. (2010). Sustainability education: Perspectives and practice across higher education: Earthscan.
Krathwohl, D., Bloom, B., & Masia, B. (1971). Taxonomy of educational objectives: the classification of educational goals: handbook II: affective domain. [S.l.]: Longman.
Martin, B., & Briggs, L. (1986). The affective and cognitive domains : integration for instruction and research. Englewood Cliffs, N.J.: Educational Technology Publications.
Martin, E., & Beatley, T. (1993). Our Relationship with the Earth: Environmental Ethics in Planning Education. Journal of Planning Education and Research Journal of Planning Education and Research, 12(2), 117-126.
McKeown, R., Hopkins, C. A., Rizi, R., & Chrystalbridge, M. (2002). Education for sustainable development toolkit. Knoxville, TN: Energy, Environment and Resources Center, University of Tennessee.
Orr, D. (1989). Ecological literacy. Conservation biology: the journal of the Society for Conservation Biology, 3(4), 334-335.
Orr, D. (2002). The nature of design: ecology, culture, and human intention. New York: Oxford University Press.
Pijawka, D., Yabes, R., Frederick, C. P., & White, P. (2013). Integration of sustainability in planning and design programs in higher education: evaluating learning outcomes. Journal of Urbanism: International Research on Placemaking and Urban Sustainability, 6(1), 24-36.
Pintrich, P. R., Marx, R. W., & Boyle, R. A. (1993). Beyond Cold Conceptual Change: The Role of Motivational Beliefs and Classroom Contextual Factors in the Process of Conceptual Change. Review of Educational Research, 63(2), 167-199.
Ramaswami, A., Weible, C., Main, D., Heikkila, T., Siddiki, S., Duvall, A., . . . Bernard, M. (2012). A Social-Ecological-Infrastructural Systems Framework for Interdisciplinary Study of Sustainable City Systems. Journal of Industrial Ecology, 16(6), 801-813. doi: 10.1111/j.1530-9290.2012.00566.x
Reigeluth, C. M. (1992). Commentary: Elaborating the Elaboration Theory. Educational Technology Research and Development, 40(3), 80-86. doi: 10.2307/30220015
Reigeluth, C. M. (1999). Instructional design theories and models: a new paradigm of instructional theory. Hillsdale, N.J.: Lawrence Erlbaum Associates.
Reigeluth, C., Merrill, M., Wilson, B., & Spiller, R. (1980). The elaboration theory of instruction: A model for sequencing and synthesizing instruction. Instructional Science, 9(3), 195-219. doi: 10.1007/BF00177327
Schahn, J., & Holzer, E. (1990). Studies of Individual Environmental Concern. Environment and Behavior, 22(6), 767-786.
Shearman, R. (1990). The meaning and ethics of sustainability. Environmental Management Environmental Management, 14(1), 1-8.
Shephard, K. (2008). Higher education for sustainability: seeking affective learning outcomes. International Journal of Sustainability in Higher Education, 9(1), 87-98.
Sibbel, A. (2009). Pathways towards sustainability through higher education. International Journal of Sustainability in Higher Education, 10(1), 68-82.
Simpson-Beck, V. (2011). Assessing classroom assessment techniques. Act. Learn. High. Educ. Active Learning in Higher Education, 12(2), 125-132.
Sinatra, G. M. (2005). The “Warming Trend” in Conceptual Change Research: The Legacy of Paul R. Pintrich. Educational Psychologist, 40(2), 107-115.
Sterling, S. R. (2001). Sustainable education: Green Books for the Schumacher Society.
Sterling, S., & Thomas, I. (2006). Education for sustainability: the role of capabilities in guiding university curricula. IJISD International Journal of Innovation and Sustainable Development, 1(4).
Svanström, M., Lozano-García, F. J., & Rowe, D. (2008). Learning outcomes for sustainable development in higher education. International Journal of Sustainability in Higher Education, 9(3), 339-351.
Thomas, I. (2009). Critical thinking, transformative learning, sustainable education, and problem-based learning in universities. Journal of Transformative Education, 7(3), 245-264.
Van Patten, J., Chao, C. I., & Reigeluth, C. M. (1986). A Review of Strategies for Sequencing and Synthesizing Instruction. Review of Educational Research, 56(4), 437-471. doi: 10.2307/1170341
Warburton, K. (2003). Deep learning and education for sustainability. International Journal of Sustainability in Higher Education, 4(1), 44-56.
Wheeler, S., & Beatley, T. (2004). The sustainable urban development reader. London; New York: Routledge.
Wiek, A., Withycombe, L., & Redman, C. L. (2011). Key competencies in sustainability: a reference framework for academic program development. Sustainability Science, 6(2), 203-218.