INSTRUCTIONAL DESIGN

overview
daniel f. oswald
charles m. reigeluth

anchored instruction
nancy j. vye

case-based reasoning
janet l. kolodner

direct instruction
m. david merrill

learning communities
katerine bielaczyc

learning through design
shelley goldman

pedagogical agents and tutors
natalie k. person
arthur c. graesser

problem-based learning
cindy e. hmelo-silver

OVERVIEW

Instructional-design theory provides guidance on how to help people learn (or develop) in different situations and under different conditions. This guidance includes what to teach and how to teach it. To do this, instructional-design theory must take into account both methods and situations. Just as a carpenter uses different tools for different situations, so do instructional design theories offer instructional designers and teachers different tools for facilitating learning in different situations.

Elements of Instructional-Design Theory

Elements of instructional-design theory include instructional outcomes, conditions, methods, and values. Instructional values are an individual's or group's philosophy or beliefs about instruction. Instructional design theories ought to inform possible users (teachers and instructional designers) of the values about learning and instruction with which the theory was constructed, for they are the values that users and students must hold in order for the theory to work well.

Instructional outcomes include both results that are intentional and those that are incidental. Outcomes include the instruction's effectiveness, efficiency, and appeal. Instructional outcomes should not be confused with learning outcomes. Instructional outcomes focus on the degree of success in attaining the desired learning outcomes (the effectiveness of instruction) but also include the efficiency and appeal of the instruction.

Instructional conditions are factors beyond the influence of the instructional designer that impact upon the effects of the methods of instruction. Conditions may include the nature of what is being learned (the content), the learner, the learning environment, and the instructional development constraints (e.g., time and money). Instructional-design theory, in attempting to provide guidance for people to help others learn, ought to state explicitly the conditions under which different methods should and should not be used.

Instructional methods are the "how to" for facilitating human learning. They are the elements of guidelines that inform designers and teachers what to do to help students learn. They can be very general, such as "provide opportunities for practice," or they can be broken down into much more detailed specifications, such as (for learning concept classification) presenting previously unencountered examples and nonexamples of the concept in random order and asking the learner to identify those that are examples of the concept.

Instructional methods are situational rather than universal. This means that there are values, desired instructional outcomes, and instructional conditions (collectively referred to as instructional situations) in any context that influence whether or not a given instructional method should be used. Hence, instructional-design theory should specify the values, outcomes, and conditions for which each method should be used. Also, instructional methods are probabilistic rather than deterministic. That is, their use can only increase the probability that the desired outcomes will be attained.

Differences

Instructional design theories differ most importantly by the methods they offer. But the methods differ because of differences in the outcomes, values, and conditions for which they are intended.

For example, regarding instructional outcomes, some theories may focus more on effectiveness of the instruction, while others may focus more on appeal or efficiency. Also, regarding learning outcomes, different instructional theories can promote very different kinds of learning: from memorization to deep understandings or higher-order thinking and self-regulatory skills; from cognitive goals to such affective goals as emotional and social development.

Instructional values may differ, and they lead one to select different goals and different methods to attain those goals. Traditional instruction systems design (ISD), "a systematic approach to the planning and development of a means to meet instructional needs and goals" (Briggs, p. xxi), specifies that goals should be selected based on an assessment of learners' needs. However, in 1999 Charles Reigeluth proposed that users also consider teachers' and learners' values about goals. Furthermore, designers have tried to rely on experimental research to determine which methods are best for any given situation. Reigeluth countered that users also consider teachers' and learners' values about methods. If a teacher does not value learner-centered methods, then forcing the teacher to use them is not likely to ensure success.

Instructional conditions may also differ across instructional design theories. First, the nature of what is to be learned (the content) may differ. For example, some theories, such as those of David Perkins and Chris Unger, focus on deep understandings, which are taught differently than skills. Second, the nature of the learner may differ, including prior knowledge, skills, understandings, motivation to learn, and learning strategies. Third, the nature of the learning environment may differ. And finally, different instructional design theories may be intended for different constraints on instructional development (time and money). In essence, different instructional theories use different methods to attain different outcomes under different conditions and based on different values.

Major Trends

Two major trends in the field of instructional-design theory are apparent: the increasing predominance of an information-age paradigm of theories and the broadening of the kinds of learning and human development addressed by instructional theorists.

Information-age paradigm. Scholars, such as Bernie Trilling and Paul Hood, are increasingly drawing attention to the need for an attainment-based, "learning-focused paradigm" of instruction to meet learners' new educational needs in the information age, compared to the time-based, "sorting-focused paradigm" of the industrial age. Reigeluth (1999) distinguishes between the industrial and information ages with certain "key markers" (see Table 1).

In the early twenty-first century there is a growing recognition that the current system of education is beginning to fail society, not in its ability to attain traditional goals, but in its ability to provide what is increasingly needed in the emerging information society. There has begun a societal transition in which the complexity of human activity systems is growing dramatically, and learning has become the "indispensable investment" according to the National Commission on Excellence in Education 1983 report, A Nation at Risk. This has important implications for both what should be taught and how it should be taught.

Broadening the scope of instructional theory.

Much of the work that has been done in relation to and with instructional-design theory has been focused on teaching and learning procedural tasks, which are performed by following sets of defined mental or physical steps that were predominant in the industrial age. However, educational and corporate settings increasingly require people to solve problems in ill-structured and complex domains–problems for which there is not a clear solution or just one way of doing things. These "heuristic" tasks entail the use of causal models and "rules of thumb," along with other kinds of typically tacit knowledge that require different methods of instruction. This heuristic knowledge because of its nature often takes years for experts to develop through trial and error, if at all. Therefore, it would be valuable for schools and corporations to be able to teach it well.

Several new methods and tools are designed to assist learners with real-world problem solving, including just-in-time instruction and electronic performance support systems (EPSSs). However, they do not provide the appropriate amount or types of support for learning this usually tacit heuristic knowledge. Only toward the close of the twentieth century have instructional deign theorists seriously attempted to address this complex type of learning. Promising work has been done in the area of problem-based learning by such theorists as John D. Bransford and colleagues, David Jonassen, Laurie Nelson, and Roger Schank.

Other current areas of promising instructional-design theory include collaborative learning, self-regulated learning, and such affective areas as emotional development and social development.

Peter Senge highlighted the importance of the "learning organization," which he defined as "an organization that is continually expanding its capacity to create its future" (p.14) through the use of five disciplines: systems thinking, personal mastery, mental models, building shared vision, and team learning. A challenge for instructional design theorists is to develop comprehensive theories that foster such organizational learning.

The preceding offers only a sampling of areas in which instructional-design theory is currently being developed. Due to the nature of human learning, there exist many more domains of instructional guidance that require greater study.

Controversial Issues

Three controversial or problematic issues are discussed below: (1) Should instructional design theories be "theoretically pure" or eclectic? (2) Are traditional research methods appropriate for advancing instructional design theories, or is a different paradigm of research needed? (3) Should instructional-design theories be strictly "local" in scope, or should they generalize across settings?

Eclecticism versus purism. Some scholars, such as Anne K. Bednar and colleagues, argue that an instructional-design theory should be "theoretically pure" in that it should follow a set of assumptions from a single theoretical perspective, such as constructivism or behaviorism. Others, such as Peggy Ertmer and Timothy Newby, believe that such is true for descriptive theories, but that design theories, with their goal orientation, should draw on all useful methods for accomplishing the stated goals. For example, a behaviorist perspective would offer the method of drill and practice to help learners remember important information, whereas a cognitive perspective would offer the use of mnemonics to relate the new information to meaningful information. Perhaps there are some situations where good mnemonics cannot be developed, in which case drill and practice would be suggested by a design theory. Is it unwise for a teacher to draw on both kinds of methods because they hail from different theoretical perspectives? This issue is particularly important because it greatly influences the nature of an instructional theory.

Traditional versus new research methods. Many scholars advocate experimental and/or descriptive case studies or other kinds of descriptive research to advance our knowledge about design theories. Other theorists, such as James Greeno and colleagues, advocate new forms of research, such as "design experiments" and "formative research" (Reigeluth and Frick). "Design experiments" is the term Greeno, Allan Collins, and Lauren Resnick have come to use to refer to educators collaborating to analyze and design changes in institutional practice. "Formative research" is a form of research developed by Charles Reigeluth and Theodore Frick that is meant to help improve instructional-design theory. In an analysis of this issue, Glenn Snelbecker argued in 1974 that descriptive theories in the field are evaluated by how truthfully they describe why learning occurs, whereas instructional theories are evaluated by how useful their methods are for attaining their stated goals. Given this very different orientation toward usefulness rather than truthfulness, Reigeluth (1999) has proposed that the major concern in research on design theory should be "preferability" (whether or not a given method is more useful than the alternatives), rather than "validity" (whether or not the description is truthful). He has also suggested that the

TABLE 1

focus for research on a design theory should be to improve it rather than to prove it, because most of our methods of instruction are not nearly as successful as we need them to be. There is also clearly a role for descriptive research on instructional design theories, however. It is occasionally helpful to compare one method with another for a given situation, and descriptions of what a highly effective teacher or computer program does can be helpful for improving an instructional theory.

Although most researchers recognize that different research methods are useful for different purposes, perhaps there has not been enough emphasis on research to improve the preferability of instructional-design theories.

Generalizable versus local knowledge. Some scholars argue that instructional design theories should be "local" in scope because every situation is unique and methods that work well in one situation may not work well in another. Others believe that the purpose of an instructional-design theory is to generalize across situations–that if it loses this quality, it has little usefulness. Given that the standard for a design theory is usefulness rather than truthfulness, the issue may boil down to whether a highly local theory is more useful than a highly generalizable theory, or even whether a design theory that is intermediate between local and general may be the most useful.

There is another consideration that may enlighten this issue. Design theories are made up of not only methods, but also situations (values, desired outcomes, and conditions) that serve as a basis for deciding when to use each method. If a design theory offers different methods for different situations, the theory is at once both local and generalizable. It recognizes the unique needs of each situation but also offers methods for a wide range of situations. In this manner perhaps the profession can transcend "either—or" thinking and be both local and global.

Further Directions

Instructional-design theory bridges the gap between descriptive theory and practice and offers powerful guidance for practitioners. It has the potential to spur tremendous improvements in practice, but it currently constitutes a minor percentage of scholarly efforts devoted to education. Partnering of researchers and practictioners to develop and improve more powerful instructional-design theories can provide valuable insights and improvements for more useful design theory to facilitate human learning and development.

See also: Cooperative and Collaborative Learning; Instructional Design, subentries on Anchored Instruction, Direct Instruction, Learning through Design, Problem-Based Learning.

bibliography

Bednar, Anne k.; Cunningham, Donald; Duffy,

Thomas. M.; and Perry, J. David. 1991. "Theory into Practice: How Do We Link?" In Instructional Technology: Past, Present, and Future, ed. Garry J. Anglin. Englewood, CO: Libraries Unlimited.

Bielaczyc, Katerine, and Collins, Allan. 1999. "Learning Communities in Classrooms: A Reconceptualization of Educational Practice." In Instructional-Design Theories and Models, ed. Charles M. Reigeluth. Mahwah, NJ: Erlbaum.

Briggs, Leslie, ed. 1977. Instructional Design: Principles and Applications. Englewood Cliffs, New Jersey: Educational Technology.

Corno, Lyn, and Randi, Judi. 1999. "A Design Theory for Classroom Instruction in Self-Regulated Learning?" In Instructional-Design Theories and Models, ed. Charles M. Reigeluth. Mahwah, NJ: Erlbaum.

Ertmer, Peggy A., and Newby, Timothy J. 1993.

"Behaviorism, Cognitivism, Constructivism: Comparing Critical Features from an Instructional Design Perspective." Performance Improvement Quarterly 6 (4):50–72.

Greeno, James g.; Collins, Allan; and Resnick, Lauren B. 1996. "Cognition and Learning." In Handbook of Educational Psychology, ed. David C. Berliner and Robert C. Calfee. New York: Macmillan.

Kemp, Jerrold e.; Morrison, Gary R.; and Ross, Steven M. 1998. Designing Effective Instruction, 2nd edition. Upper Saddle River, NJ: Merrill.

Lewis, Catherine; Watson, Marilyn; and Schaps, Eric. 1999. "Recapturing Education's Full Mission: Educating for Social, Ethical, and Intellectual Development." In Instructional-Design Theories and Models, ed. Charles M. Reigeluth. Mahwah, NJ: Erlbaum.

Merrill, M. David. 1999. "Instructional Transaction Theory: Instructional Design Based on Knowledge Objects." In Instructional-Design Theories and Models, ed. Charles M. Reigeluth. Mahwah, NJ: Erlbaum.

National Commission on Excellence in Education. 1983. A Nation at Risk: The Imperative for Educational Reform. Washington, DC: U.S. Government Printing Office.

Nelson, Laurie M. 1999. "Collaborative Problem Solving." In Instructional-Design Theories and Models, ed. Charles M. Reigeluth. Mahwah, NJ: Erlbaum.

Reigeluth, Charles M. 1999. "What Is Instructional-Design Theory and How Is It Changing?" In Instructional-Design Theories and Models, Vol. II: A New Paradigm of Instructional Theory, ed. Charles M. Reigeluth. Mahwah, NJ: Erlbaum.

Reigeluth, Charles M., and Garfinkle, Robert j. 1994. Systemic Change in Education. Englewood Cliffs, NJ: Educational Technology Publications.

Senge, Peter M. 1990. The Fifth Discipline: The Art and Practice of the Learning Organization. New York: Doubleday.

Snelbecker, Glenn E. 1974. Learning Theory, Instructional Theory, and Psychoeducational Design. New York: McGraw-Hill.

Stone-McCown, Karen, and McCormick, Ann H. 1999. "Self-Science: Emotional Intelligence for Children." In Instructional-Design Theories and Models, ed. Charles M. Reigeluth. Mahwah, NJ: Erlbaum.

Trilling, Bernie, and Hood, Paul. 1999. "Learning, Technology, and Education Reform in the Knowledge Age or 'We're Wired, Webbed, and Windowed, Now What?"' Educational Technology 39 (3):5–18.

Daniel F. Oswald

Charles M. Reigeluth

ANCHORED INSTRUCTION

Anchored instruction (AI) is an example of an approach to curriculum and instruction that provides opportunities for students to learn important content while attempting to understand and solve authentic problems that arise within particular disciplines. Other related approaches are case-based learning, which is used in law and business education, and problem-based learning, sometimes used in medical education. Another way of organizing instruction around problem solving is through project-based learning.

The Problem of Inert Knowledge

In 1929 the English philosopher Alfred Whitehead identified a major problem in schools, namely the problem of inert knowledge. Inert knowledge is knowledge than can be recalled when people are explicitly prompted to remember it, but is not spontaneously used to solve problems even though it is relevant. A major goal of AI is to create learning environments that overcome the inert knowledge problem.

Research suggests that the degree to which knowledge remains inert is strongly affected by the way the information was learned initially. One factor contributing to the problem of inert knowledge is that traditional instruction too often consists of learning isolated facts and procedures. As a consequence, students do not learn when or how to use what they have learned. The knowledge is not organized in memory with information on the conditions under which to apply it. In AI students are provided with opportunities to solve realistic problems–called anchors–that help them learn when and how to apply knowledge.

The Role of Prior Knowledge in Learning

Research indicates that learning is affected by the knowledge that people bring to the learning situation. Sometimes people's prior knowledge of a situation enables them to understand with little effort the meaning and significance of new information. More typically, especially in the case of young learners, prior knowledge of the situation is limited and the learner is unable to make sense of new information and has difficulty discriminating important from less important aspects of the information. When learners lack sufficient prior knowledge, information is treated as facts to be memorized. Anchored instruction was developed to compensate for learners' lack of experience and knowledge. Anchors consist of multimedia (e.g., video or audio with pictures) scenarios that are designed to improve learners' understanding of the problems to be solved.

Experience Being an Expert

Another major goal of AI is to help people learn the kinds of problems that experts in various areas encounter and to experience how experts identify, represent, and solve problems. The problems that experts encounter are more complex and open ended than the problems that students are asked to solve in school. Experts also assume greater autonomy than students in solving problems, including learning new skills and knowledge on an as-needed basis to solve problems. Anchors are designed to afford these kinds of experiences.

An Example of AI: The Adventures of Jasper Woodbury

Some of the original work on AI was conducted in the domain of middle school mathematics by the Cognition and Technology Group at Vanderbilt. These efforts culminated in a series called The Adventures of Jasper Woodbury. Jasper consists of twelve anchors (on videodisc or CD-ROM) that are designed for students in grades five and up. To promote transfer of learning, multiple related anchors are available to provide extra practice on core concepts and problem schemas. Three anchors relate to each of the following topics: statistics and business planning, trip planning, geometry, and algebra. Each anchor contains a short (about fifteen minutes) story on video, which ends in a complex challenge. The adventures are like good detective novels, where all the data necessary to solve the adventure (plus additional solution-irrelevant data) are embedded in the story.

In The Big Splash, one of the anchors related to statistics and business planning, the main character is a junior high school student named Chris. Chris's school is having a Fun Fair to raise money to buy a new camera for the school TV station. Chris wants to set up a dunking booth at the fair. Students would buy tickets for the opportunity to try to dunk their teachers in a pool of water. Chris needs to develop a business plan to get his dunking booth project approved by the school and to obtain a loan from the school principal. The plan must include an estimate of revenue and expenses for the dunking booth and must meet constraints set by the principal with respect to the maximum amount of the loan and the requisite profit. The video story shows Chris collecting information for his business plan.

Design principles. Jasper anchors were designed according to a set of principles. Each is video based. Research indicates that video helps students, especially poor readers, comprehend problems better. Video is also motivating to students. Anchor problems are presented in story form, instead of expository form. Stories are used because they are easy to remember.

Anchors are not simply traditional word problems on video–they are more representative of problems that an expert might solve. They are complex; more than one solution is possible and many steps (and hours) are required to solve them. Traditional story problems explicitly present the problem to be solved (there is usually only a single problem) and the relevant data. Anchors use a generative format. Each story ends with a challenge and students must generate the problems to be solved. For example, in The Big Splash many different business plans can be generated, based on the information presented. There are several options for filling the dunking booth with water and each option differs in terms of cost, risk, and the amount of time required. The challenge also involves some important statistical concepts. Chris conducts a survey to collect information on whether students at his school would be interested in dunking a teacher and how much they would pay to do so. Data from the survey can be used to extrapolate an estimate of revenue for the whole school.

All of the data needed to solve each challenge is contained in the story; students revisit the videos on an as-needed basis to look for and record data. Some of the Jasper anchors also contain ideas for how to solve parts of the challenge. These are called embedded teaching scenes. The embedded teaching scenes provide students with models for how to approach particular problems that may not be familiar to them.

Perspective on Pedagogy

Anchored instruction is consistent with a class of instructional theories known as constructivist theories. Constructivism rejects the idea that students learn by passively "soaking up" knowledge that is transmitted to them by teachers or others. Instead they assume students learn more if the teacher engages them in activities, such as defining problems, clarifying misunderstandings, generating solutions, and so forth, instead of lecturing or "telling" students how to solve problems.

Because of their complexity, anchors are effective for use in cooperative learning groups. Depending on the skill level of the class, teachers may structure the small-group work in different ways. If the class is quite skilled, the teacher may simply ask students to solve the challenge posed at the end of the video. For other classes, the teacher may ask groups to work and report on some part of the challenge or may focus the task even more by asking groups to brainstorm and report on ideas for how to solve a part of the challenge. An important aspect of AI pedagogy relates to how teachers mediate group problem solving. Because a goal of AI is for students to be as intellectually autonomous as possible in solving the anchors, it is important for teachers to interact with student in ways that do not usurp this autonomy.

Research on Anchored Instruction

One of the largest studies on AI involved a field implementation of four anchors from The Adventures of Jasper Woodbury. These anchors were used over the course of a school year by teachers in seventeen classes in seven states in the southeastern United States. In the majority of the classes, the Jasper instruction took the place of the students' regular mathematics instruction. Ten comparison classes that were matched on key demographic variables, including socioeconomic status, location, gender, minority representation, and mathematical achievement, were included in research. All students were administered a series of tests at the beginning and end of the school year. One test examined students' word-problem-solving skills. In spite of the fact that Jasper students had not received additional practice on written word problems, they performed significantly better than comparison students at the end of the school year. In this way, Jasper students were able to transfer the skills they had acquired in the context of solving Jasper problems to written word problems.

Students were also administered a series of tests designed to assess their abilities to define and formulate problems. They were given complex story problems in written form and were asked to identify goals that would need to be addressed to solve these problems. They were also shown mathematical formulations and were asked to identify the goal that each formula would satisfy. These aspects of problem solving are unique to the Jasper anchors and are not part of traditional problem-solving instruction. As expected, Jasper students performed better than comparison students on the posttest.

Finally, self-report measures of students' attitudes were collected. Jasper students showed more positive change relative to comparison students in five areas: they showed a reduction in mathematics anxiety, an increase in their beliefs about their ability to perform successfully in mathematics, greater interest in mathematics, greater interest in solving complex problems, and they thought mathematics was more useful in solving problems from everyday life.

The goal of anchored instruction is to help students learn information such that it can be remembered later on and flexibly applied to solve problems. Relevant research suggests that pedagogical approaches such as anchored instruction can enhance students' complex problem solving skills and positive attitudes towards learning.

See also: Instructional Design, subentries on Case-Based Reasoning, Learning through Design, Problem-based Learning.

bibliography

Barron, Brigid j. s.; Schwartz, Daniel l.; Vye, Nancy j.; Moore, Allison; Petrosino, Anthony; Zech, Linda; Bransford, John D.; and Cognition and Technology Group at Vanderbilt. 1998. "Doing with Understanding: Lessons from Research on Problem-and Project-Based Learning." The Journal of the Learning Sciences 7:271–311.

Barrows, Howard S. 1985. How to Design a Problem-Based Curriculum for the Preclinical Years. New York: Springer.

Bereiter, Carl. 1984. "Implications of Postmodernism for Science, or Science as Progressive Discourse." Educational Technology 29:3–12.

Bransford, John d.; Brown, Ann L.; and Cocking, Rodney R. 2000. How People Learn: Brain, Mind, Experience, and School. Washington, DC: National Academy Press.

Bransford, John. d.; Franks, Jeffery j.; Vye, Nancy J.; and Sherwood, Robert D. 1989. "New Approaches to Instruction: Because Wisdom Can't Be Told." In Similarity and Analogical Reasoning, ed. Stella Vosniadou and Andrew Ortony. New York: Cambridge University Press.

Cognition and Technology Group at Vanderbilt. 1990. "Anchored Instruction and Its Relationship to Situated Cognition." Educational Researcher 19:2–10.

Cognition and Technology Group at Vanderbilt. 1992. "The Jasper Experiment: An Exploration of Issues in Learning and Instructional Design." Education Technology Research and Development 40:65–80.

Cognition and Technology Group at Vanderbilt. 1997. The Jasper Project: Lessons in Curriculum, Instruction, Assessment, and Professional Development. Mahwah, NJ: Erlbaum.

Dewey, John. 1933. How We Think, A Restatement of the Relation of Reflective Thinking to the Educative Process. Boston: Heath.

Gick, Mary L., and Holyoak, Keith J. 1983. "Schema Induction and Analogical Transfer." Cognitive Psychology 15:1–38.

Paivio, Allan. 1971. Imagery and Verbal Processes. New York: Holt, Rinehart and Winston.

Sharp, Diana l.; Bransford, John d.; Goldman, Susan r.; Risko, Victoria j.; Kinzer, Charles K.; and Vye, Nancy J. 1995. "Dynamic Visual Support for Story Comprehension and Mental Model Building by Young, At-Risk Children." Educational Technology Research and Development 24:417–435.

Vygotsky, Lev S. 1978. Mind in Society: The Development of the Higher Psychological Processes. Cambridge, MA: Harvard University Press.

Whitehead, Alfred N. 1929. The Aims of Education. New York: Macmillan.

Williams, Susan M. 1992. "Putting Case-Based Instruction into Context: Examples from Legal and Medical Education." The Journal of the Learning Sciences 2:367–427.

Nancy J. Vye

CASE-BASED REASONING

Case-based reasoning (CBR) is a kind of analogical reasoning that focuses on reasoning based on previous experience. A previous experience can play several roles, such as:

• suggesting a solution to a new problem or a way of interpreting a situation,
• warning of a problem that will arise, or
• allowing the potential effects of a proposed solution to be predicted.

These are the types of inference necessary for addressing the kinds of ill-defined or complex problems that occur every day in the workplace, at school, and at home. One might, for example, create a new recipe by adapting one made previously. To understand why someone's boss reacted a certain way, one might remember a situation when his own boss reacted similarly. People might persuade themselves that a strategic plan will work based on the similarities between their company's situation and that of another company that is progressing in a similar way.

Cognitive Foundations

Case-based reasoning views analogical reasoning as the centerpiece of the ability to function as human beings. It posits that the most natural and powerful learning strategies are the automatic ones that situate learning in real-world experience. Previous experience and knowledge are naturally brought to bear in interpreting new situations, trying to explain when things are not as expected (based on the predictions made by previous experiences and knowledge), drawing conclusions based on explanations and on similarities between situations, and anticipating when some new thing just learned might be applicable. To do these things automatically there must be some internal processes and representations that allow a new experience to call up similar ones from memory.

Key to such reasoning is a memory that can access the right experiences (cases) at the times they are needed (the indexing problem). Case-based reasoning identifies two sets of procedures that allow such recognition to happen. First, at insertion (encoding) time, while engaging in an experience, a reasoner interprets the situation and identifies at least some of the lessons it can teach and when those lessons might most productively be applied. The case is labeled according to its applicability conditions, that is, the circumstances in which it ought to be retrieved. The most discriminating labels on a case will be derived by a reasoner who has taken the time and effort, and who has the background knowledge, to carefully analyze a case's potential applicability. Second, at retrieval time, while engaging in a new situation, a reasoner uses his or her current goals and understanding of the new situation as a probe into memory, looking for cases that are usefully similar to the new one. The extent to which a reasoner is willing or able to interpret the new situation determines the quality of the probe into memory. An uninterpreted situation is likely to yield poorer access to the contents of memory than is one that is more embellished. The more creative a reasoner is at interpreting a situation, the more likely he or she is to find relevant knowledge and experience to use in reasoning about it.

Learning, in the CBR paradigm, means extending one's knowledge by interpreting new experiences and incorporating them into memory, by reinterpreting and reindexing old experiences to make them more usable and accessible, and by abstracting out generalizations over a set of experiences. Interpreting an experience means creating an explanation that connects one's goals and actions with resulting outcomes. Such learning depends heavily on the reasoner's ability to create such explanations, suggesting that the ability and need to explain are key to promoting learning.

Case-based reasoning thus gives failure a central role in promoting learning because failure promotes a need to explain. When the reasoner's expectations fail, he or she is alerted that his or her knowledge or reasoning is deficient. When such failures happen in the context of attempting to achieve a personally meaningful goal, the reasoner wants to explain so that he or she can be more successful. Crucial to recognizing and interpreting failure is useful feedback from the world. A reasoner who is connected to the world will be able to evaluate his or her solutions with respect to what results from them, allowing in dexing that discriminates usability of old cases and allowing good judgments later about reuse.

Because one's first explanations might not be complete or accurate, iterative refinement is central to CBR. Explanations (and thus, knowledge) are revised and refined over time. People explain and index any experience the best they can at the time, and later on, when a similar situation comes up, remember and try to apply what was learned from the past experience. The ability to accurately explain develops over time through noticing similarities and differences across diverse situations, suggesting that a variety of experiences with a concept or skill–personal ones and vicarious ones–are necessary to learn it to its full complexity.

Implications for Promoting Learning

Case-based reasoning suggests five important facilitators for learning effectively from experience:

1. having the kinds of experiences that afford learning what needs to be learned;
2. interpreting those experiences so as to recognize what can be learned from them, to draw connections between their parts so as to transform them into useful cases, and to extract lessons that might be applied elsewhere;
3. anticipating the usefulness of those extracted lessons so as to be able to develop indexes for these cases that will allow their applicability to be recognized in the future;
4. applying what one is learning and experiencing failure of one's conceptions to work as expected, explaining those failures, and trying again (iteration); and
5. learning to use cases effectively to reason.

Case-based reasoning suggests that the easiest kinds of experiences to learn from are those that afford concrete, authentic, and timely feedback, so that learners have the opportunity to confront their conceptions and identify what they still need to learn. It also suggests that learners be given the opportunity to iteratively move toward increasingly better development of the skills and concepts they are learning so as to experience them in a range of situations and under a variety of conditions, and that, with each iteration, they have a chance to explain things that did not go exactly as expected and identify what else they need to learn. According to CBR, the iterative cycle of applying what is known, interpreting feedback, explaining results, and revising memory explains how expertise is developed and how an expert uses personal experiences and those of others to reason and learn.

Designing Instruction

Case-based reasoning makes two kinds of suggestions about designing instruction. First it suggests ways of orchestrating and sequencing classroom activities, including the roles teachers and peers can play in that orchestration and ways of integrating hands-on activities, software tools, and reflection. Second it suggests several kinds of software tools for scaffolding and enhancing reasoning and for promoting productive kinds of reflection.

Design of learning environments. Case-based reasoning suggests a style of education in which students learn by engaging in problem solving and other activities that motivate the need to learn and that give students a chance to apply what is being learned in ways that afford real feedback. In such an environment students might engage in solving a series of real-world problems (e.g., managing erosion, planning for a tunnel, designing locker organizers) requiring identification of issues that need resolution and knowledge that needs to be learned to address those issues; exploration or investigation or experimentation to learn the needed knowledge; application of that knowledge to solve the problem; and generation and assessment of a solution. Thinking about the problem they are trying to solve should help learners identify what they need to learn; they should have opportunities to learn those things; and they should get to apply what they are learning over and over again, with help along the way aimed at allowing them to successfully solve the problem and successfully learn the targeted knowledge and skills. Two approaches to the design of full learning environments have come from CBR. Roger C. Schank's group at Northwestern University's Institute for the Learning Sciences proposed the notion of a goal-based scenario as a fully automated learning environment. Janet L. Kolodner's group at Georgia Institute of Technology proposes the notion of their trademarked Learning by Design, a way of orchestrating a classroom for combined learning of content and important skills.

Goal-based scenarios (GBS). A goal-based scenario is a learning environment that places students in a situation where they have to achieve some interesting goal that requires them to learn targeted knowledge and skills. In Advise the President, for example, students play the role of advisers to the president in dealing with a hostage situation in a foreign land, in the process learning about several hostage-taking events, which have happened in history, and also learning some foreign policy. In Sickle-Cell Counselor students advise couples about their risk of having children with sickle-cell anemia, in the process learning about genetics in the context of sickle-cell disease. Using Broadcast News students put together a news story, in the process learning both history and writing skills. Students learn about history or genetics or writing because they need to learn those things to successfully achieve the challenge set for them. The trick is to design challenges that both engage the students and focus them on whatever content and skills they should be learning.

The student engaged in a goal-based scenario is provided with a case library of videos of experts telling their stories, strategies, and perspectives, which might help them with their task. When they reach an impasse in achieving their goal, they ask a question of the case library, and an appropriate video is retrieved and shown. Sometimes a story will suggest a topic they should learn more about or a skill they need to learn; other times it will tell how that expert dealt with some difficult issue the student is addressing. Based on suggestions made by the case library students move forward with their task–choosing a policy to recommend to the president, choosing a blood test, making recommendations to couples about whether or not they should have children, or deciding how to refer to a leader. The software takes on an additional role to clearly inform students when they have failed at their task. The case library can be consulted again, this time to help with explaining and recovering from a failure.

Learning by Design. Learning by Design is a project-based inquiry approach to middle school science, which uses design challenges as compelling contexts for learning science concepts and skills. Design challenges provide opportunities for engaging in and learning complex cognitive, social, practical, and communication skills. For example students design miniature vehicles and their propulsion systems to learn about forces, motion, and Newton's laws; and ways of managing the erosion near a basketball court to learn about erosion and accretion, erosion management, and the relationship between people and the environment.

Learning by Design's curriculum units are centered on the design and construction of working devices or models that illustrate physical phenomena. Learning by Design's focus on design challenges comes from CBR's suggestion that learning requires impasses and expectation failures. Designing, building, and testing working devices provides the kinds of failure experiences and feedback that promote good learning as well as opportunities for trying again to achieve the challenge based on what's been newly learned.

Case-based learning purports that learning from experience requires reflecting on one's experiences in ways that will allow learners to derive well-articulated cases from their experiences and insert them well into their own memories. Learning by Design includes integration of classroom "rituals" that promote such reflection. "Poster sessions" provide a venue for reporting on and discussing investigative results and procedures. "Pin-up sessions" give small groups the opportunity to share their plans with the whole class and hear other students' ideas. "Gallery walks" provide a venue for presenting one's designs in progress to the rest of the class and explaining why one's device behaves the way it does. Each provides opportunities for students to publicly present the way they engaged in important science skills, to see how others have engaged in those skills, and to discuss the ins and outs of the skills being practiced. Preparing for presentations requires doing the kinds of reflection on their activities that CBR suggests will lead to lasting learning.

Successfully engaging in design and investigative activities and in reflecting on those activities in ways that lead to productive learning requires help, and in Learning by Design, that help is distributed among the teacher, peers, cases, software, and paper-and-pencil tools. Cases that are read as part of the investigation or during design planning help students identify what they need to learn more about and give them ideas for their designs. Paper-and-pencil design diary pages help them keep track of decisions they make and data they collect while designing and testing so that they will be able to remember and reconstruct their experiences. SMILE's (Supportive Multi-User Interactive Learning Environment) Design Discussions help students plan investigative activities, summarize investigative experiences, justify design decisions, and explain design experiences, and its Lessons Learned helps them reflect back on a full design experience (several weeks long) and articulate what they've learned. The tools act as resources to help students create cases for others to use, help students keep track of what they have been doing, and help students reflect on their experiences and turn them into cases in their own memories. Learning by Design's classroom rituals get students and teachers involved in sharing and discussing experiences, providing advice, and abstracting across the experiences of different groups in a class.

Design of instructional tools. Case-based reasoning suggests three types of software tools for promoting learning: (1) case libraries as a resource; (2) supports for reflection; and (3) realistic simulation and modeling environments.

Case libraries as a resource. The most common place where CBR has influenced learning tools is in the creation of case libraries. A case library offers the opportunity for students to learn from others' experiences. Case libraries as a resource can offer a variety of different kinds of information of value to learners:

• Advice in the form of stories
• Vicarious experience using a concept or skill
• The lay of the domain and guidance on what to focus on
• Strategies and procedures
• How to use cases

Archie–2, for example, provides cases for architecture students to use while designing. Its cases describe public buildings, focusing on libraries and courthouses. As students work on designing buildings they consult Archie periodically for advice. Another program, STABLE, is designed to help students learn the skills involved in doing object-oriented design and programming. It uses a webbased (hypermedia) collection of cases made from previous students' work. Many goal-based scenarios include case libraries at their cores.

The context in which case libraries are used is critical to their effectiveness. For cases to be a useful resource to students, the students must be engaged in an activity where their impasses might be answered by cases in the case library. If students are facing challenges that arise naturally in problem solving (e.g., "How do I model a situation like this?" or "What's a good starting point for this kind of problem?"), then a case library of relevant situations and problems can help them address those impasses.

Building case libraries can be as valuable educationally as using case libraries, sometimes even more valuable. One of the findings in using Archie–2 was that the graduate students who were building the case library seemed to be learning as much or more than the students who were using the case library in their design work. The activity of building a case library is frequently motivating for students because it is creating a public artifact whose purpose is to help future students. Cognitively the need to explain to others in a way that will allow them to understand requires reflecting on a situation, sorting out its complexities, making connections between its parts, and organizing what one has to say into coherent and memorable chunks.

Support for reflection. Case-based reasoning purports that the most productive reflection for deep and lasting learning includes connecting one's goals, plans, actions, and their outcomes to tell the fully interpreted story of an experience, and then extracting the lessons learned and making predictions about the circumstances when those lessons might be applicable in the future. Several software tools have been designed to help learners engage in such reflection, each asking learners to be authors of cases describing their experiences. Jennifer Turns's Reflective Learner helps undergraduate engineering students write "learning essays" about their design experiences. Its prompting asks students to (a) identify and describe a problem that they had encountered when undertaking the current phase of their design project; (b) describe their solution to the problem; (c) say what they had learned from the experience; and (d) anticipate the kinds of situations where a similar solution might be useful. Amnon Shabo's Javacap and its successor, Kolodner's and Kris Nagel's Storyboard Author, provide structuring and prompts to help middle school students summarize their project-based science experiences, extract from them what they have learned, and write them up as stories for publication in a permanently accessible case library for use by other students. The networked computer creates motivation for the students' reflection; students enhance their own learning as they write summaries, which can act as guides and supports to future students. Nagel and Kolodner's Design Discussions, mentioned earlier, provides prompting to help students write up the results of experiments they've done, ideas about achieving project challenges or solving problems they are working on, or what happened when they constructed and tested a design idea.

Realistic modeling and simulation. Case-based reasoning's model of learning puts emphasis on experiencing failure as a motivation for deep learning. Case-based reasoning thus suggests that learners should have opportunities to try out their conceptions, failing softly when their predictions fail, and getting timely and interpretable feedback that they can use to identify and explain their misconceptions. Thus, the ability to try out and see the results of one's conceptions is fundamental to any learning environment based on CBR's model. Sometimes one can construct artifacts and try them out. For example, in Learning by Design, students design, construct, and test miniature vehicles to learn about combining forces. But often processes have time scales, size, cost, or safety constraints that make authentic feedback impractical. In those situations, CBR suggests making available to students realistic modeling and simulation environments.

Evidence of Learning

There has not been a great deal of evaluation and assessment of case-based tools and learning environments, but indicators are positive. Teachers and trainers who use CBR-informed materials come back energized. Teachers feel that they are able to reach more of their students with this methodology. Both students at the top and those at the bottom seem to be drawn in more by these activities than they are in a normal aim-toward-the-middle classroom. Concepts and skills are being learned, teachers think, in ways that will encourage students to remember and reuse them. Students surprise the teachers with ideas they come up with and the connections they are able to draw.

Evaluation and assessment in Learning by Design classrooms shows that indeed students are learning, often better than students in a traditional classroom. Results indicate that students who participate in Learning by Design learn the science content as well as or better than students in more traditional science classes. More important, results show that Learning by Design students learn targeted science skills and communication, collaboration, project, and learning practices such that they can apply them in novel situations. Indeed, Learning by Design students in typical-achievement classes perform these skills and practices as well as or better than honors students who have not been exposed to Learning by Design, while Learning by Design honors students perform the targeted skills and practices almost like experts.

See also: Instructional Design, subentries on Anchored Instruction, Learning through Design, Problem-Based Learning.

bibliography

Bareiss, Ray, and Beckwith, Richard. 1993. "Advise the President: A Hypermedia System for Teaching Contemporary American History." Paper presented at the annual meeting of the American Educational Research Association, Atlanta, GA.

Bell, Benjamin; Bareiss, Ray; and Beckwith, Richard. 1994. "Sickle Cell Counselor: A Prototype Goal-Based Scenario for Instruction in a Museum Environment." Journal of the Learning Sciences 3:347–386.

Domeshek, Eric A., and Kolodner, Janet L. 1993. "Using the Points of Large Cases." Artificial Intelligence for Engineering Design, Analysis and Manufacturing (AIEDAM) 7 (2):87–96.

Ferguson, William; Bareiss, Ray; Birnbaum, Larry; and Osgood, Richard. 1992. "ASK Systems: An Approach to the Realization of Story-Based Teachers." Journal of the Learning Sciences 2 (1):95–134.

Guzdial, Mark, and Kehoe, Colleen. 1998. "Apprenticeship-Based Learning Environments: A Principled Approach to Providing Software-Realized Scaffolding through Hypermedia." Journal of Interactive Learning Research 9:289–336.

Hammond, Kristian J. 1989. Case-Based Planning. New York: Academic Press.

Hmelo, Cindy e.; Holton, Douglas L.; and Kolodner, Janet L. 2000. "Designing to Learn About Complex Systems." Journal of the Learning Sciences 9 (3):247–298.

Holyoak, Keith J. 1984. "Analogical Thinking and Human Intelligence." In Advances in the Psychology of Human Intelligence, Vol. 2, ed. R. J. Sternberg. Hillsdale, NJ: Erlbaum.

Kass, Alex, and Guralnick, D. 1991. "Environments for Incidental Learning: Taking Road Trips Instead of Memorizing State Capitals." In Proceedings of the International Conference on the Learning Sciences (ICLS) 1991. Evanston, IL: American Association for Computers in Education.

Kolodner, Janet L. 1993. Case-Based Reasoning. San Mateo, CA: Kaufmann.

Kolodner, Janet L. 1997. "Educational Implications of Analogy: A View from Case-Based Reasoning." American Psychologist 52:57–66.

Kolodner, Janet l.; Crismond, David; Gray, Jackie; Holbrook, Jennifer; and Puntambekar, Sadhana. 1998. "Learning by Design from Theory to Practice." In Proceedings of the International Conference on the Learning Sciences (ICLS) 1998. Charlottesville, VA: American Association for Computers in Education (AACE).

Kolodner, Janet L., and Guzdial, Mark. 1999. "Theory and Practice of Case-Based Learning Aids." In Theoretical Foundations of Learning Environments, ed. Daniel H. Jonassen and Susan M. Land. Mahwah, NJ: Erlbaum.

Kolodner, Janet l.; Hmelo, Cindy E.; and Narayanan, N. Hari. 1996. "Problem-Based Learning Meets Case-Based Reasoning." In Proceedings of the International Conference on the Learning Sciences (ICLS) 1996, ed. Daniel C. Edelson and Eric A. Domeshek. Charlottesville, VA: American Association for Computers in Education (AACE).

Kolodner, Janet L., and Nagel, Kris. 1999. "The Design Discussion Area: A Collaborative Learning Tool in Support of Learning from Problem-Solving and Design Activities." In Proceedings of Computer Support for Collaborative Learning (CSCL) 1999. Palo Alto, CA: Stanford University.

Kolodner, Janet L., and Simpson, Robert L. 1989.

"The MEDIATOR: Analysis of an Early Case-Based Reasoner." Cognitive Science 13:507–549. Nagel, Kris, and Kolodner, Janet L. 1999.

"SMILE: Supportive Multi-User Interactive Learning Environment. Poster Summary." In Proceedings of Computer Support for Collaborative Learning (CSCL) 1999. Palo Alto, CA: Stanford University.

Puntambekar, Sadhana, and Kolodner, Janet L. 1998. "The Design Diary: A Tool to Support Students in Learning Science by Design." In Proceedings of the International Conference of the Learning Sciences (ICLS) 1998. Charlottesville, VA: American Association for Computers in Education (AACE).

Redmond, Michael. 1992. "Learning by Observing and Understanding Expert Problem Solving." Ph.D. diss., College of Computing, Georgia Institute of Technology.

Riesbeck, Christopher K., and Schank, Roger C. 1989. Inside Case-Based Reasoning. Mahwah, NJ: Erlbaum.

Schank, Roger C. 1982. Dynamic Memory. New York: Cambridge University Press.

Schank, Roger C. 1999. Dynamic Memory Revisited. New York: Cambridge University Press.

Schank, Roger C., and Cleary, Chip. 1994. Engines for Education. Mahwah, NJ: Erlbaum.

Schank, Roger c.; Fano, Andrew; Bell, Benjamin; and Jona, Menachim. 1994. "The Design of Goal-Based Scenarios." Journal of the Learning Sciences 3:305–346.

Shabo, Amnon; Nagel, Kris; Guzdial, Mark; and Kolodner, Janet. 1997. "JavaCAP: A Collaborative Case Authoring Program on the WWW." In Proceedings of Computer Support for Collaborative Learning (CSCL) 1997. Toronto: University of Toronto/OISE.

Turns, Jennifer a.; Newstetter, Wendy; Allen, Janet K.; and Mistree, Farrokh. June 1997. "The Reflective Learner: Supporting the Writing of Learning Essays That Support the Learning of Engineering Design through Experience." In Proceedings of the 1997 American Society of Engineering Educators' Conference. Milwaukee, WI: American Society of Engineering Education.

Zimring, Craig m.; Do, Ellen; Domeshek, Eric.; and Kolodner, Janet L. 1995. "Supporting Case-Study Use in Design Education: A Computational Case-Based Design Aid for Architecture." In Computing in Engineering: Proceedings of the Second Congress, ed. Jafar P. Mohsen. New York: American Society of Civil Engineers.

Janet L. Kolodner

DIRECT INSTRUCTION

Instruction is an illusive term that is often used indiscriminately to describe any presentation of information. In this discussion instruction is limited to those situations that, in addition to providing relevant information, include the following characteristics: (1) a particular educational goal has been specified; (2) the information has been organized to facilitate the acquisition of the desired knowledge or skill; (3) appropriate practice with feedback has been provided; and (4) guidance is available to assist learners to acquire the desired knowledge or skill.

Learning is also a term that is often equated with instruction. Learning occurs in all situations, whether or not there was a deliberate attempt to promote acquisition of a particular goal. Instruction is limited to those situations where there is a deliberate attempt to promote learning of specified knowledge or skill.

Direct instruction is a subset of instructional situations in which there is some instructor or instructional agent that is not only providing information but also monitoring the instructional activities of the student and providing guidance and feedback as appropriate. Ruth Clark describes four instructional architectures: receptive, directive, guided discovery, and exploratory. Receptive instruction is typified by a lecture where information is provided, but there is no attempt to ensure learning by providing practice or guidance. Directive instruction, often called tutorial instruction, involves presenting segments of information followed by appropriate practice with feedback and guidance. This is the type of instruction that is the subject of this entry. Guided discovery provides students with problems to solve and engages them in microworlds or simulations of the real world where they can explore a variety of approaches. The amount of guidance provided varies widely in this type of instruction. Highly guided situations are very similar to direct instruction, whereas those with little or no guidance depend more on student discovery of the knowledge or skill being promoted. Exploratory architectures are typically unstructured. Learners are provided some problem to solve and given a rich library of resource material. Students must structure their own learning as they investigate various resources and attempt to solve the problem.

Types of Instructional Models

There are a wide variety of instructional models and theories available to guide the design of directive instructional products. Charles M. Reigeluth includes summaries of more than twenty such models. Robert Tennyson et al. and Sanne Dijkstra et al. include summaries of a number of international models of instruction. David Jonassen includes a number of articles summarizing research related to direct instruction. Perhaps the most widely used model for direct instruction is the 1985 work of Robert Gagné, elaborated and extended by David Merrill in 1994. A newer model of some importance is the 4C/ID model of J. J. G. van Merriënboer.

The author has examined the above and other sources to identify instructional principles prescribed by this body of theory and research to which all of these authors would agree. These instructional design models represent a wide range of philosophical orientation and each emphasizes different aspects of the instructional situation. Some of these models apply more directly to architectures other than direct instruction but have implications for direct instruction nevertheless.

Many of these instructional models suggest that the most effective learning situations are those that are problem-based and involve students in four distinct phases of learning: (1) activation of prior experience; (2) demonstration of skills; (3) application of skills; and (4) integration of these skills into real-world activities.

Principles for Direct Instruction

The various instructional theorists and researchers all seem to agree on the following underlying principles for direct instruction:

• 1. Learning is promoted when learners are engaged in solving real-world problems.
• a. Learning is promoted when learners are shown the task that they will be able to do or the problem they will be able to solve as a result of completing a module or course.
• b. Learning is promoted when learners are engaged at the problem or task level, not just the operation or action level.
• c. Learning is promoted when learners solve a progression of problems that are explicitly compared to one another.
• 2. Learning is promoted when relevant previous experience is activated.
• a. Learning is promoted when learners are directed to recall, relate, describe, or apply knowledge or skill from relevant past experience that can be used as a foundation for the new knowledge or skill.
• b. Learning is promoted when learners are provided relevant experience that can be used as a foundation for the new knowledge or skill.
• c. Learning is promoted when learners are provided or encouraged to recall a structure than can be used to organize the new knowledge.
• 3. Learning is promoted when the instruction demonstrates what is to be learned rather than merely telling information about what is to be learned.
• a. Learning is promoted when the demonstration is consistent with the learning goal: examples and nonexamples for concepts, demonstrations for procedures, visualizations for processes, and modeling for behavior.
• b. Learning is promoted when learners are provided appropriate learner guidance including some of the following: learners are directed to relevant information, multiple representations are used for the demonstrations, and multiple demonstrations are explicitly compared.
• c. Learning is promoted when media plays a relevant instructional role.
• 4. Learning is promoted when learners are required to apply their new knowledge or skill to solve problems.
• a. Learning is promoted when the application (practice) and the posttest are consistent with the stated or implied objectives: information-about practice is to recall or recognize information; parts-of practice is to locate, name, and/or describe each part; kinds-of practice is to identify new examples of each kind; how-to practice is to do the procedure; and what-happens practice is to predict a consequence of a process given conditions, or find faulted conditions given an unexpected consequence.
• b. Learning is promoted when learners are guided in their problem solving by appropriate feedback and coaching, including error detection and correction, and when this coaching is gradually withdrawn.
• c. Learning is promoted when learners are required to solve a sequence of varied problems.
• 5. Learning is promoted when learners are encouraged to integrate (transfer) the new knowledge or skill into their everyday life.
• a. Learning is promoted when learners are given an opportunity to publicly demonstrate their new knowledge or skill.
• b. Learning is promoted when learners can reflect on, discuss, and defend their new knowledge or skill.
• c. Learning is promoted when learners can create, invent, and explore new and personal ways to use their new knowledge or skills.

See also: Instructional Design, subentry on Overview.

bibliography

Andre, Thomas. 1997. "Selected Microinstructional Methods to Facilitate Knowledge Construction: Implications for Instructional Design." In Instructional Design: International Perspective: Theory, Research, and Models, Vol. 1, ed. Robert D. Tennyson, Franz Schott, Norbert Seel, and Sanne Dijkstra. Mahwah, NJ: Erlbaum.

Clark, Ruth. 1998. Building Expertise: Cognitive Methods for Training and Performance Development. Washington, DC: International Society for Performance Improvement.

Dijkstra, Sanne; Seel, Norbert; Schott, Franz; and Tennyson, Robert D. 1997. Instructional Design International Perspective, Vol. 2: Solving Instructional Design Problems. Mahwah, NJ: Erlbaum.

GagnÉ, Robert M. 1985. The Conditions of Learning and Theory of Instruction, 4th edition. New York: Holt, Rinehart and Winston.

Jonassen, David H. 1996. Handbook of Research for Educational Communications and Technology. New York: Macmillan.

McCarthy, Bernice. 1996. About Learning. Barrington, IL: Excell.

Merrill, M. David. 1994. Instructional Design Theory. Englewood Cliffs, NJ: Educational Technology.

Reigeluth, Charles M. 1999. Instructional Design Theories and Models: A New Paradigm of Instructional Theory, Vol. 2. Mahwah, NJ: Erlbaum.

Schwartz, Daniel; Lin, Xiaodong; Brophy, Sean; and Bransford, John D. 1999. "Toward the Development of Flexibly Adaptive Instructional Designs." In Instructional Design Theories and Models: A New Paradigm of Instructional Theory, Vol. 2. Mahwah, NJ: Erlbaum.

Tennyson, Robert d.; Schott, Franz; Seel, Norbert; and Dijkstra, Sanne. 1997. Instructional Design: International Perspective, Vol. 1: Theory, Research, and Models. Mahwah, NJ: Erlbaum.

van MerriËnboer, Jeroen J. G. 1997. Training Complex Cognitive Skills. Englewood Cliffs, NJ: Educational Technology.

M. David Merrill

LEARNING COMMUNITIES

At the end of the twentieth century in America there has developed a learning-communities approach to education. In a learning community the goal is to advance the collective knowledge and, in that way, to support the growth of individual knowledge. The defining quality of a learning community is the presence of a culture of learning in which everyone is involved in a collective effort of understanding.

There are four characteristics that such a culture must have: (1) diversity of expertise among its members, who are valued for their contributions and given support to develop; (2) a shared objective of continually advancing the collective knowledge and skills; (3) an emphasis on learning how to learn; and (4) mechanisms for sharing what is learned. It is not necessary that each member assimilate everything that the community knows, but each should know who within the community has relevant expertise to address any problem. This marks a departure from the traditional view of schooling, with its emphasis on individual knowledge and performance and the expectation that students will acquire the same body of knowledge at the same time. Classrooms organized as learning communities differ from most classrooms along a number of dimensions.

Learning Activities

Because the goals focus on fostering a culture of learning, the activities of learning communities must provide a means for (1) both individual development and collaborative construction of knowledge,(2) sharing knowledge and skills among members of the community, and (3) making learning processes visible and articulated. The learning activities described in a learning-communities approach and those found in most classrooms may share some similarities. For instance, methods such as cooperative learning can be used to support a learning community's goals, but they can equally well support more traditional learning aimed at inculcating particular knowledge among students.

Teacher Roles and Power Relationships

In a learning-communities approach the teacher takes on roles of organizing and facilitating student-directed activities, whereas in most classrooms the teacher tends to direct the activities. The power relationships shift as students become responsible for their own learning and the learning of others. Students also develop ways to assess their own progress and work with others to assess the community's progress. In contrast, in most classrooms the teacher is the authority, determining what is studied and assessing the quality of students' work.

Identity

As members of a learning community take on different roles and pursue individual interests toward common goals, students develop individual expertise and identities. In contrast, in most classrooms students work on the same things and are all expected to reach a base level of understanding. Students tend to form their identity through being measured or by measuring themselves against this base level. In a learning-communities approach there is also the notion of a community identity. By working toward common goals and developing a collective awareness of the expertise available among the members of the community, a sense of "who we are" develops.

Resources

Both a learning-communities approach and many traditional classrooms use resources outside of the classroom, including disciplinary experts, telementors, the Internet, and so forth. However, in learning communities both the content learned and the processes of learning from the outside resources are shared more among members of the community and become part of the collective understanding. A further distinction between learning communities and most classrooms is that in learning communities, both the members themselves and the collective knowledge and skills of the communities are viewed as important resources.

Discourse

In the learning-communities approach, the language for describing ideas and practices in the community emerges through interaction with different knowledge sources and through co-construction and negotiation among the members of the community. Also, learning communities develop a common language for more than just content knowledge and skills. The community develops ways to articulate learning processes, plans, goals, assumptions, and so forth. In contrast, in most classrooms the teacher and texts tend to promulgate the formal language to be learned.

Knowledge

In learning communities the development of both diverse individual expertise and collective knowledge is emphasized. In order for students to develop expertise, they must develop an in-depth understanding about the topics that they investigate. There is also a circular growth of knowledge, wherein discussion within the community about what individuals have learned leads individuals to seek further knowledge, which they then share with the community. In most classrooms the goals tend toward covering all the topics in the curriculum (breadth over depth) and teaching everyone the same thing.

Products

In a learning-communities approach, members work together to produce artifacts or performances that can be used by the community to further their understanding. There is sustained inquiry and development of products over months. In contrast, most classrooms tend toward individual or small group assignments with little sharing or collective products. Usually work is produced in short periods of time.

A key idea in the learning-communities approach is to advance the collective knowledge of the community, and in that way to help individual students learn. This is directly opposed to the approaches found in most schools, where learning is viewed as an individual pursuit and the goal is to transmit the textbook's and teacher's knowledge to students. The culture of schools often discourages sharing of knowledge by inhibiting students talking, working on problems or projects together, and sharing or discussing their ideas. Testing and grading are administered individually. When taking tests, students are prevented from relying on other resources, such as students, books, or computers. The whole approach is aimed at ensuring that students have all the knowledge in their heads that is included in the curriculum. Thus the learning-community approach is a radical departure from the theory of learning and knowledge underlying traditional schooling.

See also: Computer-Supported Collaborative Learning; Cooperative and Collaborative Learning; Peer Relations and Learning.

bibliography

Bielaczyc, Katerine, and Collins, Allan. 1999. "Learning Communities in Classrooms: A Reconceptualization of Educational Practice." In Instructional Design Theories and Models: A New Paradigm of Instructional Theory, ed. Charles M. Reigeluth. Mahwah, NJ: Erlbaum.

Brown, Ann, and Campione, Joseph. 1996. "Psychological Theory and the Design of Innovative Learning Environments: On Procedures, Principles, and Systems." In Innovations in Learning: New Environments for Education, ed. Leona Schauble and Robert Glaser. Mahwah, NJ: Erlbaum.

Scardamalia, Marlene, and Bereiter, Carl. 1994. "Computer Support for Knowledge-Building Communities." Journal of the Learning Sciences 3:265–283.

Katerine Bielaczyc

LEARNING THROUGH DESIGN

Design principles, perspectives, and processes in K–12 classroom practice became a topic of educational research during the final decade of the twentieth century. Research was preceded by design-based practices in the classrooms of many K–12 teachers, who engaged students in a variety of design projects and activities, such as proposing environmental strategies, creating plans for future communities, building bridges, designing machines, creating art and theater, and planning for community events. Classroom design projects thrived among teachers, who appreciated their value, but received little notice from researchers of learning. Design activities were the legacy of John Dewey and other progressive influences on schools and teachers, with roots in "experiential education," "learning by doing," and "project work." This entry is concerned exclusively with the nascent research field exploring the cognitive, metacognitive, and social contributions of design-based approaches to learning.

Research Background and Interests

The research community's interest in design-based approaches to curriculum overlap with several movements in education during the last two decades of the twentieth century. During the 1980s an education policy atmosphere developed that called for connecting what was taught and learned in school with the emerging demands of the work world. At the same time, cognitive and social research on learning evolved in new directions. Finally, research and development efforts with educational technologies emerged, bringing new concerns, tools, and methods to classroom research. In each of these arenas there was interest in problem solving, multidisciplinary approaches to content learning, social aspects of the learning process, applications of school content to the real world, and development of new tools. Because design embodies these qualities, it became a focus of research.

Policy. At the policy level, standards documents supported the importance of design-based learning experiences. The national science content standards included a strand titled "Science and Technology Standards," which called for students to engage actively in design work, from stating problems to designing solutions and evaluating them. The standards "emphasize abilities associated with the process of design and the fundamental understandings about the enterprise of science and its various linkages with technology" (National Research Council, p. 106). The Standards for Technological Literacy included design activities, stating that design is "as fundamental to technology as inquiry is to science and reading is to language arts" (International Technology Education Association, p. 90). Although not pointing directly to design, other standards documents from the National Council of Teachers of Mathematics supported problem solving, modeling, and connecting schoolwork to real applications. The trend to include design was international as well. In a 1996 multinational examination of science, mathematics, and technology curricula in thirteen countries, Paul Black and J. Myron Atkin identified movement toward design-oriented curricula that coincided with the desire of educators to have relevant, applied contexts as well as contexts for students to develop their practical knowledge and understandings. Although the policy call was clear, there was little account of existing school practices and no evidence of their effectiveness.

Design-based curricula. The most comprehensive examination of design-based learning in the United States, a 1997 report by Meredith Davis and colleagues under the umbrella of the National Endowment for the Arts, attempted to find and study design-based curricula to show what teachers and students do in design and the promise it holds for educational reform. The study used an "exploratory, hypothesis-generating approach" because design practices in classrooms were not widespread. Scores of design approaches were discovered and examined. The study's contribution was in providing descriptions of design practice in classrooms and identifying effective models and outcomes for using design. The study emphasized the ways design opens learning opportunities across curriculum subjects for teachers and learners and connects them to the world and a future beyond school.

Designing for Results

Changes took place in what was of interest to study as well as in the ways social and cognitive researchers approached and conducted research. Concerns about the learning goals of schools and their connections to students' futures as citizens and workers interested cognitive and social-learning researchers. Prior research results indicated that, under the right conditions, children were capable of complicated understandings and skills, and more researchers became interested in identifying and understanding learning in practice. This brought researchers out of laboratories and into the rough and tumble of classroom life. Researchers began to connect with teachers and reformers, expanding the goals, methods, and reach of their research. New models of research became established, such as the "design experiment" and "interactive research and design," both less aimed at researching design-based learning and more focused on researchers designing, implementing, studying, and tweaking classroom activities until they achieved sought-after results. Some researchers partnered with teachers to conduct classroom-based cognitive research and to develop tools and curriculum based on research findings that specifically incorporated design.

Middle-School Mathematics through Application Project (MMAP). Research and development of educational technologies became an additional and significant intersection point. Technology and design have a symbiotic relationship. Once technology tools for problem solving were in the hands of teachers and students, design-based activities became a possibility for widespread use. The Middle-School Mathematics through Application Project (MMAP) developed and researched an approach to middle school mathematics that integrated technology as a tool for mathematics learning in the context of design-based projects. Students were introduced to mathematics as they needed it, to solve pre-specified and constrained design problems, such as building a research station in Antarctica or recommending environmental population-control policies. Design contexts were treated as resources for mathematical interaction and explanation, and they led to increasing student engagement with mathematics topics. Design contexts made real-world connections of mathematics obvious to students and gave them situational resources for developing mathematical concepts. Design contexts provided opportunities for problem definition, problem solving, and performance-based assessments.

Jasper Project. The Jasper Project incorporated design-based approaches to upper-elementary mathematics learning with technology. The Jasper series focused on mathematical problem finding and solving, reasoning, communications, and making connections to other content areas for elementary students. The Jasper Project encouraged collaborative activity on extended problems over time, while offering deep understanding of mathematical concepts. Both MMAP and the Jasper Project emphasized the emergent aspects of problem solving and the utilization of various mathematical concepts and skills along the way to solving real-world problems. They both emphasized the roles technologies could play in supporting complex, design-based work in the mathematics curriculum and the supports needed for teacher professional development and assessment.

Yasmin Kafai and Mitchell Resnick bring together research from both school and informal technology, using settings to demonstrate how design activities with games, textile patterns, and robots empower children and connect them to important mathematics and science ideas. They examined subject-matter learning and provided compelling cases for how design processes provide meaningful and productive learning settings.

Learning by Design. Research independent of technology has concentrated on connections between design contexts and content learning, especially in mathematics and science. The Learning by Design project employed design approaches in science learning. The researchers based their work on prior results to create environments for learning science concepts and their applicability through a curriculum focused on "design and build challenges." They studied how, in the context of design work, students gained a conceptual understanding of complex systems and practices and created and field tested pedagogical rituals and processes for making design projects effective.

Modeling. With a concentration in both mathematics and science, work by Richard Lehrer and Leona Schauble investigated students' understanding of modeling, which is central to design work. They studied four schools where teachers were moving away from emphasizing facts and procedures to approaches focusing on constructing, evaluating, and reviewing models. They chronicled the development of models across the K–5 years and outlined key features in teacher professional development. In other analyses, they examined the development of mathematical inscription devices for representing and communicating about data in experiments and designs.

Geometry. Several studies of geometry understanding explored design contexts. In a 1998 study on understanding space, Lehrer, Cathy Jacobson, and colleagues examined how the geometric ideas of transformation and symmetry were developed through a quilt-design activity. They found that quilt design provided students with opportunities to explore ideas of symmetry and transformation and that informal knowledge of drawing and aesthetics played a mutual role in mathematical argumentation and notation. James Middleton and Robert Corbett examined the contexts of engineering and architecture to see if realistic situations helped students develop notions of physical structure that they could in turn connect to their understanding of geometry. Students made toothpick models of geometric solids, tested their strengths, then created suspension bridges and tested their stability. Results were mixed, with conceptions of geometric contributions to stability present but applied naively in designs.

Features of Design-Based Learning under Study by Researchers

A closer look at the features of design ideas and practices reveals what has made them appealing to teachers and researchers and indicates areas for future research. As a collaborative process, design requires discussion and clarification of goals, problems, subproblems, and actions. Researchers of learning have interests in problem exploration, the development of organizational skills and logic, and the ways they connect to, or enhance, learning of content and concepts.

Once problems and solution constraints are understood, design-based brainstorming encourages participation because it requires students to offer ideas and suggestions for solving problems in the context of a low-pressure forum. This engages students as stakeholders at the start of a complex set of activities. Brainstorming reinforces equity principles by providing for cultural knowledge to be relevant, brought to the table, and consequential for engagement and participation.

Success and inclusion are encouraged because multiple solutions to design problems are sought and appreciated. Variation is an expected outcome of the design experience. Variation in response to problems requires that students keep track of their decisions and provide rationales for them.

The features mentioned, including problem definition, brainstorming, open access to varied solution paths, and collaborative work, provide support for specific discourse processes in the classroom. Discussion, making and testing conjectures, rationales, argumentation, and explanation are necessary. Students must talk about possibilities, resources, and constraints. The more students discuss the problems they are trying to solve, the more they suggest and assess the viability of specific solutions while trafficking in the vocabulary and discourse practices of the discipline, then the more they are learning.

Design work and its tools require students to interact with multiple media and representations of information and data, and these are an increasing focus of research. Students may have to create constraint lists, flowcharts, data tables, graphs, and drawings as they use numbers and number sense, measurement, and both natural language and symbolic formulas while working from problem definition to solution. Researchers recognize the ability and ease with which technologies and media can put multiple representations of problems and processes on the student's desktop.

Research on design-based approaches creates opportunities for studying performance-based assessment practices. Iterative review processes are part of design work, leaving teachers and students with more opportunities for performance-based and peer-based assessment. Elaborating and agreeing on goals, evaluating processes, giving and receiving feedback, and prioritizing and making revisions are practices in design and worthwhile assessment practices as well.

In design, students' work with extended real-world problems connects them to practical professions, such as art, architecture, and engineering. Links are created between the real world and models, between simulations and solution processes, giving students access to practical knowledge and general processes of problem solving.

Finally, design-based learning has examined teacher practices. Evidence from MMAP suggests that the design process has the effect of decentering instruction from the teacher, and provides students with more explanation opportunities, more agency, and a more balanced position of power in relation to their work progress.

Barriers to widespread design-based approaches are also evident. Design-based learning is considered difficult to adopt and implement and remains only a specialized classroom practice. Design-based projects and activities are disruptively different from traditional approaches and perspectives. They impact on "business as usual" in the classroom in terms of management, planning, pedagogy, and content focus, disrupting classroom routines and requiring new kinds of work and attention. They are complex and take extended periods of time, making it necessary to reorganize classroom activity structures.

Design projects place new demands on teachers and students. Teachers need professional development to learn how to structure and manage design-based activities and many of the design-focused research projects also depend on teacher learning and preparation. Design work supports new levels of participation from students as well. Students often make strong connections to their design projects, taking extra time and often working at home. This is a positive effect, with implications for supporting partnerships between school, home, and community, but one that needs to be negotiated.

With collaborative work, emergent, complex problems, and real-world distractions, design projects are difficult to grade. Researchers are working to find ways to articulate standards, assessments, and complex learning environments, such as design projects.

Summary

Design-based approaches to learning have a longstanding place in the K–12 classroom, yet research attention to the role and effects of design-based learning experiences is in its infancy. Recent movements in educational reform, changes in the conduct of social and cognitive research on learning, and the growth of research and development in educational technologies have contributed to design-based learning becoming a topic of interest and inquiry. Current research reports findings from work in the social and cognitive sciences on problem solving, discourse and learning, teaching and learning processes, culture and learning processes, and classroom assessment.

See also: Instructional Design, subentries on Anchored Instruction, Case-Based Reasoning, Problem-based Learning.

bibliography

Barron, Brigid J. 2000. "Achieving Coordination in Collaborative Problem-Solving Groups." Journal of the Learning Sciences 9:403–436.

Barron, Brigid j.; Schwartz, Daniel l.; Vye, Nancy j.; Moore, a.; Petrosino, Anthony; Zech, Linda; Bransford, John D.; and Cognition and Technology Group at Vander-built. 1998. "Doing with Understanding: Lessons from Research on Problem-and Project-Based Learning." Journal of the Learning Sciences 7:271–312.

Berg, Rick, and Goldman, Shelley. 1996. "Why Design Activities Involve Middle Schoolers in Learning Mathematics." Paper presented at the American Educational Research Association Meeting, New York.

Black, Paul, and Atkin, J. Myron, eds. 1996. Changing the Subject: Innovations in Science, Mathematics and Technology Education. New York: Routledge.

Bottrill, Pauline. 1995. Designing and Learning in the Elementary School. Reston, VA: International Technology Education Association.

Bushey, Beverly. 1997. "Student Reflection in Emergent Mathematical Activity." Ph.D. diss., Stanford University.

Cognition and Technology Group at Vanderbilt. 1997. The Jasper Project: Lessons in Curriculum, Instruction, Assessment, and Professional Development. Mahwah, NJ: Erlbaum.

Cole, Karen; Coffey, Janet; and Goldman, Shelley. 1999. "Using Assessments to Improve Equity in Mathematics Education." Educational Leadership 56 (6):56–58.

Collins, Allan. 1990. Toward a Design Science of Education. New York: Bank Street College of Education.

Davis, Meredith. 1998. "Making a Case for Design-Based Learning." Arts Education Policy Review 100 (2):7–14.

Davis, Meredith; Hawley, Peter; McMullan, Bernard; and Spilka, Gertrude. 1997. Design as a Catalyst for Learning. Alexandria, VA: Association for Supervision and Curriculum Design.

Dewey, John. 1900. Education and Society. Chicago: University of Chicago.

Goldman, Shelley, and Greeno, James G. 1998. "Thinking Practices: Images of Thinking and Learning in Education." In Thinking Practices in Mathematics and Science Education, ed. James G. Greeno and Shelley Goldman. Hillsdale, NJ: Erlbaum.

Goldman, Shelley; Knudsen, Jennifer; and Latvala, Michelle. 1998. "Engaging Middle Schoolers in and through Real-World Mathematics." In Mathematics in the Middle, ed. Larry Leutzinger. Reston, VA: National Council of Teachers of Mathematics.

Goldman, Shelley, and Moschkovich, Judit. 1998. "Technology Environments for Middle School: Embedding Mathematical Activity in Design Projects." In Proceedings of the ICLS 98: International Conference of the Learning Sciences, ed. Amy Bruckman et al. Atlanta: Georgia Tech University.

Greeno, James g.; McDermott, Ray; Cole, Karen; Engle, Randi a.; Goldman, Shelley; Knudsen, Jennifer; Lauman, Beatrice; and Linde, Charlotte. 1999. "Research, Reform, and Aims in Education: Modes of Action in Search of Each Other." In Issues in Education Research: Problems and Possibilities, ed. Ellen Lagemann and Lee Shulman. San Francisco: Jossey-Bass.

Greeno, James G., and Middle-School Mathematics through Applications Project Group. 1997. "Theories and Practices of Thinking and Learning to Think." American Journalof Education 106:85–126.

Hmelo, Cindy; Holton, Doug; and Kolodner, Janet. 2000. "Designing to Learn about Complex Systems." The Journal of the Learning Sciences 9:247–298.

International Technology Education Association. 2000. Standards for Technological Literacy: Content for the Study of Technology. Reston, VA: International Technology Education Association.

Kafai, Yasmin, and Resnick, Mitchel, eds. 2000. Constructionism in Practice: Designing, Thinking, and Learning in a Digital World. Mahwah, NJ: Erlbaum.

Katz, Lillian G., and Chard, Silvia C. 2000. Engaging Children's Minds: The Project Approach. Stamford, CT: Ablex.

Kolodner, Janet; Gray, Jacquelyn T.; and Burks Fasse, Barbara. 2000. Promoting Transfer through Case-Based Reasoning: Rituals and Practices in Learning by Design Classrooms. Atlanta, GA: Georgia Institute of Technology.

Lehrer, Richard, and Chazan, Daniel, eds. 1998. Designing Learning Environments for Developing Understanding of Geometry and Space. Mahwah, NJ: Erlbaum.

Lehrer, Richard; Jacobson, Cathy; Thoyre, Greg; Kemeny, Vera; Strom, Dolores; Horvath, Jeffrey; Gance, Stephen; and Koehler, Matthew. 1998. "Developing Understanding of Geometry and Space in the Primary Grades." In Designing Learning Environments for Developing Understanding of Geometry and Space, ed. Richard Lehrer and Daniel Chazan. Mahwah, NJ: Erlbaum.

Lehrer, Richard, and Schauble, Leona. 2000. "Modeling in Mathematics and Science." In Advances in Instructional Psychology: Educational Design and Cognitive Science, Vol. 5, ed. Robert Glaser. Mahwah, NJ: Erlbaum.

Lehrer, Ricahrd; Schauble, Leona; Carpenter, S.; and Penner, D. 2000. "The Interrelated Development of Inscriptions and Conceptual Understanding." In Symbolizing and Communicating in Mathematics Classrooms: Perspectives in Discourse, Tools, and Instructional Design, ed. Paul Cobb, Eerna Yackel, and Kay McClain. Mahwah, NJ: Erlbaum.

Lehrer, Richard; Schauble, Leona.; and Petrosino, Anthony. 2001. "Reconsidering the Role of Experimentation in Science Education." In Designing for Science: Implications from Everyday, Classroom, and Professional Settings, ed. Kevin Crowley, Christian Schunn, and Takeshi Okada. Mahwah, NJ: Erlbaum.

Lichenstein, Gary; Weissglass, Julian; and Ercikan-Alper, Kadriye. 1998. Final Evaluation Report: Middle School Mathematics through Applications Project: MMAP II. Denver, CO: Quality Education Designs.

Middleton, James A., and Corbett, Robert. 1998. "Sixth-Grade Students' Conceptions of Stability in Engineering Contexts." In Designing Learning Environments for Developing Understanding of Geometry and Space, ed. Richard Lehrer and Daniel Chazan. Mahwah, NJ: Erlbaum.

National Council of Teachers of Mathematics. 1989. Curriculum and Evaluation Standards for School Mathematics. Reston, VA: National Council of Teachers of Mathematics.

National Council of Teachers of Mathematics. 2000. Principles and Standards for School Mathematics. Reston, VA: National Council of Teachers of Mathematics.

National Research Council. 1996. National Science Education Standards. Washington, DC: National Academy Press.

Owen-Jackson, Gwyneth. 2000. Learning to Teach Design and Technology in the Secondary School: A Companion to School Experience. New York: Routledge Falmer.

Perkins, David. 1997. Design as Knowledge. Hillsdale, NJ: Erlbaum.

Stevens, Reed. 2000. "Who Counts What as Math? Emergent and Assigned Mathematics Problems in a Project-Based Classroom." In Multiple Perspectives on Mathematics Teaching and Learning, ed. Jo Boaler. Westport, CT: Ablex.

Todd, Ronald. 1999. "Design and Technology Yields a New Paradigm for Elementary Schooling." Journal of Technology Studies 25 (2):26–33.

Watt, Daniel L. 1998. "Mapping the Classroom Using a CAD Program: Geometry as Applied Mathematics." In Designing Learning Environments for Understanding of Geometry and Space, ed. Ricahrd Lehrer and Daniel Chazan. Mahwah, NJ: Erlbaum.

Wolf, Dennie Palmer, and Baron, Joan Boykoff. 1996. Performance-Based Assessment: Challenges and Possibilities. Chicago: University of Chicago Press.

Shelley Goldman

PEDAGOGICAL AGENTS AND TUTORS

The creation of pedagogical agents is a fairly new enterprise that has emerged from previous work done in autonomous agents, intelligent tutoring systems, and educational theory. Pedagogical agents are autonomous agents that occupy computer learning environments and facilitate learning by interacting with students or other agents. Although intelligent tutoring systems have been around since the 1970s, pedagogical agents did not appear until the late 1980s. Pedagogical agents have been designed to produce a range of behaviors that include the ability to reason about multiple agents in simulated environments; act as a peer, colearner, or competitor; generate multiple, pedagogically appropriate strategies; and assist instructors and students in virtual worlds.

Animated Pedagogical Agents

A new breed of pedagogical agents has begun to appear in learning environments and on websites: animated pedagogical agents. The advent of animated pedagogical agents is the result of recent advancements in multimedia interfaces, text-to-speech software, and agent-generation technologies. Some of the more high-profile systems are described below.

• ALI is an automated laboratory instructor that monitors and guides undergraduates as they solve problems while interacting with chemistry simulations.
• ADELE (Agent for Distance Learning—Light Edition) helps students work through problem-solving exercises for courses that are delivered over the Internet. ADELE-based courses have been developed for continuing medical education and geriatric dentistry.
• Auto Tutor simulates the dialogue moves of human tutors while participating in conversations with students. Auto Tutor is currently designed to help college students learn about topics in computer literacy and conceptual physics.
• Cosmo exploits deictic behaviors to offer problem-solving advice to students learning about network routing mechanisms in the Internet Advisor learning environment.
• Herman the Bug inhabits the Design-A-Plant learning environment and helps children learn about botanical anatomy and physiology.
• PPP Persona provides online help instructions while helping users navigate through web-based materials.
• STEVE (Soar Training Expert for Virtual Environments) interacts with learners in an immersive virtual environment and has been used in naval training tasks such as operating engines on U.S. Navy surface ships.
• Vincent helps workers in shoemaking factories learn about production-line control time.

These agents exhibit lifelike behaviors and have the potential to bolster student-learning outcomes by exploiting both the auditory and visual channels of the learner. In general, animated pedagogical agents are lifelike personas, which execute behaviors that involve emotive responses, interactive communication, and effective pedagogy.

Emotive responses. Clark Elliott, Jeff Rickel, and James Lester argue in their 1999 article that animated agents displaying appropriate emotions provide a number of educational benefits to learners. First, agents that appear to care about students' progress may convince students to care about their own progress. Second, agents that are sensitive to learners' emotions (e.g., boredom or frustration) can provide feedback that prevents students from losing interest. Third, agents that convey enthusiasm for the subject matter are more likely to evoke the same enthusiasm in learners. Finally, agents that have rich and interesting personalities make learning more enjoyable for the learner.

Agents can display appropriate emotions through facial expressions, gestures, locomotion, and intonation variations. For example, Cosmo uses a recorded human voice and full-body emotive behaviors to express a wide range of pedagogically appropriate emotions. When a student experiences success in the Internet Advisor learning environment, Cosmo may applaud, point to relevant information on the screen, and provide positive feedback (e.g., "You chose the fastest subnet. Also, it has low traffic. Fabulous!"). Another system, Auto Tutor, synchronizes facial expressions and intonation variations to provide feedback that reflects the quality of students' natural language contributions. If a student provides a good answer to a question, Auto Tutor may respond simultaneously with an enthusiastic "Okay!" a fast head nod, and a smile. However, if the student's answer is only partially correct, Auto Tutor may respond with a less enthusiastic "Okay," a slower head nod, and no smile.

Interactive communication. Most educational websites and software packages are designed to be mere information delivery devices that occasionally employ unsophisticated reward systems as metrics of student understanding. Pedagogical agents, however, facilitate interaction in learning environments and force students to be active participants in the learning process. Agents and learners can collaboratively perform tasks, solve problems, and construct explanations. STEVE, the agent that teaches procedural knowledge involved in operating engines on navy ships, demonstrates for learners how to perform tasks and solve problems. A learner may choose to intervene and finish the demonstration. When this happens, STEVE monitors the learner's actions and a mixed-initiative demonstration occurs. Specifically, learners can take the initiative by asking questions or performing actions, or STEVE can mediate the interaction by providing hints, asking questions, giving feedback, or demonstrating a task. Learning sessions with ADELE are interactive in that ADELE interrupts students when "she" detects student errors and suggests alternative actions to be performed instead (e.g., "Before ordering a chest X ray, it would be helpful to listen to the condition of the lungs."). Students who reach impasses during problem solving may receive hints and ask "why" questions while interacting with ADELE and STEVE. In other systems, such as Auto Tutor, the agent and student have a conversation with each other. Throughout the conversation, Auto Tutor simulates human-tutor-dialogue moves (e.g., hints, prompts, assertions, and corrections), which allow the agent and student to jointly construct answers and explanations to deep-reasoning problems.

Effective pedagogy. In order to be considered value-added entities of learning environments, pedagogical agents must be effective teachers and, therefore, adaptive and dynamic in their teaching strategies. They must be able to adjust their teaching to fit a particular problem state or learning scenario, and they must be capable of adjusting their pedagogy to accommodate students' knowledge and ability levels. Pedagogical agents should be able to ask and answer questions, provide hints and explanations, monitor students' understanding, provide appropriate feedback, and keep track of what has been covered in the learning session. All of the pedagogical agents mentioned above are, to some extent, capable of each of these functions. Of course the litmus test for any pedagogical agent is whether it produces positive student-learning outcomes.

Learning Outcomes

It has been well documented that users prefer learning environments with animated agents over those that do not have agents. Specifically, participants assigned to learning conditions with animated agents (even ones that are not particularly expressive) perceive their learning experiences to be considerably more positive than participants assigned to learning conditions that do not include animated agents. This recurring finding is known as the persona effect. The persona effect is somewhat enigmatic in that it generally is not related to student outcome or performance measures. That is, most researchers who report evidence of the persona effect also report no differences between agent and no-agent conditions for retention and learning measures.

Several recent empirical studies, however, indicate that pedagogical agents do promote learning on both retention and transfer tasks. Robert Atkinson reported that students who received explanations from an animated agent about how to solve proportion word problems outperformed other learning conditions on both near and far transfer problems. In a study conducted by Roxana Moreno et al., college students and seventh graders attempted to learn about how to design plants that could survive in a number of different environments. One group of students interacted with a pedagogical agent, Herman the Bug, while another group of students received identical graphics and textual explanations but no pedagogical agent. The results indicated that students in the pedagogical agent condition outperformed students in the no-agent condition on transfer tests but not on retention tests. In another study, Natalie Person et al. (2001) reported that the effect size for Auto Tutor was .6 compared to the other learning conditions; human tutoring studies typically report effect sizes around .5 compared to other learning controls. Given the results of these learning-outcome studies and the fact that learners perceive their interactions with agents quite favorably, the future for pedagogical agents looks quite promising.

See also: Cooperative and Collaborative Learning; Peer Relations and Learning; Technology in Education, subentry on Trends.

bibliography

AndrÉ, Elisabeth; Rist, Thomas; and mÜller, Jochen. 1998. "Integrating Reactive and Scripted Behaviors in a Life-Like Presentation Agent." In Proceedings of the Second International Conference on Autonomous Agents. Minneapolis-St. Paul, MN: ACM Press.

Baylor, Amy L. 2001. "Investigating Multiple Pedagogical Perspectives through MIMIC (Multiple Intelligent Mentors Instructing Collaboratively)." In Artificial Intelligence in Education: AI-ED in the Wired and Wireless Future, ed. Johanna D. Moore, Carol L. Redfield, and W. Lewis Johnson. Amsterdam: IOS Press.

Carbonell, Jamie R. 1970. "AI in CAI: An Artificial Intelligence Approach to Computer-Assisted Instruction." IEEE Transactions on Man-Machine Systems 11:190–202.

Cassell, Justine; Pelachaud, Catherine; Badler, Norman; Steedman, Mark; Achorn, Brett; Becket, Tripp; Douville, Brett; Prevost, Scott; and Stone, Matthew. 1994. "Animated Conversation: Rule-Based Generation of Facial Expression, Gesture and Spoken Intonation for Multiple Conversational Agents." Computational Graphics 28:413–420.

Chan, Tak-Weil. 1996. "Learning Companion Systems, Social Learning Systems, and the Global Social Learning Club." Journal of Artificial Intelligence in Education 7:125–159.

Chan, Tak-Weil, and Baskin, Arthur B. 1990. "Learning Companion Systems." In Intelligent Tutoring Systems: At the Crossroads of Artificial Intelligence in Education, ed. Claude Frasson and Gilles Gauthier.

Dillenbourg, Pierre; Jermann, Patrick; Schneider, Daniel; Traum, David; and Buiu, Catalin. 1997. "The Design of MOO Agents: Implications from an Empirical CSCW Study." In Proceedings of Eighth World Conference on Artificial Intelligence in Education, ed. Ben du Boulay and Riichiro Mizoguchi. Amsterdam: IOS Press.

D'Souza, Aaron; Rickel, Jeff; Herreros, Bruno; and Johnson, W. Lewis. 2001. "An Automated Lab Instructor for Simulated Science Experiments." In Artificial Intelligence in Education: AI-ED in the Wired and Wireless Future, ed. Johanna D. Moore, Carol L. Redfield, and W. Lewis Johnson. Amsterdam: IOS Press.

Elliott, Clark; Rickel, Jeff; and Lester, James. Carl. 1999. "Lifelike Pedagogical Agents and Affective Computing: An Exploratory Synthesis." In Artificial Intelligence Today, ed. Michael Wooldridge and Manuela Veloso. Berlin: Springer-Verlag.

Frasson, Claude; Mangelle, Thierry; and Aimeur, Esma. 1997. "Using Pedagogical Agents in a Multi-Strategic Intelligent Tutoring System." In Proceedings of the AI-Ed '97 Workshop on Pedagogical Agents. Amsterdam: IOS Press.

Frasson, Claude; Mangelle, Thierry; Aimeur, Esma; and Gouarderes, Guy. 1996. "An Actor-Based Architecture for Intelligent Tutoring Systems." In Proceedings of the Third International Conference on Intelligent Tutoring Systems, LNCS. Berlin: Springer-Verlag.

Graesser, Arthur c.; Hu, Xiangen; Susarla, Suresh; Harter, Derek; Person, Natalie k.; Louwerse, Max; Olde, Brent; and Tutoring Research Group. 2001. "Auto Tutor: An Intelligent Tutor and Conversational Tutoring Scaffold." In Artificial Intelligence in Education: AI-ED in the Wired and Wireless Future, ed. Johanna D. Moore, Carol L. Redfield, and W. Lewis Johnson. Amsterdam: IOS Press.

Graesser, Arthur c.; Person, Natalie K.; Harter, Derek; and Tutoring Research Group. 2000. "Tactics in Tutoring in Auto Tutor." In ITS 2000 Proceedings of the Workshop on Modeling Human Teaching Tactics and Strategies. Montreal, Canada: Springer-Verlag.

Johnson, W. Lewis, and Rickel, Jeff. 1998. "STEVE: An Animated Pedagogical Agent for Procedural Training in Virtual Environments." SIGART Bulletin 8:16–21.

Johnson, W. Lewis; Rickel, Jeff; and Lester, James C. 2000. "Animated Pedagogical Agents: Face-to-Face Interaction in Interactive Learning Environments." International Journal of Artificial Intelligence in Education 11:47–78.

Lester, James c.; Converse, Sharolyn A.; Kahler, Susan e.; Barlow, S. Todd; Stone, Brian A.; and Bhogal, Ravinder S. 1997. "The Persona Effect: Affective Impact of Animated Pedagogical Agents." In Proceedings of CHI 1997. Atlanta, GA: ACM Press.

Lester, James c.; Voerman, Jennifer l.; Towns, Stuart G.; and Callaway, Charles B. 1999. "Deictic Believability: Coordinating Gesture, Locomotion, and Speech in Life-Like Pedagogical Agents." Applied Artificial Intelligence 13:383–414.

Loyall, A. Bryan, and Bates, Joseph. 1997. "Personality-Rich Believable Agents That Use Language." In Proceedings of the First International Conference on Autonomous Agents. Marina del Rey, CA: ACM.

Marsella, Stacy C., and Johnson, W. Lewis. 1997. "An Instructor's Assistant for Team-Teaching in Dynamic Multi-Agent Virtual Worlds." In Proceedings of the Fourth International Conference on Intelligent Tutoring Systems, LNCS. Berlin: Springer-Verlag.

Moreno, Roxana; Mayer, Richard e.; Spires, Hiller A.; and Lester, James C. 2001. "The Case for Social Agency in Computer-Based Teaching: Do Students Learn More Deeply When They Interact with Animated Pedagogical Agents?" Cognition and Instruction 19:177–213.

Paiva, Ana, and Machado, Isabel. 1998. "Vincent, an Autonomous Pedagogical Agent for on the-Job Training." In Intelligent Tutoring Systems, ed. Valerie Shute. Berlin: Springer-Verlag.

Person, Natalie k.; Craig, Scotty; Price, Penelope; Hu, Xiangen; Gholson, Barry; Graesser, Arthur C.; and Tutoring Research Group. 2000. "Incorporating Human-Like Conversational Behaviors into Auto Tutor." In Agents 2000 Proceedings of the Workshop on Achieving Human-like Behavior in the Interactive Animated Agents. Barcelona, Spain.

Person, Natalie k.; Graesser, Arthur C.; Kreuz, Roger j.; Pomeroy, Victoria; and Tutoring Research Group. 2001. "Simulating Human Tutor Dialog Moves in Auto Tutor." International Journal of Artificial Intelligence in Education 12:23–29.

Person, Natalie k.; Klettke, Bianca; Link, Kristen; Kreuz, Roger J.; and Tutoring Re-search Group. 1999. "The Integration of Affective Responses into Auto Tutor." In Proceedings of the International Workshop on Affect in Interactions. Siena, Italy: Springer-Verlag.

Rickel, Jeff, and Johnson, W. Lewis. 1999. "Animated Agents for Procedural Training in Virtual Reality: Perception, Cognition, and Motor Control." Applied Artificial Intelligence 13:343–382.

Shaw, Erin; Ganeshan, Rajaram; Johnson, W. Lewis; and Millar, Douglas. 1999. "Building a Case for Agent-Assisted Learning As a Catalyst for Curriculum Reform in Medical Education." In Proceedings of the International Conference on Artificial Intelligence in Education. Berlin: Springer-Verlag.

Shaw, Erin; Johnson, W. Lewis; and Ganeshan, Rajaram. 1999. "Pedagogical Agents on the Web." In Proceedings of the Third International Conference on Autonomous Agents. New York: ACM Press.

Sleeman, Derek, and Brown, John, eds. 1982. "Intelligent Tutoring Systems." New York: Academic Press.

Towns, Stuart g.; Callaway, Charles b.; Voerman, Jennifer L.; and Lester, James C. 1998. "Coherent Gesture, Locomotion, and Speech in Life-Like Pedagogical Agents." IUI '98: International Conference on Intelligent User Interfaces. New York: ACM Press.

Wenger, Etienne. 1987. Artificial Intelligence and Tutoring Systems: Computational and Cognitive Approaches to the Communication of Knowledge. Los Altos, CA: Morgan Kaufmann.

Natalie K. Person

Arthur C. Graesser

PROBLEM-BASED LEARNING

Problem-based learning (PBL) is one of a class of instructional methods that situates learning in complex contexts. In PBL, students learn through guided experience in solving complex, open-ended problems, such as medical diagnosis or designing a playground. Developed by Howard Barrows for use in medical schools, it has expanded to other settings such as teacher education and K–12 instruction.

Problem-based learning was designed with five goals: to help students (1) construct flexible knowledge; (2) develop effective problem-solving skills;(3) develop self-directed learning skills; (4) become effective collaborators; and (5) become motivated to learn.

With its emphasis on learning through problem solving and on making key aspects of expertise visible, PBL exemplifies the cognitive apprenticeship model. In this model, knowledge is constructed by learners working on real-world problems. One key characteristic that distinguishes PBL from other cognitive apprenticeship approaches is its potential for covering an entire integrated curriculum through a well-chosen set of problems. Concepts and thinking skills are used in a variety of problems. This redundancy affords learners the opportunity to construct a deep understanding by revisiting concepts from many perspectives and by experiencing a variety of situations in which skills are applied.

The Problem-Based Learning Tutorial Process

A PBL tutorial session begins by presenting a group, typically 5 to 7 students, with a small amount of information about a complex problem. From the outset, students question the facilitator to obtain additional problem information; they may also gather facts by doing experiments or other research. At several points, students pause to reflect on the data they have collected so far and generate questions about that data and ideas about solutions. Students identify concepts they need to learn more about to solve the problem (i.e., learning issues). After considering the case with their existing knowledge, students divide up and independently research the learning issues they identified. They then regroup to share what they learned, and reconsider their ideas. When completing the task, they reflect on the problem to consider the lessons learned, as well as how they performed as self-directed learners and collaborative problem solvers.

While working, students use white boards to help guide their problem solving. The white board is divided into four columns to help them record where they have been and where they are going. The columns help remind the learners of the problem-solving process. The white board serves as a focus for group deliberations. Figure 1 shows an example of white board entries made by engineering students working on a chemical release problem. The Facts column holds information that the students obtained from the problem statement. The Ideas column serves to keep track of their evolving hypotheses about solutions, such as reducing the storage of hazardous chemicals. The students place their questions for further study into the Learning Issues column. They use the Action Plan column to keep track of plans for resolving the problem or obtaining additional information.

The Role of the Problem

Cognitive research and experience with PBL suggest that to foster learning, good problems have several

FIGURE 1

characteristics. Problems need to be complex and open ended; they must be realistic and connect with the students' experiences. Good problems require multidisciplinary solutions and provide feedback that allows students to evaluate the effectiveness of their knowledge, reasoning, and learning strategies. Problems should promote conjecture and discussion and should motivate the students' need to go out and learn. As students generate and defend their ideas, they publicly articulate their current understanding, thus enhancing knowledge construction and setting the stage for future learning.

Each problem requires a final product or performance that allows the students to demonstrate their understanding. For example, PBL has been used to help middle school students learn life science by designing artificial lungs. They conducted experiments and used a variety of other resources to learn about breathing. Their final products were models of their designs.

The Role of the Facilitator

The term facilitator refers to someone trained to facilitate student learning through PBL. In PBL facilitators are expert learners, able to model good learning and thinking strategies, rather than being content experts. The facilitator is responsible for moving students through the various stages of PBL and for monitoring the group process–ensuring that all students are involved and encouraging them to externalize their own thinking and to comment on each other's thinking. The facilitator plays an important role in modeling the thinking skills needed when self-assessing reasoning and understanding. For example, the facilitator encourages students to explain and justify their thinking as they propose solutions to problems. Their questions help model the use of hypothetical-deductive reasoning as they encourage students to tie inquiry to their hypotheses. Facilitators progressively fade their scaffolding as students become more experienced with PBL, until their questioning role is largely adopted by the students. However, they continue to actively monitor the group, making moment-to-moment decisions about how to facilitate the PBL process.

Collaborative Learning in Problem-Based Learning

Collaborative problem-solving groups are a key feature of PBL. Its small group structure helps distribute the work among the members of the group, taking advantage of individual strengths by allowing the whole group to tackle problems that would normally be too difficult for any student alone. Students often become experts in particular topics. Small group discussions and debate enhance higher-order thinking and promote shared knowledge construction.

Reflection in Problem-Based Learning

Reflection on the relation between doing and learning is needed to help the learners understand that the tasks they are doing are in the service of the questions they have asked and that these questions arise from the learning goals they have set. Thus, each task is not an end in itself but a means to achieve a self-defined learning goal.

One potential danger of PBL is that knowledge may become bound to the problem in which it is learned. Learners need to understand what principles are at play in a given task and further understand how those principles might apply to new problems. To avoid this difficulty, learners must use concepts and thinking skills in multiple problems and to reflect on their learning. Reflection is important in helping students (1) relate their new knowledge to prior understanding; (2) mindfully abstract knowledge; and (3) understand how the strategies might be applied in new situations. Problem-based learning incorporates reflection throughout the tutorial process and when completing a problem. As students make inferences that tie the general concepts and skills to the specifics of the problem that they are working on, they construct more coherent understanding. The facilitator-guided reflection helps students prepare to take what Gavriel Salomon and David Perkins, in their 1989 study, call the "high road" to transfer as they consider how their new knowledge might be useful in the future and the effectiveness of their learning and problem-solving strategies.

Empirical Support for Problem-Based Learning

Research results are converging to show that some of these goals have been successfully met. Students in problem-based curricula are more likely to use their knowledge during problem solving and to transfer higher-order thinking skills to new situations. Cindy Hmelo has studied PBL in medical students and found that when asked to provide an explanation for a patient problem, the students in problem-based curricula were more accurate in their diagnoses, more likely to apply scientific concepts, and constructed better quality explanations than students in traditional curricula. This study provides evidence that PBL students transfer their knowledge and strategies to new problems. Shelagh Gallagher and William Stepien have studied the effects of PBL on gifted high school students. In one study, they examined the effect of a PBL intervention on content knowledge in social studies and found that students in PBL learned as much content as students in traditional instruction. In another study Gallagher, Stepien, and Hilary Rosenthal, comparing students taking a PBL science and society elective with students taking other classes, found that PBL students became better at problem-finding than comparison students. Most studies of PBL have been conducted either in medical schools or in other highly selected populations such as gifted high school students, but Hmelo, Douglas Holton, and Janet Kolodner conducted a 2000 preliminary study with middle school students learning life science. The students in the PBL intervention learned more than a comparison class, but because the students were not actually able to get feedback by implementing their solution, they did not achieve as deep an understanding as the investigators had expected.

Conclusion

Problem-based learning was designed to help students become flexible thinkers. Although the research on PBL is promising, the effects of PBL need to be examined more widely. The challenge ahead lies in understanding how the potential of PBL can be harnessed in diverse settings. Understanding the nature of the tutorial process, including the role of the problem and facilitator, collaboration among peers, and the importance of student reflection is necessary to successfully implement PBL and to prepare students to think in the world beyond school.

See also: Instructional Design, subentries on Anchored Instruction, Case-Based Learning, Learning through Design.

bibliography

Barrows, Howard S. 1985. How to Design a Problem-Based Curriculum for the Preclinical Years. New York: Springer.

Blumenfeld, Phyllis c.; Marx, Ronald w.; Soloway, Elliot; and Krajcik, Joseph S. 1996. "Learning with Peers: From Small Group Cooperation to Collaborative Communities." Educational Researcher 25 (8):37–40.

Chi, Micheline t. h.; Bassok, Miriam; Lewis, Matthew w.; Reimann, Peter; and Glaser, Robert. 1989. "Self-Explanations: How Students Study and Use Examples in Learning to Solve Problems." Cognitive Science 13:145–182.

Collins, Allan; Brown, John Seely; and Newman, Susan E. 1989. "Cognitive Apprenticeship: Teaching the Crafts of Reading, Writing, and Mathematics." In Knowing, Learning, and Instruction: Essays in Honor of Robert Glaser, ed. Lauren B. Resnick. Hillsdale, NJ: Erlbaum.

Gallagher, Shelagh A., and Stepien, William J. 1996. "Content Acquisition in Problem-Based Learning: Depth Versus Breadth in American Studies." Journal for the Education of the Gifted 19:257–275.

Gallagher, Shelagh a.; Stepien, William J.; and Rosenthal, Hilary. 1992. "The Effects of Problem-Based Learning on Problem Solving." Gifted Child Quarterly 36:195–200.

Hmelo, Cindy E. 1998. "Cognitive Consequences of PBL for the Early Development of Medical Expertise." Teaching and Learning in Medicine 10:92–100.

Hmelo, Cindy E. 1998. "Problem-Based Learning: Effects on the Early Acquisition of Cognitive Skill in Medicine." Journal of the Learning Sciences 7:173–208.

Hmelo, Cindy e.; Holton, Douglas; and Kolodner, Janet L. 2000. "Designing to Learn About Complex Systems." Journal of the Learning Sciences 9:247–298.

Hmelo, Cindy E., and Lin, Xiaodong. 2000. "Becoming Self-Directed Learners: Strategy Development in Problem-Based Learning." In Problem-Based Learning: A Research Perspective on Learning Interactions, ed. Dorothy H. Evensen and Cindy E. Hmelo. Mahwah, NJ: Erlbaum.

Kolodner, Janet l.; Hmelo, Cindy E.; and Narayanan, N. Hari. 1996. "Problem-Based Learning Meets Case-Based Reasoning." In Proceedings of the Second International Conference of the Learning Sciences, ed. Daniel C. Edelson and Eric A. Domeshek. Charlottesville, VA: Association for the Advancement of Computing Education.

Koschmann, Timothy d.; Myers, Ann c.; Feltovich, Paul J.; and Barrows, Howard S.1994. "Using Technology to Assist in Realizing Effective Learning and Instruction: A Principled Approach to the Use of Computers in Collaborative Learning." Journal of the Learning Sciences 3:225–262.

Salomon, Gavriel, and Perkins, David N. 1989. "Rocky Roads to Transfer: Rethinking Mechanisms of a Neglected Phenomenon." Educational Psychologist 24:113–142.

Torp, Linda, and Sage, Sara. 1998. Problems As Possibilities: Problem-Based Learning for K–12 Education. Alexandria, VA: Association for Supervision and Curriculum Development.

Williams, Susan m.; Bransford, John d.; Vye, Nancy j.; Goldman, Susan R.; and Carlson, Kristen. 1993. "Positive and Negative Effects of Specific Knowledge on Mathematical Problem Solving." Paper presented at the American Educational Research Association annual meeting, Atlanta, Georgia.

Cindy E. Hmelo-Silver