Augmented reality simulations on handheld computers

University of Wisconsin-Madison

Teacher Education, Massachusetts Institute of Technology

For correspondence, please contact Kurt Squire (kdsquire@education.wisc.edu), School of Education, UW Madison, Madison, WI.

RUNNING HEAD: AUGMENTED REALITY, ENVIRONMENTAL ENGINEERING

AUGMENTED REALITY SIMULATION GAMING ON HANDHELD COMPUTERS

Introduction

The use of computer simulations is changing the very nature of scientific investigation (Casti, 1998) and providing us unique insights into the way the world works (Wolfram, 2002). Scientists can now experiment in a virtual world of complex, dynamic systems in a way that was impossible just years ago. These tools have led to discoveries on topics ranging from the origins of planets to the spread of diseases through human populations. In an effort to engage students in the authentic making of science, many science educators (e.g. Feurzeig & Roberts, 1999) have begun using models and simulations in classrooms as well (c.f. Colella, Klopfer, & Resnick, 2001; Friedman, & diSessa, 1999; Stratford, Krajcik, & Soloway, 1998).To date most computer simulations have been tethered to the desktop, as they have relied on the processing power of desktop computers, but more ubiquitous and increasingly powerful portable devices make entirely new kinds of simulation experiences possible (Klopfer, Squire, Jenkins, & Holland, 2001).

Handheld computers’ portability, social interactivity, context sensitivity, connectivity, and individuality open new opportunities for creating participatory and augmented reality simulations where players play a part in a simulated system, coming to understand systems’ properties through social interactions (Colella, 1999). One possible genre of applications are augmented reality simulations, simulations where virtual data is connected to real world locations and contexts (Klopfer et al., 2001). In fields such as environmental science, where investigations are profoundly rooted in the particulars of local context, augmented reality applications invite science educators to bring the environment into the investigation process, but explore phenomena impossible to produce in the real world, such as toxic chemicals flowing through watersheds or diseases. In short, investigating how a toxin such as (Trichloroethylene or TCE) spreads through a watershed might be educationally valuable (particularly for environmental engineering students who might eventually conduct such investigations), but is obviously too dangerous to pursue. In this paper, we argue that augmented reality applications have promise in environmental science curricula because they allow curricula designers to design game trade-offs around core disciplinary dilemmas (Cobb et al., 2003), ill-structured dilemmas that are central to a field. This allows students to learn through failure, by intellectual play with robust disciplinary problems. Students’ failure combined with carefully crafted social organization allows students to explore difficult and complex tasks while building expertise in the field.

This study is a design-based research project (c.f. Brown, 1992) with the following foci: (1) investigating how learning occurs in augmented realities, (2) developing instructional models for using augmented realities in environmental science curricula, and (3) the affordances and challenges of using augmented reality applications in learning environments. Working with environmental science faculty at MIT, we developed augmented reality simulations of a carcinogenic toxin (TCE) flowing through an urban watershed. In a series of four case studies with approximately 75 students, we examine: (1) How the physical environment mediates students’ problem investigations; (2) How students integrate real and virtual data in problem-solving and conducting their scientific investigations; (3) How students construct the problem and how the contrived, constrained nature of the investigation is perceived as authentic or inauthentic (4) What instructional supports are useful in supporting learning? We argue that augmented reality simulations are powerful learning tools in situations when educators want the physical environment to be a part of students’ thinking and scientific reasoning.
Theoretical Approach: Situated Cognition

Over the past decades, a growing number of educational theorists and researchers in the learning sciences have argued for the importance of understanding cognition in context (e.g. Brown, Collins, & Diguid, 1989; Barab & Kirschner, 2001; Cognition and Technology Group at Vanderbilt, 1990; Greeno, 1998; Kirshner & Whitson, 1996). Whereas traditional cognitive models treat the workings of the mind as somewhat independent, a host of emerging, complementary approaches to cognition treat cognition and context as inextricably linked. How these different approaches construct the notion of context depends on their underlying theoretical framework. In this paper, we use an ecological approach to describe cognition, arguing for augmented realities as useful frameworks for thinking about digitally enhanced, immersive learning environments that couple the user with the affordances of the ecological environment.

Because learning is a process of creating meaning in situ, the environment plays an important role in the processes of knowing and learning; the environment constrains activity, affords particular types of activity or performance, and supports performance (Dewey, 1938; Peirce, 1868/1992; Salomon, 1993). From an ecological perspective, much of learning can be described as processes of becoming attuned to one’s environment, learning to read environmental affordances, which are interactions between the attributes of an object and an organism that determine how the object can be used (Gibson, 1979). Restated, if one adopts a distributed approach to cognition, then the physical environment becomes a critical component of cognition (c.f. Norman, 1988). Effective action is a coupling of the actor and the environment, and a major indictment of school is the way that traditional educational approaches cleave the learner from the physical world, processing information so that learners are handed “digested” information rather than having opportunities to learning the affordances of the environment (Barab et al., 1999). In the case of environmental science, this means being handed prepackaged research techniques (such as sampling strategies), or investigative design heuristics (such as investigations are social processes that involve managing budgets and constraints) without having opportunities to develop such understandings through action.

Greeno and others (e.g. Kirshner and Barab, 2001), describe a core challenge of cognitive science as how to account for the evolving nature of problem spaces; as actors participate in problem spaces, they constantly negotiate and redefine the components of the problem space, such as redefining adapting goals, adopting new strategies, and even reconceptualizing the problem itself. Learners’ intentions constrain activity in the system establishing boundary conditions in which the remainder of the activity exists (Barab et al. 1999). A situated lens should not confused with environmental determinism; rather, from the situated perspective, individuals’ intentions interact with the environment to produce activity. Intentions might be thought of as the subject’s goals and intended transformations on the object transformations. Thus, activity is considered as a coupling between the individual and his/ her environment (Barab, et al., 1999; Dewey, 1981). Kirshner and Barab (2001) describe this constant as a part of dynamic learning environments, in that the environment itself is expected to evolve in reciprocal relations with human activity; in other words, the researcher is challenged with capturing not only the learners’ performance in situ, but how the environment co-evolves with the participants.

From this situated perspective, the key unit of analysis is the learner operating within the environment. Effective action is always situated within environmental constraints and affordances, and a mark of expertise is one’s ability to “see” the environment in particular ways (Goodwin, 19**). If we are to take a situated view of environmental engineering, then a primary goal is to help students learn to “see” the environment as an environmental engineer might. We need to help students become attuned to the affordances of doing environmental science, particularly navigating complex problem spaces with multiple variables and solutions. Unfortunately, apprenticing students to experts is not always feasible. We argue that augmented reality simulations are one way to engage learners in complex investigations within a socially safe context.

Learning scientists have developed a number of pedagogical approaches predicated on situated learning theory, ranging from problem-based learning to case-based scenarios to anchored instruction. Barab and Duffy (1999) review a number of kindred approaches, distinguishing between instructional approaches designed around artificial problems and scenarios (practice fields) and those designed around social organization (communities of practice). Barab and Duffy describe eight features of effective practice field learning environments: Doing domain-related practices, ownership of the inquiry, coaching and modeling of thinking skills, opportunity for reflection, ill-structured dilemmas, support the learner rather than simplify the dilemma, collaborative and social work, motivating learning context. In a similar review, Jonassen (1999) uses a spatial metaphor for talking about constructivist learning environments, describing the environment in terms of a problem space, which is situated within and mediated by supporting tools and resources and social / cultural contexts. It is beyond the scope of this paper to compare these approaches to understanding the design of practice field, or problem-based learning environments. However, we do want to highlight the agreement across these approaches in the importance of facilitating student ownership over the problem space (helping students perceive problem authenticity) and the role that tools and resources play in mediating students’ relationship with the problem. Tools such as Model-It (Spitulnik, Studer, Funkel, Gustafson, & Soloway, 1995) or Climate Watcher (Edelson, Pea, & Gomez, 1996) mediate how students encounter dilemmas, collaborate in solving problems, and represent problem solutions (Solomon, 1993).

The affordances of handheld computers can be used to create unique situationalities for learners. Specifically, their (1) portability –can take the computer to different sites and move around within a site; (2) social interactivity – can exchange data and collaborate with other people face to face; (3) context sensitivity– can gather data unique to the current location, environment, and time, including both real and simulated data; (4) connectivity – can connect handhelds to data collection devices, other handhelds, and to a common network that creates a true shared environment; and (5) individuality – can provide unique scaffolding that is customized to the individual’s path of investigation. In the context of environmental science, handheld computers allow students to collect data in doing complex field investigations, access authentic tools and resources, and participate in collaborative learning practices while in the field. Whereas traditional desktop VR applications or 3D gaming technologies such as MUVEEs burden the computer with reproducing reality in 3D, augmented realities exploit the affordances of the real world, instead providing users layers of data that augment their experience of reality. As a result, simulations are untethered from the desktop and learners can participate in technology-enhanced investigations, location-based games, or participatory simulations. Because players are free to move throughout the world, novel opportunities exist for learners to interact with the physical environment, literally reading the landscape as they conduct environmental investigations or historical studies.

Not only can Pocket PCs provide users’ data, they can immerse players use interactive storytelling techniques multi media to create virtual worlds with compelling problems, and narrative contexts that might entice learners into engaging in complex practices and adopting the personae of scientists. Pocket PCs, which can display video, text, and host webs of information in intranets can create virtual worlds that go beyond just presenting data, but provide narrative context, similar to problem-based learning or anchored instruction environments. Leveraging design techniques from role playing games (c.f. Gee, 2003), we believe that opportunities exist for immersive gaming environments to recruit players into assuming new identities as environmental investigators, scientists, and environmental activists, thereby encouraging students to adopt epistemic frames that might be ideal preparation for future learning.

Augmented Reality Environmental Investigations

Augmented reality applications hold particular promise in disciplines such as environmental engineering, where spatial information and location matter. In field studies, such as investigating and cleaning toxic spills, spatial information about the distribtuion of the spill and location sensitive information about the proximity of the spill to other parts of the environment are core components of the investigation. But the investigative process, sampling strategies, and remediation strategies are all mediated by social factors (c.f. Vernon & Mehenna, 1998)¹. Students often have difficulty recognizing the situated nature of environmental engineering investigations and learning to act effectively within the many constraints (Nepf, 2002). First, environmental investigations are affected by resource constraints. The amount of time, money, equipment, and human power available affects what strategies are feasible in any given context. Second, the physical particulars of the research context drive an investigation, and research goals are often reprioritized in relation to local context. For example, discovering a lethal toxin in groundwater in close proximity to a major source of drinking water might be cause for re-evaluating a research approach. Third, there is an interplay between desktop research and collecting field data. In some cases, a knowledgeable informant can save investigators time and money by pointing investigators to probably culprits. Finally, social constraints affect both the investigative process and remediation strategies, as investigators need to manage how their work is perceived by others (particularly the press). Investigators need to avoid generating unwarranted public alarm, or in some cases, generating bad press for clients. A few environmental educators have begun exploring how immersing students in problems based on current events might serve as useful pedagogical models in environmental and chemical engineering (c.f. Dorland, and Buria, 1995; Patterson, 1980).

This study examines the implementation of a particular augmented-reality simulation, Environmental Detectives in three university classes and one high school class. This study is a part of a larger design research agenda (Collins, 1992) exploring the potential of augmented reality for supporting learning in environmental education. Environmental Detectives is an augmented-reality simulation game for the Pocket PC developed by the investigating team with the .NET compact framework. Environmental Detectives was designed in consultation with environmental engineering faculty and is matched to introductory environmental engineering course goals as well as Advanced Placement environmental science standards and is designed to be used in high school and college courses.

The curricular goal of Environmental Detectives is to give students an experience of leading a complex environmental science investigation. The game scenario was designed in consultation with two environmental science faculty and designed around a core dilemma of environmental science: how to conduct effective environmental investigations within social, geographic, and temporal constraints. This scenario requires students to (1) develop sampling strategies, (2) analyze and interpret data, (3) read and interpret scientific texts to understand the problem, (4) and ultimately design a viable remediation plan for core constituents. Scientific investigations are frequently presented to students as closed-ended problems with one right answer which can be solved linearly (c.f. Zolin, R., Fruchter, R., and Levitt, 2003), whereas in the field, scientists continuously frame and reframe the problem in response to budgetary and time constraints, local conditions, and what is known about the problem. As an example, researchers design sampling strategies in relation to the chemical and physical properties of a toxin, its potential health and environmental effects, legal issues surrounding its spread, as well as local conditions, such as nearby waterways and impediments to sampling (i.e. man made physical structures or waterways). Consistent with efforts such as the problem, project, product, process, people-based learning laboratory at Stanford University (Fruchter, 2004; http://pbl.stanford.edu), our goal is to immerse students in complex problem spaces where they draw on diverse resources, design creative solutions, and work across complex distributed environments in solving problems.

In Environmental Detectives, participants work in teams of 2-3 students playing the role of environmental engineers investigating a simulated chemical spill on a watershed. In phase one this watershed was surrounding the students’ university, including a nearby river, while in phase two the watershed was associated with a nature center and working farm. Both real-world watersheds include streams, trees, and other natural elements which are then augmented by a simulation of an environmental disaster; in this case a toxic spill of TCE that can potentially contaminate ground and surface water. Moving about in the real world, the handheld computers (Pocket PCs) provide a window into that simulation where students can take simulated sample readings, interview virtual people and get local geographical information (See Figure 1).

The spread of TCE is simulated on a location-aware Pocket PC, which, equipped with a GPS device, allows players to sample chemical concentrations in the groundwater depending on their location. So, for example, if the player is standing at point a, which is near the source of the spill (See Figure 1), she might take a reading of 85 parts per billion, where as a student standing on the opposite end of campus (point b) might take a reading of 10 points per billion. Players are given three reusable drilling apparatuses which they can use to drill for water samples. After drilling for a sample, players must wait three minutes for the drilling to complete and an additional one to three minutes for a sample to be processed, meaning that students can only take three samples at a time and are forced to develop sampling strategies in order to optimize the amount of ground that they can cover in limited time.

Environmental Detectives contains a multimedia database of resources which students can use to learn more about the chemical make-up of TCE, where TCE is found on campus, the health risks associated with exposure to TCE, how TCE flows through ground water, relevant EPA regulations TCE, remediation strategies for cleaning up TCE, and the political and economic consequences of EPA violations on campus. Students access these resources by obtaining interviews from virtual experts who we have spread across the area at locations roughly corresponding with actual operations. Because time is limited and there is not enough time to interview everyone or to drill more than a handful of wells, students must make choices between collecting interviews, gathering background information, and drilling wells, adjusting and reprioritizing goals as new information becomes available.

Critically, students in this game must combine both the real-world and virtual-world data to get to the bottom of the problem. The precise location of the spill is unknowable to students, and there is no one perfect solution to remediating the problem; each solution involves political, financial, and practical trade-offs that must be considered. Consistent with problem-based learning frameworks (e.g. Barron et al. 1998), students use their handheld computers as tools for gathering first hand data about the location and severity of the spill, and as a resource for accessing archives of information about toxicology, hydrology, similar cases, and local environmental conditions.

This version of Environmental Detectives takes 2-3 hours to complete, including introduction, game play, and debriefing, although a teacher might extend or shorten the game in order to meet her classroom needs. The simulation is designed to be flexibly adaptive, so that teachers might easily add extension activities (such as exploring the properties of TCE, the health effects of TCE, hydrology, water treatment plans, or similar cases) or remove activities as local conditions suggest (See Squire, MaKinster, Barnett, et al., 2003).

In the first phase of the project, we examined Environmental Detectives in three courses at a private technical university in the Eastern United States. One course was a freshmen environmental engineering course; the other two courses were undergraduate research writing. In both courses, the game was used to introduce students to issues around conducting real-world environmental investigations and used as a prelude for a larger research project. All three classes were two hours in length.

The second phase of the project took place at a nature center in an East Coast metropolitan area and involved an environmental science class of 18 high school students. The session involved roughly twenty minutes of introduction time, ninety minutes of game play and twenty minutes of debriefing. The pedagogical goals of the game were developed with nature center educators interested in engaging students more robust activities than traditional fieldtrip “scavenger hunt” exercises. They hoped that Environmental Detectives would engage students in interacting with the environment, geography and history of the site as well as in domain-based problem solving.
Methodology

In this study, we used a naturalistic case-study methodology (Stake, 1995) to gain a holistic view of the activity that unfolded during gameplay, understand how learning occurred through participation in these activities, and remain responsive to unanticipated issues which might arise during the research. Because we were interested in accounting for student-computer, student-student, and culture – student interactions, we employed quasi-ethnographic techniques designed to capture student actions at the molar level (Goodwin, 1994). How to capture an ecology, including the many tools, resources, and social structure that characterize any particular context of activity is challenging and still being negotiated in educational research (Engestrom & Cole, 1993). In describing a “situation” as a unit of analysis, Cole (1995) concentrated on practice, activity, contexts, situations, and events. We use narrative case studies to provide a broad flow of events that take each of these factors into consideration (c.f. Hoadley, 2002). We use discourse analysis (Gee, 1992) to examine how students constructed and framed problems more closely, and to study relations between class discourses and student’s scientific investigations.

Two researchers viewed and analyzed all researcher field notes, video tapes, and students’ projects using the constant comparative method (Glaser & Strauss, 1967), to generate relevant themes from the data. Consistent with Stake’s responsive method (1995), we paid special attention to unexpected and unintended consequences, given the exploratory nature of this research. After each round of videotape viewing, we developed emergent hypotheses, re-examining and refining these hypotheses as we watched subsequent tapes looking for disconfirming evidence or counter-hypotheses. We then wrote a case study for one group from the first phase of the project as a means of communicating to the reader the flavor of the activity, and give the reader a basis for generating contrary interpretations of the activity.

Using data from the second phase, we developed an additional case study for a group to give the reader a vicarious experience of the case and generate assertions across groups. In this case we focus specifically on the discourse of one group in order to understand how students frame the problem and generate meaning in situ. We provide a discourse analysis, an analysis of how language “enacts activities, perspectives, and identities" (Gee, 1999, p. 4-5). Researchers transcribed the interactions of one group that was representative (typical) of talk across groups. Consistent with Gee (1999) we focused on how language, specifically, word choice, cues, syntactic and prosodic markers, cohesion devices, discourse organization, contextualization signals, and thematic organization in language created the activity. Essentially, this analysis is toward understanding meaning, how it is made, enacted, and represented in situ. We specifically looked for places where meaning was negotiated and shared understandings were mobilized to solve problems, where meanings generated further action. Specifically, we examined how participants framed the problem, constructed the reason behind the activity, and negotiated problem-solving strategies.
Results

The following two brief case studies describe the results of the first phases of our design experiment. We start by describing with an illustrative example, a case study of a typical group. (1) privileging quantitative data; (2) framing the problem as a unidimensional one of “tracking down the toxin to its source” as opposed to a multi-dimensional problem involving probable cause, potential health effects, potential legal effects, and suggested remediation strategies; (3) integrating prior knowledge of the environment with students’ reasoning; (4) creatting emergent sampling strategies, such as triangulation. (5) “voting with their feet” as they decided which problem-solving path to pursue.

After a classroom briefing of the problem at hand all of the student groups started in the center of campus and learned to use their GPS and Pcoket PC, which took about 15 minutes to accomplish. Most groups headed out in different directions, immediately drilling samples. One group, tailed by a researcher, headed up away from the river and toward campus. One of the student inquired, “How many samples do we need?” It was not clear whether the question was addressed to the researcher or the rest of the group, but no one responded. The student holding the Pocket PC had previous GPS experience and started to guide the group. He drilled for one sample and then walked to nearby locations to take two more samples, the maximum amount of concurrent samples permitted. He chose a triangular configuration, though when another student asked why he chose this arrangement, he cited no particular reason.

Students retraced their steps as they waited for the required three minutes between sample drilling and reading. Finally the sample was retrieved. The reading was 88. Another student asked if 88 was good or bad. One student hypothesized that the number could be a percentage, but none of the other students in the group could answer definitively. Regardless, they decided that their next step was to collect more data.

As they walked to collect the other two drill rigs, a student not holding the Pocket PC asked what the data looked like. The student with the handheld described their current readings by pointing to three locations in physical space (as opposed to showing on the handheld) and citing the readings. Students debated the meanings of these readings. One student hypothesized that the readings were in parts per million. The student holding the Pocket PC suggested that they should go toward the “higher numbers”, pointing into the distance. They walked several hundred yards through several buildings toward the higher number and placed more drills.

This pattern was similar across groups. This exchange, taken from another group, reveals a similar strategy.

Lisa: The reading is 4.

Ben: It’s obviously good. Come on now.

Lisa: I don’t think it is good.

Ben: It’s obviously good.

Lisa: Four. Like four is a bad reading. Like four on a scale of one to five. Four is real bad.

Ben: On a scale of one to fifty though, four is pretty damn good.

Mel: True, but what is this scale? We don’t know that.

Ben: We don’t know that.

Lisa: We have no idea. It could even be that the top one is the best.

Mel: OK. So we need to dig another well.

Ben: Let’s get this one first [referring to an already dug well].

After several minutes, the readings from this second round of drill placements returned from the lab. One student noted that the new readings were very high in one direction. They walked in the direction of the higher readings, as if following a trail or scent, pausing briefly to interview a staff member in environmental policy, who happened to be nearby. The interview yielded little information, but it did reveal that they could conduct a second interview with a TCE supplier from facilities at a new location across campus (building 54), which they needed to visit within the next half hour because the informant was leaving for another meeting. They decided to immediately go to building 54 although there was no discussion about what information they hoped to find, or evaluation its anticipated value. Along the way they looked at the gradient and one student hypothesized that the concentration was likely to be higher on the other side of the building (the one they hadn’t visited yet).

The second interview revealed where TCE is used on campus, and the student holding the Pocket PC summarized this for the others. Meanwhile, the group took another reading. One student (not holding the Pocket PC) realized that the highest concentration appeared to be surrounding one building, and suggested that they should drill more wells there. This idea was dismissed by another student who assured that they already knew the source of the leakage, which was building three. Using the information from the toxin supplier combined with pre-existing knowledge of the activities in building three (that is where the university machine shop is), he correctly located the source of the toxin and suggested that they obtain interviews to help interpret their data. It is worth noting that although they had spent nearly 50% of their time already, the group did not know what units the readings were in (and indeed, one student hypothesized incorrectly that they were a percentage), what levels of TCE were dangerous, how likely the TCE was to spread throughout the environment (including into a nearby river), or what legal repercussions the university might face if the TCE were to leak off of university property. Most groups had similar problem-solving strategies, although one group, notably, stopped at a computer and used google to find a good deal of information on TCE.

Seeing another interview nearby, they headed in that direction. One student noticed the time, and suggested that they use their last 15 minutes wisely. The Pocket PC changed hands briefly to a different group member, but was quickly given back to the student who has been holding it most of the time as they had trouble tracking down the point that they were headed to. After several minutes of circling the building, they finally accessed the interview which explained how groundwater flows through campus.

As the students headed back to class, they discussed the implications of their findings. One student looked at the building where they hypothesized the toxin originated and then back at the river, declaring that by the time the pollution gets to the river the pollution is likely to be highly reduced (although they have no evidence to base this assertion on).

Debriefing. Next, the group presented their case to class. They pinned the location of the spill down to a particular building (which was correct) and theorized that the spill came from the machine shop. They argued that the spill was not a problem because the groundwater is not a source for drinking water, and the river was too far from the source of the pollution to be a problem. They recommended planting trees to mitigate the problem and monitoring the situation over time. They noted that this solution would cause little alarm in the community, and not destroy the only grassy area on campus.

Cross Group Discussions. Most of the twelve groups that we studied made similar findings. Most relied heavily on sampling, and roughly 75% of the groups accurately determined the location in the time allotted. Most groups also suggested the politically expedient answer of planting trees and monitoring the situation because they saw no immediate legal or health threat. Only three groups, which all focused on collecting interview data, correctly surmised that regardless of whether the spill was an immediate health hazard it was definitely a legal threat, and that it should be cleaned up to avoid EPA fines. Students across all the groups were very sensitive of the political ramifications of falsely calling too much attention to the problem. Successful groups gathered both samples and interview data, recursively examining what was known, reframing the research questions, and gathering new data.

Case 2: High School Students

Most college students had framed the problem as one of collecting samples to obtain the one correct solution of where the spill occurred, as opposed to an investigation into a socially situated problem. Therefore, we redesigned the interface to provide the player more information for weighing the potential value of interview data. In this case study, we focus more specifically on group talk to examine the processes by which the problem was framed. Across the groups we examined, four main motifs emerged in the talk: (1) Negotiating the environment in the investigative process, (2) Within and intergroup interpreting the problem as gathering information to complete a puzzle, (3) Discussion and problem solving which integrated the physical world, paper-based resources, and PDA-mediated resources, and (4) emerging of inter-group power dynamics. This section reports results from one group, using a brief discourse analysis to examine emergent learning practices.

In the following passage, a group discusses the best method for reaching an interview with an expert who is in the horse farm. Several physical structures enter their thinking.

1. Stacey: There’s a fence there I can’t get over it.

2. Gina: Then I don’t know what we’re going to do. We’re stumped. Let’s call the guy [facilitator on the walkie talkie] so we can find out what we’re doing.

3. Stacey: What does it look like?

4. Gina: We’re close. That’s the thing.

5. Stacey: Ok fine. Can we go over this fence [it is barbed wire]?

6. Gina: I don’t know.

7. Stacey: Maybe we can get on the other side by walking somewhere else.

8. Louis: Maybe we can walk the fence. No, there are trees.
Consistent with the goals of the exercise, environmental constraints and affordances immediately had an impact on their problem solving process. The constraints of the environment, namely fences (1,5,8) barbed wire (5), and trees (8) guide their problem solving path. All of Gina’s statements are declarative, assessing their progress and directing activity, whereas other students raise ideas as suggestions, couching them with qualifiers (i.e. maybe). The problem is about designing strategies in relation to local environmental affordances.

Roughly ten minutes into the activity, students have negotiated the particulars of the environment, with Gina having taken a lead in defining group work. They conducted their first virtual interview and now meet another group, which asks them how many interviews they have gathered. A shared understanding emerges whereby the point of the activity gets framed as collecting boxes, akin to a scavenger hunt.

9. Girl (group 2): How many did you get so far?

10. Louis: None, nothing.

11. Stacey: We’ve only gotten one box. How many have you got?

12. Girl (group 2): One so far. We were going for another one.

13. Boy (group 2): Three. Oh. You meant the boxes?

14. Gina: Did you dig?

15. Boy (group 2): Yeah.

16. Gina: Can you dig anywhere?

17. Boy (group 2): Yeah. I think so -- I did.

18. Gina: Cool. We got an interview. That’s all we did. We don’t have much time. We have to go.

In the final sample we examine, students are near the end of the activity. They are using real maps, the actual environment, and the Environmental Detectives-based maps interchangeably. They just collected an interview, and is now about to get another one. The students are concerned that they do not have enough information to solve the problem adequately. We pick up the discussion as they decide what to do next.

19. Stacey: Let’s go to that one [pointing to the learning center]. We just traipsed through a field.

20. Louis: I like how he was standing up there [pointing towards the house] and reading it.

21. Gina: Yeah, I know.

22. Louis: He got to stand at the house and we had to stand in the water [in the field].

23. Stacey: I know. I am so wet.

24. Louis: My socks are so wet.

25. Camera: We should head back soon.

26. Gina: Yeah, it is 12:50.

27. Louis: How far away is the thing [place they should return to]?

28. Gina: Where do we have to go again?

29. Stacey: Alan Morgan center? That is…

30. Louis: [Looking around]. Not around here.

31. Stacey: Right here [points at paper map].

32. Gina: And we’re right here [points at ppc].

33. Stacey: That’s not bad.

34. Louis: But we have to go through the tunnel.

35. Stacey: How are we supposed to make recommendations?

36. Gina: I don’t know.

37. Louis: Just read off of the information that we got.

38. Gina: I thought we could dilly-dally but we actually did work

39. Louis: For once.
Stacey initiates the conversation by suggesting that they go to the learning center, as the group is tired of “traipsing through fields,” which “got their socks wet.” Louis notes that their path back to the nature center will take them through the tunnel (34), a feature of the environment which earlier had been the cause of considerable discussion, as a group of birds flew out and scared the group. Stacey notes their lack of information (they had located several interviews, but dug few, if any wells), and asks the group how they are supposed to make recommendations (35). As in the other changes, Stacey queries the group for strategies, and Gina gives the response (36). Louis (37) suggests that they just “read off their information.” Gina sums up the group’s dilemma: they thought that the exercise would be relatively thoughtless – that they could “just dilly” dally, but instead, they “actually did work,” (38) which Louis agreed with (39). Students use maps (19, 31), the real environment (20, 30), and PDA resources as tools (32) for communicating.
Cross Case Discussion

The Environment in Augmented Realities

Across both groups, students had little difficulty negotiating augmented reality and within minutes were thinking across this distributed environment. Students mapped virtual data onto the real world context or pointed to locations in the real world and described the concentrations at those locations using data and information off of the handhelds. Using maps and computers, they continuously worked across the distributed problem solving context with little difficulty. More importantly, students used knowledge of the surroundings to solve the problem. The college students, who were more familiar with the environment than the high school students (who were on a field trip) investigated sites of known printing presses, metal shops, and other places with large machinery, which were noted as being associated with TCE early on in the investigation. College students used hypotheses of the activities in each building to guide their thinking, yet they were less intimately connected to their surroundings.

In the high school case, where students had to traverse rougher terrain, environmental constraints affected their problem solving paths to an even greater extent. From the first challenge of climbing a fence to the final challenge of negotiating a tunnel, students’ problem solving was concrete, and specific environmental constraints (fences and trees), affordances (such as the tunnel) and local demands (time considerations) were a part of students’ thinking. Students, rarely, however, used the physical environment to talk about toxin spreads, as they framed the activity as collecting and synthesizing information rather than gathering data, constructing a narrative and designing a solution. Indeed, this group struggled to balance the need to gather background information with that of drilling and sampling (as environmental engineers might predict). They defined the activity as a scavenger hunt where the moment-to-moment goal was to collect interviews as quickly as possible. This meaning was created through both intragroup and intergroup communications whereby students negotiated the focus of the activity as collecting information. How and why the activity got framed as a scavenger hunt is the result of several factors, including the nature of fieldtrip and students’ past experiences (as evidenced by Gina and Louis’ comments that this “actually was work.” (38-39).

Augmented reality simulations may have communication advantages (i.e. gestures, facial expressions) over their purely virtual counterparts. These groups debated in real time using their voices, gestures and physical locations as tools. While similar representations exist in virtual worlds, they require negotiated standards that must be adopted and accepted over time. Emoticons in chat and hand signals by avatars are two examples of these emergent standards. Students in augmented realities do not need to learn these standards, as evidenced by these cases, since they employ the modes of communication with which they are the most familiar. More importantly, we also saw group members “voting with their feet” in determining the next location to go. While this did not always result in democratic decision making (the person holding the computer seemed to have a larger vote), it did make immediately apparent what people’s opinions were, and provoked critical dialog.

A major theme guiding the design of this project was to authentically recreate core environmental engineering practices, building the activity around the central dilemma of balancing multiple data sources and the evolving, competing needs of an investigation. The university students were driven almost exclusively by the collection of water quality data from the wells. Most students collected samples at the starting location or traveled to where the initial reading was found. When students did conduct interviews, it was because interviews were near desirable sampling sites. In fact, each group collected between 6-10 water samples before they ever determined what the units meant or what level was considered toxic. This problem (not knowing toxic levels) was often discussed, but dismissed in favor of collecting more samples, perhaps hoping that a pattern would emerge that would put the readings in perspective. In short, wherever there was a problem, the answer was typically to drill more samples. The holes in students’ understandings were made more evident when they presented their assessment and remediation plans. For example, several groups reported that the TCE was unlikely to reach nearby surface water because it was “far away,” even though they did not know how fast the TCE was moving or how long it had been in the ground (which might indicate that it had already spread to the river). Other groups made assumptions about the use of groundwater for drinking water, though they had no evidence to support these assertions.

When collecting water quality samples, the majority of groups who actually collected data used a “warmer/colder” strategy for locating the source. They would take two samples and move in the direction of the sample with the higher concentration. This method proved to be largely successful, though it was susceptible to getting stuck on local minima (due to local variability or a smaller secondary spill built into the game), and was very data intensive. Two other strategies that were employed were triangulation, and concentric circles. Triangulation (perhaps suggested by the three simultaneous wells limitation in the game) involved drilling three wells in some relatively small area and then moving in the direction of the highest concentration. The concentric circles strategy was designed to start at the original site of contamination and then move out from there sampling along different radii. Neither of these strategies was more successful in the context of this game, though they might have involved fewer wells, and been less susceptible to local variation.

Students often recognized shortcomings in their information, citing the lack of data on flow rates, or toxic levels, but then proceeded to make recommendations based upon these incorrect or incomplete assumptions. Regardless of this information, the proposed solutions were fairly consistent—because this is largely a drinking water problem, and since we don’t drink the groundwater we should plant trees (which have been found to have a measurable but quite minimal effect on TCE) and monitor the situation. We have classified this solution as the “political solution”—on the surface it seems like it should satisfy the parties involved (doesn’t alarm the population, detract from the aesthetics of campus, or call attention to any environmental wrongdoings), but would be largely ineffective against any real problem. In reality this problem has no one solution that could satisfy everyone and address the real environmental and legal concerns (the pollution is likely to wind up in the river, which might upset environmentalists, though it might not have real environmental consequences, but flowing off the property at any level has legal implications). Students seemed unable or unwilling to make the hard tradeoffs and address this solution.

The high school students, described the second phase of the study, also struggled to understand the nature of the environmental investigations, but in a different way. They defined the activity as a scavenger hunt where the goal was to collect as many interviews as possible. This meaning was negotiated through both intragroup and intergroup communications (i.e. 10-18). Through intergroup exchanges, students negotiated and agreed upon a focus of the activity of one as collecting information, as one might collect pieces of a puzzle. How and why the activity got framed as a scavenger hunt (which was unique and contradicted earlier case studies) is the result of several factors, including the nature of fieldtrip and students’ past experiences (as evidenced by Gina and Louis’ comments that this “actually was work.” (38-39). Consistent with instructional goals, students’ environmental investigations were deeply embedded in the particulars of this physical location. Fences, trees, fields, tunnels, and marshes played a role in students’ thinking and problem solving. In most instances, students used these features as navigation devices, seemingly thinking across people, paper-based resources, and PDA-mediated data. Students used the physical environment in deciding which interviews to get (few groups got a critical interview which was at the top of a steep hill), but rarely used the physical environment to talk about toxin spreads, as they framed the activity as one of information collection and synthesis.

Failure in Learning

Students across both cases had difficulty negotiating “soft” qualitative information gained in interviews and “hard” quantitative data gathered through physical samples, suggesting that the game captured a real and relatively hard problem for these students. The differences in appropriation can be attributed to several factors, but minimally these cases remind of us of the power of local cultures in shaping how tools are used. The game was only one object in the activity system and encompassing cultural models of schooling, specific academic practices, as well as students’ goals, shaped the activity. As the student who complained about the difficulty would suggest, this activity was more complex than normally demanded from them at school, and one can easily understand how this activity could be misconstrued in this context as a relatively simple fact-collecting exercise.

Students’ failure to discern what information was needed for an effective solution represents an interesting failure that might be leveraged for future learning. If learning to redefine the research problem given new information is a central dilemma in environmental engineering investigations, then perhaps allowing students to make these mistakes – to make choices and experience their consequences within a sandbox-like virtual world is a good thing. One way to think about the pedagogical value of competitive games in education is to consider their role in inducing failure states, in providing a socially acceptable context for trying different strategies, experimenting with ideas, and then revising those ideas. Play theorists (e.g. Salen and Zimmerman, 2004), emphasize the importance of creating safe spaces where people can experiment with new ideas and new identities.

One particularly promising pattern we observed was different participants arguing for different investigative strategies. In most groups, dominant personalities (as in the high school case), or a combination of social factors (as in the college students cases) drove students to privilege one approach over another and prematurely close off strategy discussions. In future iterations of Environmental Detectives, we hope to explore ways of creating game dynamics and groupings so that these strategies are more even. Further, we have begun scaffolding students’ problem solving by lengthening the game and providing mid-course reviews that force students’ to articulate what they know and do not know. The distributed nature of the game makes in game coaching difficult, although communication technologies that would allow teachers to better monitor students’ work could be integrated into the game.

Over the last few years, games and simulations have been criticized for their contrived, nature and contrasted with the social “authenticity” of engaging in communities of practice, either through participation in extended communities of practice, or through establishing classroom-bound communities of practice engaging in authentic inquiry (e.g. Barab & Duffy, 1999). Quoting Lave (1993), Barab and Duffy pit learning environments predicated upon a practice field metaphor against one predicated on Lave and Wenger’s (1991) communities of practice, arguing that (in practice fields) the problems, although authentic in the complexity they bring to the learner, are not authentic in the sense that they are an integral part of the ongoing activity of the society. With the practice field, education is viewed as preparation for some later sets of activities, not as “meaningful activity in its own right” (p.48-49).

We argue that augmented reality games such as Environmental Detectives offer unique learning opportunities that allow learners to experience intellectually productive problems central to science in a psychologically “safe space” where they can try new ideas (and identities) and learn through failure. Environmental Detectives draws from traditional “practice field” models of education, but deviates from most these forms in that students are placed under time pressures and forced to make decisions, which has consequences on students’ success, but within a safe environment where failure is acceptable. What sampling strategies students use, what information students decide to pursue, and when students decide to jump from sub-goal to sub-goal can have critical ramifications for student performance. This decision structure is designed to not only be engaging, but to model authentic scientific and engineering practices, including planning research strategies, evaluating the value of data sources, and constructing arguments in debating with team members.

Perhaps one way to think about the role of strategy games in learning environments is as precursors to conducting full scale investigations. The teachers we worked with saw Environmental Detectives as a useful tool for helping students understand some of the trade-offs in doing larger research projects. Perhaps games can provide one way for overcoming some of the challenges to more open-ended forms of inquiry-based learning, such as a lack of student engagement, or student’s experiencing cognitive overload at the challenges of conducting open-ended inquiry. Games such as Environmental Detectives might provide scaffolding for conducting larger investigations, serving as “simplified, but authentic” conditions for larger, more complex tasks. The fact that Environmental Detectives explicitly bridges real and simulated worlds suggests that it may also help to bridge these practice fields with subsequent actual fields. We believe that by bringing the physical world into the game space, augmented reality gaming applications have unique educational affordances when compared to their purely virtual counterparts. In purely digital simulations, we ask students to make connections between wholly constructed digital virtual environments and the physical landscape. Augmented reality applications allow the physical environment to enter the problem space and students’ thinking, and these cases suggest just how environmental affordances can affect a problem-solving path within an augmented reality environment. In future studies, we will examine how the physical environment enters students’ thinking across a variety of environmental engineering tasks, comparing students thinking in virtual and augmented reality environments..

Barab, S., Cherkes-Julkowski, M., Swenson, R., Garrett, S., Shaw, R. (1999) Principles of Self-Organizing Systems: Ecologizing the Learner-Facilitator System. Journal of Learning Sciences, 8 (3/4).

Barab, S. A., & Duffy, T. (2000). From practice fields to communities of practice. In D. Jonassen & S. Land (Eds.), Theoretical foundations of learning environments. Hillsdale, NJ: Lawrence Erlbaum Associates.

Barab S.A. & Kirshner D., (in press). Methodologies for capturing learner practices occurring as part of dynamic learning environments. To appear in Journal of the Learning Sciences.

Barab, S.A. & Luehmann, A. (in press). Building Sustainable Science Curriculum:
Acknowledging and Accommodating Local Adaptation. To appear in Science Education.

Barron, B. J. S., Schwartz, D. L., Vye, N. J., Moore, A., Petrosino, A., Zech, L., Bransford, J. D., & The Cognition and Technology Group at Vanderbilt. (1998). Doing with understanding: Lessons from research on problem- and project-based learning. The Journal of the Learning Sciences, 7, 271-311.

Blumenfeld, P., Fishman, B., Krajcik, J., Marx, R., and Soloway, E. (2000). Creating Usable Innovations in Systemic Reform: Scaling up Technology-Embedded Project-Based Science in Urban Schools, Educational Psychologist, 35 (3), 149-164.

Brown, A. (1992). Design experiments: Theoretical and methodological challenges in creating complex interventions in classroom settings. The Journal of the Learning Sciences, 2(2), 141-178.

Brown, J.S., Collins, A., & Duguid, P. (1989). Situated cognition and the culture of learning. Educational Researcher, 18, 32-42.

Bransford, J. D., Franks, J.J., Vye, N.J. & Sherwood, R.D. (1989). New approaches to instruction: Because wisdom can’t be told. In S. Vosniadou and A. Ornony, (Eds). Similarity and analogical reasoning, (pp. 470-497). New York: Cambridge, University Press.

Bransford, J.D., Brown, A.L., & Cocking, R.R. (Eds.). (1999). How people learn: Brain, mind, experience, and school. Washington DC: National Academy Press.

Casti, J., (1998) Would-be worlds : How simulation is changing the frontiers of science, John Wiley & Sons, New York, NY.

Caillois,R. (1979). Man, play, and games. Schocken Books: New York.

Cognition and Technology Group at Vanderbilt (CTGV), (1990). Anchored instruction and its relationship to situated cognition. Educational Researcher, 19 (6), 2-10.

CTGV. (1993). Anchored instruction and situated cognition revisited. Educational Technology, 33 (3), 52- 70.

Cole, M. (1995). The supra-individual envelope of development: Activity and practice; situation and context. In J. Goodnow, P. Miller, & F. Kessel (Eds.), Cultural practices as contexts for development (pp. 105-117). San Francisco: Jossey-Bass.

Colella, V., (2001) Participatory simulations: Building collaborative understanding through immersive dynamic modeling Journal of the Learning Sciences, pp. 471-500.

Cobb, P. (2000). Conducting teaching experiments in collaboration with teachers. In A. E. Kelly & R. A. Lesh (Eds.), Handbook of research design in mathematics and science education (pp. 307-333). Mahwah, NJ: Lawrence Erlbaum Associates, Publishers.

Dewey, J. (1938). Context and thought. In R. Bernstein, (Ed.) On experience, nature and freedom. (pp. 88-110). New York: Library of Liberal Arts.

Dorland, D. & Baria, D.N. (1995). Hazardous waste processing in the chemical engineering curriculum. Chemical Engineering Education, 29(3), pp. 178-181.

Edelson, D. C., Pea, R. D., & Gomez., L. (1996). Constructivism in the collaboratory. In B. G. Wilson (Ed.), Constructivist learning environments: Case studies in instructional design, (pp. 151-164). Englewood Cliffs, NJ: Educational Technology Publications.

Elliot, J., Adams, L, & Bruckman, A. (2002) No Magic Bullet: 3D Video Games in Education. Proceedings of ICLS 2002, Seattle, WA, October 2002.

Engeström, Y. (1993). Developmental studies of work as a testbench of activity theory: The case of primary care medical practice. In S. Chaiklin & J. Lave (Eds.) Understanding practice: Perspectives on activity and context (pp. 64-103). Cambridge, MA: Cambridge University Press.

Engestrom Y. & Cole, M. (1997). Situated cognition in search of an agenda. In D. Kirshner, & J.A. Whitson (Eds). Situated cognition: Social, semiotic, and psychological perspectives, (pp. 301-310). Mahwah, NJ: Erlbaum.

Feurzeig, W., and N. Roberts (1999), Modeling and simulation in precollege science and mathematics, Springer, New York, NY.

Friedman, & diSessa

Gibson, J.J. (1979). The ecological approach to visual perception. Boston: Houghton Mifflin.

Glaser, B. G., & Strauss, A. L. (1967). The Discovery of Grounded Theory: Strategies for Qualitative Research. Aldine.

Goodwin, C. (1994) Professional vision. American Anthropologist, 96(3), 606-633.

Greeno, J. G. (1998). The situativity of knowing, learning, and research. American Psychologist, 53, 5-26.

Guba, E. G., & Lincoln, Y. S. (1983). Epistemological and methodological bases of naturalistic inquiry. In G. F. Madaus, M. S. Scriven., and D. L. Stufflebeam (Eds.), Evaluation models: Viewpoints on educational and human services evaluation (pp. 311-334). Boston, MA: Kluwer

Jonassen, D. (1999). Designing Constructivist Learning Environments. Instructional-Design Theories and Models. C. Reigeluth. Mahwah, New Jersey and London, Lawrence Erlbaum.

Krajcik, J., Blumenfeld, P., Marx, R., Bass, K., Fredricks, J., & Soloway, E. (1998). Inquiry in project-based science classrooms. Journal of the Learning Sciences, 7(3&4), 313-351.

Lave, J. (1988). Cognition in practice. Cambridge: Cambridge University Press.

Lave, J., & Wenger, E. (1991). Situated learning: Legitimate peripheral participation. New York: Cambridge University Press.

Leont'ev, A. (1981). Problems in the development of mind. Moscow: Progress Publishers.

Nelson, L. M. (1999). Collaborative problem solving. In C. M. Reigeluth (Ed.), Instructional-design theories and models: A new paradigm of instructional theory. Mahwah, NJ: Lawrence Erlbaum Associates.

Norman, D. A. (1988). The design of everyday things. New York: Doubleday.

Patterson, J.W. (1980). Environmental engineering education: Academia and an evolving profession. Environmental Science and Technology, 14(5), p. 524-532.

Pea, R. (1993). Practices of distributed intelligence and designs for education. In G. Salomon (Ed.), Distributed cognitions: Psychological and educational considerations (pp. 47-87). Cambridge, UK: Cambridge University Press.

Peirce, C.S. (1868/1992). Questions concerning faculties claimed for man. Journal of Speculative Philosophy 2. Reprinted in N. Houser & C. Kloesel (Eds.) The Essential Peirce (Vol 1, pp. 11-27).. Bloomington, IN: Indiana University Press.

Roth, W.-M. (1996). Knowledge diffusion in a grade 4-5 classroom during a unit on civil engineering: An analysis of a classroom community in terms of its changing resources and practices. Cognition and Instruction, 14, 179-220.

Schank, Roger C. (1994). Goal-Based Scenarios: A Radical Look at Education. Journal of the Learning Sciences, 3(4), 429-453.

Silverman, D. (1993). Interpreting qualitative data: Methods for analysing talk, text, and interaction. Newbury Park, CA: SAGE.

Soloway, E., Grant, W., Tinker, R., Roschelle, J., Mills, M., Resnick, M., Berg, R., & Eisenberg, M. (1999). Science in the palm of their hands. Communications of the ACM, 42(8), 21-26.

Spitulnik, J., Studer, S., Finkel, E., Gustafson, E., Laczko, J., Soloway, E. (1995). The RiverMUD Design Rationale: Scaffolding for Scientific Inquiry Through Modeling, Discourse, and Decision Making in Community-Based Issues. Technical Report, Highly Interactive Computing Research Group, University of Michigan.

Squire, K.D., Makinster, J., Barnett, M., Barab, A.L., & Barab, S.A. (2003). Designed Curriculum and Local Culture: Acknowledging the Primacy of Classroom Culture. Science Education. 87:1– 22

Stratford, S. J., Krajcik, J., & Soloway, E. (1998). Secondary students' dynamic modeling processes: Analyzing, reasoning about, synthesizing, and testing models of stream ecosystems. Journal of Science Education and Technology, 7(3), 215-234.

Sutton-Smith, B. (1997). The ambiguity of play. Cambridge, MA: Harvard University Press.

Walkerdine, V. (1997). Redefining the subject in situated cognition theory. In D. Kirshner & J.A. Whitson (Eds). Situated cognition: Social, semiotic, and psychological perspectives, pp. 57-70. Mahwah, NJ: Erlbaum.

Wenger, E. (1998). Communities of practice: Learning, meaning, and identity. Cambridge, MA: Cambridge University Press.

Zimmerman, E. & Salen, K. (2004). Game + design: An interactive design handbook. Cambridge, MA: MIT Press.


Figure 1 – A screen shot (left) of Environmental Detectives. The red dot indicates the players current location and is guided by real world position as supplied by GPS. The pink markers represent locations of interviews, while the blue markers show where the player has already sampled the water. On the right is shown some of the textual resources that players can uncover.
Figure 2 – Players beginning a round of Environmental Detectives spot their current location on a handheld computer and await readings from a recently placed sample.

Fruchter, R., “Degrees of Engagement in Interactive Workspaces,” International Journal of AI &

Society, accepted for publication, 2004.

Zolin, R., Fruchter, R., and Levitt, R., “Realism and Control: Problem-based learning

Education (IJEE) 2003.