Instructional Science 29: 403–427, 2001. © 2001 Kluwer Academic Publishers. Printed in the Netherlands.

403

Epistemology, situated cognition, and mental models: ‘Like a bridge over troubled water’ NORBERT M. SEEL Department of Educational Science, Albert-Ludwigs University, 79085 Freiburg, Germany (E-mail: [email protected])

Abstract. Over the past two decades important educational implications have been drawn mainly from two movements of epistemology: Constructivism and situated cognition. Aside from a meta-theoretical use of constructivism, the concept ‘situated cognition’ refers to a conception of situational context bound to a historically and socially determined situational logic. Focusing on educational processes, situated cognition is considered to be a central construct for instruction, as is the closely related concept of the construction of mental models. There are various kinds of the construction and change of mental models in a situational context: self-guided inductive construction is one example; another is the processing of a conceptual model provided to the learner. An emerging question is how the preconception changes and if the effects of such a model transition are stable. An exploratory study will be sketched which investigates the significance of a conceptual model provided at the beginning of the learning process; it has been hypothesized, that such a conceptual model significantly impacts the stability (i.e. the successful reconstruction) of mental models built in the course of learning. Also considerable intraindividual differences and changes between two points of assessing the learners’ causal explanations were found. Similarities of the individuals’ reconstructions could be explained with regard to similarities of the structures of the learning situations and the related instructional intervention. In general, the results of this exploratory study support the assumption that mental models are constructed in dependence on the demands of learning situations. Keywords: cognitive apprenticeship, educational diagnosis, learning environments, mental models, situated cognition

Introduction In accordance with a long tradition of educational science of operating with theoretical concepts of philosophy and psychology we can still observe a tremendous trend to adopt new theoretical constructs from cognitive science. But at the same time, it must be recognized that the resulting educational movements often are not theoretically sound because the adapted theoretical constructs are not precisely defined, interrelated to other constructs, or accurately assessed (cf. Snow, 1990). Consequently, Strauss (1996) compared the relations between cognitive science and education with ‘troubled water’, and

404 Anderson et al. (1996) identified several important claims of new educational approaches as empirically invalid. This article aims at demonstrating that mental models could be considered a ‘bridge over troubled water’ insofar as this theoretical concept mediates successfully between constructivism, situated cognition approaches, and education. Actually, over the past two decades important educational implications have been drawn from two movements of epistemology: Constructivism and situated cognition (cf. Anderson et al., 1996; Reynolds et al., 1996). Whereas constructivism refers, first of all, to a particular philosophical position that is relevant for education and instruction primarily at a meta-theoretical level (cf. Dinter, 1998; Dinter & Seel, 1994), the concept ‘situated cognition’ has immediate and significant educational implications because it provides numerous prescriptions for instructional practice. Therefore, we easily can find similar ideas in the instructional sciences over the last several decades. So, for example, there was a strong situation-oriented approach in the field of curriculum development 25 years ago. It was argued that future life situations should always be considered in close relation to the learners’ former history as well as to their social situation, i.e. their problems, needs, and opportunities to learn. Accordingly, situation-oriented curricula should aim to strengthen the autonomy and competence of pupils by going back to familiar situations in everyday life (cf. Hemmer & Zimmer, 1975). Although the concept of situated cognition and learning has a long tradition in education and curriculum development, it has been considered from a different perspective than in cognitive science. Here, situated cognition is normally a part of constructivism that is centrally concerned with the construction of ‘scientific languages’ in the spirit of both Peirce’s pragmatism (e.g. Peirce, 1973) and Wittgenstein’s (1969) ‘language games’. Although constructivism aims at the allocation of shared subject domains such as situations of teaching and learning, the notion of situated cognition as discussed currently in educational psychology is usually not placed on such a meta-theoretical level, but rather centers around broad sociocultural models of teaching and learning (cf. Strauss, 1996). Moreover, in the related literature a principal distinction is made between situated cognition theories and models of situated learning promising supplements to conventional forms of instruction (cf. Renkl et al., 1996). Both approaches evidently involve ideological implications for education and school reform. However, this kind of reform in education is unlikely to take place unless we are willing to bring to the surface the implicit assumptions in which the new educational movements are rooted so that we may subject them to critical interrogation. In this article I wish to contribute to the discussion on situated cognition by inquiring into one of its root assumptions:

405 namely, the notion that learners actively construct their own interpretations of objects and events within the intrinsically dialectical process of accruing knowledge.

Situativity of cognition and learning Reynolds et al. (1996) have pointed out that the situated cognition approach evolved recently from artificial intelligence and cognitive science to capture the broader situational context of learning and communication, and thus to overcome the encapsulation of school learning as it has been criticized by Engeström (1991), Resnick (1987), and others. With respect to the common distinction between symbolic and connectionist models of cognition, the situated cognition approach clearly evolved from symbolic models. Bruner (1990, p. 2) captured the central idea of cognitive science as follows: “Let me tell you first what I and my friends thought the [cognitive] revolution was about there in the late 1950s. It was, we thought, an all-out effort to establish meaning as the central concept of psychology. It focused upon the symbolic activities that human beings employed in constructing and in making sense not only of the world, but of themselves.” This corresponds with the view of semantics according to which meaning is a construction of human thinking that necessitates computational processes utilizing symbolic notation (cf. Greeno, 1989). Thinking and reasoning are considered a symbol manipulation process which enables individuals to form and express subjective experiences, ideas, thoughts, and feelings. Evidently, central theoretical concepts of the situated cognition approach, such as mental models, center around the notion that human information processing presupposes the use of symbols to represent ideas and thoughts (Seel, 1991). Accordingly, the theory of situated cognition aims to account for how individuals learn in environments that contain the external world to be understood, the individual’s perceptions and internal representations of this world, and the individual’s symbolic interactions with it. In this context, learning is defined as the individual’s ability to construct meaning by extracting and organizing information from a given environment. Thus, the cognitive processes that occur as the individual interacts with an environment are at the core of situated cognition, and cognition (learning, thinking, acting) is considered as consisting of those interactions between learners and situations.

406 Situated cognition in cognitive science In the field of cognitive science situated cognition can be considered a novelty – or even a paradigm shift (cf. Sandberg & Wielinga, 1992). However, cognitive science and educational psychology differ in their understandings of situativity. Educational psychologists focus on the dialectic nature of learning in instructionally designed environments. Cognitive scientists, on the other hand, primarily relate situated cognition to the referentiality of propositions. Accordingly, situated cognition is seen as means of mental representation by referring knowledge to structures of the external world (cf. Seel, 1991). Correspondingly, situational logic assumes that historical and sociological explanations of events consist of model-like descriptions of the situations in which people act. Individuals behave in accordance with the situation’s requirements whereby the situation encompasses the individual’s dispositions (e.g. objectives and abilities) as well as the specific environment in which the individual acts. Consequently, situation semantics emphasizes different aspects of situativity than cognitive scientists do. However, situation semantics can also be considered a recent theory of semantics developed by Barwise (1981), namely as a generalization of the semantics of possible worlds which aims to address the argument that the meaning of a proposition is always dependent on its context (i.e. situation). Moreover, semantics of possible worlds maintains that only objects and situations are given as possible worlds, whereas characteristics and relationships of individuals must be inferred. Contrarily, in situation semantics objects and situations as well as their characteristics and relations are considered given, and worlds are defined as maximally consistent situations. According to the semantics of possible worlds, the meaning of a proposition results from a mapping of propositions into the truth values TRUE and FALSE, whereas in situation semantics the meaning of a proposition is grounded in the relationship between the context within which the sentence is expressed and the situation described by this sentence. In both cognitive and educational psychology situated cognition is considered a product of the internal operations which occur when a learner interacts with a physical and social situation. In accordance with the vocabulary of practical philosophy, situations include all environmental states to which subjects are exposed and which are not self-constructed models. However, this definition creates a dichotomy between external situations and the mind’s constructions. Therefore, the question arises of how the suggested interactions of a learner with a physical or social situation may occur. The answer of those who advocate situated cognition is that the learner constructs a mental model in order to simulate relevant aspects of the situation to be cognitively mastered.

407 Situated cognition and mental models Mental models emerged in recent years as a central theoretical construct to encompass both situated cognition as well as qualitative reasoning. Greeno (1989), for example, argued that both comprehension of and reasoning in specific situations (e.g. in mathematics, physics, and real-life situations) necessarily involve the use of mental models of different qualities. Modelbased reasoning occurs when an individual interacts with the objects involved in a situation in order to manipulate them mentally so that the cognitive operations simulate (in the sense of thought experiments) specific transformations of these objects which may occur in real-life situations. That means, mental models ‘run in the mind’s eye’ to produce qualitative inferences with respect to the situation to be cognitively mastered. Accordingly, mental models are occasionally called ‘situation models’ (e.g. Morrow et al., 1989). In accordance with symbolic models of cognition it is widely recognized that the construction of mental models necessarily presupposes the use and manipulation of signs (used as index, icon, or symbol) to the extent that mental models are used to organize the symbols of experience and thinking to achieve a systematic representation of this thinking as a means of understanding and explanation (Seel & Winn, 1997; Seel, 1999a). Accordingly, in cognitive psychology and similarly in educational psychology, mental models are considered qualitative mental representations which are developed by subjects on the basis of their available world knowledge aiming at solving problems or acquiring competence in a specific domain. However, this concept of mental models is really not new. Historically seen, it was Craik (1943) who introduced the similar idea of internal models with the notion of a working model into psychology. This author developed a far-reaching epistemological perspective for research on human cognition. In contrast to the behaviorism that dominated psychology at this time Craik assumed that people experience reality only mediated by mental constructs such as internal models (discussed here in terms of mental models). Thus, Craik’s idea of internal models presupposes that an individual who intends to give a rational explanation for something must develop practicable methods in order to generate adequate explanations on the basis of restricted knowledge and limited information processing capacity. In other words, the individual constructs a model that both integrates the relevant knowledge of the world and meets the requirements of the situation to be explained. This mental model ‘works’ when it fits the subject’s knowledge bases as well as the explanatory needs with regard to the particular situation to be cognitively mastered. Thus, internal models serve a twofold epistemological function: they represent human knowledge and they generate subjective plausibility with regard to the external world and its situations.

408 In summary, mental models are cognitive artifacts, i.e. inventions of the mind that represent, organize and restructure the subject’s domain-specific knowledge in such a way that even complex phenomena of the (observable or imagined) world become plausible. Craik’s constructivist epistemology has been adopted and continued by Johnson-Laird (1983), Greeno (1989), Halford (1993), and Seel (1991) and it provides an appropriate framework for the conceptualization of higher-ordered constructions of knowledge and related cognitive functions. Constructivists basically assume that every form of human information processing requires constructive activities with regard to both the reception of sensory data and a deeper understanding of the world (cf. Dinter, 1993; Stachowiak, 1973). They argue that mental models guide and regulate all human perceptions of the physical and social world. Compared to the unqualified informational input provided by raw sensory data, perceptions are meaningful constructions. They are generated on the basis of former experiences represented in mental models that ‘tell’ the individuals what they may perceive. According to this view, human knowledge is based on experiences produced by the material world (i.e. situations and environments). But reality does not provide a discernible, fixed and all finished shape that could be mapped by the human mind. Rather it is the other way around: the mind projects order onto the diversity of world phenomena. Mental models reflect the structure of the external world because they are constructed to structure it and not because they reproduce or copy a given external structure. However, the human mind is not free to arbitrarily impose any possible structure onto the world, but rather the mind is constrained through the external world. Mental models are constructed with regard to the subject’s requirements within specific environmental contexts. According to the epistemology of Piaget (1950), they are products of accommodation aiming at cognitive adjustments of a subject’s internal structures to an environment’s structure whenever the subject is not able to assimilate this external structure (cf. Seel, 1991). That is, mental models should be considered a central means of equilibration between the mind and the environment to be cognitively mastered. Consequently, although the material, social, and historical particulars of cognition may vary in relevant ways, it is certainly not the case that only some cognition is situated and other cognition is not. Although advocates of situated cognition agree on the central role of mental models, they often did not focus on the internal processes that occur when learners construct mental models. Correspondingly, Greeno (1989) and other advocates of situated cognition remain rather vague as to what the mind does to construct mental models. Several years ago Klix (1971) described the cognitive processes on which the interactions of a learner with a physical or

409 social situation are based. He considered them as learning processes that are principally oriented toward the environment (or situation) in that they start with the perception of external stimuli provided by the environment. These learning processes lay the foundations for knowledge acquisition as well as affecting behavior, and furthermore, they have a strong influence on higherordered cognition such as the construction of mental models. According to Klix the essential conditions for such learning consist of (a) the learner’s perceptual faculties corresponding to the demands of the specific environment, (b) the adjustment of cognition and behavior to the environment, and (c) the stimulation of domain-specific memory as well as the internal evaluation of perceptions to which the decisions on behavioral patterns are oriented. The intervention of learning in the perceptual process significantly alters the information circulation between the learners and their environments. The actual utilization of adjusted cognitive behaviors is only possible if a better or optimal decision is continuously being connected with the characteristics of the situation in which these behaviors have proved successful. Connecting characteristics of situations with the adjusted behaviors of the mind constitutes the essential function of memory in the course of learning. In addition to such cognitive learning, human information processing always entails strong motivational aspects (for example, the evaluation of the features and weighting of the usefulness of a decision for a particular alternative) that are usually ignored by the advocates of situated cognition. Accordingly, effects of environmental states are perceived as complex stimuli, and then, they are transformed into mental representations which in turn activate components of a preformed memory structure associated with particular procedures as ‘answers’ to these stimuli. Parallel to this, the perceived stimuli are evaluated in conjunction with and in terms of the learner’s motives and needs. Often, a correction (accretion, tuning, or restructuring) of cognitive structures is necessary and will be stored in a case-based format. In subsequent cases, similar situations activate these modified memory structures which must then prove their success once more. Following Seel’s (1991) theory of mental models, these cognitive artifacts are not fixed structures of the mind, but rather they must be constructed when needed – for example, in order to master a learning situation with its particular demands.

Change of mental models through instruction From an instructional point of view, Snow (1990) integrated mental models into a more comprehensive framework. He defined the mental model progression as a specific kind of learning-development transition between preconceptions, defined as initial states of the learning process, and causal

410 explanations, defined as the desired end states of learning. This conception largely corresponds with Piaget’s epistemology presupposing that learning always involves dynamic modifications of cognitive structures in the aforementioned sense. From an instructional perspective, mental models are dynamic structures created to solve problems and answer questions in learning situations. As cognitive artifacts, they are both created from prior knowledge and they are also constrained by it insofar as the construction of a mental model heavily depends on the existence of stored individual experiences as well as on the competence to both retrieve this knowledge and to apply it in accordance with the requirements of a situation. Moreover, the construction of a mental model also depends on the individual’s ability to reason analogically and to transfer knowledge to new but similar tasks (cf. Collins & Gentner, 1987). That means, the model-constructing individual must be able to recognize when a given task or situation has some similarities with prior experiences. In this case, the individual can apply a Schema which includes these experiences, and which provides the fundamental basis for analogical reasoning. Accordingly, schemas are often considered the building blocks of mental modeling (cf. Norman, 1983). Following Piaget (e.g. 1950), it has been argued that schemas emerge from the abstraction of imitating and constructive activities with object-related nature into the space of mental representations (cf. Wetzel, 1980). As a result of such mindful abstraction a schema involves both the relevant domainspecific knowledge and those operations and procedures that are necessary for constructive cognitive activities such as analogical reasoning as a prerequisite for the construction of a mental model. However, from the instructional point of view, this inductive way of constructing and successively completing a mental model qua analogical reasoning and ‘fleshing out’ (i.e. a procedure which examines the ‘truth’ of a model by means of a reductio ad absurdum) is only one side of the coin. The other one consists in the instructional method to provide learners with an appropriate conceptual model of the facts to be causally explained. In this case, the mental model results from the learner’s ability to adopt other people’s explanations in a semantically sensitive manner. However, this presupposes that the learner is sensitive to characteristics of the learning environment, such as the availability of certain information at a given time, the way this information is structured and mediated, and the ease with which it can be found in the environment. Such model-related learning heavily depends on (a) the learner’s domain-specific knowledge and related cognitive structures, (b) the nature of the material to be learned, (c) the modality in which the contents to be learned are presented and delivered by media, and (d) the method of teaching applied.

411 Clearly, there might exist learning environments that can initiate learning in the sense of free explorations by invention, but in instructional contexts well-designed learning environments predominate which constrain the students’ learning processes to various extents. Nevertheless, there are different opportunities to enhance exploratory learning and problem solving through instruction aiming at (a) self-guided discovery learning (cf. Hammer, 1997), (b) guided discovery learning (in the sense of Bruner, 1966), or (c) receptive meaningful learning (in the sense of Ausubel, 1968). With respect to the construction of mental models, discovery methods evidently constrain learning and problem solving less than expository teaching methods aiming at receptive meaningful learning. In guided discovery, for example, the learner searches continuously for information in the learning environment in order to complete or stabilize effective mental models. In this case, mental models are more proactive and direct the learning experiences so that the result of learning is dependent on the initial model, defined here as the learner’s ‘a priori understanding’ of the material to be learned. In expository teaching, a teacher explicitly directs the mental model progression by providing, usually as a first step, an explicit conceptual model which should direct the learner’s comprehension of the learning material and may evoke those conceptual changes as they are intended in the learning-dependent progression of mental models from preconceptions to causal explanations. Both guided-discovery and expository teaching are geared toward conceptual changes on the learners’ part. However, since preconceptions can be stable and widespread among students they can be strongly held and resistant to change. Consequently, a substantial conceptual change often does not take place rapidly, and relatively stable intermediate states of causal understanding often precede the instructionally intended conceptual change (cf. Galili et al., 1993). Incidentally, Briggs (1990) demonstrated in a case-study that an instructional strategy aiming at discovery learning may dramatically increase the probability of stabilizing false initial mental models. Therefore, the idea of several authors (such as Norman, 1983) to provide students, especially novice learners, with a designed conceptual model acquires some instructional appeal. Actually, it might be easier for a novice learner to assimilate a given causal explanation (provided through a conceptual model) rather than to induce one individually. In this case, the provided conceptual model will be functionally incorporated and related information can be successively integrated in a more or less consistent manner. However, this procedure of an instructionally guided generation of mental models raises several questions: 1. When the exposure to a designed conceptual model should enable the learner to adopt the delivered causal explanation of phenomena, one must take into account that students may dynamically modify and refine

412 their knowledge structures and attitudes when they evaluate the externally provided information as more plausible and convincing than their prior knowledge (cf. Dole & Sinatra, 1998). Therefore, the question comes up how permanent and stable are such mental models which are generated immediately after a learner’s early exposure to a conceptual model. 2. More basically, it might be asked whether the students will actually adopt a provided conceptual model as a fundamental basis or starting point for the construction of mental models. In accordance with the situated cognition approach it can be argued that learners will extract that information from the learning environment which they consider necessary or helpful to construct idiosyncratic mental models that can but must not correspond to various degrees with the provided conceptual model. It can be argued that the similarity between a provided conceptual model and a learner’s mental model depends primarily on the similarity of the experienced learning situations and their requirements. The aforementioned questions were investigated in an exploratory study embedded in a more comprehensive research project1 concerning the guided construction of mental models in the domain of economics. More specifically, the research focused on the effectiveness of a computerized learning environment designed according to the principles of the Cognitive Apprenticeship framework exemplified by Collins et al. (1989). The research is explicitly intended to investigate the learning-dependent progression of mental models over a long period and in a complex field of interest.

Method Participants Nineteen 12th grade students (10 females and 9 males) of a more comprehensive study participated in this exploration study on a voluntary basis. These volunteers were selected because they had achieved better than average results in previous studies, especially with regard to the results of verbal protocols (cf. Al-Diban & Seel, 1999). The learning environment In order to meet the critics of Royer et al. (1993) and Snow (1990) that educationally-relevant mental model research previously has been done piece-meal, in small scale, specialized contexts and limited learning situations, in the study to be reported here the subject matter domain of economics and dynamic economical systems, respectively, were chosen due to their

413

Figure 1. The structure of the learning environment according to the cognitive apprenticeship methods including the assessment strategy.

complexity. To narrow down the field of economics, two topics were selected to be taught through the multimedia-based learning software Financial Systems (http://www.ezw.uni-freiburg.de/forschung.html). The topic area was the financial politics of the Federal Bank of Germany, including (a) the whole economic system, and (b) the forthcoming European monetary union, because the introduction of the EURO provides the wider context for German financial politics. Since 1994, a series of evaluation studies has been carried out in order to improve the instructional effectiveness of the multimedia program Financial Systems which is designed according to the principles of Cognitive Apprenticeship (cf. Al-Diban & Seel, 1999; Seel et al., 2000). Figure 1 gives an overview of the program’s structure and methods. On average, successful learning with this program requires about six hours.

414

Figure 2. A dynamic model of an open national economy.

Material Macroeconomic models provide good examples for complex dynamic systems (cf. Bagozzi, 1980) that contain at least two variables at two different times that are related causally: Y(t) = f(Y(t – 1), . . . , Y(t – k)). That means, at time t, vector Y causally depends on the former states of Y between times t – 1 and t – k. The system’s dynamics consist in the fact that states or interventions in former times affect the state of Y at point t. In Figure 2, a dynamic model of an open national economy is described. A more comprehensive specification of dynamic systems and a description of their mathematical characteristics has been provided by Hübner (1989). Figure 2 also gives an example of the type of conceptual models applied in the multimedia program Financial Systems. As part of a more comprehensive evaluation study, the degree to which such conceptual models may influence the construction of the subjects’ mental models, in the course of learning with the multimedia program, has been explored. Two different teaching strategies, an explanation-based and an example-based strategy, have been experimentally varied in order to investigate their influences on the subjects’ constructions of mental models (see, for more details, Al-Diban & Seel, 1999; Seel et al., 1998).

415 Design Up to now, four evaluation studies have been carried out in order to investigate the learning effectiveness of the multimedia program Financial Systems. These studies were, to a large extent, exploratory due to the novelty of research on the generation of mental models by using multimedia and due to a lack of knowledge in this field that could lead to statistically testable hypotheses. Moreover, these studies were designed as replication studies that aimed both at the empirical evaluation of the multimedia program and the assessment of mental modeling during the multi-session learning phases. Results of these investigations are reported in Al-Diban and Seel (1999), Seel et al. (1998). In the fourth evaluation study the research focus was extended to the learning-dependent progression of mental models insofar as the long-term stability of mental models was investigated while testing the students four months after the multi-session learning phase. In order to assess substantial changes in the constructed mental models as well as those in domain-specific knowledge a pretest-posttest design with multiple stages of testing was applied with different test procedures (see also Figure 1). Assessment procedures The diagnosis of the learning-dependent progression of mental models included several test procedures. First, the domain-specific declarative knowledge was measured with a test developed by Beck (1993). Secondly, causal diagrams were applied for the assessment of mental models. In the past, causal diagrams have been used effectively in order to illustrate how complex systems work (cf. Hale & Barsalou, 1995) and to investigate knowledge acquisition and application in steering dynamic systems (cf. Funke, 1985; Müller, 1993). These authors consider causal diagrams as suitable instruments for the assessment of a learner’s subjective assumptions about the causality of a dynamic system as well as appropriate representations of the learner’s mental model of a dynamic system. In the exploratory study reported here, the subjects had to draw either at a flip-chart-wall or on a sheet of paper the causal relationships between the main sectors of economics (see, for example, Figures 4 and 5) by differentiating between causes, aims and processes. In order to achieve a high degree of comparability of the learning results as well as a reliable indicator of the long-term stability of mental models the subjects were asked to draw causal diagrams during the learning phase Reflection as well as four months later. Additionally, techniques of receptive interviews (Kleinig, 1991) and ‘teach back’ procedures (Sasse, 1991) were applied to produce verbal data

416 with regard to the subjects’ mental models. The technique of receptive interview was especially applied in order to assess the students’ preconceptions, and the ‘teach back’ procedure was applied in the last part of the study as a means of realizing metacognitive control in articulation and reflection. Results According to the research questions of the project this exploratory study was designed to obtain empirical data with regard to both the learning-dependent mental model progression and the effectiveness of providing students with conceptual models of dynamic systems. The different assessment methods produced interesting results. Preconceptions First, the students’ preconceptions were assessed by asking the students in a receptive interview to explain causally which effects particular decisions of the Federal Bank of Germany (e.g. lowering of the discount and Lombard rate) may have on the macroeconomics of Germany. Only a few students were able to give adequate explanations on this task. A representative example to demonstrate the quality of the students’ preconceptions about economics are the verbal explanations of subject #401 which are reproduced in Figure 3. In general, the verbal data of the receptive interviews of all subjects indicated a substantial inability to explain the economic functions of the Federal Bank of Germany. We hypothesized that this inability could be explained through a lack of domain-specific prior knowledge but the results of the knowledge pretest (Beck, 1993) revealed a relatively good prior knowledge about economics (Mean = 14.4, SD = 2.5, Xmax = 22). Pretest-posttest-retention test In order to assess the effectiveness of the multimedia program on the acquisition of domain-specific knowledge the knowledge test of Beck was administered again after the completion of the program (post-test) and four months later (stability test). Concerning the posttest scores the Mean was 16.6 with a SD = 3.6. The corresponding statistics for the stability test were Mean = 16.8 and SD = 3.2. Significant differences could be found with respect to the differences between the pretest scores with both the posttest and stability test scores whereas the differences between posttest and stability test scores were not significant. Evidently, the multimedia program improved the domain-specific knowledge, and this improvement revealed itself to be relatively stable over a period of more than four months.

417

Figure 3. Preconception of subject #401 assessed through a receptive interview before learning with the multimedia program.

In accordance with Beck’s (1993) prescriptions, the quality of the domainspecific knowledge was differentiated by applying the dimensions of Bloom’s (1972) taxonomy of educational objectives. This differentiation is a central part of the validation procedure of Beck’s test and facilitates a classification of declarative knowledge. Table 1 summarizes the results of this analysis. Table 1 indicates a significant improvement between the posttest and the stability test in the dimensions Comprehension and Analysis whereas in Application a tendency for a decrease could be observed. This particular effect on the comprehension of economic facts and relationships supports the suggestion that the program had a strong influence on the learners’ causal understanding of economics. In a next step, this assumption was examined in a more qualitative manner with the methodology of causal diagrams. Causal diagrams The method of causal diagrams (i.e. a particular ‘structure-spreadingtechnique’) was applied in order to assess the constructed mental models

418 Table 1. Results of the data analysis according to Bloom’s taxonomy. I, II, III, IV, V = Knowledge, Comprehension, Application, Analysis, Evaluation; KTB = Pretest, Posttest, Stability, p = asymptotic significance between different treatment groups according to the Friedmann-Test, M = Mean of percentages, SD = Standard Deviation, N = 19 Dimension (Bloom)

Time of assessment, i.e. part in KTB

I Knowledge

Pretest Posttest Stability

II Comprehension

Chi2

P

N = 19 M SD

2,6

0.26

86,8 73,7 71,1

22,6 30,6 34,6

Pretest Posttest Stability

14,2

0.00

38,2 62,5 68,4

14,7 20,4 15,4

III Application

Pretest Posttest Stability

11,1

0.00

57,9 75,9 70,2

13,8 17,9 18,1

IV Analysis

Pretest Posttest Stability

3,7

0.15

54,9 50,4 66,3

19,8 18,1 18,9

V Evaluation

Pretest Posttest Stability

1,0

0.61

45,6 36,8 35,1

29,8 29,2 30,4

during the learning process as well as their stability over four months after completing the multimedia program. In order to assess the structural quality of the causal diagrams produced at both measurements a global index, the ‘Goodness of Causal Diagrams’ (GCD), developed by Funke (1985) was adopted. The formula for the GCD is: GCD = Ci Ri /Cmax Rmax − Fj Sj /Cmax Rmax with Cmax and Rmax > 0. Ci is the number of correct nodes (economic concepts) and Ri of the correct relations, whereas Fj is the number of false concepts and Sj of incorrect relations between nodes of a diagram. Cmax and Rmax correspond to the maximum numbers of correct concepts and relations in expert model.

419 The expectancy value of GCDa is E(GCDa ) = 1/Rmax E(Ci Ri ) – 1/Rmax E(Fj Sj ) with E(Ci Ri ) = pi Cmax Rmax, and pi as the probability of an accidental effect. For the causal diagrams after the completion of the learning phase (= posttest) a GCDposttest = 0.72 with a variance of V = 11.6 resulted. Evidently, the subjects have been able to put down correctly more than 70% of the relevant economic sectors and relations into their causal diagrams, whereas they achieved in the stability test (i.e. four months after the program’s completion) only a GCDstability = 0.54 with a variance of V = 18.8. Thus, the structural quality of causal diagrams decreased significantly (chi2 = 30.48, p < 0.05) with regard to both nodes and relations although the subjects were able to produce correctly more than 50% of the economic concepts and relationships in the stability test. Beyond this, the coefficient of contingency for repeated measures was computed for both measurements with C = 0.23. Evidently, the causal diagrams constructed during and after learning with the multimedia program were revealed as similar to a certain degree, but they did not correlate significantly. These results support the assumption that the causal diagrams constructed at different times of measurement are representations of those mental models which the subjects have constructed in the related test situations. According to the different demands of these situations the subjects’ causal diagrams turned out to be different, too. This can be demonstrated with two individual examples. Selected examples Subject #206, the first example, developed in the posttest and stability test different causal diagrams as illustrated in Figure 4. Concerning the program’s effectiveness on the acquisition of declarative knowledge, subject #206 did not improve significantly her knowledge in both the posttest and the stability test. However, as her causal diagrams show this student was able to recall most of the economic concepts taught in the program Financial Systems and to interrelate them in a correct manner. She began with a sketch and continued with distinct examples whereby the first ones were strongly influenced by her personal experiences. Several statements and the sketch about the sectors of the national economy showed that this learner continuously made references to the ‘systemic cycle model’ presented in the multimedia program, but not in such a way that the causal diagrams produced did not look like the conceptual models applied in the program Financial Systems. On the whole, subject #206 showed a deep understanding of multicausal and systemic aspects of the field of economics. The causal diagram produced in the stability test indicated relatively stable results concerning the subject’s concepts of economics, and she evidenced a

420

Figure 4. Causal diagrams generated by subject #206 in the posttest and the stability test.

421 substantial knowledge of multicausal and systemic relations. Interestingly, this diagram proved to be more detailed and complete with regard to the named causes, aims and processes involved than did the causal diagram produced in the posttest. Accordingly, the GCDstability was 0.69, whereas the GCDposttest was 0.52. Thus, this subject was not only able to reconstruct four months later a more detailed view of important factors of economic systems but she also made several attempts to ‘run different economic aspects in the mind’s eye’, insofar as there were five explicit attempts at simulations concerning the processes ‘state supporting programs’, ‘measures of the Federal Bank of Germany’, ‘interest rates of private banks’, ‘employment rates’, and ‘basis of one’s livelihood’. In each case, a close look at positive and negative shapings of these processes and their causes and consequences became evident. The causal diagram in the posttest included only three such attempts at simulations. Subject #401, the second example, started also with good prior knowledge about economics (Xpre = 14 in Beck’s test), and he could increase this knowledge in the posttest (Xpost = 17) and even more in the delayed stability test (Xstability = 20). After the completion of the learning process, this subject produced a nearly correct causal diagram which was, however, quite dissimilar to the conceptual models taught. This holds true also with respect to the causal diagram in the stability test. A comparison of the causal diagrams of subject #401 (see Figure 5) shows that the mental representations of this learner obviously involved essential factors and important relationships of economic cycles. The economic components and relations that subject #401 described in both causal diagrams were clearly structured, but grounded first of all on personal experiences. Both causal diagrams had a sufficient quality, namely GCDposttest = 0.65 and GCDstability = 0.38. In his long-term causal diagram this subject demonstrated the ability to explain basic functions and processes of economics. He could also connect the relevant sectors of the national economy, including the Federal Bank of Germany and its tasks. Moreover, there are some reasons to assume that subject #401 was also involved in simulations of the mental model concerning ‘money supply’ and ‘purchasing power’ in the posttest and ‘local taxes’ in the stability test.

Discussion Mental models can be considered a central theoretical construct of the situated cognition approach since it is assumed that the learner constructs a mental model in order to simulate relevant properties of the situation to be cognitively mastered. An instructionally far-reaching assumption of the

422

Figure 5. Causal diagrams developed by subject #401 in the post- and stability test.

423 situated cognition approach is that these cognitive artifacts are not fixed structures of the mind, but are rather constructed when needed – for example, in order to master a learning situation with its specific demands. In accordance with this, numerous studies in the research field of conceptual change (cf. Dole & Sinatra, 1998) indicate that students dynamically modify and restructure their knowledge bases when externally provided information is evaluated as more plausible and convincing than their prior knowledge. However, this depends to a large extent on externally provided information, for example in the form of a designed conceptual model, aiming at the facilitation and improvement of constructing an appropriate mental model to master the learning situation and its cognitive demands. Taking into account that instruction generally aims at the construction of stable knowledge structures and elaborated cognitive skills, the reported exploratory study investigated how permanent and stable those mental models are which have been generated after a student’s early exposure to a designed conceptual model. Actually, the investigations indicated that subjects who worked with the multimedia-program Financial Systems showed only a small tendency to adopt the designed conceptual models provided especially within the modeling-component of the learning program according to the principles of Cognitive Apprenticeship framework. However, this observation corresponds only partially with the assumption that in the case that no relevant preconceptions are available learners tend to adopt provided conceptual models in order to construct idiosyncratic mental models of the tasks they are trying to learn. Accordingly, the subjects in our exploratory study drew causal diagrams which indicated varying degrees of similarity to the provided conceptual models. Among the different methods of assessment the causal diagrams were revealed to be a highly appropriate procedure to assess mental models. Nevertheless, referring to Seel’s (1999b) theoretical positions concerning the educational diagnosis of mental models, several problems of accurately assessing abstract theoretical constructs such as mental models by means of ‘structure-spreading-techniques’ and drawings remain unsolved, especially with regard to the validity of these methods. However, when we take into consideration both the results of this exploration and the results of previous evaluation studies of the multimedia program Financial Systems there are good reasons to assume that most students applied initially constructed mental models in order to master the learning tasks of the program. From our point of view, the subjects’ initially constructed mental models became increasingly stable although they evidently changed after the learning period in accordance with the requirements of the learning situations. Thus, we can assert that mental models are acquired with significant properties of learning situations and the learners’ interactions

424 with them. Moreover, we can basically argue that effective mental models are stored in memory as effective means for having mastered a specific situation, but they never will become ‘sleeping copies’ which can be reproduced in the case of need. The results of the stability test clearly show that mental models have been generated on the basis of the learner’s former experiences with similar situations and that they served as a means to master the new situation. Considering mental models as temporary structures built by a learner when confronted with a new situation that cannot be mastered with available conceptual structures implies that a mental model is really not a fixed permanent structure of the human mind. Therefore, we cannot suppose a factual persistence of a mental model over time, but rather more plausible is the assumption of a situation-dependent reconstruction of previously generated mental models. This fits in with the observation of the subjects’ slightly changed mental models after four months. Obviously, these mental models assessed through causal diagrams have met the requirements of the new situation which in turn closely resembled the former learning situations. Thus, we can state a successful construction of a mental model as a cognitive artefact that organizes the learner’s knowledge as well as it produces plausibility concerning the task to be mastered cognitively. The research reported here, offers only a snapshot of a huge and complex learning phenomenon. We still have to confirm and elaborate the assumptions drawn from the theory of mental models, and we also have to do more hypothesis-testing research in this field of interest. At the moment, the research group is performing a more extensive and systematic study to find better and more precise answers to challenging questions concerning the construction and application of mental models in learning situations. This research is oriented to the following statement: “The primary virtue a model must have if we are to learn from its failures is that it, and the experimental and heuristic tools we have for analyzing it, are structured in such a way that we can localize its errors and attribute them to some parts, aspects, assumptions, or subcomponents of the model. If we can do this, then ‘piecemeal engineering’ can improve the model by modifying its offending parts” (Wimsatt, 1987, p. 30).

Acknowledgement We gratefully acknowledge financial support for this research from a generous grant provided by the German Research Association (Deutsche Forschungsgemeinschaft) with Grant-No. Se 399/4.

425 References Al-Diban, S. & Seel, N.M. (1999). Didaktisches Design multimedialer Lernsysteme als Forschungsaufgabe die “Cognitive Apprenticeship” auf dem Prüfstand. Unterrichtswissenschaft 27: 1, 29–60. Anderson, J.R., Reder, L.M. & Simon, H.A. (1996). Situated learning and education. Educational Researcher 25 (4): 5–11. Ausubel, D.P. (1968). Educational Psychology. A Cognitive View. New York: Holt, Rinehart and Winston. Bagozzi, R.P. (1980). Causal Models in Marketing. New York: Wiley. Barwise, J. (1981). Scenes and other situations. The Journal of Philosophy 8: 369–397. Beck, K. (1993). Dimensionen der ökonomischen Bildung. Messinstrumente und Befunde. Nürnberg: Universität (Abschlussbericht zum DFG-Projekt ’Wirtschaftskundlicher Bildungs-Test [WBT]. Normierung und internationaler Vergleich). Bloom, B.S., ed. (1972). Taxonomie von Lernzielen im kognitiven Bereich. Weinheim: Beltz. Briggs, P. (1990). The role of the user model in learning as an internally and externally directed activity. In D. Ackermann & M.J. Tauber, eds, Mental Models and Human-Computer Interaction 1, pp. 195–208. Amsterdam: Elsevier Publications. Bruner, J. (1966). Toward a Theory of Instruction. Cambridge, MA: Harvard University Press. Bruner, J. (1990). Acts of Meaning. Cambridge, MA: Harvard University Press. Collins, A., Brown, J.S. & Newman, S.E. (1989). Cognitive apprenticeship: Teaching the crafts of reading, writing, and mathematics. In L.B. Resnick, ed., Knowing, Learning, and Instruction, pp. 453–494. Hillsdale, NJ: Lawrence Erlbaum Associates, Publishers. Collins, A. & Gentner, D. (1987). How people construct mental models. In D. Holland & N. Quinn, eds, Cultural Models in Language and Thought, pp. 243–265. Cambridge: Cambridge University Press. Craik, K.J.W. (1943). The Nature of Explanation. Cambridge: Cambridge University Press. Dinter, F.R. (1993). Mentale Modelle als Konstrukt der empirischen Erziehungswissenschaft. Saarbrücken: Universität (Dissertation). Dinter, F. (1998). Zur Diskussion des Konstruktivismus im Instruktionsdesign. Unterrichtswissenschaft 26 (3): 254–287. Dinter, F.R. & Seel, N.M. (1994). What does it mean to be a constructivist in I.D.? An epistemological reconsideration. In J. Lowyck & J. Elen, eds, Modelling I.D.-Research, pp. 49–66. Leuven: Proceedings of the first workshop of the SIG on instructional design of EARLI. Dole, J.A. & Sinatra, G.M. (1998). Reconceptualizing change in the cognitive construction of knowledge. Educational Psychologist 33 (2/3): 109–128. Engeström, Y. (1991). Non scholae sed vitae discimus: Toward overcoming the encapsulation of school learning. Learning and Instruction 1 (3): 243–259. Funke, J. (1985). Steuerung dynamischer Systeme durch Aufbau und Anwendung subjektiver Kausalmodelle. Zeitschrift für Psychologie 193 (4): 443–465. Galili, I., Bendall, S. & Goldberg, F. (1993). The effects of prior knowledge and instruction on understanding image formation. Journal of Research in Science Teaching 30 (3): 271–301. Greeno, J.G. (1989). Situations, mental models, and generative knowledge. In D. Klahr & K. Kotovsky, eds, Complex Information Processing, pp. 285–318. Hillsdale, NJ: Lawrence Erlbaum Associates, Publishers. Hale, C.R. & Barsalou, L.W. (1995). Explanation content and construction during system learning and troubleshooting. The Journal of the Learning Sciences 4 (4): 385–436.

426 Halford, G.S. (1993). Children’s Understanding. The Development of Mental Models. Hillsdale, NJ: Lawrence Erlbaum Associates, Publishers. Hammer, D. (1997). Discovery learning and discovery teaching. Cognition and Instruction 15 (4): 485–529. Hemmer, K.P. & Zimmer, J. (1975). Der Bezug zu Lebenssituationen in der didaktischen Diskussion. In K. Frey, ed., Curriculum Handbuch, Band II, S. 188–201. München: Piper. Hübner, R. (1989). Methoden zur Analyse und Konstruktion von Aufgaben zur kognitiven Steuerung dynamischer Systeme. Zeitschrift für Experimentelle und Angewandte Psychologie 36 (2): 211–238. Johnson-Laird, P.N. (1983). Mental Models. Towards a Cognitive Science of Language, Inference, and Consciousness. Cambridge: Cambridge University Press. Kleinig, G. (1991). Das qualitative Experiment. In O. Flick, E. von Kardorff, H. Keupp, L. von Rosenstiel & S. Wolff, eds, Handbuch Qualitative Sozialforschung, S. 263–266. München. Klix, F. (1971). Information und Verhalten. Bern: Huber. Mayer, R.E. (1989). Models for understanding. Review of Educational Research 59 (1): 43–64. Morrow, D.G., Bower, G.H. & Greenspan, S. (1989). Updating situation models during narrative comprehension. Journal of Memory and Language 28: 292–312. Müller, H. (1993). Komplexes Problemlösen: Reliabilität und Wissen. Bonn: Holos Verlag. Norman, D.A. (1983). Some observations on mental models. In D. Gentner and A.L. Stevens, eds, Mental Models, pp. 7–14. Hillsdale, NJ: Erlbaum. Peirce, Ch.S. (1973). Lectures on Pragmatism. Vorlesungen über Pragmatismus. (Ed. E. Walther). Hamburg. Piaget, J. (1950). La Construction du Réel chez l’Enfant. Neuchâtel, Suisse: Delachaux et Niestlé S.A. Renkl, A., Mandl, H. & Gruber, H. (1996). Inert knowledge: Analyses and remedies. Educational Psychologist 31 (2): 115–121. Resnick, L.B. (1987). Learning in school and out. Educational Researcher 16 (9): 13–20. Reynolds, R.E., Sinatra, G.M. & Jetton, T.L. (1996). Views of knowledge acquisition and representation: A continuum from experience centered to mind centered. Educational Psychologist 31 (2): 93–104. Royer, J.M., Cisero, C.A. & Carlo, M.S. (1993). Techniques and procedures for assessing cognitive skills. Review of Educational Research 63 (2): 201–243. Sandberg, J. & Wielinga, B. (1992). Situated cognition: A paradigm shift? Journal of Artificial Intelligence in Education 3: 129–138. Sasse, M.A. (1991). How to t(r)ap users’ mental models. In M.J. Tauber & D. Ackermann, eds, Mental Models and Human-Computer Interaction 2. Amsterdam: Elsevier Science Publ. Seel, N.M. (1991). Weltwissen und mentale Modelle. Göttingen: Hogrefe. Seel, N.M. (1999a). Semiotics and Structural Learning Theory. Journal of Structural Learning and Intelligent Systems 14 (1): 11–28. Seel, N.M. (1999b). Educational diagnosis of mental models. Assessment problems and technology-based solutions. Journal of Structural Learning and Intelligent Systems 14 (1): 153–185. Seel, N.M., Al-Diban, S., Held, S. & Hess, C. (1998). Didaktisches Design multimedialer Lernumgebungen: Theoretische Positionen, Gestaltungsprinzipien, empirische Befunde. In G. Dörr & K.L. Jüngst, eds, Lernen mit Medien. Ergebnisse und Perspektiven zu medial vermittelten Lehr- und Lernprozessen, S. 87–119. Weinheim: Juventa. Seel, N.M., Al-Diban, S. & Blumschein, P. (2000). Mental models and instructional planning. In M. Spector & T.M. Anderson, eds, Integrated and Holistic Perspectives on Learning,

427 Instruction and Technology: Understanding Complexity, pp. 129–158. Dordrecht: Kluwer Academic Publishers. Seel, N.M. & Winn, W.D. (1997). Research on media and learning: Distributed cognition and semiotics. In R.D. Tennyson, F. Schott, S. Dijkstra & N.M. Seel, eds, Instructional Design Internantional Perspectives. Volume 1: Theories and Models of Instructional Design, pp. 293–326. Hillsdale, NJ: Lawrence Erlbaum Associates, Publishers. Snow, R.E. (1990). New approaches to cognitive and conative assessment in education. International Journal of Educational Research 14 (5): 455–473. Stachowiak, H. (1973). Allgemeine Modelltheorie. Wien: Springer. Strauss, S. (1996). Confessions of a born-again constructivist. Educational Psychologist 31 (1): 15–21. Wetzel, F.G. (1980). Kognitive Psychologie. Eine Einführung in die Psychologie der kognitiven Strukturen von Jean Piaget. Weinheim: Beltz. Wimsatt, W.C. (1987). False models as means to truer theories. In M. Nitecki, ed., Neural Models in Biology, pp. 23–55. London: Oxford University Press. Wittgenstein, L. (1969). Philosophische Untersuchungen. In G.E.M. Anscombe & G.H. von Wright, eds, Schriften von Ludwig Wittgenstein, Band 1, S. 278–544. Frankfurt/M.