New Generation Computing, 21(2003)3-12 Ohmsha, Ltd. and Springer-Verlag

GEIE~tIDN COMPUTING @Ohmsha, Ltd. 2003

Metacognitive and Computational Aspects of Chance Discovery Ruediger OEHLMANN School of Computing and Information Systems, Kingston University, Kingston upon Thames, KTI 2EE, UK R.Oehlmann~kingston.ac.uk

Received 26 February 2002

Abstract

Chance discovery is concerned with events or situations that affect human decision making; such events or situations are viewed as opportunities or risks. Perspectives are mental representations that describe partial knowledge of a task domain (cognitive perspective) as well as knowledge about other participants (social perspectives). Based on verbal protocols and a computational model of these protocols, it is argued that perspective taking is a suitable strategy to achieve chance discovery. Therefore the cognitive mechanisms underlying this strategy have been investigated and the results implicate metacognition as necessary requirement to achieve chance discovery. Keywords:

§1

Chance Discovery, Metacognition, Perspective Taking, Computer Model.

Introduction

In a number of different areas, researchers became recently interested in events or situations that affect human decision making in that they are viewed as opportunities or risks. Such events or situations have been labelled as chance and noticing them as discovery. Chance discovery has been characterized in terms of "awareness of a chance and the explanation of its significance." 13) The studies from psychology and artificial intelligence described in this paper have focused on a particular reasoning strategy, perspective taking. Perspectives can be viewed as mental representations that describe partial knowledge of a task domain, including causal links between knowledge components, (cognitive perspective) and as representations about other participants (social perspectives) including knowledge about how these participants might see the world. As the social aspects of perspective taking are beyond the scope of this

4

R. Oehlmann

paper, we will only consider the cognitive perspectives. Changing the way we look at a problem or a task in a suitable way may lead to a chance for a new and better solution. 14 ) It also may lead to a better explanation, which is suitable to explain previous observations as well as a new observation that is inconsistent with previous explanations. Finally a perspective change may lead to a chance for designing new and better artifacts. 15) Investigations of cognitive perspectives have focused on the role of perspectives in problem solving and creativity. Ohlsson 12) has argued that an impasse can occur on a problem even if the problem solver is competent to solve it. This happens because the initial encoding of the problem does not activate the relevant competence in memory. Constructing a different representation of the problem can break the impasse. If the new representation is sufficiently different from the initial representation, memory activation will involve different memory components. The impasse is broken by taking a new perspective. More recently perspectives or viewpoints have been discussed in creativity research. For instance, Sugimoto, Hori and Ohsuga 15) described a system that supported creative insight by visualizing different viewpoints. A more systematic treatment of this idea can be seen in Nakakoji's7) concept of collective creativity that relies on the perspectives of different designers who have solved similar design problems in the past. This has been explored in the EVIDII system. 10,11) In contrast, Sternberg 14 ) investigated the creative behaviour of individuals and found that selective encoding and selective construction are among those processes that are crucial for creative insight. Selective encoding involves distinguishing between relevant and irrelevant information and hence establishing a perspective. Selective combination is concerned with the synthesis of isolated pieces of information to a unified whole. This type of evaluation and monitoring is related to metacognition and is consistent with experimental results, which suggest that human cognitive performance can be improved by the use of metacognitive skills such as self-monitoring, prediction and self-questioning. 16 ) The term metacognition has been used in different ways. 3) However, there is an increasing tendency to follow Flavell,2) who viewed metacognition as referring to the "active monitoring and consequent regulation" of the agent's own cognitive processes. As a result, we can regard the term metacognition as describing two distinct but related issues: the issue of knowledge about cognition and the issue of regulating cognition. The first term includes awareness of the agent's resources with respect to the demands of the agent's thinking; for example, the availability of analogous knowledge during an analogical mapping tasks. Oehlmann8 ) has suggested that this type of awareness is affected by attentional processes on a metacognitive level. Recently, Horiguchi and Hirashima 5) have related awareness to the notion of counter examples. The second issue involves self-regulating mechanisms such as planning and monitoring. 1) Such meta-cognitive skills should be improved with growing experience. Reasoners should be able to re-use such experience when they maintain mental models about metacognition. The remainder of this paper will focus on the abilities mentioned in the

Metacognitive and Computational Aspects of Chance Discovery

5

definition of chance discovery such as being able to become aware of a chance and to explain its significance and discuss them from a metacognitive perspective. Based on the analysis of verbal protocols, the next section will identify various stages of the process of chance discovery. It also will show where metacognition takes place and what effects it has. The subsequent section describes a computational model of the type of reasoning identified in the protocols. The program was able to demonstrate the same stages and metacognitive processes that were identified in the verbal protocols. Furthermore, it showed that awareness, monitoring and explaining can be achieved by metacognitive reasoning about goals.

§2

Metacognitive Aspects of Perspective Change

This section investigates the various stages of chance discovery and will focus particularly on those stages that require metacognition. The investigation involved the analysis of verbal protocols that were recorded while participants attempted to understand the mechanism of a mechanical clock. The analysis of the protocols focused on understanding the chime of the clock because this potentially involved the discovery of a chance for a perspective change and consequently for acquiring a deeper understanding of the mechanism. 2.1

Method

[1]

Design All participants were individually presented with the same task of understanding the mechanism of a mechanical clock mechanism.

[ 2 ] Subjects Participants were 16 university employees with a background in social sciences (age range 26 - 50 years, means 36.8 years).

[ 3 ] Materials Throughout the investigation, a mechanical clock was used after the case was removed so that subjects were able to observe every single wheel of the clock. The clock was equipped with two springs, one for the chime and one for the actual clock mechanism. Figure 1 shows the structure of the chime, the components of the clock which connect the chime and the clock mechanism. and the actual clock mechanism. The sequence of wheels that control the chime could be viewed as a branched path. The common part of the path originates at the spring that drives the hammer (hammer spring). Initially the spring drives a wheel referred to as hammer spring wheel. This wheel transfers the force of the spring via a number of wheels to the first intermediate hammer wheel. At this wheel the path is divided. One branch transfers the force via the hammer lever wheel to the hammer, whereas a second branch leads via the second intermediate hammer wheel to a wheel referred to as pin wheel. This is a vertical wheel with a horizontal pin attached. A small lever in contact with the pin stops the wheel and is released when the large hand points upwards. As the pin wheel is indirectly

6

R.Oehlmann

Ratchet

Chime

Chime Control

Small Clock

Clock

Hand,

Fig. 1 The Clock Domain

connected with the hammer, stopping the pin wheel results in interrupting the hammer movement. The lever movement is controlled by a mechanism, which in Fig. 1 is labelled as "Chime Control." This mechanism consists of a number of interacting levers and depends on the positions of the hands of the clock. Similarly, the arrangement of the wheels in the actual clock mechanism can be viewed as branched path. The common path leads from the spring that drives the hands of the clock (hand spring) to the large hand wheel. Here the path

Metacognitive and Computational Aspects of Chance Discovery

7

is split. One branch transfers the force of the hand spring to the hands of the clock. Another branch connects the large hand wheel with ratchet and pendulum. Again the second branch transfers the counterforce that in this case is provided by ratchet and pendulum to the large hand wheel and therefore balances the force that drives the hands of the clock. [ 4 ] Procedure 16 subjects were separately presented with the same mechanical clock after the case of the clock was removed. Verbal protocols of all interactions with the clock were recorded. The task presented to participants was to understand and to explain mechanism of the chime. 2.2

Result The protocols revealed that all subjects knew that the wheels transfer the force of the hammer-spring-wheel to the hammer. The movement of one wheel drives the movement of the neighbouring wheel. Subjects attempted to identify a chain of consequences responsible for the hammer movement. In the course of these considerations subjects performed "experiments." They set the large hand in the 12:00 position to start the chime. All subjects started with the perspective that the spring drives the hammer only at certain times. 12 participants discovered the wheel/lever arrangement and changed their perspective. During the experiments, these subjects observed that the two-wing wheel turned and noted that a lever is moved into the orbit of the pin-wheel. The lever touches the pin and interrupts the movement of the pin-wheel. The unexpected observation led to a change of the reasoning perspective. Rather than reasoning about the consequences of the movements of the pin-wheel, the 12 participants focused now on the consequences of the interruption of these movements. Subsequently, they considered the mechanism from the perspective that the spring would drive the hammer all the time and only the lever when in contact with the pin stops the mechanism from doing so. From the 12 protocols the following schema has been derived: 1. Evidence Phase

a. collecting evidence in support of the initial perspective b. making an observation that is inconsistent with the initial perspective 2. Awareness Phase

a. Noting inconsistency between the observation and the previous perspective b. Rejection of the new observation and explaining why there is no inconsistency c. Evaluation (making sense of the world by using accepted theories and realising that this is not possible given the new observation.)

3. Utilisation Phase

8

R.Oehlmann

a. Perspective change by using strategies such as generalisation or case-based type of analogies 4. Explanation Phase

a. Arguing for the new perspective in relation to the inconsistent observation

5. Elaboration Phase a. Elaboration of the new perspective b. Theory Change and extension of the new theory During the evidence phase participants showed the usual preference for supporting evidence, as observed by Klayman. 6 ) The awareness phase was characterised by participants maintaining for some time the initial perspective as well as the contradictory new observation without discarding either of them. This situation is extended in the utilisation phase where participants consider an alternative perspective that explains the new observation. Again for some time they maintained the initial perspective and the new perspective before finally committing themselves to the new perspective. Characteristics of this change included a weak explanation why the new observation should be discarded ("not relevant for the hammer mechanisms but for another part of the clock") followed by tentative attempts to explain the new observation. Whereas in the explanation phase the new perspective is considered in relation to the new observation, in the elaboration phase the perspective is extended and tested in relation to all known observations. In addition, the perspective is detailed to arrive at a new theory that explains all the observations.

2.3

Discussion The reported results suggest a correspondence between the ability to become aware of a chance and to explain its significance on one hand and the metacognitive aspects of awareness of an agent's resources and self-regulating mechanisms on the other. The protocols indicate that chance discovery takes place in the form of competition between alternative strategies of supporting an adopted perspective and of supporting a new contradictory observation. This competition between strategies led to an awareness of a chance as cognitive resource. It is of particular interest that participants were typically able to maintain several perspectives in parallel as long as they did not consider one of them as their accepted perspective. If this happened, reasoning about other perspectives did no longer take place. Finally, the significance of a chance results from using the observation as a cognitive resource, i.e., considering an observation as significant was not a pre-requisite for using the observation, but a consequence.

§3

A Computational Model of Chance Discovery

In the previous section, the main stages of chance discovery have been identified by analysing verbal protocols. This section will describe a computer architecture referred to as Chance Discoverer (CD) that is suitable to simulate

9

Metacognitive and Computational Aspects of Chance Discovery

...

~ Functional dependencies between process modules

--~">~

Sentence Strategy Planner

Data input and output

Introspection Planner

Fig.2 The CD Architecture

such protocols and hence the steps of chance discovery. The program uses planning of sentences such as questions, answers, hypotheses and factual statements to model reasoning. In addition, it is capable of planning experiments. The main task of the system is the generation of new explanations to revise an initial theory. Figure 2 shows the main modules of the system: sentence planner, sentence strategy planner, experiment planner and introspection planner. All these modules are implemented as case-based planners. 4) The sentence planner module accepts a problem description as input and generates a protocol to address the input problem. Depending on the input, it has several options such as generating a hypothetical statement or a question about the problem followed by an answer. Often answers and factual statements are generated by performing experiments and using the obtained information for the answer. Sometimes a statement cannot be generated because information is missing. Also this situation is addressed by performing additional experiments. During the process of sentence-based reasoning and experimentation-based activity, sentences focus on objects of the domain to be investigated such as pin-wheel and small-lever (Fig. 1). In addition, the sentence planner generates sentences that focus on the systems reasoning process. This type of sentence generation requires additional meta-knowledge that is not available to the sentence planner. Acquiring this knowledge is the task of the introspection planner. 8) Successful sequences of sentence plan executions can be cached in the form of a sentence strategy plan that is then executed by the sentence strategy planner. This approach of a two-layered plan execution provides an advantage in performance in that it reduces search time for various sentence plans as well as a model of sentence learning. The basic knowledge structures of the system are experiments and plans that are used as cases. An experiment consists of two components: an experimental setting (e.g. a partial description of a mechanical clock with wheels,

10

R.Oehlmann

springs and hands) and the result of an experiment such as the statement that the hammer of the chime is moving when the large hand is in upright position. Experiments are represented by objects and relations between objects. Objects are represented as Memory Units which contain an object frame, a content frame and a context frame. The object frame contains general information about the object. The content frame stores several sets of intentional descriptor values referred to as view, whereas relations with other objects are described in the context frame. Sentence plans are used to apply case-based planning techniques to the generation of single sentences. For instance, the question "What does the PINWHEEL turn?" can be built by combining the substructures "What", "does", "the OBJECTl" and "turn". OBJECTl is a variable which is instantiated with the string "PIN-WHEEL" during plan execution. A sentence plan has two main parts: a set of descriptors used for indexing the plan and a sequence of steps. A plan is retrieved by matching its index with the current situation; this is characterized by the goals the system pursues in asking the question. If plan execution fails, the usual explanation-based repair mechanism is employed. 4l It is an important advantage of the case-based planning approach that new sentences can be learned by modifying previous sentence plans. The case-based planning approach to sentence generation is highly flexible because it only depends on the current situation and the goals the system is attempting to pursue. Moreover, new plans can be generated by adapting existing plans to new situations. An introspective sentence refers to the reasoners internal knowledge and internal processes. A sentence plan that has to generate an introspective sentence contains special steps. Executing such as step results in a call of the introspection planner. This planning process provides the information needed by the sentence planner which can then complete the sentence. Introspection plans address different metacognitive tasks such as assessing goals, reasoning strategies, resources needed to perform a given reasoning strategy, reasoning failures that occurred previously and conditions that have to be satisfied in order that a strategy can be executed. In addition, the system uses experimentation plans to perform experiments. Experimentation plans describe the steps which have to be executed in order to perform an experiment. Executing an experimentation plan results in a call to a rule-based experiment simulator which is not part of the CD architecture but represents the environment of the cognitive model. The simulator returns an experimental result. The experimental setting and the result of a plan execution are stored as new case. The same basic plan structure used for sentence plans has been employed for experimentation plans, although the index vocabulary differs. Tests of the cognitive model have shown that from the generated sentences and experiments the same phases of chance discovery were emerging that were identified in the verbal protocols. Moreover, the program was able to demonstrate the same competitiveness between strategies and between perspectives as

Metacognitive and Computational Aspects of Chance Discovery

11

found among human reasoners. However, whereas the verbal protocols gave only implicit indications for metareasoning about goals, the cognitive model clearly showed that awareness, monitoring and explaining can be achieved by reasoning about goals. In order to clarify this goal-oriented aspect additional psychological experiments are needed.

Conclusion

§4

In the previous sections, it has been demonstrated that the strategy of perspective taking facilitates chance discovery. The analysis of verbal protocols and a computational model have been used to identify stages of the process of chance discovery. Two of these stages involved metacognition. In the awareness stage, the competing goals of maintaining the initial perspective and explaining an observation that is inconsistent with that perspective lead to an awareness of cognitive resources and, in this particular case, of an opportunity for perspective change. In the utilization stage, the reasoner utilises the opportunity by addressing two competing perspectives and finally accepting the new perspective. It is important to note that in both stages the reasoners maintain contradictory knowledge constructs for some time before they commit themselves to one of them. The results also suggest a correspondence between aspects of chance discovery, such as awareness of a chance and the explanation of its significance, and features of metacognition, such as awareness of cognitive resources and control of cognitive processes. Apparently metacognitive processes of awareness and control are required to facilitate the decisions to be made during chance discovery. This correspondence in turn suggests that the process of chance discovery may benefit from training on metacognitive strategies such as self-questioning and self-monitoring. It should however be noted that the current investigation has a important limitation. The current results do not consider the interactions between team members, because the social context and hence the second part of our definition of perspective taking was beyond the scope of this investigation.' 1 Future work will have to address interactions with collaborators to develop a complete theory of chance discovery.

References 1)

Brown, A., "Metacognition, Executive Control, Self-Regulation and Other More Mysterious Mechanisms," in Metacognition, Motivation and Human Performance (Weinert, F. and Kluwe, R., eds.), Lawrence Erlbaum Associates, Hillsdale, pp.65-116,1987

2)

Flavell, J., "Metacognitive Aspects of Problem Solving," in The Nature ofIntelligence (Resnick, L., ed.), Lawrence Elbaum Associates, Hillsdale, 1976.

3)

Gavelek, J. and Raphael, T., "Metacognition, Instruction, and the Role of Questioning Activities," in Metacognition, Cognition and Human Performance (ForstPressley, D., MacKinnon, G. and Waller, T., eds.), Academic Press, 1985.

4)

Hammond, K., Case-Based Planning, Academic Press, 1989 .

•1

An initial discussion of collaborative perspective taking is forthcoming. 9)

R. Oehlmann

12

5)

Horiguchi, T. and Hirashima, T., "The Role of Counterexamples in Discovery Learning Environment: Awareness of the Chance for Learning," in On-site Notes of The First International Workshop on Chance Discovery, 2001.

6)

Klayman, J. and Ha, Y., "Confirmation, Disconfirmation, and Information in Hypothesis Testing," Psychological Review, 94, pp. 211-228, 1987.

7)

Nakakoji, K., Ohira, M. and Yamamoto, Y., "Computational Support for Collective Creativity," Knowledge-based Systems Journal, Elsevier Science, 13, 7-8, pp. 451-458, 2000.

8)

Oehlmann, R., "Metacognitive Attention: Reasoning about Strategy Selection" in Proc. of the 17th Annual Conference of the Cognitive Science Society, 1995 (Moore, J. and Lehman, J., eds.), Lawrence Erlbaum Associates, Hillsdale, pp. 66-71, 1995.

9)

Oehlmann, R., "A Computational Model of Reasoning as Socially-Constructed Process," in Proc. of the Seventh Pacific Rim International Conference on Artificial Intelligence, 2000 (accepted for publication).

10)

Ohira, M., Yamamoto, K. and Nakakoji, K., "EVIDII; An Environment That Supports Understanding "Differences" among People," International Conference on Cognitive Science '99 (ICCS99), pp. 466-471, 1999.

11)

Ohira, M., Yamamoto, K. and Nakakoji, K., "EVIDII: A System That Supports Mutual Understanding through Visualizing Differences of Impressions," Journal of Information Processing Society of Japan, Special Issue on Knowledge and Information Sharing (in Japanese), 41, 10, pp. 2814-2826, 2000.

12)

Ohlson, St., "Restructuring Revisited. I. Summary and Critique of the Gestalt Theory of Restructuring and Insight," Scandinavian Journal of Psychology, 25, pp. 65-78, 1984.

13)

Ohsawa, Y., "Chance Discoveries for Making Decisions in Complex Real World," New Generation Computing, 20, 2, 2002

14)

Sternberg, R., "A Three-facet Model of Creativity," The Nature of Creativity (Sternberg, R., ed.), Cambridge University Press, pp. 125-147, 1988. Sugimoto, M., Hori, K. and Ohsuga, S., "A System for Supporting Researcher's Creativity by Visualizing Different Viewpoints," Workshop on Creativity and Cognition, Loughborough, 1996.

15)

16)

Wong, B., "Self-questioning Instructional Research. A Review," Review of Educational Research, 55, pp. 227-268, 1985.

Ruediger Oehlmann, Ph.D.: He is a senior lecturer in the Cognitive Science Laboratory, School of Computing and Information Systems, Kingston University, London. He received his degrees in Mathematics, Computer Science and Psychology. His doctoral thesis describes a model of discovery learning, which he has extensively tested using psychological experiments as well as computer programs. His current research interests include perspective taking, creativity, chance discovery and collaborative work in design domains. He is a member of the British Computer Society and the Cognitive Science Society.