User Modeling and User-Adapted Interaction 7: 1–55, 1997. c 1997 Kluwer Academic Publishers. Printed in the Netherlands.

1

A Model for Adapting Explanations to the User’s Likely Inferences HELMUT HORACEK ? Universit¨at Bielefeld, LILI-Fakult¨at, Postfach 100 131, D-33501 Bielefeld, Germany (Received: 8 June 1995; accepted in final form: 15 October 1996) Abstract. In order to generate natural, high quality textual presentations in technical domains, good explanations must not only be adapted to the knowledge attributed to the intended audience, but they must also take into account the inferential capabilities of the addressees. In this paper, we present a model for anticipating contextually-motivated inferences addressees are likely to draw. This model is used to motivate choices in presenting or omitting individual pieces of information; it takes into account the addressees’ domain expertise and expectations about logical consequences of purposefully presented information. Several kinds of empirical evidence are incorporated into a text planning process that aims at exploiting conversational implicature, so that a most suitable portion of the plan can be selected for being uttered explicitly. This way, our method adds to discourse planners based on Rhetorical Structure Theory (RST) the ability to omit easily inferable information. Thus, it overcomes one of the main shortcomings of RST. In the course of this process, rules anticipating user inferences are invoked to determine contextually justified derivability of information. In this manner, text variants can be composed on the basis of a text plan entailing annotations about the inferability of pieces of information. Moreover, pragmaticallymotivated preference criteria can be used to choose among several plausible variants. The model is formulated in a reasonably domain-independent way, so that the rules expressing aspects of conversational implicature can be incorporated into typical RST-based text planners. Key words: explanation, inference, natural language generation, stereotype user model.

1. Introduction Several computer programs, primarily those producing presentations in the form of natural language texts, have been designed to exhibit similar presentation skills as humans do, at least in some crucial aspects. These skills comprise the selection of a suitable presentation strategy, adequate content determination, and choices concerning perspective, terminology, and rhetorical devices. Presentation skills generally make use of situational contexts and adapt themselves to the needs of their addressees. Some methods worth mentioning incorporate devices addressing at least one, but maybe several of the aforementioned areas: the presentation of taxonomic definitions by taking the addressee’s knowledge into account (Paris 1988, Paris 1993), object descriptions focusing on the avoidance of wrong implications (Reiter, 1990), and the selection of an adequate perspective according to ? Present address: Universit¨at des Saarlandes, FB 14 Informatik, Postfach 1150, D-66041 Saar-

br¨ucken.

PREPROOF *INTERPRINT* J.V.: PIPS Nr.: 124531 MATHKAP user112.tex; 26/08/1997; 7:01; v.6; p.1

2

HELMUT HORACEK

the role of a person who needs advice – for instance, an ordinary user or a system debugger (Bateman, Paris 1989). In addition, particular emphasis is put on reactive explanation techniques for selecting appropriate content according to the contextually-motivated interpretation of requests (Moore, Swartout 1989) and for making use of content and discourse planning devices (Cawsey, 1990). However, the majority of systems developed to date share the implicit assumption that some information to be conveyed needs to be uttered explicitly in order to affect the hearer’s associated beliefs. While plan recognition systems generally perform an extensive inference analysis to understand the intention behind a piece of discourse (for instance, (Carberry, 1988; Litman, Allen, 1987)), generation systems mostly confine their rhetorical devices to direct the provision of information. In particular, this holds for text planners based on Rhetorical Structure Theory (e.g., (Moore, Paris, 1989)). The best known exceptions in generation mainly address the avoidance of false implicature, either by providing additional information to prevent the associated inferences (Joshi, Webber, Weischedel, 1984; Zukerman, McConachy. 1993), or by selecting alternative descriptions that do not carry wrong implications (Reiter, 1990). In contrast, some more recent approaches aim at exploiting implications entailed in text portions by omitting information from planned discourse that can be contextually inferred (Horacek, 1991; Lascarides; Oberlander, 1992, Zukerman, McConachy, 1993). As we will demonstrate in the next section, the assumption that information to be conveyed must be uttered explicitly may cause serious deficits in texts produced by these systems. In order to overcome these deficits, a system must exhibit several capabilities to select its presentation content:  Avoiding the presentation of redundant information, unless doing this would serve another communicative purpose, such as putting emphasis on a particular issue.  Maintaining coherence in the discourse it produces, as well as in cases where the system wants the user to believe some pieces of information that are implied, but not uttered explicitly.  Adapting its choices of expressing pieces of information explicitly or leaving them to be uncovered by the addressee’s inferential capability, according to evidence about the addressee’s domain knowledge and discourse preferences.

In two earlier papers (Horacek, 1991; Horacek, 1994), we have focused on the first two points; further technical details can be found in (Meier, 1991). This paper primarily emphasizes the last point. The method presented here is designed to answer questions about solutions proposed by an expert system. It is based on the system DIAMOD (Peters, 1993), which illustrates the behavior of OFFICE-PLAN (Karbach, Linster, Voß 1989). OFFICE-PLAN is able to appropriately assign a set of employees to a set of office rooms, guided by a number of constraints expressing various kinds of the people’s requirements.

user112.tex; 26/08/1997; 7:01; v.6; p.2

A MODEL FOR ADAPTING EXPLANATIONS

3

First, we discuss the motivations underlying our enterprise, which include limitations of current systems in producing natural discourse and empirical justifications for attacking these limitations. We follow with a description of the key ideas underlying our method, and we motivate crucial principles underlying reasoning about implicit information in discourse. When outlining the associated formalizations, we argue how the principles envisioned are met in the formal model. Then we demonstrate how inferences the addressee is expected to make can be incorporated into a text plan. The presentation of the role of user modeling and its exploitation for discourse planning in our model is preceded by a section in which we report about results of an inquiry carried out to gain evidence about human preferences in comparable situations. The achievements of our method are illustrated by discussing several ways to express a moderately complex text plan, depending upon presumed properties of the audience. We conclude with a comparison to other approaches and discuss some future prospects.

2. Motivation 2.1. LIMITATIONS OF CURRENT SYSTEMS The inadequacy of the assumption that all information conveyed needs to be uttered explicitly has become evident only recently when systems began to address deeper explanations with more involved content. Earlier systems either produced short discourse (Appelt, 1985), or they generated a set of well organized, comparably simple facts, whose implications were of minor or even of negligible importance (McKeown, 1985; Paris, 1988). In domains with inferentially rich discourse, a predominant assumption requires that all information content conveyed must be expressed explicitly, which frequently leads to redundant, tedious texts. Consider, for example, text (1a) in Figure 1 which has been generated by EES (Moore, Swartout, 1989), a system that aims at explaining improvement measurements applied to LISP programs. We believe that people would generally agree on the opinion that the underlying rationale can be better expressed in a concise way, by text (2a) or (3a) (see also (Carroll, 1990) for the benefit of minimalist manuals). These kinds of alternatives in providing explanatory information are quite common, as the texts on the right side of Figure 1 illustrate, which stem from the domain of office planning. In this field of application, employees have to be suitably assigned to rooms according to some given specifications. If a human expert is asked why his/her proposal contains the assignment of a certain person A to a particular room B, he/she may simply answer by text (2b) or by text (3b). In order to be comprehensible, answer (3b) presumes some shared knowledge about the room topology and some domain knowledge, that is, knowledge about the office planning issue on the side of the addressee. In any case, the expert would

user112.tex; 26/08/1997; 7:01; v.6; p.3

4

HELMUT HORACEK

Redundant (1), natural, concise (2, 3) and puzzling (4, 5) explanations, form the domains of Improving LISP programs – the EES domain

Office planning – our domain

‘Why should I replace Setq by Setf?’ (1a) ‘I’m trying to enhance the maintainability of the program by applying transformations that enhance maintainability, Setq-to-setf is a transformation that enhances maintainability.’ (2a) ‘Because we are applying transformations that enhance maintainability.’ (3a) ‘Because Setq-to-setf is a transformation that enhances maintainability.’ (4a) ‘To make use of the concept of generalized variables.’ (5a) ‘Because setf is applicable to generalized variables.’

‘Why is employee A assigned to room B?’ (1b) ‘In assigning employees to rooms some conditions must be fulfilled, including the requirement that group leaders must be assigned to single rooms. A is a group leader, and B is a single room.’ (2b) ‘Because group leaders must be assigned to single rooms.’ (3b) ‘Because A is a group leader.’ (4b) ‘Because group leaders frequently have meetings.’ (5b) ‘Because he has been nominated as a group leader in 1991.’

Figure 1. Commonalties of redundant, concise, and puzzling explanations across two domains.

certainly avoid a mechanical enumeration of the underlying rules and facts like in text (1b), which, though verbalized in a moderately skillful way, would be a rather boring presentation of the underlying rationale. Leaving out information, however, that could be contextually inferred is not without limitation as is illustrated by sentences (4a), (4b), (5a), and (5b) in Figure 1. Sentences (4a) and (5a) present the essential piece of information justifying the system’s original proposal to replace setq by setf, as expressed by sentence (1a). Nevertheless, comprehending the relevance of the information communicated via sentences (4a) or (5a), in the context of the request made, requires a good deal of understanding and background knowledge. In particular, this includes acquaintance with the fact that programs mostly entailing generalized variables can be maintained easier than other programs. Only if the presumed addressee is familiar with that, which are probably not what the system designers had in mind, sentences (4a) and (5a) sound adequate as responses to the original request. Otherwise, these arguments tend to look puzzling, especially to a novice, and should be uttered only after the justification (3a) has been given. Likewise, an office planning expert would certainly be able to avoid overdoing his/her job in explaining the rationale behind his/her reasoning. He/she would probably not consider sentence (4b) as an adequate answer, even though this is a deeper reason for the assignment proposed. In the actual context, however, this response seems rather unusual, at best. In fact, this statement would be an adequate response to a ‘Why?’ question following the explanation ‘Group leaders must be assigned to single rooms.’

user112.tex; 26/08/1997; 7:01; v.6; p.4

A MODEL FOR ADAPTING EXPLANATIONS

5

Under different circumstances, chaining of inferences and leaving some arguments implicit may be adequate – however, one must remember the arguments justifying sentences (4a) and (5a) as adequate answers in the LISP programming domain. If an explanation-seeking question refers to the problem-solving process itself by asking ‘Why has employee A been assigned prior to employee E?’, either ‘Group leaders are processed prior to project leaders’ or a more detailed version like ‘Employees associated with a large number of constraints are processed with priority’ and ‘There are usually more constraints associated with group leaders than with project leaders’, or even ‘A is a group leader, and E is a project leader’ are perfectly adequate. Furthermore, the response ‘For reasons of efficiency’ is also acceptable, but it seems to be adequate only for users with little technical background. For users with technical interest, this explanation is weak, since it can hardly be considered to be informative. They know that the ordering of elements fed to an iterative process is motivated by an expected increase of efficiency. There is, however, one specific approach that is very close to ours in which the user’s likely inferences are anticipated and exploited in one or another way to convey information indirectly – namely, the system WISHFUL-II (Zukerman, McConachy, 1993). The principal aims pursued by this system are quite in accordance with ours in that they produce concise explanations and avoid unnecessary redundancy. In addition, WISHFUL-II has several elaborate features: strategies for producing shallow or deep discourse (Zukerman, McConachy, 1994b), operationalized definitions of boredom and overload have been developed (Zukerman, McConachy, 1995), and a constraint-based optimization mechanism is used which aims at maximizing the user’s degree of belief while simultaneously minimizing boredom and overload. WISHFUL-II, thus, aims at more easily understandable explanations. The optimization mechanism relies on two strategies: conveying less information and breaking up the material to convey into smaller chunks. These are the same strategies as those used by our explanation mechanism for constraints (Horacek, 1992), which provides the initial specifications for the presentation techniques described in this paper. In Figure 2, we have listed two typical explanations WISHFUL-II can generate. When comparing these with the explanations in the domain of office planning (see the right half of Figure 1), some crucial differences emerge. These contrasts are especially apparent with the redundant versions, which make some of the underlying inferences explicit. In Zukerman and McConachy’s approach, a set of domain propositions is presented, which refer to some objects interrelated via specialization links in a taxonomic hierarchy. In contrast, generic regularities and referential facts partially instantiating these regularities occur in the domain of office planning. Thus, the work of Zukerman and McConachy pertains to descriptions, while our work applies to argumentation. This difference has decisive consequences for the complementary aims envisioned by the two approaches. In our approach, where instantiation and abstraction rather than specialization and generalization are the

user112.tex; 26/08/1997; 7:01; v.6; p.5

6

HELMUT HORACEK

‘DOS and UNIX’ discourse

‘Racing car’ discourse

DOS is an operating system. It has a command line interface, which is an interface where you type commands at a text prompt, e.g., mkdir, ls. It is a single user operating system and it does not allow multitasking, which is doing more than one job at a time. UNIX is like DOS, however it is a multiuser operating system and does allow multitasking. Some UNIX commands are the same as DOS, e.g., mkdir, however some are different, e.g., pwd. DOS runs on PC compatibles. In addition to PC compatibles UNIX runs on workstations.

An indicar is an American racing car. It has a very powerful engine, wide tires, huge breaks and big wings to make lots of downforce. Lots of downforce helps it go around corners quickly, however lots of downforce does not help it go straight quickly. A formula 1 car is like an indicar, however a formula 1 car is a European racing car.

Figure 2. Two texts produced by the system WISHFUL-II (Zukerman, McConachy, 1994a, pp. 44).

primary conceptual relations dealt with, the crucial task is to relate the entities in the current focus of attention to generic counterparts in a meaningful way. This task involves building sets of objects in reference to generic counterparts, which is associated with ambiguities and preferences of various sorts. Setting up a system of rules by which these issues can be handled, together with a mechanism for applying these rules, is the primary concern of our approach. Hence, while the aims of WISHFUL-II and our approach are the same on a certain abstract level, the different types of information presented demand the elaboration of different, but widely complementary, inferential aspects. 2.2. EVIDENCE FROM EMPIRICAL RESEARCH Empirical studies relevant to our enterprise primarily concern the issue of humaninferencing behavior in interpreting the relation between two consecutively uttered propositions. Insights from psychological research are crucial for building a model of text production that adapts its content presentation to the addressee’s inferential capabilities. There is ample evidence from psychological experiments that humans draw causal inferences during reading to close gaps left implicit in narrative texts (Kintsch, Keenan, McKoon, 1974). This reasoning is presumably done by building forward-oriented expectations and by drawing backward-driven inferences (Garnham, 1982). The experiments carried out by (Th¨uring; Wender, 1985) make it plausible to accept that both associated theories, the immediate inference theory and the deferred inference theory,1 are valid to a certain degree. However, these studies mostly address sequences of facts typically occurring in narratives rather than a mixture of generic regularities and individual facts, which is the typical pattern observed in the type of explanations in which we are mostly interested. 1 The immediate inference theory puts the perspective on inferences based on forward-oriented expectations. The deferred inference theory, on the other hand, advocates in favor of the view that inferences are drawn in a backward-driven fashion.

user112.tex; 26/08/1997; 7:01; v.6; p.6

A MODEL FOR ADAPTING EXPLANATIONS

7

Nevertheless, we believe that human cognitive behavior is comparable in these situations. In the experiments by Th¨uring and Wender, the task is to assess the processing effort associated with understanding the relation between two sentences presented in sequence, whose relation is not explicitly expressed in the text. Th¨uring and Wender distinguish between ‘direct causes’ (which can easily be understood) and ‘indirect causes’ and ‘unrelated facts’ (which both require considerable reasoning effort) to infer or, at least, to guess a relation, or to resign in the attempt of doing so. The distinction between ‘direct causes’ and ‘indirect causes’, however, is always delicate to a certain degree. In the approach by Th¨uring and Wender, the distinction is established on a statistical basis. In some pre-experiments, test-subjects have been asked to specify a follow-up sentence to each element of a set of pre-given sentences. Through carrying out these experiments, commonalties about default expectations across the set of subjects have been uncovered; the most frequently named follow-up sentences were categorized as ‘direct causes’. Within the cases classified as ‘indirect causes’, some of them have been built by chaining two ‘direct causes’, and some others have been derived by building a specific case of a ‘direct cause’. For example, if the first sentence is ‘he missed the bus’, the subsequently uttered sentence ‘he was late in the concert hall’ is considered a ‘direct cause’. The sentence ‘he missed the guitar solo’, however, is considered an ‘indirect cause’, since the proposition conveyed follows from being late in the concert hall and not being late somewhere else. These experiments tell us to envision several things in a formal model:  Expectations can and should be exploited in building discourse contributions, but one must be aware of the fact that expectations are almost always subjective to a considerable degree.  The suitability of discourse contributions may vary significantly in dependency of the task to be accomplished and according to the capabilities and experience of the addressee.

We believe that the experiments by Th¨uring and Wender are very relevant to our enterprise, and they strengthen empirical motivations for our approach. Therefore, we aim at taking these results into account by leaving ‘direct causes’ to be inferred by the reader, and by explicitly expressing ‘indirect causes’. In order to distinguish ‘direct causes’ from ‘indirect causes’, we rely on our causal domain model and some addressee dependent factors. 3. The Key Ideas of Our Model 3.1. CONTEXTUALLY MOTIVATED INFERENCES We concentrate our efforts on generating explanations that show causal chains of arguments underlying a proposition being questioned. The effects described in the previous section occur most frequently in explanations of this type, so that this

user112.tex; 26/08/1997; 7:01; v.6; p.7

8

HELMUT HORACEK

Figure 3. Relations between logically justified and contextually motivated inferences.

choice enables us to demonstrate the obtained improvements most effectively. In a general context, other strategies, such as (1) giving an example, (2) providing an analogy, or (3) using terminological definitions can suitably be applied; these strategies may also be combined with ours. The content of explanations considered here basically consists of individual facts and domain-specific regularities that can be causally chained. Texts composed of these ingredients are intended to provide deeper insights into the reasons underlying an explanation-seeking request: relations between parts of an explanation indicate which facts contribute to it and how they depend on each other, which regularities are relevant, and to which entities they apply in a concrete instance. Inferences in understanding utterances embodying these ingredients comprise purely logical conclusions, such as (1) substitution and deduction, (2) plausible abductive reasoning, as well as (3) contextually motivated assumptions and expectations. Hence, we ground our approach on the following hypotheses:  Logical reasoning is a good way to model a user’s understanding of an explanation.  The logic must be interpreted in context: assumptions and expectations must be taken into account.  We have evidence that certain regular interpretation patterns expressible by rules are used by the addressee, which accounts for aspects of conversational implicature (Grice, 1975).

Figure 3 shows the basic pattern relating logically justified and contextually motivated inferences to each other. Figures 4 and 5 show some minor variations of it. Deductive inferences are indicated by full lines, abductive inferences by dashed lines. In the central and the lower parts of these Figures, illustrations of the well-established inference operations – deduction and abduction – can be found. In the left, right, and upper parts of these Figures, illustrations of the remaining inferences, the contextually motivated ones, can be found. They comprise  derivations of a suitable instance from a generic rule

user112.tex; 26/08/1997; 7:01; v.6; p.8

A MODEL FOR ADAPTING EXPLANATIONS

9

Figure 4. A slightly generalized variation of the basic pattern in Figure 3.

Figure 5. Another slightly generalized variation of the basic pattern in Figure 3.

– yielding an instantiated premise, termed deduction-enabling hypothetical instantiation, – yielding an instantiated conclusion, termed abduction-enabling hypothetical instantiation,  the derivation of the relevance of a rule from instances of one of its parts, namely – from its premise termed deduction-enabling hypothetical causality, and – from its conclusion termed abduction-enabling hypothetical causality. The patterns depicted in Figures 3, 4, and 5 illustrate the hypothetical inferences for rules of different degrees of complexity. The basic pattern in Figure 3 can capture rules like ‘All men are mortal’, where P (x) encodes ‘x is a man’, and Q(x) ‘x is mortal’. Hence, knowing the rule P (x) ! Q(x) and a particular instance, P (e), ‘Socrates is a man’, the fact Q(e), ‘Socrates is mortal’, can be derived by applying logical deduction. In Figure 3, this can be identified by following the arrows labeled ‘deduction’. Similarly, abduction can be identified by following

user112.tex; 26/08/1997; 7:01; v.6; p.9

10

HELMUT HORACEK

the arrows labeled ‘abduction’, when Q(e) is known instead of P (e). Moreover, under the perspective of relevance, only becoming acquainted with one of the propositions involved, P (e) or Q(e), or the rule P (x) ! Q(x) may be sufficient to trigger an inference. For instance, in a context where P (e), ‘Socrates is a man’, is uttered, the relevance of knowing about Q (here, mortality) triggers the relevance of the rule P (x) ! Q(x) through deduction-enabling hypothetical causality (see the arrows labeled this way in Figure 3). Likewise, the relevance of that rule is triggered through abduction-enabling hypothetical causality in a context where Q(e), ‘Socrates is mortal’, is uttered, and knowing about P is relevant (here, being a man or not). Finally, when the rule P (x) ! Q(x) itself is uttered, it must prove its contextual relevance, which can be done by inferring a suitable ‘e’ for which either P (e) or Q(e) hold. In Figure 3, these kinds of inferences are represented by the arrows labeled ‘deduction-enabling hypothetical instantiation’ and ‘abductionenabling hypothetical instantiation’, respectively. In addition to the basic pattern in Figure 3, which merely covers simplistic rules such as ‘All men are mortal’, Figures 4 and 5 cover minor variations, that is, extensions of the underlying rule pattern. An example for the rule pattern covered by Figure 4, P (x) ^ R(x; y ) ! Q(y ), is ‘Group leaders must be assigned to single rooms’, where P (x) stands for ‘x is a group leader’, R(x; y ) for ‘x must be assigned to y ’, and Q(y ) for ‘y is a single room’. An example for the rule pattern covered by Figure 5, P (x) ^ R(y ) ! Q(x; y ), is ‘Smoker and smoker-intolerant persons must be assigned to different rooms’, where P (x) stands for ‘x is a smoker’, R(y ) for ‘y is a smoker-intolerant person’, and Q(x; y ) for ‘x and y must be assigned to different rooms’.2 As far as the logically motivated inferences are concerned, the difference between the basic pattern in Figure 3 and the extended patterns in Figures 4 and 5 manifests itself in the conjunction of the two premises. That is, both P (e) and R(e; f ), must be true to apply deduction in Figure 4. And both, P (e) and R(e; f ), can be derived abductively. The cases for the rule pattern in Figure 5 work analogously. In contrast to that, an optimistic attitude is driving the applicability of the contextually-motivated inference rules. Only one of the predicates in the rules’ premise, P or R, must be considered relevant so that abduction-enabling hypothetical causality can be applied to Q(f ) and Q(e; f ), respectively. Similarly, knowing an instantiation of only one of the predicates P and Q is sufficient to drive deduction-enabling hypothetical causality. These weak application conditions may lead to the applicability of several rules, and may yield propositions with identical predicates applying to different entities. Since some of these are usually in conflict

2 In the formalizations used in our model, these rules are expressed with slightly more predicates to limit the set of predicate names used in all domain rules. For instance, separate predicates for room categories, assignment of employees to rooms, and relations between rooms are used instead of the predicate associated with the complex meaning ‘must be assigned to different rooms’. However, this measurement does not influence the principal functionality of hypothetical causality and instantiation.

user112.tex; 26/08/1997; 7:01; v.6; p.10

A MODEL FOR ADAPTING EXPLANATIONS

11

with each other, competitions among contextually-motivated inferences must be resolved, which follows the principles discussed in the next section. The principle of relevance: All generic entities introduced in the arguments given must be related meaningfully to individuals in the current focus of attention, and the predications contained in the arguments must contribute to gaining increased evidence about the issue to be explained. The principle of minimal complexity: The simplest interpretation, that is, the one involving the smallest number of relations and propositions, is preferred over more complex interpretations. The principle of maximal coverage: The most connected interpretation, that is, the one involving the largest number of entities in the current focus of attention, is preferred over interpretations referring to a smaller number of entities. The principle of unambiguity: Two or more equally simple and connected, but conflicting interpretations must not be accepted, unless the resulting ambiguity can be tolerated. Figure 6. Four principles underlying the application of the contextually-motivated rules.

3.2. PRINCIPLES UNDERLYING THE APPLICATION OF INFERENCE RULES Unlike a classical logical rule, which can always be applied correctly once its premise is fulfilled, the force behind a contextually motivated rule is partially grounded in expectations and subjective preferences. Consequently, the confident application of such an inference rule in the generation process, that is, assuming the addressee will be able to draw that inference if confronted with partial information only, requires circumstances which reflect certain principles. We conceptualize these requirements into the four principles listed in Figure 6. The principle of relevance drives hypothetical instantiation, that is, the derivation of an instance from a generic rule is primarily governed by the expectation that the rule is relevant in the given context. That means, if the regularity expressed by the rule is a part of an explanation, there must be entities to which the rule is applicable in the actual context. For example, if the rule ‘Group leaders must be assigned to single rooms’ is uttered, there must be one or several group leaders to which this regularity applies in the concrete situation. Or, alternatively, one or several rooms must qualify as single rooms, so that the rule can be applied abductively. The candidate entities are typically to be found in the current focus of attention. Unfortunately, since knowledge is frequently incomplete and limited, it may be the case that no entities in the current focus of attention are known to fit into the description introduced by a regularity mentioned. However, hypothetical instanti-

user112.tex; 26/08/1997; 7:01; v.6; p.11

12

HELMUT HORACEK

ation still works in such cases by increasing the amount of hypotheses associated with the underlying inferences. If no sure match with a given description can be established, all other entities in the current focus of attention become candidates for matching with the categories introduced. In the above example, this means that in case no entity in the current focus of attention is known to the addressee as a group leader, all persons being in the current focus of attention with an administrative role unknown to the addressee are candidates to be considered as group leaders. The other three principles featuring minimal complexity, maximal coverage, and unambiguity, strongly influence the choice among alternative rule applications in a given context. If the addressee knows that an entity in the current focus of attention fits into a rule uttered in a given situation, no hypothesis is made whether or not other entities in the current focus of attention do so as well, according to the principle of minimal complexity. Moreover, if several entities simultaneously qualify for being subject to a rule application, all of them are accepted as suitable substitutes, according to the principle of maximal coverage.3 However, an ambiguity frequently must not be tolerated – if several entities qualify equally well for being substituted into a rule uttered, all resulting assertions must be consistent with each other. For instance, if a constraint expresses that two persons being in the current focus of attention must not share a room, and the rule ‘Smokers and smoker-intolerant persons must not share a room’ is uttered, it is at first unclear who is the smoker and who is the smoker-intolerant person. The communicative act considered would lead to a vague interpretation, unless the attitude towards smoking of one of the persons referred to is known. If this is not the case, and if the discourse purpose demands it, expanding the utterance will usually clarify this issue. In the example above, mentioning the attitude towards smoking of one of the persons involved would do the job. Similarly to hypothetical instantiation, the principle of relevance bears a crucial role for hypothetical causality. That means a rule is considered to be meaningfully applicable to a fact conveyed – either deductively or abductively – if the new information that results from a successful instantiation of the conclusion or the premise of a rule contributes to the issue to be explained. For example, if the fact ‘A is a group leader’ is uttered, the rule ‘Group leaders must be assigned to single rooms’ is useful for becoming acquainted with reasons justifying assignments of persons to rooms since the rule’s conclusion expresses a condition about this process. Moreover, an interesting phenomenon applies to rules with at least two variables, or even two predicates, involved in the premise: it seems that evidence about an instance of some part of a rule’s premise may be enough to trigger its relevance. For instance, if the assertion ‘A is a smoker’ is provided as an argument in favor of a particular assignment, the relevance of the rule expressing that smokers 3

The reader should note that the scope of complexity and coverage in these principles refers to a particular aspect of the explanation: properties of alternative sets of entities that match a description. This interpretation should not be confused with the complexity and the coverage of an explanation as a whole, the detail of the argumentation given, and the number of explainable cases covered.

user112.tex; 26/08/1997; 7:01; v.6; p.12

A MODEL FOR ADAPTING EXPLANATIONS

13

and smoker-intolerant persons should not share a room is apparent, even though no persons with smoker-aversion might be known. This example provides one piece of evidence justifying the ‘_’ joining P (e) and R(f ) in the extended pattern in Figure 5 (extended with respect to the basic pattern in Figure 3): knowing either P (e) or R(f ), that is ‘e is a smoker’ or ‘f is a smoker-intolerant person’ is sufficient to trigger the relevance of the rule P (x) ^ R(y ) ! Q(x; y ), that is, ‘Smokers and smoker-intolerant persons should be assigned to different rooms’. This reasoning is triggered by knowing whether or not it is relevant in the domain that the predication Q(x; y ) holds for some x and y (here, a relation between rooms that influences the feasibility of assignments) – see (Kass, Finin, 1987) for similar acquisition rules. The other three principles influence hypothetical causality in a similar way as hypothetical instantiation. According to the principle of minimal complexity, mentally searching for a suitable regularity only makes sense if there is no rule expressed explicitly in the explanation to which the fact considered applies. Furthermore, if the fact considered holds for several entities, all of them become subject to the application of a rule inferred by hypothetical causality, according to the principle of maximal coverage. Finally, there must not be several rules but only one to which hypothetical causality applies; otherwise, one of them is prominently preferred over the others. This requirement is not entirely based on an eventually-resulting inconsistency due to multiple matches, but it mainly comes from the need to control the complexity of the reasoning process. Some words need to be said about how obtaining a preferred element out of a set of candidates is done. This is usually a delicate problem. The associated assessments typically bear some subjective component, so that degrees of preference cannot be established on general grounds. In our approach, we use a simple scoring function comprising the rule’s specificity in terms of the number of variables involved, and the number of matches of variables in a rule with entities in the current focus of attention. Consider, for instance, a situation where group leader A is the only group leader in the current focus of attention, and two candidate rules, ‘Group leaders must be assigned to single rooms’ and ‘Group leaders and secretaries must be assigned to adjacent rooms’, are possible matches for being subject to hypothetical causality. If A’s secretary is not in the current focus of attention, the former rule is preferred due to the principle of minimal complexity. However, if not only A, but also A’s secretary, is in the current focus of attention, the latter rule is preferred due to increased specificity since this rule applies to a pair of employees instead of a single one. Thus, the principle of maximal coverage, which applies here, overrules the principle of minimal complexity in case these two principles are in conflict with each other. Unfortunately, not all cases are as clear-cut as this one; hence, exhibiting cautiousness in this respect seems to be a suitable strategy for the machine. Therefore, our model accepts only those rules as preferred ones which fit into the context at least as well as their competitors in all relevant aspects, and are superior in at least one aspect.

user112.tex; 26/08/1997; 7:01; v.6; p.13

14

HELMUT HORACEK

4. Formalization of the Model 4.1. FORMALIZING INFERENCE RULES In this section, we present formalizations of the regularities and principles underlying conversational implicature, as introduced in the previous section. For that purpose, rules are expressed in a language similar to that developed by (Moore, Paris, 1989, Moore, Paris, 1993), which is also used to express the text planning operators presented in Section 5. In an earlier version of or model, domain regularities and referential facts have simply been collected in a list, and inferability has been checked across elements of this list. In the current version, these pieces of information are composed into a text plan based on Rhetorical Structure Theory (Mann, Thompson, 1987). This enhancement achieves two purposes: the inferability of propositions is checked within their rhetorical context, and the mechanism developed can be better compared with work done by others. Formal definitions and associated informal descriptions of contextuallymotivated rules are given in Figure 7. Furthermore, this Figure shows a reformulation of some aspects of scalar implicature, as introduced by (Hirschberg, 1991), which is contextually-motivated in the same way as the deduction- and abductionenabling hypothetical instantiation and causality rules. To complete the set of rules used for expressing aspects of conversational implicature, we present formalizations of the classical logical inferences substitution, deduction and abduction in the same language. These formalizations are displayed in Figure 8. The rules consist of a premise, which usually is a conjoined expression connected by the junctor AND, and of a conclusion, which is headed by the predicate IMPLIES to mark the fact(s) newly inferred in case the rule succeeds. The conclusions also entail a rhetorical relation, which expresses how the newly inferred information is linked to known information. Variable names are headed by a ‘?’, to distinguish them from predicate names. Moreover, variable names are chosen in such a way that recognizing types of variables is supported: ?X , ?Y, ?Z, ?G, ?XG, and ?YG stand for entities, ?R and ?R1 for rules, ?PRED and ?PREDN for predicates, ?task for the issue at hand, which may be ‘assigning’ or ‘ordering’ here, and ?speaker and ?hearer stand for the speaker and its addressee, respectively. Some of the predicates contained in these rules need to be clarified. COLLECTALL is similar to FORALL, but it does not simply yield true or false depending on the results of the individual operations made. Instead, all instantiations of variables quantified by COLLECT-ALL for which the loop operations are successful are collected in a list. The predicates PREMISE and CONCLUSION yield true when being applied to a domain rule and the subformula respectively representing the premise or the conclusion of that rule. This way, it is possible to access a variable in the rule’s premise or conclusion, as well as the entailed predicate. The predicate ENTAILS (used in the definition of PREFERRED in Figure 11) is like a recursive version of MEMBER, and it includes variable matching. The predicate FOCUS yields true for an entity that is in the current focus of attention, which is typically

user112.tex; 26/08/1997; 7:01; v.6; p.14

A MODEL FOR ADAPTING EXPLANATIONS

15

Figure 7. Contextually-motivated inference rules expressing aspects of conversational implicature.

the case for entities being referred to by the explanatory request considered. The predicate UNKNOWN designates that the agent referred to does not know whether or not a certain proposition holds true. This means that the proposition is satisfiable in principle, that is, its types of restrictions are compatible with the types of the

user112.tex; 26/08/1997; 7:01; v.6; p.15

16

HELMUT HORACEK

Figure 8. Logically justified inference rules (conversational implicature rules).

referential entities. Hence, UNKNOWN is used as an abbreviation for (NOT (OR (KNOW ?X ?P) (KNOW ?X (NOT ?P)))). Finally, (PREFERRED ?R1 ?hearer ?task FROM ) stands for a procedure which evaluates the suitability of several candidates ?R specified in . It yields one entity, ?R1, as its result, if a ‘clear’ preference can be established for that entity; otherwise, ?R1 is not instantiated. We give more details on the definition of the predicate PREFERRED in Section 4.2. In the form presented in Figures 7 and 8, the rules can only pick out one variable at a time from a domain rule. This corresponds to the basic reasoning pattern shown in Figure 3 in which only rules like ‘All men are mortal’ are covered. Hence, additional rules are defined in the model’s repertoire of conversational knowledge, which are appropriate variations to cover the structures corresponding to the domain rules used by the expert system. The database entailing the domain knowledge of OFFICE-PLAN is not large. It consists of eleven domain-specific inference rules and four task-specific problem-solving rules, with a maximum of six variables per rule, four of which, at most, are introduced in the premise. In practice, however, not all possible combinations usually occur. As it turned out in our application, introducing only four additional variations extending the basic pattern are sufficient to cover the cases found in OFFICE-PLAN’s knowledge base.

user112.tex; 26/08/1997; 7:01; v.6; p.16

A MODEL FOR ADAPTING EXPLANATIONS

17

Table I. Domain-specific inference rules and task-dependent problem-solving rules in DIAMOD Domain-Specific Inference Rules (re-phrased freely here) ‘Resources should not be overused’ ‘Rooms should not be overfilled’ ‘Everyone should be put in a room’ ‘Group leaders should be assigned to single rooms’ ‘Group leaders and secretaries should be assigned to adjacent rooms’ ‘Group leaders and project leaders should be assigned to near rooms’ ‘Group leaders should be assigned to a room near the discussion room’ ‘Employees that frequently have meetings and full-time employees should be assigned to different rooms’ ‘Smokers and smoker-intolerant persons should be assigned to different rooms’ ‘Employees working on the same project should be assigned to different rooms’ ‘Employees with no common themes should be assigned to different rooms’ Task-Dependent Problem Solving Rules (re-phrased freely here) ‘Group leaders are associated with more constraints than project leaders’ ‘Project leaders are associated with more constraints than secretaries’ ‘Secretaries are associated with more constraints than ordinary employees’ ‘Employees associated with a larger number of constraints are assigned prior to others’

Figure 9. A domain inference rule and two problem-solving rules used in OFFICE-PLAN.

Table 1 entails a complete list of these rules, freely paraphrased in natural language. Figure 9 shows formalizations of one domain rule and two problem-solving rules. One of the problem-solving rules, Rule-C, needs some justification because the formalization entails a certain simplification. In our domain model, group leaders are considered to be always associated with more restrictions than project leaders. The effect of the requirements ‘must be assigned to a single room’ and ‘must be next door to his/her secretary’ are considered so restrictive that these requirements

user112.tex; 26/08/1997; 7:01; v.6; p.17

18

HELMUT HORACEK

outscore the set of individually motivated constraints for project leaders (e.g., due to smoker-aversion). In fact, a certain project leader can even be associated with more constraints than a group leader, but the overall effect, which cannot be quantified in simple terms, is still considered stronger for the group leader. However, in the formulation of our model, simply the predicate GREATER is used to express these circumstances. Now, let us return to the variations of the basic rule patterns in Figure 7 and 8. Figure 10 shows such a variation, namely C-Rule 1a, which originates from C-Rule 1. C-Rule 1a deviates from C-Rule 1 only insofar as it can match domain rules with a slightly more complex pattern in their premise. In concrete, C-Rule 1a can capture the relevant aspects of domain rule Rule-A, ‘Group leaders must be assigned to single rooms’, as far as deduction-enabling hypothetical instantiation is concerned. For abduction-enabling hypothetical instantiation, C-Rule 2 is sufficient, since the conclusion of Rule-A consists of a single proposition only. In C-Rule 1a, not only one predicate is accessible in the domain rule’s premise, as in C-Rule 1, but three of them: ?PRED1, ?PRED2, and ?PRED3. The remaining parts of C-Rule 1 and CRule 1a are identical, since only ?PRED1 is of interest for the kind of inferencing required here. In Rule-A, for example, the GROUP-LEADER predicate is the relevant one, while the ROOM and the ASSIGNED-TO predicates only serve the purpose of appropriately binding the variable associated with the group leader’s room, which is relevant for the rule’s conclusion. The particular role of some predicates (such as the GROUP-LEADER predicate) demands adherence to some conventions in the way in which the domain rules are specified; the proposition entailing the prominent predicate must precede the other ones. An alternative to this strategy would be reformulating the domain rule into the form P (x) ! Q(x; y ), where P would stand for the GROUP-LEADER predicate, and Q(x; y ) would mean ‘x is assigned to room y , which must be a single room’, thus conflating several predicates into one. We did not choose this form, because it is unsuitable for purposes of natural language generation. Altogether, we preferred to obey some conventions in setting up our small domain model, rather than applying some set of predicate transformations that would have been otherwise necessitated. We take C-Rule 1a as an example for demonstrating how applying and instantiating such a rule works in detail. Let us assume that an INFORM speech act is generated, whose propositional content is the domain rule Rule-A, and C-Rule 1a is the contextually-motivated inference rule considered to determine the addressee’s likely inferences. Moreover, let us assume that the current focus of attention is determined by the occurrence of the question ‘Why has employee A been assigned to room B?’ Hence, A and B are in the current focus of attention. Furthermore, let us assume that the explanation-seeking person does not know what kind of employee A is. If ?speaker and ?hearer match with the conversants in the actual context, ?R matches with Rule-A, as the propositional content of the INFORM speech act, and it also fulfills the categorial predication (RULE ?R). Next, the premise of

user112.tex; 26/08/1997; 7:01; v.6; p.18

A MODEL FOR ADAPTING EXPLANATIONS

19

Figure 10. A variant of a contextually-motivated inference rule and one of its instantiations.

Rule-A gets matched against the pattern ‘(PREMISE (AND (?PRED1 ?Y1 ...))’, as specified in C-Rule 1a. This operation is successful, and its result yields the following instantiations for the variables in C-Rule 1a: ?PRED1 is instantiated to GROUP-LEADER, ?PRED2 to ROOM, and ?PRED3 to ASSIGNED-TO. ?Y and ?Y1 match with the variables in Rule-A, so that they do not yield instantiations. Thereafter, the addressee’s knowledge with respect to group leaders is tested. The entities in the current focus of attention, A and B, are bound to ?Y, one after the other, thus testing the proposition ‘(NOT (KNOW ?hearer (AND (GROUP-LEADER ?Y)...))’. For both A and B, evaluating this proposition yields true according to the assumption made because the addressee does not know A as a group leader. Next, ?Z is matched with A and B, one after the other, and the next proposition, ‘(UNKNOWN ?hearer (?PRED ?Z))’, is evaluated. However, this proposition yields true only for employee A, since the addressee is assumed to know that rooms (in particular, room B) can never be group leaders. Hence, the whole premise has yielded successful matches for the variables ?R, ?PRED1, and ?Z, so that the conclusion of C-Rule 1 gets instantiated to ‘(BELIEVE (USER (ELABORATION Rule-A (GROUP-LEADER A))))’.

user112.tex; 26/08/1997; 7:01; v.6; p.19

20

HELMUT HORACEK

Not all variations of contextually-motivated inference rules deviate as little from their master rules as C-Rule 1a deviates from C-Rule 1. For example, not only the pattern matching of the domain rule’s premise or conclusion must be expanded in order to capture domain rules involving two employees. In addition, combinations of the variable slots in the predicates obtained from the domain rule must be matched against the entities from the current focus of attention. Thus, the single loop initiated by ‘(FORALL ?Y ...)’ is typically extended by another loop ‘(FORALL ?Y1 ...)’ so that not only a functional relation between these entities can be captured. Similarly, the proposition (?PRED ?Z) occurring twice in the last two lines of C-Rule 1 and C-Rule 2 gets expanded to a more complex proposition. For the sake of simplicity, we always refer to the rules shown in Figures 7 and 8 when discussing their applications, even though one of their variations is typically meant. Apparently, it would be desirable to represent some of these variations compactly in one rule, but that would require increasing the expressive power of the pattern language. In order to augment the generality of the contextually-motivated inference rules from merely covering the cases occurring in OFFICE-PLAN’s knowledge base, an increasing number of variations of the basic pattern of these rules must be built. These would tend to cover the cases occurring in domain models of an increasing number of systems. Eventually, some generalizations of variations could be specified, and the variations subsumed by a particular generalization could be automatically generated to save some effort in building the inference rules by hand. Furthermore, each variation could be equipped with association links to the domain rules to which it is applicable in order to reduce the search effort in checking inferability in a concrete application. In the course of evaluating the contextually motivated rules, reference is made to domain rules in various places. The way this is done, however, is merely matching predicates and variables of the domain rules symbolically. Hence, it is the contextually-motivated rules which are evaluated, and not the domain rules. Doing the latter would mean a consistency check for the solution of the problemsolving process rather than contributing to the construction of the explanation’s content. These domain-specific rules are in fact the link between the problem-solving component and the explanation component. In the explanation component, the domain rules become subject to matching operations when the contextuallymotivated implicature rules are invoked to determine the cross-inferability of propositions. In the problem-solving component, however, constraints are derived from the domain-specific rules, by abstracting from the underlying domain rationale. For instance, a constraint may merely express that two employees must be assigned to different rooms, irrespective of the underlying domain principle – be it due to smoker-aversion, or due to achieving the dissemination of experience.

user112.tex; 26/08/1997; 7:01; v.6; p.20

A MODEL FOR ADAPTING EXPLANATIONS

21

4.2. HOW THE RULES MEET THE PRINCIPLES In this subsection, we illustrate in what ways the principles set out in Section 3 are met by the formal reconstruction of aspects of conversational implicature, as shown in C-Rules 1 to 4. 4.2.1. The Principle of Relevance In C-Rules 1 and 2, relevance manifests itself in the effect when applying these rules. The result of such a rule application creates a belief in facts about discourse entities that are related to a domain rule uttered (?R), in the way expressed by the rule’s body. Hence, the effect when applying these rules is stronger than the force of a speech act stating ?R, which would merely comprise the acceptance of the truth of the domain regularity expressed, or of another mental attitude towards it. In C-Rules 3 and 4, relevance manifests itself in an even stronger way. The mere applicability of a regularity to a fact uttered is not sufficient to trigger its relevance. In order to prominently influence the addressee’s attention, those parts of the domain rule that did not match with the fact mentioned in the discourse must provide new information that contributes to the task to be achieved. For example, if we are interested in assignments of employees to rooms and a group leader’s situation is taken into account, a rule that expresses a room category requirement for group leaders is considered relevant, whereas a rule that expresses ingredients of salary regulations of group leaders is not. In our model, the distinction of relevance from irrelevance is made on the basis of the predicates that appear in corresponding parts of domain rules, which must be a room category or a relation between rooms in the context of the assignment goal. This criterion constitutes a simplification, but it proves to be sufficient for our concrete application. In more general cases, however, capturing more indirect relations may also be needed, which would require a less restrictive formalization of relevance. 4.2.2. The Principles of Minimal Complexity and Maximal Coverage Two further principles, minimal complexity and maximal coverage, primarily govern preferences among alternatives in the case that several interpretations, including different sets of individuals, result from an application of a contextually-motivated inference rule. The formulation of the relevant aspects of these two principles is done on the basis of domain-independent criteria. Minimal complexity manifests itself in C-Rules 1 to 4 in two respects. One of them concerns the necessity to invoke hypothetical instantiation or causality at all. Looking for a set of yet unknown individuals that match with a mentioned domain rule is reasonable only if no individual in the current focus of attention is already known to fit into that regularity (see [2] in C-Rules 1 and 2). Only if no suitable entities are known, is new information inferred by means of hypothetical instantiation (see [3] in C-Rules 1 and 2). Similarly, if a fact is mentioned, the

user112.tex; 26/08/1997; 7:01; v.6; p.21

22

HELMUT HORACEK

Figure 11. Definition of the predicate PREFERRED.

mental search for a suitable domain rule is primarily confined to rules being in the current focus of attention (see [2] in C-Rules 3 and 4). Only if this fails, are further task-relevant domain regularities taken into account as potential reasons underlying that fact (see [3] in C-Rules 3 and 4). The other way minimal complexity manifests itself in the C-Rules concerns only hypothetical causality, that is, C-Rules 3 and 4. Preference among several candidate rules is expressed by the predicate PREFERRED, whose precise definition is given in Figure 11. In essence, the underlying purpose is to pick up one rule that wins over the other candidates in terms of specificity, in view of the context given by the entities in the current focus of attention. For instance, a domain rule about group leaders alone is considered to be preferable to a domain rule about group leaders and their secretaries, provided the entities in the current focus of attention only fit to the predicate group-leader, but not as well to the predicate secretary. This case has already been mentioned in the previous section, when introducing the principles on an intuitive basis. In contrast, the principle of maximal coverage leads to the opposite preference if both predicates, group-leader and secretary, fit to entities in the current focus of attention. Another manifestation of the principle of maximal coverage is encoun-

user112.tex; 26/08/1997; 7:01; v.6; p.22

A MODEL FOR ADAPTING EXPLANATIONS

23

tered in C-Rules 1 and 2 only. If the variable ?Z in the predication (?PRED ?Z) yields successful matches for several individuals, all of them become subject to the force of the rule; that is, the predication (?PRED ?Z) becomes a belief of the addressee for all individuals matching. The application of these two principles may interfere with problems associated with incompleteness and imperfection of knowledge. These problems concern the decision which of the entities in the current focus of attention can or cannot satisfy the predication (?PRED ?Z). For example, consider a situation where ?PRED is instantiated to the predicate GROUP-LEADER and there are two persons in the current focus of attention: a local employee ?X1 and a visitor ?X2. Moreover, the intended inference is that only the local employee is considered a group leader by the addressee, but not the visitor. Intuitively, we would expect even a moderately intelligent user to conclude that information, on the basis of general world knowledge about typical situations in companies. However, knowledge about the user’s beliefs may be incomplete, so that there is no conclusive reason for assuming whether or not the user believes that visitors can be group leaders. Another potential source for uncertainty is present in case knowing whether or not visitors can be group leaders is beyond the competence of the underlying domain model. This example illustrates a general problem: even in situations that may be obvious to practically all users, it may always be the case that a system is incapable of deriving the feasibility or unfeasibility of a certain proposition in question, or the system may be incapable of judging the user’s competence in this respect. In our model, we apply a cautious strategy to counter this problem. The question whether a predicate is applicable to a certain individual or not is decided exclusively on the basis of class memberships. Thus, if a predicate is not applicable to a certain individual because the type required by the predicate is incompatible with the individual’s class membership, then this incompatibility holds for sure – provided the taxonomy is correct. In some occasions, however, this criterion may be too weak; the potential roles of a visitor, for instance, may not be derivable purely on the basis of the system’s limited taxonomic knowledge. The criterion used can be expected to yield acceptable, though not always optimal, results. For instance, if a system’s background knowledge is insufficient to indicate whether or not an addressee is able to draw an inference in a particular situation, a redundant piece of discourse may result if, in fact, the addressee is able to draw that inference. Going back to the example mentioned above, the system would follow the utterance ‘Group leaders must be assigned to single rooms’ by ‘Employee ?X1 is a group leader’ to make sure that the addressee believes that the employee, but not the visitor, is the group leader. If, however, ?X2 is known to denote a computer rather than a visitor, the addressee is expected to draw the necessary inference. Hence, the inferential capability of the addressee can be exploited under certain circumstances to produce a more natural discourse, such as the motivating examples in Section 1, but the mechanism would at worst produce redundant discourse in cases where the system’s knowledge is insufficient.

user112.tex; 26/08/1997; 7:01; v.6; p.23

24

HELMUT HORACEK

4.2.3. The Principle of Unambiguity As with the other principles, the principle of unambiguity is met in different ways by C-Rules 1 to 4. In C-Rules 3 and 4, evaluating the predication headed by PREFERRED is responsible for producing a single solution, if possible, so that the overall outcome of the rule application is always unambiguous. Several components contribute to this result, mainly concerning the degree to which variables appearing in the rules considered and entities in the current focus of attention can be related to each other (see the definition of the predicate PREFERRED in Figure 11). These components are:  the degree of matching in a positive sense, that is, a maximum number of variables in the current focus of attention should be taken into account by the matching process (see [3] and [6] in Figure 11, which are derived from the principle of maximal coverage),  the degree of matching in a negative sense, that is, a minimum number of variables appearing in the rules should be left unmatched by the reasoning process (see [4] and [7] in Figure 11, which are derived from the principle of minimal complexity),  the degree of salience attributed to domain rules (see [5] and [8] in Figure 11).

In order to qualify as a preferred item, a rule should be considered not worse than any other rule with respect to these criteria (see [3] to [5] in Figure 11), and the preferred rule should be superior to all other rules in at least one of these aspects (see [6] to [8] in Figure 11). In our application, we barely elaborated the last of these criteria, that is, the salience of all domain rules is considered identical, except for the context in which they are invoked. For example, a rule about assignment constraints is considered more salient if an explanation about assignments is required (see RuleA in Figure 9), while a rule about ordering employees for an assignment task is more salient in the context of explanations about problem-solving aspects, which include ordering (see Rule-B and Rule-C in Figure 9). In a more complex model, however, it could make sense to assign salience measures to rules. For instance, a preference of general rules over specialized ones could be expressed, thus marking a rule about group leaders in general as more salient than a rule about group leaders nominated within the last year. In C-Rules 1 and 2, the principle of unambiguity is realized in a different way than this is done by the predicate PREFERRED in C-Rules 3 and 4. In the majority of usually simple cases, no problems of ambiguity arise since the predicates against which entities in the current focus of attention are matched typically entail enough restrictions to yield only the desired facts. For instance, if the predicate groupleader, a one-place predicate, is matched against candidate entities, all successful matches are collected. In case of multiple matches, this set is identical to the set of entities intended to be inferred; otherwise a source for a wrong implication must be detected.

user112.tex; 26/08/1997; 7:01; v.6; p.24

A MODEL FOR ADAPTING EXPLANATIONS

25

Complications arise if n-ary predicates (n > 1), or several predicates with compatible types are subject to the matching process, and at least two of these slots yield multiple matches with entities in the current focus of attention. The set of resulting legal combinations of matches usually is a proper superset of the true facts that are intended to be inferred. Typically, a reduction of potentially true facts can be obtained by cross-evaluating the semantics of predicates, which includes exploiting properties like incompatibility and singularity. For example, if a rule refers to a smoker and a smoker-intolerant person, none of the candidate entities may match both predicates simultaneously. Another example concerns assignment requirements for group leaders and secretaries. If several candidates for both the group leader and secretary roles exist, potential secretaries have to be related to group leaders in a one-to-one fashion if the domain model requires that each secretary is assigned to one group leader exclusively. However, a concrete communicative situation may not always require strict precision. It may even be sufficient to convey only the vague information that one person involved in a requirement is a smoker and the other is a smoker-intolerant person, irrespective of who is what. Or, it may be perfectly adequate to merely know who are the secretaries and who are the group leaders, but it may not be important to know to which group leader each of the secretaries belongs. However, all considerations about cross-evaluating predicate semantics and relaxing precision due to the communicative purpose are too complicated to be incorporated into our formal model. Hence, our approach simply is to require all inferable combinations of matches to be true facts; otherwise, the implicature rule under consideration cannot be applied in the given context. However, by conveying additional facts that rule can be made applicable in a modified context, but these deviating cases must be specified first in order to make the conversational implicature rules work. 5. Incorporating Inferences into Text Planning 5.1. TEXT PLANNING OPERATORS In this section, we illustrate the exploitation of knowledge about conversational implicature for producing texts that take into account assumptions about the addressee’s inferential capabilities. For this purpose, a text plan is incrementally built in such a way that evidence about contextually-motivated inferences is incorporated into the text plan in terms of suitable annotations. The annotations establish a link between one or several leaf nodes and another leaf node whose associated content can be inferred from the contents associated with the nodes linked to it. Text planning is essentially realized as a top-down hierarchical process, pretty much in the style adopted by (Moore, Paris, 1989) and (Moore, Swartout, 1989), and based on Rhetorical Structure Theory (RST) (Mann, Thompson, 1987). In a previous version (Horacek, 1991), our model was based on a mere set of propositions not bound by rhetorical relations. Moreover, simpler forms of conversationally-

user112.tex; 26/08/1997; 7:01; v.6; p.25

26

HELMUT HORACEK

motivated rules were responsible for maintaining a set of propositions conveyed explicitly in contrast to those conveyed implicitly. In the current version, there are two reasons to use RST:  Inferences are additionally based on rhetorical relations holding between individual propositions, which are composed into tree-like structures rather than simply collected in a list.  By the use of RST, we can rely on known techniques from text planning, and our approach is more comparable to several other approaches in the field.

Following the taxonomy developed by (Maier, Hovy, 1993), the set of rhetorical relations relevant for our purposes comprises the relations CAUSE, GENERALCONDITION, JOINT, GENERAL-SPECIFIC, and ABSTRACT-INSTANCE. The last two relations are specializations of ELABORATION, and JOINT simply links multiple expansions together, as in standard RST. Enhancing other approaches, plan expansion is interleaved with checking operations, which determine whether or not a rhetorical relation and one of the propositions connected are inferable in a given context. Occasionally, the conditions of inferability may be satisfied only as a consequence of a contextual change achieved by planning steps performed later. Therefore, if the applicability test of a contextually-motivated rule fails, a record specifying where the failure occurred is kept in an agenda. Whenever the reason for such a failure disappears during subsequent planning, that is, the truth value of the responsible proposition changes, the checking operation formerly failing is repeated. It may yield success in the new context. A priori selection of rules for testing is done by matching the rhetorical relation introduced by the plan step expanded and the relation specified in the rule’s conclusion. Only if this quick pre-selection succeeds, is a detailed matching check performed. Appropriate annotations are made in a text plan, in case the content associated with a newly generated plan step is assumed to be common knowledge, either due to evidence about the addressee’s expertise or as a consequence of speech acts associated with already expanded plan steps, from which the proposition at hand is considered inferable. These annotations are updated as more propositions become inferable. In order to apply a contextually-motivated inference rule with confidence, not only the content of the proposition left implicit must be inferable, but also the rhetorical relation which links it to some part of the text plan. Hence, the relation specified in the conclusion of the inference rule must subsume the relation occurring at the corresponding position in the text plan. For the relations considered here – in particular, for the relations GENERAL-CONDITION and ABSTRACTINSTANCE, it is hard to think of plausible alternatives. However, it may be the case that the relation between two facts is not a priori clear from the facts themselves. For example, given the assertion ‘he opposed the new law’, the subsequently uttered assertion ‘he is a trade union representative’ may provide an evidence for or a contrast to the first assertion, depending on the expected attitude of the trade unions towards the new law. In the latter case, the rhetorical relation must be conveyed

user112.tex; 26/08/1997; 7:01; v.6; p.26

A MODEL FOR ADAPTING EXPLANATIONS

27

Figure 12. Text planning operators.

explicitly, in order to avoid confusion about the true circumstances: ‘He opposed the new law, even though he is a trade union representative’. In Figure 12, the most important text planning operators used for explanations of the type discussed are presented. These operators express the following text plan expansion steps:  Increasing the belief of the addressee in a certain state by an actually-holding condition which implies this state (Operator 1).  Introducing the generic rule which the condition is an instance of (Operator 2).  Providing more specific details about a generic rule or some part of it (Operator 3).  Elaborating a generic rule by introducing the entities it applies to in a concrete instance (Operator 4).

The assumption behind the usage of ‘FORALL’ in Operator 1 is that the complete set of facts constituting the reason for state ?S is required – here these facts are a set of assignment constraints. Admittedly, this assumption is rather strong. It is justified by the domain, since only the combination of all responsible constraints causes the assignment in question to be infeasible, and not some subset of these. However, the strategy pursued may be changed in another domain, when knowing about partial causes is sufficient.

user112.tex; 26/08/1997; 7:01; v.6; p.27

28

HELMUT HORACEK

Operator 2 and Operator 3 may be used to provide additional information to the shallow reason already included in a text plan by Operator 1 (here, the mere constraints). Since the suitability of further details depends on the concrete circumstances, these operators are marked as optional. In case the causal explanation consists of a large number of constraints, it hardly makes sense to state the generic principles behind each of them since the explanation would then grow long. Even for four or five constraints, the underlying generic principles should, at best, be included only selectively. The admittedly simple strategy incorporated in the model expands the causes further if they consist of at most three constraints. Intensive testing would be required to establish better motivated complexity assessing parameters. The other operators are marked as required since their application is necessary to generate a consistent explanation which, in our model, comprises a generic rationale and the precise way it applies to relevant individuals in the given situation. Both contextually-motivated rules as well as text planning operators can access domain-specific knowledge represented in terms of rules. 5.2. BUILDING A TEXT PLAN We demonstrate the basic functionality of our model by generating a response to the question: ‘Why is person A in room B and not in room C?’, thereby illustrating the effects of exploiting conversational implicature. The corresponding text plan is depicted in Figures 13, 14, and 15 at different stages of processing. When answering that request, text planning starts by providing the cause for the assignments mentioned, according to Operator 1. The set of responsible arguments is determined by the method described in (Horacek, 1992). For the sake of simplicity, we assume that only one condition, cond-1, is responsible for the assignment in question, and an appropriate speech act is inserted in the text plan. Since the information obtained so far essentially paraphrases the explanatory request, optional expansion of the text plan is performed by applying Operator 2. This leads to the insertion of Rule-A into the text plan, and a check is performed whether C-Rule 6 can be applied to one of the plan steps generated most recently – only this rule is applicable to the GENERAL-CONDITION relation introduced by Operator 2. When addressing an external user, all of these checks fail, and the unmatched proposition (BELIEVE ?hearer (GROUP-LEADER ?X)) is recorded in the agenda. For a local employee, it would be unnecessary to make some of the further checking operations since he/she is assumed to know all group leaders including, in particular, A, and the relations cond-1, inst-1, inst-2, and inst-3. So far, the partial text plan built is shown in Figure 13. According to our explanation model, the relevance of Rule-A is then elaborated by applying Operator 4, which introduces inst-1 into the text plan, stating that ‘A is a group leader’. From there, several things are inferable (see the dashed arrows in Figure 14, which mark the inferability of propositions). Since A is the only entity in the current focus of attention to which the predicate GROUP-LEADER can be

user112.tex; 26/08/1997; 7:01; v.6; p.28

A MODEL FOR ADAPTING EXPLANATIONS

29

Figure 13. Preliminary steps in building a text plan for explaining a particular assignment.

Figure 14. Intermediate step in building an annotated text plan for explaining a particular assignment.

meaningfully applied, proposition inst-1, ‘A is a group leader’, is inferable from Rule-A by virtue of C-Rule 1. By virtue of C-Rule 3, which is applicable if the addressee is familiar with domain rules about group leaders, the inference also works the other way round if Rule-A is preferred among these rules in the given context, as this is the case here. Next, inst-2 is introduced (see Figure 15). Once Rule-A and inst-1 are known, applying C-Rule 7 to them yields proposition inst-2, ‘B is a single room.’ Now, CRule 6, which formerly failed, can be applied successfully to the three propositions Rule-A, inst-1, and inst-2, to yield proposition cond-1. Finally, the proposition

user112.tex; 26/08/1997; 7:01; v.6; p.29

30

HELMUT HORACEK

Figure 15. Completed and annotated text plan for explaining a particular assignment.

inst-3 is introduced, because C is mentioned in the question. This proposition, ‘C is not a single room,’ can be inferred from inst-2 by virtue of C-Rule 5. By inspecting the completed and annotated text plan in Figure 15, we note that from the proposition ‘Group leaders must be assigned to single rooms’ all other propositions and the rhetorical relations holding between them are considered inferable by the addressee. In case the addressee is a domain expert, this is even possible for the proposition ‘A is a group leader’. Hence, either of the texts ‘A cannot be assigned to C, since group leaders must be assigned to single rooms’ or ‘A cannot be assigned to C, since he/she is a group leader’ would adequately realize the text plan, depending on the aspects of domain knowledge attributed to the addressee. While we have seen that deduction is working rather nicely in this example, we have to admit our model fails to exploit abduction to yield the required inferences here. Let us examine why this is the case. First, let us consider testing abductionenabling hypothetical instantiation, that is, attempting to derive instances of rooms from the rule ‘Group leaders must be assigned to single rooms’. The application of C-Rule 2 to Rule-A yields B and C as possible matches for the (room ?X)

user112.tex; 26/08/1997; 7:01; v.6; p.30

A MODEL FOR ADAPTING EXPLANATIONS

31

predication. Because of the principle of maximal coverage, both B and C must be single rooms, since they match to this predicate in the conclusion of Rule-A. However, the consequences resulting from this assumption are not sound. Assuming only A being a group leader leads to a contradiction (see proposition assign-1). Assuming another person not being in the current focus of attention to be a group leader assigned to room C violates the principle of minimal complexity. Hence, applying C-Rule 2 to Rule-A fails. However, if the request considered would be ‘Why is A and not E assigned to room B?’, abduction would yield inferring B as a single room uniquely, and the remaining inference process would be similar as that described for the case where deduction can be applied successfully. The inability of our mechanism to apply abduction-enabling hypothetical instantiation in the original example does not mean that abductive reasoning would not succeed here, in principle. However, it requires more complex and indirect reasoning. In the example considered, abductive reasoning could go along the following lines: Following the principle of maximal coverage in determining the rooms that fit into the rule ‘Group leaders must be assigned to single rooms’, B and C must both be single rooms. Since this leads to a contradiction, as argued above, the rule responsible for that contradiction, C-Rule 2, is reconsidered. In this rule, there are references to the hearer’s knowledge about the predications appearing in the domain rule’s conclusion, which instantiates here to the proposition ‘rooms in the current focus of attention are single rooms’. Since not both B and C can be single rooms at the same time, only one of them can be a single room, according to the increased knowledge obtained by the inferences made so far. Hence, C-Rule 2 is reevaluated, once with B and once with C as a single room exclusively. Out of these two cases, only the first one survives, because C being a single room is not consistent with the domain rule and the assignment made. As one can see, the mechanism could also succeed in reasoning abductively, allowing the application of the contextually-motivated rules to be defeasible, as far as the aspect of the relevant principles is concerned. This complication, however, is not incorporated in our model. Then, let us consider the other kind of abduction-enabling hypothetical inferencing, that is, causality. When checking the inferability the other way round, that is, from the assertion that B is a single room, there are several equally plausible reasons why A should be assigned to a single room, other than that of being a group leader; for instance, A could be a secretary or the only smoker facing a set of smoker-intolerant colleagues. Hence, applying C-Rule 4 to inst-2 fails, too, but this time due to an ambiguity with respect to the underlying rationale. Nevertheless, the argument ‘B is a single room’ is certainly an explanation to the question asked. It is, however, a shallow one, because it does not support the inference to the underlying domain regularity. So far, the techniques presented essentially rely on an understanding of logic and relevance in a communicative situation in general. In some places, subjective elements in the associated reasoning have been mentioned without providing much

user112.tex; 26/08/1997; 7:01; v.6; p.31

32

HELMUT HORACEK

detail about typicality, individual variations, and their concrete realization. Hence, while the emphasis in the last section was on general techniques applied in our presentation method, the next section reports on assumptions and evidence about subjective properties of potential addressees, the representation of these ingredients in a user model, and its exploitation in the presentation task. 6. Empirically Determining Human Preferences 6.1. MOTIVATING AN INQUIRY So far, we have demonstrated that our text planning mechanism is capable of generating annotations in text plans to mark the cross-inferability among sets of propositions. By leaving selected propositions unexpressed, these annotations can be exploited for producing alternative texts in varying degrees of explicitness. Moreover, suitable choices among these alternatives should be in accordance with typical human preferences. In order to learn about these preferences, we have confronted several people with the example discussed in the previous section. The overwhelming majority of people voted in favor of simply uttering the domain rule responsible for the assignments made, that is, ‘Group leaders must be assigned to single rooms’, thereby leaving the class memberships of the entities involved to be inferred by the addressee. People also agreed that simply stating the class membership of the only employee mentioned in the question, ‘A is a group leader’ might even be preferable for addressing a domain expert, provided that the domain rule intended to be inferred is prominently associated with the assignment of group leaders. We believe that these assessments provide further justifications for our enterprise, and the generated annotations provide an important prerequisite for producing the envisioned concise, explanatory texts. However, the example discussed above seems to be a comparably simple and clear-cut case. In more complex cases, it is not so clear how humans would judge the suitability of alternative texts, and on what factors preferences may be based in varying environments. In order to find out more systematically how humans would act in comparable situations, we have initiated an inquiry by asking some colleagues to play the role of an explanation-giving device in some well-defined situations. Thus, our study records two populations’ beliefs about the need of some hypothetically specified populations. Unfortunately, we were not able to exploit the expertise of real domain professionals for our inquiry. Therefore, the assessments of our test-subjects are influenced to a certain extent by their assumptions about the domain – we intentionally gave them descriptions in rather general terms, such as ‘address the explanation to a local expert’. This was done because we did not want to constrain the situation descriptions too much in order to meet a variety of eventually divergent intuitions test-subjects might have about the domain. Two groups of people participated in the inquiry: six students of a course on human computer interaction, and seven academic professionals with varying degrees of experience and scientific background. This number is admittedly small, but it

user112.tex; 26/08/1997; 7:01; v.6; p.32

33

A MODEL FOR ADAPTING EXPLANATIONS

Table II. Alternative texts used in our empirical inquiry Text Number of Number of labels rules facts (a) (b)

0 1

1 0

(c)

1

0

(d)

1

1

(e)

1

1

(f)

2

0

(g)

2

1

Explanation texts ‘A is a group leader’ ‘Group leaders are associated with more constraints than other employees’ ‘An employee associated with a larger number of constraints is assigned with priority’ ‘Group leaders are associated with more constraints than other employees. A is a group leader’ ‘An employee associated with a larger number of constraints is assigned with priority. A is a group leader’ ‘An employee associated with a larger number of constraints is assigned with priority. Group leaders are associated with more constraints than other employees’ ‘An employee associated with a larger number of constraints is assigned with priority. Group leaders are associated with more constraints than other employees. A is a group leader’

has already been noted in the context of subjective evaluations of interfaces that even a small number of subjects seems to be adequate for testing the quality of the interfaces (Nielsen, 1994). Moreover, as we will see in this section, the assessments of our test-subjects are very pronounced. 6.2. SPECIFICATIONS FOR THE INQUIRY The test-subjects have been confronted with a description of an issue to be explained, a characterization of several environments, and a repertoire of possible actions, that is, alternative texts (cf. Table 2). The task set forth was to select suitable text alternatives in each situation, and to rank these alternatives according to their degrees of appropriateness. The explanation task was an aspect of the problem-solving behavior of OFFICEPLAN, referred to by the request ‘Why did the system assign employee A prior to employee B?’ In the task specification, the information relevant to answer this question was specified as consisting of two generic and chainable domain rules and appropriate instantiations applying in the given situation. The two domain rules considered can be paraphrased by ‘Employees associated with a larger number of constraints must be processed with priority’ and ‘Group leaders are associated with more constraints than other employees’. These specifications are almost identical to the task-specific problem-solving rules Rule-B and Rule-C used in our model (see Figure 9). In addition to these two rules, only the assertion expressing the instantiation ‘A is a group leader’ was specified in the task description. For the

user112.tex; 26/08/1997; 7:01; v.6; p.33

34

HELMUT HORACEK

purpose of this inquiry, Rule-C, which can be paraphrased by ‘Group leaders are associated with more constraints than project leaders’ has been slightly modified by replacing the term ‘project leaders’ by the more general term ‘other employees’. Through the introduction of this simplification, we intended to reduce the number of plausible alternative explanations, so that the task of the people rating these texts should be less time-consuming. The suitability of this decision was implicitly confirmed by the reaction of our test-raters: they selected one or several suitable texts in each of the given contexts, and no test-rater complained about missing information concerning the role of employee B in the text alternatives available. Since no test-rater proposed to add the text ‘and B is not’ to the assertion ‘A is a group leader’ we assume that the class membership of employee B was considered inferable in all contexts to the degree of precision needed, that is, B being anything but a group leader. The text alternatives which constitute the repertoire of actions available to the human raters are compositions of subsets of three propositions: namely, the two problem-solving rules and one instantiated fact. From these ingredients, seven alternative texts can be built by systematic combinations: three consisting of a single proposition, another three consisting of a pair of propositions, and, finally, one text comprising all three propositions together. To ease readability, these alternative texts are explicitly presented in Table 2 and labeled for later reference. These text alternatives differ with respect to conciseness, the degree of detail, and the degree of explicitness. In order to find out the raters’ subjective views about the relative merits of these texts, some hypothetical contexts have been specified, including the following factors:  Discourse goal: a distinction is made between a user who wants to comprehend the system behavior, and a user who intends to change system specifications (here: concerning its problem-solving heuristics).  Domain knowledge: a distinction is made between an office planning expert and a novice in that domain.  Acquaintance with the local environment: a distinction is made between a ‘local’ and an ‘external’ user. The assumption is made that the local user knows the roles of the employees in this office – in particular, who the group leaders are, whereas the external user does not know about these things.

By systematically combining varying values of these factors, eight contexts can be built. As with the text alternatives listed in Table 2, these contexts are described in Table 3 and labeled for later reference. 6.3. RESULTS OF THE INQUIRY In each of the contexts (I) to (VIII), the test-raters have selected those texts from the alternatives a) to g) which they considered appropriate in the specified situation. Moreover, if several texts have been chosen in some context, these texts have

user112.tex; 26/08/1997; 7:01; v.6; p.34

35

A MODEL FOR ADAPTING EXPLANATIONS

Table III. The context specifications used in our empirical inquiry Context Local or Expert/ Change or labels external or novice examine (I) (II) (III)

local expert local expert external expert

(IV)

external expert

(V) local novice (VI) local novice (VII) external novice (VIII) external novice

Textual description

change A local expert intends to change system specifications examine A local expert wants to examine the system behavior change An external expert intends to change system specifications examine An external expert wants to examine the system behavior change A local novice intends to change system specifications examine A local novice wants to examine the system behavior change An external novice intends to change system specifications examine An external novice wants to examine the system behavior

been ranked according to their degrees of appropriateness, in case the suitability was considered different among candidate texts, partial ties being possible. In exposing the results, we distinguish between the group of academic professionals and the group of students. While the assessments by the test-subjects are relatively consistent within each group, they differ in essential aspects across the two groups. The results are illustrated in Table 4 for the academic professionals and in Table 5 for the students. In both tables, each context occupies a row, and each text a column. Thus, information about the appropriateness of some text in a particular context can be looked up at the cell where the corresponding row and column intersect. The numbers appearing in such a cell indicate how frequently the corresponding text was considered an appropriate candidate in the corresponding context. The leftmost number stands for the nominations as the best candidate, the number next to the right stands for the second best candidate, and so on. A dash in a cell indicates that this text variant was never considered a candidate explanation in the corresponding context by the respective group of test-subjects. Hence, by summing up the leftmost numbers in a row, the result is at least the number of test-subjects (academic professionals or students, respectively). It may be more in the event that several variants are considered equally best by at least one test-subject. Looking at the results of this inquiry, as presented in Table 4 for the academic professionals, several interesting observations can be made:  In general, texts considered appropriate for experts are more concise, while texts addressing novices are more explicit. In the table, this observation manifests itself in values clustered in the left half for contexts I to IV and, even more pronounced, in the two outmost right columns for contexts V to VIII.

user112.tex; 26/08/1997; 7:01; v.6; p.35

36

HELMUT HORACEK

Table IV. Context-dependent suitability, as judged by the academic professionals Context Local Expert Change Text Text Text Text labels or or or (a) (b) (c) (d) external novice examine (I) (II) (III) (IV) (V) (VI) (VII) (VIII)

local expert change 1 4 1 local expert examine 1 1 5 external expert change 1 0 1 0 1 external expert examine 2 0 1 0 1 local novice change — 1 0 1 local novice examine — 0 0 1 external novice change — — external novice examine — —

02 02 01 — 01 01 — —

Text Text Text (e) (f) (g)

01 — 121 11 011 0002 111 1101 31 11 01 221 4 01 02 112 — 001 43 22 0001 012 61 13 101 02 11 52 001 11 01 61

Table V. Context-dependent suitability, as judged by the students Context Local Expert Change Text labels or or or (a) external novice examine (I) (II) (III) (IV) (V) (VI) (VII) (VIII)

local expert change local expert examine external expert change external expert examine local novice change local novice examine external novice change external novice examine

Text (b)

Text (c)

Text Text Text Text (d) (e) (f) (g)

102 041 032 12 221 0011 021 032 11 121 01 011 021 12 301 001 011 021 12 111 001 0001 101 0011 011 1 0 2 0 (4)1 0 0 0 1 0 0 1 1 0 1 1 — 0 (4)1 0 0 0 1 0 0 0 2 1 0 2 1 1 0 (4)1 0 (4)1 0 0 0 2 0 0 2

33 32 21 22 13 03 13 04

32 5 51 5 41 5 41 5

 All text variants available are considered suitable by at least one rater in at least one context. Only variant (c) has not been considered a first choice text under any constellation.  The discourse goal affects the raters’ choices only marginally – this result may have its source in the particular kind of task treated here.  There is a comparably strong agreement among the raters about the most suitable alternative in each context, while the degree of agreement decreases for second and third best choices.  Different discourse goals barely affected the raters’ choices in our inquiry, but they established preferences for distinctive text alternatives for local experts, external experts, local novices, and external novices.

Though the results for the group of students deviate significantly from those obtained for the group of academic professionals, all but the last observation are

user112.tex; 26/08/1997; 7:01; v.6; p.36

A MODEL FOR ADAPTING EXPLANATIONS

37

also valid for that group to a considerable degree. The major difference between the two groups of raters manifests itself in two respects:  Unlike for the academic professionals, the distinction between ‘local’ and ‘external’ users virtually does not affect the decision of the student raters.  The distinction between experts and novices affects the student raters’ selection among alternatives only insofar as more concise variants are gradually preferred as second- or third- best choices for experts, but not so much for novices.

Consequently, the most explicit text variant is preferred by our student raters in all contexts. The reasons for the differences observed between the two groups probably goes back to individual assumptions about the domain knowledge and communicative goals that our test-raters had to make. This looks plausible in view of a few statements deliberately made by some of the raters: ‘Be as concise as possible for the expert, and provide all details in a completely explicit form for the novice’ and ‘do not tell “local” users about the roles of their colleagues, but do so for “external” users’ are some of the statements made by the academic professional raters, rephrased freely in English here. In contrast to that, ‘alternative (g) is preferable in all contexts, because it is the most accurate version’, and ‘for experts, explanations may also be expressed in a less exact way, because experts have more background knowledge’ are some of the remarks made by the student raters. We do not know why each of the rater groups internally behaved in a relatively consistent way, while the assessments deviate across the groups in certain aspects. It may be the case that academic professionals are intensively trained at elaborating distinctions in tasks facing varying environments, while students are generally educated to express themselves explicitly — but this view constitutes only a subjective feeling. Because the ratings given by the academic professionals are more systematic and entail more distinctions, we restrict further discussions to the assessments given by this group. These elaborations concern reasoning about underlying motivations, as well as orienting formalizations for a user model on these assessments. Following the numbers shown in Table 4, the preferred alternatives by our academic professional raters are illustrated in Table 6. These results demonstrate that the academic professional raters carefully attempted to adapt their responses to the contextual environment, and that they mostly preferred distinct alternatives for different settings. Unlike the explanation addressing ‘external’ users, the proposition ‘A is a group leader’ is usually not included in explanations for ‘local’ users. Thus, the majority of the raters consider acquaintance with a fact an important prerequisite for leaving the recognition of the relevance of this fact to the user’s inferential capability. A minority still prefers to express this fact explicitly in such cases. Another minority apparently trusts the user to be able to infer this fact as caused by the force of conversational implicature, even though the user is assumed to be unfamiliar with this piece of information.

user112.tex; 26/08/1997; 7:01; v.6; p.37

38

HELMUT HORACEK

Table VI. Best candidate texts in dependency of the context, as obtained through our inquiry Context labels

Local Expert Change or or or external novice examine

(I) and (II) local

expert

(b)

(III) and external expert (IV) (V) and local novice (VI)

(d)

(VII) and external novice (VIII)

(g)

(f)

Text variation considered best

‘Group leaders are associated with more constraints than other employees’ ‘Group leaders are associated with more constraints than other employees. A is a group leader’ ‘An employee associated with a larger number of constraints is assigned with priority. Group leaders are associated with more constraints than other employees’ ‘An employee associated with a larger number of constraints is assigned with priority. Group leaders are associated with more constraints than other employees. A is a group leader’

As for the distinction between experts and novices, only experts are credited in an overwhelming number of cases with the capability of inferring the relevance of the rule ‘An employee associated with a larger number of constraints is assigned with priority’ from the proposition ‘Group leaders are associated with more constraints than other employees’. Some test-raters favor expressing both rules explicitly, even for experts. A few others seem to trust even novices to be able to infer the general relevance of the number of constraints for ordering from the case of group leaders, or these raters believe that it is better to reduce the degree of detail in the content presented to keep the communicative effort low. The proposal of merely uttering the deeper reason when facing a novice seems to be motivated by focusing on the central point of the issue — an eventually following explanatory discourse can explore this case in more detail. Finally, the minority of raters favoring the text variant ‘A is a group leader’ for experts seems to trust in the prominence of these two domain rules in the context of ordering employees. In the following, we illustrate how these empirical results are incorporated into our user model. 7. The User Model 7.1. CATEGORIES OF KNOWLEDGE AND THEIR ROLE IN THE USER MODEL In order to take full profit of informative explanations like those discussed in the preceding sections, an addressee must exhibit several capabilities of varying quality. Expressed in terms of the office planning domain, these capabilities comprise, among others: (1) recognizing the relevance of a prominent assignment heuristic in view of the categories of the employees involved, (2) knowledge about the ordering heuristics applied in the problem-solving method, and (3) acquaintance with the

user112.tex; 26/08/1997; 7:01; v.6; p.38

A MODEL FOR ADAPTING EXPLANATIONS

39

topology of the office and with roles and relevant properties of the employees to be assigned to the rooms. Different users exhibit these capabilities to different degrees. We believe that these judgments can reasonably be assumed to be based on the experts’ assumption that users also differ in their communicative preferences. The results obtained through our inquiry provide an additional piece of evidence by the variations observed in the judgments of the information providers. In order to adapt the behavior of an explanation model to the varying needs of its users, we have developed a user model that enables one to make decisions about the relative suitability of alternative presentations. Since the capabilities referred to in the previous paragraph are of a rather different nature, we have to integrate several categories of knowledge adequately. In order to facilitate the assignment of individual pieces of knowledge to appropriate categories, we introduce two partially overlapping category systems. One of these categorizations is task-oriented, while the other one is user-oriented. The motivation behind having these categorizations is that the task-oriented one should support the categorization of knowledge in a new domain, while the user-oriented categorization is the one needed ultimately for building the user model. We distinguish the following task-oriented categories:  knowledge about forces underlying human conversation (category 1),  general knowledge about how a certain task is accomplished (category 2),  particular knowledge about methods for accomplishing a certain task (category 3),  knowledge about the particularities of the environment in which the task is carried out (category 4).

Moreover, we believe that a good deal of knowledge belonging to either of these categories can plausibly be ascribed to users on the basis of stereotypes4 with a reasonable degree of confidence. For the purpose of associating pieces of knowledge with types of users, we distinguish between the following user-oriented categories of knowledge:  knowledge shared by all users,  knowledge that can be typically ascribed to certain classes of users,  pieces of knowledge attributed to individual users (we do not elaborate this category here).

As the results of the inquiry suggest, the model reflects the distinctions necessary to justify choices among alternatives tailored to the intended audience. 4

Ascribing knowledge to a user on the basis of stereotypes does not by far mean that a user’s knowledge necessarily is in accordance with the information found in the associated stereotypes; however, we believe that using stereotypes provides a simple method to obtain a plausible approximation under limited evidence.

user112.tex; 26/08/1997; 7:01; v.6; p.39

40

HELMUT HORACEK

7.2. BUILDING THE USER MODEL In order to build a user model on the basis of these categorizations, the following activities have to be carried out:  Pieces of domain knowledge have to be categorized in task-specific terms.  Subcategorizations may be introduced concerning both the classes of users and the specificity of tasks.  A mapping between task-oriented and user-oriented categories needs to be established.

In our model, we pursue the following strategy. Acquaintance with the small fraction of regularities underlying human conversation that is relevant here is attributed to all users. Similarly, all users are assumed to be acquainted with those pieces of domain-relevant knowledge that can be characterized as simple everyday knowledge. The justification for this design decision lies in the relatively consistent communicative behavior of people concerning their inferential capabilities observed in comparable everyday situations, as this is demonstrated by the experiments carried out by Th¨uring and Wender. In principle, deviations could be incorporated into our model by deactivating conversational implicature rules, or by allowing information being entered into the user model on an individual basis, but we do not elaborate these aspects. Unlike for general communicative capabilities, we assume the command of domain knowledge to differ significantly across classes of users and individuals. In order to schematically organize pieces of domain expertise, we have to categorize the domain rules in our model. Moreover, we subcategorize general knowledge about how a task is accomplished (category 2) into general world knowledge relevant for the domain and general domain expertise. Some examples for these categories of knowledge are (a complete record is given in Table 7):  General world knowledge relevant for the domain (parts of category 2): Even in specialized domains, some simple pieces of everyday knowledge typically contribute to any model in such a domain. In office planning, the rule expressing that ‘Smoker and smoker-intolerant persons should be assigned to different rooms’ falls into this category. In addition, there are rules in OFFICE-PLAN which are so obvious and general that they are not even handed over from the problem-solving component to the explanation module. ‘Everyone should be put in a room’, and ‘Rooms should not be overfilled’ are examples.  General domain expertise (parts of category 2): This category concerns pieces of knowledge on which all experts in the domain widely agree, irrespective of their acquaintance with particular systems. We consider the rule ‘Group leaders should be assigned to single rooms’ a typical example in office planning, which can be assumed to be part of the majority of systems in that domain.  Special domain knowledge (category 3): This knowledge category requires acquaintance with particularities of the concrete problem-solving mechanism,

user112.tex; 26/08/1997; 7:01; v.6; p.40

41

A MODEL FOR ADAPTING EXPLANATIONS

Table VII. Categorizing domain inference knowledge and problem-solving knowledge for building user stereotypes Domain Rules (re-phrased freely here)

Known to User Category

‘Resources should not be overused’ ‘Rooms should not be overfilled’ ‘Everyone should be put in a room’ ‘Group leaders should be assigned to single rooms’ ‘Group leaders and secretaries should be assigned to adjacent rooms’ ‘Group leaders and project leaders should be assigned to near rooms’ ‘Group leaders should be assigned to a room near the discussion room’ ‘Employees that frequently have meetings and full time employees should be assigned to different rooms’ ‘Smokers and smoker-intolerant persons should be assigned to different rooms’ ‘Employees working on the same project should be assigned to different rooms’ ‘Employees with no common themes should be assigned to different rooms’

all users all users all users general domain experts

Problem Solving Rules (re-phrased freely here) ‘Group leaders are associated with more constraints than project leaders’ ‘Project leaders are associated with more constraints than secretaries’ ‘Secretaries are associated with more constraints than ordinary employees’ ‘Employees associated with a larger number of constraints are assigned prior to others’

general domain experts general domain experts special domain experts special domain experts all users special domain experts general domain experts

Known to User Category general domain experts general domain experts general domain experts general domain experts

the strategies employed, and the assumptions and assessments incorporated; some of them may be atypical for the domain. ‘Employees working on the same project should be assigned to different rooms’ is probably the most unusual rule in OFFICE-PLAN. In this system, the motivation is to disseminate experiences made within individual projects across other projects, whereas the usual motivation in the assignment of employees is to ease communication within a project and not across projects.  Knowledge about the local environment (category 4): This category comprises acquaintance with the room topology in this office and with the roles of the employees to be assigned. When assigning pieces of knowledge to user model

user112.tex; 26/08/1997; 7:01; v.6; p.41

42

HELMUT HORACEK

Figure 16. Specialization hierarchy of user stereotypes.

entries, we merely distinguish between complete acquaintance and complete unfamiliarity with the relevant aspects of the local environment. We build our user stereotypes according to these subcategorizations. The relations between the resulting stereotypes are illustrated in Figure 16. Inheritance of pieces of knowledge along this hierarchy works in an orthogonal way, thereby combining domain-specific expertise and acquaintance with the local environment. In our model, the assignment of the pieces of knowledge to stereotypes does not require performing inheritance in a non-monotonic way. In principle, introducing more subtle distinctions with respect to the underlying expertise is possible. This may include distinctions concerning partial acquaintance with properties of a domain, intermediate levels between general domain expertise and acquaintance with particular systems, and partial acquaintance with the local environment. In view of the local environment presented, the distinctions made are mostly sufficient. Especially for a larger environment, where partial acquaintance can be ascribed to some users in a motivated way, both known and unknown areas of the local environment may be relevant for an explanation. In such a case, the resulting explanations may be influenced by a finer-grained model. The concrete assignment of individual pieces of knowledge to stereotypes may be debatable in some instances. Therefore, we propose to apply a cautious strategy: if in doubt, do not credit a certain category of persons with knowing a certain piece of information. Nevertheless, even if a certain assignment made in a stereotype unfortunately proves to be wrong and becomes crucial in the course of an explanation generation process, the resulting utterance should still not lead to a severe problem. Either the utterance entails an avoidable redundancy if the hearer is told something he/she already knows or can infer with ease, or the hearer is missing a piece of information needed to understand the utterance. In such a case, a clarification dialog is needed, but we did not investigate this issue — this issue can be achieved by incorporating other work on RST-based explanations (e.g., (Moore, 1989)). Such situations may occur from time to time even between human conversants, since human communication is inherently fallible. However, the general

user112.tex; 26/08/1997; 7:01; v.6; p.42

A MODEL FOR ADAPTING EXPLANATIONS

43

aim of our method is to improve the quality of the utterances produced over a significantly large set of situations and we tolerate an occasional failure due to wrong assumptions about the addressee in favor of reducing boring redundancies in a significant number of instances. We continue with applying the assumptions represented in our user model for inserting annotations in a slightly more complex text plan than that one illustrated in Section 5, thereby contrasting the effects of using two different user stereotypes. These elaborations are oriented on the preferences expressed by the academic professionals in the course of our inquiry. 8. Impact of Assumptions about The Addressee 8.1. BUILDING ANNOTATED TEXT PLANS We demonstrate the impact of assumptions about the addressee by generating explanations to answer the request ‘Why has person A been assigned prior to person E?’ Up to a certain stage, the plan expansion process proceeds in a similar way as that described in the example in Section 5, when the explanation to ‘Why is person A in room B and not in room C?’ is elaborated. Only one structural difference occurs since the domain model can provide here more specific details about the underlying generic condition: stating that ‘Employees associated with a larger number of constraints are processed prior to others’ is further elaborated by the rule ‘Group leaders are associated with more constraints than project leaders’. Let us look at the plan expansion process in more detail. As in the example discussed in Section 5, CAUSE (cond-1) is provided for the fact under question (order-1), which is elaborated by a general-condition (Rule-B). At this stage, the only structural difference to the previous example occurs, which consists in additionally inserting Rule-C (via the relation generic-specific); Rule-C constitutes a more specific reason for order-1 than Rule-B does. From there, appropriate instantiations, inst-1 and inst-2, are introduced through an abstract-instance relation, all of them linked by a JOINT relation, similarly to the explanation demonstrated in Section 5. The resulting text plan is depicted in Figures 17 and 18 in two versions, which differ only in terms of the annotations made. Figure 17 entails an annotated text plan for addressing a novice. The two instantiations inst-1 and inst-2 are derived from the conjunction of Rule-B and Rule-C via C-Rule 1. Moreover, cond1 is derived from all these propositions via C-Rule 6, that is, logical substitution. Figure 18 entails the same text plan with annotations suitable for an expert. These are the same as those for the novice; in addition, annotations are shown which mirror the inferences deriving the relevance of Rule-C from inst-1 from which, in turn, the relevance of Rule-C is derived. In Figure 18, the two annotations specific to the expert are marked by thicker arrows (both due to applications of C-Rule 3). In contrast to the plan expansion process, the inferences that allow performing annotations in the text plans differ significantly from those made in connection with the example discussed in Section 5. These annotations crucially depend on plausible

user112.tex; 26/08/1997; 7:01; v.6; p.43

44

HELMUT HORACEK

Figure 17. Annotated text plan explaining the ordering of elements in a process to a novice.

expectations about the addressees for which it is necessary to know whether the user is acquainted with the two domain rules ‘Group leaders are associated with more constraints than project leaders’ and ‘Employees associated with a larger number of constraints are assigned prior to others’. According to the user stereotypes shown in Section 7, experts are assumed to be acquainted with both pieces of knowledge, but novices with none of them. According to these assumptions, the following inferences are supported. For all users, including novices: According to the assumptions made about these classes of users, two kinds of inferences are licensed. Both propositions asserting class memberships of the persons in the current focus of attention, ‘A is a group leader’ and ‘E is a project leader’, can be inferred from the conjunction of the two domain rules ‘Group leaders are associated with more constraints than

user112.tex; 26/08/1997; 7:01; v.6; p.44

A MODEL FOR ADAPTING EXPLANATIONS

45

Figure 18. Annotated text plan explaining the ordering of elements in a process to an expert.

project leaders’ and ‘Employees associated with a larger number of constraints are assigned prior to others’, the former being substituted into the latter. In each case, these inferences are licensed through the force of C-Rule 1, that is, deductionenabling hypothetical instantiation. Note that, in contrast to the example discussed in Section 5, neither scalar implicature nor deduction yield successful inferences here. Scalar implicature would only allow one to conclude that E is not a group leader, but this rule is too weak to identify E as a project leader. However, it could be argued that knowing E not to be a group leader is sufficient to understand the ordering priority. While this is certainly true, our model misses some steps in the required line of reasoning. The database only has rules expressing that group leaders are associated with more constraints than project leaders, that project leaders are associated with more constraints than secretaries, etc. However, the inference

user112.tex; 26/08/1997; 7:01; v.6; p.45

46

HELMUT HORACEK

which yields the fact that group leaders are associated with more constraints than any other kind of employee is not drawn since such an abstraction from all several rules is beyond the system’ capabilities. The reason for the failure of deduction lies in the structural difference between Rule-B, the domain rule involved in the example discussed here, and Rule-A, which is relevant in the example discussed in Section 5. In that example, the fact that B is a single room is deductively inferable because that predicate appears in the rule’s conclusion. In contrast, the predicate PROJECT-LEADER appears in the premise of the Rule-B, as does the predicate GROUP-LEADER. Hence, E being a project leader is derived the same way as A’s class membership, via hypothetical instantiation. In addition, acquaintance with both instantiations relevant here, inst-1 and inst-2, as well as evidence about the relevance of the rules ‘Group leaders are associated with more constraints than project leaders’ and ‘Employees associated with a larger number of constraints are assigned prior to others’ enables one to conclude cond-1 (‘A is required to be processed before E’) via logical substitution, as expressed by C-Rule 6. In addition to the inferences stated above, expert knowledge enables the derivation of the general domain rule, Rule-B, ‘Employees associated with a larger number of constraints are assigned prior to others’ from the more special one, Rule-C, ‘Group leaders are associated with more constraints than project leaders’, via C-Rule 3. The latter rule, in turn, can be derived from the conjunction of the two class memberships, ‘A is a group leader’ and ‘E is a project leader’, by C-Rule 3 again. This inference is licensed by the prominence of Rule-C for group leaders and project leaders in the context of ordering, a subtask in the problem-solving process. The latter aspect is established by the context-dependent evaluation of the predicate SALIENT, which contributes to obtain preferences among competing domain rules. As can be seen from the annotations made in the text plan in Figure 17, appropriate explanation texts conveying the full information to novices always include the two domain rules ‘Group leaders are associated with more constraints than project leaders’ and ‘Employees associated with a larger number of constraints are assigned prior to others’. Other propositions can optionally be mentioned, too. If one of these rules is missing, the explanation can still be considered useful, but it does not fully convey the intended information. This fact is quite in accordance with the results obtained from our inquiry; at least one of the two domain rules is stated explicitly by almost each of the raters, with the exception of a few proposals made by members of the student group. When addressing an expert, there are basically two choices. Either the more special domain rule stating that ‘Group leaders are associated with more constraints than project leaders’ or the two class memberships, ‘A is a group leader’ and ‘E is a project leader’ must be included in the explanatory text. The second variant is very similar to the shortest variant in the situation underlying our inquiry – ‘A is a group leader’ – since the communicative intention was specified a bit more

user112.tex; 26/08/1997; 7:01; v.6; p.46

A MODEL FOR ADAPTING EXPLANATIONS

47

modestly there: person E only needs to be identified as an employee other than a group leader, whereas that person should be identified as a project leader here. Some effects that result when deviating from the central parts of the explanation for the expert are worth mentioning. As in the case before, other propositions may be optionally added. Only mentioning the more general rule instead of the more special one results in a loss of information, as indicated by our inference mechanism. If only the more important class membership, ‘A is a group leader’, is included in the explanation, some probably tolerable loss of precision occurs in conveying the reason for the ordering adopted by the system. 8.2. CHOOSING FROM OPTIONS – DETERMINING PREFERENCES AND EXPLOITING THEM The scope of our inference mechanism is confined to the generation of annotations in text plans. Through these, options are created for excluding the explicit utterance of certain propositions according to the inferential capabilities attributed to the audience. Hence, for deciding about the concrete text variant to be presented to the user, additional criteria must be invoked to guide the selection of preferred alternatives among all options available. The choice should be oriented on the general preferences established through our inquiry, which may be modified in view of particular pragmatic requirements favoring conciseness or verbosity. Recapitulating the results from our inquiry, the preferences observed can be characterized in the following terms:  If an individual fact (here: a class membership) is unknown to the audience, it is stated explicitly, even if it could be contextually inferred from a generic regularity included in the explanation; if, however, the individual fact constitutes shared knowledge, its relevance for the actual situation is left to be inferred by the addressee(s).  If the addressee is credited with expertise in the domain of application, the inference concluding a general rule from a more specific one is left implicit. Moreover, stating a rule is generally preferred to stating a referential fact, if one can be inferred from the other. However, if the rule is considered prominent in view of a fact, this choice may be inverted, according to a minority of our test-raters. We come back to this issue in subsection 8.3. when comparing the two explanations discussed in Sections 5 and 8, respectively.

From these considerations, we have extracted the following preference criteria:  Concerning C-Rules 5 to 8 (scalar implicature, logical substitution, deduction and abduction): Always leave the results of simple applications of these rules implicit, which seems to meet rather general cases of ‘direct causes’.  Concerning C-Rules 3 and 4 (hypothetical instantiation): Implicitly infer the relevance of a rule from another rule, as a special case of C-Rules 3 and 4.

user112.tex; 26/08/1997; 7:01; v.6; p.47

48

HELMUT HORACEK

Table VIII. Explanations favored according to context Local or Expert or external novice

Text favored according to context

local external

expert ‘Group leaders are associated with more constraints than project leaders’. expert ‘Group leaders are associated with more constraints than project leaders. A is a group leader, and E is a project leader’. local novice ‘An employee associated with a larger number of constraints is assigned with priority. Group leaders are associated with more constraints than project leaders’. external novice ‘An employee associated with a larger number of constraints is assigned with priority. Group leaders are associated with more constraints than project leaders. A is a group leader, and E is a project leader’.

However, leave the inference of a rule from a fact implicit only if the rule is prominently inferable from the fact with a high degree of certainty.  Concerning C-Rules 1 and 2 (hypothetical causality): Implicitly infer the relevance of a fact from a rule, if the fact is known to the audience. However, if the fact is not known to the audience, one should discharge the addressees’ inferential capability by explicitly mentioning that fact. In our example, these preference criteria favor the following explanations according to assumptions made about the audience, as illustrated in Table 8. Moreover, if conciseness is particularly favored, the explanation ‘A is a group leader, and E is a project leader’ constitutes a reasonable alternative when addressing an expert. In this context, conciseness is interpreted as the minimal number of facts and domain regularities conveyed explicitly, domain regularities counting twice as much as the simple facts. As one can easily verify, these texts are entirely in accordance with the results obtained by our inquiry for the group of academic professionals; the differences stem from the slight change in the situation presented in the inquiry and the situation underlying the explanation discussed in this section. 8.3. REVIEWING PREFERENCES AND UNDERLYING REASONS Some final remarks in this section are devoted to the differences between the examples illustrated in Figures 15, 17, and 18, which give rise to additional considerations about the suitability of alternative realizations, depending on particularities of the environment. In the example discussed in this section, the deeper reason stating that ‘Because group leaders are associated with more conditions than project leaders, and A is a group leader’ can be considered even a reasonable, though probably not optimal explanation for a novice. The fact that A is associated with more constraints than E can be inferred, and this fact is recognizable as the reason for ordering employees in the assignment process. However, there is no evidence for

user112.tex; 26/08/1997; 7:01; v.6; p.48

A MODEL FOR ADAPTING EXPLANATIONS

49

the addressee that the system’s domain knowledge entails an explicit rule (Rule-B) relating the number of constraints and the assignment priority. In the example illustrated in Figure 15, we have seen that even merely stating ‘A is a group leader’ should be sufficient to illustrate the reason for the order of processing to an expert. However, an expansion like ‘A cannot be assigned to C, since he/she is a group leader – he/she has been nominated in 1991’ can rarely be abbreviated to ‘A cannot be assigned to C, since he/she has been nominated as a group leader in 1991’ and ‘A cannot be assigned to C, since group leaders must be assigned to single rooms – this is because group leaders have meetings frequently’ can also not be abbreviated with confidence to ‘A cannot be assigned to C, since group leaders have meetings frequently’. These utterances fall under the category of ‘indirect causes’ according to (Th¨uring, Wender, 1985), which can be considered as empirical evidence for the low suitability of these abbreviated texts. The lines of reasoning demonstrated below, which are beyond the capabilities of our model, are likely to contribute to a principled distinction between ‘direct’ and ‘indirect’ causes. In the first case, ‘A cannot be assigned to C, since he/she has been nominated as a group leader in 1991’, the additional argument provides further evidence for the fact that A is a group leader rather than for Rule-A, so that uttering this argument pursues a different rhetorical goal. Hence, a reader might infer that the date is important to the assignment rather than to the fact that A is a group leader. Thus, if the goal is to trigger the relevance of Rule-A, it would be easier, and, therefore, also be expected by the hearer, that simply ‘A is a group leader’ is stated rather than details about the person’s promotion; this is another aspect of conversational implicature, following the Gricean maxim of brevity. These abbreviated variants seem only acceptable in special environments, where the fact mentioned is prominently important in the conversation, and the addressee is credited with an exceptionally good inference capability and the willingness to exploit it. However, the slightly different verbalization, ‘A is a group leader now’ would be perfectly acceptable, even under no special assumptions. In the second case, ‘A cannot be assigned to C, since group leaders have meetings frequently’, the additional argument provides a motivation for Rule-A rather than further evidence for the assignment itself. Hence, it is hardly conceivable to think of an addressee to whom this variant would be adequate: if he/she does not know the domain rule, its relevance cannot be triggered, and if he/she knows it, that is, if he/she is an expert, also the motivation is known to him/her, so that no new information is presented. Even though reasoning about these subtleties is beyond the capabilities of our model, we think that our inference model generally exhibits a reasonable behavior by being oriented on the insights gained through our inquiry.

user112.tex; 26/08/1997; 7:01; v.6; p.49

50

HELMUT HORACEK

9. Comparisons with Other Approaches In this paper, we have concentrated our efforts on applying contextually-motivated inferences to those types of rhetorical relations that prove particularly relevant for our application, that is CAUSE, GENERAL-CONDITION, JOINT, GENERALSPECIFIC, and ABSTRACT-INSTANCE. Further types of relations to which our mechanism can be suitably applied are SEQUENCE, RESULT, PROCESS-STEP, and their specializations, since these relations express aspects of state-action transitions and action sequences which are widely associated with default expectations. The cross-inferability between actions and their resulting states has been demonstrated in several places, for instance, by examining multi-lingual maintenance manuals (R¨osner, 1993). There is ample evidence for the effect of default expectations due to empirical experiments (Kintsch, Keenan, McKoon, 1974, Th¨uring, Wender, 1985). As a consequence of the concise realizations obtained by our approach, more content can be provided at one shot in an explanation. A side effect of this achievement is that the necessity of follow-up questions is reduced which arose frequently in the examples presented in (Moore, Swartout, 1989). 9.1. APPROACHES TO USER MODELING Taking user concerns into account in generating explanations and in other kinds of communicative actions requires activity in several areas: representing evidence and assumptions about the intended audience, updating the resulting model in the course of interaction and maintaining consistency within the model. This does not necessarily mean that all user beliefs are consistent with each other; additionally, it exploits the information represented in a user model to purposefully select among available actions. In our model, we have addressed the first and the last aspect. Unlike most principled approaches to represent information in a user model (for instance, BGP-MS (Kobsa, 1990)), we have confined our model to user beliefs, and we have neglected user goals. We have introduced this simplification in order to concentrate our efforts on modeling the cross-inferability of generic and referential pieces of information. Assumptions about user beliefs are collected in stereotypes, which can be combined through orthogonal multiple inheritance, as in (Kobsa, M¨uller, Nill, 1994), an application of BGP-MS. Moreover, the content of user beliefs to be represented in our model amounts to simple facts and rules whose internal structure does not affect the beliefs about them. Hence, there is no need to analyze the content of user beliefs in our application as, for instance, this is done in connection with the hierarchical decomposition of plans in (Peter, R¨osner, 1994). As a consequence of neglecting user goals, beliefs can simply be represented as flat lists, which is in contrast to nested contexts typically used in the field for representing the beliefs and goals of the participating agents. A further justification for this simplification is given by our inquiry whose results can essentially be reproduced on the basis of beliefs, as we have done in our model.

user112.tex; 26/08/1997; 7:01; v.6; p.50

A MODEL FOR ADAPTING EXPLANATIONS

51

Modeling an explanation situation in its entirety requires a considerable amount of effort, since explanation is commonly seen as an interactive process rather than a single-shot activity. Therefore, we have restricted the scope of our method to aspects of a snapshot in this process. Hence, our approach elaborates a particular aspect mostly relevant for explanations, which could certainly be combined with complementary ingredients: (1) means to phrase a text in terms familiar to the user (Bateman, Paris, 1989), (2) acquisition of knowledge about the user’s propositional attitudes (Kass, Finin, 1987), and (3) exploitation of user feedback to compensate for the unreliability of user models (Moore, Paris, 1992).

9.2. APPROACHES TO EXPLOIT INFERENCES In general, a variety of approaches exploit evidence about the user’s mental state, but only a few ones address the user’s inferential capability as we do. We are aware of at least three approaches which bear strong similarities to ours: an earlier approach to presenting lines of reasoning, which is the explanation facility BLAH (Weiner, 1980), and two more recent approaches exploiting the user’s inferential capabilities (Green, Carberry, 1994, Zukerman, McConachy, 1993). Here, we address the first two approaches only, since we already have discussed Zukerman’s and McConachy’s work in Section 2.1. The explanation facility BLAH applies content structuring measurements, which includes omitting a generic regularity from the message to be communicated in case the actually relevant instances of this regularity are to be conveyed. Our approach constitutes progress over this capability of BLAH in two respects: Firstly, it does not only exploit implications between a generic regularity and some of its instances, but also between a regularity and instances of its premise or conclusion. Secondly, it is integrated in a text planner so that its specifications can be interpreted in a linguistically motivated way to produce a variety of surface expressions. Green and Carberry aim at the generation of indirect answers by reasoning on the basis of coherence rules. The effect they achieve is similar to ours in so far as part of the information to be conveyed is not uttered explicitly, as a consequence of consulting applicable coherence rules. There is, however, a crucial difference in the kind of knowledge which guides drawing inferences: in their approach, state-event transitions and potential obstacles preventing intended achievements are modeled by coherence rules, whereas we mostly aim at capturing inference relations between generic and referential pieces of knowledge through rules expressing aspects of conversational implicature. Moreover, the planning process in their approach covers the coherent composition of individual, mostly single-clause, utterance by planning it in the context of a whole discourse, whereas our model aims at planning a text for an eventually longer single-shot dialog contribution.

user112.tex; 26/08/1997; 7:01; v.6; p.51

52

HELMUT HORACEK

10. Realization and Extensions The model presented in this paper has been designed in the course of developing the system DIAMOD (Peters, 1993), whose aim was to generate explanations about the solutions obtained by OFFICE-PLAN and about its problem-solving behavior. Within the scope of DIAMOD, a simplified control mechanism has been implemented that evaluates the contextually-motivated inference rules in view of a logical form that represents a full specification of the explanation to be generated. As a consequence of this evaluation, the logical form is modified so that those pieces of information considered inferable by the addressee are eliminated. The original logical form is composed by schema-based mechanism (Sprenger, 1993), which does not allow an easy integration of our methods for controlling the crossinferability of information. After work on the system DIAMOD has been finished, the mechanism evaluating the contextually-motivated inference rules has been redesigned in such a way that it is widely compatible with techniques used in other approaches. This redesign led to the form shown in this paper, in which the plausibility of the incorporation of our mechanism into typical RST-based text planners has been demonstrated. As it turned out, the model presented is quite complex, if considering it in all details. In some sense, this is not surprising, since the forces underlying the associated reasoning processes are modeled on a rather deep level and in a rather principled way. In our view, there are two major sources for this complexity: first, instantiating the conversationally motivated rules requires matching the variables appearing in these rules with sets and subsets of discourse entities, and, second, the number of variants of conversationally motivated rules may eventually grow too much in larger domains. The second issue is a mere technical one, and it may be overcome by designing and using a more powerful description language for these rules. The first issue is of a more principled nature. A possible strategy would be to exploit domain-specific restrictions and to incorporate them in the conversationally motivated rules which should lead to a simplification of the formalizations of these rules. Finally, restricting the chaining of inference steps left implicit, which is done here by implementing the distinction between ‘direct’ and ‘indirect causes’, as advocated for by Th¨uring and Wender, may not easily carry over to other domains. In such a case, reasonable assumptions about the addresses’ reasoning capabilities must be made, thus limiting the number of chained inference steps left implicit in the presentation. Major sources for extending the approach constitute the incorporation of aggregation and cross-dependencies to the issue of building referring expressions. The latter may occasionally demand that a non-minimal portion of the text plan is uttered explicitly to support building referring expressions. This is the case if generic and referential assertions occur freely mixed in the text plan and some of them need not being uttered explicitly. Performing aggregation creates new propositions and, therefore, it influences the contextual inferability of steps in the

user112.tex; 26/08/1997; 7:01; v.6; p.52

A MODEL FOR ADAPTING EXPLANATIONS

53

text plan obtained before aggregation. Hence, in an integrated approach, performing aggregation operations during text planning might make revision operations necessary. 11. Conclusion In this paper, we have presented a model for anticipating contextually-motivated inferences addressees are likely to draw. The model is used to motivate choices in presenting or omitting individual pieces of information, thereby taking into account the addressees’ domain expertise and their expectations about logical consequences of purposefully presented information. We have formulated and motivated principles which guide the application of these kinds of inferences, and we have presented detailed formalizations in terms of rules. Furthermore, we have shown how the evaluation of these rules can be incorporated into typical RST-based text planners, which leads to annotated text plans indicating the cross-inferability of information, in specific situation. To support systematicity in adapting this method to particularities of the addressee, we have presented a structured user model in which relevant information typically derived from stereotypes can be represented. This information is used for making reasonable assumptions about the addressee with respect to the inferences he/she is likely to draw, as well as for choosing among alternative texts in which parts of the conveyed content are left to be uncovered by the addressee’s inferential capabilities. An inquiry has been carried out to put our assumptions and preferences on a plausible empirical basis. By incorporating the mechanism developed in a text planning module, we have achieved an important improvement. This additionally represents a prerequisite for generating explanations in different degrees of depth and detail which is, in turn, partially motivated by the effort to convey the information required. As we have shown in detail, the method is powerful enough to achieve the improvements illustrated in Figure 1 for the domains of EES and OFFICE-PLAN. It is also moderately sensitive to the issues discussed in the introduction: avoiding boring redundancy, but maintaining coherence and ease of comprehension. Acknowledgements We would like to thank the following colleagues for participating in our inquiry and providing us with assessments about alternative explanations: Hermann Kaindl, Elisabeth Maier, Andreas Mertens, Stephan Mehl, Jens-Uwe M¨oller, Katharina Morik, Thomas Rist, and Manfred Stede. We are particularly indebted of Russell Block and his students, whose assessments about alternative explanations provided us with an additional amount of empirical evidence. Finally, we would like to thank Louise Senior for proof-reading the paper.

user112.tex; 26/08/1997; 7:01; v.6; p.53

54

HELMUT HORACEK

References Appelt, D.: 1985, ‘Planning English Sentences’. Cambridge University Press. Bateman, J. and Paris, C.: 1989, ‘Phrasing a Text in Terms the User can Understand’. In: Proceeding of IJCAI-89, Detroit, 1511–1517. Carberry, S.: 1988, ‘Modeling the User’s Plans and Goals’. In: Computational Linguistics 14(3), 23–37. Carroll, J.: 1990, ‘The Nurnberg Funnel: Designing Minimalist Instruction for Practical Computer Skill’. MIT Press, Cambridge, Massachusetts. Cawsey, A.: 1990, ‘Generating Explanatory Discourse’. In: R. Dale, C. Mellish, M. Zock (eds): Current Issues in Natural Language Generation, Academic Press, 75–102. Garnham, A.: 1982, ‘Testing Psychological Theories about Inference Making’. In: Memory and Cognition 10(4), 341–349. Green, N. and Carberry, S.: 1994, ‘A Hybrid Reasoning Model for Indirect Answers’. In: Proceedings of ACL-94, Las Cruces, New Mexico. Grice, H.: 1975, ‘Logic and Conversation’. In: Syntax and Semantics: Vol 3, Speech Acts, Academic Press, 43–58. Hirschberg, J.: 1991, ‘A Theory of Scalar Implicature’. Garland Press, New York. Horacek, H.: 1991, ‘Exploiting Conversational Implicature for Generating Concise Explanations’. In: J. Kunze, D. Reimann (eds): Proceedings of EACL-91, Berlin, 191–193. Horacek, H.: 1992, ‘Explanations for Constraint Systems’. In: B. Neumann (ed): Proceedings of ECAI-92, Vienna, 500–504. Horacek, H.: 1994, ‘How to Avoid Explaining Obvious Things (Without Omitting Central Information)’. In: T. Cohn (ed):Proceedings of ECAI-94, Amsterdam, 520–524. Joshi, A., Webber, B. and Weischedel, R.: 1984, ‘Living Up to Expectations: Computing Expert Responses’. In: Proceedings of AAAI-84, Austin, Texas, 169–175. Karbach, W., Linster, M. and Voß, A.: 1989, ‘OFFICE-PLAN: Tackling the Synthesis Frontier’. In: D. Metzing (ed.): Proceedings of GWAI-89, Springer, Berlin, 379–387. Kass, R. and Finin, T.: 1987, ‘Rules for the Implicit Acquisition of Knowledge about the User’. In: Proceedings of AAAI-87, Seattle, WA, 295–300. Kintsch, W., Keenan, J. and McKoon, G.: 1974, ‘Memory for Information Inferred During Reading’. In: W. Kintsch (ed.): The Representation of Meaning in Memory, Earlbaum, Hillsdale, New Jersey. Kobsa, A.: 1990, ‘Modeling a User’s Conceptual Knowledge in BGP-MS, a User Modeling Shell System’. In: Computational Intelligence 6, 193–208. Kobsa, A., M¨uller, D. and Nill, A.: 1994, ‘KN-AHS: An Adaptive Hypertext Client of the User Modeling System BGP-MS’. In: A. Kobsa (ed.): Proceedings of the 4th International Conference on User Modeling, Hyannis, MA, 99–105. Lascarides, A. and Oberlander, J.: 1992, ‘Abducting Temporal Discourse’. In: R. Dale, E. Hovy, D. R¨osner, O. Stock (eds): Aspects of Automated Language Generation, Springer, Berlin, 167–182. Litman, D. and Allen, J.: 1987, ‘A Plan Recognition Model for Subdialogues in Conversation’. In: Cognitive Science 11, 163-200. Maier, E. and Hovy, E.: 1993, ‘Organizing Discourse Structure Relations Using Metafunctions’. In: H. Horacek, M. Zock (eds): New Concepts in Natural Language Generation, Pinter, London, 69-86. Mann, W. and Thompson, S.: 1987, ‘Rhetorical Structure Theory: A Theory of Text Organization’. In: L. Polanyi (ed.): The Structure of Discourse, Norwood, Ablex. McKeown, K: 1985, ‘Discourse Strategies for Generating Natural Language Text’. In: Artificial Intelligence 27, 1-41. Meier, J.: 1991, ‘Ein Algorithmus zur Erzeugung konversationeller Implikaturen’. DIAMOD-Report Nr. 9, University of Bielefeld. Moore, J.: 1989, ‘A Reactive Approach to Explanation in Expert and Advice-Giving Systems’. Ph. D. Dissertation, University of Southern California, Los Angeles. Moore, J. and Paris, C.: 1989, ‘Planning Text for Advisory Dialogs’. In: Proceedings of ACL-89, Vancouver, 203-211. Moore, J. and Paris, C.: 1992, ‘Exploiting User Feedback to Compensate for the Unreliability of User Models’. In: User Modeling and User-Adapted Interaction 2(4), 287–330.

user112.tex; 26/08/1997; 7:01; v.6; p.54

A MODEL FOR ADAPTING EXPLANATIONS

55

Moore, J. and Paris, C.: 1993, ‘Planning Text for Advisory Dialogs: Capturing Intentional and Rhetorical Information’. In: Computational Linguistics 19(4), 651–694. Moore, J. and Swartout, W.: 1989, ‘A Reactive Approach to Explanation’. In: Proceedings of IJCAI89, Detroit, 1504–1510. Nielsen, J.: 1994, ‘Estimating the Number of Subjects Needed for a Thinking Aloud Test’. In: International Journal of Human Computer Studies 41, 385–397. Paris, C.: 1988, ‘Tailoring Object Descriptions to a User’s Level of Expertise’. In: Computational Linguistics 14, 64–78. Paris, C.: 1993, ‘User Modeling in Text Generation’. Pinter, London. Peter, G. and R¨osner, D.: 1994, ‘User-Model-Driven Generation of Instructions’. In: User Modeling and User-Adapted Interaction 3, 289–319. Peters, K.: 1993, ‘Generierung nat¨urlichsprachlicher Erkl¨arungen in DIAMOD – Eine Systembeschreibung’. DIAMOD-Report Nr. 24, University of Bielefeld. Reiter, E.: 1990, ‘Generating Descriptions that Exploit a User’s Domain Knowledge’. In: R. Dale, C. Mellish, M. Zock (eds): Current Issues in Natural Language Generation, Academic Press, 257–285. R¨osner, D.: 1993, ‘Intentions, Rhetoric, or Discourse Relations? A Case from Multilingual Document Generation’. In: O. Rambow (ed.): Intentionality and Structure in Discourse Relations, Workshop held at the ACL-93, Columbus, Ohio, 106–109. Sprenger, M.: 1993, ‘Explanation Strategies for KADS-Based Expert Systems’. In: H. Horacek, M. Zock (eds): New Concepts in Natural Language Generation, Pinter, London, 27–56. ¨ Th¨uring, M. and Wender, K.: 1985, ‘Uber kausale Inferenzen beim Lesen’. In: Sprache und Kognition 2, 76–86. Weiner, J.: 1980, ‘BLAH, A System Which Explains its Reasoning’. In: Artificial Intelligence 15, 19–48. Zukerman, I. and McConachy, R.: 1993, ‘Generating Concise Discourse that Addresses a User’s Inferences’. In: Proceedings of IJCAI-93, Chambery, France, 1202–1207. Zukerman, I. and McConachy, R.: 1994a, ‘Discourse Planning as an Optimization Process’. In: Proceedings of 7th International Workshop on Natural Language Generation, Kennebunkport, Massachusetts, 37–44. Zukerman, I. and McConachy, R.: 1994b, ‘Being Concise Versus Being Shallow: Two Competing Discourse Paradigms’. In: Proceedings of ECAI-94, Amsterdam, The Netherlands, 515–519. Zukerman, I. and McConachy, R.: 1995, ‘Generating Discourse Across Several User Models: Maximizing Belief while Avoiding Boredom and Overload’. In: Proceedings of IJCAI-95, Montreal, Canada, 1251–1257.

Author’s Vita Dr. H. Horacek University of Saarbr¨ucken, Dept. of Computer Science, PO. Box 1150, D-66041 Saarbr¨ucken, Germany Dr. H. Horacek is a research staff member at the Dept. of Computer Science at the University of Saarbr¨ucken, working in the area of natural language generation. He recieved his Doctoral degree at the Technical University of Vienna, in the area of computer chess. Dr. Horacek has worked in research projects on several aspects of natural language processing at the universities of Vienna, Hamburg, and Bielefeld. His research interests include natural language generation, explanations, and search methods. His contribution describes the detailed elaboration of a model of pragmatically motivated inferences. This work originates from a research project that aimed at the generation of explanations in expert system, which was a joint effort of the University of Bielefeld and the GMD FIT, St. Augustin.

user112.tex; 26/08/1997; 7:01; v.6; p.55