Instructional Science 27: 373-401, 1999. 9 1999 KluwerAcademic Publishers. Printed in the Netherlands.

373

Supplementing floundering text with adjunct displays KENNETH A. KIEWRA 1, DOUGLAS F. KAUFFMAN t, DANIEL H. ROBINSON 2, NELSON E DUBOIS 3 & RICHARD K. STALEY 3 1University of Nebraska-Lincoln; 2 University of South Dakota; 3SUNY Oneonta

Abstract. Three experiments compared the learning potential of text versus outline and matrix displays. In Experiments 1 and 2, college students read or heard a passage about fish and then studied the text, an outline, or a matrix. In Experiment 3, students heard a passage about wildcats, and then studied text, outline, or matrix displays. In all experiments, the text, outline, and matrix formats were informationally equivalent. However, the two-dimensional matrix appeared more computationally efficient than the linear organized text or outline because it (a) positioned related information about fish or wildcats in closer proximity so that local relations within a single category (such as "size") were learned, and (b) organized information spatially so that global relations across categories (such as size and diet) were learned. The learning potential of text, outline, and matrix displays was also examined in combination with variations in thematic organization, amount of study time, and time of testing. The most important and consistent findings were that (a) outline and matrix displays produced greater relational learning than the text, and (b) matrix displays produced greater relational learning than outlines. Keywords: graphic organizers, spatial representations, text learning

Suppose that you are a biology instructor teaching students about human bones. You line several bones across the floor so that students can examine each one carefully. In so doing, students might learn specific information about particular bones (e.g., a rib bone is curved) but little about which bones are interconnected or the structures that bones form. Moreover, the notion that these bones form the human skeleton is obscured. Knowing how bones interrelate to form a hand or skeleton is an example of structural knowledge. Structural knowledge is knowledge about the interrelationships among ideas (Jonassen, Beissner & Yacci, 1993). Interrelated information is more meaningful than the sum of its parts just as an assembled puzzle is more meaningful than a random collection of its pieces. According to Mandler (1983), meaning does not exist until some structure or organization is achieved. Does the text below achieve organization? Are the interrelationships among fish apparent or does the text present ideas one at a time and obscure relationships much like bones lined across the floor?

374

Characteristics of fish Fish fall into one of three social groupings: solitary, small or school. Solitary fish generally do not socialize with other fish. Examples of solitary fish are the Hat and the Arch. Although the Hat and Arch are both solitary fish, they differ in several ways. The Hat swims at a depth of approximately 200 feet, whereas the Arch swims at approximately 400 feet. The Arch is 300 cm in size; the Hat is 150 cm. The Hat is a brown color and eats algae. The Arch is blue and eats minnows. Fish in small groups are also varied. They can swim at a depth of 200 feet like the Lup or at 600 feet like the Tin. The Lup is 150 cm, eats algae, and is black. The Tin is 500 cm, eats flounder, and is tan. Fish in schools also vary along several dimensions. The Bone, for example, is 300 cm and swims at a depth of 400 feet. In contrast, the scale is 500 cm and swims at 600 feet. The Bone is orange and eats minnows, whereas the Scale is yellow and eats flounder. We investigated the potential of this text and two other displays - the outline and matrix - for communicating text structure and facilitating relational learning.

Text, outline, and matrix displays Text, outline, and matrix displays vary in their presentation formats which, in turn, influence how students process them. The text above presents information sequentially in a paragraph form. Readers usually follow a single processing route as they read left-to-right and top-to-bottom. The outline in Figure 1 presents information hierarchically in a list form. Two fish are listed beneath each social group; characteristics pertaining to depth, size, color, and diet are listed beneath each fish. An outline's list-like format encourages students to follow a top-to-bottom processing route. The matrix in Figure 2 presents information hierarchically and coordinately in a two-dimensional form. The matrix's hierarchical structure, like that of the outline, is evident when read vertically. When read horizontally, the matrix displays coordinate relations among fish with respect to the shared characteristics of depth, size, color, and diet. A matrix's two-dimensional format encourages students to follow both a top-to-bottom and horizontal processing route.

375

Social Groupings ofFish I.

Solitary A. Hat 1. D e p t h - 200 ft 2. S i z e - 150 c m 3. C o l o r - B r o w n 4. Diet - A l g a e B. A r c h 1. D e p t h - 400 ft 2. S i z e - 300 c m 3. Color - B r o w n 4. D i e t - M i n n o w s

II.

Small A. L u p 1. D e p t h - 200 ft 2. S i z e - 150 c m 3. C o l o r - Black 4. Diet - A l g a e B. Tin 1. D e p t h - 600 ft 2. Size - 500 c m 3. Color - Tan 4. Diet - F l o u n d e r III. School A. B o n e l. D e p t h - 400 ft 2. S i z e - 300 c m 3. Color - O r a n g e 4. Diet - M i n n o w s B. Scale 1. D e p t h - 600 ft 2. S i z e - 500 c m 3. C o l o r - Yellow 4. Diet - F l o u n d e r

Figure 1. Social outline o f fish. Social G r o u p i n g s o f F i s h

Solitary Fish: Depth: Size: Color: Diet:

Hat 200 ft 150 c m Brown Algae

Arch 400 ft 300 c m Blue Minnows

Small Lup 200 ft 150 c m Black Algae

Tin 600 ft 500 c m Tan Flounder

Figure 2. Social matrix o f fish.

School Bone 400 ft 300 c m Orange Minnows

Scale 600 ft 500 c m Yellow Flounder

376 Theoretical framework

Why might one of these three displays increase relational learning more than another? One simple reason is that one display might include more information than the others. Informational equivalence was maintained, however, by incorporating the same ideas in all displays (Larkin & Simon, 1987). The more intriguing reason why one display might prove most effective deals with computational efficiency (Larkin & Simon, 1987). If displays are informationally equivalent but relational information is drawn more quickly and accurately from one, then that display is considered more computationally efficient. More specifically, a display is more computationally efficient if it localizes information and facilitates perceptual enhancement (Larkin & Simon, 1987). In terms of localization, the text and outline separate related information. For example, when examining fish size, there are six intervening facts between the comparable depths of the Hat and Lup fish (i.e., 200 feet) in the text and seven intervening facts in the outline. Those same size designations appear on the same line of the matrix with only one intervening fact between them. Consider also the revised outline and matrix displays in Figures 3 and 4, respectively. Now organized by depth rather than social group, both depth displays improve localization over Figures 1 and 2, respectively. Still the matrix is superior to the outline in terms of localization. The depth outline contains three intervening facts between common characteristics associated with color, size, and diet whereas the depth matrix contains no intervening facts between those common characteristics. Improved localization within the matrix should increase or speed (Robinson & Skinner, 1996) the learning of l o c a l r e l a t i o n s - relations pertaining to a single characteristic such as depth or color. Students studying a matrix need only examine a single matrix row to recognize local relations, such as the Lup and Hat fish both eat algae, whereas students studying an outline must transverse across outline sections to recognize the same relations. Perceptual enhancement occurs when the "big picture" (Winn, 1988) or overriding structure is readily apparent. The depth matrix in Figure 4 seems to facilitate perceptual enhancement better than the other displays. Examining the depth matrix vertically, it is immediately clear that fish at 200 feet eat algae, are 150 cm in length, and are dark colored. The text and outlines surrender this global relationship only with greater search and effort. Examining the depth matrix both vertically and horizontally the big picture emerges: As fish swim deeper they consume larger prey, increase in size, become lighter in color, and swim in larger social groups. Extracting this pattern from

377

Depth ofFish 1.

200 ft A. L u p 1. Social G r o u p - S m a l l 2. C o l o r - B l a c k 3. Size - 150 c m 4. Diet - A l g a e B. Hat 1. Social G r o u p - Solitary 2. C o l o r - B r o w n 3. S i z e - 150 c m 4. Diet - A l g a e II. 400 ft A. A r c h 1. Social G r o u p - Solitary 2. C o l o r - B l u e 3. S i z e - 300 c m 4. Diet - M i n n o w s B. B o n e 1. Social G r o u p - School 2. C o l o r - O r a n g e 3. Size - 300 c m 4. Diet - M i n n o w s III. 600 ft A. Scale 1. Social G r o u p - School 2. C o l o r - O r a n g e 3. Size - 500 c m 4. Diet - F l o u n d e r B. Tin 1. Social G r o u p - Small 2. Color - Tan 3. Size - 500 c m 4. Diet - F l o u n d e r

Figure 3.

D e p t h outline o f fish.

Depth of Fish

Fish: Social Grouping: Color: Size: Diet:

20Oft

40Oft

6oo ft

Lup Hat Small Solitary Black Brown 150 c m Algae

Arch Bone Solitary School Blue Orange 300 c m Minnows

Scale Tin School Small Yellow Tan 500 c m Flounder

Figure 4.

D e p t h m a t r i x o f fish.

378 the outlines or text involves more computation. The pieces of the puzzle are there, but lie scattered and unattached-like bones lined up across the floor. Improved perceptual enhancement should facilitate learning global relations. For instance, students studying the depth matrix should more likely note the overriding relation that as fish swim deeper they increase in size.

Related research

A review of related research reveals a handful of published studies investigating the relative effectiveness of matrix displays. (See Kiewra (1994) for a more comprehensive review of matrix studies.)Matrix displays were compared most often with text and outline displays among students engaging in problem solving, lecture learning, or text learning. The studies point toward the matrix's general effectiveness with one study in particular (Robinson & Schraw, 1994) supporting the localization and perceptual enhancement arguments. Early matrix investigations were conducted by Schwartz (1971) and colleagues (Schwartz & Fattaleh, 1972). They presented who-done-it type problems for which solvers had to reason deductively from provided sentence clues. In one problem, participants were given a list of statements pertaining to (a) the names of patients in a hospital, (b) their illnesses, and (c) their room numbers. Sentences presented partial information such as "The man in room 101 has asthma" or "Mr. Jones has cancer." The problem entailed determining what disease Mr. Young had. Results showed that when sentence information was displayed in a matrix, rather than a list or hierarchy, problems were solved more accurately. A study reported by Day (1988) showed how studying a medication schedule displayed in a matrix form produced more accurate problem solving than studying it in a list form. The list contained the names of six drugs in a column with dosage alongside each drug. The matrix also listed the drugs in a column but used time designations as side headings. A check mark in a matrix cell indicated when a drug should be taken. Results showed that the matrix produced higher performance than the outline on inferential problems such as "if you leave home in the afternoon and will not be back until breakfast the next day, how many Inderal should you take along?" Day concluded that the matrix was superior because it provided the union of medication and time information. Two lecture learning studies by Kiewra and colleagues demonstrate the matrix's effectiveness across learning outcomes. In each experiment, students viewed a 19-min videotaped lecture about five types of creativity. For each

379 type of creativity, nine different characteristics were discussed in a changing order. In one experiment (Kiewra, DuBois, Christian & McShane, 1988), students viewed the lecture before reviewing provided notes. The provided notes were either in text, outline, or matrix form. Both the outline and matrix groups recalled more lecture ideas than the text group probably because the outline and matrix highlighted presented ideas better than the text. On a transfer test measuring both relational learning and concept learning, only the matrix group outperformed the text group. This finding perhaps occurred because the matrix helped in learning relations across the types of creativity. A second experiment (Kiewra, DuBois, Christian, McShane, Meyerhoffer & Roskelley, 1991) introduced the matrix to lecture note taking. Students who took and/or reviewed matrix notes recalled more than those who took and/or reviewed conventional notes (which generally follow a linear form). The matrix groups, however, did not outperform conventional or outline groups on a test measuring relational learning. Two text learning studies by Robinson and colleagues demonstrate the matrix's effectiveness across learning outcomes. In one study (Robinson & Kiewra, 1995), students studied a chapter-length text describing psychological disorders. Students read the text alone or the text supplemented by several outline displays or several matrix and hierarchy displays. Results across two experiments revealed that the matrix and hierarchy displays produced higher performance than outline displays on tests measuring hierarchical concept relations and coordinate concept relations. The latter test primarily assessed global relations. Students had to compare and contrast psychological disorders. The text learning study most closely aligned with the present study was conducted by Robinson and Schraw (1994). Subjects read an adaptation of the fish passage appearing in this introduction. Subjects read the passage and then studied the passage, or a corresponding outline or matrix display for an additional five minutes (Experiment 1) or one minute (Experiment 2). Truefalse items assessed ability to verify both local and global concept relations. Results indicated that studying the matrix display produced greater relational learning than studying the text (across item types) under both abbreviated (1 min) and extended (5 min) study periods.

The present study Our primary interest was in determining whether a matrix or outline display bolsters relational learning more than text and more than one another. Our first prediction was that matrix and outline displays would produce greater

380 relational learning than the text display, and that the matrix display would produce greater relational learning than the outline display. The prediction was based on the displays' relative computational efficiency. The matrix is superior to the outline and the outline superior to the text in positioning related information in close proximity and displaying information so that local and global relationships are readily apparent. Our second interest was in determining whether thematic organization influenced relational learning. In Experiment 1, outline and matrix displays were organized by depth or social group. Our second prediction was that displays organized by depth (see Figures 3 and 4) would produce greater relational learning than those organized by social group (see Figures 1, 2, and the text). Based upon our informal analysis of localization and perceptual enhancement, displays organized by depth increase localization and present the overriding relations better than those organized by social group. Our third interest was in determining the relative learning efficiency of text, outline, and matrix displays. In Experiments 2 and 3, we varied study time and investigated how quickly students learn from the various displays. Our third prediction was that the computational advantage of matrices would be most pronounced when study time is brief. Under these conditions, learners using the texts or outlines do not have sufficient time to perform the computations necessary to uncover and learn the embedded relations. Our fourth interest was in determining whether a display's advantages are most apparent upon immediate or delayed testing or following an opportunity to relearn the displayed material. In most studies investigating adjunct displays, testing occurs without delay (e.g., Hawk, 1986; Guri-Rosenblit, 1988; Kiewra et al., 1988, 1991). Our fourth prediction was that the matrix's effectiveness would increase following a delay because its computational advantages in localization and perceptual enhancement would help learners build more durable connections in memory (Mayer, 1984) than either outlines or text. We also predicted that relearning opportunities would still not foster relational learning among those studying text or outline displays because their linear formats would make it difficult to learn relations even though criterion knowledge and additional study time are afforded. The present study was both a replication and extension of the Robinson and Schraw (1994) study. It sought to substantiate their finding that matrix displays increase relational learning over texts and extend their findings in several ways. First, Experiments 1 and 2 measured students' ability to formulate and recall relationships rather than verify provided ones. The recall format should be examined given that organized displays purportedly have greater effects on recall than recognition measures (Kintsch, 1970;

381 McCormack, 1972). Second, the present study manipulated learning efficiency in a single experiment. This permitted insight into the interactive effects of learning time and display form. Third, the relative effects of displays were examined under both immediate and delayed conditions within the same experiment. Robinson and Schraw (1994) investigated delay alone in Experiment 3. Test delay in their study was only 25 minutes versus the more challenging two-, five-, and seven-day delays used here. Fourth, only the present study investigated varying displays' thematic organization. Fifth, only the present study (Experiment 3) used realistic, chapteMength materials. Last, only the present study examined the savings potential of displays.

Experiment 1 Experiment 1 examined whether various displays (text, outline, and matrix) and different thematic organizations (social group and depth) produce differential effects on learning local relations and global relations. The effects of displays and thematic organization were assessed immediately following acquisition and following a two-day delay.

Me~od Subjects and design. Sixty-nine undergraduate students enrolled in an educational psychology course at a large midwestem university voluntarily participated in this study to fulfill a research requirement. Participants were assigned randomly to a text-only control group or to one cell of a 2x2 design. The first factor was display format (outline or matrix) and the second factor was thematic organization (social group or depth). The individual groups (and their respective sizes) were as follows: text-only (14), social outline (14), depth outline (15), social matrix (13), and depth matrix (13). Materials. Materials included the text passage, four adjunct displays of the passage, and tests. The 190-word passage (presented in the introduction and adapted from Friedman & Greitzer, 1972) contained 30 facts - five about each of six fictitious fish. The passage's organization by social group actually obscured the local relations between fish (e.g., both the Hat and Lup fish are dark, 150 cm, and eat algae) and the global relations among all fish (e.g., fish swimming at progressively deeper depths are lighter in color, larger in size, and eat larger food). There were four adjunct displays that were studied along with the text. The four adjunct displays (shown in Figures 1-4) included the same 30 facts that appeared in the text but differed from each other with respect to either format

382 (outline or matrix) or thematic organization (social group or depth). These format and organization differences in turn produced measurable differences in localization, the degree to which similar information appeared in close proximity, and apparent differences in perceptual enhancement, the degree to which the overriding patterns or "big picture" emerged. The social outline and depth outline (each containing about 75 words) differed in terms of localization. Two comparable factors favored the depth outline over the social outline. First, related information (e.g., the size of the Hat and Lup fish) was located beneath different Roman Numerals in the social outline but beneath the same Roman Numeral in the depth outline. Second, seven intervening facts separated most related information in the social outline compared with three intervening facts in the depth outline. Although neither outline clearly presented the overriding or global relations, the depth outline seemingly required less data manipulation, or computation, to observe the global relations involving depth and the other variables (e.g., as depth increases size increases). The social matrix and depth matrix were more efficient than outlines because the common category names (e.g., size, diet) appeared only once along the left margin. The two-dimensional structure allowed learners to read vertically about a single fish or adjacent fish, and read horizontally across a common category to compare two or more fish along that dimension. In terms of localization, the depth matrix was superior to the social matrix. The social matrix included one intervening fact between most related facts, whereas the depth matrix generally placed related facts next to one another or listed them one time between adjacent matrix cells. The single listing of overlapping facts permitted the depth matrix to use fewer words (42) than the social matrix (54). The depth matrix seemingly offered the most integrated "picture" of the global relations as well. Without intervening information, no data manipulation was necessary to observe that as depth increased, size, food size, and social group increased and coloration lightened. Two types of tests were administered: local relations and global relations. The 30-item local relations test was divided into 24 explicit items and six implicit items. The explicit items assessed ability to recall the two fish that explicitly shared a common characteristic between them with respect to social group, depth, diet, or size. For example, given the question "which two fish eat algae?" the correct response was Hat and Lup. The implicit items assessed recall for the fishes' color. Although no two fish shared the same color, there was an implicit color relation between fish: The Lup and Hat fish were dark; the Arch and Bone fish were bright; the Scale and Tin fish were light.

383 The global relations test included four cued-recall questions asking students to describe the relationship between the depth of fish and their size, diet, social group, and color.

Procedure. All participants studied instructional materials (the text alone or the text plus one of the four adjunct displays) for 15 rain in preparation for unspecified tests. Participants then took the global relations test and the local relations test in that order without reference to their study materials. Two days later they completed an unannounced local relations test. This delayed test was identical to the immediate version except the items were reordered. The global relations test was not readministered because exposure to the immediate local relations test, in this case, might provide cues that unjustly facilitate performance on a subsequent global relations test. Results and discussion Scoring. Local relations tests were scored using an established key. One point was awarded for each correct answer. The maximum number of explicit local relations was 24; the maximum number of implicit local relations was 6. The global relations test was scored on a 0-2 scale for each of the four items. A score of 2 was awarded when the relationship was correctly stated in some way. For example, the correct answer to the item "What is the relationship between depth and diet?" was "As fish swim deeper the size of their diet increases." Scores of zero were assigned when the relationship was incorrectly stated (e.g., as fish swim deeper they eat smaller food), or when accurate information was provided but no global relationship was stated (e.g., fish at 200 feet eat algae; fish at 400 feet eat minnows; and fish at 600 feet eat flounders). A score of one was assigned when a partial relationship was stated (e.g., fish that swim deep eat large fish). Explicit local relations. Average percents correct on the immediate and delayed explicit local relations items for the five display groups are shown in rows 1 and 2, respectively, in Table 1. Visual inspection of the means shows a pattern consistent with predictions based upon the localization theory. Displays containing the most intervening facts (i.e., social outline and text) produced the lowest scores whereas materials containing the fewest intervening facts (i.e., the social matrix and depth matrix) produced the highest scores. One-way analyses of variance (ANOVAs) confirmed these observations. The ANOVA for immediate explicit local relations was significant, F(4,64) = 3.77, p < 0.009, MSe = 14.71 as was the ANOVA for delayed explicit local relations, F(4,64) = 4.70, p < 0.003, MSe = 28.03. Tukey HSD follow-

384 Table 1. Mean percent correct among display groups on tests in Experiment 1

Tests

Display groups Text

Social

Depth

Social

Depth

outline

outline

matrix

matrix

Local relations Explicit items (immediate)

80b

80b

91

96

98a

Explicit items (delayed)

73

61 b

77

90a

93a

Implicit items (immediate)

89

86

80

89

94

Implicit items (delayed)

82

81

75

78

91

62

55

75

68

83

Global relations

Note: Means with "a" subscripts differ significantly from means with "b" subscripts within the same row.

ups (p < 0.05) for the immediate test indicated that the depth matrix group outperformed the text group and the social outline group. Followups for the delayed test indicated that the social matrix group and the depth matrix group outperformed the social outline group. These findings confirmed the importance of positioning related facts near each other. When facts are "localized" in a matrix form, students recall more explicit local relations. Findings also demonstrated that an outline following the text's structure (i.e., the social outline) was more dysfunctional than the text itself for delayed performance when compared with matrix displays. The main and interactive effects of display (outline and matrix), thematic organization (social and depth), and testing occasion (immediate and delayed) were also examined with regard to explicit local relations using a 2 x 2 x 2 mixed model ANOVA. Testing occasion was the within-subjects factor. This analysis showed a main effect for display, F(1,51) = 17.31, p < 0.001, M S e = 25.46, a main effect for testing occasion, F(1,51) = 22.77, p < 0.001, M S e = 9.00, and a display by testing occasion interaction, F(1,53) = 5.87, p < 0.020, M S e = 9.00. The main effect of organization, F(1,51) = 3.92, p < 0.054, M S e = 25.46, and the other interactions (largest F = 1.91, p < 0.74 for display by organization) were not significant. Regarding the main effects, immediate scores were naturally higher than delayed scores and the .matrix produced higher performance than the outline as observed in the margins of Table 2. Visual inspection of the cell means in Table 2 reveals that the display by testing occasion interaction was the result of a greater decrease in performance for the outline group than the matrix group from immediate to delayed testing. Followup tests confirmed that the matrix group outperformed the outline group on the immediate F(1,53) = 8.07, p < 0.007, M S e = 11.27

385 Table 2. Mean percent correct for matrix and outline display groups on immediate and delayed explicit local relations items in Experiment 1

Display groups Outline Matrix Both

Explicit local relations Immediate test Delayedtest 86 97 89

69 91 78

Overall 78 94

and delayed tests, F(1,53) = 15.63, p < 0.001, MSe = 24.94, but that both groups showed a decrement in performance from immediate to delayed testing. The decrement, however, was somewhat less pronounced for the matrix group, F(1,25) = 7.10, p < 0.014, MSe = 3.32 than for the outline group, F(1,28) = 17.93, p < 0.001, MSe = 13.62. Overall, these findings support the localization principle and our predictions that the matrix displays facilitate learning local relations better than outline displays. Implicit local relations. Average percents correct on the immediate and delayed implicit local relations items pertaining to color relations are shown in rows 3 and 4, respectively, in Table 1. As predicted and consistent with findings for explicit local relations, the depth matrix (which best localized related information and contained the fewest intervening facts) produced the highest performance. One-way ANOVAs, however, indicated that the groups did not differ significantly on the immediate F(4,64) = 0.68, p < 0.612, MSe = 1.89 or delayed test, F(4,64) = 0.73, p < 0.579, MSe = 2.57. A 2• mixed model ANOVA examining the main and interactive effects of display, organization, and testing occasion for implicit local relations indicated a main effect for testing occasion, F(1,51) = 4.37, p < 0.042, MSe = 0.75. Performance was naturally higher on the immediate test (M = 87%) than the delayed test (M = 81%). No other main effects or interactions were significant. Overall, these results indicate that the treatments did not have the same profound effects on implicit local relations as with explicit local relations. The descriptively higher performance by the depth matrix group, however, shows that the localization of implicit relations is somewhat helpful. Global relations. It was predicted that displays that increased localization and perceptual enhancement would help learners recognize overriding patterns and relationships - the Big Picture - throughout the materials. More specif-

386 ically, displays following a matrix rather than outline format and following a thematic organization by depth rather than social group were predicted to produce the highest performance on the global relations test. Results from a one-way ANOVA indicated that the five display groups did not differ reliably, F(4,64) = 1.98, p < 0.108, MSe = 5.38. An inspection of the mean percents in the bottom row of Table 1, however, shows the same pattern of results seen with the explicit local relations. The highest performance was by the depth matrix group and the lowest performance by the social outline and text groups. This pattern is consistent with our display and organization predictions based upon perceptual enhancement and again suggests that a poorly organized (social) outline is relatively debilitating. Results from a 2 x 2 ANOVA assessing the main and interactive effects of display and thematic organization, indicated a main effect for thematic organization, F(I,51) = 5.14, p < 0.029, MSe = 5.32. Materials organized by depth (M = 79%) produced higher performance than those organized by social group (M = 61%). Although the matrix groups had somewhat higher scores than the outline groups (Ms = 76% and 65%, respectively), this difference was not reliable, F(1,51) = 1.89, p < 0.177, MSe = 5.32, nor was the display by thematic organization interaction, F(1,51) = 0.154, p = ns. In this case, thematic organization played an important role in learning global relations. This stands to reason given that materials organized by social group obscured the overriding relations built around the depth variable. In contrast, the displays organized by depth better accented the relations between depth and the other variables. Display format did not play a significant role in learning global relations but the ordering of means suggests a mild advantage for the matrix over the outline.

Experiment 2 The purpose of Experiment 2 was to reexamine the outline and matrix displays' potential for facilitating relational learning under various conditions. We compared their learning efficiency by varying and reducing study time. We compared their retention potential by extending the test delay. And, we compared their savings potential by affording students a second opportunity to learn from displays following testing. In line with these investigations, six methodological changes were made from Experiment 1. 1. The independent variable of thematic organization was omitted to focus more on display format. 2. The text was read aloud to participants who then studied either the text, matrix, or outline. This ensured that outline and matrix participants focused attention exclusively on the display rather than divide attention

387 between it and the text. Reading the text aloud also controlled initial reading time. 3. Participants studied the text, outline, or matrix for 5, 10, or 15 rain. This provided a comparison of how efficiently these displays were encoded. 4. A five-day delay was interspersed between immediate and delayed tests to determine the relative retention value associated with the displays. 5. Following the delayed test, participants had three rain to relearn the material studied previously. They were then tested for a third time. This assessed the relative savings potential of the displays. 6. Two new forms of the global relations test were used. The first was a summary test that asked participant to "rapidly recall any overriding relations they observed." Recall cues were deleted to better test what relations were encoded rather than reconstructed upon questioning. The other was an application test that asked students to "predict the color, size, diet, and social group of a new fish discovered at a depth of 800 ft." A correct response to this problem depended upon understanding the global relations among the fish. For example, knowing that as depth increases, size increases, one should predict the newly discovered fish to be larger than 500 cm. MeNod

Ninety-seven new participants drawn from the same subject pool as those in Experiment 1 were first randomly assigned to one of three classrooms where study time was 5, 10, or 15 rain. Within each classroom, participants were then randomly assigned to one of three display formats (text only, outline, or matrix). The same displays from Experiment 1 were used, with the outline and matrix organized by depth only. Next, the passage was read aloud to all participants. Participants then studied their materials for 5, 10, or 15 min as previously directed. After studying, participants were administered the summary test, the application test, and the local relations test in that order. Five days later, pm-ticipants took the delayed local relations test which was identical to the immediate test except for its varied item order. Participants then restudied their displays for three rain before taking the savings test, which was identical to the immediate and delayed local relations tests except for its varied item order. Results" and discussion Scoring. The local relations test was scored in the same manner as in Experiment 1. The summary test, assessing the free recall of global relations, and the application test, assessing the application of global relations, were each

388 Table 3. Mean percent correct for display groups studying for 5, 10, or 15 rain on immediate (I), delayed (D), and savings (S) items assessing explicit local relations in Experiment 2

Time

Display groups Text I D S

Outline I D

S

Matrix I D

S

5m in 10min 15min

68 77 79

84 94 97

99 97 99

97 94 98

97 97 99

59 63 66

73 77 83

76 80 97

90 89 95

scored awarding one point for each correct relationship or response with a maximum score of four for each. Explicit local relations. Average percents correct on immediate and delayed items are shown beneath their respective columns in Table 3. The main and interactive effects of display (text, outline, or matrix) study time (5, 10, or 15 rain) and testing occasion (immediate and delayed) were examined using a 3 x3 x 2 mixed model ANOVA with testing occasion as the within-subjects factor. Main effects were followed-up using the Tukey HSD procedure (p < 0.05). This analysis revealed main effects for display, F(2,87) = 29.01, p < 0.001, MSe = 20.35; study time, F(2,87) = 4.19, p < 0.018, MSe = 20.35; and testing occasion, F(1,87) = 35.83, p < 0.001, MSe -- 5.23. The three-way interaction and two-way interactions were not significant (largest F = 2.20, p < 0.118 for display by testing occasion). With respect to display, the outline (M = 88%) and matrix (M = 94%) groups outperformed the text group (M = 68%) as predicted. The matrix group, however, did not statistically outperform the outline group as predicted. Still, the observed differences favored the matrix, consistent with findings from Experiment 1. With respect to study time, studying for 15 min (M = 90%) naturally produced higher test scores than studying for 5 min (M = 78%). The uniformly ascending means from 5 to 15 min (M = 83% for 10 rain) suggest that study time was positively and linearly related to performance, With respect to testing occasion, performance was naturally better immediately following studying (M = 88%) than following a 5-day delay (80%). Although the interactions involving display, study time, and testing occasion were not significant, the patterns of means in Table 3 involving the outline and matrix groups are noteworthy. For immediate testing, it appears

389 Table 4. Mean percent correct for display groups studying for 5, 10, or 15 rain on immediate (I), delayed (D), and savings (S) items assessing implicit local relations in Experiment 2 Display groups Text

Outline

Time

I

S

I

5min

48

33

85

79

63

10min

54

43

80

83

75

15m in

65

59

85

92

85

95

D

D

Matrix S

I

D

S

100

82

63

93

97

95

90

100

100

86

100

that the matrix holds some practical advantage over the outline when study time is restricted to 5 min (M = 97% and 84%, respectively). This comparison suggests that the matrix is somewhat more efficient for learning explicit local relations than the outline. For delayed testing, the matrix's practical advantage over the outline is observed at both 5 rain (M = 90% and 76%, respectively) and 10 rain (M = 89% and 80%, respectively). This suggests that when study time is relatively limited, the outline has somewhat less retention potential than the matrix. Taken together, the immediate and delayed means for 10 rain mirror those found in Table 2 for Experiment 1. They show a greater decrement in delayed performance for the outline group (14%) than the matrix group (5%). Implicit local relations. Scores were analyzed in the same manner used for the explicit local relations test. Average percents correct for immediate and delayed tests are shown beneath their respective columns in Table 4. Again, main effects were found for display, F(2,87) = 18.60, p < 0.001, MSe = 4.08; study time, F(2,87) = 5.45, p < 0.007, MSe = 4.08; and testing occasion, F(1,87) = 19.44, p < 0.001, MSe = 1.08. As was true with explicit local relations, it was confirmed that studying the outline (M = 79%) and matrix (M = 87%) displays produced higher performance than studying the text (M = 50%); studying for 15 rain (M = 83%) naturally produced higher performance than studying for 5 rain (M = 62%), with study time again showing a positive and linear relationship with performance (M = 76% for 10 rain); and immediate performance (M = 79%) was naturally higher than delayed (M = 68%) performance. Although no interactions were significant (largest F = 1.24, p < 0.295 for time by testing occasion), the patterns of means in Table 4 involving the outline and matrix groups are again noteworthy. Given minimal study time (5 rain), the outline and matrix groups performed similarly (immediate

390 scores: 79% and 82%, respectively; delayed scores: 63% for both) and generally low. Compared with explicit local relations (Table 3), implicit local relations seem to require greater learning time. Given moderate study time (10 rain), the matrix format produced somewhat more efficient learning than the outline as seen on the immediate items (M = 95% v. 83%), and greater retention as seen on the delayed items (M = 90% v. 75%). Given ample study time (15 min), the groups again performed comparably, although the matrix group showed a slight advantage on the immediate test (M = 100% v. 92%). Savings test. To determine if display or study time influence the savings of explicit or implicit local relations, separate 3 x3 ANOVAs were conducted on explicit local relations and implicit local relations savings scores. The means for explicit local relations appear in Table 3 and those for implicit local relations appear in Table 4. Both analyses revealed a main effect for display, F(2,87) = 28.48, p < 0.001, MSe = 8.19 for explicit local relations; and F(2,87) = 8.50, p < 0.001, MSe = 2.13 for implicit local relations. The main effect for study time and the study time by display interaction were not significant in either analysis. It was confirmed for explicit and implicit local relations that students studying the outline (M = 98% and 92%, respectively) or matrix (M = 98% and 96%, respectively) displays outperformed those studying the text display (M = 75% and 78%, respectively). The relative ineffectiveness of learning from text without the aid of an outline or matrix supplement is particularly obvious among students studying the text for the maximum 15 rain who still only managed savings scores in the low to mid 80s. These findings show that even when students have criterion knowledge and a second opportunity to study, studying a text still results in poorer performance than studying an outline or matrix. Global relations. Two tests assessed global relations. The first was the summary test which asked students to rapidly summarize relationships among variables. The second was the application test which asked students to predict the characteristics of a new fish (based upon the existing global relations among fish). Each set of scores was analyzed using a 3 x 3 ANOVA in which one factor was display and the other factor was study time. Both analyses showed a main effect for display, F(2,88) = 11.81, p < 0.001, MSe = 5.58 for the summary test; and F(2,88) = 9.01, p < 0.001, MSe = 1.90 for the application test. The main effect for study time and the two-way interaction were not significant in either analysis. Tukey HSD tests confirmed that the outline (M = 83%) and the matrix (M = 63%) groups outperformed the text group (M = 12%) on the summary test. On the application test, the outline (M = 86%) and matrix (M = 85%)

391 groups again outperformed the text group (M = 52%). The generally higher scores on the application test are perhaps attributable to the stringent time limits imposed only for the summary test and the summary test's uncuing nature. It asked students to report rapidly any relationships observed whereas the application test requested predictions about specific characteristics (e.g., diet) given a new fish's depth. The descriptively higher performance by the outline group over the matrix group on the summary test was unexpected and in contrast to (a) results from Experiment l involving global relations, and (b) results for local relations in Experiments 1 and 2. They are also in opposition to those from the application test in Experiment 2. It appears that the outline and matrix groups learned a comparable number of overriding relations in Experiment 2 (as seen on the application test) but that the outline group was somewhat better at reporting them under conditions involving time constraints and an absence of retrieval cues (as seen on the summary test).

Experiment 3 Experiment 3 also examined the effects of adjunct displays and study time on relational learning. In so doing, it addressed limitations of Experiments 1 and 2. First, a longer text passage was used. The text material in Experiments 1 and 2 contained less than 200 words; the passage in Experiment 3 contains more than 2,000 words. Second, the present text is realistic. Whereas the fish passage in Experiments 1 and 2 was fictional, this one about wildcats contains true facts. Third, the two- and five-day intervals between immediate and delayed testing used in Experiments 1 and 2, respectively, were extended to seven days in Experiment 3 to better assess long-term retention. Last, Experiment 3 adds a signaled text condition. This addition helps determine whether relational learning differences are the result of a display's highlighted content or its form.

Me~o~ Subjects and design. Fifty-one college students, drawn from the same subject pool as Experiments 1 and 2, were assigned randomly to one cell of a 4 x 2 design. The first factor was display (text, signaled text, outline, or matrix) and the second factor was study time (10 rain or 20 rain). Materials. Materials included the four displays, an audiotaped lecture, and two relational tests.

392 The 2200 word text display was typed single space in two columns on six pages. It described six different wildcats (e.g., tiger and bobcat) according to 13 different characteristics (e.g., call and weight). The 13 characteristics fit into four major categories (classification, physical features, life style, and hunting behaviors). The text contained 102 important details pertaining to the wildcats and their characteristics (e.g., the bobcat weighs 30 pounds). The signaled text display was identical to the unmarked text except key information was highlighted. Wildcat names and major categories were in bold type, the 13 characteristics were italicized, and details were underlined. The outline and matrix displays contained the identical information highlighted in the signaled text. The outline display organized the information in two columns on three pages using 373 words. The wildcat names appeared as Roman Numerals 1-6. The major categories appeared as capital letters A-D, beneath each Roman numeral. The 13 characteristics appeared as numbers 1-4 (because there were a maximum of 4 per major category) beneath the major categories, and the details were assigned letters and subsumed beneath their respective characteristics. The matrix display was a two-dimensional classification table appearing on a single page and containing 235 words. It listed the wildcats along the top row, the major categories and their characteristics down the left column, and the details within the matrix cells. The matrix display appears in Figure 5. A 13-min audiotaped lecture, identical to the text and signaled text, was developed. It was presented at roughly 170 words per minute. There were two relational tests. The local relations test assessed students' ability to relate the six wildcats across a single characteristic such as call or range. Sample items included "Which cat has the largest range?" and "Which two cats hiss and purr?" This 16-item recognition test required 28 responses. The global relations test assessed ability to relate the six wildcats across two characteristics, such as call and weight, simultaneously. A sample question is "What is the relationship between call and weight?" Students selected the best response from among four unique choices for each of the 10 items. Procedure. All students were instructed to learn relationships in preparation for tests measuring local and global relations (which were explained). Students then listened to the 13-min audiotaped lecture without note taking. After the audiotaped lecture, students studied their displays for 10 or 20 rain (in different classrooms) without the benefit of note taking. Immediately following the study period, students took the anticipated global relations test and local relations test in that order. Upon completion, all materials were collected and students were dismissed.

393 Tigers

Lions

Jaguars

Leopards

Cheetah

Bobcat

Panthera

Panthera

Panthera

Panthera

Acinonyx

Lynx

Call:

Roar

Roar

Roar, growl

Roar, growl

Hiss, purr

Hiss, purr

Maximum Weight (lb.):

450

400

200

150

125

Yellow with black circles containing black spots

Yellow with black Yeltow with black circles spots

Genus: Physical Features:

Coat:

Yellow-ormage Light brown with black stripes

Disoncfive Characteristic: Powerthlupperbody

30 Rust color

PowertM upperbody

Keen eyes and ears Tremendous Powerful, athletic strength, legs Keen eyes and ears

Keen eyes and ears

Lifestyle: Habitat:

Jungle

Plains

Jungle

Jungle

Plains

Forests

Range (sq. miles):

30

150

5

15

50

30

Social Behavior:

Solitary

Groups

Solitm'y

Solitary

Groups

Solitary

25

20

10

8

6

Small animals

Medium~sized mammals

Medium-sized mammals

Small mammals

Night

Life Span (yr.): 25 Hunting: What:

Medium-sized mammals

Medium-sized mammals

When:

Sundown

Sunrise & sundown Night

Daytime

Sundown

Method:

Stalks prey Knocks prey over Hides uneaten portion for future meals

Grot~p stalks prey from all angles Knocks prey over Will scavenge

Stalks prey Ambushes prey Fats what it can and from trees leaves the rest Hoists prey into trees and hides for future meals

Group stalks prey Runs 65 mph to capture prey Prey often stolen by lions and hyenas Will scavenge

Abushes prey from behind trees Eats what it can and leaves the rest

Frequency:

Weekly

Daily

Dally

Dally

Daily

Weekly

Figure 5. Matrix of wildcats.

One week later, students were retested without warning using the identical tests. Following the delayed tests, all students studied their displays once more for 10 minutes "in preparation for the same test." Following this relearning opportunity, the same tests were administered to assess the "savings" potential of the displays.

Results and discussion Scoring. Both local and global relations tests were scored using established keys. One point was assigned for each correct response in the 28-response local test and the 10-response global test.

394 Table 5. Mean percent correct for display groups on immediate and delayed local relations tests in Experiment 3

Display groups

Test time Immediate

Delayed

Total

Text Signaled text Outline Matrix All groups

67 71 76 84 75

67 70 65 76 69

66 70 71 81 72

L o c a l relations. A 4 x 2 x 2 mixed-model ANOVA was conducted on scores

for the local relations test. The first factor was display (text, signaled text, outline, or matrix). The second factor was study time (10 rain or 20 min), and the within-subject factor was testing occasion (immediate and delayed). The Tukey HSD procedure followed-up significant effects (p < 0.05). There was a main effect for study time, F(1,43) = 5.40, p < 0.05, M S e = 20.44. Naturally, students studying 20 rain (M = 76%) outperformed those studying 10 min (M = 68%). There was a main effect for display, F(3,43) = 3.03, p < 0.05, M S e = 20.44. As seen in the right column of Table 5, students studying the matrix display outperformed those studying text, signaled text, or outline displays. This is further evidence that matrix displays localize related information more effectively than other displays. There was also a main effect for testing occasion, F(1,43) = 6.54, p < 0.05, M S e = 4.68. As seen in the bottom row of Table 5, immediate test performance was naturally higher than delayed test performance. There was an interaction between display and testing occasion, F(3,43) = 4.02, p < 0.05, M S e = 4.68. As seen within columns one and two in Table 5, all display groups' scores decreased somewhat from immediate to delayed testing with significant decreases occurring to the outline and matrix groups. From another perspective, the display groups differed significantly on the immediate test but not the delayed test. On the immediate test, the matrix group outperformed the text and signaled text group, and the outline group outperformed only the text group. This pattern of results runs counter to that in Experiment 1, where matrix studiers outperformed outline studiers on both the immediate and delayed test. Savings scores on the local test were analyzed using a 4 x 2 ANOVA with the first factor being display and the second factor being study time.

395 Table.6. Mean percent correct for study time groups on immediate and delayed global relations tests in Experiment 3 Test time Study time

Immediate

Delayed

Total

l0 rain

76

64

70

20 min

74

73

74

Both groups

75

69

72

Results showed only a main effect for display, F(3,43) = 5.19,p < 0.01, MSe = 14.53. The matrix group (M = 89%) outperformed all other groups: text (M = 66%), signaled text (M = 77%), and outline (M = 75%). In addition, the signaled text group outperformed the text group. Results from the savings test are evidence that it is better to study matrix displays than other display formats when given an opportunity to relearn and retest. Global relations. A 4 x 2 x 2 mixed model ANOVA was also conducted on global relations test scores. Again, there was a main effect for display, F(3,43) = 4.23, p = < 0.05, MSe = 4.80. The matrix group (M = 83%) outperformed the text group (M = 60%) and the signaled text group (M = 71%), but not the outline group (M = 74%). The outline group also outperformed the text group. This finding shows the advantage of outline and matrix displays over text displays for learning global relations. The main effect for testing occasion was significant, F(l,43) = 6.86, p < 0.05, MSe = 1.24. As seen in the bottom row of Table 6, performance was naturally higher on the immediate test than the delayed test. Testing occasion interacted with study time, F(1,43) = 5.16, p < 0.05, MSe = 1.24. As shown within the cells of Table 6, study time made little difference for immediate test performance but significantly affected delayed performance with 20 rain studiers outperforming 10 rain studiers. Savings scores on the global test were analyzed using a 4 (display) x 2 (study time) ANOVA. Only the main effect for display was significant, F(3,43) = 4.91, p < 0.01, MSe = 2.92. Only the matrix group (M = 88%) outperformed the text group (M = 61%). The other display groups performed within this range: signaled text (M = 75%) and outline (M = 73%). This result points to the matrix's, not but the outline's, advantage over text for helping students relearn global relations.

396 General discussion

This research addressed the following four questions about studying displays. 1. What display is generally most effective for learning relationships: text, outlines, or matrices? 2. Does the thematic organization of displays influence relational learning? 3. Does time spent studying displays influence relational learning? 4. Does testing occasion (immediate, delayed, or following an opportunity for relearning) influence the value of displays for relational learning?

What display is generally most effective ? Given the popularity of text learning, a subquestion worth addressing is whether studying outlines or matrices boosts relational learning more than studying texts alone. Results were clear: As predicted, studying outlines or matrices produced greater relational learning than studying text alone. Across three experiments, nine different results showed that students studying matrix or outline displays outperformed students studying text. No findings favored studying text over studying matrix or outline displays. Why are outline and matrix displays better than text displays for relational learning? The displays are informationally equivalent (Larkin & Simon, 1987) proving that observed differences are not the result of information availability. Differences are likely the result of outline and matrix displays being more computationally efficient (Larkin & Simon, 1987) than text displays. The former use fewer words and physically organize ideas by topics and categories so that relations among ideas are more discernible. The physical organization component of outlines and matrices seems most important. In Experiment 3, a signaled text display was used that highlighted the exact topics, categories, and details appearing in the outline and matrix displays and used roughly the same number of words. Even when students were told that the highlighted portion was all they needed to study for the test, their performance mirrored the text only group and often fell below that of students studying the outline or matrix. This unique finding extends previous research (see Kiewra, 1994) by showing that selective attention to highlighted text ideas is not sufficient for relational learning. The ideas must be extracted and physically arranged so that interrelations are observed. Knowing that outline and matrix displays boost relational learning beyond text displays, the second part of our question is, What display is best, outline or matrix? Present findings support the prediction that matrix displays produce greater relational learning than outline displays. The most compelling findings occurred in Experiments 1 and 3. Most notably, matrix studiers outperformed outline studiers on immediate and delayed local tests

397 in Experiment 1 and on local overall (immediate and delayed combined) and savings tests in Experiment 3. In addition, the matrix group relearned more global relations than the text group (Experiment 3), whereas the outline group did not. Although no significant findings emerged from Experiment 2. The ordering of means favored matrices over outlines on all local tests. Across Experiments 1 and 3, matrix groups performed uniformly higher than outline groups. The present study and that of Robinson and Schraw (1994) provide the clearest picture of displays' effect on learning local and global relations. The benefit of matrix displays over outline displays is resident primarily in learning local relations. Local relations are those along a single characteristic such as the relative size of fish or the relative hunting methods of wildcats. The matrix's localization benefit (Larkin & Simon, 1987) is probably due to its two-dimensional structure that positions similar information about various fish sizes or various wildcats' hunting methods on the same horizontal line for easy comparison. In contrast, the outline's linear structure organizes information about fish size or wildcats' hunting methods vertically and separates it with intervening and unrelated facts. The present study provided some evidence favoring matrices over outlines for global relearning. This shows that students aware of the global relationships they must know, relearn them more completely when studying matrices than outlines. Matricies' two-dimensional structure makes it easier to compare several fish or wildcats along two categories (such as size and diet) than do outlines with their linear structure. The matrix presents all the related information in two rows whereas the outline presents the same information throughout the outline separated by intervening information. Our meager global results are perhaps the result of students not taking full advantage of the matrix structure. Perhaps students are unaware of how to study matrices to learn global relations. Future studies should add a training component to better examine the m a n x ' s potential for improving global learning.

Does the thematic organization of displays influence learning? Predictions involving thematic organization were partially confirmed in Experiment 1. Those whose displays were organized by depth rather than social group performed best on the global relations test. The displays organized by depth better arranged facts so that overriding relations were apparent. Among the various characteristics (depth, social group, size, color, and diet), social group had the most subtle relationship with the other characteristics. Therefore, material organized by social group made learning interrelationships among characteristics relatively difficult.

398 It might be argued that our earlier described findings favoring the outline and matrix displays over the text are due to the poor thematic organization of the fish text, which was organized by social group rather than depth. This argument is weakened, however, by our own findings and those from a related study. In Experiment 1, the social matrix group outperformed the social outline group and scored about 16 percentage points higher than the text group on the explicit local relations items. These results, all involving groups whose materials were organized by social group, suggest that display format, and not thematic organization, influenced relational performance. Results from a related study (Kiewra, Levin, Kim, Meyers, Renandya & Hwang, 1994) employing displays similar to those used in Experiment 1 and 2, indicated that studying a text passage organized by depth still produced significantly lower performance than studying a matrix display also organized by depth. It might also be argued that texts are inherently more relational than the texts used here. Our own observations are that texts commonly present information one topic after another without necessarily drawing relationships among topics. Consider several books we examined about planets. The books presented information about one planet after another but neglected to describe relationships among them. The contrast between inner and outer planets with respect to size, surface features, revolution time, and moons went unsaid.

Does time spent studying displays influence learning ? Results related to study time were rather straightforward. Studying for longer time periods versus shorter time periods produced greater local relations performance. Increased study time did not differentially affect global performance. It could be that regardless of study time, students really do not know how to study effectively for global tests. Therefore, increased study time has little effect on global performance. Recall that matrix displays too had minimal effect on global performance. The predicted interaction between study time and displays did not occur. We had thought that studying matrix displays instead of text or outline displays would especially be effective when study time was limited. Although not significant, the pattern of means in Experiment 2 supported our prediction. Inspection of Table 3 suggests a practical advantage for the matrix over the outline and text when study time is limited to 5 rain. On the immediate explicit local relations test, the matrix group scored 97% correct whereas the outline and text groups scored 84% and 68%, respectively. This educationally important difference might have been larger had matrix scores not been near the ceiling. Larger differences might also have resulted if study time was reduced further (see Robinson & Skinner, 1996).

399

Does testing occasion influence the value of displays? In general, performance was naturally lower on delayed tests than immediate tests. This occurred when tests were delayed 2 days (Experiment 1), 5 days (Experiment 2), and 7 days (Experiment 3). Test occasion had only mild effects on the value of displays. In Experiment 1, performance dropped more from immediate to delayed testing for the outline group versus the matrix group. Although this predicted pattern was evident in Experiments 2 and 3, it was not reliable. Results from savings tests (Experiments 2 and 3) confirmed predictions that outline and, particularly, matrix displays promote greater relearning than text displays. Studying text was relatively ineffective even when students had an opportunity to relearn information for tests about which they had explicit criterion knowledge. For instance, text studiers in Experiment 2 still only scored about 77% correct on the savings test whereas outline and matrix groups scored about 97% correct. These unique savings findings are evidence that two educational cannons for maximizing performance - criterion knowledge and distributed practice - are of minimal value if the learning materials are in a form where relationships are not easily drawn.

Conclusions Generally speaking, our results showed consistent effects favoring the matrix over the outline for local relations, thereby supporting the localization principle, but inconsistent effects for global relations, thereby not fully supporting the perceptual enhancement principle. Perhaps students do not naturally make an integrative search of the matrix - examining it both vertically within topics and horizontally across topics - to discern the global relation or "Big Picture" that is readily apparent. A limitation of our study was failing to assess or promote an integrative matrix search. Tracking students' eye movements or having them "think aloud" as they study a matrix are potential means to assess our contention. In addition, researchers might train students in how to study or introduce study questions (see Kauffman & Kiewra, 1998) as means to facilitate an integrative search of study materials. Other research avenues are available based upon our study's limitations and findings. First, efficiency can be reexamined when study time is curtailed further than the liberal study times we imposed. Robinson and Skinner (1996) found that students locate answers to comparative questions more rapidly from a matrix than an outline or text. Robinson's and Sldnner's finding, coupled with our results from Experiment 2, suggest that the matrix's benefit might be observed under conditions involving very brief study times. Second, the present study examined only four types of text displays. Displays used

400 in the present study can be compared with or supported by other text aids. Such is the case in a similar study (Kiewra, Levin, Kim, Meyers, Renandya & Hwang, 1994) involving displays, illustrations, and mnemonics. Third, our study was limited to using texts with a comparison structure - one of five text structures proposed by Meyer (1985). Researchers might examine how displays facilitate learning from other text structures. Last, we encourage researchers to further examine the retention and savings effects of displays. Our procedures limited the retention and savings interval to one week. The relative benefit of displays over a semester length course, for example, is unknown. Instructionally, we suggest that teachers and textbook authors supply students with matrix text displays. An analysis of current educational psychology textbooks revealed a wide range in the incorporation of adjunct displays and the inclusion of few matrices (Kiewra & Gubbels, 1997). The matrix appears especially helpful for learning local relations and somewhat effective for learning global relations. In contrast, the age-old routine of studying text without adjunct displays produces relatively poor relational learning.

References Day, R.S. (1988). Alternative representations. In G. Bower, ed., The Psychology of Learning and Motivation, Vol. 22, pp. 261-303. Friedman, M.R & Greitzer, EL. (1972). Organization and study time in learning from reading. Journal of Educational Psychology 63: 609-616. Guri-Rosenblit, S. (1988). Impact of diagrams on recalling sequential elements in expository texts. Reading Psychology 9: 121-139. Hawk, P.R (1986). Using graphic organizers to increase achievement in middle school life science. Science Education 70: 81-87. Jonassen, D.H., Beissner, K. & Yacci, M. (1993). Structural Knowledge: Techniques for Representing, Conveying, and Acquiring Structural Knowledge. Hillside, NJ: Lawrence Erlbaum. Kauffman, D.F. & Kiewra, K.A. (1998). The Matrix Organizer Boosts Relational Learning more than Text or Outlines. Paper presented at the annual meeting of the American Educational Research Association, San Diego, CA. Kiewra, K.A. (1994). The matrix representation system: Orientation, research, theory, and application. In R. Perry, ed., Higher Education: Handbook of Theory and Research. New York: Agathon Press. Kiewra, K.A., DuBois, N.E, Christian, D. & McShane, A. (1988). Providing study notes: A comparison of three types of notes for review. Journal of Educational Psychology 80: 595-597. Kiewra, K.A., DuBois, N.F., Christian, D., McShane, A., Meyerhoffer, M. & Roskelley, D. (1991). Notetaking functions and techniques. Journal of Educational Psychology 83: 240245.

401 Kiewra, K.A. & Gubbels, R (1997). Are educational psychology courses educationally and psychologically sound? What textbooks and teachers say. Educational Psychology Review 9: 121-149. Kiewra, K.A., Levin, J.R., Kim, S., Meyers, T., Renandya, W.A. & Hwang, Y. (1994). Fishing for Text Facilitators: The Lure of the Mnematrix. Paper presented at the annual conference of the American Educational Research Association, New Orleans, LA, U.S.A. Kintsch, W. (1970). Models for free recall and recognition. In D.A. Norman, ed., Models of Human Memory (pp. 331-373). New York: Academic Press. Larkin, J.H. & Simon, H.A. (1987). Why a diagram is (sometimes) worth ten thousand words. Cognitive Science 11: 65-99. Mandler, J. (1983). Stories: The Function of Structure. Paper presented at the annual convention of the American Psychological Association, Anaheim, CA. Mayer, R.E. (1984). Aids to text comprehension. Educational Psychologist 19: 30-42. McCormack, RD. (1972). Recognition memory: How complex a retrieval system? Canadian Journal of Psychology 26: 19-41. Meyer, B.J.E (1985). Signaling the structure of text. In D.H. Jonassen, ed., Technology of Text, Vol. 2. Englewood Cliffs, NJ: Educational Technology. Robinson, D.H. & Skinner, C.H. (1996). Why graphic organizers facilitate search processes: Fewer words or computationally effective indexing? Contemporary Educational Psychology 2l: 161-180. Robinson, D.H. & Kiewra, K.A. (1995). Visual argument: Graphic organizers are superior to outlines in improving learning from text. Journal of Educational Psychology 87: 455-467. Robinson, D.H. & Schraw, G. (1994). Computational efficiency through visual argument: Do graphic organizers communicate relations in text too effectively. Contemporary Educational Psychology 19: 399-415. Schwartz, S.H. (1971). Modes of representation and problem solving: Well evolved is half solved. Journal of Experimental Psychology 91: 347-350. Schwartz, S.H. & Fattaleh, D.L. (1972). Representation in deductive problem solving: The matrix. Journal of Experimental Psychology 95: 343-348. Winn, W. (1988). Recall of the pattern, sequence, and names of concepts represented in instructional diagrams. Journal of Research in Science Teaching 25: 375-386.