Selective Review of Research Studies Showing Media Effectiveness: A Primer for Media Directors

J O H N A. M O L D S T A D The field of instructional technology always seems to be in a dilemma! On the one hand, important national and state studies such as To Improve Learning, the report to the President and the Congress of the United States by the Commission on Instructional Technology; the Carnegie Report, The Fourth Revolution; The Fleischmann Report, New York State's Commission Report on the Quali W, Cost, and Financing of Elementary and Secondary Education; and the New York State Regent's position paper, Instructional Technology, A Statement of Policy and Proposed Action, unanimously conclude that greater efficiency and economy in educational practice can only be achieved through efficient, effective use of technology. "In the conviction that technology can make education more productive, individual, and powerful, make learning more immediate, give instruction a more scientific base, and make access to education more equal," the President's Commission on Instructional Technology, To Improve Learning

]ohn A. Moldstad is professor of education in the division of instructional systems technology at Indiana University, Bloomington, Indiana 47401. AVCR VOL. 22, NO. 4, WINTER 1974

387

AV C O M M U N I C A T I O N REVIEW 1974 WINTER

:

388

(Tickton, 1970), concludes, "the nation should increase its investment in instructional technology, thereby upgrading the quality of education, and ultimately, the quality of individual's lives and of society generally" (p. 10). Few educational media specialists, and I venture most teachers, would do anything but say "amen" to the above! On the other hand, the dilemma comes with the pressing question, "How effectively is instructional technology being used in today's educational process?" Here noted educators are quick to come up with "educated guesses" and undocumented research claims. For example, the authors of To Improve Learning cite these opinions in their costly two-year study; Instructional technology is largely supplementary to the two primary media of instruction, the textbook and the teacher. Eliminate either of these and the educational system would be transformed. Eliminate all of the technology, and education would go on with hardly a missed lesson [Tickton, 1970, p. 21-Norman D. Kurland, Director, Center of Innovation in Education, New York State Education Department]. They are used primarily to give data which the reasonably effective teacher could give anyway and/or at least to furnish a momentary diversion from business as usual [p. 66-67-Wilbur Rippy, curriculum resources specialist at Bank Street College of Education]. According to informed opinion, audiovisual media are generally employed intermittently and then only to enrich and supplement the familiar patterns of classroom instruction. For the most part, they merely augment the conventional teaching strategy which has hardly changed for more than a century [p. 66]. Isn't it strange that this national report to the President and the Congress of the United States seems forced to base many of its conclusions on the basis of such "soft data" evaluations, especially when the U.S. Office of Education has over the past ten years spent millions through NDEA, Title VII and other sources with the expressed purpose of getting facts concerning the use of instructional technology in American education? The challenge to media professionals at work on all instructional levels seems one of deciding how to refute claims such as those quoted above. From long experience seeing media used extensively as an integrated part of instruction by both

RESEARCHSHOWINGMEDIAEFFECTIVENESS :

389

teachers and students, they know that instructional technology is not an "extra," providing something that teachers could well do without[ Ironically, most media professionals have built a case for the justification (or defense) of their programs upon equally "soft data." These data usually consist of utilization statistics, teacher testimonials or questionnaires and media attitude responses, and student documentation of use and preferences. Perhaps with today's increased emphasis on accountability and cost effectiveness, we need to add a new dimension to our presentations: research and/or evaluation data based upon comparative studies evaluating the relative student learning outcomes when systems incorporating instructional technology are pitted against traditional learning results. Typically, experimental psychologists have never been very enthusiastic about evaluative experiments in which learning from instructional programs using the so-called "newer media" is compared with learning results based upon conventional instruction. (Schramm, 1964; Allen, 1971) Their primary objection centers on the inexactness in controlling variables associated with either the experimental (newer media) approach or the control (conventional instruction) approach. Therefore, difficulties in determining the nature of the learner, learning stimuli and their interactions, and what specific learning results are achieved have precluded the development of anything approaching a science of instruction. However valid these objections may be as limitations in generating theory-oriented, generalizable learning principles, such evaluative studies should not be ignored[ They are doubtlessly useful in assisting educators to "prove" to themselves (and their constituencies) that newly designed instructional programs actually work. As decision-oriented efforts 1,

~Cronbach distinguishes "decision-oriented" from "'conclusion-oriented" investigations by indicating that "in decision-oriented study the investigator is asked to provide information to a declsion-maker: a school administrator, a governmental policy-maker, the manager of a project to develop a new biology textbook, or the like. The conclusion-oriented study, on the other hand, takes its direction from the investigator's commitments and hunch." (Cronbach & Suppes, 1969).

AV COMMUNICATION

R E V I E W 1974 W I N T E R

:

390

they provide valuable data to assist the school administrator in making instructional program decisions that are intellectually motivating and cost effective. Scriven (1967) agrees that when an administrator needs to know if method A is "better" than method B, the comparative study is the best way to find out. In the article, "Instructional Media Research: Past, Present, and Future," Allen (1971) discusses the history of such "evaluative" comparisons and writes, ... the educational establishment demanded proof of the effectiveness of these innovative techniques, and the baseline for comparison was clearly the current teaching practices. As a consequence, the general perception of instructional media research even today is in these terms [p. 6]. The educational establishment is still demanding such proof through research documentation! This became abundantly clear to the author following an address on comparative media research studies presented at the 1973 New York State Educational Communications Association annual convocation. Many requests were received from media directors for more detailed research data for use in their program justifications. Fortunately, twenty years of decision-oriented media research have produced significant evidence to justify the following claims when instructional technology is carefully selected and used: 1. Significantly greater learning often results when media are integrated into the traditional instructional program. 2. Equal amounts of learning are often accomplished in significantly less time using instructional technology. 3. Multimedia instructional programs based upon a "systems approach" frequently facilitate student learning more effectively than traditional instruction. 4. Multimedia and/or audiotutorial instructional programs are usually preferred by students when compared with traditional instruction. The studies selected for review in this article were of two types. One group documented what effect added stimuli using instructional technology (media plus current practice) had

RESEARCH S H O W I N G MEDIA EFFECTIVENESS

INTEGRATING MOTION PICTURES IN INSTRUCTION

:

391

on student learning when compared with conventional instruction where technology was either not used or used very sparingly. A second group included studies in which media were used to replace traditional instruction, i.e. programed learning, computer-assisted instruction, etc. The purpose here was not to be comprehensive or to review the many no-significant-difference media studies. Rather it was to call attention to selected studies in these categories that document the four above-mentioned claims. It is hoped that such information will be useful to media directors in their challenging task of justifying their media programs. The question of what effect integrating 16mm motion pictures into the traditional lecture-discussion format has on student learning has been with us for a long time. In the early 1930s, Rulon (1933) of Harvard University studied what effect integrating specially designed science films might have on student learning of both rote (factual) items and eductive (application) type problems. This classic study produced startling results in favor of the film-enriched instructional approach. The 14.8 percent superiority and 24.1 percent superiority of the text-plus-film groups over the textonly groups in immediate learning of "rote" and "eductive" items, respectively, and the 33.4 percent and 41 percent superiority on "rote" and "eductive" items, respectively, of text-plus-films groups over the text-only groups in retention of information after three and a half months is one of the most encouraging studies on film use. This is a classic study, still regarded as one of the best designed and executed. Also, it is one of the few showing the advantages of films in aiding students to apply conceptual understandings to new problem situations. Often teachers and administrators wonder whether use of several films will actually make a difference in student learning in a particular unit of study. Courtenay Nelson (1952) conducted just such a study. He wondered what effect, if any, using films in two of the ten class periods, normally including only lecture and discussion, would have on student learning in a unit on sulphur. Nelson found that students who had the advantage of seeing the sulphur films did significantly better on the comprehensive examination given at the end of the unit and also on the retention test five weeks later.

AV COMMUNICATION

OVERHEAD TRANSPARENCIES

R E V I E W 1974 W I N T E R

:

392

Wendt and Butts (1960) hypothesized that the wealth of U.S. history films should be an important asset to student learning of history concepts. Obtaining exceptional cooperation from the seven Carbondale, Illinois, high schools, they divided 315 tenth-grade history students into two groups, one group receiving traditional history instruction for the two-semester course and the second group spending only o n e semester covering content normally taking two semesters but using 54 carefully selected history films. Although spending only half the normal time, students who received the filmenriched approach learned 86 percent as much history as the control group when measured by mean scores on the Midwest High School Achievement Examination, Form A for World History. At the University of Texas, Clayton Chance (1960) and two other instructors of engineering descriptive geometry studied what effect the additional use of 200 specially prepared transparencies would have on student learning. In comparing this instructional approach (transparencies plus current practice) with their traditional lecture-discussion approach, covering identical content, the researchers arrived at the following conclusions: 1. The groups having the added use of the transparencies did significantly better on mean final course examination scores and final course grades (at .05 level of confidence). 2. The three faculty members unanimously agreed on the desirability of using these transparencies in their teaching. 3. Use of the transparencies resulted in an average savings of 15 minutes per class period. 4. Students reported overwhelming preference for instruction using transparencies.

USING FILMSTRIPSTO TEACH READING

Few comparative studies have been undertaken to assess the value of integrating filmstrips in instructional situations, even though surveys have revealed filmstrips to be one of the most used media of instruction. Kelley (1961) conducted a comparative study with first graders at Michigan City, Indiana, to study the effect of using filmstrips in teaching reading. He found that the youngsters who had the advantage of using filmstrips in their reading did significantly better on the Gates Primary

RESEARCHSHOWINGMEDIAEFFECTIVENESS :

EVALUATING THE EFFECTIVENESS OF LANGUAGE LABORATORIES

DO PUPILS LEARNFROM TELEVISION?

393

Reading Tests in word recognition (.01 level of confidence) and sentence reading (.05 level of confidence). The teachers reported the filmstrips improved student interest, stimulated class discussion, helped to fix basic vocabulary, encouraged the timid child, reduced teacher lesson-preparation time, and helped in phonetic and structural analysis. Although not a research report, Glenn McCracken's (1965) article reporting the 12-year New Castle, Pennsylvania, experiment in teaching reading to primary students via the filmstrip plus textbook approach was most encouraging. Every beginning reading class was reported to have achieved well into the third grade level as measured by standardized reading tests with scarcely a child-even those of 80 to 100 I Q s scoring below national norms on standardized reading tests. McCracken further reports that, in 1964, 27 beginning classes involving 731 pupils from other Pennsylvania cities learned to read by this filmstrip approach and achieved substantially the same results. A high percentage of these 731 cases scored above 2.5 while only 19 scored below the national norm of 1.8. Sarah Lorge's (1963) two-year comparative stu.dy of the effectiveness of language laboratories in ten schools involving 17 classes in New York City produced the following significant results in favor of their use, provided they were used at ]east twice a week for a minimum of 20 minutes: 1. The ninth-grade students using the language laboratory proved significantly superior to the nonlaboratory groups in French speech fluency. 2. The tenth-grade laboratory groups proved significantly superior in both speech fluency and intonation. 3. Eleventh-grade laboratory groups proved significantly superior to the nonlaboratory control groups on French comprehension at both slow and fast speeds. Chu and Schramm's monumental review, Learning from Television (1967), ably answers this question. There can no longer be any real doubt that children and adults learn a great amount from instructional television, just as they do from any other experience that can be made to seem relevant to them-experiences as different as watching someone rotate a hula hoop or reading the encyclopedia. The effectiveness of television has now been demonstrated in well over 100 experiments, and sev-

AVCOMMUNICATIONREVIEW1974WINTER :

394

eral hundred separate comparisons, performed in many parts of the world, in developing as well as industrialized countries, at every level from pre-school through adult education, and with a great variety of subject matter and method [p. 1].

Many research studies comparing cooperative television teacher-classroom teacher instruction with conventional instruction could be cited to show the resulting superiority of the "team" approach. C. F. Kelley (1964) made over 300 such matched achievement test comparisons between 1956 and 1961 in areas of mathematics, English, science, and social studies. Kelley found students generally exhibited good learning achievement when television was used as a regular resource. In fact, significantly higher achievement scores were made by the television groups in one out of every four comparisons! Persons researching such studies should refer to the above two reviews and to Reid and MacLennan's (1967) Research in Instructional Television and Film. In seeking to learn how students are affected when a school system adopts television as an additional instructional resource and integrates it fully into its instructional program, one naturally turns to Hagerstown, Maryland, and Anaheim," California. Serena Wade (1967) summarized performance gains made by Hagerstown students. Here are six significant gains reported after the introduction of television as a major resource in 1959: 1. In grades 3 through 6, rural pupils, averaging half a grade below the national norm in arithmetic (on Iowa Test of Basic Skills) before television, all came to exceed the norm: grades 3 and 4 after one year of television, the others after two years. In grade 5 arithmetic, the pupils gained an average of 1.9 years in knowledge of arithmetic concepts in one school year. 2. In junior-high general mathematics, the average achievement level of urban pupils on a standardized test of concepts rose in four years of televised instruction from the 31st percentile to the 84th percentile, and on a standardized test of problem-solving from the 33rd to the 68th percentile. Rural schools on the same tests rose from the 14th to the 38th per-

RESEARCH SHOWING MEDIA EFFECTIVENESS

:

395

TABLE1 Comparison of Growth with Pupils Taught Conventionally and by Television Taught Conventionally Taught by Television

Ability Level (Grade 6 Science)

Average IQ

Achievement Growth

Average IQ

Achievement Growth

111-140

117

12 too.

118

15 m o .

90-110

100

11 m o .

100

14 m o .

57-89

83

6 too.

83

13 m o .

centile on concepts, but made very slight overall gains in problem-solving. 3. In tenth-grade mathematics, urban schools rose from the 34th percentile before television to the 51st. 4. Analysis of sixth-grade science achievement showed television pupils improving more than conventionally taught pupils at all ability levels as is shown by Table 1 taken from Learning from Television (Chu & Schramm, 1967, p. 5). 5. In both the city and rural Hagerstown schools, grade 8 general science achievement on standardized tests was two years higher after several years of television than it had been before television was introduced into the program. 6. When television was introduced as an additional resource in the teaching of U.S. history in outlying Hagerstown schools, the percentile ranks on national norms increased from 28 in 1958 before television to 45 in 1959, 46 in 1960, and 50 in 1961. The Anaheim City School District's evaluation report (1963) of instructional television from 1959 to 1963 was clone by Welty Lefever and staff from the University of Southern California. Their study included two separate, but similar, investigations. The first studied 1157 fifth-grade pupils during a nine-month period; the second included 1016 fourthgrade students and extended over a period of 26 months9 Results of these two studies dramatically demonstrated that: 1. Of 48 comparisons using the California Achievement Test in evaluating the differences between post-television achievement with before-television achievement on the

AV C O M M U N I C A T I O N REVIEW 1974 W I N T E R

:

396

basic skills of similar groups of students, it was found that: a. Thirty-two comparisons favored-the post-television instructional groups at the .05 level of significance or better. b. None favored the before-television groups. c. There was an overall mean advantage of four months in achievement for the post-television groups. 2. Of 23 comparisons evaluating the difference between televised instruction plus regular instruction in regular-size classrooms and instruction without television in regularsize classrooms, it was found that: a. Eleven comparisons favored television-enriched instruction at the .05 level of significance or better. b. No comparison favored the classroom without televised instruction. 3. Lefever made further comparisons of large classrooms (75 students) using television instruction combined with conventional teaching by two subject matter teachers versus regular-size classrooms (25 students) with the regular teacher using traditional methods of instruction without television. The results were: a. Seven comparisons favored the televised instruction in large classrooms (75 pupils). b. Five comparisons showed no differences. c. Two comparisons favored self-contained classrooms without televised instruction. PROGRAMED INSTRUCTION: DOES IT HAVEA PLACEIN THE INSTRUCTIONAL PICTURE?

When programed instruction appeared as a promising tool in the instructional arsenal, numerous studies soon appeared evaluating its instructional potential as compared to more conventional means of instruction involving textbooks, lecturing, etc. In reviews of these evaluative studies, Lysaught and Williams (1963), Schramm (1964), and Silberman (1962) seem to agree that research on programed instruction confirms: a) that students can learn effectively, often more effectively, from all types of programed materials, whether in the form of linear or branching programs, and from programs on machines or programs in texts, than from more conventional instructional stimuli; and b)that frequently students learn equal amounts in far less time.

RESEARCH S H O W I N G MEDIA EFFECTIVENESS

:

397

Several studies showing significantly superior learning results in favor of prograrned learning approaches are cited below. In studying the relative effectiveness of various programing approaches in the teaching of elementary probability to 186 freshman engineering students at UCLA, Roe (1962) compared test results with student achievements using the nonprogramed lecture. All programed methods proved significantly superior to the lecture approach on criterion measures. Fincher and Fillmer (1965) studied the relative effectiveness of programed instruction with 309 fifth-grade arithmetic students and reported significantly superior results (.05 level of confidence) on criterion tests in favor of groups taught addition and subtraction of fractions using programed textbooks rather than by traditional lecture-discussion methods. Goldbeck (Goldbeck, Shearer, Campeau, & Willis, 1962) investigated the effectiveness of integrating programed texts with regular classroom instruction. Using 150 high school students in six government classes, he reported that programed texts combined with regular classroom instruction were significantly more effective than either programed texts or classroom instruction used separately. Hughes and McNamara (1961) reported that a class of IBM customer engineers using programed texts in a 7070 training course scored significantly higher on a criterion test than another group taught the same content by means of the lecture-discussion method. In addition, a substantial saving of instructional time was found in favor of the programed textbook group: a mean of 8.8 hours compared to 15 hours. Eighty-seven percent of the engineers indicated preference for programed instruction over conventional instruction. Porter (1959) reports a study in which two groups of sixthgrade pupils learned spelling, one by means of teaching machines and the other through conventional methods of spelling instruction. The achievement of the machine-instructed group was found to be significantly greater, except for the upper-IQ groups. Further, the machine-instructed group completed the program in about one-third the time. Ripple (1963) conducted a study using 240 students in an introductory psychology class at Cornell University. Students

AV C O M M U N I C A T I O N REVIEW 1974 W I N T E R

:

398

received one of four treatments: a standard programed text, a conventional text form, conventional lecture-discussion, or the programed text without feedback provisions. The standard programed learning group achieved significantly superior results as measured by the course criterion test. Marsh and Pierce-Jones (1968) studied the entire class of 295 students in eight sections of a semester-long adolescent psychology course to assess the effects on learning of using adjunct programed materials. Analysis of a 100-item, multiple-choice final examination revealed that those students using the programed materials as a supplement to regular instruction scored significantly higher than the control groups who attended the same laboratory but participated in tasks not involving programed instruction. One of the larger, nationwide experiments designed to evaluate the efficiency of programed materials in teaching core micro- and macroeconomics sections of the typical elementary economics course was reported by Attiyeh, Bach, and Lumsden (1969). Involving 48 schools and 4121 students from colleges and universities throughout the United States, the researchers' primary objective was to compare the performance of students using programed instruction, either by itself or as a supplement, with that of students participating in a conventionally taught basic economics course. Based on forms of the Test of Understanding in College Economics, student performance analyses revealed the following results: 1. On average, by spending 12 hours studying a programed learning text, students learned practically as much micro- or macroeconomics as did students in seven weeks of a conventionally taught elementary course. 2. On the basis of the test question breakdowns, students who used only programed learning materials, as compared to conventionally taught students, performed better on "applications" of theory than on simple "concept recognition." 3. Students had a generally positive attitude toward programed instruction [Attiyeh, Bach, & Lumsden, 1969, p. 217]. Finally, Price (1963) reported a study with 36 mentally retarded students with IQs ranging from 42 to 66. Two groups studied the 12-factor table in addition and

RESEARCH S H O W I N G MEDIA EFFECTIVENESS

MULTIMEDIA A N D AUDIOTUTORIAL SYSTEMS C O M E OF A G E

:

399

subtraction using different programed materials. The control group was taught by conventional instruction. Posttest results revealed no significant difference in amount of learning; however, the conventionally taught group spent 130 class periods while the programed text groups averaged 86 class periods. The application of a "systems concept" in instructional development has resulted in many multimedia systems designed to provide more effective and efficient instruction. Results of research studies comparing these new multimedia approaches with more traditional instruction should encourage educators to support this new methodology with its more judicious integration of instructional technology. Louis Romano (1955) did a pioneering study to determine what learning differences, if any, the integration of 16mm motion pictures and projected still pictures would make in the learning of science vocabulary in the fifth, sixth, and seventh grades. Children from two fifth, two sixth, and two seventh grade groups in the public schools of Shorewood and Whitefish Bay, Wisconsin, participated in the study with each group rotated to serve in turn as an experimental and a control group. In both control and experimental groups, blackboards, bulletin boards, charts, models, flat pictures, and field trips were used; but motion pictures and projected still pictures (filmstrips, 2 x 2 and 3-88 x 4 slides, and pictures used with the opaque projector) were used only in the experimental groups. Fifty-item vocabulary tests based upon science textbooks from the third-grade through the ninth-grade level were developed for each of the six science units studied. Prior to teaching the science units, vocabulary pretests were administered for each unit of study. Five weeks following the initiation of the unit of study, a vocabulary test for that particular unit was given both the experimental and control groups to determine the extent of vocabulary gain. A retention test consisting of these same vocabulary tests was administered to all participants six months after the conclusion of each unit in order to determine the amount of vocabulary retained over that period. Further data were gathered by obtaining the reactions of

AVCOMMUNICATIONREVIEW1974WINTER :

400

TABLE2 Percentage of Gain Achieved by Control Group Compared to the Experimental Group in All Units of Study in Grades Five, Six, and Seven

Actual Mean Gain Grade Level

Unit of Study

Control I

Experimental II

Percentage of Gain Achieved by Control Group Compared to Experimental Group

10.48

16.38

63.9

5

Electricity

5

Rocks

3.76

14.31

26.2

6

Astronomy

7.72

19.16

40.3

6

Sound

6.45

16.84

38.3

7

Air

4.36

13.00

33.5

7

Soil

3.84

11.56

33.0

teachers and pupils to the use of audiovisual materials in the classroom. 1. All experimental groups using the motion picture films and projected still pictures evidenced larger gains in vocabulary over the control groups in all units of study. 2. Retest scores of the experimental group in the rocks and astronomy units of study were lower than the final test scores, while in all the other units of study for the experimental group and for all the units of the control groups, the retest scores were higher than the final test scores. There was a loss in retention for the experimental group in only two of the units of study. 3. All of the teachers in this study pointed out the intrinsic value of the use of audiovisual materials in creating more effective teaching-learning situations. 4. The boys and girls participating in this study pointed out that an instructional program using many audiovisual materials enhances the learning situation. Table 2 summarizes the results of the study. Since the first audiotutorial program was initiated in botany at Purdue University by Samuel Postlethwait in 1961, hundreds of similar programs have been developed. Strangely, interest in evaluating the effectiveness of such programs has not paralleled interest in developing them.

RESEARCH S H O W I N G MEDIA EFFECTIVENESS

:

401

Sparks and Unbehaun (1971) reported a study done at Wisconsin State University at LaCrosse to evaluate achievement of students using an audiotutorial program as compared with student performance in a conventional I~ )logy course. One hundred-ninety students registered for the audiotutorial section of the general biology course were equated with a control group of 180 students registered for the conventional lecture-lab section of the same course. The natural science section of the American College Test was used as a pretest to check initial equality of the control and experimental groups. Analysis of results in presenting similar subject matter under two different instructional methods was based on scores on a 274-item biology exam, tested for content validity by a panel of biology instructors. Test results indicated that students in the experimental group (audiotutorial) did significantly better (.05 confidence level) than students in the control (lecture-discussion) group. On subtopic tests in chemistry, plant reproduction, and ecology-evolution, the audiotutorial group was significantly superior to the conventional instruction group, which failed to excel significantly on any of the nine subtopics. Meleca's (1970) study provides additional evidence that students enrolling in audiotutorial sections achieve significantly more than students in conventional lecture-laboratory sections. Hinton (1970) and Brewer (1970) reaffirm the general report of an upward skewing of grades in audiotutorial classes when compared with traditional grade distributions. In addition to audiotutorial laboratory systems where audiotape is the primary instructional medium, other types of multimedia systems are emerging where several media alternatives are offered students in open-laboratory environments. Such a variation of the audiotutorial science laboratory approach was reported by Edwards, Williams, and Roderick (1968). Beginning business students from Lansing Community College were subjects in two studies designed to compare their performance in learning beginning typing and first semester business machine operations when taught by two different methods. Students in the experimental group attended an open lab-

AV C O M M U N I C A T I O N REVIEW 1974 W I N T E R

COMPUTERADMINISTERED INSTRUCTION: HOW EFFECTIVE?

:

402

oratory and received instruction through programed materials and printed instruction sheets, continuous-loop sound films, tape-slide sets, and drill tapes while the control groups were taught in the traditional manner. In both courses, the students using the audiotutorial approaches learned significantly more (.05 level of significance) than the control groups as measured by end-of-term performance examinations. Students generally preferred the audiotutorial, open-laboratory method of learning. Stuck and Manatt (1970) investigated still another audiotutorial approach, the simulated "in-basket" problem-solving approach, in teaching 18 concepts of school law to preservice teachers and compared its effectiveness with the traditional lecture-discussion method. Two hundred-nineteen preservice teachers at Iowa State University were randomly divided into two groups. The control group attended seven hours of live lecture-discussion while the experimental group was given one week in which to complete simulated "in-basket" problems while role-playing a high school principal solving these problems with the aid of reference materials on school law. Pre-/posttest gains on a carefully validated criterion instrument indicated that the audiotutorial group's performances were significantly superior to the traditionally taught group's performance (.01 level of confidence). An additional significant finding was that the audiotutorial group spent 38.44 percent less time learning the 18 school law concepts. Evidence of the effectiveness and potential contribution of computer-assisted instructions (CAI) is beginning to accumulate. CAI programs such as PLATO, TICCIT, the work of Suppes and Hansen (1965) in elementary school mathematics and of Atkinson and Hansen (1966) in initial reading instruction in progress at Stanford University promise to provide continuing research data on which to judge the effectiveness of this new technological tool. In comparing CAI approaches with more traditional approaches, Hansen (1966) cites three comparative studies to support this conclusion: One of the most consistent findings with CAI tutorial applications is the marked saving in instructional time along with no loss in postinstructional achievement test performance [p. 596].

RESEARCH S H O W I N G MEDIA EFFECTIVENESS

:

403

Grubb and Selfridge (1964) taught the first half of a beginning descriptive statistics course to small numbers of college students via CAl. Their achievements were compared with those of college students receiving instruction in the conventional lecture-discussion mode. The CAI students spent a mean of 5.8 hours instructional and review time and achieved a midterm mean score of 94.3 percent. The conventionally taught students spent a mean of 54.3 hours in lecture, homework, and review time and had a rnidterm mean score of 58.4 percent. Goodman (1964) reported a study in which 3000 airline ticket agents were instructed via a CAI application used for airline reservation purposes. When compared with an equated group of ticket agents who received conventional instruction, the CAI group needed only half the training time and performed approximately 5 percent better as reflected by final test grades. Schurdak (1965) instructed 48 college students on one section of a Fortran programing course. After equating for mental ability, the CAI students were found to have performed approximately 10 percent better on the criterion measure than comparable students using either a standard or a programed text. The CAI students also used approximately 10 percent less time to complete the course. Posttest questionnaires administered to subjects in each of the three above mentioned studies revealed positive reactions concerning their CAI instructional experiences. A recent CAI study (Cartwright, Cartwright, & Robine, 1972) sought to compare the relative effectiveness of two forms of a program designed to prepare classroom teachers to engage in the early identification of children with problems expected to affect their school progress adversely. One hundred-fourteen college students enrolled at the Pennsylvania State University in an introductory course in special education were randomly assigned to one of two groups. One received all instruction by computer-assisted instruction; the other received instruction in the conventional lecture-discussion mode. The CAI group performed significantly better on the 75item final criterion test (.001 level of significance). Their mean score was 24 percent higher than that of the conven-

AV COMMUNICATION

COMPARATIVE MEDIA STUDIES: BOON OR BANE?

REFERENCES

R E V I E W 1974 W I N T E R

:

404

tional instruction group. In addition, the CAI students completed the three-credit course in 12 hours less time (33 percent) than the conventionally taught group. In contemplating the evaluative studies reviewed in this article, the reader may feel, as the author does, that gaps and inadequacies still exist. Better research designs, more insightful questions, more adequate sampling, and better integration of technology into instructional programs are but a few obvious needs. However, many educational decisions must be made by administrators and school board members on information that might be considered somewhat incomplete by educational researchers. Decision-oriented media research studies help in these decisions by: a)providing confirmation that specific instructional expenditures are resulting in improved student learning; b) establishing "proof" that instructional innovations, such as television, audiotutorial laboratories, computeradministered instruction, etc., are capable of producing student achievement levels as high or higher than those obtained with traditional approaches often at substantial savings of time, money, and resources; and c)providing longitudinal "track records" of student achievement under alternative strategies so as to better match individuals with the instructional approaches most suitable to their learning styles, interests, and motivational patterns. W. S. McCulloch says, "Don't bite my finger, look where I am pointing." The challenge is evident! Each media professional must work toward building a more extensive and higher quality base of research evidence to document the effectiveness and efficiency of using technology to improve instruction.

Allen, W. H. Instructional media research: Past, present, and future. A V Communication Review, 1971, 19, 5-18. Anaheim City School District. Summary of instructional television evaluation. Anaheim, Calif.: Anaheim City School District, 1963. (Unpublished manuscript.) Atkinson, R. C., & Hansen, D. N. Computer-assisted instruction in initial reading: The Stanford project. Reading Research Quarterly, 1966, 2, 5-25.

RESEARCH S H O W I N G MEDIA EFFECTIVENESS

:

405

Attiyeh, R. E., Bach, G. L., & Lumsden, K. G. The efficiency of programmed learning in teaching economics: The results of a nationwide experiment. American Economic Review, 1969, 59, 217-223. Brewer, I. M. An audio-visual method of teaching and learning in biology. Unpublished manuscript, Sidney University (Australia), 1970. Cartwright, C. A., Cartwright, G. P., & Robine, G. G. CAI course in the early identification of handicapped children. Exceptional Children, 1972, 38, 453-459. Chance, C. W. Experimentation in the adaption of the overhead projector utilizing 200 transparencies and 800 overlays'in teaching engineering descriptive geometry curricula. (USOE Project 243) Washington, D.C., 1960. (University Microfilm Pub. No. 61-3610). Chu, G. C., & Schramm, W. Learning from television: What the research says. (USOE Contract 2 EFC 70894) Stanford, Calif.: Stanford University, 1967. Cronbach, L. J., & Suppes, P. Research for tomorrow's schools. New York: Macmillan, 1969. Edwards, R. K., Williams, M. L. & Roderick, W. W. An experimental pilot study to explore the use of an audio-visual-tutorial laboratory in the secretarial offerings at the community college level in Michigan. Lansing, Michigan: Lansing Community College, 1968. (Ed 027387) Fincher, G. E., & Fillmer, H. T. Programmed instruction in elementary arithmetic. Arithmetic Teacher, 1965, 12, 19-23. Goldbeck, R. A., Shearer, J. W., Campeau, P. L., & Willis, M. B. Integrating programed instruction with conventional classroom teaching. Palo Alto, Calif.: American Institutes for Research, 1962. Goodman, L. S. Computer-based instruction: Today and tomorrow. Data Processing for Education, 1964, 3, 2-5. Grubb, R. E., & Selfridge, L. D. Computer tutoring in statistics. Computers and Automation, 1964, 13, 20-26. Hansen, D. N. Computer-assisted instruction. Review of Educational Research, 1966, 36, 588-603. Hinton, J. R. Audio-tutorial practices in California community colleges. Unpublished manuscript, Cabrillo College, California, 1970. Hughes, J. L., & McNamara, W. J. A comparative study of programmed and conventional instruction in industry. Journal of Applied Psychology, 1961, 45, 225-231. Kelley, C. F. The efficacy of television in the schools. Dissertation Abstracts, 1964, 24, 224. Kelley, T. D. Utilization of filmstrips as an aid in teaching beginning reading. Unpublished doctoral dissertation, Indiana University, 1961.

AVCOMMUNICATIONREVIEW1974WINTER :

406

Lorge, S. W. The relative effectiveness of four types of language laboratory experiences. (New York State Research Project A-61/62) New York: New York City Board of Education, 1963. Lysaught, J. P., & Williams, C. M. A guide to programmed instruction. New York: Wiley, 1963. Marsh, L. A., & Pierce-Jones, J. Programmed instruction as an adjunct to a course in adolescent psychology. Paper presented at the annual meeting of the American Educational Research Association, Chicago, Illinois, February 9, 1968. (Ed 018989) McCracken, G. Texffilm, enrichment for primary reading programs. Educational Screen, 1965, 44, 16-17, 30. Meleca, C. B. The use of multivariate analysis in the evaluation of audio-tutorial programs I and II. BioScience, 1970, 20, 23-30. Nelson, C. M. Effectiveness of sound motion pictures in teaching a unit on sulphur in high school chemistry. School Science and Mathematics, 1952, 52, 8-10. Porter, D. Some effects of year long teaching machine instruction. In E. Galanter (Ed.), Automatic teaching: The state of the art. New York: 1959. Pp. 85-90. Price, J. Automated teaching programs with mentally retarded students. American Journal of Mental Deficiency, 1963, 68, 69-72. Reid, J. C. & MacLennan, D. W. Research in instructional television and film. Washington, D.C.: U.S. Department of Health, Education, and Welfare, Office of Education, 1967. Ripple, R. E. A comparison of the effectiveness of a programmed text with three other methods of presentation. AID, 1963. Pp. 218-220. Roe, A. Automated teaching methods using linear programs. Journal of Psychology, 1962, 40, 198-201. Romano, L. The role of sixteen millimeter motion pictures and projected still pictures in science unit vocabulary learnings at grades five, six, and seven. Unpublished doctoral dissertation, University of Wisconsin, 1955. Rulon, P. J. The sound motion picture in science teaching. Cambridge, Mass.: Harvard University Press, 1933. Schramm, W. The research on programmed instruction: An annotated bibliography. U.S. Department of Health, Education, and Welfare, Office of Education, 1964, Contract OE 3-16-004. Schurdak, J. J. An approach to the use of computers in the instructional process and an evaluation. (Research Report RC-1432) Yorktown Heights, N.Y.: IBM Watson Research Center, 1965. Scriven, M. S. The methodology of evaluation. In R. Tyler and others, Perspectives of curriculum evaluation. Chicago: Rand McNally, 1967. Pp. 39-83. Silberman, H. F. Characteristics of some recent studies of instructional methods. In J. E. Coulson (Ed.), Programmed learning and computer-based instruction. New York: Wiley, 1962. Pp. 13-24.

RESEARCHSHOWINGMEDIAEFFECTIVENESS :

407

Sparks, P. D., & Unbehaun, L. M. Achievement of audiotutorial and conventional biology students-a comparative study. BioScience, 1971, 21,574-576. Stuck, L. L., & Manatt, R. P. A comparison of audiotutorial and lecture methods of teaching. The Journal of Educational Research, 1970, 63, 414-418. Suppes, P., & Hansen, D. Accelerated program in elementaryschool mathematics--The first year. Psychology in the Schools, 1965, 2, 195-203. Tickton, S. G. (Ed.) To improve learning: An evaluation of instructional technology (Vol. I). New York: Bowker, 1970. Wade, S. Hagerstown: A pioneer in closed-circuit televised instruction. New educational media in action: Case studies for planners-L Paris: UNESCO and International Institute for Educational Planning, 1967. Wendt, P. R., & Butts, G. K. A report of an experiment in the acceleration of teaching tenth grade world history with the help of an integrated series of films. Carbondale, Ill.: General Publications, 1960.