European Journal ofPsychology ofEducation 1996. Vol. xt. n° 4.443-457 © / 996. I.s.P..4.

An evaluation of two cognitive learning methods in adults on pre-qualification schemes: Logo and logical reasoning workshops (ARL) Daniel Chartier AFPA, Montreuil, France

Two groups of adults on AFPA pre-qualificati on training schemes were each exposed to a different programme. explici tly aimed at improving th eir cognit iv e s kills : Ateliers de Ra isonn em ent Logique/Logical Reasoning Workshops and Logo exercises. They were compared with a control group receiving special treatment. members being informed that they were participating in an experiment. This latter group was exposed. for an essentially similar period, to training in the use ofoffice -work softw are and computer-assisted education. The three groups were tested. at the beginning and end of the scheme. on a number of cognitive dimensions. The level of self-esteem ofparticipants. their attitude to comp uters and the internality of their attributions were also evaluated. Results showed: I) a modest increase in cognitive effi ciency in each group at the end oftraining and ambiguous variations in conative variables; 2) no superiority of the Logo group over the control group ; 3) superiority of the ARL group for two cognitive variables of the ten studied (combinatory analysis and p erceptive fi eld independence). The extent of the difference was invariably less that one standard-deviation of the distribution of sc ores. These results. and in particular the absence of transfer of acquisitions. seem insufficient to justify the conclusion ofany spec ific effic acy ofthis last method.

Introduction Long term unemployment currently affects virtually all work categories. It is nevertheless unden iable that those without any vocational qual ifications are the most vulnerable to the mechanism s of exclusion. Vocat ional trainin g thus appears to offer a solut ion likely to bolster the chances of such individual s establishing themselves. However, as production systems now exist, the majorit y of job s are increasingl y demanding in terms of cognit ive skills , with the additional requirement of a degree of intellectual mobility, an ability to constantl y adjust to frequent changes in the nature of jobs. Many people, who for various reasons have failed to achieve an adequate level of initial training, are unable to take on such jobs. In some, lack of We thanl: the lnst itut National d 'Et ude du Travail et d'Orientation Profe ssionnelle (I.N.E.T.O.P, Paris) for the help given in data p rocessing .

444

D. CHARTIER

academic and/or cultural skills is such that they cannot even obtain access to qualifications training since this is always subject to certain prerequisites in terms of aptitude or knowledge. The Association nationale pour la Formation Professionnelle des Adultes (A.F.P.A.) (National Adult Vocational Training Association) offers such individuals preparatory schemes intended: I) to bring them up to an adequate academic level, 2) familiarize them with the technological environment which they will have to master during their subsequent training, and 3) more generally provide them with the intellectual tools which will enable them to take on the challenge of qualifications training with a good chance of success. In order to achieve this third aim, several cognitive education methods are used at A.F.P.A.: the Feuerstein's Instrumental Enrichment Programme (IEP), Ateliers de Raisonnement Logique (ARL) (Logical Reasoning Workshops) and Logo-educabilite (Logo-educability). The choice of method is left to trainers, who use the deficiencies which they have identified in their trainees to define their particular aims and their educational background. The first of these methods has been evaluated recently (Loarer, Chartier, Huteau, & Lautrey, 1995). The present study reports the results of evaluation of the other two methods, using similar conditions regarding the type of individuals and features of the schemes (length and general objectives) concerned. The place of actions aimed at unqualified adults in the wider context of cognitive education will be defined, followed by a description of the two methods, with a review of the results of published evaluations. General problems raised by the evaluation of educational methods will also be considered. Cognitive Education in the Training of Unqualified Adults The term cognitive education (or learning) covers a number of educational methods which share the objective not of the acquisition of academic or vocational knowledge, but rather "the reconstruction of the cultural tools of learning and thought" (Buchel, 1995). This implies methods which are aimed at helping individuals to acquire the intellectual means they need for subsequent learning. It is useful, in this respect, to draw a distinction, following Buchel (\ 995) between two broad categories of objectives targeted by cognitive education methods. Some provide learners with procedures and strategies enabling them to improve their learning abilities. In order to facilitate the acquisition of knowledge, they are shown particular techniques (e.g., memorization) and problem-solving strategies applicable to learning situation categories of varying broadness. Others, more ambitiously, seek to educate their thought process, to develop their intelligence and reasoning abilities. Greater emphasis is placed here on the development of metacognition, independently of the content of knowledge. The two methods evaluated in this study are aimed at a general improvement in cognitive processes using exercises without any vocational content and hence fall within the second group of methods, i.e., that their objective is to develop a very broad reasoning ability. Cognitive education methods were initially developed in the context of special education, or to help pupils in great difficulty. All are based upon the "postulate of educability" (e.g., Bruner, 1983) according to which the intelligence of any individual can always be modified. In the field of adult education, highly concrete objectives of adjustment to current situations in the world of work have been assigned to cognitive education. Adults are to be provided with cognitive tools, mental flexibility in particular, enabling them to better adjust to tasks with requirements in a constant state of evolution. It will be seen that, from this standpoint, unqualified adults are grouped together with those who, for various reasons, show retarded development which must and can be dealt with. Methods and their Objectives Both methods evaluated in this study refer back to Piaget's operative development theory. However they are sufficiently different regarding the specificity of exercises as well as targeted aims to justify being described individually.

LOGO AND ARL IN ADULTS' TRAINING

445

Logo-Educability. Logo is a programming language, initially developed by S.Papert and M. Minsky at MIT in the I970s, for educational purposes. The authors, devotees of the operative theory of intellectual development, wished to provide children with a learning environment enhancing the onward move from concrete operations to categorical operations. According to Papert (1981), the everyday environment does not provide sufficient opportunities to apply so-called formal operations, in that the tasks with which the child is confronted most often call for action at a concrete level. The objective of the essentially play activities based upon Logo programming language is to bring the child into contact with "formal objects" and to place him/her in a situation of engaging in a hypothetico-deductive process. The child first familiarizes him/herself with a command language close to natural language. He/she gradually learns to control the basic movements of a cursor (the tortoise) which draws geometric figures of varying complexity on the screen. Once the direct command mode is mastered, the user can plan (in editor mode) the construction of basic figures, store the figures which he has created and move on to composite procedures enabling the production of increasingly complex figures. Cognitive abilities most likely to be developed by using Logo are: a) identification of an aim to be achieved and its breakdown into an orderly series of sub-aims; identification of actions required and their translation into Logo language; b) confirmation that the aim has been achieved; and c) identification and correction of possible errors. It can already be imagined how the use of Logo could be envisaged for cognitive remediation purposes, including in adults, for example by endowing the figures to be drawn using the tortoise with a technical content: production of technical drawings or simulated driving of mini-robots. This is how Logo is used in adult education. It provides a base for an educational approach aimed at the remediation of cognitive operations deemed defective. It should be noted that the desired aim is not the acquisition of skills, even modest, in programming. Programming activities are seen as a base for the acquisition of general thought strategies. During Logo exercises, the confrontation of viewpoints of participants is encouraged by group work in which individual strategies are shown to the group and errors discussed. Trainers often refer to the concept of socio-cognitive conflict (for a detailed description of educational Logo in vocational training, see Morseau, 1991). We are unaware of any evaluational study up to now of the use of Logo for cognitive remediation purposes. In contrast, extensive literature exists on educational Logo in school, and chiefly in young children. Particular interest has been shown in the effects on development of mathematical thought, the transfer of acquisitions to other situations and effects in the shorter or longer term. Published results are contradictory and only a few examples will be given here, Some authors have found no effect on logical development expressed as Piagetian stages (Howell, Scott, & Diamond, 1987) nor on cognitive or language abilities (Bailey, 1992). In an extensively documented review of the question, Salomon and Perkins (1987) noted that the transfer of acquisitions to remote tasks was almost never seen. In contrast, Black, Swan, and Schwartz (1988) or Yelland (1993) found effects on problem-solving strategies. Clements (1987) noted a long term effect on analogical reasoning, vocabulary and punctuation, and Noss (1987) found evidence of greater mastery of certain geometric concepts (conservation oflengths and angles). It would seem very difficult to summarize the various studies of the efficacy of Logo in a school context. Tests used to study direct effects or transfer, sizes of the samples involved as well as educational procedures used and periods of time differ widely. Many studies are also affected by methodological bias: they often involve small numbers of subjects, together with the initial equivalence between the control group and the Logo group being based upon the results of tests different to those used for post-treatment comparison. Positive effects are even sometimes inferred solely on the basis of intra-group variations between the start and end of treatment (no control group). The true existence and extent of these effects are also strongly linked to the age of subjects. It seems that mastery of concrete operations are a prerequisite for children to derive benefit from Logo exercises. Motivational or personality factors also appear to play the role of

446

D. CHARTIER

moderating variables. In short, the effects of Logo appear to essentially concern the acquisition and maintenance of strategies specific to problems of geometric construction or moving in space, without the possibility of these strategies transferring spontaneously to situations remote from those forming the Logo environment.

Logical Reasoning Workshops. Logical Reasoning Workshops (which will be referred to by their French abbreviation: ARL) (Higele, Hommage, & Perry, 1991) were originally conceived in the context of cognitive remediation for unqualified adults. They are "progressive exercises for the learning of intellectual operations". Their primary objective is "cognitive remediation (oo.) aimed at rehabilitation of defective cognitive functions". As with Logo, the reference theory is Piaget's model of intellectual development. Aspects borrowed from social interaction theories (Perret-Clermont, 1979; Mugny, 1985) are also invoked in the theoretical framework of the method, without their application being precisely explained in practical instructions to trainers. Participants are assigned to homogeneous groups on the basis of an assessment of their logical development level. Workshop sessions consist of series of progressive individual exercises enabling self-training in the various operations of seriation, classification, combinatorial analysis, propositions logic etc. In each session, individual work is followed by collective interchange on the solutions found and ends by seeking, under the trainer's supervision, situations in which the intellectual operations just performed might apply. There have been few studies up to now of the effects of ARL. In two of them, undertaken by the originator of the workshops (Higele & Martin, 1979; Higele, 1992), the control group received no particular treatment during the experiment. Evidence was found post-test of superiority of the experimental group in tests analogous to workshop exercises, but no difference between the two groups as to general cognitive efficiency nor personality dimensions. Another study (Chartier & Rabine, 1989) included, in addition to a control group without treatment, a control group whose treatment consisted of educational sessions concerning vocational choices. Subjects were adolescents (41 per group) in their last upper secondary school year. A posttest of the level oflogical development failed to differentiate between the three groups. The study reported here should enrich the few available empirical data on the effects of ARL. Evaluation Difficulties The evaluation of cognitive education methods raises a number of problems. Some are shared by all field research areas and can be partially resolved by the use of appropriate observation methods, aimed at neutralizing any bias encountered in a quasi-experiment. However, other problems are specific to the evaluation of educational methods. This applies to the choice of evaluation devices (Paour, Jaume, & de Robillard, 1995). It is essential to seek the closest possible harmony between the cognitive change objectives of the method evaluated and the techniques used to confirm that these objectives have been attained. We were forced to compromise between three constraints of different types: a) the taking into account of objectives described by the originators of methods (these objectives being identified from the available literature - summarised in the paragraph above - but also in relation to effects seen subjectively by trainers and reported at a working meeting; b) the taking into account of the specific characteristics of learners, who were adults with little schooling; c) material demands requiring the least possible use of individual tests, which would have been too time-consuming. There is also widespread agreement that the essential criterion in evaluation of the efficacy of a cognitive educational method is the transfer to new situations of acquisitions obtained during learning (Buchel, 1980; Higele, 1992; DeCorte, 1992; Huteau & Loarer, 1992; Paour et al., 1995). It is a confirmation that trained individuals can resolve problems not included in their training which shows that their cognitive function has been modified rather than that they have merely been trained in the solution of specific problems.

LOGO AND ARL IN ADULTS' TRAINING

447

Expected Effects At the End a/Training Any educational intervention lasting a certain time has effects on at least certain characteristics of the individuals concerned. The hypotheses which will be put forward here concern the specific effects of the two methods examined, i.e., what, in progress achieved at the end of training, can be attributed to the method used, in comparison with the effects seen at the end of "ordinary" training of similar length and also given to similar individuals making up a control group.

Common Objectives a/the Two Methods. Logo and ARL share the objective of cognitive remediation, an objective expressed in terms of the development of intellectual abilities. Specific exercises target the acquisition of a general ability to "learn how to learn" which can be transferred to other tasks, e.g., solving problems. Thus Higele et al. (1991, p. 15) talk of "restructuring operative mechanisms". Similarly, Logo is expected to provide "development of intellectual flexibility" (Morseau, 1991, p. 189). Following training using one of these methods, participants would thus be expected to show a significant improvement (differential) in their intellectual efficiency. Because of their general nature, it should be possible for these acquired abilities to be transferred to tasks not included in training, in particular planning tasks with a non-technical concrete content. Mistakes are given a special status in the stated educational principles of each method. Routine analysis of mistakes should encourage trainees to more accurately attribute their successes and failures to their own actions rather than to external influences (luck, difficulty of task or supervisor's attitude). This implies that increased internality of causal attributions was expected in experimental groups. Specific Objectives. One of the chief activities involved in Logo consists of an analytical breakdown of increasingly complex geometric figures, then recomposing these elements to produce other figures. Cognitive operations of this type are involved in tests of Field Dependence-Independence. A notable improvement in field-independence is expected at the end of training in subjects using Logo. In addition, the spatial visualization processes also exploited and trained by these activities should lead to significant advances in spatial aptitude tests. It might be reasonable to consider that using a computer could help to familiarize partici pants with such devices and could have a favourable influence on the attitude of participants to the world of computerization. The authors of ARL mention a number of changes which they believe they have seen in trainees (Higele et al., 1991; pp. 65-66). It is possible to identify, among potentially modifiable individual characteristics, some which hark back to psychological variables: chiefly impulsivity and self-esteem. The concept of impulsivity has been examined in the context of studies on the cognitive style of reflection-impulsivity, brought into play in perceptive recognition tasks (Ancillotti, 1985). Impulsive subjects are defined as answering quickly and making many mistakes, while reflective subjects show an opposite pattern: they have longer latent periods and make few mistakes. ARL would be expected to have an effect leading to a differential decrease in the impulsivity of trainees in a picture recognition task. Changes in self-perception, announced by the authors of ARL, should also result in the finding of a differential increase in self-esteem.

Method Nineteen trainers and sixteen psychologists of the Association nationale pour la Formation Professionnelle des Adultes (A.F.P.A.) took part in the study. Trainers of Logo and ARL groups were chosen on the basis of their experience of the method concerned. Prior to the study, all received specific training in the method used and had

448

D.CHARTIER

actually been using it for several years. Psychologists participated in data collection and some ofthem co-led ARL sessions. A smaller group of trainers and psychologists (about 10 people) participated in decisions concerning the choice oftests and setting up of the observation procedure. Subjects Subjects were initially 250 unqualified adults (80% men) participating in an A.F.P.A. qualifications training preparatory scheme. Their level of skills and/or aptitude was such that they were unable to directly enter skilled worker training, and pre-training of about three months had been recommended. These schemes, including about 15 participants, took place at training centres throughout France. Because of occasional absences or departures during schemes, 84, 70 and 85 subjects participated in both the pre- and post-test in the Logo, ARL and Control groups respectively. Observation Design The observation design set up was a quasi-experimental design (Huteau & Loarer, 1992) consisting of three groups of subjects. A first experimental group took part in Logo-Educability sessions (I h30min twice a week for thirteen weeks). A second experimental group took part in ARL sessions with a similar time pattern. In order to neutralize the novelty effect or Hawthorne effect likely to be seen in experimental groups, the control group also received additional treatment (placebo). This treatment, not aimed at cognitive remediation, was chosen on the basis of the characteristics of the experimental groups, and chiefly those of the Logo group. It was necessary in the latter to draw a distinction between the actual effect of Logo and the effect of introducing computer work into traditional training. The control group, members of which were clearly informed that they were participating in an experiment, participated in computer workshops for an hour a day, described by the trainer as an innovation in comparison with ordinary schemes. Subjects were provided with word processing software, spreadsheets and didactic mathematics software. They were given a minimum of instructions concerning their use. It may be noted that in terms of duration, the control group received specific training for considerably longer than that of the two experimental groups. However, insofar as access to computers was "self-service", the actual time of contact with softwares of each subject was difficult to assess and was, on average, probably less than the theoretical duration. Subjects of all three groups underwent a pre-test during their first two weeks on the scheme and a post-test about three weeks before the end of the scheme. The time between taking the two tests was therefore about ten weeks. The same tests were used each time, with the exception of the planning task which had two isomorphic versions. Material Apart from the planning test (see below), tests used were of pencil and paper or questionnaire type, taken as a group. Intellectual Efficiency. Four "Piagetian" type tests were used. Raw scores obtained by subjects were considered to be indices of their intellectual efficiency in different areas of content. These scores could also, subject to good hierarchisation of items, be used to assess a level of logical development, which was primordial for the creation of ARL working groups. Scores were expressed as the number of correct answers. Areas evaluated concerned: a) Combinatorial analysis: Letters sub-test of the Echelle Collective de Developpement Logique/Logical Development Collective Rating Scale (ECDL, Horneman, 1974).

LOGO AND ARL IN ADULTS' TRAINING

449

This involves finding all the combinations which can be made with 3, 4,5 then 6 different letters. For 5 and 6 letters, subjects must also justify their answer by indicating how they found the number of combinations (all phrases explaining the calculation of n! are considered to be correct). b) Propositions logic: this is an original test, used in ARL initial assessment. A "game" consists of eight cards each showing zero to three symbols (heart, club, diamond). A rule is stated (e.g., "if the club is present, there should be neither heart nor diamond"), and all cards not satisfying the rule must be discarded. This test has the advantage of minimizing the need for verbal comprehension. c) Logical multiplication of classes: adaptation of the ECDL Crossovers sub-test. The original test had to be modified in order to lessen the risks of bias due to the limited vocabulary of subjects. The subject is presented with five objects and must find those belonging to two categories indicated, each of which has two representatives. For example, the element forming the intersection of classes must be found among four distractors: (bull, ox, heifer, cow) and (foal, lamb, piglet, colt) - (answer: calt). d) Integration of simultaneous variations of two dimensions on pictorial material (ECDL Drawings sub-test). A drawing in perspective shows a cylinder covered by a sheet of paper and equipped with a pencil fastened to a bar parallel to the axis of rotation of the cylinder. This pencil can move along the axis. A phrase tells the subject to imagine certain simultaneous movements of the drum and the pencil (e.g.,: a turn of the drum and a single movement of the pencil). The subject must draw what the pencil would mark on the paper. Success in this test implies that the subject is capable of interiorising the combination of two distinct mental operations. e) A spatial aptitude test which saturates the Visualization factor: Bricks test of the l.N.n.p. Research Centre Battery (Reuchlin & Valin, 1953). A pi.e of 2, 3 or 4 parallelepipeds is drawn in perspective. The subject must discover, among four drawings proposed, which represents the pile of bricks seen from a different viewpoint symbolised by an arrow. t) An individual planning test was also proposed. This involved simulation of the preparation then accomplishment of a delivery involving a certain number of constraints, in particular regarding time (Lorries test, Loarer & Lautrey, 1994). This test is designed to evaluate the possible transfer of general metacognitive abilities (interiorisation of action, breakdown of an aim into sub-aims, checking during execution and final check) to a situation not studied during training. The subject is provided with a model lorry, cubes representing parcels, a map oftowns to be delivered and customers' order slips. These slips indicate in particular the customer's demands (delivery in the morning, the afternoon or at any time). The subject must draw on the map the delivery route which they feel will meet the various conditions imposed and which is also the most economic in terms of distance covered. Once this itinerary is drawn, the subject loads the lorry then carries out the delivery. The test is taken in interactive mode. The operator questions the subject about reasons for choices and points out any possible failure to comply with instructions. It is very likely that learning of the task occurs as the test is being taken. In order to compensate for this learning effect, which might have made the task too easy in the post-test for the majority of subjects, the mission in post-testing was a structurally identical variant of that 0 f the pre-test: names of customers and towns have been changed, the route was a mirror image of the first and the lorry was no longer loaded from the back but from the side. Previous use of this test has shown that these modifications are sufficient to avoid any post-test ceiling effect. A number of indices are noted by the operator as the test is taken, essentially in terms of the presence or absence of particular behaviour (e.g.,: classification or not of order

450

D. CHARTIER

slips by the subject). These indices are then combined to give three composite scores: a) latent time between the instruction being given and when the subject begins to act (drawing route or loading lorry); b) the number of planning errors (taking into account the number of itinerary rough drafts used, the number of parcels incorrectly loaded and the number of reloadings necessary during delivery); and c) the quality of the route drawn as well as the quality ofthe explanation provided when the route has been drawn and when the lorry is loaded. The score for this last index is 0 points when there is no explanation, I point when the explanation is only descriptive and 2 points when a principle is stated (e.g., "l loaded the lorry in reverse order of the delivery in order to have the last parcel to be delivered the farthest in, then the last but one, etc.). Field Dependence-Independence. The GEFT (Oltman, Raskin, & Witkin, 1971/1985) was used, a test in which the subject must locate a simple picture camouflaged within a compicated picture. Reflection-impulsivity. Instructions of Thurstone's Identical Pictures Test (1959) were adapted in order to be able to obtain an error score and a speed score. The test was taken with out any time limit and subjects were given two instructions: they were to answer as quickly as possible without making mistakes. Each of the sixty items showed as a model a diagrammatic picture alongside five other almost similar pictures except one which was the precise replica of the model. The subject was to identify the drawing identical to the model as quickly as possible. After distributions of errors and test times were collected, raw scores were converted to reduced centred grades (z grades). Impulsivity index was defined as the difference between the error grade and test time (Ivze-zt). In an individual with an impulsive tendency, this index was positive and all the higher when the subject made more mistakes than average, answering faster than average. On the contrary, individuals with a reflective tendency would have a negative index, being all the smaller when less mistakes were made while taking much more time to answer. In general, time measurements are very greatly affected by extreme values. The distribution of index I is hence dyssmetrical. Thus in the pre-test, mean and median were 0.35 and 0.305 respectively. However, this index was kept as the impulsivity score in that 95% of subjects' in which the distribution was close to normal, were between -2 and +3. Causal attributions. An adaptation of Rotter's questionnaire (1966) developed during a previous study (Loarer, Libert, Chartier, Huteau, & Lautrey, 1991) was used. The adaptation consisted above all of simplification of formulations of items to render them accessible to subjects of low cultural level. It is a 16 item forced choice questionnaire in which the subject must decide between two propositions; one indicative of an external attribution and the other of an internal attribution. E.g.: Most of our difficulties are due to bad luck (external) contrasting with Most ofour problems are due to our mistakes (internal). Scores are internality scores and are all the higher when internal answers are selected. Self-esteem. This is an abridged version of Coopersmith's Self-Esteem Questionnaire (1984) including the 27 items most saturated by the work self-esteem factor. Subjects must state whether each of the 27 propositions resembles them or not. E.g.: I do my job as well as I can. Two self-esteem scores were used to take into account the factorial structure of answers to the pre-test questionnaire. Only the first two factors could be interpreted, and accounted for 35% of total variance. Thus two self-esteem scores were taken into account: a "self-denigration" score (Factor I), higher when the subject gave answers indicative of a self-deprecatory attitude and a social self-image score (Factor 2). Attitude toward computers. An original questionnaire consisting of 20 items expresssing

LOGO AND ARL IN ADULTS' TRAINING

451

varying degrees of positive or negative attitude (from Agree completely to Do not agree at all) to personal use of domestic or didactic applications of computers. E.g.: 1 should like to belong to a computer club. Factorial analysis of the questionnaire in the pre-test led to the identification of two dimensions of attitudes to computers: - a first dimension (89% of variance explained) was saturated by items evoking domestic and didactic uses of computers; - a second dimension (10% of variance explained) combined items referring to games aspects of computers. The two scale identified in the pre-test were used to calculate post-test scores.

Results It was to be considered that one of the experimental situations (Logo or ARL) had a specific effect if a significant difference was found at post-test between the results of the experimental group concerned and those of the control group. The arbritrary value generally adopted in this type of study for the a threshold is .05. This involves the risk of erroneously rejecting the null hypothesis in a single comparison. We actually applied a series of 15 comparisons to post-test means. The possibility exists, under such circumstances, that certain differences might be deemed significant by chance. The global a threshold (Stevens, 1992) for all comparisons was then .05 x 15, i.e., .75. In order to ensure a global aof .05, complying with conventions, we had to use a corrected threshold of .05/15, i.e., .0033, for each basic comparison. The fact of considering a difference as being significant implies the existence of an effect, but does not indicate whether it is marked or not. The practical problem which arises in the context of training is not so much whether an effect exists as its degree. It might be wished to compare the effects of one or more methods not with the absence of any action but with an expected effect, reflected by a mean increase in performance considered notable according to external criteria, specific to the "semantics of the field" (ReuchIin, 1977, 1992) It might be decided, for example, that training had a notable effect if 95% of learners showed evidence of performance at the end of the training scheme exceeding the initial mean of the group. When comparing the means of two groups, indication of the size of the effect (Cohen, 1977; Corroyer & Rouanet, 1994) was obtained by relating the difference in mean to the mean standard-deviation of distributions (d=D/om). The presentation of results will be directed towards the size of effects, though without neglecting their statistical significance. It may be noted to clarify these ideas that when d=.05o; almost one third of subjects in the weakest group had score higher than the mean of the strongest group. Comparisons Between Pre- and Post-Test for Each Group

The first phase involves consideration of variations over the course of time in the various characteristics evaluated. Table I shows the size of effects seen for each group, between pretest and post-test. The difference found between pre- and post-test in a group is not a univocal indicator of the effect of training. Progress between pre- and post-test must be attributed to the effect of at least two factors: the impact of training and the retest effect, and there is no easy way of distinguishing between these two effects. It is only comparisons between the various groups which will then be analysed, capable of providing indications of relative progress, since it is reasonable to assume that the retest effect was similar in all three groups.

D. CHARTIER

452

Table I

Effe ct s izes (as fra ction of standard deviation) between p re- and post-t est fo r each group Group Variable Intersection of classes Combinatorial analysis Coordination of two conversions Propositional logic Spatial aptitude Field independence Reflection-impulsivity"

Internalityof control General self-esteem? Social self-image Computing (utilitarian) Computing( games)

Logo 0,06 0,07 0,18 0,10 0,36 0,36 0,01 0,17 -0,11 0,07 -0,25 0,09

ARL

Control

0,42 0,67 0,22 0,19 0,55 0,84 0,06 0,09 -0,34 0,09 -0,01 0,18

0,26 0,38 0,10 0,19 0,34 0,36 -0, 13 -0,01 -0,24 0,03 -0,09 -0,35

Note. a This concerned variations in impulsivity index. A negative effect indicates a greater reflectivity; b Self-denigra-

tionindex. A negative effect indicates an improvement in self-esteem. Effects greater thanhalf a standard-deviationarein bold type. Progress found at the end of training was generally positive but relatively slight. Taking

.50 as the value indicative of a moderate effect (Corroyer & Rouanet, 1994), it emerged that onl y the ARL group progressed to an y notable extent and in th ree areas onl y: a) combinatorial analysis, b) spatial ap titude and c) field-independence.

Comparisons Between Groups at Post-Test In order to take into account inequalities concerning the initi al level of the three groups, post-test means were corrected by analysis of covariance (the covariable being the pre-test score). It should be remembered that the point in q uestion was the differential effect of each of the methods evaluated, i.e., the add itiona l progress which it possibl y produced in relation to standard learning from training assessed on the basi s of progress in the control g ro up. Results are reported in Table 2.

Table 2

Comparisons ofpost-test sco res of the experimental groups with those of the control group (Afte r correction ofmeans for covariance) Variable Intersection of classes Combinatorial analysis Coordinationof two conversions Propositional logic Spatial aptitude Planning(time) Planning(mistakes) Planning(quality of answers) Field independence Reflection-impulsivity Internality of control Generalself-esteem Social self-image Computing(utilitarian) Computing( games)

Comparison

Effect sizea

ns ARL > Control (p<.0023) ns ns ns os ns

0,33 0,76 0,37 0,19 0,21 0,18 0,31 0,06 0,26 0,19 0,20 0,14 0,27 0,18 0,08

ns

ARL > Control (p<.0008) os ns os ns ns ns

Note. a As fraction of standard deviation. When a difference is significant, the corresponding effect size is shown, in

boldtype. When differences arenotsignificant, theeffect sizewiththegreatest absolute value is shown.

LOGO AND ARL IN ADULTS' TRAINING

453

It will no doubt be noticed in Table 2 that non-significant differences were sometimes accompanied by greater effects than those seen when differences were significant. This might seem surprising, but it must be kept in mind that standard-deviations of distributions, upon which the size of effect depends in particular, varied widely according to the tests used, while frequencies in groups, upon which the probability of rejection of the null hypothesis depended, were fixed. Significant differential effects were seen for only two variables, in both cases to the advantage of the ARL group. For the Combinatorial analysis test, the ARL-Logo difference was .71 standard-deviations, (significant at p<.OOO 1) and the ARL-Control difference was. 76 standard-deviations (significant atp<.0023). In the field dependence-independence test, ARL-Control and ARL-Logo differences were .26 standard-deviations, significant at p<.0008).

Discussiolli Variations Between Pre-Test and Post-Test

The effects of training, taking all groups together, on cognitive variables were all positive but modest in extent. The most marked progress was seen in the combinatorial analysis, spatial aptitude and field dependence-independence tests. These three tests are based upon a figurative support. It is likely that this progress could be partially explained by technical exercises (and more particularly familiarization with diagrams). It may be noted that the ARL group made the most progress, the effect of training being moderate in size. In the conative area there was evidence in all groups of a decrease in self-denigration. This situation effect was no doubt attributable to the educational type of pre-qualification schemes which are in many ways (e.g., practical exercises, no final examination) different from school situations of which unqualified adults are likely to have retained a memory associated with feelings of personal failure. Differential Effects at Post-Test

The Logo group showed no superiority over the control group. The absence of specific Logo effect was no doubt due, in part, to the fact that these schemes were relatively short. It is unlikely that learners, without any initial idea of programming, would have had the time to become skilled with the various Logo commands at a sufficient level to enable them to produce relatively complex technical drawings or automation conceptions. Under such circumstances, repeated exercises involving inevitably basic geometric figures could lead to a degree of demotivation, signs of which are found, for instance in the less positive attitude to computers found at post-test. The ARL group had significantly better results in only two tests of the ten, including only one of the Piagetian tests used (Combinatorial analysis, d=.760'). Our explanation for these results is that the combinatorial analysis exercises in which subjects were trained during workshops lent themselves more particularly to the learning of a highly spatialized and easily autornatable specific strategy. This did not apply to other areas of logical reasoning in which training was provided. Taking the mean effect on the four tests relative to logical reasoning (i.e., .320'), this effect was notably greater than that (.24) found in an earlier study (Chartier & Rabine, 1989) in which subjects were adolescents in their 4th year of secondary school. These results also share certain features with those of studies evaluating another cognitive education method wittl similar objectives. Thus the effects of IEI' on intelligence tests had a median value of .33 in the study of Feuerstein, Rand, Hoffman, and Miller (1980) in which the programme was used for two years. A median effect of .l Szr was found when the same programme was used for 15 weeks in adults on vocational pre-training (Loarer et aI., 1995).

454

D. CHARTIER

A differential effect of .26lT was obtained in the ARL group for Field DependenceIndependence. This effect must be viewed in light of progress made between pre-test and posttest in the spatial aptitude test, because of the strong geometric component in the items of the DIF test. An effect of similar extent, taking into account the longer period of use of the programme (.820) was reported in the study of Feuerstein et al. (1980) where, similarly, one of the most marked effects was seen in Thurstone's Spatial Relations test. With regard to personality variables (attitude, self-esteem) the absence of effects seen is in agreement with the results of the studies cited above.

Conclusion Expected differential effects were only very partially seen, and for only one of the methods evaluated. In particular, no evidence could be shown of the transfer of acquired skills to situations not covered by training (planning activities in the delivery test). It would seem that under the conditions adopted in this study, Logical Reasoning Workshops, and even less Logo-educability, do not fulfil the targets of cognitive education aimed at in the training of unqualified adults. There could be several reasons for this absence of effect. First, methods aimed at unqualified adults are extensively derived from research in the domain of special education and practices based upon such research (e.g., Paour, 1992). Overhasty consideration that the difficulties of these adults are identical to those of mentally retarded children or adolescents seems dubious at the very least. While it is true that cognitive performance, in certain situations, of unqualified adults is similar to that of the mentally retarded, they show evidence most of the time - in everyday life, for instance - of adaptational behaviour indicative of normal cognitive ability. A plausible hypothesis is that the counterperformance of adults can be explained more by affective/motivational factors linked to their often unhappy contact with school type skills than by retarded development. In addition, both ARL and Logo-educability are founded upon the Piagetian model of child intellectual development. One of the props of this model is the existence of a logical overall structure independent of the content of knowledge and it is the adoption of this model which justifies the choice of exercises with a trite content. More recent research has nevertheless shown that I) each area of knowledge has relative specificity (for example memorization ability is not independent of the material to be memorized, Chi, 1978) and 2) the structuration of material imposes constraints on the learning of new knowledge (see in particular the role of "preconceptions" and of intuitive theories, in the learning of concepts of physics, Clement, 1993). Far more than schoolchildren, adults have encountered during their existence opportunites to construct and consolidate intuitive theories concerning their surroundings. They must sometimes "forget" knowledge acquired prevously while doing another job which they had to give up. Exercises used for cognitive educational purposes have a deliberately trite and noncontextualised content. It is unlikely that the mental operations applied in these exercises as well as the cognitive representations developed at the same time could be transferred to areas, for instance technical, with very different contents and which possess their own structuration. Findings in this study are along the same lines as those emerging from other research (e.g., Feuerstein et al., 1980; Chartier & Rabine, 1989; Loarer et al., 1995). This might raise the question of the pertinence of cognitive education methods in adult training. It may neverthess be noted that the incorporation of pedagogic principles in cognitive education has had an extremely positive impact on the representations and practices of trainers. This applies to the adoption of the modifiability postulate which has brought about a major turnabout in ways of considering the potential of scheme participants. Similarly, the taking into account of learners' mistakes as a particularly fruitful source of information on

LOGO AND ARL IN ADULTS'TRAINING

455

their cognitive functionning has made trainers aware that learners are not merely receptacles of knowledge. Last but not least, the role played by social interaction in learning has considerably modified educative practices in adult training. In conclusion, concerning cognitive education, mention should be made again of a number of suggestions made by Huteau et a!. (1994). We feel that the success of cognitive education in the training of unqualified adults requires the discarding of overambitious objectives aimed at trying to change general intellectual ability. It also involves definite adjustment to adults of the pedagogic principles of cognitive education, by very strongly attaching them to the characteristics of work situations. Very close attention must be paid to analysis of the cognitive operations actually used in specific work situations, taking into account the most recent research in cognitive psychology and ergonomics, in order to better master the conditions and limitations of expected transfers. Rather than expecting an effect of cognitive changes on affective/motivational factors, these must be taken into account as variables which it is possibe to try to influence in order to facilitate learning.

References Ancillotti, J.P. (1985). Dimension reflexive-impulsive de la personnalite et fonctionnement cognitif. Cahiers de Psychologie Cognitive, 5,667-686. Bailey, J. (1992). Developing language and cognition in young children with Logo. British Journal of Educational Technology. 23, 89-94. Black, J.B., Swan, K., & Schwartz, D.L. (1988). Developing thinking skills with computer. Teachers Col/ege Record, 89,384-407. Bruner, J. (1983). Le developpement de I 'enfant. Savoir faire. Savoir dire. Paris: Presses Universitaires de France. Buchel, F.-P. (1980). Der Transfer von Lernstrategien. Institut de Psychologie de l'Universite de Bale. Buchel, F.-P. (Ed.] (1995). L'education cognitive. Le developpement de la capacite d'apprentissage et son evaluation. Neuchatel: Delachaux et Niestle. Chartier, D., & Rabine, P. (1989). Evaluation d'une methode de remediation cognitive. Le cas des ateliers de raisonnement logique, L 'Orientation Scolaire et Professionnel/e, 18, 127-137. Chi, M.T.H. (1978). Knowledge structure and memory development. In R. Siegler (Ed.), Children's thinking: what develops? (pp. 73-96). Hillsdale, NJ: Erlbaum. Clement, J. (1993). Using bridging analogies and anchoring intuitions to deal with students' preconceptions in physics. Joumal of Research in Science Teaching, 30, 1241-1257. Clements, D.H. (1987). Longitudinal study of the effects of Logo programming on cognitive abilities and achievement. Journal cfEducational Computing Research, 3, 73-94. Cohen, J. (19771. Statistical power analysis for the behavioral sciences (2nd ed.). New York: Academic Press. Coopersmith, S. (1984). Manuel de I'Inventaire d'Estime de Soi. Paris: Editions du Centre de Psychologie Appliquee. Corroyer, D., & Rouanet, H. (1994). Sur I'importance des effets et ses indicateurs dans I'analyse statistique des donnees. L 'Annee Psychologique, 94,607-624. De Corte, E. (1992). On the learning and teaching of problem-solving skills in mathematics and Logo programming. Applied Psychology, 41,317-331. Feuerstein, R., Rand, Y., Hoffman, M., & Miller, R. (1980). Instrumental enrichment: An intervention program for cognitive modifiability. Baltimore, MD: University Park Press. Fletcher-Flinn, C.M., & Gravatt, B. (1995). The efficacy of computer assisted instruction (CAl): A meta-analysis. Journal ofEducational Computing Research, 12,219-242. Gordon, L. V. (1982). Manuel de I'inventaire de personnalite GPP-I. Paris: Editions du Centre de Psychologie Appliquee.

456

D. CHARTIER

Higele, P. (1992). Evaluation des effets de transfert des ateliers de raisonnement logique. In F. Ginsbourger, V. Merle, & G. Vergnaud (Eds.), Formation et apprentissage des adultes peu qualifies (pp. 122-124). Paris: La Documentation Francaise. Higele, P., & Martin, B. (1979). Une experience d'apprentissage d'operations intellectuelles. Revue Francoise de Pedagogie, 46, 16-29. Higele, P., Hommage, G., & Perry, E. (1991). Ateliers de Raisonnement Logique. Livret duformateur. Nancy: CAFOC. Hornemann, J. (1974). Echelle Collective de Developpement Logique. Issy les Moulineaux: Editions des Applications Psychotechniques. Howell, R.D., Scott, P.B., & Diamond, J. (1987). The effects of "instant" Logo computing language on the cognitive development of very young children. Journal ofEducational Computing Research, 3,249-260. Huteau, M., & Loarer, E. (1992). Comment evaluer les methodes d'educabilite cognitive? L 'Orientation Scolaire et Professionnelle, 21,47-74.

a

Huteau, M., Lautrey, I., Chartier, D., & Loarer, E. (1994). Apprendre apprendre... la question de I'educabilite cognitive. In G.Yergnaud (Ed.), Apprentissages et didactiques, ou en est-on? (pp. 151-178). Paris: Hachette. Loarer, E., & Lautrey, J. (1994). Une epreuve de planification de l'activite: "le test des camions". In M.Huteau (Ed.), Les Techniques Psychologiques d'Evaluation des Personnes. Issy les Moulineaux: Editions des Applications Psychotechniques. Loarer, E., Chartier, D., Huteau, M., & Lautrey, J. (1995). Peut-on eduquer !'intelligence? L 'evaluation d'une methode d'education cognitive. Berne: Peter Lang. Loarer, E., Libert, M.F., Chartier, D., Huteau, M., & Lautrey, J. (1991). L 'evaluation du programme d'enrichissement instrumental de Feuerstein (P.E.!.) dans les stages prequalifiants de I 'A.F.P.A. Rapport au Ministere de Ia Recherche. Document roneote,

Morseau, J.D. (1991). La pratique de Logo en educabilite cognitive. In M.Sorel (Ed.), Pratiques nouvelles en education et en formation. L 'educabilite cognitive (pp. 173-186). Paris: L'Harmattan. Mugny, G. (Ed.) (1985). Psychologie sociale du developpement cognitif. Berne: Peter Lang.

Noss, R. (1987). Children's learning of geometrical concepts through Logo. Journal for Research in Mathematics Education, 18,343-362. Oltman, P.K., Raskin, E., & Witkin, H.A. (1971/1985). Manuel du Test des Figures Encastrees . Paris: Les Editions du Centre de Psychologie Appliquee. Paour, Ll., (1992). Induction oflogic structures in the mentally retarded: An assessment and intervention instrument. In H.C. Haywood & D.Tzuriel (Eds.), Interactive assessment (pp. 119-166). New York: Springer Verlag. Paour, Ll.; Jaume, J., & de Robillard, O. (1995). De I'evaluation dynamique a l'education cognitive: reperes et questions. In F.P. Buchel (Ed.), L 'education cognitive. Le developpement de la capacite d'apprentissage et son evaluation (pp. 47-102). Neuchatel: DeJachaux et Niestle. Papert, S. (1981). Jaillissement de I 'esprit. Paris: Flammarion.

Perret-Clermont, A.-N. (1979). La construction de l'intelligence dans l'interaction sociale. Berne: Peter Lang. Reuchlin, M. (1977). Epreuves d'hypotheses nulles et inference fiduciaire en Psychologic. Journal de Psychologie, 3, 277:292. Reuchlin, M. (1992). Introduction

ala recherche en Psychologie.

Paris: Nathan.

Reuchlin, M" & Valin, E. (1953). Tests collectifs du Centre de Recherches, Bulletin de l 'Institut National d'Orientation Professionnelle, 3, 1-152. Rotter, J.B. (1966). Generalized expectancies for internal versus external control of reinforcement. Psychological Monographs, 80, Whole n° 609. Salomon, G., & Perkins, D.N. (1987). Transfer of cognitive skills from programming: when and how? Journal of Educational Computing Research, 3, 149-169. Stevens, J. (1992). Applied multivariate statistics for the social sciences. 2nd ed. Hillsdale, NJ: Erlbaum.

457

LOGO AND ARL IN ADULTS' TRAINING

Thurstone, L.L. (1959). Manuel du Test des Figures ldentiques. Paris: Editions du Centre de Psychologie Appliquee. Yelland, N. (1993). Young children learning with Logo: An analysis of strategies and interactions. Journal of Educational Computing Research, 9,465-486.

Deux groupes d'adultes en formation pre-qualifiante a l'AFPA ont beneficie chacun d'un programme different, explicitement destine a ameliorer leurs capacites cognitives: les ateliers de raisonnement logique pour l'un et les exercices LOGO pour l'autre. lis ont ete compares a un groupe temoin ayant recu, pendant une duree sensiblement analogue, une formation a l'utilisation de logiciels de bureautique et d'enseignement assiste par ordinateur. Les trois groupes ont ete testes, en debut et en fin de stage, sur un certain nombre de dimensions cognitives. Le niveau d'estime de soi des stagiaires, leur atitude a l'egard de l'informatique, ainsi que l'internalite de leurs attributions ont egalement ete evalues. Les resultats montrent: 1) une augmentation modeste de l'efjicience cognitive de chaque groupe a l'issue de la formation et une evolution ambigiie des variables conatives; 2) aucune superiorite du groupe LOGO par rapport au groupe temoin; 3) une superiorite du groupe ARL pour deux variables cognitives sur les dix etudiees (combinatoire et independance a l'egard du champ perceptif). Ces resultats semblent insufjisants pour conclure a une efficacite specifique de cette methode sur une periode de temps correspondant aux stages de type

preparatoire.

Key words. Adults' training; Logical Thinking Workshops; Logo programming; Teaching of thinking skills.

Received: January 1995 Revision received: January 1996

Daniel Chartier, A.F.P.A, Departement Etudes et Methodes en Psychologie du Travail, 13, place du Gal De Gaulle, F-93108 Montreuil Cedex, France. Current theme ofresearch:

Interaction of cognitive and motivational factors in adults' learning. Job analysis applied to recruitement and training. Most relevant publications in the field ofEducational Psychology:

Chartier, D., & Lautrey, J. (1992). Peut-on apprendre it connaitre et it controler son propre fonctionnement cognitif? L 'Orientation Scolaire et Professionnelle, 21,27-46. Huteau, M., Lautrey, J., Chartier, D., & Loarer, E. (1994). Apprendre it apprendre... La question de l'educabilite cognitive. In G.Vergnaud (Ed.), Apprentissages et didactiques, OU en est-on? (pp. 151-178). Paris: Hachette. Loarer, E., Chartier, D., Huteau, M., & Lautrey, J. (1995). Peut-on eduquer l'intelligence? L 'evaluation d'une methode d'education cognitive. Berne: Peter Lang.