Integr Psych Behav (2007) 41:28–34 DOI 10.1007/s12124-007-9008-9 C O M M E N TA R I E S
Psychology is About Processes Stellan Ohlsson
Published online: 18 July 2007 # Springer Science + Business Media, LLC 2007
Abstract Out of the eight points of methodological criticism against contemporary psychology formulated by Watson (Psychological Bulletin 31:755–776, 1934) and put forward by Toomela in this issue, the overemphasis on prediction, the neglect of individual differences, the habit of the differences between the mental states of subjects in objective experimental conditions are particularly important. Modern cognitive psychology has began to remedy those problems, in part by proposing broad, integrative theories. It is not useful to subdivide psychology into “schools of thought” defined by their methodological practices. Keywords Cognitive architecture . Explanation . Individual differences . Methodology . Prediction . Process theories . Statistical inference . Typology Toomela’s indictment of contemporary mainstream psychology on eight counts of methodological error is an opportunity to step back from individual studies and engage in meta-level reflection on our investigatory practices. As Toomela documents, contemporary psychology journals do indeed publish numerous articles that illustrate the eight criticisms from Watson’s (1934) article. I find particular value in the observation that what Toomela calls “physical controls”—experimental conditions that are defined in terms of stimulus displays, tasks, training procedures, feedback regimens, and the like—do not guarantee that all the experimental participants are in the same mental state and that their behaviors therefore fall under the same explanation (Toomela’s point number 2). This is a very real concern in, for example, studies of skill acquisition. It is of some practical importance and considerable theoretical interest to determine how a coach, tutor, or a computer-based training system can best help a learner acquire a new skill. The S. Ohlsson (*) Department of Psychology, University of Illinois at Chicago, BSB 1050C, M/C 285, Chicago, IL 60607, USA e-mail:
[email protected]
Integr Psych Behav (2007) 41:28–34
29
standard experimental approach to this problem is to define different feedback conditions in terms of their values on such variables as the type of feedback (positive or negative), the rate at which the feedback is given, the exact content of the feedback messages, and so on. The outcome of interest is the relative post-training performance of group that learned under different conditions. (My own research team has carried out multiple studies of this sort; e.g., Corrigan-Halpern and Ohlsson 2004.) However, this approach is problematic because the exact feedback a learner encounters is always a function of his or her performance, which in turn is a function of prior knowledge, cognitive ability, motivation, and other factors. A smart, knowledgeable and motivated learner will commit fewer errors during training than his or her opposite, and hence draw more positive feedback from the training environment than the latter, regardless of experimental condition. Feedback regimen is not an independent variable. The relative effectiveness of objectively defined feedback regimens could only be determined by comparing groups if all participants were in the same mental state at the outset of training, but they are not. This severely limits the conclusions that can be drawn from the hundreds of experimental studies that systematically vary feedback conditions in order to determine their effects on learning. Exactly how to overcome this methodological conundrum is at this time an open question, but partitioning data sets by the participants_ values on auxiliary variables is a useful technique (e.g., Nokes and Ohlsson 2005). Another research topic that is affected by this problem and by the closely related problem of focusing exclusively on group means without careful qualitative descriptions of individual participants (points number 1 and 4) is insight problem solving. Researchers tend to classify certain problems as “insight problems” while others are “non-insight problems.” (This is true of my own past work as well as the work of others; e.g., Kershaw and Ohlsson 2004) But being an insight problem is not an objective property of a problem, analogous to the way in which height is a property of a building. Whether a problem is an insight problem could only be decided once and for all if every person were guaranteed to be in the same mental state when they encounter the problem. Instead, any given problem is an insight problem for a particular person, if that person's background knowledge is such that the problem leads him or her into an unproductive initial approach and thus forces him or her to restructure the representation of the problem before the solution can be found. For example, Maier_s famous Two-String Problem requires the problem solver to make a pendulum out of a hanging rope by tying a heavy object to its end —an action that is hard to think of for many but which might nevertheless be easy to think of for someone who restores old pendulum clocks or watches old Tarzan movies. In most studies, the researchers do not demonstrate that the problems referred to as insight problems actually require insight on the part of the participants. The result is that the set of behaviors observed in such experiments is likely to be heterogeneous within each experimental condition: some solutions having come about via insight and others not, throwing noise into the data and confusion into theoreticians' efforts to explain all those behaviors with one and the same theory. The solution in this case is to do what Toomela recommends: to collect qualitative data on each participant so as to be able to group the participants into subgroups in which the members’ knowledge states are similar in the relevant respects (e.g., prior
30
Integr Psych Behav (2007) 41:28–34
experience with pendulums and ropes). This makes such studies considerably more laborious to perform. Two other points in Watson’s list of methodological ills that I find particularly important are the neglect of individual differences and the overemphasis on prediction. The study of cognition—or the intellect, as it was called before the 1950s—has developed along two independent lines of research, cognitive psychology and psychometrics, dealing with group averages and individual test scores respectively (Cronbach 1957). But this division is not a fruitful division of labor. In every experimental study performed in my laboratory, the differences among individuals within each experimental group are typically larger than the differences among the group means. To the extent that variability is information, experimental psychologists throw away most of the information generated by their experimental procedures by sweeping interindividual variation under the conceptual rug as “error variance.” School science often emphasizes prediction, both because the ability to predict events is an impressive achievement and because prediction is possible in mechanics, often the first serious science that students encounter. Nevertheless, prediction is not typical of the successful sciences. An evolutionary biologist cannot predict which species will evolve next, and for a chemist to predict the properties of a substance ahead of synthesizing the relevant molecule remains a hard task. Against this background, the oft-repeated statement that the goal of psychology is to be able to “predict behavior” is not sensible. If we could do as well as biologists and chemists, we would do well indeed. Toomela’s critique would have been more useful if he had dug deeper than Watson and tried to diagnose the root cause of the eight weaknesses rather than merely repeating them. This task cannot be undertaken within the boundaries of this commentary, but let me suggest that one cause is the overuse of statistical significance tests as the arbiter for whether the result of an experiment is interesting and publishable. The widespread adoption of this technique forces studies into a straightjacket. For the technique to be applicable, a study has to be cast as a comparison between groups, the groups have to be defined in terms of objective conditions (“physical controls”) and the researcher must be willing to pretend that individual differences are noise in the data. The result—whether there is a significant difference—is seldom interesting unless the researcher claimed to be able to predict which way the comparison would fall out ahead of time. Comparisons are often useful and interesting, but it is not the only type of scientific study. I find studies that identify patterns in behavior—particularly patterns over time—more informative and thought provoking than group comparisons. As examples, consider the insight sequence (Ohlsson 1992), the power law learning curve (Ohlsson and Jewett 1997), multiple overlapping waves (Siegler 1996, p. 89) and the rise and decline in cognitive speed with age (Salthouse 1996). Each of these patterns ultimately has to be explained, and explaining them is a difficult theoretical task. Explaining all of them at once is more difficult still; in a forthcoming book (Ohlsson 2007) I make an attempt in that direction. The point here is that the eight weaknesses are not mere mistakes or oversights, nor do they spring directly from any well-articulated philosophical outlook. Instead, each is understandable as a side effect of the overuse of a particular type of statistical analysis. A healthier science would recognize many
Integr Psych Behav (2007) 41:28–34
31
different types of studies as worthwhile contributions, a stance that we might call methodological pluralism. Although I am in agreement with Watson and Toomela’s critique regarding the focus on comparisons between group means, the reliance on physical specifications of experimental conditions, the overemphasis on prediction and the systematic neglect of individual differences, I do not agree with the remaining points. I also disagree with Toomela’s sweeping claim that “the Zeitgeist of contemporary mainstream Anglo-American psychology can be characterized as being very similar to the pre-WWII North American psychology,” that is, to the behaviorist research tradition that dominated experimental psychology up to World War II. To explain this difference in judgment, I need to introduce two distinctions that are missing from Toomela's paper. The first is the difference between different fields of research; they do not all develop in lock step. Toomela makes all eight points appear as features of “contemporary mainstream psychology” by zooming out to such a broad perspective that he can select his examples from such diverse areas as clinical psychology, cognitive psychology, and psychometrics. But the basic research questions of these areas differ and so do their methodologies. As a result, there is a certain sociological separation. The people in one area do not interact much with people in another, a natural consequence of specialization, which in turn becomes necessary as a science grows in complexity and size. As a side effect, the different areas do not develop at the same rate or in the same way. Toomela’s perspective is so broad that he overlooks such differences. A second distinction that is missing from Toomela’s paper is the distance between the average researcher in a field and its avant garde. There are still psychologists who believe that all of human behavior can be accounted for by Pavlov’s conditioned reflexes. This is an extreme example, but the point is general: Every scientific advance leaves some researchers behind, but those researchers have tenure and continue to publish. The result is that one can dip into the literature of almost any field and retrieve current examples of regressive and backward statements. It is more interesting to observe what happens at the leading edge. Modern cognitive psychology—the area which I know best and is hence most comfortable discussing at this meta-level—have adopted methodological practices that address most of the eight criticisms. For example, qualitative descriptions derived from verbal protocols and other types of detailed traces that reveal the strategies of individual participants are routinely used in cognitive psychology. Indeed, case studies, either of average people or of exceptional people, are, if not common, nevertheless an accepted and recurring feature of empirical work in modern cognitive psychology. A significant proportion of empirical studies are not cast as comparisons between groups, and take individual differences into account. This is illustrated by the fact that when Toomela wants to document his first point, he cites Robert Siegler, a leading representative of contemporary North American mainstream psychology! Furthermore, cognitive psychologists recognize the need for deep explanatory theories, and such theories have been forthcoming. In 1973, Allen Newell wrote a critique of experimental cognitive psychology along the same theme as Toomela’s point 7, about the need for a systematic approach to theory building (Newell 1973). Since then, cognitive theories with broad scope—sometimes referred to as theories
32
Integr Psych Behav (2007) 41:28–34
of the cognitive architecture—have been proposed by Allen Newell himself (Newell 1990), John R. Anderson (Anderson and Lebiere 1998), Ron Sun (Sun et al. 2005), and others. In short, the claim that “psychology” suffers form these eight weaknesses is too broad and unspecific to be useful. Each field within psychology exhibits a different profile with respect to these eight weaknesses, and the leading elements of modern cognitive psychology have gradually altered their investigatory practices to address them. The claim that contemporary psychology does not differ from pre-WWII research is false with respect to cognitive psychology. To his credit, Toomela not only criticizes but also proposes alternatives. In particular, he outlines the kind of approach that he thinks would cure psychology of its fragmented approach and its neglect of individual differences. According to Toomela, we should return to the pre-WWII program of developing a typology of individuals, a set of classes or categories in which to put people. I see no value in this proposal. First, I am interested in understanding the mind, but I do not see how a typology of persons provides any explanatory power. For example, consider the task of explaining why someone performed a task in a particular way, or learned (or failed to learn) such-and-such a subject matter, or made such-and-such a mistake in operating a piece of machinery, all of which are of some importance in cognitive psychology. To explain any such event or phenomenon by saying that “person P behaved like B in situation S, because P is a T-type person” seems to me devoid of interest and to provide no insight or understanding. I also wonder why Toomela believes that such typology is possible. Researchers in the field of personality have long sought to formulate such a theory in terms of personality traits, but they have recently moved away from this goal. Instead, they seek to explain how so-called traits arise out of the interactions among more basic mental processes and structures (Cervone 2005). It seems to me that individuals potentially differ along so many dimensions (working memory capacity, threshold for long-term memory retrieval, time to encode a new memory element, disposition to act on negative feedback, etc.) that the space of possible types will be larger than the number of living persons available to populate the typology. Instead of returning to pre-WWII psychology in the literal sense of digging up the corpse of Kretschmer’s (1925) research program, I suggest that we look to what has been accomplished since then. I believe that the last 50 years of research has revealed that psychology is a science of mental processes. The purpose of a psychological explanation is to show how particular behaviors and regularities come about. This is accomplished by (a) hypothesizing basic processes, i.e., processes that are so simple that we feel no need to break them down into yet simpler components, (b) specifying how those basic processes combine to form processes of larger scope, and (c) demonstrating how those larger processes generate behavioral patterns. As this enterprise continues, the repertoire of basic processes that we need to postulate to understand how the mind works (encoding, retrieval, spread of activation, and so on) will presumably converge on a small standard set, and the rules of combination should eventually crystallize into a stable set. The result will be an explanatory (but necessarily predictive) theory of human cognition. Modern cognitive psychology has made great strides along this path. Given this background, I strongly disagree with Toomela’s points 3, 7, and 8. The shared theme of those points is the need to build systematic theories of broad scope
Integr Psych Behav (2007) 41:28–34
33
within which individual facts can find a place. But the claim that contemporary cognitive psychology does not construct such theories is a caricature. Worse, Toomela’s point 8, which claims that contemporary North American psychologists do not think (while German-inspired ones do), is an ad hominem claim—an insult, to those of us without Latin—which no serious scholar would want to include in a paper and no respectable scientific journal would want to publish. However, the most destructive feature of Toomela’s article is that he falls for the perennial temptation to divide psychology into “schools of thought” based on their approaches or methodological practices, in his case “Austrian-German” versus “North American.” This is a bad old European tradition that indeed extends all the way back to the pre-WWII era and the sooner we abandon it, the better. Methods are tools, not articles of faith. The only productive attitude is that the more tools we have, the better. Every tool comes in handy sometime. Other sciences tend to organize themselves around problems, questions and phenomena (organic chemistry, particle physics, evolutionary biology, crystallography, etc.) and each field uses every method, type data and style of analysis that can help throw light on its substantive issues. I think psychology would likewise benefit from doing away with methodologically defined “schools.” From this point of view, a return to pre-World War II psychology is precisely what we should avoid and another article like Toomela’s is precisely what we do not need.
References Anderson, J. R., & Lebiere, C. (1998). The atomic components of thought. Mahwah, NJ: Erlbaum. Cervone, D. (2005). Personality architecture: Within-person structures and processes. Annual Review of Psychology, 56, 423–452. Corrigan-Halpern, A., & Ohlsson, S. (2004). Feedback effects in the acquisition of a hierarchical skill. In K. Forbus, D. Gentner, & T. Regier (Eds.), Proceedings of the 26th Annual Conference of the Cognitive Science Society (p. 1646). Cronbach, L. J. (1957). The two disciplines of scientific psychology. American Psychologist, 12, 671–684. Kershaw, T., & Ohlsson, S. (2004). Multiple causes of difficulty in insight: The case of the nine-dot problem. Journal of Experimental Psychology. Learning, Memory, and Cognition, 30(1), 3–13. Kretschmer, E. (1925). Physique and character. New York: Harcourt Brace. Newell, A. (1973). You can’t play 20 questions with nature and win. In W. G. Chase (Ed.), Visual information processing (pp. 283–308). New York: Academic Press. Newell, A. (1990). Unified theories of cognition. Cambridge, MA: Harvard University Press. Nokes, T., & Ohlsson, S. (2005). Comparing multiple paths to mastery: What is learned? Cognitive Science, 29, 769–796. Ohlsson, S. (1992). Information processing explanations of insight and related phenomena. In M. Keane & K. Gilhooly (Eds.), Advances in the psychology of thinking (Vol. 1, pp. 1–44). London, UK: Harvester-Wheatsheaf. Ohlsson, S. (2007). Deep learning: How the mind overrides experience. Cambridge, MA: Cambridge University Press (in press). Ohlsson, S., & Jewett, J. J. (1997). Ideal adaptive agents and the learning curve. In J. Brzezinski, B. Krause, & T. Maruszewski (Eds.), Idealization VIII: Modelling in psychology (pp. 139–176). Amsterdam, The Netherlands: Rodopi. Salthouse, T. A. (1996). The processing-speed theory of adult age differences in cognition. Psychology Review, 103, 403–428. Siegler, R. S. (1996). Emerging minds: The process of change in children’s thinking. New York: Oxford University Press.
34
Integr Psych Behav (2007) 41:28–34
Sun, R., Slusarz, P., & Terry, C. (2005). The interaction of the explicit and the implicit in skill learning: A dual-process approach. Psychological Review, 112, 159–192. Toomela (2007). Culture of science: Strange history of the methodological thinking in psychology. Integrative Psychological and Behavioral Science (doi:10.1007/s12124-007-9004-0). Watson, G. (1934). Psychology in German and Austria. Psychological Bulletin, 31, 755–776.
Stellan Ohlsson is Professor of Psychology and Adjunct Professor of Computer Science at the University of Illinois at Chicago (UIC), Chicago, Illinois, USA, since 1996. He received his Ph.D. in Psychology at the University in Stockholm, Sweden, in 1980. He has since held academic appointments in Sweden, Australia and the USA. He was Senior Scientist at the Learning Research and Development Center (LRDC) in Pittsburgh 1990–1995. Dr. Ohlsson is currently completing a book length integration of his research in the areas of creativity, skill acquisition and conceptual change, to be published by Cambridge University Press under the title Deep Learning: How the Mind Overrides Experience.