Atten Percept Psychophys (2011) 73:2168–2179 DOI 10.3758/s13414-011-0169-8

Tracking objects that move where they are headed Nicole L. Jardine & Adriane E. Seiffert

Published online: 16 July 2011 # Psychonomic Society, Inc. 2011

Abstract Previous work has demonstrated that the ability to keep track of moving objects is improved when the objects have unique visual features, such as color or shape. In the present study, we investigated how orientation information is used during the tracking of objects. Orientation is an interesting feature to explore in moving objects because it is directional and is often informative of the direction of motion. Most objects move forward, in the direction they are oriented. In the present experiments, participants tracked a subset of moving isosceles triangles whose orientations were constant, related, or unrelated to the direction of motion. In the standard multiple object tracking (MOT) task, tracking performance improved when orientations were unique and remained constant, but not when orientation and direction of motion were aligned. In the target recovery task, in which MOT was interrupted by a brief blanking of the display, performance did improve when orientation and direction were aligned. In the final experiment, results showed that orientation was not used before the blank to predict future target locations, but was instead used after the blank. We concluded that people use orientation to compare a stored representation to target position for recovery of lost targets. Keywords Orientation . Motion . Multiple object tracking . Target recovery

N. L. Jardine : A. E. Seiffert (*) Department of Psychology, Vanderbilt University, Nashville, TN 37240, USA e-mail: [email protected] N. L. Jardine e-mail: [email protected]

To successfully navigate a dynamic and complex world, people must often attend to and keep track of multiple objects moving about the environment. Without the ability to monitor many things at once, it would be difficult for a cyclist to find his or her way along a road while attending to passing cars, traffic lights, and pedestrians. To study this capacity, Pylyshyn and Storm (1988) devised the multiple object tracking (MOT) paradigm that has since inspired a body of literature on attentional tracking (for a review, see Cavanagh & Alvarez, 2005). In these studies, participants attempted to keep track of a few target dots that bounced around inside a box among identical distractor dots. Much investigation has already documented some of the key factors that influence tracking ability, such as motion of the dots, proximity of targets to distractors and attentional load (Cavanagh & Alvarez, 2005). The goal behind much of this research is to understand how people use visual information to successfully maintain accurate representations of target locations and differentiate targets from distractors. One way to address this general goal is to determine the types of information that people use to track objects. Some research has demonstrated that people can use surface features such as color and shape to improve tracking performance. Unique surface features can help discriminate objects from one another and effectively reduce crowding, or be used to search for and find targets if they are lost during tracking (Horowitz, Klieger, Fencsik, Yang, Alvarez, & Wolfe, 2007; Makovski & Jiang, 2009; Oksama & Hyönä, 2004). These results support a two-process model of object tracking that includes: (1) a memory-based process that stores information such as location of targets, motion of targets, assignment of objects to targets or distractors, and possibly other features of these objects, and (2) a spatiotemporal correspondence process that handles the moment-to-moment perception of objects as

Atten Percept Psychophys (2011) 73:2168–2179

they move, allowing for representations of objects to sustain continuity with the visual input from the appropriate object in the world (Horowitz et al., 2007; Tombu & Seiffert, 2011). Several existing models of tracking include components similar to these, with variants including a role for visual attention (Cavanagh & Alvarez, 2005; d’Avossa, Shulman, Snyder, & Corbetta, 2006; Oksama & Hyönä, 2008), although others propose other mechanisms (Pylyshyn, 2001; Vul, Frank, Tenenbaum, & Alvarez, 2009). Results showing improved tracking performance from unique surface features suggest that features support the memory-based process, but not the spatiotemporal correspondence process. To expand on this approach, we investigated how orientation could affect tracking performance by affecting either memory or spatiotemporal correspondence. There are four different ways that orientation could facilitate tracking. First, when targets have unique features, such as different orientations, people could use them to facilitate the memory-based process. If the surface features of the targets were stored in memory, that information could be used to search for targets when they are lost (Horowitz et al., 2007; Makovski & Jiang, 2009). Second, similar to colors or shapes, objects with unique orientations could be easier to discriminate from one another, improving perceptual parsing. This would improve the spatiotemporal correspondence process and reduce the crowding that has been shown to impair tracking performance (Tombu & Seiffert, 2008; though see Bettencourt & Somers, 2009). Third, unlike color or shape, orientation is directional. Most objects in the natural world move in the same direction they are oriented. Orientation might help predict future positions of targets, which may also guide the spatiotemporal correspondence process. Fourth, orientation may improve the memorybased process even if the only information in memory is target location (Oksama & Hyönä, 2008), if orientation could be used to disambiguate potential alternatives on the basis of the assumption that orientation indicates trajectory. For example, two objects equidistant from a remembered target location may be equally likely to be a target according to location alone, but the target could be correctly identified if its orientation aligned with the remembered location. We will refer to these four ways that orientation could have an impact on tracking performance as feature search, perceptual parsing, position prediction, and disambiguation. We manipulated the relationship between orientation and motion of tracked objects to determine the ways in which orientation contributes the online maintenance of representations of moving targets. Previous research exploring the interaction between orientation and motion suggests that orientation can have an impact on the perception of motion direction. Morikawa (1999) tested the perception of a linearly translating, symmetrical object that was aligned or misaligned with its

2169

trajectory. When misaligned, participants’ trajectory judgments were systematically biased toward the angle of orientation. When an object was misaligned with its trajectory by 15 degrees to the right, for example, the judgment was biased by about 10 degrees to the right from the true trajectory. Other research has similarly demonstrated that direction judgments can be biased by orientation (Freyd & Finke, 1984; McBeath, Morikawa, & Kaiser, 1992; Palmer, 1980; Werkhoven, Snippe, & Koenderink, 1990;). Freyd (1983) demonstrated that just the structure of an object can imply motion that makes people perceive the last position of a moving object to be displaced in the direction implied by the orientation. Vinson and Reed (2002) found that pointed objects were perceived to be shifted more in their direction of motion than were blunt objects. Freyd and Pantzer (1995) have suggested that the implied directionality of pointed objects distorts the memory of the object’s positions. Orientation, then, seems to be unique as a surface property that may interact with motion information and influence the perception of an object’s trajectory. In the present series of experiments, we investigated how the orientation of tracked objects might affect tracking performance. In the first two experiments, we investigated the effects of orientation on the standard MOT task. To anticipate, results were consistent with the idea that orientation was used for feature search and perceptual parsing, but not position prediction or disambiguation. In the remaining experiments, we investigated the use of orientation in the target recovery (TR) task, which is the same as the MOT task except there is a momentary blank inserted into the trial. The TR task was employed to further investigate the use of memory, since memory is required to sustain representations during the blank. Results show that orientation can be used to support accurate localization of the targets after the blank, and relating orientation and motion direction has a unique benefit.

Experiment 1: orientation effects in MOT In the first experiment, we investigated how the reliability of orientation information affects performance in the MOT task. Participants were asked to keep track of targets bouncing around inside a box with distractors. As is illustrated in Fig. 1, we created displays of isosceles triangles whose orientations throughout the trial were all the same (hereafter called the same condition), were different for each object and consistently unique (unique) were different but changing randomly at each bounce (random change), or were changing at each bounce because they were aligned with object trajectories (aligned). If orientation is useful because it allows lost targets to be found with feature search, then performance should be better when orientation is consistently unique (unique) than

2170

Atten Percept Psychophys (2011) 73:2168–2179

Fig. 1 An illustration of the orientation manipulation in Experiment 1. Arrows show the direction of motion relative to the orientation of the triangles and were not shown in the displays. a In the same condition, all triangles were always upright orientation regardless of their direction

of motion. b In the unique and random change conditions, triangles were each different orientations and were unrelated to their direction of motion. c In the aligned condition, triangles always pointed in the direction that they moved

when it is not unique (same) or changing (random change and aligned). If orientation is also useful because it helps perceptual parsing of objects from one another and reduces crowding, thus improving spatial correspondence, then performance should be better whenever the orientations of objects are different from one another (unique, random change, and aligned) rather than when they are all the same. Finally, if orientation is useful when it conveys trajectory information by improving position prediction or disambiguating alternatives, then performance should be better when orientations align with trajectories (aligned) than when they do not (all other conditions). Note that all or any subset of these influences of orientation could occur simultaneously. We also included a condition in which participants tracked dots instead of triangles to determine whether triangles, per se, were difficult or easy to track and so that the results could be more easily related to previous work.

viewing distance was not constrained, measurements of degrees visual angle are approximate. Triangles had a height of 1.3º and width 0.6º of visual angle (hereafter º), and dots had a diameter of 0.72º, so that the areas of the dots and triangles were similar (16.1 and 16.3 deg2, respectively). Black triangles and dots were shown on a white screen within a 21º × 21º black-outlined box. Objects moved linearly and independently at 7.4º/s, could occlude each other, and bounced off the walls of the box. Objects bounced by immediately moving in a direction away from the wall that was a reflection with an added random jitter of 0 to ±22.5 degrees. All of the triangles in each display were oriented with respect to the condition. In the same condition, all of the triangles had the same orientation—upright—and retained that orientation throughout the trial. In the unique condition, each triangle was given a unique orientation that was randomly chosen at the beginning and was retained throughout the trial. In the random change condition, each triangle was given a unique orientation at the beginning of the trial and then changed orientation randomly at each wall bounce. Random changes were not constrained so it was possible, but very unlikely, for two triangles to have the same orientation after bouncing. Finally, in the aligned condition, triangles always were oriented along their direction of motion, changing at each wall bounce accordingly. Directions were not constrained, so it was possible, but very unlikely, for two triangles to have the same orientation and moving in the same direction.

Method Participants Thirty-six adults from the Vanderbilt University campus and Nashville community gave informed and written consent as approved by the Vanderbilt University Institutional Review Board. Each received $10 for an hour of participation. Apparatus and stimuli Stimuli were programmed in MATLAB using the Psychophysics Toolbox (Brainard, 1997; Pelli, 1997) and were displayed on an Apple eMac PowerPC G4. The integrated CRT received a 1,024 × 768 resolution signal with 32-bit color at 89 Hz from an ATI Radeon 9600 dedicated graphics processor. Participants were seated in a testing room approximately 57 cm from the monitor. Participants were encouraged to remain in the same position throughout the experiment; however, because

Procedure In this experiment, participants engaged in the MOT task. Each trial began with a static array of 10 objects randomly placed within an invisible 5 × 5 matrix of locations with some random jitter, so that no objects overlapped. The four targets were simultaneously cued with surrounding red circles for 2 s. After the cues disappeared, the display

Atten Percept Psychophys (2011) 73:2168–2179

remained static for 1 s. Then, all objects moved in straight paths and bounced off the walls of the box for 5.6 s. In the conditions in which orientations changed, orientation was immediately updated when the object changed direction. At the end of the trial, all objects remained in their final locations and orientations while participants attempted to select the targets. Participants used the mouse to point the cursor and pressed the spacebar to select objects. Feedback was presented after each selection by playing a high tone if the selected object was a target and a low tone if not. Participants first performed a practice session of five trials (one for each condition), then completed four blocks of five repetitions per condition, for a total of 20 data points per condition, or 100 trials. Conditions were randomly mixed within each block. Accuracy was calculated as the percentage of correctly selected targets within trials. Results and discussion Mean tracking accuracy as a function of condition is graphed in Fig. 2. A one-way repeated measures ANOVA indicated that condition (dots, same, unique, random change, and aligned) significantly affected tracking performance, F(4, 136) = 5.89, p < .001. To describe the results further, we compared conditions to one another using t tests. Because there were eight t tests that were theoretically motivated, we adjusted the alpha criterion for significance to be 0.00625 by using Bonferroni correction. First, tracking dots did not produce significantly different performance than did tracking triangles averaged over orientation conditions, t(35) < 1 , ns. Second, when the orientations of objects were always the same, performance was the worst. The same condition produced worse performance than dots, but this comparison did not pass the statistical test adjusted for multiple comparisons, t(35) =

2171

2.80, p = .008. Performance in the same condition was significantly worse than that in the unique condition, t(35) = 4.23, p < .001, providing evidence to support the idea that orientation information can be used to facilitate object tracking when it provides uniquely identifying information, as has been shown in previous work using other features (Horowitz et al., 2007; Makovski & Jiang, 2009). This result could stem from either the benefits of perceptual parsing or feature search on tracking performance. A perceptual parsing account would predict benefits in the random change and aligned conditions as well, because items had different orientations. Although the means showed these trends, results did not pass the adjusted test when comparing the same condition to the random change condition, t(35) = 2.68, p = .02, and aligned condition, t(35) = 2.31, p = .03. A feature search account would predict better performance when orientation was consistent and unique over those in which orientation changed. Although the means showed these trends, results did not pass the adjusted test when comparing unique to random change, t(35) = 2.38, p = .03, and unique to aligned, t(35) = 2.53, p = .02. It is possible that the unique condition showed the only significant effect because weak benefits from perceptual parsing and feature search were combined. As such, this experiment provided only weak evidence for the idea that orientation improves performance because a memory-based process stores the orientation of targets. We addressed this idea with a different paradigm in Experiment 3. Finally, when triangles changed orientation throughout the trial, there were no differences in performance between triangles aligned with their trajectories and triangles that were not aligned (aligned vs. random change, t(35) < 1, ns). As such, there is no evidence from this experiment to support the idea that orientation interacts with motion to facilitate position prediction and improve tracking. Although the evidence is weak, these results are most consistent with the idea that orientation is used similarly to other surface features and may affect tracking performance through a memory-based system, but not through spatiotemporal correspondence.

Experiment 2: misalignment effects in MOT

Fig. 2 Results of Experiment 1. Percentage of correctly selected targets as a function of condition. Error bars show standard errors of the means

The results of the previous experiment showed a benefit of different orientations, but no benefit of aligning orientation with the direction of motion when compared with a condition in which orientation changed randomly. Experiment 2 was designed to test the possibility that a deliberate offset of orientation may produce performance decrement. A consistent misalignment of orientation and motion can produce an error in trajectory perception (Morikawa, 1999), which could impair position prediction. Aligned orientations may have information that can be directly used to interpret motion

2172 Fig. 3 An illustration of the orientation manipulation in Experiment 2. Arrows show the direction of motion relative to the orientation of the triangle (dashed line) in the three conditions

trajectory, whereas misaligned orientations may be misleading. In Experiment 2, we tested MOT performance using triangles that were aligned or misaligned to their trajectories. In all trial types, orientation maintained a consistent relationship with motion. For misaligned triangles, however, orientation and motion were consistently offset. If misalignment interferes with perception of motion trajectory or the processing of target motion in some other way, then tracking performance should be better when triangles are aligned than when they are misaligned. To increase the likelihood of finding an alignment effect, we tested three orientation conditions across a range of tracking loads. Method The apparatus, stimuli, and display were identical to those in Experiment 1, with exceptions as described below. Thirteen undergraduate students from the Vanderbilt University campus participanted for partial course credit. Stimuli Triangles were white on a black background. Triangles were oriented relative to their direction of motion, depending on the condition (Fig. 3). Aligned triangles always were oriented along their direction of motion, as in Experiment 1. Misaligned 15° triangles were oriented with a 15 degrees offset relative to their trajectories. We chose to test this offset because previous work demonstrated that an misalignment of 15 degrees is enough to produce a misperception of trajectory (Morikawa, 1999). Degree and direction of misalignment was constant for each object throughout each trial, so that in both aligned and misaligned conditions, the orientation of each object had a consistent relationship to the object trajectory. Because of practical constraints, we were unable to include all possible misalignments that might have had an effect as separate factors in the experiment. Instead, we tested a third condition—misaligned random—in which all triangles had one misalignment value randomly chosen from a uniform distribution between 22.5° and 135° for each trial. When triangles were misaligned, their offsets relative to trajectories were clockwise or counter-clockwise, counterbalanced across trials.

Atten Percept Psychophys (2011) 73:2168–2179

Procedure There were three, four, or five targets. Participants first performed a practice session of nine trials (one for each condition); then, they completed four blocks of four repetitions per condition, for a total of 16 data points per condition, or 144 trials. All other aspects of the procedure were identical to those in Experiment 1. Results and discussion Mean tracking accuracy as a function of set size and orientation condition is shown in Fig. 4. We analyzed mean accuracy with a 3 X 3 ANOVA testing for effects of set size (three, four, five targets) and orientation (aligned, misaligned 15, and misaligned random). Percent correct decreased significantly as number of targets increased, F(2, 24) = 51.5, p < .001 (means 82%, 75%, and 69% for three, four, and five targets, respectively). There was neither a significant main effect of orientation condition (F < 1), nor an interaction between orientation and set size (F < 1). Participants did not perform better at the tracking task when the triangles were aligned with their trajectories as compared with when they were misaligned. Results are consistent with the conclusion that surface features do not directly affect the spatiotemporal correspondence process of MOT. As with previous work, the benefit of unique orientations, as found in Experiment 1, may stem from a benefit in perceptual parsing objects from one another or the use of visual short-term memory to search for targets momentarily lost during tracking (Horowitz et al., 2007; Oksama & Hyönä, 2008; Makovski & Jiang, 2009). Object orientations may have enhanced perceptual segmentation between overlapping or nearby objects in Experiment 2,

90%

Aligned Misaligned 15

80% Random Misaligned 70%

60%

50% 3

4 Number of Targets

5

Fig. 4 Results of Experiment 2. Percentage of correctly selected targets as a function of the number of targets and triangle alignment in the multiple object tracking task. Error bars show standard errors of the means

Atten Percept Psychophys (2011) 73:2168–2179

but this would have been equally likely regardless of whether they were aligned or misaligned, so would not have been evident in the results. The relationship between orientation and trajectory was not unique for each object (as it was for some conditions in Experiment 1), so there was no unique surface feature that could be used to search for lost targets. Finally, if orientations did affect motion perception by enhancing the appropriate motion for aligned triangles and misleading motion perception for misaligned triangles (Morikawa, 1999), then these effects were either too small to be measured with the present methods or did not have an impact on the tracking process. Regardless, the results showed no evidence for a facilitation of the tracking process from object alignment. This pattern of results is most consistent with the idea that orientation does not influence the spatiotemporal correspondence process of object tracking.

Experiment 3: orientation effects in TR Previous research using identifying features and trends in Experiment 1 suggested that distinguishing features can be used to search for targets that are lost during tracking (Horowitz et al., 2007; Makovski & Jiang, 2009). In Experiment 3, we examined this idea further by employing the target recovery (TR) paradigm in which participants track multiple moving objects that momentarily disappear at some point during the trial, creating a momentary blank in the display (Fencsik, Klieger, & Horowitz, 2007; Keane & Pylyshyn, 2006). This task differs from MOT because targets are not always visible, so the positions of the targets are not continuously available. Memory of the location of targets before the blank must be used to allow the targets to be localized after the blank. We refer to the task as “target recovery” to accentuate this difference between the task demands. Although the tasks are different, the possible ways that orientation could affect tracking performance delineated in the introduction could also be used to affect target localization in the TR task. Orientation could allow for (a) feature search for targets after the blank, (b) a benefit in localization from enhanced perceptual parsing after the blank, (c) position prediction of the target after the blank, and (d) disambiguation of a target from distractors. In Experiment 3, we tested whether a TR task revealed effects of orientation that we investigated in Experiment 1. In this experiment, we also tested whether object orientations that are aligned with their trajectories improve TR performance over conditions with no consistent relationship between orientation and motion. Previous work has demonstrated that TR performance is impaired when objects continue to move while they are invisible and reappear ahead of where they disappeared, as compared

2173

with a condition in which they pause during the blank (Keane & Pylyshyn, 2006). These results suggested that people primarily use position information to localize objects after the blank. More recent work, however, has demonstrated that motion information is used to localize targets under some circumstances, such as when objects move during the blank and the target load is low (Fencsik et al., 2007). In the present study, we investigated the effects of orientation when objects paused or moved during the blank to determine whether orientation has more of an impact on performance when objects move during the blank because position information is insufficient. Results could support the notion that orientation is different from other features because it can help TR when motion information is important. Method The apparatus, stimuli, and display were identical to those in Experiment 1, with the following exceptions. The 12 new participants always tracked three targets. At 4.2 s after the beginning of a trial, all objects disappeared, and the display remained blank for 450 ms. During the blank, all objects either paused or moved along their trajectories while invisible. The objects then reappeared, continued to move for 1 s, and then stopped for response selection. The five orientation conditions were the same as those in Experiment 1. Participants performed a practice block of five trials (one per condition) and four experimental blocks of four trials per condition, for 16 trials per condition or 160 trials. Results and discussion Accuracy was analyzed with a two-way repeated measures ANOVA with two factors: reappearance type (pause, move) and orientation condition (dots, same, unique, random change, and aligned, as in Experiment 1). Figure 5 shows the mean tracking accuracy in each condition. Tracking performance was significantly higher when objects remained in the same locations than when they moved forward during a disappearance, F(1, 11) = 221.5, p < .001, as has been found in previous work (Keane & Pylyshyn, 2006). There was no significant main effect of orientation condition, F(4, 44) = 1.5, p > .2, but there was a significant interaction between reappearance type and orientation condition, F(4, 44) = 3.52, p < .02. A simple effects analysis demonstrated no effect of orientation condition when objects paused during the blank, F(4, 44) = .4, p > .7, but a significant effect of orientation condition when objects moved during the blank, F(4, 44) = 3.38, p < .02. This result could support the idea that orientation is not used when position information stored in memory is sufficient to localize the targets after the blank.

2174

Atten Percept Psychophys (2011) 73:2168–2179

Percent Correct

90% 80% 70% 60% 50%

Dots Same Uniq Rand Align

Dots Same Uniq Rand Align

Pause

Move

Fig. 5 Results from the target recovery task used Experiment 3. Percentage of correctly selected targets in each condition. Left bars indicate when objects paused during the blank, and right bars indicate

when objects moved during the blank. Conditions within each group are the same as in Experiment 1. Error bars show standard errors of the means. Uniq unique, Rand random, Align aligned

To describe the significant difference between orientation conditions when objects moved during the blank, we compared conditions to one another using t tests. First, dots did not produce significantly different performance than triangles that remained in the same orientation, t(11) < 1. Object shape did not have an effect on TR performance. Second, unlike in Experiment 1, performance was not significantly worse when the orientations of objects were all the same as compared with when orientations were different from one another [mean percent correct for same = 61.5% vs. unique = 64.1%, t(11) < 1; same vs. random change = 57.5%, t(11) < 1]. If orientation was used to parse objects from one another and reduce crowding during tracking, this effect was not reliable in the results. Third, performance was significantly better when each triangle’s orientation was consistently unique (64.1%) than when it was changing randomly (57.5%), t(11) = 2.31, p < .05. This result could support the idea that orientation is used to search for lost targets. However, that hypothesis would also have predicted better performance with unique orientations over same orientations, which did not obtain, t(11) < 1, though the means showed a trend in the right direction (64.1% vs. 61.5%). Finally, there was better performance when orientations aligned with trajectories (aligned = 66.7%) than when orientations changed but did not align (random change = 57.5%), t(11) < 3.79, p < .005. Orientation related to object motion helped participants localize targets after the blank, possibly because orientation helped participants predict target location after it moved during the blank (Fencsik et al., 2007; St. Clair, Huff, & Seiffert, 2010) or because orientation helped disambiguate targets from distractors. These ideas are further investigated in Experiment 4. Overall, this pattern of results is most consistent with the idea that orientation facilitates a memory-based process (rather than any spatiotemporal correspondence process) by allowing participants to recover targets that have disappeared by using their unique or informative orientations.

Experiment 4: alignment effects in TR Similar to Experiment 2, in Experiment 4, we tested whether tracking performance was impaired by misaligned orientations that may provide misleading information about the targets’ trajectories. Here, we used a TR task to test the use of orientation information for aligned triangles and misaligned triangles that were offset from their trajectory by 15 degrees or more. As in Experiment 2, triangles did not differ in the uniqueness of orientation, or in the reliability of the orientation information across time. Rather, in this experiment, we tested only the relationship between orientation and motion information in the TR task. Method The apparatus, stimuli, and display were identical to those in Experiment 2, with the following exceptions. The 25 new participants always tracked three targets. At 4.2 s after the beginning of a trial, all triangles disappeared from the screen for 450 ms. During the blank, all triangles either paused or moved along their trajectories during the blank. The triangles then reappeared, continued to move for 1 s, and then stopped for response selection. Participants performed a practice block of six trials (one per condition) and four experimental blocks of six trials per condition, for 24 trials per condition, or 144 trials. Results and discussion As is shown in Fig. 6, tracking performance was better overall when objects paused (mean percent correct = 84%) than when they moved (67%), F(1, 24) = 185.3, p < .001, replicating Keane and Pylyshyn (2006). Orientation significantly affected performance, F(2, 48) = 8.5, p < .002, and interacted with whether objects moved during the blank, F(2, 48) = 3.6, p < .05. The effect of alignment was larger in the move condition than in the pause condition. A

Atten Percept Psychophys (2011) 73:2168–2179 90% Aligned Misaligned 15

80%

Random Misaligned 70%

60%

50%

Pause

Move

Fig. 6 Results of Experiment 4. Percentage of correctly selected targets as functions of triangle alignment and motion condition in the target recovery task. Error bars show standard errors of the means

simple effects analysis revealed a main effect of orientation only for the move, F(2, 48) = 8.31, p < .001, but not for the pause condition F(2, 48) = 1.99, ns. These results suggest that participants primarily relied on the last visible positions of the objects to localize targets after the blank, and when position was reliable, orientation information did not have an impact on performance. When objects moved during the blank, however, aligned triangles yielded better performance (70%) than did triangles in the misaligned random condition (63%), t(24) = 4.0, p < .001. Misaligned 15° yielded significantly better performance (67%) than did misaligned random (63%), t(24) = 2.2, p < .04. Aligned triangles produced marginally better performance than did triangles that were misaligned 15°, t(24) = 1.9, p = .07. Although performance declined overall when objects moved during the blank, performance was best when triangles were aligned to their direction of motion, and performance decreased as the offset increased between orientation and direction. Alignment, then, can affect our ability to recover targets that have disappeared momentarily. Similar to previous work with identifiable objects (Horowitz et al., 2007; Makovski & Jiang, 2009; Oksama & Hyönä, 2008), these results indicate that surface features can facilitate performance because people can use the visual feature information stored in memory to find targets. Also, similar to previous work using the TR paradigm (Fencsik et al., 2007), these results could be interpreted to indicate that directional information may be available to facilitate TR in some circumstances. For example, participants would be more successful identifying a target that is moving and oriented leftward next to a distractor that is oriented and moving rightward, if the memory for the target position or motion trajectory is buttressed by its orientation. The observed alignment effect in this experiment with the TR task contrasts with the null effect observed in Experiment 2

2175

with the MOT task, possibly because position information is always available in MOT but not in TR. Here, orientation aligned with the motion of targets facilitated recovery of targets that moved while they were not visible. Directional surface features may be related to motion information stored in memory for successful performance at the TR task. The larger effect of alignment for trials in which the objects moved during the blank than for trials in which the objects paused is consistent with this account, since motion information is most helpful in the move condition (Fencsik et al., 2007; Keane & Pylyshyn, 2006). In addition, we found a larger effect for the misaligned random condition than for the misaligned 15 degrees condition presumably because there were larger misalignments of orientation in the former. A closer look at the misaligned random condition revealed no correlation between performance and the range of tested misalignments from 22 degrees to 135 degrees from the direction of motion (r = .05). A null effect was also observed when binning the data into four levels of offset. One possibility is the effect is not evident because fewer trials were tested at each orientation in this condition, allowing random variation to obscure a trend. Another possibility is that the absence of a correlation could reflect that the alignment effect asymptotes with larger misalignments, as has been found for trajectory perception in previous work with misalignments greater than 20 degrees (Morikawa, 1999). Either way, the evidence clearly supports the notion that orientation that is informative about motion direction facilitates TR. The observed effect of alignment may reflect either position prediction, on the basis of the relationship between trajectory and orientation, or disambiguation of targets from distractors by comparing orientation to target locations stored in memory. These ideas were directly tested in Experiment 5.

Experiment 5: alignment effects before and after the blank in TR In Experiment 4, we demonstrated that people can use alignment to help localize targets that move during a brief disappearance. In Experiment 5, we expanded this result by testing when orientation has an effect. Orientation could influence perception before the blank, such as by influencing how direction is extrapolated to predict the future location of targets (position prediction), or orientation could be used after the blank, such as by influencing how the objects that have reappeared are related to the positions or trajectories stored in memory (disambiguation). To test these hypotheses, we manipulated the pre- and postblank shapes, including a replication of the conditions from Experiment 4 and a condition in which objects were always dots.

2176

Method The methods were identical to those in Experiment 4, with the following exceptions. We manipulated pre- and postblank shapes to make objects start as either dots or triangles, and to make them reappear after the blank as either dots or triangles. All objects were always the same shape within a display: Dots and triangles were never presented at the same time. When the objects were triangles, we tested two orientation conditions from Experiment 2—namely, aligned and misaligned random. The orientation condition was the same throughout the trial, and for any particular object, the postblank orientation was the same as the preblank orientation. During the blank, objects always moved. We randomized the start time of the blank to 2, 2.5, 3, 3.5, or 4 s after the start of the trial to make the time of the blank less predictable. Total tracking duration was always 5.6 s. Dot radius was approximately 0.38° visual angle. Twenty-six participants volunteered for this study. We tested a variety of speeds across subjects: 5°/s (N = 7), 7°/s (N = 9), and 10°/s (N = 10). All participants performed a practice block of 5 trials. After practice was finished, participants performed four experimental blocks for a total of 140 trials, or 20 trials per shape change condition. Conditions were randomized within each block, and the orientation offsets were clockwise or counterclockwise to the trajectory counterbalanced across trials. Results and discussion As is shown in Figure 7, we replicated the findings of Experiment 4: When the pre- and postblank shapes are both triangles, tracking accuracy was better when the triangles were aligned (mean percent correct = 74%) than when they were misaligned (69%), t(25) = 3.9, p < .002. There was also a significant between-subjects effect of speed, Fig. 7 Results of Experiment 5. Percentage of correctly selected targets as functions of alignment and shape change condition in the target recovery task. Error bars show standard errors of the means. Lines connecting two conditions show t test comparisons with significance levels p < .01

Atten Percept Psychophys (2011) 73:2168–2179

F(2, 25) = 1226.4, p < .001. Tracking performance declined as speeds increased from 5°/s (87%) to 7°/s (65%) to 10°/s (55%). This result replicates previous work finding that faster objects are harder to track (Alvarez & Franconeri, 2007; Liu et al., 2005;). Speed did not significantly interact with either shape change condition or alignment (Fs < 1), so it will not be discussed further. Analyzing accuracy when objects did not change shape within a trial allowed us to establish whether aligned triangles produced better performance than dots. Performance was better with aligned triangles than with dots, t(25) = 3.1, p < .01. Performance with misaligned triangles was not significantly different from that with dots, t(25) = 0.61, ns. This result suggests that TR performance is facilitated by alignment, a result consistent with those found in Experiment 3. Participants were not misled by the misalignment, but were using aligned orientation information to bolster performance. Analysis of the conditions with a shape change revealed when people use orientation to help recover targets. Alignment was effective only when triangles appeared after the blank. A 2 X 2 ANOVA that tested shape change (dot to triangle, triangle to dot) and alignment (aligned, misaligned) found a significant main effect of shape change, F(1, 25) = 7.6, p < .05, and an interaction between shape change and alignment, F(1, 25) = 5.8, p < .05. If participants were using orientation to predict future target position, then the alignment effect would have been found when the preblank shape was a triangle. This is not what we found, as is shown on the right side of Fig. 7. When triangles became dots, triangle alignment before the blank did not affect performance, t(25) < 1, p > .5. Instead, surprisingly, results showed that people used alignment only after the blank. When dots became triangles, the mean percent correct was higher when the triangles were aligned (71%) than when they were misaligned (66%), t(25) = 3.1,

Atten Percept Psychophys (2011) 73:2168–2179

p < .005. In addition, performance was marginally better when dots became aligned triangles (71%) than when dots remained dots (68%), t(25) = 2.1, p < .06. These results indicate that orientation information consistent with the direction of motion can facilitate performance when participants attempt to find the targets after they reappear. This pattern of results is most consistent with the idea that orientation facilitates performance by disambiguating alternatives that appear after the blank in relation to the memory of target positions.

General discussion Many objects in the world—people and cars, for example— move in a direction consistent with their orientation most of the time. In the present experiments, we tested whether, how, and when people use orientation information to sustain representations of moving targets. People attended to target triangles with orientations that were either completely uninformative, unique, aligned, or systematically misaligned with their trajectories in MOT and TR tasks. In the MOT task, orientation facilitated performance possibly by allowing for better perceptual parsing and/or feature search of lost targets (Experiment 1), as has been described with other features (Makovski & Jiang, 2009). Misalignment did not impair performance over alignment in the MOT task (Experiment 2). However in the TR task, in which all objects disappeared briefly, people did use orientation to localize targets if the objects moved during the blank (Experiment 3). Also, targets that were aligned with their motion were found more successfully than ones that were consistently misaligned (Experiment 4). In recovering aligned triangles, people used orientation after the blank rather than before the blank (Experiment 5). These results are most consistent with the hypothesis that people use orientation to disambiguate targets from distractors when objects reappear. We interpret these results to indicate that orientation facilitates the memory process and does not seem to improve the spatiotemporal correspondence process during target tracking and recovery. People seem to take advantage of orientation information stored in memory, rather than use orientation to sustain perceptual object representations, to improve performance on these tasks. Previous work has also characterized the role of surface features in object tracking as a role for memory. Tracking uniquely identifiable targets among uniquely identifiable distractors yield better performance than tracking nonunique objects (Horowitz et al., 2007; Makovski & Jiang, 2009; Oksama & Hyönä, 2008). People track and identify familiar objects and faces better than they do unfamiliar objects and faces (Oksama & Hyönä, 2004, 2008).

2177

Similarly, but without requiring identification, Horowitz and colleagues (2007) found that performance for tracking a subset of different cartoon animals was slightly better than performance for tracking a subset of identical animals. Manipulating features and identity independently, Makovski and Jiang (2009) demonstrated that unique features of objects, such as colors or shapes, also can lead to improvement in tracking performance. Features were helpful only when they could uniquely identify targets and/or discriminate targets from distractors. Participants could find a lost target at any time during tracking by simply searching for the unique feature that defined the lost target (Makovski & Jiang, 2009). In Experiments 1 and 3, we expanded on these observations by testing orientation as the unique feature of tracked objects. Results were consistent with the previous work showing that the feature of orientation could be used to find lost targets when it was unique to each object. These results support the notion that memory-based processes are an important component to the performance at object tracking. In the present study, we also investigated whether orientation would facilitate tracking in other ways as well. Orientation is the only surface feature that is directional and has a strong relationship with motion perception. Results of these experiments suggested that, even when orientation is not unique, it can assist target recovery if it is informative about other pertinent information such as object trajectory. Alignment between orientation and direction did not strongly facilitate object tracking by supporting the spatiotemporal correspondence process of tracking, as evidenced from the negligible effects of alignment over misaligned orientations in Experiment 2. Rather, orientation allowed for improved performance in TR, as in Experiment 4, possibly because target features stored in memory helped disambiguate targets from distractors. Taken together, these results suggest that tracking objects with informative surface features may engage a memory-based process that is separate from the spatiotemporal correspondence process. It remains to be seen whether relating other features to motion can yield the same benefit, or whether the effects found here are particular to orientation. Our results also indicate when surface features are used. Aligning orientations with objects’ trajectories facilitated performance when the orientation was presented after the blank, but not before it (Experiment 5). These results suggest that participants are not extrapolating target positions on the basis of orientation information that appears before the blank. This result is compatible with the conclusion made by Keane and Pylyshyn (2006) that people do not use optimally all of the target information, such as motion history, to appropriately extrapolate target position during a disappearance. It is also consistent with a

2178

position-sensitive tracking mechanism that maintains target continuity across time by periodically sampling and storing position information (Kazanovich & Borisyuk, 2006; Pylyshyn, 2001), if it is also assumed that surface features affect this sampling mechanism. However, this observation is not well accounted for by models of tracking in which all of the target features are stored and matched for recovery, as has been suggested by previous work (Horowitz et al., 2007; Makovski & Jiang, 2009; Oksama & Hyönä, 2008). Such a strategy would predict the most performance benefits when orientation remains consistent across the blank, by appearing both before and after the blank. Our results, however, show that even when shape is inconsistent because orientation occurs only after the blank, there is an improvement in performance with aligned orientations. Therefore, these results are best accounted for with a strategy in which memory of target positions is matched to the orientation information in the objects that have reappeared to disambiguate targets from distractors. For example, if two objects are equidistant from the remembered target location, the target can be correctly identified with orientation information, assuming that orientation is aligned with target motion (Fig. 8). Orientation information also could potentially override proximity, allowing distractors closer than the target to the remembered target location to be appropriately rejected. These experiments do not allow us to determine exactly the nature of this process. Orientations could be used to backward extrapolate to the positions stored in memory where targets disappeared, or else, a number of possible forward extrapolated positions could be narrowed by the postblank orientation. Orientation could also be used to match with stored information about the trajectory of targets. Regardless, it is clear that orientation as a surface feature affects TR strategy at the time when targets must be localized in the postblank display. Finally, these observations also indicate that orientation was not automatically combined with motion information to facilitate object tracking. Trajectories of many common objects are redundant with their orientation, because objects such as people, cars, and bicycles move forward most of the

Fig. 8 Disambiguation: Orientation can help identify a target (red triangle) and reject a distractor nearby the last remembered location of the target (blue X) because the target’s orientation is aligned with the target’s direction of motion.

Atten Percept Psychophys (2011) 73:2168–2179

time. Combining orientation and motion information would yield more reliable information for predicting object motion in these cases. Previous research has indicated that orientation can affect the perceived direction of moving objects (Morikawa, 1999). However, we did not find performance benefits when objects were aligned with their trajectories during object tracking. Rather, the benefit of orientation information found presently occurred for TR. Orientation, it seems, can be used similarly to other features, such as color or shape, as a cue for finding moving targets that are momentarily lost (as in MOT) or absent (as in TR). Unlike color or shape, however, we found that orientation was useful because of its directionality. Orientation aligned with a target’s motion trajectory was helpful either because of a facilitation between orientation and motion information stored in memory, or as a pointer to the remembered location of the target. Taken together, this research indicates that people use surface features to successfully match memory representations of attended targets to new visual information for the purpose of keeping track of or recovering the locations of objects that move. In some sense, the most surprising aspect of these results may be that object orientation wasn’t more powerful in affecting tracking performance. It seems to be a common belief that misleading orientations disrupt the ability to track moving objects. After all, players of ball sports, such as basketball and football, sometimes will deliberately face in a direction different from where they expect to catch or throw the ball, a strategy that seems to successfully confuse novices (Sebanz & Shiffrar, 2009). Animals also benefit from surface properties, such as highcontrast markings, that mislead the motion perception of predators (Stevens, 2007; Troscianko, Benton, Lovell, Tolhurst, & Pizlo, 2009) and can even hinder human predators (Stevens, Yule, & Ruxton, 2008). Similar tactics were used by military to camouflage navy ships during World War I, so that enemies would not know the direction or speed of the ships (Behrens, 1999). Nevertheless, the present experiments demonstrated little, if any, impact of misleading orientation on object tracking. It is possible that misleading orientations may be more effective in natural settings, as when combined with social cues to intentions and/or disruptive effects of camouflage. To successfully navigate the dynamic world, people seem to be able to use orientation when it is aligned with motion to help them recover lost objects, but are not strongly misled by misalignment. Author Note The authors thank Rebecca St. Clair and Laura Thomas for valuable assistance with providing advice on the experimental designs and interpretation, and Jeremy Wolfe and Jonathan Flombaum for their insightful suggestions. Support provided by P30-EY008126.

Atten Percept Psychophys (2011) 73:2168–2179

References Alvarez, G. A., & Franconeri, S. L. (2007). How many objects can you track?: Evidence for a resource-limited attentive tracking mechanism. Journal of Vision, 7, 1–10. doi:10.1167/ 7.13.14 Behrens, R. R. (1999). The role of artists in ship camouflage during World War I. Leonardo, 32, 53–59. doi:10.1162/002409499553000 Bettencourt, K. C., & Somers, D. C. (2009). Effects of target enhancement and distractor suppression on multiple object tracking capacity. Journal of Vision, 9, 9. Brainard, D. H. (1997). The Psychophysics Toolbox. Spatial Vision, 10, 433–436. Cavanagh, P., & Alvarez, G. (2005). Tracking multiple targets with multifocal attention. Trends in Cognitive Sciences, 9, 349–354. doi:10.1016/j.tics.2005.05.009 D’Avossa, G., Shulman, G. L., Snyder, A. Z., & Corbetta, M. (2006). Attentional selection of moving objects by a serial process. Vision Research, 46, 3403–3412. Fencsik, D. E., Klieger, S. B., & Horowitz, T. S. (2007). The role of location and motion information in the tracking and recovery of moving objects. Perception & Psychophysics, 69, 567–577. Freyd, J. J. (1983). The mental representation of movement when static stimuli are viewed. Perception & Psychophysics, 33, 575–581. Freyd, J. J., & Finke, R. A. (1984). Representational momentum. Journal of Experimental Psychology: Learning, Memory, and Cognition, 10, 126–132. doi:10.1037/0278-7393.10.1.126 Freyd, J. J., & Pantzer, T. M. (1995). Static patterns moving in the mind. In S. M. Smith, T. B. Ward, & R. A. Finke (Eds.), The creative cognition approach (pp. 181–204). Cambridge: MIT. Horowitz, T. S., Klieger, S. B., Fencsik, D. E., Yang, K. K., Alvarez, G. A., & Wolfe, J. M. (2007). Tracking unique objects. Perception & Psychophysics, 69, 172–184. Kazanovich, Y., & Borisyuk, R. (2006). An oscillatory neural model of multiple object tracking. Neural Computation, 18, 1413–1440. Keane, B., & Pylyshyn, Z. (2006). Is motion extrapolation employed in multiple object tracking? Tracking as a low-level, nonpredictive function. Cognitive Psychology, 52, 346–368. doi:10.1016/j.cogpsych.2005.12.001 Liu, G., Austen, E. L., Booth, K. S., Fisher, B. D., Argue, R., Rempel, M. I., et al. (2005). Multiple-object tracking is based on scene, not retinal, coordinates. Journal of Experimental Psychology: Human Perception & Performance, 31, 235–247. doi:10.1037/ 0096-1523.31.2.235 Makovski, T., & Jiang, Y. V. (2009). The role of visual working memory in attentive tracking of unique objects. Journal of Experimental Psychology: Human Perception & Performance, 35, 1687–1697. doi:10.1037/a0016453 McBeath, M. K., Morikawa, K., & Kaiser, M. K. (1992). Perceptual bias for forward-facing motion. Psychological Science, 3, 362– 367. doi:10.1111/j.1467-9280.1992.tb00048 Morikawa, K. (1999). Symmetry and elongation of objects influence perceived direction of translational motion. Perception & Psychophysics, 61, 134–143.

2179 Oksama, L., & Hyönä, J. (2004). Is multiple object tracking carried out automatically by an early vision mechanism independent of higher order cognition? An individual difference approach. Visual Cognition, 11, 631–671. doi:10.1080/13506280344 000473 Oksama, L., & Hyönä, J. (2008). Dynamic binding of identity and location information: A serial model of multiple identity tracking. Cognitive Psychology, 56, 237–283. doi:10.1016/j. cogpsych.2007.03.001 Palmer, S. E. (1980). What makes triangles point: Local and global effects in configurations of ambiguous triangles. Cognitive Psychology, 12, 285–305. doi:10.1016/0010-0285(80)90012-2 Pelli, D. G. (1997). The VideoToolbox software for visual psychophysics: Transforming numbers into movies. Spatial Vision, 10, 437–442. Pylyshyn, Z. (2001). Visual indexes, preconceptual objects and situated vision. Cognition, 80, 127–158. Pylyshyn, Z., & Storm, R. (1988). Tracking multiple independent targets: Evidence for a parallel tracking mechanism. Spatial Vision, 3, 179–197. Sebanz, N., & Shiffrar, M. (2009). Detecting deception in a bluffing body: The role of expertise. Psychonomic Bulletin & Review, 16, 170–175. St. Clair, R., Huff, M., & Seiffert, A. E. (2010). Conflicting motion information impairs multiple object tracking. Journal of Vision, 10, 18. Stevens, M. (2007). Predator perception and the interrelation between protective coloration. Proceedings of the Royal Society B: Biological Sciences, 274, 1457–1464. doi:10.1098/ rspb.2007.0220 Stevens, M., Yule, D. H., & Ruxton, G. D. (2008). Dazzle coloration and prey movement. Proceedings of the Royal Society B: Biological Sciences, 275, 2639–2643. doi:10.1098/ rspb.2008.0877 Tombu, M., & Seiffert, A. E. (2008). Attentional costs in multipleobject tracking. Cognition, 108, 1–25. Tombu, M., & Seiffert, A. E. (2011). Tracking planets and moons: Mechanisms of object tracking revealed with a new paradigm. Attention, Perception, & Psychophysics, 73, 738–750. Troscianko, T., Benton, C. P., Lovell, P. G., Tolhurst, D. J., & Pizlo, Z. (2009). Camouflage and visual perception. Philosophical Transactions of the Royal Society of London B: Biological Sciences, 364, 449–461. Vinson, N. G., & Reed, C. L. (2002). Sources of object-specific effects in representational momentum. Visual Cognition, 9, 41–65. doi:10.1080/13506280143000313 Vul, E., Frank, M., Alvarez, G., & Tenenbaum, J. (2009). Explaining human multiple object tracking as resource-constrained approximate inference in a dynamic probabilistic model. In Y. Ben- gio, D. Schuurmans, J. Lafferty, C. K. I. Williams, & A. Culotta (Eds.), Advances in Neural Information Processing Systems 22 (pp. 1955–1963). Werkhoven, P., Snippe, H. P., & Koenderink, J. (1990). Metrics for the strength of low-level motion perception. Journal of Visual Communication and Image Representation, 1, 176–188. doi:10.1016/1047-3203(90)90006-H