A key component of humans’ striking creativity in solving problems is our ability to construct novel descriptions to help us characterize novel concepts. Bongard problems, which challenge the problem solver to come up with a rule for distinguishing visual scenes that fall into two categories, provide an elegant test of this ability. Bongard problems are challenging for both human and machine category learners because only a handful of example scenes are presented for each category, and they often require the open-ended creation of new descriptions. A new type of Bongard problem called Physical Bongard Problems (PBPs) is introduced, which requires solvers to perceive and predict the physical spatial dynamics implicit in the depicted scenes. The PATHS (Perceiving And Testing Hypotheses on Structures) computational model which can solve many PBPs is presented, and compared to human performance on the same problems. PATHS and humans are similarly affected by the ordering of scenes within a PBP. Spatially or temporally juxtaposing similar (relative to dissimilar) scenes promotes category learning when the scenes belong to different categories, but hinders learning when the similar scenes belong to the same category. The core theoretical commitments of PATHS, which we believe to also exemplify open-ended human category learning, are a) the continual perception of new scene descriptions over the course of category learning, b) the context-dependent nature of that perceptual process, in which the perceived scenes establish the context for the perception of subsequent scenes, c) hypothesis construction by combining descriptions into explicit rules, and d) bi-directional interactions between perceiving new aspects of scenes and constructing hypotheses for the rule that distinguishes categories.
Tag Archives: Perception
Carving joints into nature: reengineering scientific concepts in light of concept-laden evidence
A new wave of proposals suggests that scientists must reassess scientific concepts in light of accumulated evidence. However, reengineering scientific concepts in light of data is challenging because scientific concepts affect the evidence itself in multiple ways. Among other possible influences, concepts (i) prime scientists to overemphasize within-concept similarities and betweenconcept differences; (ii) lead scientists to measure conceptually relevant dimensions more accurately; (iii) serve as units of scientific experimentation, communication, and theory-building; and (iv) affect the phenomena themselves. When looking for improved ways to carve nature at its joints, scholars must take the concept-laden nature of evidence into account to avoid entering a vicious circle of concept-evidence mutual substantiation.
Going Beyond Formalisms: A Grounded and Embodied Learning Approach to the Design of Pedagogical Statistics Simulations
Computer-based interactive simulations that model the processes of sampling from a population are increasingly being used in data literacy education. However, these simulations are often summarized by graphs designed from the point of view of experts which makes them difficult for novices to grasp. In our ongoing design-based research project, we build and test alternative sampling simulations to the standard ones. Based on a grounded and embodied learning perspective, the core to our design position is that difficult and abstract sampling concepts and processes should: be grounded in familiar objects that are intuitive to interpret, incorporate concrete animations that spontaneously activate learners’ gestures, and be accompanied by verbal instruction for a deeply integrated learning. Here, we report the results from the initial two phases of our project. In the first iteration, through an online experiment (N=126), we show that superficial perceptual elements in a standard simulation can lead to misinterpretation of concepts. In the second iteration, we pilot test a new grounded simulation with think-aloud interviews (N=9). We reflect on the complementary affordances of visual models, verbal instruction, and learners’ gestures in fostering integrated and deep understanding of concepts.
The Counterintuitive Interpretations Learned from Putatively Intuitive Simulations
Reasoning about sampling distributions is notably challenging for humans. It has been argued that the complexity involved in sampling processes can be facilitated by interactive computer simulations that allow learners to experiment with variables. In the current study, we compared the effects of learning sampling distributions through a simulation-based learning (SBL) versus direct instruction (DI) method. While both conditions resulted in similar improvement in rule learning and graph identification, neither condition improved more distant transfer of concepts. Furthermore, the simulation-based learning method resulted in unintuitive and surprising kinds of misconceptions about how sample size affects estimation of parameters while the direct instruction group used correct intuitive judgments more often. We argue that similar perceptual properties of different sampling processes in the SBL condition overrode learners’ intuitions and led them to make conceptual confusions that they would not typically make. We conclude that conceptually important differences should be grounded in easily interpretable and distinguishable perceptual representations in simulation-based learning methods.
Categorical perception meets El Greco: categories unequally influence color perception of simultaneously present objects
Broad empirical evidence suggests that higher-level cognitive processes, such as language, categorization, and emotion, shape human visual perception. Do these higher-level processes shape human perception of all the relevant items within an immediately available scene, or do they affect only some of them? Here, we study categorical effects on visual perception by adapting a perceptual matching task so as to minimize potential non- perceptual influences. In three experiments with human adults (N = 80; N = 80, N = 82), we found that the learned higher-level categories systematically bias human perceptual matchings away from a caricature of their typical color. This effect, however, unequally biased different objects that were simultaneously present within the scene, thus demonstrating a more nuanced picture of top-down influences on perception than has been commonly assumed. In particular, perception of only the object to be matched, not the matching object, was influenced by animal category and it was gazed at less often by participants. These results suggest that category- based associations change perceptual encodings of the items at the periphery of our visual field or the items stored in concurrent memory when a person moves their eyes from one object to another. The main finding of this study calls for a revision of theories of top-down effects on perception and falsify the core assumption behind the El Greco fallacy criticism of them.
Tonal Emergence: An agent-based model of tonal coordination
Humans have a remarkable capacity for coordination. Our ability to interact and act jointly in groups is crucial to our success as a species. Joint Action (JA) research has often concerned itself with simplistic behaviors in highly constrained laboratory tasks. But there has been a growing interest in understanding complex coordination in more open-ended contexts. In this regard, collective music improvisation has emerged as a fascinating model domain for studying basic JA mechanisms in an unconstrained and highly sophisticated setting. A number of empirical studies have begun to elucidate coordination mechanisms underlying joint musical improvisation, but these findings have yet to be cached out in a working computational model. The present work fills this gap by presenting Tonal Emergence, an idealized agent-based model of improvised musical coordination. Tonal Emergence models the coordination of notes played by improvisers to generate harmony (i.e., tonality), by simulating agents that stochastically generate notes biased towards maximizing harmonic consonance given their partner’s previous notes. The model replicates an interesting empirical result from a previous study of professional jazz pianists: feedback loops of mutual adaptation between interacting agents support the production of consonant harmony. The model is further explored to show how complex tonal dynamics, such as the production and dissolution of stable tonal centers, are supported by agents that are characterized by (i) a tendency to strive toward consonance, (ii) stochasticity, and (iii) a limited memory for previously played notes. Tonal Emergence thus provides a grounded computational model to simulate and probe the coordination mechanisms underpinning one of the more remarkable feats of human cognition: collective music improvisation.
Reconstructing Maps from Text
Avery, J. E., Goldstone, R. L, & Jones, M. N. (2021). Reconstructing Maps from Text. Cognitive Systems Research. doi: https://doi.org/10.1016/j.cogsys.2021.07.007
Previous research has demonstrated that Distributional Semantic Models (DSMs) are capable of reconstructing maps from news corpora (Louwerse & Zwaan, 2009) and novels (Louwerse & Benesh, 2012). The capacity for reproducing maps is surprising since DSMs notoriously lack perceptual grounding . In this paper we investigate the statistical sources required in language to infer maps, and the resulting constraints placed on mechanisms of semantic representation. Study 1 brings word co-occurrence under experimental control to demonstrate that standard DSMs cannot reproduce maps when word co-occurrence is uniform. Specifically, standard DSMs require that direct co-occurrences between city names in a corpus mirror the proximity between the city locations in the map in order to successfully reconstruct the spatial map. Study 2 presents an instance-based DSM that is capable of reconstructing maps independent of the frequency of co-occurrence of city names.
The Influences of Category Learning on Perceptual Reconstructions
We explore different ways in which the human visual system can adapt for perceiving and categorizing the environment. There are various accounts of supervised (categorical) and unsupervised perceptual learning, and different perspectives on the functional relationship between perception and categorization. We suggest that common experimental designs are insufficient to differentiate between hypothesized perceptual learning mechanisms and reveal their possible interplay. We propose a relatively underutilized way of studying potential categorical effects on perception, and we test the predictions of different perceptual learning models using a two-dimensional, interleaved categorizationplus- reconstruction task. We find evidence that the human visual system adapts its encodings to the feature structure of the environment, uses categorical expectations for robust reconstruction, allocates encoding resources with respect to categorization utility, and adapts to prevent miscategorizations.
Categories affect color perception of only some simultaneously present objects
There is broad empirical evidence suggesting that higher-level cognitive processes, such as language, categorization, and emotion, shape human visual perception. For example, categories that we acquire throughout lifetime have been found to alter our perceptual discriminations and distort perceptual processing. However, many of these studies have been criticized as unable to differentiate between immediate perceptual experience and the arguably concomitant processes, such as memory, judgment, and some kinds of attention. Here, we study categorical effects on perception by adapting the perceptual matching task to minimize the potential non-perceptual influences on the results. We found that the learned category-color associations bias human color matching judgments away from their category ideal on a color continuum. This effect, however, unequally biased two objects (probe and manipulator) that were simultaneously present on the screen, thus demonstrating a more nuanced picture of top-down influences on perception than has been assumed both by the theories of categorical perception and the El Greco methodological fallacy. We suggest that only the concurrent memory for visually present objects is subject to a contrast-from-caricature distortion due to category-association learning.
A three-site reproduction of the Joint Simon Effect with the NAO robot
The generalizability of empirical research depends on the reproduction of findings across settings and populations. Consequently, generalizations demand resources beyond that which is typically available to any one laboratory. With collective interest in the joint Simon effect (JSE) – a phenomenon that suggests people work more effectively with humanlike (as opposed to mechanomorphic) robots – we pursued a multi-institutional research cooperation between robotics researchers, social scientists, and software engineers. To evaluate the robustness of the JSE in dyadic human-robot interactions, we constructed an experimental infrastructure for exact, lab-independent reproduction of robot behavior. Deployment of our infrastructure across three institutions with distinct research orientations (well-resourced versus resource-constrained) provides initial demonstration of the success of our approach and the degree to which it can alleviate technical barriers to HRI reproducibility. Moreover, with the three deployments situated in culturally distinct contexts (Germany, the U.S. Midwest, and the Mexico-U.S. Border), observation of a JSE at each site provides evidence its generalizability across settings and populations.
Coordination and consonance between interacting, improvising musicians
Setzler, M., & Goldstone, R. L. (2020). Coordination and consonance between interacting, improvising musicians. Open Mind: Discoveries in Cognitive Science, 4, 88—101. https://doi.org/10.1162/opmi_a_00036.
Joint action (JA) is ubiquitous in our cognitive lives. From basketball teams to teams of surgeons, humans often coordinate with one another to achieve some common goal. Idealized laboratory studies of group behavior have begun to elucidate basic JA mechanisms, but little is understood about how these mechanisms scale up in more sophisticated and open-ended JA that occurs in the wild. We address this gap by examining coordination in a paragon domain for creative joint expression: improvising jazz musicians. Coordination in jazz music subserves an aesthetic goal: the generation of a collective musical expression comprising coherent, highly nuanced musical structure (e.g. rhythm, harmony). In our study, dyads of professional jazz pianists improvised in a “coupled”, mutually adaptive condition, and an “overdubbed” condition which precluded mutual adaptation, as occurs in common studio recording practices. Using a model of musical tonality, we quantify the flow of rhythmic and harmonic information between musicians as a function of interaction condition. Our analyses show that mutually adapting dyads achieve greater temporal alignment and produce more consonant harmonies. These musical signatures of coordination were preferred by independent improvisers and naive listeners, who gave higher quality ratings to coupled interactions despite being blind to condition. We present these results and discuss their implications for music technology and JA research more generally.
Quantifying Emergent, Dynamic Tonal Coordination in Collaborative Musical Improvisation
Groups of interacting individuals often coordinate in service of abstract goals, such as the alignment of mental representations in conversation, or the generation of new ideas in group brainstorming sessions. What are the mechanisms and dynamics of abstract coordination? This study examines coordination in a sophisticated paragon domain: collaboratively improvising jazz musicians. Remarkably, freely improvising jazz ensembles collectively produce coherent tonal structure (i.e. melody and harmony) in real time performance without previously established harmonic forms. We investigate how tonal structure emerges out of interacting musicians, and how this structure is constrained by underlying patterns of coordination. Dyads of professional jazz pianists were recorded improvising in two conditions of interaction: a ‘coupled’ condition in which they could mutually adapt to one another, and an ‘overdubbed’ condition which precluded mutual adaptation. Using a computational model of musical tonality, we show that this manipulation effected the directed flow of tonal information amongst pianists, who could mutually adapt to one another’s notes in coupled trials, but not in overdubbed trials. Consequently, musicians were better able to harmonize with one another in coupled trials, and this ability increased throughout the course of improvised performance. We present these results and discuss their implications for music technology and joint action research more generally.
How do people code information in working memory when items share features?
Featured
A large literature suggests that the way we process information is influenced by the categories that we have learned. We examined whether, when we try to uniquely encode items in workingmemory, the information encoded depends on the other stimuli being simultaneously learned. Participants were required to memorize unknown aliens, presented one at the time, for immediate recognition of their features. Some aliens, called twins, were organized into pairs that shared every feature (nondiscriminative feature) except one (discriminative feature), while some other aliens, called hermits, did not share feature. We reasoned that if people develop unsupervised categories by creating a category for a pair of aliens, we should observe better feature identification performance for nondiscriminative features compared to hermit features, but not compared to discriminative features. On the contrary, if distinguishing features draw attention, we should observe better performance when a discriminative rather than nondiscriminative feature was probed. Overall, our results suggest that when items share features, people code items in working memory by focusing on similarities between items, establishing clusters of items in an unsupervised fashion not requiring feedback on cluster membership.
Bias to (and away from) the Extreme: Comparing Two Models of Categorical Perception Effects
Categorical Perception (CP) effects manifest as faster or more accurate discrimination between objects that come from different categories compared to objects that come from the same category, controlling for the physical differences between the objects. The most popular explanations of CP effects have relied on perceptual warping causing stimuli near a category boundary to appear more similar to stimuli within their own category and/ or less similar to stimuli from other categories. Hanley and Roberson (2011), on the basis of a pattern not previously noticed in CP experiments, proposed an explanation of CP effects that relies not on perceptual warping, but instead on inconsistent usage of category labels. Experiments 1 and 2 in this paper show a pattern opposite the one Hanley and Roberson pointed out. Experiment 3, using the same stimuli but with different choice statistics (i.e., different probabilities of each face being the target), obtains the same pattern as the one Hanley and Roberson showed. Simulations show that both category label and perceptual models are able to reproduce the patterns of results from both experiments, provided they include information about the choice statistics. This suggests two conclusions. First, the results described by Hanley and Roberson should not be taken as evidence in favor of a category label model. Second, given that participants did not receive feedback on their choices, there must be some mechanism by which participants monitor their own choices and adapt to the choice statistics present in the experiment.
Building Inner Tools
Humans show a striking penchant for creating tools to benefit our own thought processes. Andy Clark (2003, 2008) has convincingly argued that the tools that we as humans recruit become integrated parts of an extended cognitive system that includes us as just one component. By extending cognition beyond our brains, Clark presents an “embiggened” perspective on what it means to be a cognizer and a person more generally. This perspectival shift runs counter to some recent forms of argumentation that in effect work to minimize personhood. For example, arguments for lack of personal culpability can take the form of “It wasn’t my fault. It was the fault of my ___ ” to be filled in, perhaps, by “upbringing,” “genes,” “neurochemistry,” “diet,” or “improperly functioning amygdala.” Instead, Clark (see also Dennett 1989) offers the opposite line of argumentation, according to which we consist not only of our amygdalae and hippocampi but also potentially our glasses, notebooks, friends, supporting technologies, and culture.
Patterns of coordination in simultaneously and sequentially improvising jazz musicians
In Joint Action (JA) tasks, individuals must coordinate their actions so as to achieve some desirable outcome at the grouplevel. Group function is an emergent outcome of ongoing, mutually constraining interactions between agents. Here we investigate JA in dyads of improvising jazz pianists. Participants’ musical output is recorded in one of two conditions: a real condition, in which two pianists improvise together as they typically would, and a virtual condition, in which a single pianist improvises along with a “ghost partner” – a recording of another pianist taken from a previous real trial. The conditions are identical except for that in real trials subjects are mutually coupled to one another, whereas there is only unidirectional influence in virtual trials (i.e. recording to musician). We quantify ways in which the rhythmic structures spontaneously produced in these improvisations is shaped by mutual coupling of co-performers. Musical signatures of underlying coordination patterns are also shown to parallel the subjective experience of improvisers, who preferred playing in trials with bidirectional influence despite not explicitly knowing which condition they had played in. These results illuminate how mutual coupling shapes emergent, group-level structure in the creative, open-ended and fundamentally collaborative domain of expert musical improvisation.
A Computational Model of Scientific Discovery in a Very Simple World, Aiming at Psychological Realism
We propose a computational model of human scientific discovery and perception of the world. As a prerequisite for such a model, we simulate dynamic microworlds in which physical events take place, as well as an observer that visually perceives and makes interpretations of events in the microworld. Moreover, we give the observer the ability to actively conduct experiments in order to gain evidence about natural regularities in the world. We have broken up the description of our project into two pieces. The first piece deals with the interpreter constructing relatively simple visual descriptions of objects and collisions within a context. The second phase deals with the interpreter positing relationships among the entities, winding up with elaborated construals and conjectures of mathematical laws governing the world. This paper focuses only on the second phase. As is the case with most human scientific observation, observations are subject to interpretation, and the discoveries are influenced by these interpretations.
Models of human scientific discovery
The scientific understanding of scientific understanding has been a long-standing goal of cognitive science. A satisfying formal model of human scientific discovery would be a major intellectual achievement, requiring solutions to core problems in cognitive science: the creation and use of apt mental models, the prediction of the behavior of complex systems involving interactions between multiple classes of elements, high-level perception of noisy and multiply interpretable environments, and the active interrogation of a system through strategic interventions on it – namely, via experiments. Over the past decades there have been numerous attempts to build formal models that capture what Perkins (1981) calls some of the “mind’s best work” – scientific explanations for how the natural world works by systematic observation, prediction, and testing. Early work by Hebert Simon and his colleagues (Langley, Simon, Bradshaw, & Zytkow, 1987) developed production rule systems employing heuristics to tame extremely large conjoint search spaces of experiments to run and hypotheses to test. Qualitative physics approaches seek to understand physical phenomena by building non-numeric, relational models of the phenomena (Forbus, 1984). Some early connectionist models interpreted scientific explanation in terms of emerging patterns of strongly activated hypotheses that mutually support one another (Thagard, 1992).
Data visualization as a domain to research areas in cognitive science
How people are able to turn information in the environment into meaning is a critical question for cognitive science. That environment is increasingly data-driven. Using data to inform decisions and improve understanding of the world is a valuable component of critical thinking, and serves as the foundation of evidence-based decision making. Designing graphical representations can make those data more accessible, such that users may engage the visual system and capacity for visual pattern recognition to discern regularities and properties of data. We ultimately want to understand the connection between the initial perception of data visualizations and conceptual understanding of information. Data visualizations, broadly, are the representation of recorded values in visual form, including scientific visualizations such as brain scans, or live visualizations such as stock market monitoring; the work discussed through this symposium is of the type used in science, business, and medical settings to display data abstractly.
Visual flexibility in arithmetic expressions
We investigated whether, and in what, ways people use visual structures to evaluate mathematical expressions. We also explored the relationship between strategy use and other common measures in mathematics education. Participants organized long sum/products when visual structure was available in algebraic expressions. Two experiments showed a similar pattern: One group of participants primarily calculated from left to right, or combined identical numbers together. A second group calculated adjacent pairs. A third group tended to group terms which either produced easy sums (e.g., 6+4), or participated in a global structure. These different strategies were associated with different levels of success on the task, and, in Experiment 2, with differential math anxiety and mathematical skill. Specifically, problem solvers with lower math anxiety and higher math ability tend to group by chunks and easy calculation. These results identify an important role for the perception of coherent structure and pattern identification in mathematical reasoning.
Experientially grounded learning about the roles of variability, sample size, and difference between means in statistical reasoning
Despite its omnipresence in this information-laden society, statistics is hard. The present study explored the applicability of a grounded cognition approach to learning basic statistical concepts. Participants in 2 experiments interacted with perceptually rich computer simulations designed to foster understanding of the relations between fundamental statistical concepts and to promote the ability to reason with statistics. During training, participants were asked to estimate the probability of two samples coming from the same population, with sample size, variability, and difference between means independently manipulated. The amount of learning during training was measured by the difference between participants’ confidence judgments and those of an Ideal Observer. The amount of transfer was assessed by the increase in accuracy from a pretest to a posttest. Learning and transfer were observed when tailored guidance was given along with the perceptually salient properties. Implications of our quantitative measures of human sensitivity to statistical concepts were discussed.
Science map metaphors: a comparison of network versus hexmap-based visualizations
Most maps of science use a network layout; few use a landscape metaphor. Human users are trained in reading geospatial maps, yet most have a hard time reading even simple networks. Prior work using general networks has shown that map-based visualizations increase recall accuracy of data. This paper reports the result of a comparison of two comparable renderings of the UCSD map of science that are: the original network layout and a novel hexmap that uses a landscape metaphor to layout the 554 subdisciplines grouped into 13 color-coded disciplines of science. Overlaid are HITS metrics that show the impact and transformativeness of different scientific subdisciplines. Both maps support the same interactivity, including search, filter, zoom, panning, and details on demand. Users performed memorization, search, and retrieval tasks using both maps. Results did not show any significant differences in how the two maps were remembered or used by participants. We conclude with a discussion of results and planned future work.
Categorization and Concepts
Concepts are the building blocks of thought. They are critically involved when we reason, make inferences, and try to generalize our previous experiences to new situations. Behind every word in every language lies a concept, although there are concepts, like the small plastic tubes attached to the ends of shoelaces, that we are familiar with and can think about even if we do not know that they are called aglets . Concepts are indispensable to human cognition because they take the “blooming, buzzing confusion” (James, 1890, p. 488) of disorganized sensory experiences and establish order through mental categories. These mental categories allow us to make sense of the world and predict how worldly entities will behave. We see, hear, interpret, remember, understand, and talk about our world through our concepts, and so it is worthy of reflection time to establish where concepts come from, how they work, and how they can best be learned and deployed to suit our cognitive needs.
Can we Reproduce it? Toward the Implementation of good Experimental Methodology in Interdisciplinary Robotics Research
The insufficient level of reproducibility of published experimental results has been identified as a core issue in the field of robotics in recent years. Why is that? First of all, robotics focuses on the abstract concept of computation and the creation of technological artifacts, i.e., software that implements these concepts. Hence, before actually reproducing an experiment, the subject of investigation must be artificially created, which is non-trivial given the inherent complexity [5]. Second, robotics experiments usually include expensive and often customized hardware setups (robots), that are difficult to operate for non-experts. Finally, there is no agreed upon set of methods in order to setup, execute, or (re-)conduct an experiment.
To this end, we introduce an interdisciplinary and geographically distributed collaboration project that aims at implementing good experimental methodology in interdisciplinary robotics research with respect to: a) reproducibility of required technical artifacts, b) explicit and comprehensible experiment design, c) repeatable/reproducible experiment execution, and d) reproducible evaluation of obtained experiment data. The ultimate goal of this collaboration is to reproduce the same experiment in two different laboratories using the same systematic approach which is presented in this work.
Promoting Spontaneous Analogical Transfer by Idealizing Target Representations
Recent results demonstrate that inducing an abstract representation of target analogs at retrieval time aids access to analogous situations with mismatching surface features (i.e., the late abstraction principle). A limitation of current implementations of this principle is that they either require the external provision of target-specific information or demand very high intellectual engagement. Experiment 1 demonstrated that constructing an idealized situation model of a target problem increases the rate of correct solutions compared to constructing either concrete simulations or no simulations. Experiment 2 confirmed that these results were based on an advantage for accessing the base analog, and not merely on an advantage of idealized simulations for understanding the target problem in its own terms. This target idealization strategy has broader applicability than prior interventions based on the late abstraction principle, because it can be achieved by a greater proportion of participants and without the need to receive target-specific information.
Even when people are manipulating algebraic equations, they still associate numerical magnitude with space
The development of symbolic algebra transformed civilization. Since algebra is a recent cultural invention, however, algebraic reasoning must build on a foundation of more basic capacities. Past work suggests that spatial representations of number may be part of that foundation, but recent studies have failed to find relations between spatial-numerical associations and higher mathematical skills. One possible explanation of this failure is that spatial representations of number are not activated during complex mathematics. We tested this possibility by collecting dense behavioral recordings while participants manipulated equations. When interacting with an equation’s greatest [/least] number, participants’ movements were deflected upward [/downward] and rightward [/leftward]. This occurred even when the task was purely algebraic and could thus be solved without attending to magnitude (although the deflection was reduced). This is the first evidence that spatial representations of number are activated during algebra. Algebraic reasoning may require coordinating a variety of spatial processes.
A computer model of context dependent perception in a very simple world
We propose the foundations of a computer model of scientic discovery that takes into account certain psychological aspects of human observation of the world. To this end, we simulate two main components of such a system. The first is a dynamic microworld in which physical events take place, and the second is an observer that visually perceives entities and events in the microworld. For reason of space, this paper focuses only on the starting phase of discovery, which is the relatively simple visual inputs of objects and collisions.
Adapting perception, action, and technology for mathematical reasoning
Formal mathematical reasoning provides an illuminating test case for understanding how humans can think about things that they did not evolve to comprehend. People engage in algebraic reasoning by 1) creating new assemblies of perception and action routines that evolved originally for other purposes (reuse), 2) adapting those routines to better fit the formal requirements of mathematics (adaptation), and 3) designing cultural tools that mesh well with our perception-action routines to create cognitive systems capable of mathematical reasoning (invention). We describe evidence that a major component of proficiency at algebraic reasoning is Rigged Up Perception-Action Systems (RUPAS), via which originally demanding, strategically-controlled cognitive tasks are converted into learned, automatically executed perception and action routines. Informed by RUPAS, we have designed, implemented, and partially assessed a computer-based algebra tutoring system called Graspable Math with an aim toward training learners to develop perception-action routines that are intuitive, efficient, and mathematically valid.
Modeling Mathematical Reasoning as Trained Perception-Action Procedures
We have observed that when people engage in algebraic reasoning, they often perceptually and spatially transform algebraic notations directly rather than first converting the notation to an internal, non spatial representation. We describe empirical evidence for spatial transformations, such as spatially compact grouping, transposition, spatially overlaid intermediate results, cancelling out, swapping, and splitting. This research has led us to understand domain models in mathematics as the deployment of trained and strategically crafted perceptual-motor processes working on grounded and strategically crafted notations. This approach to domain modeling has also motivated us to develop and assess an algebra tutoring system focused on helping students train their perception and action systems to coordinate with each other and formal mathematics. Overall, our laboratory and classroom investigations emphasize the interplay between explicit mathematical understandings and implicit perception action training as having a high potential payoff for making learning more efficient, robust, and broadly applicable.
Mastering algebra retrains the visual system to perceive hierarchical structure in equations
Formal mathematics is a paragon of abstractness. It thus seems natural to assume that the mathematical expert should rely more on symbolic or conceptual processes, and less on perception and action. We argue instead that mathematical proficiency relies on perceptual systems that have been retrained to implement mathematical skills. Specifically, we investigated whether the visual system—in particular, object-based attention—is retrained so that parsing algebraic expressions and evaluating algebraic validity are accomplished by visual processing. Object-based attention occurs when the visual system organizes the world into discrete objects, which then guide the deployment of attention. One classic signature of object-based attention is better perceptual discrimination within, rather than between, visual objects. The current study reports that object-based attention occurs not only for simple shapes but also for symbolic mathematical elements within algebraic expressions—but only among individuals who have mastered the hierarchical syntax of algebra. Moreover, among these individuals, increased object-based attention within algebraic expressions is associated with a better ability to evaluate algebraic validity. These results suggest that, in mastering the rules of algebra, people retrain their visual system to represent and evaluate abstract mathematical structure. We thus argue that algebraic expertise involves the regimentation and reuse of evolutionarily ancient perceptual processes. Our findings implicate the visual system as central to learning and reasoning in mathematics, leading us to favor educational approaches to mathematics and related STEM fields that encourage students to adapt, not abandon, their use of perception.
Index of Supplemental Videos for “An Integrated Computational Model of Perception and Scientific Discovery in a Very Simple World, Aiming at Psychological Realism”
Below is an index of supplemental videos for the manuscript:
Lara-Dammer, F., Hofstadter, D. R., & Goldstone, R. L. (under review).An Integrated Computational Model of Perception and Scientific Discovery in a Very Simple World, Aiming at Psychological Realism
Free Space in a Circle (Tricycle)
Boyle’s Law Sophisticated A (Tricycle)
Boyle’s Law Sophisticated B (Tricycle)
Boyle’s Law Sophisticated C (Tricycle)
Non-ideal Gas A (non-success, Tricycle)
Non-ideal Gas B (Tricycle)
Non-ideal Gas C (Tricycle)
Understanding Noise (Tricycle)
Failed Discovery (Tricycle)
Thinking in Groups A (Tricycle)
Thinking in Groups B (Tricycle)
Informal mechanisms in mathematical cognitive development: The case of arithmetic
The idea that cognitive development involves a shift towards abstraction has a long history in psychology. One incarnation of this idea holds that development in the domain of mathematics involves a shift from non-formal mechanisms to formal rules and axioms. Contrary to this view, the present study provides evidence that reliance on non-formal mechanisms may actually increase with age. Participants – Dutch primary school children – evaluated three-term arithmetic expressions in which violation of formally correct order of evaluation led to errors, termed foil errors. Participants solved the problems as part of their regular mathematics practice through an online study platform, and data were collected from over 50,000 children representing approximately 10% of all primary schools in the Netherlands, suggesting that the results have high external validity. Foil errors were more common for problems in which formally lower-priority sub-expressions were spaced close together, and also for problems in which such sub-expressions were relatively easy to calculate. We interpret these effects as resulting from reliance on two non-formal mechanisms, perceptual grouping and opportunistic selection, to determine order of evaluation. Critically, these effects reliably increased with participants’ grade level, suggesting that these mechanisms are not phased out but actually become more important over development, even when they cause systematic violations of formal rules. This conclusion presents a challenge for the shift towards abstraction view as a description of cognitive development in arithmetic. Implications of this result for educational practice are discussed.
The Simple Advantage in Perceptual and Categorical Generalization
Recent research in relational learning has suggested that simple training instances may lead to better generalization than complex training instances. We examined the perceptual encoding mechanisms that might undergird this Simple advantage by testing category and perceptual learning in adults with simplified and traditional (more complex) Chinese scripts. In Experiment 1, participants learned Chinese characters and their English translations, performed a memorization test, and generalized their learning to the corresponding characters written in the other script. In Experiment 2, we removed the training phase and modified the tests to examine transfer based purely on the perceptual similarities between simplified and traditional characters. We found the simple advantage in both experiments. Training with simplified characters produced better generalization than training with traditional characters when generalization relied on either recognition memory or pure perceptual similarities. On the basis of the results of these two experiments,we propose a simple processmodel to explain the perceptual mechanism that might drive this simple advantage, and in Experiment 3 we tested novel predictions of this model by examining the effect of exposure duration on the simple advantage. We found support for our model that the simple advantage is driven primarily by differences in the perceptual encoding of the information available from simple and complex instances. These findings advance our understanding of how the perceptual features of a learning opportunity interact with domain-general mechanisms to prepare learners for transfer.
The Cognitive Costs of Context: The Effects of Concreteness and Immersiveness in Instructional Examples
Prior research has established that while the use of concrete, familiar examples can provide many important benefits for learning, it is also associated with some serious disadvantages, particularly in learners’ ability to recognize and transfer their knowledge to new analogous situations. However, it is not immediately clear whether this pattern would hold in real world educational contexts, in which the role of such examples in student engagement and ease of processing might be of enough importance to overshadow any potential negative impact. We conducted two experiments in which curriculum-relevant material was presented in natural classroom environments, first with college undergraduates and then with middle-school students. All students in each study received the same relevant content, but the degree of contextualization in these materials was varied between students. In both studies, we found that greater contextualization was associated with poorer transfer performance. We interpret these results as reflecting a greater degree of embeddedness for the knowledge acquired from richer, more concrete materials, such that the underlying principles are represented in a less abstract and generalizable form.
Getting from here to there: Testing the effectiveness of an interactive mathematics intervention embedding perceptual learning
We describe an interactive mathematics technology intervention From Here to There! (FH2T) that was developed by our research team. This dynamic program allows users to manipulate and transform mathematical expressions. In this paper, we present initial findings from a classroom study that investigates whether using FH2T improves learning. We compare learning gains from two different instantiations of FH2T (retrieval practice and fluid visualizations), as well as a control group, and investigate the role of prior knowledge and content exposure in FH2T as possible moderators of learning. Findings, as well as implications for research and practice are discussed.
A computational model for learning structured concepts from physical scenes
Category learning is an essential cognitive mechanism for making sense of the world. Many existing computational category learning models focus on categories that can be represented as feature vectors, and yet a substantial part of the categories we encounter have members with inner structure and inner relationships. We present a novel computational model that perceives and learns structured concepts from physical scenes. The perception and learning processes happen simultaneously and interact with each other. We apply the model to a set of physical categorization tasks and promote specific types of comparisons by manipulating presentation order of examples. We find that these manipulations affect the algorithm similarly to human participants that worked on the same task. Both benefit from juxtaposing examples of different categories – especially ones that are similar to each other. When juxtaposing examples from the same category they do better if the examples are dissimilar to each other.
It does belong together: Cross-modal correspondences influence cross-modal integration during perceptual learning
Experiencing a stimulus in one sensory modality is often associated with an experience in another sensory modality. For instance, seeing a lemon might produce a sensation of sourness. This might indicate some kind of cross-modal correspondence between vision and gustation. The aim of the current study was to provide explore whether such cross-modal correspondences influence cross-modal integration during perceptual learning. To that end, we conducted 2 experiments. Using a speeded classification task, Experiment 1 established a cross-modal correspondence between visual lightness and the frequency of an auditory tone. Using a short-term priming procedure, Experiment 2 showed that manipulation of such cross-modal correspondences led to the creation of a crossmodal unit regardless of the nature of the correspondence (i.e., congruent, Experiment 2a or incongruent, Experiment 2b). However, a comparison of priming-effects sizes suggested that cross-modal correspondences modulate cross-modal integration during learning and thus leading to new learned units that have different stability over time. We discuss the implications of our results for the relation between cross-modal correspondence and perceptual learning in the context of a Bayesian explanation of cross-modal correspondences.
Brain self-portraits: the cognitive science of visualizing neural structure and function
With several large-scale human brain projects currently underway and a range of neuroimaging techniques growing in availability to researchers, the amount and diversity of data relevant for understanding the human brain is increasing rapidly. A complete understanding of the brain must incorporate information about 3D neural location, activity, timing, and task. Data mining, highperformance computing, and visualization can serve as tools that augment human intellect; however, the resulting visualizations must take into account human abilities and limitations to be effective tools for exploration and communication. In this feature review, we discuss key challenges and opportunities that arise when leveraging the sophisticated perceptual and conceptual processing of the human brain to help researchers understand brain structure, function, and behavior.
The extent and arrangement of assistance during training impacts test performance
Various kinds of assistance, including prompts, worked examples, direct instruction, and modeling, are widely provided to learners across educational and training programs. Yet, the effectiveness of assistance during training on long-term learning is widely debated. In the current experiment, we examined how the extent and schedule of assistance during training on a novel mouse movement task impacted unassisted test performance. Learners received different schedules of assistance during training, including constant assistance, no assistance, probabilistic assistance, alternating assistance, and faded assistance. Constant assistance led to better performance during training than no assistance. However, constant assistance during training resulted in the worst unassisted test performance. Faded assistance during training resulted in the best test performance. This suggests that fading may allow learners to create an internal model of the assistance without depending upon the assistance in a manner that impedes successful transfer to unassisted circumstances.
Fitting Perception in and to Cognition
Perceptual modules adapt at evolutionary, lifelong, and moment-to-moment temporal scales to better serve the informational needs of cognizers. Perceptual learning is a powerful way for an individual to become tuned to frequently recurring patterns in its specific local environment that are pertinent to its goals without requiring costly executive control resources to be deployed. Mechanisms like predictive coding, categorical perception, and action-informed vision allow our perceptual systems to interface well with cognition by generating perceptual outputs that are systematically guided by how they will be used. In classic conceptions of perceptual modules, people have access to the modules’ outputs but no ability to adjust their internal workings. However, humans routinely and strategically alter their perceptual systems via training regimes that have predictable and specific outcomes. In fact, employing a combination of strategic and automatic devices for adapting perception is one of the most promising approaches to improving cognition.
Absolute Judgment of Musical Interval Width
When a musical tone is sounded, most listeners are unable to identify its pitch by name. Those listeners who can identify pitches are said to have absolute pitch perception (AP). A limited subset of musicians possesses AP, and it has been debated whether musicians’ AP interferes with their ability to perceive tonal relationships between pitches, or relative pitch (RP). The present study tested musicians’ discrimination of relative pitch categories, or intervals, by placing absolute pitch values in conflict with relative pitch categories. AP listeners perceived intervals categorically, and their judgments were not affected by absolute pitch values. These results indicate that AP listeners do not infer interval identities from the absolute values between tones, and that RP categories are salient musical concepts in both RP and AP musicianship.
Similarity-based ordering of instances for efficient concept learning
Theories in concept learning predict that interleaving instances of different concepts is especially beneficial if the concepts are highly similar to each other, whereas blocking instances belonging to the same concept provides an advantage for learning low-similarity concept structures. This suggests that the performance in concept learning tasks can be improved by grouping the instances of given concepts based on their similarity. To explore this hypothesis, we use Physical Bongard Problems, a rich categorization task with an open feature space, to analyze the combined effects of comparing dissimilar and similar instances within and across categories. We manipulate the within- and between-category similarity of instances presented close to each other in blocked, interleaved and simultaneous presentation schedules. The results show that grouping instances to promote dissimilar within- and similar between category comparisons improves the learning results, to a degree depending on the strategy used by the learner.
Spatial organization and presentation mode in the representation of complex data
External representations are more effective when spatial dimensions are used to represent numeric variables. However, this principle may result in suboptimal representations when the number of numeric variables to be represented is large. To test this possibility, participants studied a set of graphs representing a parametrized function under different parameter values. The graphs were displayed either using a grid organization, with parameter values represented by spatial dimensions (horizontal and vertical position of the graphs), or juxtaposed in a single area, with parameter values represented by non-spatial dimensions (color and texture). Juxtaposed organization led to better learning. However, this advantage was eliminated when the graphs were presented successively rather than simultaneously. The results suggest that juxtaposed organization can improve comprehension of complex data by facilitating comparison between parts of the data. Such organization may be preferable even if it precludes use of spatial dimensions for some numeric variables.
Perceptual Learning
Through perceptual learning, perceptual systems are gradually modified so as to better fit an organism’s environment and frequently occurring needs. We consider psychological and neurophysiological evidence that changes to perception can be early in the stream of information processing. Three specific mechanisms of perceptual learning are described: attentional tuning, unitization, and attribute differentiation. These mechanisms allow organisms to emphasize important perceptual information, to construct single functional units that are activated when a familiar complex configuration arises, and to isolate perceptual attributes that were originally psychologically fused. We describe ways by which people modify their perceptual systems so as to better meet their goals, and the implications of these modifications for the cognitive penetrability of perception, relations between perception and higher-order reasoning, and education.
Learning Visual Units After Brief Visual Experience in 10-month-old Infants
How does perceptual learning take place early in life? Traditionally, researchers have focused on how infants make use of information within displays to organize it, but recently, increasing attention has been paid to the question of how infants perceive objects differently depending upon their recent interactions with the objects. This experiment investigates 10-month-old infants’ use of brief prior experiences with objects to visually organize a display consisting of multiple geometrically-shaped three-dimensional blocks created for this study. After a brief exposure to a multi-part portion of the display, each infant was shown two test events, one of which preserved the unit the infant had seen and the other of which broke that unit. Overall, infants looked longer at the event that broke the unit they had seen prior to testing than the event that preserved that unit, suggesting that infants made use of the brief prior experience to (a) form a cohesive unit of the multi-part portion of the display they saw prior to test and (b) segregate this unit from the rest of the test display. This suggests that infants made inferences about novel parts of the test display based on limited exposure to a subset of the test display. Like adults, infants learn features of the three-dimensional world through their experiences in it.
Concreteness fading in mathematic and science instruction: A systematic review
A longstanding debate concerns the use of concrete versus abstract instructional materials, particularly in domains such as mathematics and science. Although decades of research have focused on the advantages and disadvantages of concrete and abstract materials considered independently, we argue for an approach that moves beyond this dichotomy and combines their advantages. Specifically, we recommend beginning with concrete materials and then explicitly and gradually fading to the more abstract. Theoretical benefits of this “concreteness fading” technique for mathematics and science instruction include: (1) helping learners interpret ambiguous or opaque abstract symbols in terms of well-understood concrete objects, (2) providing embodied perceptual and physical experiences that can ground abstract thinking, (3) enabling learners to build up a store of memorable images that can be used when abstract symbols lose meaning, and (4) guiding learners to strip away extraneous concrete properties and distill the generic, generalizable properties. In these ways, concreteness fading provides advantages that go beyond the sum of the benefits of concrete and abstract materials.
Flexibility in data interpretation: Effects of representational format
Graphs and tables differentially support performance on specific tasks. For tasks requiring reading off single data points, tables are as good as or better than graphs, while for tasks involving relationships among data points, graphs often yield better performance. However, the degree to which graphs and tables support flexibility across a range of tasks is not well-understood. In two experiments, participants detected main and interaction effects in line graphs and tables of bivariate data. Graphs led to more efficient performance, but also lower flexibility, as indicated by a larger discrepancy in performance across tasks. In particular, detection of main effects of variables represented in the graph legend was facilitated relative to detection of main effects of variables represented in the x-axis. Graphs may be a preferable representational format when the desired task or analytical perspective is known in advance, but may also induce greater interpretive bias than tables, necessitating greater care in their use and design.
The structure of integral dimensions: Contrasting topological and Cartesian representations
Diverse evidence shows that perceptually integral dimensions, such as those composing color, are represented holistically. However, the nature of these holistic representations is poorly understood. Extant theories, such as those founded on multidimensional scaling or general recognition theory, model integral stimulus spaces using a Cartesian coordinate system, just as with spaces defined by separable dimensions. This approach entails a rich geometrical structure that has never been questioned but may not be psychologically meaningful for integral dimensions. In particular, Cartesian models carry a notion of orthogonality of component dimensions, such that if 1 dimension is diagnostic for a classification or discrimination task, another can be selected as uniquely irrelevant. This article advances an alternative model in which integral dimensions are characterized as topological spaces. The Cartesian and topological models are tested in a series of experiments using the perceptual-learning phenomenon of dimension differentiation, whereby discrimination training with integral-dimension stimuli can induce an analytic representation of those stimuli. Under the present task design, the 2 models make contrasting predictions regarding the analytic representation that will be learned. Results consistently support the Cartesian model. These findings indicate that perceptual representations of integral dimensions are surprisingly structured, despite their holistic, unanalyzed nature.
When Seeing a Dog Activates the Bark: Multisensory Generalization and Distinctiveness Effects
The goal of the present study was to find evidence for a multisensory generalization effect (i.e., generalization from one sensory modality to another sensory modality). The authors used an innovative paradigm (adapted from Brunel, Labeye, Lesourd, & Versace, 2009) involving three phases: a learning phase, consisting in the categorization of geometrical shapes, which manipulated the rules of association between shapes and a sound feature, and two test phases. The first of these was designed to examine the priming effect of the geometrical shapes seen in the learning phase on target tones (i.e., priming task), while the aim of the second was to examine the probability of recognizing the previously learned geometrical shapes (i.e., recognition task). When a shape category was mostly presented with a sound during learning, all of the primes (including those not presented with a sound in the learning phase) enhanced target processing compared to a condition in which the primes were mostly seen without a sound during learning. A pattern of results consistent with this initial finding was also observed during recognition, with the participants being unable to pick out the shape seen without a sound during the learning phase. Experiment 1 revealed a multisensory generalization effect across the members of a category when the objects belonging to the same category share the same value on the shape dimension. However, a distinctiveness effect was observed when a salient feature distinguished the objects within the category (Experiment 2a vs. 2b).
Benefits of Graphical and Symbolic Representations for Learning and Transfer of Statistical Concepts
Past research suggests that spatial configurations play an important role in graph comprehension. The present study investigates consequences of this fact for the relative utility of graphs and tables for interpreting data. Participants judged presence or absence of various statistical effects in simulated datasets presented in various formats. For the statistical effects introduced earlier in the study, performance was better with graphs than with tables, while for the effect introduced last in the study, this trend reversed. Additionally, in the later sections of the study, responses with graphs, but not tables, reflected increasing influence from the presence of stimulus features which had been relevant earlier in the study, but were no longer relevant. The findings suggest that graphs, relative to tables, may better facilitate perception of complex relationships among data points, but may also bias readers more strongly to favor some perspectives over others when interpreting data.
How to present exemplars of several categories? Interleave during active learning and block during passive learning
Research on how information should be presented during inductive category learning has identified both interleaving of categories and blocking by category as beneficial for learning. Previous work suggests that this mixed evidence can be reconciled by taking into account within- and between-category similarity relations. In this paper we present a new moderating factor. One group of participants studied categories actively, either interleaved or blocked. Another group studied the same categories passively. Results from a subsequent generalization task show that active learning benefits from interleaved presentation while passive learning benefits from blocked presentation.
Grouping by Similarity Helps Concept Learning
In inductive learning, the order in which concept instances are presented plays an important role in learning performance. Theories predict that interleaving instances of different concepts is especially beneficial if the concepts are highly similar to each other, whereas blocking instances belonging to the same concept provides an advantage for learning lowsimilarity concept structures. This leaves open the question of the relative influence of similarity on interleaved versus blocked presentation. To answer this question, we pit withinand between-category similarity effects against each other in a rich categorization task called Physical Bongard Problems. We manipulate the similarity of instances shown temporally close to each other with blocked and interleaved presentation. The results indicate a stronger effect of similarity on interleaving than on blocking. They further show a large benefit of comparing similar between-category instances on concept learning tasks where the feature dimensions are not known in advance but have to be constructed.
An experiment on the cognitive complexity of code
What simple factors impact the cognitive complexity of code? We present an experiment in which participants predict the output of ten small Python programs. Even with such simple programs, we find a complex relationship between code, expertise, and correctness. We use subtle differences between program versions to demonstrate that small notational changes can have profound effects on comprehension. We catalog common errors for each program, and perform an in-depth data analysis to uncover effects on response correctness and speed.
Cognitive Architectures: A Way Forward for the Psychology of Programming
Programming language and library designers often debate the usability of particular design choices. These choices may impact many developers, yet scientific evidence for them is rarely provided. Cognitive models of program comprehension have existed for over thirty years, but lack quantitative definitions of their internal components and processes. To ease the burden of quantifying existing models, we recommend using the ACT-R cognitive architecture: a simulation framework for psychological models. In this paper, we provide a high-level overview of modern cognitive architectures while concentrating on the details of ACT-R. We review an existing quantitative program comprehension model, and consider how it could be simplified and implemented within the ACT-R framework. Lastly, we discuss the challenges and potential benefits associated with building a comprehensive cognitive model on top of a cognitive architecture.
The Importance of Being Interpreted: Grounded Words and Children’s Relational Reasoning
Although young children typically have trouble reasoning relationally, they are aided by the presence of relational words (e.g., Gentner & Rattermann, 1991). They also reason well about commonly experienced event structures (e.g., Fivush, 1984). Relational words may benefit relational reasoning because they activate well-understood event structures. Two candidate hypotheses were tested: (1) the Schema hypothesis, according to which words help relational reasoning because they are grounded in schematized experiences and (2) the Optimal Vagueness hypothesis, by which words benefit relational reasoning because the activated schema is open enough (without too much specificity) so that it can be applied analogically to novel problems. Four experiments examine these two hypotheses by examining how training with a label influences schematic interpretations of a scene, the kinds of scenes that are conducive to schematic interpretations, and whether children must figure out the interpretation themselves to benefit from the act of interpreting a scene as an event. Experiment 1 shows the superiority of schema-evoking words over words that do not connect to schematized experiences. Experiments 2 and 3 further reveal that these words must be applied to vaguely related perceptual instances rather than unrelated or concretely related instances in order to draw attention to relational structure. Experiment 4 provides evidence that even when children do not work out an interpretation for themselves, just the act of interpreting an ambiguous scene is potent for relational generalization. The present results suggest that relational words (and in particular their meanings) are created from the act of interpreting a perceptual situation in the context of a word grounded in meaningful experiences.
Going to extremes: The influence of unsupervised categories on the mental caricaturization of faces and asymmetries in perceptual discrimination
Recent re-analysis of traditional Categorical Perception (CP) effects show that the advantage for between category judgments may be due to asymmetries of within-category judgments (Hanley & Roberson, 2011). This has led to the hypothesis that labels cause CP effects via these asymmetries due to category label uncertainty near the category boundary. In Experiment 1 we demonstrate that these “within-category” asymmetries exist before category training begins. Category learning does increase the within-category asymmetry on a category relevant dimension but equally on an irrelevant dimension. Experiment 2 replicates the asymmetry found in Experiment 1 without training and shows that it does not increase with additional exposure in the absence of category training. We conclude that the within-category asymmetry may be a result of unsupervised learning of stimulus clusters that emphasize extreme instances and that category training increases this caricaturization of stimulus representations.
Download PDF version of this paper
Re-learning labeled categories reveals structured representations
How do people learn to group and re-group objects into labeled categories? In this paper, we examine mechanisms that guide how people re-represent categories. In two experiments, we examine what is easy and what is hard to relearn as people update their knowledge about labeled groups of objects. In Study 1, we test how people learn and re-learn to group objects that share no perceptual features. Data suggest that people easily learn to re-label objects when the category structure remains the same. In Study 2, we test whether more general types of labeling conventions — words that do or do not correspond with object similarities — influence learning and re-learning. Data suggest that people are able to learn either kind of convention and may have trouble switching between them when re-structuring their knowledge. Implications for category learning, second language acquisition and updating representations are discussed.
Download PDF version of this paper
Teaching the Perceptual Structure of Algebraic Expressions: Preliminary Findings from the Pushing Symbols Intervention
We describe an intervention being developed by our research team, Pushing Symbols (PS). This intervention is designed to encourage learners to treat symbol systems as physical objects that move and change over time according to dynamic principles. We provide students with the opportunities to explore algebraic structure by physically manipulating and interacting with concrete and virtual symbolic systems that enforce rules through constraints on physical transformations. Here we present an instantiation of this approach aimed at helping students learn the structure of algebraic notation in general, and in particular learn to simplify like terms. This instantiation combines colored symbol tiles with a new touchscreen software technology adapted from the commercial Algebra Touch software. We present preliminary findings from a study with 70 middle-school students who participated in the PS intervention over a three-hour period.
Download PDF version of this paper
Concepts and Categorization
Issues related to concepts and categorization are nearly ubiquitous in psychology because of people’s natural tendency to perceive a thing as something. We have a powerful impulse to interpret our world. This act of interpretation, an act of “seeing something as X” rather than simply seeing it (Wittgenstein, 1953), is fundamentally an act of categorization. The attraction of research on concepts is that an extremely wide variety of cognitive acts can be understood as categorizations (Murphy, 2002).
Grounded and Transferable Knowledge of Complex Systems using Computer Simulations (supported by IES and NSF)
Click here to learn more about this project
Perceptual Learning
Perceptual learning consists of long-lasting changes to an organismʼs perceptual system that improve its ability to respond to its environment in specific ways. These changes persist over time; more ephemeral perceptual changes are typically considered to be adaptation, attentional processes, or strategy shifts, rather than perceptual learning. These changes are due to environmental inputs; perceptual changes not coupled to the environment are considered maturation, rather than learning. Perceptual learning benefits an organism by tailoring the processes that gather information to the organismʼs needs for and uses of information.
Improving Perception to Make Distant Connections Closer
One of the challenges for perceptually grounded accounts of high-level cognition is to explain how people make connections and draw inferences between situations that superficially have little in common. Evidence suggests that people draw these connections even without having explicit, verbalizable knowledge of their bases. Instead, the connections are based on sub-symbolic representations that are grounded in perception, action, and space. One reason why people are able to spontaneously see relations between situations that initially appear to be unrelated is that their eventual perceptions are not restricted to initial appearances. Training and strategic deployment allow our perceptual processes to deliver outputs that would have otherwise required abstract or formal reasoning. Even without people having any privileged access to the internal operations of perceptual modules, these modules can be systematically altered so as to better subserve our high-level reasoning needs. Moreover, perceptually-based processes can be altered in a number of ways to closely approximate formally sanctioned computations.
Brain Calisthenics
‘Brain Calisthenics for Abstract Ideas’ article in the New York Times (June 6, 2011)
Early perceptual learning
Goldstone, R. L., Son, J. Y, & Byrge, L. (2011). Early perceptual learning. Infancy, 16, 45-51.
Bhatt and Quinn (2011) present a compelling case that human learning is early in two very different, but interacting, senses. Learning is developmentally early in that even infants show strikingly robust adaptation to the structures present in their world. Learning is also early in an information processing sense because infants’ adapt their perceptual encodings and organizations at an early stage of neural processing. Both senses of ”early” speak to the importance of learning because they imply that learners are adapting their representations of their environment in a way that affects all ”down-stream” processing. Developmentally speaking, the learning that an infant enacts serves as the groundwork for all subsequent learning. In terms of information processing, adapting early-stage sensory and perceptual processes in turn affects all subsequent cognitive processes. There is evidence from neuroscience that interactions with an environment do cause early changes to primarysensory cortices (Goldstone, 1998; Vogels, 2010). One might generally suppose that it is advisable to be conservative in making such environment driven cortical changes, given the ripples of influence caused by early learning in both senses. Manipulating grounding representations is a risky proposition. However, the evidence indicates that systems that need to respond effectively to their environment need to engage in both kinds of learning.
Connecting instances to promote children’s relational reasoning
Son, J. Y., Smith, L. B., & Goldstone, R. L. (2011). Connecting instances to promote children’s relational reasoning. Journal of Experimental Child Psychology, 108, 260-277.
The practice of learning from multiple instances seems to allow children to learn about relational structure. The experiments reported here have focused on two issues regarding relational learning from multiple instances: (1) what kind of perceptual situations foster such learning and (2) how particular object properties, such as complexity or similarity, interact with relational learning. Two kinds of perceptual situations were of interest here: simultaneous view, where instances are viewed at once, and sequential view, instances are viewed one at a time, one right after the other. We examine the influence of particular perceptual situations and object properties using two tests of relational reasoning: a common match-to-sample task (where new instances are compared to a common sample) and a variable match-to-sample task (where new instances are compared to a sample that varies on each trial). Experiments 1 and 2 indicate that simultaneous presentation of even highly dissimilar instances, one simple and one complex, effectively connects them together and improves relational generalization in both match-to-sample tasks. Experiment 3 showed simple samples are more effective than complex ones in the common match-to-sample task. However, when one instance is not used a common sample and various pairs of instances are simply compared (Experiment 4), simple and rich instances are equally effective at promoting relational learning. These results bear on our understanding of how children connect instances and how those initial connections affect learning and generalization.
Computing representations for bound and unbound object matching
Shyi, G. S. -W., Goldstone, R. L., Hummel, J. E., & Lin, C. (submitted). Computing representations for bound and unbound object matching.
Five experiments examined the nature of object representation. Participants made same-different judgments between two multipart 3-D objects, according to rules where either the object parts and their spatial relationship had to be considered (role-relevant, RR) or just the object parts (role-irrelevant, RI). Results indicate that it was easiest to judge two identical and orientationally aligned objects according to either rule, followed by judging those that shared identical parts located in different positions according to the RI rule. It was most difficult to judge the latter according to the RR rule when they were misaligned by rotation. These findings lend support to the hypothesis that object representations at the image level, part level, or full structural description level may be computed and used for making same-different judgements. The implications of our findings for object recognition in general and the role of spatial attention in particular are discussed.
Analogical Transfer from a Simulated Physical System
Previous research has consistently found that spontaneous analogical transfer is strongly tied to concrete and contextual similarities between the cases. However, that work has largely failed to acknowledge that the relevant factor in transfer is the similarity between individuals’ mental representations of the situations, rather than the overt similarities between the cases themselves. Across several studies, we find that participants are able to transfer strategies learned from a perceptually concrete simulation of a physical system to a task with very dissimilar content and appearance. This transfer is reflected in better performance on the transfer task when its underlying dynamics are consistent rather than inconsistent with the preceding training task. Our data indicate that transfer in these tasks relies on the perceptual and spatial nature of the training task, but does not depend on direct interaction with the system, with participants performing equally well after simply observing the concrete simulation. We argue that participants in these studies are using the concrete, spatial, dynamic information presented in the training simulation as the basis for a concretely similar mental model of the dissimilar transfer task. Unexpectedly, our data consistently showed that transfer was independent of reported recognition of the analogy between tasks: while such recognition was associated with better overall performance, it was not associated with better transfer (in terms of applying an appropriate strategy). Together, these findings suggest that analogical transfer between overtly dissimilar cases may be much more common—and much more relevant to our cognitive processing—than is generally assumed.
Distinguishing Levels of Grounding that Underlie Transfer of Learning
We find that transfer of learning from a perceptually concrete simulation to an isomorphic but superficially dissimilar text- based problem is sensitive to the congruence between the force dynamics common to both systems and the kinesthetic schema induced via action in the first, perceptually concrete, simulation. Counterintuitively, incompatibility between the force dynamics and the kinesthetic schema has a beneficial effect on transfer, relative to compatibility as well as an unrelated control. We suggest that this incompatibility between action and system dynamics may make the system’s relational structure more salient, leading to a more flexible conceptualization that ultimately benefits transfer. In addition, we suggest that too much “action concreteness” in hands-on learning may actually limit transfer, by fostering an understanding that is tied to that action and therefore less available for transfer in situations where that action is no longer relevant.
Domain-creating constraints
Goldstone, R. L., & Landy, D. H. (2010). Domain-creating constraints. Cognitive Science.
The contributions to this special issue on cognitive development collectively propose ways in which learning involves developing constraints that shape subsequent learning. A learning system must be constrained to learn efficiently, but some of these constraints are themselves learnable. To know how something will behave, a learner must know what kind of thing it is. While this has led previous researchers to argue for domain-specific constraints that are tied to different kinds/domains, an exciting possibility is that kinds/domains themselves can be learned. General cognitive constraints, when combined with rich inputs, can establish domains, rather than these domains necessarily pre-existing prior to learning. Knowledge is structured and richly differentiated, but its “skeleton” must not always be pre-established. Instead, the skeleton may be adapted to fit patterns of co-occurrence, task requirements, and goals. Finally, we argue that for models of development to demonstrate genuine cognitive novelty, it will be helpful for them to move beyond highly pre-processed and symbolic encodings that limit flexibility. We consider two physical models that learn to make tone discriminations. They are mechanistic models that preserve rich spatial, perceptual, dynamic, and concrete information, allowing them to form surprising new classes of hypotheses and encodings.
Perceptual expertise: Bridging brain and behavior
Goldstone, R. L. (2010). Foreward. in I. Gauthier, M. J. Tarr, & D. Bubb (Eds.) Perceptual expertise: Bridging brain and behavior. Oxford, England: Oxford University Press. (pp. v – x).
perceptual learning is important for two reasons—because it is perceptual and because it is learning. Changes to perception are particularly important because they affect all subsequent cognitive processes that occur downstream. There is good evidence, both neurophysiological and behavioral, that perceptual learning can involve early changes to the primary visual, auditory, and somatosensory cortices. One might feel that the early perceptual system ought to be hardwired—it is better not to mess with it if it is going to be depended upon by all processes later in the information processing stream. There is something right with this intuition, but it implicitly buys into a ‘‘stable foundations make strong foundations’’ assumption that it is appropriate for houses of cards, but probably not for flexible cognitive systems. For better models of cognition, we might turn to Birkenstock shoes and suspension bridges, which provide good foundations for their respective feet and cars by flexibly deforming to their charges. Just as a suspension bridge provides better support for cars by conforming to the weight loads, perception supports problem solving and reasoning by conforming to these tasks.
If perceptual learning is crucially perceptual, it is also crucially learning. Consistent with the ripples of downstream influence that early perceptual changes exert, perceptual systems should generally be designed to change slowly and conservatively, so as not to disrupt their downstream consumers. For this reason, this book’s focus on perceptual expertise is appropriate. Expertise typically requires at least 10 years to attain (Ericsson, Krampe, & Tesch-Römer, 1993), sufficient time to influence perception, not simply decision trees or explicitly memorized strategies. The protracted time course of acquiring new perceptual tools is certainly frustrating for those in the business of judging wines, rock samples, cell structures, dives, or manufacturing flaws. One of the reasons why wisdom can’t be simply told (Bransford, Franks, Vye, & Sherwood, 1989) but rather must be lived is that wisdom is frequently perceptual and thus must be built into one’s neurological wiring.
Categorical Perception
Goldstone, R. L., & Hendrickson, A. T. (2010). Categorical Perception. Interdisciplinary Reviews: Cognitive Science, 1, 65-78.
Categorical perception (CP) is the phenomenon by which the categories possessed by an observer influences their perception. Experimentally, CP is revealed when an observer’s ability to make perceptual discriminations between things is better when those things belong to different categories rather than the same category, controlling for the physical difference between the things. We consider several core questions related to CP: Is it caused by innate and/or learned categories, how early in the information processing stream do categories influence perception, and what is the relation between ongoing linguistic processing and CP? CP for both speech and visual entities are surveyed, as are computational and mathematical models of CP. CP is an important phenomenon in cognitive science because it represents an essential adaptation of perception to support categorizations that an organism needs to make. Sensory signals that could be linearly related to physical qualities are warped in a non-linear manner, transforming analog inputs into quasi-digital, quasi-symbolic encodings.
The education of perception
Goldstone, R. L., Landy, D. H., & Son, J. Y. (2010). The education of perception. Topics in Cognitive Science, 2, 265-284.
While the field of perceptual learning has mostly been concerned with low- to middle-level changes to perceptual systems due to experience, we consider high-level perceptual changes that accompany learning in science and mathematics. In science, we explore the transfer of a scientific principle (competitive specialization) across superficially dissimilar pedagogical simulations. We argue that transfer occurs when students develop perceptual interpretations of an initial simulation and simply continue to use the same interpretational bias when interacting with a second simulation. In arithmetic and algebraic reasoning, we find that proficiency in mathematics involves executing spatially explicit transformations to notational elements. People learn to attend mathematical operations in the order in which they should be executed, and the extent to which students employ their perceptual attention in this manner is positively correlated with their mathematical experience. For both science and mathematics, relatively sophisticated performance is achieved not by ignoring perceptual features in favor of deep conceptual features, but rather by adapting perceptual processing so as to conform with and support formally sanctioned responses. These “Rigged Up Perceptual Systems” (RUPS) offer a promising approach to educational reform.
The effect of verbal interference and the internal structure of categories on perceptual discrimination
Hendrickson, A. T., Kachergis, G., Gureckis, T. M., & Goldstone, R. L. (2010). The effect of verbal interference and the internal structure of categories on perceptual discrimination.Proceedings of the Thirty-Second Annual Conference of the Cognitive Science Society. (pp. 1216-1221). Portland, Oregon: Cognitive Science Society.
Recent research has argued that categorization is strongly tied to language processing. For example, language (in the form of verbal category labels) has been shown to influence perceptual discriminations of color (Winawer et al., 2007). However, does this imply that categorical perception is essentially verbally mediated perception? The present study extends recent findings in our lab showing that categorical perception can occur even in the absence of overt labels. In particular, we evaluate the degree to which certain interference tasks (verbal, spatial) reduce the effect of learned categorical perception for complex visual stimuli (faces). Contrary to previous findings, our results show that a verbal interference task does not disrupt learned categorical perception effects for faces. Our results are interpreted in light of the ongoing debate about the role of language in categorization. In particular, we suggest that at least a sub-set of categorical perception effects may be effectively “language-free”.
Proximity and precedence in arithmetic
Landy, D. H., & Goldstone, R. L. (2010). Proximity and precedence in arithmetic. The Quarterly Journal of Experimental Psychology, 63, 1953-1968.
How does the physical structure of an arithmetic expression affect the computational processes engaged in by reasoners? In handwritten arithmetic expressions containing both multiplications and additions, terms that are multiplied are often placed physically closer together than terms that are added. Three experiments evaluate the role such physical factors play in how reasoners construct solutions to simple compound arithmetic expressions (such as “2 + 3 × 4”). Two kinds of influence are found: First, reasoners incorporate the physical size of the expression into numerical responses, tending to give larger responses to more widely spaced problems. Second, reasoners use spatial information as a cue to hierarchical expression structure: More narrowly spaced subproblems within an expression tend to be solved first and tend to be multiplied. Although spatial relationships besides order are entirely formally irrelevant to expression semantics, reasoners systematically use these relationships to support their success with various formal properties.
Partial position transfer in categorical perception learning
Gerganov, A., Grinberg, M., & Goldstone, R. L. (2009). Partial position transfer in categorical perception learning Proceedings of the Thirty-First Annual Conference of the Cognitive Science Society, 1828-1833. Amsterdam, Netherlands: Cognitive Science Society.
Two experiments are reported. The first shows incomplete transfer of explicit categorical learning at a distance of 4.5 degrees of visual angle and the second is a control experiment with a non-learning task. The results suggest that some early visual plasticity takes place even in a simple, explicit categorical learning task. We claim that perceptual learning is a much more common phenomenon than believed before and that it plays an important role in everyday tasks including higher-level learning.
Perceptual unitization in part-whole judgments
Hendrickson, A T., & Goldstone, R. L. (2009). Perceptual unitization in part-whole judgmentsProceedings of the Thirty-First Annual Conference of the Cognitive Science Society, 1084-1089. Amsterdam, Netherlands: Cognitive Science Society.
Categorization relies upon the vocabulary of features that comprise the target objects. Previous theoretical work (Schyns, Goldstone, & Thibaut, 1998) has argued this vocabulary may change through learning and experience. Goldstone (2000) demonstrated this perceptual learning during a categorization task when new features are added that create a single feature unit from multiple existing units. We present two experiments that expand on that work using whole-part judgments (Palmer, 1978) to elicit the feature representation learned through categorization. The implications for different classes of computational models of categorization are discussed.
A well grounded education: The role of perception in science and mathematics
Goldstone, R. L., Landy, D., & Son, J. Y. (2008). A well grounded education: The role of perception in science and mathematics. In M. de Vega, A. Glenberg, & A. Graesser (Eds.) Symbols, embodiment, and meaning. Oxford Press (pp . 327-355).
One of the most important applications of grounded cognition theories is to science and mathematics education where the primary goal is to foster knowledge and skills that are widely transportable to new situations. This presents a challenge to those grounded cognition theories that tightly tie knowledge to the specifics of a single situation. In this chapter, we develop a theory learning that is grounded in perception and interaction, yet also supports transferable knowledge. A first series of studies explores the transfer of complex systems principles across two superficially dissimilar scenarios. The results indicate that students most effectively show transfer by applying previously learned perceptual and interpretational processes to new situations. A second series shows that even when students are solving formal algebra problems, they are greatly influenced by non-symbolic, perceptual grouping factors. We interpret both results as showing that high-level cognition that might seem to involve purely symbolic reasoning is actually driven by perceptual processes. The educational implication is that instruction in science and mathematics should involve not only teaching abstract rules and equations but also training students to perceive and interact with their world.
Learning to see and conceive
Goldstone, R. L., Gerganov, A., Landy, D., & Roberts, M. E. (2008). Learning to see and conceive. In L. Tommasi, M. Peterson, & L. Nadel (Eds.) The New cognitive sciences (part of the Vienna Series in Theoretical Biology). Cambridge, MA.: MIT Press. (pp. 163-188).
Human concept learning depends upon perception. Our concept of Car is built out of perceptual features such as “engine,” “tire,” and “bumper.” However, recent research indicates that the dependency works both ways. We see bumpers and engines in part because we have acquired Car concepts and detected examples of them. Perception both influences and is influenced by the concepts that we learn. We have been exploring the psychological mechanisms by which concepts and perception mutually influence one another, and building computational models to show that the circle of influences is benign rather than vicious.
The effect of internal structure of categories on perception
Gureckis, T. M., & Goldstone, R. L. (2008). The effect of internal structure of categories on perception. Proceedings of the Thirtieth Annual Conference of the Cognitive Science Society,(pp. 1876-1881). Washington, D.C.: Cognitive Science Society.
A novel study is presented that explores the effect that learning internally organized categories has on the ability to subsequently discriminate category members. The results demonstrate the classic categorical perception effect whereby discrimination of stimuli that belong to different categories is improved following training, while the ability to discriminate stimuli belonging to the same category is reduced. We further report a new within-category perceptual effect whereby category members that share the same category label but fall into different sub-clusters within that category are better discriminated than items that share the same category and cluster. The results show that learners are sensitive to multiple sources structure beyond simply the labels provided during supervised training. A computational model is presented to account for the results whereby multiple levels of encoding (i.e., at the item-, cluster-, and category- level) may simultaneously contribute to perception.
Search in external and internal spaces
Hills, T. T., Todd, P. M., & Goldstone, R. L. (2008), Search in external and internal spaces.Psychological Science, 19, 676-682.
There is compelling molecular and behavioral evidence that goal-directed cognition is an evolutionary descendent of spatial-foraging behavior. Across animal species, similar dopaminergic processes modulate between exploratory and exploitative foraging behaviors and control attention. Consequently, we hypothesized that spatialforaging activity could prime attentional cognitive activity. We examined how searching in physical space influences subsequent search in abstract cognitive space by presenting participants with a spatial-foraging task followed by a repeated Scrabble task involving search for words that could be made from letter sets. Participants who searched through clumpier distributions in space behaved as if words were more densely clumped in the Scrabble task. This was not a function of arousal, but was consistent with predictions of optimal-foraging theory. Furthermore, individual differences in exploratory search were conserved across
the two types of tasks. Along with the biological evidence, our results support the idea that there are generalized cognitive search processes.
How the appearance of an operator affects its formal precedence
Landy, D. H., Jones, M. N., & Goldstone, R. L. (2008). How the appearance of an operator affects its formal precedence. Proceedings of the Thirtieth Annual Conference of the Cognitive Science Society, , (pp. 2109-2114). Washington, D.C.: Cognitive Science Society
Two experiments test predictions of a visual process-driven model of multi-term arithmetic computation. The visual process model predicts that attention should be drawn toward multiplication signs more readily than toward plus signs, and that narrow spaces should draw gaze comparably to multiplication signs. Although both of these predictions are verified by behavioral response measures and eye-tracking, the visual process model cannot account for patterns of early looking. The results suggest that people strategically deploy visual computation strategies.
Simplicity and generalization: Short-cutting abstraction in children’s object categorizations
Son, J. Y., Smith, L. B., & Goldstone, R. L. (2008). implicity and generalization: Short-cutting abstraction in children’s object categorizations. Cognition, 108, 626-638.
Development in any domain is often characterized by increasingly abstract representations. Recent evidence in the domain of shape recognition provides one example; between 18 and 24 months children appear to build increasingly abstract representations of object shape [Smith, L. B. (2003). Learning to recognize objects. Psychological Science, 14, 244– 250]. Abstraction is in part simplification because it requires the removal of irrelevant information. At the same time, part of generalization is ignoring irrelevant differences. The resulting prediction is this: simplification may enable generalization. Four experiments asked whether simple training instances could shortcut the process of abstraction and directly promote appropriate generalization. Toddlers were taught novel object categories with either simple or complex training exemplars. We found that children who learned with simple objects were able to generalize according to shape similarity, typically relevant for early object categories, better than those who learned with complex objects. Abstraction is the product of learning; using simplified – already abstracted instances – can short-cut that learning, leading to robust generalization.
Introduction to a special issue on the Development of Categorization
Ionescu, T., & Goldstone, R. L. (2007). Introduction to a special issue on the Development of Categorization, Cognition, Brain, and Behavior, 11, 629-633.
Categorization is indubitably an important cognitive process for humans (as well as other animals, Murai, Kosugi, Tomonaga, Tanaka, Matsuzawa, & Itakura, 2005), one that we constantly engage in to adapt to a very rich environment. We have a powerful impulse to interpret our world. This act of interpretation is fundamentally an act of categorization. We can go back in history at least to Aristotle (see his work on Categories, 350 B.C.E.) and along this way we find discussions of categories often appearing in philosophers’ books. The issue of categorization is also an historically early topic in psychology (see Hull’s experiment in 1920), and a considerable amount of research has been continuously dedicated to it up until the present. One could ask then: Why a special issue on categorization at this point in time? Although the general topic of categorization is venerable, relatively recently we cognitive scientists have changed our view about categorization. We have moved from considering taxonomies (or categories based in logic) as the “real,” mature kind of categorization to understanding that there are multiple kinds of similarities that are taken into account when one groups items (Barsalou, 1993, 2003; Medin, Goldstone, & Gentner, 1993; Ross & Murphy, 1999).
Formal notations are diagrams: Evidence from a production task
Landy, D., & Goldstone, R. L. (2007). Formal notations are diagrams: Evidence from a production task. Memory & Cognition, 35, 2033-2040
Although a general sense of the magnitude, quantity, or numerosity of objects is common both in untrained people and in animals, the abilities to deal exactly with large quantities and to reason precisely in complex but well-specified situations—to behave formally, that is—are skills unique to people trained in symbolic notations. These symbolic notations employ typically complex, hierarchically embedded structures, which all extant analyses assume are constructed by concatenative, rule-based processes. The primary goal of this article is to establish, using behavioral measures on naturalistic tasks, that the some of the same cognitive resources involved in representing spatial relations and proximities are also involved in representing symbolic notations: in short, formal notations are a kind of diagram. We examine self-generated productions in the domains of handwritten arithmetic expressions and typewritten statements in a formal logic. In both tasks, we find substantial evidence for spatial representational schemes even in these highly symbolic domains.
Spatial constraints on visual statistical learning of multi-element displays
Conway, C., Goldstone, R. L., & Christiansen, M. (2007). Spatial constraints on visual statistical learning of multi-element displays. Proceedings of the Twenty-ninth Annual Conference of the Cognitive Science Society. (pp. 185-190). Nashville, TN: Cognitive Science Society.
Visual statistical learning allows observers to extract high-level structure from visual scenes (Fiser & Aslin, 2001). Previous work has explored the types of statistical computations afforded but has not addressed to what extent learning results in unbound versus spatially bound representations of element cooccurrences. We explored these two possibilities using an unsupervised learning task with adult participants who observed complex multi-element scenes embedded with consistently paired elements. If learning is mediated by unconstrained associative learning mechanisms, then learning the element pairings may depend only on the co-occurrence of the elements in the scenes, without regard to their specific spatial arrangements. If learning is perceptually constrained, cooccurring elements ought to form perceptual units specific to their observed spatial arrangements. Results showed that participants learned the statistical structure of element cooccurrences in a spatial-specific manner, showing that visual statistical learning is perceptually constrained by spatial grouping principles.
Simulating conceptually-guided perceptual learning
Gerganov, A., Grinberg, M., Quinn, P. C., & Goldstone, R. L. (2007). Simulating conceptually-guided perceptual learning. Proceedings of the Twenty-ninth Annual Conference of the Cognitive Science Society. (pp. 287-292). Nashville, TN: Cognitive Science Society.
Visual statistical learning allows observers to extract high-level structure from visual scenes (Fiser & Aslin, 2001). Previous work has explored the types of statistical computations afforded but has not addressed to what extent learning results in unbound versus spatially bound representations of element cooccurrences. We explored these two possibilities using an unsupervised learning task with adult participants who observed complex multi-element scenes embedded with consistently paired elements. If learning is mediated by unconstrained associative learning mechanisms, then learning the element pairings may depend only on the co-occurrence of the elements in the scenes, without regard to their specific spatial arrangements. If learning is perceptually constrained, cooccurring elements ought to form perceptual units specific to their observed spatial arrangements. Results showed that participants learned the statistical structure of element cooccurrences in a spatial-specific manner, showing that visual statistical learning is perceptually constrained by spatial grouping principles.
Priming and conservation between spatial and cognitive search
Hills, T., Todd, P., & Goldstone, R. L. (2007). Priming and conservation between spatial and cognitive search. Proceedings of the Twenty-ninth Annual Conference of the Cognitive Science Society. (359-364). Nashville, TN: Cognitive Science Society.
Visual statistical learning allows observers to extract high-level structure from visual scenes (Fiser & Aslin, 2001). Previous work has explored the types of statistical computations afforded but has not addressed to what extent learning results in unbound versus spatially bound representations of element cooccurrences. We explored these two possibilities using an unsupervised learning task with adult participants who observed complex multi-element scenes embedded with consistently paired elements. If learning is mediated by unconstrained associative learning mechanisms, then learning the element pairings may depend only on the co-occurrence of the elements in the scenes, without regard to their specific spatial arrangements. If learning is perceptually constrained, cooccurring elements ought to form perceptual units specific to their observed spatial arrangements. Results showed that participants learned the statistical structure of element cooccurrences in a spatial-specific manner, showing that visual statistical learning is perceptually constrained by spatial grouping principles.
How abstract is symbolic thought?
Landy, D. & Goldstone, R. L. (2007). How abstract is symbolic thought? Journal of Experimental Psychology: Learning, Memory, & Cognition, 33, 720-733.
In 4 experiments, the authors explored the role of visual layout in rule-based syntactic judgments. Participants judged the validity of a set of algebraic equations that tested their ability to apply the order of operations. In each experiment, a nonmathematical grouping pressure was manipulated to support or interfere with the mathematical convention. Despite the formal irrelevance of these grouping manipulations, accuracy in all experiments was highest when the nonmathematical pressure supported the mathematical grouping. The increase was significantly greater when the correct judgment depended on the order of operator precedence. The result that visual perception impacts rule application in mathematics has broad implications for relational reasoning in general. The authors conclude that formally symbolic reasoning is more visual than is usually proposed.
The alignment of ordering and space in arithmetic computation
Landy, D. & Goldstone, R. L. (2007). The alignment of ordering and space in arithmetic computation. Proceedings of the Twenty-ninth Annual Conference of the Cognitive Science Society. (pp. 437-442). Nashville, TN: Cognitive Science Society.
In 4 experiments, the authors explored the role of visual layout in rule-based syntactic judgments. Participants judged the validity of a set of algebraic equations that tested their ability to apply the order of operations. In each experiment, a nonmathematical grouping pressure was manipulated to support or interfere with the mathematical convention. Despite the formal irrelevance of these grouping manipulations, accuracy in all experiments was highest when the nonmathematical pressure supported the mathematical grouping. The increase was significantly greater when the correct judgment depended on the order of operator precedence. The result that visual perception impacts rule application in mathematics has broad implications for relational reasoning in general. The authors conclude that formally symbolic reasoning is more visual than is usually proposed.
How space guides interpretation of a novel mathematical system
Landy, D. & Goldstone, R. L. (2007). How space guides interpretation of a novel mathematical system. Proceedings of the Twenty-ninth Annual Conference of the Cognitive Science Society.(pp. 431-436). Nashville, TN: Cognitive Science Society.
This paper investigates how people build interpretations of compound expressions in a novel formal system. In traditional arithmetic, interpretations are guided by an order of precedence convention (times and division precede addition and subtraction). This order is supported by alignment with the order of precedence. In the experiment described here, participants learned computation tables of two simple novel operators, and then were asked to discover a precedence order between them. The operators were presented with a physical spacing convention that either aligned with the precedence order, opposed it, or randomly opposed or aligned with the precedence order. Participants were more likely to reach a criterion of successful performance when the order of operations aligned with the precedence order, and did so more quickly than either other group. The results indicate that reasoners integrate salient perceptual cues with formal knowledge following particular conventions, even on novel systems.
Grounding symbol structures in space: Formal notations as diagrams
[Winner of the 2007 Marr Prize for Best Student Paper at the 2007 Meeting of the Cognitive Science Society]
Although a general sense of the magnitude, quantity, or numerosity is common both in untrained people and animals, the abilities to deal exactly with large quantities and to reason precisely in complex but well-specified situations—to behave formally, that is—are skills unique to people trained in symbolic notations. These symbolic notations employ typically complex, hierarchically embedded structures, which all extant analyses assume are the product of concatenative, rule-based processes. The primary goal of this article is to establish, using behavioral measures on naturalistic tasks, that the some of the same cognitive resources involved in representing spatial relations and proximities are also involved in representing symbolic notations. In short, formal notations are used as a kind of diagram. We examine selfgenerated productions in the domains of handwritten arithmetic expressions and typewritten statements in a formal logic. In both tasks, we find substantial evidence for spatial processes even in these highly symbolic domains.
The interplay between perceptual organization and categorization in the representation of complex visual patterns by young infants
Quinn, P. C., Schyns, P. G., & Goldstone, R. L. (2006). The interplay between perceptual organization and categorization in the representation of complex visual patterns by young infants.Journal of Experimental Child Psychology, 95, 116-127.
The relation between perceptual organization and categorization processes in 3- and 4-month-olds was explored. The question was whether an invariant part abstracted during category learning could interfere with Gestalt organizational processes. A 2003 study by Quinn and Schyns had reported that an initial category familiarization experience in which infants were presented with visual patterns consisting of a pacman shape and a complex polygon could interfere with infants’ subsequent good continuationbased parsing of a circle from visual patterns consisting of a circle and a complex polygon. However, an alternative noninterference explanation for the results was possible because the pacman had been presented with greater frequency and duration than had the circle. The current study repeated Quinn and Schyns’s procedure but provided an equivalent number of familiarization trials and duration of study time for the infants to process the pacman during initial familiarization and the circle during subsequent familiarization. The results replicated the previous Wndings of Quinn and Schyns. The data are consistent with the interference account and suggest that a cognitive system of adaptable feature creation can take precedence over organizational principles with which a perceptual system comes preequipped.
Asymmetries in categorization, perceptual discrimination, and visual search for reference and non-reference exemplars
Corneille, O., Goldstone, R. L., Queller, S., & Potter, T. (2006). Asymmetries in categorization, perceptual discrimination, and visual search for reference and non-reference exemplars. Memory & Cognition ,34, 556-567.
Two studies examined the representation, treatment, and attention, devoted to the members of reference (i.e., Club members) and non-reference (i.e., Not-Club members) categories. Consistent with prior work on category interrelatedness (e.g. Goldstone, 1996; Goldstone, Steyvers, & Rogosky, 2003), the findings reveal the existence of asymmetric representations for reference and non-reference categories which, however, decreased as expertise and familiarity with the categories increased (Experiment 1 and Experiment 2). Participants also more readily judged two reference than two non-reference exemplars as being the same (Experiment 1), and were better at detecting reference than non-reference exemplars in a set of novel, category-unspecified,exemplars (Experiment 2). These findings provide evidence for the existence of a feature asymmetry in the representation and treatment of exemplars from reference and non-reference categories. Membership in a reference category acts as a salient feature, thereby increasing the perceived similarity and detection of faces that belong in the reference, compared to nonreference, category.
Generalizing from simple instances: An uncomplicated lesson from kids learning objects categories
Son, J. Y., Smith, L. B., & Goldstone, R. L. (2006). Generalizing from simple instances: An uncomplicated lesson from kids learning objects categories. Proceedings of the Twenty-eighth Annual Conference of the Cognitive Science Society. Hillsdale, New Jersey: Lawrence Erlbaum Associates. (2174-2179)
Abstraction is the process of stripping away irrelevant information so that learners can generalize on relevant similarities. Can we shortcut this process by directly teaching abstractions in the form of simplified instances? We tested this prediction in the domain of shape-based generalization and found that young children were able to generalize better when taught with simplified shapes rather than complex detailed ones. Simplicity during training allowed shape novices to generalize like shape experts.
How we learn about things we don’t already understand
Landy, D., & Goldstone, R. L. (2005). How we learn about things we don’t already understand. Journal of Experimental and Theoretical Artificial Intelligence, 17, 343-369.
The computation-as-cognition metaphor requires that all cognitive objects are constructed from a fixed set of basic primitives; prominent models of cognition and perception try to provide that fixed set. Despite this effort, however, there are no extant computational models that can actually generate complex concepts and processes from simple and generic basic sets, and there are good reasons to wonder whether such models may be forthcoming. We suggest that one can have the benefits of computationalism without a commitment to fixed feature sets, by postulating processes that slowly develop special-purpose feature languages, from which knowledge is constructed. This provides an alternative to the fixed-model conception without radical anti-representationlism. Substantial evidence suggests that such feature development adaptation actually occurs in the perceptual learning that accompanies category learning. Given the existence of robust methods for novel feature creation, the assumption of a fixed basis set of primitives as psychologically necessary is at best premature. Methods of primitive construction include (a) perceptual sensitization to physical stimuli, (b) unitization and differentiation of existing (non-psychological) stimulus elements into novel psychological primitives, guided by the current set of features, and (c) the intelligent selection of novel inputs, which in turn guides the automatic construction of new primitive concepts. Modeling the grounding of concepts as sensitivity to physical properties reframes the question of concept construction from the generation of an appropriate composition of sensations, to the tuning of detectors to appropriate circumstances.
Differentiation for novel dimensions
Hockema, S. A., Blair, M. R., & Goldstone, R. L. (2005). Differentiation for novel dimensions. Proceedings of the Twenty-seventh Annual Conference of the Cognitive Science Society. Hillsdale, New Jersey: Lawrence Erlbaum Associates. (pp. 953-958)
Two experiments are reported that provide evidence for perceptual differentiation between a pair of novel, integral dimensions, in contrast to previous attempts that failed to differentiate these same two dimensions (Op de Beeck,Wagemans, & Vogels, 2003). In Experiment 1, an acquired distinctiveness effect was created on the category-relevant dimension through a categorization training regimen that gradually increased in difficulty. Response times for correct trials were faster across the category boundary. This effect was replicated in Experiment 2 using a new training procedure where participants had to predict category boundaries while watching an animation in which shapes transformed along the category-relevant dimension. Furthermore, the accuracy results of Experiment 2 also indicated that discriminability was changed on the category-relevant dimension relative to the irrelevant dimension across the entire range of the dimension, not just at the category boundary.
Relational reasoning is in the eyes of the beholder: How global perceptual groups aid and impair algebraic evaluations
Landy, D., & Goldstone, R. L. (2005). Relational reasoning is in the eyes of the beholder: How global perceptual groups aid and impair algebraic evaluations.Proceedings of the Twenty-seventh Annual Conference of the Cognitive Science Society. Hillsdale, New Jersey: Lawrence Erlbaum Associates. (pp. 2509)
Relational reasoning—reasoning that depends on the interactions of multiple elements, rather than on the intrinsic properties of the elements—is both ubiquitous and challenging. For example, children find it difficult to respond to relational commonalities when object-based similarities are present (Gentner & Rattermann, 1991). Since overt symbol systems such as algebra are external constructs, their terms can contain perceptual regularities. Models of symbolic reasoning, however, typically ignore perceptual regularities (Anderson, in press). It is reasonable to wonder whether people make use of available domaingeneral grouping processes when parsing mathematical structures.
The purpose of the experiments described here is to evaluate whether algebraic grouping is sensitive to visual grouping. If processing is strictly symbolic, then the manipulation of perceptual regularities should not affect judgments; however, if people use visual grouping to help them parse expressions, then they should make more errors in cases where the perceptual grouping gives an incorrect answer, and be more accurate when visual grouping supports the standard order of operations.
Perceptual and semantic reorganization during category learning
Goldstone, R. L., Rogosky, B. J., Pevtzow, R., & Blair, M. (2005). Perceptual and semantic reorganization during category learning. In H. Cohen & C. Lefebvre (Eds.)Handbook of Categorization in Cognitive Science. (pp. 651-678). Amsterdam: Elsevier.
Category learning not only depends upon perceptual and semantic representations; it also leads to the generation of these representations. We describe two series of experiments that demonstrate how categorization experience alters, rather than simply uses, descriptions of objects. In the first series, participants first learned to categorize objects on the basis of particular sets of line segments. Subsequently, they were given a perceptual part-whole judgment task. Categorization training influenced participants’ partwhole judgments, indicating that whole objects were more likely to be broken down into parts that were relevant during categorization. In the second series, correlations were created or broken between semantic features of word concepts (e.g., ferocious vs. timid, and group-oriented vs. solitary animals). The best transfer was found between category learning tasks that shared the same semantic organization of concepts. Together, the experiments support models of category learning that simultaneously create the elements of categorized objects’ descriptions and associate those elements with categories.
Adaptation of perceptual and semantic features
Rogosky, B. J., & Goldstone, R. L. (2005). Adaptation of perceptual and semantic features. In L. A. Carlson & E. van der Zee (Eds.), Functional features in language and space: Insights from perception, categorization and development. (pp. 257-273). Oxford, England: Oxford University Press.
This chapter examines the role of feature in theories of concepts, perception, and language. The authors define features as psychological representations of properties in the world that can be processed independently of other properties and that are relevant to a task, such as categorization. They discuss the classic view of features as entities that do not change over time. They argue for an alternative view in which features are created and adapted according to the immediate goals and context of tasks, and over longer time periods in terms of perceptual and conceptual learning and development. The authors also distinguish pairs of dimensions in terms of whether the dimensions can be processed separately (i.e. either dimension can be attended independently of the other) or integrally (i.e. the dimensions cannot be processed independently). They present a study of the classification of linguistic stimuli according to rules based on semantic features (e.g. ferocity and socialness of animals). The results indicate that changes in the integral processing of the dimensions can be induced by tasks that favor the separate processing of one dimenion. The findings support the authors’ claim that, like perceptual features, semantic features can be adapted during learning.
The transfer of scientific principles using concrete and idealized simulations
Participants in two experiments interacted with computer simulations designed to foster understanding of scientific principles governing complex adaptive systems. The quality of participants’ transportable understanding was measured by the amount of transfer between two simulations governed by the same principle. The perceptual concreteness of the elements within the first simulation was manipulated. The elements either remained concrete throughout the simulation, remained idealized, or switched midway into the simulation from concrete to idealized or vice versa. Transfer was better when the appearance of the elements switched, consistent with theories predicting more general schemas when the schemas are multiply instantiated. The best transfer was observed when originally concrete elements became idealized. These results are interpreted in terms of tradeoffs between grounded, concrete construals of simulations and more abstract, transportable construals. Progressive idealization (“Concreteness fading”) allows originally grounded and interpretable principles to become less tied to specific contexts and hence more transferable.
‘Believing Is Seeing’ article (June, 2004)
‘Believing Is Seeing’ article (June, 2004)
Believing is seeing
Goldstone, R. L. (2004). Believing is seeing. American Psychological Society Observer, 17, 23-26.
Human concept learning clearly depends upon perception. Our concept of “gerbil” is built out of perceptual features such as “furry,” “small,” and “four-legged.” However, recent research has found that the dependency works both ways. Perception not only influences, but is influenced by, the concepts that we learn. Our laboratory has been exploring the psychological mechanisms by which concepts and perception mutually influence one another, and building computational models to show that the circle of influences is benign rather than vicious.
Access this paper on-line at APS
Access this paper on-line at APS
Learning to perceive while perceiving to learn
The external world must be filtered through our perceptual systems before it can have an impact upon us. That is, the world we experience is formed by our perceptual processing. However, it is not viciously circular to argue that our perceptual systems are reciprocally formed by our experiences. In fact, it is because our experiences are necessarily based on our perceptual systems that these perceptual systems must be shaped so that our experiences are appropriate and useful for dealing with our world.
In what follows, I will argue that the “building blocks” an observer uses for construing their world depends on the observer’s history, training, and acculturation. These factors, together with psychophysical constraints, mold one’s set of building blocks. Researchers who have proposed fixed sets of hard-wired primitives are exactly right in one sense — the combinatorics of objects, words, scenes, and scenarios strongly favor componential representations. However, this does not necessitate that the components be hard-wired. By developing new components to subserve particular tasks and environments, a newly important discrimination can generate building blocks that are tailored for the discrimination. Adaptive building blocks are likely to be efficient because they can be optimized for idiosyncratic needs and environments.
Conceptual development from origins to asymptotes
Scientists studying adult concept learning are typically careful to analyze the entire pattern of responses given across all of the trials of an experiment. Often times, the early trials are the most diagnostic because categorization accuracy quickly reaches an asymptote. We take some pride in tackling the hard problem of accounting for adaptive processes that account for category learning, unlike many psychophysicists, who simply throw out the first 1000 trials because steady-state performance has not yet been reached. However, lest we grow too smug, the chapters of this book provide a great service by reminding us that even though we analyze the very first trial of our experiment, we are still studying conceptual change that occurs almost imperceptibly close to the asymptote. By the time that our 20-year-old subjects come to our laboratories, they have learned the majority of the concepts that they will ever learn and virtually all of their truly foundational concepts. Relatively brief laboratory training suffices to teach students the rule “Circle Above Square” (Bruner, Goodnow, & Austin, 1956), a particular configuration of 9 dots (Posner & Keele, 1968), or a new fact such as that grebes are birds, but this rapid learning is only possible because it builds upon a longer and more profound process by which concepts such as Above (Quinn, this volume), Bird (Mervis, Pani & Pani, this volume), Animal (Mareschal, this volume; Mandler, this volume), and Animacy (Gelman & Koenig, this volume; Rakison, this volume) are learned.
Those of us who want to develop theories of the learning and representation of adult concepts cannot afford to remain blind to the conceptual development that makes possible adult concept use. This life-long learning provides us with the fundamental representations that we subsequently combine and tweak. In assessing the contribution of developmental research on concepts and categories to our general understanding of human concepts, we will ask four questions: what are concepts; what is the relation between perception and concepts; what are the constraints on concept learning; and what are promising future directions for research on concepts?
Featural processing in face preferences
Halberstadt, J., Goldstone, R. L., & Levine, G. M. (2003). Featural Processing in Face Preferences. Journal of Experimental Social Psychology, 39, 270-278.
Two experiments examined how practice and time pressure influence holistic processing, defined as the relative importance of feature interactions, in a face preference task. Participants rated 32 cartoon faces that varied along five dichotomous features (Experiment 1) or 27 realistic morphed faces that varied along three trichotomous dimensions (Experiment 2), under high and low time pressure (operationalized as a short versus long stimulus presentation time), over a series of experimental blocks. In both experiments, the overall importance of facial features, but not of feature interactions, increased over blocks and, in one condition of Experiment 1, under high versus low time pressure. Analyses of idiosyncratic importance indicated that the feature effects were due to the increasing importance of participants’ idiosyncratically most influential features. Functional differences between face preferences and face recognition are offered to explain and predict when facial features will be processed independently versus holistically.
Altering object representations through category learning
Goldstone, R. L, Lippa, Y., & Shiffrin, R. M. (2001). Altering object representations through category learning. Cognition, 78, 27-43.
Previous research has shown that objects that are grouped together in the same category become more similar to each other and that objects that are grouped in different categories become increasingly dissimilar, as measured by similarity ratings and psychophysical discriminations. These findings are consistent with two theories of the influence of concept learning on similarity. By a strategic judgment bias account, the categories associated with objects are explicitly used as cues for determining similarity, and objects that are categorized together are judged to be more similar because similarity is not only a function of the objects themselves, but also the objectsí category labels. By a representational change account, category learning alters the description of the objects themselves, emphasizing properties that are relevant for categorization. A new method for distinguishing between these accounts is introduced which measures the difference between the similarity ratings of categorized objects to a neutral object. The results indicate both strategic biases based on category labels and genuine representational change, with the strategic bias affecting mostly objects belonging to different categories and the representational change affecting mostly objects belonging to the same category.
View This Paper (HTML)
Download PDF version of this paper (Download free reader from Adobe for this file)
The sensitization and differentiation of dimensions during category learning
Goldstone, R. L, & Steyvers, M. (2001). The Sensitization and Differentiation of Dimensions During Category Learning. Journal of Experimental Psychology: General, 130,116-139.
The reported experiments explore two mechanisms by which object descriptions are flexibly adapted to support concept learning: selective attention and dimension differentiation. Arbitrary dimensions were created by blending photographs of faces in different proportions, and mixing these blends together. Consistent with learned selective attention, positive transfer was found when initial and final categorizations shared either relevant or irrelevant dimensions, and negative transfer was found when previously relevant dimensions became irrelevant. Unexpectedly good transfer was observed when both irrelevant dimensions became relevant and relevant dimensions became irrelevant, and was explained in terms of participants learning to isolate one dimension from another. This account was further supported by experiments indicating that conditions expected to produce positive transfer via dimension differentiation produced better transfer than conditions expected to produce positive transfer via selective attention, but only when stimuli were composed of highly integral and overlapping dimensions. We discuss the relation between dimension differentiation and selective attention, mechanisms that may underlie these processes, and implications for category learning research.
The acquisition of automatic response biases through stimulus-response mapping and categorization determined by a compatibility task
Lippa, Y., & Goldstone, R. L. (2001). The Acquisition of Automatic Response Biases through Stimulus-Response Mapping and Categorization Determined by a Compatibility Task. Memory & Cognition, 29, 1051-1060
Experiments explored whether spatially neutral stimuli acquire the ability to automatically elicit spatial responses. In Experiment 1, participants associated line-drawings with either left or right key presses. Subsequently, the pictures were used in a Simon task wherein participants made left and right key presses based on the color of the picture, ignoring its shape. Participants responded more quickly when the key press previously associated with the picture matched, rather than mismatched, the response required by the picture’s color. In Experiment 2, participants learned response categories that grouped spatially ambiguous line-drawings together with pictures of left- and right-pointing arrows and fingers. A subsequent Simon task again yielded compatibility effects, indicating that the spatially ambiguous pictures inherited the response biases of the other objects in their category. Thus, responses directly associated with shapes, and indirectly associated with shapes by category membership, are both automatically triggered even when the responses are irrelevant and inappropriate.
View This Paper (HTML) [TEXT ONLY, no figures]
Download PDF version of this paper
Unitization during category learning
Goldstone, R. L. (2000). Unitization during Category Learning. Journal of Experimental Psychology: Human Perception and Performance, 26, 86-112
Five experiments explored the question of whether new perceptual units can be developed if they are diagnostic for a category learning task, and if so, what are the constraints on this unitization process? During category learning, participants were required to attend either a single component or a conjunction of five components in order to correctly categorize an object. In Experiments 1-4, some evidence for unitization was found in that the conjunctive task becomes much easier with practice, and this improvement was not found for the single component task, or for conjunctive tasks where the components cannot be unitized. Influences of component order (Experiment 1), component contiguity (Experiment 2), component proximity (Experiment 3), and number of components (Experiment 4) on practice effects were found. Using a Fourier Transformation method for deconvolving response times (Experiment 5), prolonged practice effects yielded responses that were faster than expected by analytic model that integrate evidence from independently perceived components.
View This Paper (HTML) [with some figures]
Download PDF version of this paper
A neural network model of concept-influenced segmentation
Goldstone, R. L. (2000). A neural network model of concept-influenced segmentation. Proceedings of the Twenty-second Annual Conference of the Cognitive Science Society. Hillsdale, New Jersey: Lawrence Erlbaum Associates. (pp. 172-177).
Several models of categorization assume that fixed perceptual representations are combined together to determine categorizations. This research explores the possibility that categorization experience alters, rather than simply uses, descriptions of objects. Based on results from human experiments, a model is presented in which a competitive learning network is first given categorization training, and then is given a subsequent segmentation task, using the same network weights. Category learning establishes detectors for stimulus parts that are diagnostic, and these detectors, once established, bias the interpretation of subsequent objects to be segmented.
Interactions between perceptual and conceptual learning
Goldstone, R. L., Steyvers, M., Spencer-Smith, J., & Kersten, A. (2000). Interactions between perceptual and conceptual learning. in E. Diettrich & A. B. Markman (eds.) Cognitive Dynamics: Conceptual Change in Humans and Machines. Mahwah, New Jersey: Lawrence Erlbaum Associates. (pp. 191-228).
Confusions arise when ‘stable’ is equated with ‘foundational.’ Spurred on by the image of a house`s foundation, it is tempting to think that something provides effective support to the extent that it is rigid and stable. We will argue that when considering the role of perception in grounding our concepts, exactly the opposite is true. Our perceptual system supports our ability to acquire new concepts by being flexibly tuned to these concepts. Whereas the concepts that we learn are certainly influenced by our perceptual representations, we will argue that these perceptual representations are also influenced by the learned concepts. In keeping with one of the central themes of this book, behavioral adaptability is completely consistent with representationalism. In fact, the most straightforward account of our experimental results is that concept learning can produce changes in perceptual representations, the ‘vocabulary’ of perceptual features, that are used by subsequent tasks.
This chapter reviews theoretical and empirical evidence that perceptual vocabularies used to describe visual objects are flexibly adapted to the demands of their user. We will extend arguments made elsewhere for adaptive perceptual representations (Goldstone, Schyns, & Medin, in press; Schyns, Goldstone, & Thibaut, in press), and discuss research from our laboratory illustrating specific interactions between perceptual and conceptual learning. We will describe computer simulations that provide accounts of these interactions using neural network models. These models have detectors that become increasingly tuned to the set of perceptual features that support concept learning. The bulk of the chapter will be organized around mechanisms of human perceptual learning, and computer simulations of these mechanisms.
Access site where this paper can be downloaded
Download PDF version of this paper
Perceptual Learning
Goldstone, R. L. (1998). Perceptual Learning. Annual Review of Psychology, 49, 585-612.
Perceptual learning involves relatively long-lasting changes to an organism`s perceptual system that improve its ability to respond to its environment. Four mechanisms of perceptual learning are discussed: attention weighting, imprinting, differentiation, and unitization. By attention weighting, perception becomes adapted to tasks and environments by increasing the attention paid to important dimensions and features. By imprinting, receptors are developed that are specialized for stimuli or parts of a stimuli. By differentiation, stimuli that were once indistinguishable become psychologically separated. By unitization, tasks that originally required detection of several parts come to be accomplished by detecting a single constructed unit representing a complex configuration. Research from cognitive psychology, psychophysics, neuroscience, expert/novice differences, development, computer science, and cross-cultural differences is described that relates to these mechanisms. The locus, limits, and applications of perceptual learning are also discussed.
Reuniting perception and conception.
(reprinted as: Goldstone, R. L., & Barsalou, L. (1998). Reuniting perception and conception. In S. A. Sloman and L. J. Rips (Eds.) Similarity and symbols in human thinking. (pp. 145-176). Cambridge, MA: MIT Press)
Work in philosophy and psychology has argued for a dissociation between perceptually-based similarity and higher-level rules in conceptual thought. Although such a dissociation may be justified at times, our goal is to illustrate ways in which conceptual processing is grounded in perception, both for perceptual similarity and abstract rules. We discuss the advantages, power, and influences of perceptually-based representations. First, many of the properties associated with amodal symbol systems (e.g. productivity and generativity) can be achieved with perceptually-based systems as well. Second, relatively raw perceptual representations are powerful because they can implicitly represent properties in an analog fashion. Third, perception naturally provides impressions of overall similarity, exactly the type of similarity useful for establishing many common categories. Fourth, perceptual similarity is not static but becomes tuned over time to conceptual demands. Fifth, the original motivation or basis for sophisticated cognition is often less sophisticated perceptual similarity. Sixth, perceptual simulation occurs even in conceptual tasks that have no explicit perceptual demands. Parallels between perceptual and conceptual processes suggest that many mechanisms typically associated with abstract thought are also present in perception, and that perceptual processes provide useful mechanisms that may be coopted by abstract thought.
Two Competing Attentional Mechanisms in Category Learning.
This research provides evidence for two competing attentional mechanisms. Attentional persistence directs attention to attributes previously found to be predictive, whereas contrast directs attention to stimuli that have not already been associated with a category. Three experiments provide evidence for these mechanisms. Experiments 1 and 2 revealed increased attention to an attribute following training in which that attribute was relevant, providing evidence for persistence. These experiments also revealed increased attention to an attribute following training in which another, more salient attribute was relevant, providing evidence for contrast. Experiment 3 used a subtractive method to determine the contributions of persistence and contrast to changes in attention to an attribute. The results suggest that persistence operates primarily at the level of dimensions, whereas contrast operates at the level of dimension values.
Development of features in object concepts.
According to an influential approach to cognition, our perceptual systems provide us with a repertoire of fixed features as input to higher-level cognitive processes. We present a theory of category learning and representation in which features, instead of being components of a fixed repertoire, are created under the influence of higher-level cognitive processes. When new categories need to be learned, fixed features face one of two problems: (1) High-level features that are directly useful for categorization may not be flexible enough to represent all relevant objects. (2) Low-level features consisting of unstructured fragments (such as pixels) may not capture the regularities required for successful categorization. We report evidence that feature creation occurs in category learning and we describe the conditions that promote it. Feature creation can adapt flexibly to changing environmental demands and may be the origin of fixed feature repertoires. Implications for object categorization, conceptual development, chunking, constructive induction and formal models of dimensionality reduction are discussed.
View This Paper (HTML) [without figures]
Ways of featuring in object categorization.
Schyns, P. G., Goldstone, R. L., & Thibaut, J. (1998). Ways of featuring in object categorization. Behavioral and Brain Sciences, 21, 41-49. (response to commentaries)
Learning to bridge between perception and cognition.
In building models of cognition, it is customary to commence construction on the foundations laid by perception. Perception is presumed to provide us with an initial source of information that is operated upon by subsequent cognitive processes. And, as with the foundation of a house, a premium is placed on stability and solidity. Stable edifices require stable support structures. By this view, our cognitive processes are well behaved to the degree that they can depend upon the stable structures established by our perceptual system.
Considered collectively, the contributions to this volume suggest an alternative metaphor for understanding the relation between perception and cognition. The architectural equivalent of perception may be a bridge rather than a foundation. The purpose of a bridge is to provide support, but they do so by adapting to the supported vehicles. Bridges, by design, sway under the weight of heavy vehicles, built on the principle that it is better to bend than break. Bridges built with rigid materials are often less resilient than their more flexible counterparts. Similarly, the chapters collected here raise the possibility that perception supports cognition by flexibly adapting to the requirements imposed by cognitive tasks. Perception may not be stable, but its departures from stability may facilitate rather than hamper its ability to support cognition. Cognitive processes involved in categorization, comparison, object recognition, and language may shift perception, but perception becomes better tuned to these tasks as a result.
Categorical perception of novel dimensions
Categorical perception is a phenomenon in which people are better able to distinguish between stimuli along a physical continuum when the stimuli come from different categories than when they come from the same category. In a laboratory experiment with human subjects, we find evidence for categorical perception along a novel dimension that is created by interpolating (i.e. morphing) between two randomly selected bezier curves. A neural network qualitatively models the empirical results with the following assumptions: 1) hidden ÒdetectorÓ unit become specialized for particular stimulus regions with a topologically structured competitive learning algorithm, 2) simultaneously, associations between detectors and category units are learned, and 3) feedback from the category units to the detectors causes the detectors to become concentrated near category boundaries. The particular feedback used, implemented in an “S.O.S. network,” operates by increasing the learning rate to detectors that are neighbors to a detector that produces an improper categorization.
Reasoning and the weighting of attributes in attitude judgments
The experiments examined processes by which analyzing reasons may influence attitude judgments. Participants made multiple liking judgments on sets of stimuli that varied along six a priori dimensions. In Study 1, the stimulus set consisted of 64 cartoon faces with six binary-valued attributes (e.g. a straight versus crooked nose). In Study 2, the stimuli were 60 digitized photographs from a college yearbook that varied along six dimensions uncovered through multi-dimensional scaling. In each experiment, half of the participants were instructed to think about the reasons why they liked each face before making their liking rating. Participants` multiple liking ratings were then regressed on the dimension values to determine how they weighted each dimension in their liking judgments. Results support a process whereby reasoning leads to increased variability and inconsistency in the weighting of stimulus information. Results are discussed with respect to Wilson`s model of the disruptive effects of reasoning on attitude judgments (e.g. Wilson, Dunn, Kraft, & Lisle, 1989).
Effects of categorization on color perception.
Subjects were shown simple objects and were asked to reproduce the colors of the objects. Even though the objects remained on the screen while the subjects reproduced the colors and the objects` shapes were irrelevant to the subjects` task, subjects` color perceptions were influenced by the shape category of an object. For example, objects that belonged to categories with redder objects were judged to be more red than identically colored objects belonging to another category. Further experiments showed thatX the object categories that subjects use, rather than being fixed, depend on the objects to which subjects are exposed.
Influences of categorization on perceptual discrimination
Four experiments investigated the influence of categorization training on perceptual discriminations. Ss were trained according to 1 of 4 different categorization regimes. Subsequent to category learning, Ss performed a Same-Different judgement task. Ss` sensitivities (d`s) for discriminating between items that varied on a category(ir)relevant dimensions were measured. Evidence for acquired distinctiveness (increased perceptual sensitivity for items that are categorized differently) was obtained. One case of acquired equivalence (decreased perceptual sensitivity for items that are categorized together) was found for separable, but not integral, dimensions. Acquired equivalence within a categorization-relevant dimension was never found for either integral or separable dimensions. The relevance of the results for theories of perceptual learning, dimensional attention, categorical perception, and categorization are discussed.
The role of similarity in categorization: Providing a groundwork
The relation between similarity and categorization has recently come under scrutiny from several sectors. The issue provides an important inroad to questions about the contributions of high-level thought and lower-level perception in the development of people`s concepts. Many psychological models base categorization on similarity, assuming that thing belong in the same category because of their similarity. Empirical and in-principle arguments have recently raied objections to this connection, on the grounds that similarity is too unconstrained to provide an explanation of categorization, and similarity is not sufficiently sophisticated to ground most categories. Although these objections have merit, a reassesment of evidence indicates that similarity can be sufficiently constrained and sophisticated to provide at least a partial account of many categories. Principles are discussed for incorporating similarity into theories of category formation.
View This Paper (HTML) [without figures and some equations missing]
Learning new features of representation.
Goldstone, R. L., & Schyns, P. (1994). Learning new features of representation. Proceedings of the Sixteenth Annual Conference of the Cognitive Science Society. (pp. 974-978). Hillsdale, New Jersey: Lawrence Erlbaum Associates.
One productive and influential approach to cognition maintains that categorization, object recognition, and higher-level cognitive processes operate on the output of lower-level perceptual processing.That is, our perceptual systems provide us with a set of fixed features. These features are the inputs to higher-level cognitive processes.
Recently, researchers in psychology, computer science, and philosophy have questioned this unidirectional approach, arguing that in many situations, the high-level cognitive process being executed has an influence on the lower-level features that are created. For example, in addition to categorization being based on featural descriptions of objects, it might also be the case that the categorization process partially creates the featural decriptions that are used. Rather than viewing the “vocabulary” of primitives to be fixed by low-level processes, this view maintains that the vocabulary is dependent on the higher-level process that uses the vocabulary. This symposium will investigate several issues related to bidirectional interactions between high-level and low-level cognitive processes.
- Medin, Douglas L. The Pervasiveness of Constructive Processes
- Thibaut, Jean-Pierre. Role of Variation and Knowledge on Stimuli Segmentation: Developmental Aspects
- Mozer, Michael. Computational Approaches to Functional Feature Learning
- French, Robert. Representation-building in Analogical Reasoning
- Schyns, Philippe G. A Functional Approach to Feature Learning
Letter perception: Toward a conceptual approach
We present the results of a simple experiment in lower-case letter recognition. Unlike most psychology studies of letter recognition, we include in our data set letters at the extremes of their categories and investigate the recognition of letters of multiple typefaces. We are interested in the relationship between the recognition of normal letters and the recognition of non-standard letters. Results provide empirical evidence for top-down conceptual constrains on letter perception in the form of roles and relations between perceptually-based structural subcomponents. A process model based on the hypothesis developed below is currently being implemented.
Categorization and the parsing of objects
Several models of categorization suggest that fixed inputs (features) are combined together to create categorization rules. It is also possible that categorization influences what features are perceived and used. This experiment explored the possibility that categorization training influences how an object is decomposed into parts. In the first part of this experiment, subjects learned to categorize objects based on particular sets of line segments. Following categorization training, subjects were tested in a whole-part decomposition task, making speeded judgements of “does whole X contain probe Y.” All diagnostic and nondiagnostic category parts were used as parts within the whole objects, and as probes. Categorization training in the first part of the experiment affected performance on the second task. In particular, subjects were faster to respond when the whole object contained a part that was diagnostic for categorization than when it contained a nondiagnostic part. When the probe was a diagnostic category part subjects were faster to respond that it was present than absent, and when the probe was a nondiagnostic part, subjects were faster to respond that it was absent than that it was present. These results are discussed in terms of perceptual sensitivity, response bias, and the modulating influence of experience.
Feature distribution and biased estimation of visual displays
Perceptual equivalents of confirmation biases and framing effects are observed in subjects` estimates of feature numerosity. Subjects are asked to estimate the percentage of display items that have a particular feature. Features either are randomly distributed or are spatially clustered such that features of the same type tend to be close. Subjects systematically overestimate the prevalence of features in clustered displays. The pattern of results is best explained by a regional salience bias: Features tend to be more salient if they belong to regions that have a high concentration of instruction-mentioned features. The regional salience bias is contrasted with a feature salience bias: Features tend to be more salient if they are mentioned in the instructions. The relations among the observed perceptual bias and traditional confirmation biases, numeric estimation, and attention are discussed.