Cognitive Science of Augmented Intelligence

Dubova, M., Galesic, M., & Goldstone, R. L. (in press).  Cognitive Science of Augmented Intelligence.  Cognitive Science.

Cognitive science has been traditionally organized around the individual as the basic unit of cognition. Despite developments in areas such as communication, human-machine interaction, group behavior, and community organization, individual-centric approach heavily dominates both cognitive research and its application. A promising direction for cognitive science is the study of augmented intelligence, or the way social and technological systems interact with and extend individual cognition. The cognitive science of augmented intelligence holds promise in helping society tackle major real-world challenges that can only be discovered and solved by teams made of individuals and machines with complementary skills who can productively collaborate with each other.

Download PDF of paper

Going Beyond Formalisms: A Grounded and Embodied Learning Approach to the Design of Pedagogical Statistics Simulations

Gok, S., & Goldstone, R. L., (2022). Going Beyond Formalisms: A Grounded and Embodied Learning Approach to the Design of Pedagogical Statistics Simulations.  Journal of Applied Instructional Design, 11:4, 1-16. https://dx.doi.org/10.51869/114/sgrg.

Computer-based interactive simulations that model the processes of sampling from a population are increasingly being used in data literacy education. However, these simulations are often summarized by graphs designed from the point of view of experts which makes them difficult for novices to grasp. In our ongoing design-based research project, we build and test alternative sampling simulations to the standard ones. Based on a grounded and embodied learning perspective, the core to our design position is that difficult and abstract sampling concepts and processes should: be grounded in familiar objects that are intuitive to interpret, incorporate concrete animations that spontaneously activate learners‚Äô gestures, and be accompanied by verbal instruction for a deeply integrated learning. Here, we report the results from the initial two phases of our project. In the first iteration, through an online experiment (N=126), we show that superficial perceptual elements in a standard simulation can lead to misinterpretation of concepts. In the second iteration, we pilot test a new grounded simulation with think-aloud interviews (N=9). We reflect on the complementary affordances of visual models, verbal instruction, and learners‚Äô gestures in fostering integrated and deep understanding of concepts. 

Download PDF of paper

The division of linguistic labor for offloading conceptual understanding

Andrade-Lotero, E. J., Ortiz-Duque, J. M., Velasco-Garcia, J. A., & Goldstone, R. L. (2022). The division of linguistic labor for offloading conceptual understanding.  Philosophical Transactions of the Royal Society B. 378: 20210360.

The division of linguistic labour (DLL), initially theorized by philosophers, has gained the attention of cognitive scientists in the last decade. Contrary to some controversial philosophical accounts of DLL, we propose that it is an extended mind strategy of offloading conceptual understanding onto other people. In this article, we empirically explore this proposal by providing an exploratory experimental paradigm to search for the mechanisms underwriting DLL and how they may work in practice. We developed a between-subjects experiment in which participants had to categorize two pairs of highly confusable dog breeds after receiving categorization training on just one pair of breeds. In the treatment group, participants were grouped in dyads and were allowed to interact with each other by means of the labels of these four dog breeds. In their queries to trained ‚Äėexperts‚Äô, novices frequently used labels to refer to breeds that they could not identify themselves. Experts were highly responsive to their paired novices‚Äô queries, and the rates of querying for the two members within a dyad were positively correlated. Independent categorization failure and offloading categorization success lead to subsequent increases in querying by novices, indicating adaptive use of offloading. Self-reports of breed knowledge were higher for experts within a dyad compared to isolated experts.

This article is part of the theme issue ‚ÄėConcepts in interaction: social engagement and inner experiences‚Äô.

Download PDF of paper

Plan composition using higher-order functions

Rivera, E., Krishnamurthi, S., & Goldstone, R. L. (2022).¬†¬†Plan composition using higher-order functions. In Proceedings of the 2022 ACM Conference on International Computing Education Research V.1 (ICER 2022), August 7‚Äď11, 2022, Lugano, Switzerland. ACM, New York, NY, USA, 21 pages. https://doi.org/10.1145/3501385.3543965

Program planning has been a longstanding and important problem in computing education. Finding useful primitives for planning and assessing whether students are able to understand and use these primitives remain open problems. We make progress on this problem by using higher-order functions (hofs) as planning operations. Not only are hofs increasingly prevalent in computing broadly, some data science programming sources also recommend their use in planning solutions to data-processing pipelines, giving our task additional applicability.

We find students are proficient at recognizing individual hofs through input-output examples. They use a variety of features to identify hofs, with the most prominent features being type-based. While they do have difficulty recognizing compositions of hofs presented in the same input-output example format, there may be simple explanations for this. Either way, students are able to produce correct plans that require composing hofs, and can successfully translate these plans into correct code.

Download PDF of paper

Partial copying and the role of diversity in social learning performance

Campbell, C., Izquierdo, E. & Goldstone, R. L. (2022).  Partial copying and the role of diversity in social learning performance. Collective Intelligence, 1:1, 1-15. 10.1177/26339137221081849

One major way that people engage in adaptive problem solving is by imitating others’ solutions. Prominent simulation models have found imperfect imitation advantageous, but the interactions between copying amount and other prevalent aspects of social learning strategies have been underexplored. Here, we explore the consequences for a group when its members engage in strategies with different degrees of copying, solving search problems of varying complexity, in different network topologies that affect the solutions visible to each member. Using a computational model of collective problem solving, we demonstrate that the advantage of partial copying is robust across these conditions, arising from its ability to maintain diversity. Partial copying delays convergence generally but especially in globally connected networks, which are typically associated with diversity loss, allowing more exploration of a problem space.We show that a moderate amount of diversity maintenance is optimal and strategies can be adjusted to find that sweet spot.

Download PDF of Paper

 A computational model of context-dependent encodings during category learning

Carvalho, P. F., & Goldstone, R. L. (2022).  A computational model of context-dependent encodings during category learning, Cognitive Science, 46, e13128.

Although current exemplar models of category learning are flexible and can capture how different features are emphasized for different categories, they still lack the flexibility to adapt to local changes in category learning, such as the effect of different sequences of study. In this paper, we introduce a new model of category learning, the Sequential Attention Theory Model (SAT-M), in which the encoding of each presented item is influenced not only by its category assignment (global context) as in other exemplar models, but also by how its properties relate to the properties of temporally neighboring items (local context). By fitting SAT-M to data from experiments comparing category learning with different sequences of trials (interleaved vs. blocked), we demonstrate that SAT-M captures the effect of local context and predicts when interleaved or blocked training will result in better testing performance across three different studies. Comparatively, ALCOVE, SUSTAIN, and a version of SAT-M without locally adaptive encoding provided poor fits to the results.Moreover, we evaluated the direct prediction of the model that different sequences of training change what learners encode and determined that the best-fit encoding parameter values match learners’ looking times during training.

Download PDF of Paper

Getting situated: Comparative Analysis of Language models with experimental categorization tasks

Edginer, A., & Goldstone, R. L. (2022). Getting situated: Comparative Analysis of Language models with experimental categorization tasks.  Proceedings of the 44th Annual Conference of the Cognitive Science Society. (pp. 230-236).  Toronto, Canada. Cognitive Science Society.

Common critiques of natural language processing (NLP) methods cite their lack of multimodal sensory information, claiming an inability to learn situated, action-oriented relations through language alone. Barsalou’s (1983) theory of ad hoc categories, which are formed from to achieve goals in real-world scenarios, correspond theoretically to those types of relations with which language models ought to have great difficulty. Recent NLP models have developed dynamic approaches to word representations, where the same word can have different encodings depending on the context in which it appears. Testing these models using categorization tasks with human response data demonstrates that situated properties may be partially captured through semantic analysis. We discuss possible ways in which different notions of situatedness may be distinguished for future development and testing of NLP models.

Download PDF of paper

The Counterintuitive Interpretations Learned from Putatively Intuitive Simulations 

Gok, S. & Goldstone, R. L. (2022). The counterintuitive interpretations learned from putatively intuitive simulations. Proceedings of the 44th Annual Conference of the Cognitive Science Society. (pp. 2230-2235).  Toronto, Canada. Cognitive Science Society.

Reasoning about sampling distributions is notably challenging for humans. It has been argued that the complexity involved in sampling processes can be facilitated by interactive computer simulations that allow learners to experiment with variables. In the current study, we compared the effects of learning sampling distributions through a simulation-based learning (SBL) versus direct instruction (DI) method. While both conditions resulted in similar improvement in rule learning and graph identification, neither condition improved more distant transfer of concepts. Furthermore, the simulation-based learning method resulted in unintuitive and surprising kinds of misconceptions about how sample size affects estimation of parameters while the direct instruction group used correct intuitive judgments more often. We argue that similar perceptual properties of different sampling processes in the SBL condition overrode learners’ intuitions and led them to make conceptual confusions that they would not typically make. We conclude that conceptually important differences should be grounded in easily interpretable and distinguishable perceptual representations in simulation-based learning methods. 

Download PDF of Paper

Exposing learners to variability during training has been demonstrated to improve performance

Gorman,T. E., & Goldstone, R. L. (2022).  An instance-based model account of the benefits of varied practice in visuomotor skill.  Cognitive Psychology, 137. E101491.

Exposing learners to variability during training has been demonstrated to improve performance in subsequent transfer testing. Such variability benefits are often accounted for by assuming that learners are developing some general task schema or structure. However much of this research has neglected to account for differences in similarity between varied and constant training conditions. In a between-groups manipulation, we trained participants on a simple projectile launching task, with either varied or constant conditions. We replicate previous findings showing a transfer advantage of varied over constant training. Furthermore, we show that a standard similarity model is insufficient to account for the benefits of variation, but, if the model is adjusted to assume that varied learners are tuned towards a broader generalization gradient, then a similarity-based model is sufficient to explain the observed benefits of variation. Our results therefore suggest that some variability benefits can be accommodated within instance-based models without positing the learning of some schemata or structure.

Download PDF of paper

The well measured life: Performance, well-being, motivation, and identity in an age of abundant data

Goldstone, R. L. (2022). The well measured life: Performance, well-being, motivation, and identity in an age of abundant data.  Current Directions in Psychological Science, 31(1), 1-9. https://doi.org/10.1177/09637214211053834

Our lives are being measured in rapidly increasing ways and frequency. These measurements have beneficial and deleterious effects at both individual and social levels. Behavioral measurement technologies offer the promise of helping us to know ourselves better and to improve our well-being by using personalized feedback and gamification. At the same time, they present threats to our privacy, self-esteem, and motivation. At the societal level, the potential benefits of reducing bias and decision variability by using objective and transparent assessments are offset by threats of systematic, algorithmic bias from invalid or flawed measurements. Considerable technological progress, careful foresight, and continuous scrutiny will be needed so that the positive impacts of behavioral measurement technologies far outweigh the negative ones.

Download PDF of article here

Categorical perception meets El Greco: categories unequally influence color perception of simultaneously present objects

Dubova, M., & Goldstone, R. L. (2022).  Categorical perception meets El Greco: categories unequally influence color perception of simultaneously present objects. Cognition, 223, 1-14. 105025.

Broad empirical evidence suggests that higher-level cognitive processes, such as language, categorization, and emotion, shape human visual perception. Do these higher-level processes shape human perception of all the relevant items within an immediately available scene, or do they affect only some of them? Here, we study categorical effects on visual perception by adapting a perceptual matching task so as to minimize potential non- perceptual influences. In three experiments with human adults (N = 80; N = 80, N = 82), we found that the learned higher-level categories systematically bias human perceptual matchings away from a caricature of their typical color. This effect, however, unequally biased different objects that were simultaneously present within the scene, thus demonstrating a more nuanced picture of top-down influences on perception than has been commonly assumed. In particular, perception of only the object to be matched, not the matching object, was influenced by animal category and it was gazed at less often by participants. These results suggest that category- based associations change perceptual encodings of the items at the periphery of our visual field or the items stored in concurrent memory when a person moves their eyes from one object to another. The main finding of this study calls for a revision of theories of top-down effects on perception and falsify the core assumption behind the El Greco fallacy criticism of them.

Download PDF of paper here

Tonal Emergence: An agent-based model of tonal coordination

Setzler, M., & Goldstone, R. L. (2022). Tonal Emergence: An Agent-Based Model of Tonal Coordination. Cognition, 221, 1-19. 104968.

 Humans have a remarkable capacity for coordination. Our ability to interact and act jointly in groups is crucial to our success as a species. Joint Action (JA) research has often concerned itself with simplistic behaviors in highly constrained laboratory tasks. But there has been a growing interest in understanding complex coordination in more open-ended contexts. In this regard, collective music improvisation has emerged as a fascinating model domain for studying basic JA mechanisms in an unconstrained and highly sophisticated setting. A number of empirical studies have begun to elucidate coordination mechanisms underlying joint musical improvisation, but these findings have yet to be cached out in a working computational model. The present work fills this gap by presenting Tonal Emergence, an idealized agent-based model of improvised musical coordination. Tonal Emergence models the coordination of notes played by improvisers to generate harmony (i.e., tonality), by simulating agents that stochastically generate notes biased towards maximizing harmonic consonance given their partner’s previous notes. The model replicates an interesting empirical result from a previous study of professional jazz pianists: feedback loops of mutual adaptation between interacting agents support the production of consonant harmony. The model is further explored to show how complex tonal dynamics, such as the production and dissolution of stable tonal centers, are supported by agents that are characterized by (i) a tendency to strive toward consonance, (ii) stochasticity, and (iii) a limited memory for previously played notes. Tonal Emergence thus provides a grounded computational model to simulate and probe the coordination mechanisms underpinning one of the more remarkable feats of human cognition: collective music improvisation. 

Download PDF of paper here