Reinforcement communication learning in different social network structures

Dubova, M., Moskvichev, A., & Goldstone, R. L. (2020). Reinforcement Communication Learning in Different Social Network Structures. International Conference on Machine Learning, 1st Language and Reinforcement Learning Workshop. 

Social network structure is one of the key determinants of human language evolution. Previous work has shown that the network of social interactions shapes decentralized learning in human groups, leading to the emergence of different kinds of communicative conventions. We examined the effects of social network organization on the properties of communication systems emerging in decentralized, multi-agent reinforcement learning communities. We found that the global connectivity of a social network drives the convergence of populations on shared and symmetric communication systems, preventing the agents from forming many local “dialects”. Moreover, the agent’s degree is inversely related to the consistency of its use of communicative conventions. These results show the importance of the basic properties of social network structure on reinforcement communication learning and suggest a new interpretation of findings on human convergence on word conventions.

Download PDF of paper

A three-site reproduction of the Joint Simon Effect with the NAO robot

Strait, M., Lier, F., Bernotat, J., Wachsmuth, S., Eyssel, F., Goldstone, R. L., & Sabanovic, S. (2020).  A three-site reproduction of the Joint Simon Effect with the NAO robot.  15th Annual International Conference on Human Robot Interaction.  Cambridge, England.  Association for Computing Machinery (ACM) & Institute of Electrical and Electronics Engineers (IEEE).  (pp. 103-111).  New York: ACM.

The generalizability of empirical research depends on the reproduction of findings across settings and populations. Consequently, generalizations demand resources beyond that which is typically available to any one laboratory. With collective interest in the joint Simon effect (JSE) – a phenomenon that suggests people work more effectively with humanlike (as opposed to mechanomorphic) robots – we pursued a multi-institutional research cooperation between robotics researchers, social scientists, and software engineers. To evaluate the robustness of the JSE in dyadic human-robot interactions, we constructed an experimental infrastructure for exact, lab-independent reproduction of robot behavior. Deployment of our infrastructure across three institutions with distinct research orientations (well-resourced versus resource-constrained) provides initial demonstration of the success of our approach and the degree to which it can alleviate technical barriers to HRI reproducibility. Moreover, with the three deployments situated in culturally distinct contexts (Germany, the U.S. Midwest, and the Mexico-U.S. Border), observation of a JSE at each site provides evidence its generalizability across settings and populations.

Download PDF of paper

Coordination and consonance between interacting, improvising musicians

Setzler, M., & Goldstone, R. L. (2020).  Coordination and consonance between interacting, improvising musicians.  Open Mind: Discoveries in Cognitive Science, 4, 88—101.

Joint action (JA) is ubiquitous in our cognitive lives. From basketball teams to teams of surgeons, humans often coordinate with one another to achieve some common goal. Idealized laboratory studies of group behavior have begun to elucidate basic JA mechanisms, but little is understood about how these mechanisms scale up in more sophisticated and open-ended JA that occurs in the wild. We address this gap by examining coordination in a paragon domain for creative joint expression: improvising jazz musicians. Coordination in jazz music subserves an aesthetic goal: the generation of a collective musical expression comprising coherent, highly nuanced musical structure (e.g. rhythm, harmony). In our study, dyads of professional jazz pianists improvised in a “coupled”, mutually adaptive condition, and an “overdubbed” condition which precluded mutual adaptation, as occurs in common studio recording practices. Using a model of musical tonality, we quantify the flow of rhythmic and harmonic information between musicians as a function of interaction condition. Our analyses show that mutually adapting dyads achieve greater temporal alignment and produce more consonant harmonies. These musical signatures of coordination were preferred by independent improvisers and naive listeners, who gave higher quality ratings to coupled interactions despite being blind to condition. We present these results and discuss their implications for music technology and JA research more generally.

Download PDF

Quantifying Emergent, Dynamic Tonal Coordination in Collaborative Musical Improvisation

Setzler, M., & Goldstone, R. L. (2020).  Quantifying Emergent, Dynamic Tonal Coordination in Collaborative Musical Improvisation.  Proceedings of the 42nd Annual Conference of the Cognitive Science Society. (pp. 461-466).  Toronto, CA. Cognitive Science Society.

Groups of interacting individuals often coordinate in service of abstract goals, such as the alignment of mental representations in conversation, or the generation of new ideas in group brainstorming sessions. What are the mechanisms and dynamics of abstract coordination? This study examines coordination in a sophisticated paragon domain: collaboratively improvising jazz musicians. Remarkably, freely improvising jazz ensembles collectively produce coherent tonal structure (i.e. melody and harmony) in real time performance without previously established harmonic forms. We investigate how tonal structure emerges out of interacting musicians, and how this structure is constrained by underlying patterns of coordination. Dyads of professional jazz pianists were recorded improvising in two conditions of interaction: a ‘coupled’ condition in which they could mutually adapt to one another, and an ‘overdubbed’ condition which precluded mutual adaptation. Using a computational model of musical tonality, we show that this manipulation effected the directed flow of tonal information amongst pianists, who could mutually adapt to one another’s notes in coupled trials, but not in overdubbed trials. Consequently, musicians were better able to harmonize with one another in coupled trials, and this ability increased throughout the course of improvised performance. We present these results and discuss their implications for music technology and joint action research more generally.

Download PDF of paper

How much to copy from others?

Campbell, C., Izquierdo, E. & Goldstone, R. L. (2020).  How much to copy from others? The role of partial copying in social learning.  Proceedings of the 42nd Annual Conference of the Cognitive Science Society. (pp. 916-922). Toronto, Canada: Cognitive Science Society.

One of the major ways that people engage in adaptive problem solving is by copying the solutions of others. Most of the work on this field has focused on three questions: when to copy, who to copy from, and what to copy. However, how much to copy has been relatively less explored. In the current research, we are interested in the consequences for a group when its members engage in social learning strategies with different tendencies to copy entire or partial solutions and different complexities of search problems. We also consider different network topologies that affect the solutions visible to each member. Using a computational model of collective problem solving, we demonstrate that strategies where social learning involves partial copying outperform strategies where individuals copy entire solutions. We analyze the exploration/exploitation dynamics of these social learning strategies under the different conditions.

Download PDF of paper here

Reconstructing Maps from Text

Avery, J. E., Goldstone, R. L., & Jones, M. N. (2020).  Reconstructing Maps from Text.  Proceedings of the 42nd Annual Conference of the Cognitive Science Society. (pp. 557-563).  Toronto, CA. Cognitive Science Society.

 Previous research has demonstrated that Distributional Semantic Models (DSMs) are capable of reconstructing maps from news corpora (Louwerse & Zwaan, 2009) and novels (Louwerse & Benesh, 2012). The capacity for reproducing maps is surprising since DSMs notoriously lack perceptual grounding (De Vega et al., 2012). In this paper we investigate the statistical sources required in language to infer maps, and resulting constraints placed on mechanisms of semantic representation. Study 1 brings word co-occurrence under experimental control to demonstrate that direct co-occurrence in language is necessary for traditional DSMs to successfully reproduce maps. Study 2 presents an instance-based DSM that is capable of reconstructing maps independent of the frequency of co-occurrence of city names. 

Download PDF of paper

How do people code information in working memory when items share features?


Plancher, G., & Goldstone, R. L. (2020).  How do people code information in working memory when items share features?  Experimental Psychology, 1-9.

A large literature suggests that the way we process information is influenced by the categories that we have learned. We examined whether, when we try to uniquely encode items in workingmemory, the information encoded depends on the other stimuli being simultaneously learned. Participants were required to memorize unknown aliens, presented one at the time, for immediate recognition of their features. Some aliens, called twins, were organized into pairs that shared every feature (nondiscriminative feature) except one (discriminative feature), while some other aliens, called hermits, did not share feature. We reasoned that if people develop unsupervised categories by creating a category for a pair of aliens, we should observe better feature identification performance for nondiscriminative features compared to hermit features, but not compared to discriminative features. On the contrary, if distinguishing features draw attention, we should observe better performance when a discriminative rather than nondiscriminative feature was probed. Overall, our results suggest that when items share features, people code items in working memory by focusing on similarities between items, establishing clusters of items in an unsupervised fashion not requiring feedback on cluster membership.

Download a PDF version of the paper

Why does peer instruction benefit student learning?


Tullis, J. G., & Goldstone, R. L. (2020).  Why does peer instruction benefit student learning?  Cognitive Research: Principles and Implications, 5:15, 1-12.

In peer instruction, instructors pose a challenging question to students, students answer the question individually, students work with a partner in the class to discuss their answers, and finally students answer the question again. A large body of evidence shows that peer instruction benefits student learning. To determine the mechanism for these benefits, we collected semester-long data from six classes, involving a total of 208 undergraduate students being asked a total of 86 different questions related to their course content. For each question, students chose their answer individually, reported their confidence, discussed their answers with their partner, and then indicated their possibly revised answer and confidence again. Overall, students were more accurate and confident after discussion than before. Initially correct students were more likely to keep their answers than initially incorrect students, and this tendency was partially but not completely attributable to differences in confidence. We discuss the benefits of peer instruction in terms of differences in the coherence of explanations, social learning, and the contextual factors that influence confidence and accuracy.

Download PDF of paper

Simple Threshold Rules Solve Explore/Exploit Trade-offs in a Resource Accumulation Search Task

Sang, K., Todd, P. M., Goldstone, R. L., & Hills, T. T. (2020).  Simple threshold rules solve explore/exploit tradeoffs in a resource accumulation search task. Cognitive Science, 44, e12817.

How, and how well, do people switch between exploration and exploitation to search for and accumulate resources? We study the decision processes underlying such exploration/exploitation trade-offs using a novel card selection task that captures the common situation of searching among multiple resources (e.g., jobs) that can be exploited without depleting. With experience, participants learn to switch appropriately between exploration and exploitation and approach optimal performance. We model participants’ behavior on this task with random, threshold, and sampling strategies, and find that a linear decreasing threshold rule best fits participants’ results. Further evidence that participants use decreasing threshold-based strategies comes from reaction time differences between exploration and exploitation; however, participants themselves report nondecreasing thresholds. Decreasing threshold strategies that “front-load” exploration and switch quickly to exploitation are particularly effective in resource accumulation tasks, in contrast to optimal stopping problems like the Secretary Problem requiring longer exploration.

Download a PDF of article