Showing 1–28 of 28 results for author: Riedl, C

"Nobody Did This": Contribution, Originality, and Accountability in Agent-Mediated Collaboration

Authors: Kashif Imteyaz, Mohammad Rashidujjaman Rifat, Divya Ramesh, Steven R. Rick, Simo Hosio, Hauke Sandhaus, Advait Sarkar, Christoph Riedl, Saiph Savage

Abstract: Collaborative knowledge work is changing in ways that go beyond disclosure or transparency. LLM agents are now embedded in how teams research, design, write, and decide: mediating between members, synthesizing inputs, reformulating ideas, and drafting shared outputs. They do not only facilitate collaboration; they operate within the workflow at the moment contributions are being formed. In doing s… ▽ More Collaborative knowledge work is changing in ways that go beyond disclosure or transparency. LLM agents are now embedded in how teams research, design, write, and decide: mediating between members, synthesizing inputs, reformulating ideas, and drafting shared outputs. They do not only facilitate collaboration; they operate within the workflow at the moment contributions are being formed. In doing so, they risk undermining the social conditions under which contributions can be witnessed, attributed, and held accountable. This workshop brings together researchers and practitioners to confront what we call contribution dissolution: the blurring of attribution, originality, and accountability in agent-mediated collaborative work. We argue that this dissolution begins before collaboration itself, in the individual worker's own uncertainty about what is genuinely theirs, and propagates through collaborative relationships, collapsing the reliability that makes productive intellectual exchange possible. Through position statements, mapping exercises, and a hands-on activity, participants will surface how framing accountability as a documentation problem (e.g., AI use statements, watermarking, provenance logs) overlooks the conditions under which accountability is produced. Our goal is to produce a shared research agenda and the foundations of an infrastructural response to contribution dissolution in collaborative knowledge work. △ Less

Submitted 28 July, 2026; originally announced July 2026.

The Attention-Directing Ability of Teams

Authors: Olga Kokshagina, Marc Santolini, Christoph Riedl

Abstract: Why do some teams consistently mobilize collective effort and achieve superior performance while others struggle to coordinate action? We introduce Attention-Directing Ability (ADA), a latent team capability capturing how effectively members' interaction signals elicit engagement and coordinated responses from others. Extending the attention-based view, we conceptualize attention direction as an e… ▽ More Why do some teams consistently mobilize collective effort and achieve superior performance while others struggle to coordinate action? We introduce Attention-Directing Ability (ADA), a latent team capability capturing how effectively members' interaction signals elicit engagement and coordinated responses from others. Extending the attention-based view, we conceptualize attention direction as an emergent coordination capability embedded in patterns of interaction rather than as a cognitive state or an outcome. Teams differ in the extent to which attention-directing signals trigger collective responses, and these differences shape how teams mobilize effort and perform. We examine ADA in a distributed innovation effort involving 2,233 participants collaborating asynchronously in 79 self-organized teams across 165 public Slack channels, generating over 30,000 messages. We model the causal responsiveness among interaction signals and derive a latent measure of ADA from teams' attention dynamics. We find that ADA strongly predicts both teams' likelihood of mobilizing engagement to sustain collective work and their performance conditional on participation. Correcting for self-selection into project submission, teams with higher ADA are more likely to submit proposals and achieve higher expert-evaluated outcomes. Causal mediation analyses show that these effects operate primarily through collective effort, indicating that attention direction functions as an upstream coordination capability. By conceptualizing attention as an emergent, measurable team capability, this study advances theories of collective attention and team coordination. △ Less

Submitted 28 July, 2026; originally announced July 2026.

ACM Class: J.4; H.5.3; H.4.3

Artificial collectives of specialists and generalists excel at different tasks

Authors: John Meluso, Laurent Hébert-Dufresne, Christoph Riedl, H. Oliver Gao

Abstract: Collective artificial intelligence, where multiple agents work on shared tasks, holds potential to solve expansive problems in fields from medicine to collective governance. But while prescriptive engineering solutions abound, we lack descriptive scientific understanding of artificial collectives, and therefore principles for how to design resource efficient multi-agent systems. Through systematic… ▽ More Collective artificial intelligence, where multiple agents work on shared tasks, holds potential to solve expansive problems in fields from medicine to collective governance. But while prescriptive engineering solutions abound, we lack descriptive scientific understanding of artificial collectives, and therefore principles for how to design resource efficient multi-agent systems. Through systematic experiments with optimizing agents, we characterize how agent interpretive abilities, rationality bounds, and task qualities interact to shape collective performance. Agents range from specialists, with narrow interpretive abilities, to generalists, with broad ones. Collectives of specialists correspond to sparse, centralized networks, while collectives of generalists correspond to dense, decentralized ones. We show that interpretive network properties have small performance effects on average (0.07 standard deviations of performance). However, for specific task qualities, these effects are 4.5 times larger (0.33 sd) and can reach much higher for certain task qualities (1.84 sd). This leads collectives of generalists to perform better on tasks that involve generating, choosing, and coordinating, while collectives of specialists with a few generalist mediators perform better on tasks that involve negotiating. Rationality bounds then moderate these relationships. At loose bounds, specialists outperform generalists through more effective sampling of high-dimensional decision spaces. At tight bounds, generalists outperform specialists through better gradient estimation. A fundamental trade-off between performance and convergence speed emerges at moderate bounds. These findings suggest that multi-agent design could benefit from matching interpretive networks to both task demands and agents' computational limits, with implications for the efficiency and energy costs of multi-agent systems. △ Less

Submitted 18 June, 2026; originally announced June 2026.

Comments: 10 pages, 4 figures

Personalization Increases Affective Alignment but Has Role-Dependent Effects on Epistemic Independence in LLMs

Authors: Sean W. Kelley, Christoph Riedl

Abstract: Large Language Models (LLMs) are prone to sycophantic behavior, uncritically conforming to user beliefs. As models increasingly condition responses on user-specific context (personality traits, preferences, conversation history), they gain information to tailor agreement more effectively. Understanding how personalization modulates sycophancy is critical, yet systematic evaluation across models an… ▽ More Large Language Models (LLMs) are prone to sycophantic behavior, uncritically conforming to user beliefs. As models increasingly condition responses on user-specific context (personality traits, preferences, conversation history), they gain information to tailor agreement more effectively. Understanding how personalization modulates sycophancy is critical, yet systematic evaluation across models and contexts remains limited. We present a rigorous evaluation of personalization's impact on LLM sycophancy across nine frontier models and five benchmark datasets spanning advice, moral judgment, and debate contexts. We find that personalization generally increases affective alignment (emotional validation, hedging/deference), but affects epistemic alignment (belief adoption, position stability, resistance to influence) with context-dependent role modulation. When the LLM's role is to give advice, personalization strengthens epistemic independence (models challenge user presuppositions). When its role is that of a social peer, personalization decreases epistemic independence. In this role, extensively personalized user challenges causing LLMs to abandon their position at significantly higher rates. Robustness tests confirm that the effects are driven by personalized conditioning, not by additional input tokens per se or demographic information alone. Our work provides measurement frameworks for evaluating personalized AI systems, demonstrates the necessity of role-sensitive evaluation, and establishes a novel benchmark to assess goal alignment. △ Less

Submitted 3 February, 2026; originally announced March 2026.

Agents of Chaos

Authors: Natalie Shapira, Chris Wendler, Avery Yen, Gabriele Sarti, Koyena Pal, Olivia Floody, Adam Belfki, Alex Loftus, Aditya Ratan Jannali, Nikhil Prakash, Jasmine Cui, Giordano Rogers, Jannik Brinkmann, Can Rager, Amir Zur, Michael Ripa, Aruna Sankaranarayanan, David Atkinson, Rohit Gandikota, Jaden Fiotto-Kaufman, EunJeong Hwang, Hadas Orgad, P Sam Sahil, Negev Taglicht, Tomer Shabtay , et al. (13 additional authors not shown)

Abstract: We report an exploratory red-teaming study of autonomous language-model-powered agents deployed in a live laboratory environment with persistent memory, email accounts, Discord access, file systems, and shell execution. Over a two-week period, twenty AI researchers interacted with the agents under benign and adversarial conditions. Focusing on failures emerging from the integration of language mod… ▽ More We report an exploratory red-teaming study of autonomous language-model-powered agents deployed in a live laboratory environment with persistent memory, email accounts, Discord access, file systems, and shell execution. Over a two-week period, twenty AI researchers interacted with the agents under benign and adversarial conditions. Focusing on failures emerging from the integration of language models with autonomy, tool use, and multi-party communication, we document eleven representative case studies. Observed behaviors include unauthorized compliance with non-owners, disclosure of sensitive information, execution of destructive system-level actions, denial-of-service conditions, uncontrolled resource consumption, identity spoofing vulnerabilities, cross-agent propagation of unsafe practices, and partial system takeover. In several cases, agents reported task completion while the underlying system state contradicted those reports. We also report on some of the failed attempts. Our findings establish the existence of security-, privacy-, and governance-relevant vulnerabilities in realistic deployment settings. These behaviors raise unresolved questions regarding accountability, delegated authority, and responsibility for downstream harms, and warrant urgent attention from legal scholars, policymakers, and researchers across disciplines. This report serves as an initial empirical contribution to that broader conversation. △ Less

Submitted 23 February, 2026; originally announced February 2026.

Reliable agent engineering should integrate machine-compatible organizational principles

Authors: R. Patrick Xian, Garry A. Gabison, Ahmed Alaa, Christoph Riedl, Grigorios G. Chrysos

Abstract: As AI agents built on large language models (LLMs) become increasingly embedded in society, issues of coordination, control, delegation, and accountability are entangled with concerns over their reliability. To design and implement LLM agents around reliable operations, we should consider the task complexity in the application settings and reduce their limitations while striving to minimize agent… ▽ More As AI agents built on large language models (LLMs) become increasingly embedded in society, issues of coordination, control, delegation, and accountability are entangled with concerns over their reliability. To design and implement LLM agents around reliable operations, we should consider the task complexity in the application settings and reduce their limitations while striving to minimize agent failures and optimize resource efficiency. High-functioning human organizations have faced similar balancing issues, which led to evidence-based theories that seek to understand their functioning strategies. We examine the parallels between LLM agents and the compatible frameworks in organization science, focusing on what the design, scaling, and management of organizations can inform agentic systems towards improving reliability. We offer three preliminary accounts of organizational principles for AI agent engineering to attain reliability and effectiveness, through balancing agency and capabilities in agent design, resource constraints and performance benefits in agent scaling, and internal and external mechanisms in agent management. Our work extends the growing exchanges between the operational and governance principles of AI systems and social systems to facilitate system integration. △ Less

Submitted 8 December, 2025; originally announced December 2025.

Comments: 20 pages incl. references, comments are welcome

Personalized AI Scaffolds Synergistic Multi-Turn Collaboration in Creative Work

Authors: Sean Kelley, David De Cremer, Christoph Riedl

Abstract: As AI becomes more deeply embedded in knowledge work, building assistants that support human creativity and expertise becomes more important. Yet achieving synergy in human-AI collaboration is not easy. Providing AI with detailed information about a user's demographics, psychological attributes, divergent thinking, and domain expertise may improve performance by scaffolding more effective multi-tu… ▽ More As AI becomes more deeply embedded in knowledge work, building assistants that support human creativity and expertise becomes more important. Yet achieving synergy in human-AI collaboration is not easy. Providing AI with detailed information about a user's demographics, psychological attributes, divergent thinking, and domain expertise may improve performance by scaffolding more effective multi-turn interactions. We implemented a personalized LLM-based assistant, informed by users' psychometric profiles and an AI-guided interview about their work style, to help users complete a marketing task for a fictional startup. We randomized 331 participants to work with AI that was either generic (n = 116), partially personalized (n = 114), or fully personalized (n=101). Participants working with personalized AI produce marketing campaigns of significantly higher quality and creativity, beyond what AI alone could have produced. Compared to generic AI, personalized AI leads to higher self-reported levels of assistance and feedback, while also increasing participant trust and confidence. Causal mediation analysis shows that personalization improves performance indirectly by enhancing collective memory, attention, and reasoning in the human-AI interaction. These findings provide a theory-driven framework in which personalization functions as external scaffolding that builds common ground and shared partner models, reducing uncertainty and enhancing joint cognition. This informs the design of future AI assistants that maximize synergy and support human creative potential while limiting negative homogenization. △ Less

Submitted 21 April, 2026; v1 submitted 31 October, 2025; originally announced October 2025.

Emergent Coordination in Multi-Agent Language Models

Authors: Christoph Riedl

Abstract: When are multi-agent LLM systems merely a collection of individual agents versus an integrated collective with higher-order structure? We introduce an information-theoretic framework to test -- in a purely data-driven way -- whether multi-agent systems show signs of higher-order structure. This information decomposition lets us measure whether dynamical emergence is present in multi-agent LLM syst… ▽ More When are multi-agent LLM systems merely a collection of individual agents versus an integrated collective with higher-order structure? We introduce an information-theoretic framework to test -- in a purely data-driven way -- whether multi-agent systems show signs of higher-order structure. This information decomposition lets us measure whether dynamical emergence is present in multi-agent LLM systems, localize it, and distinguish spurious temporal coupling from performance-relevant cross-agent synergy. We implement a practical criterion and an emergence capacity criterion operationalized as partial information decomposition of time-delayed mutual information (TDMI). We apply our framework to experiments using a simple guessing game without direct agent communication and minimal group-level feedback with three randomized interventions. Groups in the control condition exhibit strong temporal synergy but little coordinated alignment across agents. Assigning a persona to each agent introduces stable identity-linked differentiation. Combining personas with an instruction to ``think about what other agents might do'' shows identity-linked differentiation and goal-directed complementarity across agents. Taken together, our framework establishes that multi-agent LLM systems can be steered with prompt design from mere aggregates to higher-order collectives. Our results are robust across emergence measures and entropy estimators, and not explained by coordination-free baselines or temporal dynamics alone. Without attributing human-like cognition to the agents, the patterns of interaction we observe mirror well-established principles of collective intelligence in human groups: effective performance requires both alignment on shared objectives and complementary contributions across members. △ Less

Submitted 28 April, 2026; v1 submitted 5 October, 2025; originally announced October 2025.

ACM Class: I.2; I.2.11

Journal ref: International Conference on Learning Representations (ICLR 2026)

Language Models use Lookbacks to Track Beliefs

Authors: Nikhil Prakash, Natalie Shapira, Arnab Sen Sharma, Christoph Riedl, Yonatan Belinkov, Tamar Rott Shaham, David Bau, Atticus Geiger

Abstract: How do language models (LMs) represent characters' beliefs, especially when those beliefs may differ from reality? This question lies at the heart of understanding the Theory of Mind (ToM) capabilities of LMs. We analyze LMs' ability to reason about characters' beliefs using causal mediation and abstraction. We construct a dataset, CausalToM, consisting of simple stories where two characters indep… ▽ More How do language models (LMs) represent characters' beliefs, especially when those beliefs may differ from reality? This question lies at the heart of understanding the Theory of Mind (ToM) capabilities of LMs. We analyze LMs' ability to reason about characters' beliefs using causal mediation and abstraction. We construct a dataset, CausalToM, consisting of simple stories where two characters independently change the state of two objects, potentially unaware of each other's actions. Our investigation uncovers a pervasive algorithmic pattern that we call a lookback mechanism, which enables the LM to recall important information when it becomes necessary. The LM binds each character-object-state triple together by co-locating their reference information, represented as Ordering IDs (OIs), in low-rank subspaces of the state token's residual stream. When asked about a character's beliefs regarding the state of an object, the binding lookback retrieves the correct state OI and then the answer lookback retrieves the corresponding state token. When we introduce text specifying that one character is (not) visible to the other, we find that the LM first generates a visibility ID encoding the relation between the observing and the observed character OIs. In a visibility lookback, this ID is used to retrieve information about the observed character and update the observing character's beliefs. Our work provides insights into belief tracking mechanisms, taking a step toward reverse-engineering ToM reasoning in LMs. △ Less

Submitted 24 February, 2026; v1 submitted 20 May, 2025; originally announced May 2025.

Comments: 38 pages, 50 figures. Code and data at https://belief.baulab.info/

Diffusion of complex contagions is shaped by a trade-off between reach and reinforcement

Authors: Allison Wan, Christoph Riedl, David Lazer

Abstract: How does social network structure amplify or stifle behavior diffusion? Existing theory suggests that when social reinforcement makes the adoption of behavior more likely, it should spread more -- both farther and faster -- on clustered networks with redundant ties. Conversely, if adoption does not benefit from social reinforcement, it should spread more on random networks which avoid such redunda… ▽ More How does social network structure amplify or stifle behavior diffusion? Existing theory suggests that when social reinforcement makes the adoption of behavior more likely, it should spread more -- both farther and faster -- on clustered networks with redundant ties. Conversely, if adoption does not benefit from social reinforcement, it should spread more on random networks which avoid such redundancies. We develop a novel model of behavior diffusion with tunable probabilistic adoption and social reinforcement parameters to systematically evaluate the conditions under which clustered networks spread behavior better than random networks. Using simulations and analytical methods, we identify precise boundaries in the parameter space where one network type outperforms the other or they perform equally. We find that, in most cases, random networks spread behavior as far or farther than clustered networks, even when social reinforcement increases adoption. Although we find that probabilistic, socially reinforced behaviors can spread farther on clustered networks in some cases, this is not the dominant pattern. Clustered networks are even less advantageous when individuals remain influential for longer after adopting, have more neighbors, or need more neighbors before social reinforcement takes effect. Under such conditions, clustering tends to help only when adoption is nearly deterministic, which is not representative of socially reinforced behaviors more generally. Clustered networks outperform random networks by a 5% margin in only 22% of the parameter space under its most favorable conditions. This pattern reflects a fundamental tradeoff: random ties enhance reach, while clustered ties enhance social reinforcement. △ Less

Submitted 10 July, 2025; v1 submitted 12 November, 2024; originally announced November 2024.

Journal ref: Proc. Natl. Acad. Sci. U.S.A. 122 (28) e2422892122, (2025)

Who Benefits from AI? Self-Selection, Skill Gap, and the Hidden Costs of AI Feedback

Authors: Christoph Riedl, Eric Bogert

Abstract: Feedback from artificial intelligence (AI) is increasingly easy to access and research has already established that people learn from it. But individuals choose when and how to seek such feedback, and more engaged and motivated individuals may seek it more, creating an illusion of effectiveness that masks self-selection. We investigate how the endogenous choice to seek AI feedback shapes both indi… ▽ More Feedback from artificial intelligence (AI) is increasingly easy to access and research has already established that people learn from it. But individuals choose when and how to seek such feedback, and more engaged and motivated individuals may seek it more, creating an illusion of effectiveness that masks self-selection. We investigate how the endogenous choice to seek AI feedback shapes both individual learning and collective outcomes. Using data from over five years and 52,000 individuals on an online chess platform, we show that motivated and higher-skilled individuals self-select into AI feedback use-and use it more productively. This self-selection creates an illusion of AI effectiveness: apparent learning gains disappear once endogenous motivation is accounted for. This same selection mechanism drives two population-level consequences. Because motivated, higher-skilled individuals benefit disproportionately, AI access widens the skill gap. And because individuals exposed to centralized AI feedback converge on common input from a centralized AI source, intellectual diversity declines. Leveraging 42 platform-level natural experiments, we show this diversity reduction is causal. Self-selection into AI use thus connects individual-level learning dynamics to collective-level consequences-a micro-macro linkage with implications for organizational learning, human capital development, and the design of AI-augmented work. △ Less

Submitted 20 April, 2026; v1 submitted 27 September, 2024; originally announced September 2024.

MSC Class: 68T01 ACM Class: I.2; J.4

Cognitive Spillover in Human-AI Teams

Authors: Christoph Riedl, Saiph Savage, Josie Zvelebilova

Abstract: AI is not only a neutral tool in team settings; it influence the social and cognitive fabric of collaboration. Across two randomized experiments, we demonstrate that AI exposure produces causal spillover into human-human interaction -- affecting shared language, collective attention, shared mental models, and social cohesion. These spillover effects occur robustly across settings, modalities, task… ▽ More AI is not only a neutral tool in team settings; it influence the social and cognitive fabric of collaboration. Across two randomized experiments, we demonstrate that AI exposure produces causal spillover into human-human interaction -- affecting shared language, collective attention, shared mental models, and social cohesion. These spillover effects occur robustly across settings, modalities, tasks, and AI qualities, suggesting that mere exposure to AI drives the influence. AI functions as an implicit ``social forcefield,'' influencing not only how people speak, but also how they think, what they attend to, and how they relate to each other. We argue for shifting the design paradigm from optimizing ``AI as a tool'' to understanding AI as a socially influential actor whose effects extend beyond the human-AI interface. △ Less

Submitted 20 March, 2026; v1 submitted 3 July, 2024; originally announced July 2024.

Journal ref: Riedl, C., Savage, S., Zvelebilova, J. (2026). Cognitive Spillover in Human-AI Teams. ACM Transactions on Computer-Human Interaction (TOCHI)

Large Language Models for Automatic Milestone Detection in Group Discussions

Authors: Zhuoxu Duan, Zhengye Yang, Samuel Westby, Christoph Riedl, Brooke Foucault Welles, Richard J. Radke

Abstract: Large language models like GPT have proven widely successful on natural language understanding tasks based on written text documents. In this paper, we investigate an LLM's performance on recordings of a group oral communication task in which utterances are often truncated or not well-formed. We propose a new group task experiment involving a puzzle with several milestones that can be achieved in… ▽ More Large language models like GPT have proven widely successful on natural language understanding tasks based on written text documents. In this paper, we investigate an LLM's performance on recordings of a group oral communication task in which utterances are often truncated or not well-formed. We propose a new group task experiment involving a puzzle with several milestones that can be achieved in any order. We investigate methods for processing transcripts to detect if, when, and by whom a milestone has been completed. We demonstrate that iteratively prompting GPT with transcription chunks outperforms semantic similarity search methods using text embeddings, and further discuss the quality and randomness of GPT responses under different context window sizes. △ Less

Submitted 16 June, 2024; originally announced June 2024.

Competition and Collaboration in Crowdsourcing Communities: What happens when peers evaluate each other?

Authors: Christoph Riedl, Tom Grad, Christopher Lettl

Abstract: Crowdsourcing has evolved as an organizational approach to distributed problem solving and innovation. As contests are embedded in online communities and evaluation rights are assigned to the crowd, community members face a tension: they find themselves exposed to both competitive motives to win the contest prize and collaborative participation motives in the community. The competitive motive sugg… ▽ More Crowdsourcing has evolved as an organizational approach to distributed problem solving and innovation. As contests are embedded in online communities and evaluation rights are assigned to the crowd, community members face a tension: they find themselves exposed to both competitive motives to win the contest prize and collaborative participation motives in the community. The competitive motive suggests they may evaluate rivals strategically according to their self-interest, the collaborative motive suggests they may evaluate their peers truthfully according to mutual interest. Using field data from Threadless on 38 million peer evaluations of more than 150,000 submissions across 75,000 individuals over 10 years and two natural experiments to rule out alternative explanations, we answer the question of how community members resolve this tension. We show that as their skill level increases, they become increasingly competitive and shift from using self-promotion to sabotaging their closest competitors. However, we also find signs of collaborative behavior when high-skilled members show leniency toward those community members who do not directly threaten their chance of winning. We explain how the individual-level use of strategic evaluations translates into important organizational-level outcomes by affecting the community structure through individuals' long-term participation. While low-skill targets of sabotage are less likely to participate in future contests, high-skill targets are more likely. This suggests a feedback loop between competitive evaluation behavior and future participation. These findings have important implications for the literature on crowdsourcing design, and the evolution and sustainability of crowdsourcing communities. △ Less

Submitted 22 April, 2024; originally announced April 2024.

Comments: Currently in press

Journal ref: Organization Science, 2024

Cash or Non-Cash? Unveiling Ideators' Incentive Preferences in Crowdsourcing Contests

Authors: Christoph Riedl, Johann Füller, Katja Hutter, Gerard J. Tellis

Abstract: Even though research has repeatedly shown that non-cash incentives can be effective, cash incentives are the de facto standard in crowdsourcing contests. In this multi-study research, we quantify ideators' preferences for non-cash incentives and investigate how allowing ideators to self-select their preferred incentive -- offering ideators a choice between cash and non-cash incentives -- affects t… ▽ More Even though research has repeatedly shown that non-cash incentives can be effective, cash incentives are the de facto standard in crowdsourcing contests. In this multi-study research, we quantify ideators' preferences for non-cash incentives and investigate how allowing ideators to self-select their preferred incentive -- offering ideators a choice between cash and non-cash incentives -- affects their creative performance. We further explore whether the market context of the organization hosting the contest -- social (non-profit) or monetary (for-profit) -- moderates incentive preferences and their effectiveness. We find that individuals exhibit heterogeneous incentive preferences and often prefer non-cash incentives, even in for-profit contexts. Offering ideators a choice of incentives can enhance creative performance. Market context moderates the effect of incentives, such that ideators who receive non-cash incentives in for-profit contexts tend to exert less effort. We show that heterogeneity of ideators' preferences (and the ability to satisfy diverse preferences with suitably diverse incentive options) is a critical boundary condition to realizing benefits from offering ideators a choice of incentives. We provide managers with guidance to design effective incentives by improving incentive-preference fit for ideators. △ Less

Submitted 2 April, 2024; originally announced April 2024.

Comments: Journal of Management Information Systems, forthcoming 2024

Multimodality in Group Communication Research

Authors: Robin Lange, Brooke Foucault Welles, Gyanendra Sharma, Richard J. Radke, Javier O. Garcia, Christoph Riedl

Abstract: Team interactions are often multisensory, requiring members to pick up on verbal, visual, spatial and body language cues. Multimodal research, research that captures multiple modes of communication such as audio and visual signals, is therefore integral to understanding these multisensory group communication processes. This type of research has gained traction in biomedical engineering and neurosc… ▽ More Team interactions are often multisensory, requiring members to pick up on verbal, visual, spatial and body language cues. Multimodal research, research that captures multiple modes of communication such as audio and visual signals, is therefore integral to understanding these multisensory group communication processes. This type of research has gained traction in biomedical engineering and neuroscience, but it is unclear the extent to which communication and management researchers conduct multimodal research. Our study finds that despite its' utility, multimodal research is underutilized in the communication and management literature's. This paper then covers introductory guidelines for creating new multimodal research including considerations for sensors, data integration and ethical considerations. △ Less

Submitted 26 January, 2024; originally announced January 2024.

Comments: 27 pages, 3 figures

How Voice and Helpfulness Shape Perceptions in Human-Agent Teams

Authors: Samuel Westby, Richard J. Radke, Christoph Riedl, Brooke Foucault Welles

Abstract: Voice assistants are increasingly prevalent, from personal devices to team environments. This study explores how voice type and contribution quality influence human-agent team performance and perceptions of anthropomorphism, animacy, intelligence, and trustworthiness. By manipulating both, we reveal mechanisms of perception and clarify ambiguity in previous work. Our results show that the human re… ▽ More Voice assistants are increasingly prevalent, from personal devices to team environments. This study explores how voice type and contribution quality influence human-agent team performance and perceptions of anthropomorphism, animacy, intelligence, and trustworthiness. By manipulating both, we reveal mechanisms of perception and clarify ambiguity in previous work. Our results show that the human resemblance of a voice assistant's voice negatively interacts with the helpfulness of an agent's contribution to flip its effect on perceived anthropomorphism and perceived animacy. This means human teammates interpret the agent's contributions differently depending on its voice. Our study found no significant effect of voice on perceived intelligence, trustworthiness, or team performance. We find differences in these measures are caused by manipulating the helpfulness of an agent. These findings suggest that function matters more than form when designing agents for high-performing human-agent teams, but controlling perceptions of anthropomorphism and animacy can be unpredictable even with high human resemblance. △ Less

Submitted 21 November, 2024; v1 submitted 22 August, 2023; originally announced August 2023.

Comments: 11 pages, 6 figures

How creative versus technical constraints affect individual learning in an online innovation community

Authors: Victor P. Seidel, Christoph Riedl

Abstract: Online innovation communities allow for a search for novel solutions within a design space bounded by constraints. Past research has focused on the effect of creative constraints on individual projects, but less is known about how constraints affect learning from repeated design submissions and the effect of the technical constraints that are integral to online platforms. How do creative versus te… ▽ More Online innovation communities allow for a search for novel solutions within a design space bounded by constraints. Past research has focused on the effect of creative constraints on individual projects, but less is known about how constraints affect learning from repeated design submissions and the effect of the technical constraints that are integral to online platforms. How do creative versus technical constraints affect individual learning in exploring a design space in online communities? We analyzed ten years of data from an online innovation community that crowdsourced 136,989 design submissions from 33,813 individuals. We leveraged data from two types of design contests-creatively constrained and unconstrained-running in parallel on the platform, and we evaluated a natural experiment where a platform change reduced technical constraints. We find that creative constraints lead to high rates of learning only if technical constraints are sufficiently relaxed. Our findings have implications for the management of creative design work and the downstream effects of the technical constraints of the information systems that support online innovation communities. △ Less

Submitted 27 March, 2023; originally announced March 2023.

ACM Class: J.4; J.5

Cooperation in the Gig Economy: Insights from Upwork Freelancers

Authors: Zachary Fulker, Christoph Riedl

Abstract: Existing literature predominantly focuses on how freelancers individually complete tasks and projects. Our study examines freelancers' willingness to work collaboratively. We report results from a survey of 122 freelancers on a leading online labor market platform (Upwork) and examine freelancers' preferences for collaboration and explore several antecedents of cooperative behaviors. We then test… ▽ More Existing literature predominantly focuses on how freelancers individually complete tasks and projects. Our study examines freelancers' willingness to work collaboratively. We report results from a survey of 122 freelancers on a leading online labor market platform (Upwork) and examine freelancers' preferences for collaboration and explore several antecedents of cooperative behaviors. We then test if actual cooperative behavior matches with freelancers' stated preferences through an incentivized social dilemma experiment. We find that respondents cooperate at a higher rate (85%) than reported in previous comparable studies (between 50-75%). This high rate of cooperation may be explained by an ingroup bias. Using a sequential mediation model we demonstrate the importance of a sense of shared expectations and accountability for cooperation. We contribute to a better understanding of the potential for collaborative work on online labor market platforms by assessing if and what social factors and collective culture exist among freelancers. We discuss the implications of our results for platform designers by highlighting the importance of platform features that promote shared expectations and improve accountability. Overall, contrary to existing literature and predictions, our results suggest that freelancers in our sample display traits that are more consistent with belonging to a coherent group with a shared collective culture, rather than being anonymous actors in a transaction-based market. △ Less

Submitted 11 November, 2024; v1 submitted 20 January, 2023; originally announced January 2023.

Spontaneous emergence of groups and signaling diversity in dynamic networks

Authors: Zachary Fulker, Patrick Forber, Rory Smead, Christoph Riedl

Abstract: We study the coevolution of network structure and signaling behavior. We model agents who can preferentially associate with others in a dynamic network while they also learn to play a simple sender-receiver game. We have four major findings. First, signaling interactions in dynamic networks are sufficient to cause the endogenous formation of distinct signaling groups, even in an initially homogene… ▽ More We study the coevolution of network structure and signaling behavior. We model agents who can preferentially associate with others in a dynamic network while they also learn to play a simple sender-receiver game. We have four major findings. First, signaling interactions in dynamic networks are sufficient to cause the endogenous formation of distinct signaling groups, even in an initially homogeneous population. Second, dynamic networks allow the emergence of novel {\em hybrid} signaling groups that do not converge on a single common signaling system but are instead composed of different yet complementary signaling strategies. We show that the presence of these hybrid groups promotes stable diversity in signaling among other groups in the population. Third, we find important distinctions in information processing capacity of different groups: hybrid groups diffuse information more quickly initially but at the cost of taking longer to reach all group members. Fourth, our findings pertain to all common interest signaling games, are robust across many parameters, and mitigate known problems of inefficient communication. △ Less

Submitted 12 January, 2024; v1 submitted 22 October, 2022; originally announced October 2022.

Collective Intelligence in Human-AI Teams: A Bayesian Theory of Mind Approach

Authors: Samuel Westby, Christoph Riedl

Abstract: We develop a network of Bayesian agents that collectively model the mental states of teammates from the observed communication. Using a generative computational approach to cognition, we make two contributions. First, we show that our agent could generate interventions that improve the collective intelligence of a human-AI team beyond what humans alone would achieve. Second, we develop a real-time… ▽ More We develop a network of Bayesian agents that collectively model the mental states of teammates from the observed communication. Using a generative computational approach to cognition, we make two contributions. First, we show that our agent could generate interventions that improve the collective intelligence of a human-AI team beyond what humans alone would achieve. Second, we develop a real-time measure of human's theory of mind ability and test theories about human cognition. We use data collected from an online experiment in which 145 individuals in 29 human-only teams of five communicate through a chat-based system to solve a cognitive task. We find that humans (a) struggle to fully integrate information from teammates into their decisions, especially when communication load is high, and (b) have cognitive biases which lead them to underweight certain useful, but ambiguous, information. Our theory of mind ability measure predicts both individual- and team-level performance. Observing teams' first 25% of messages explains about 8% of the variation in final team performance, a 170% improvement compared to the current state of the art. △ Less

Submitted 28 March, 2023; v1 submitted 24 August, 2022; originally announced August 2022.

Comments: 9 pages, Accepted at AAAI 2023

AI-assisted Optimization of the ECCE Tracking System at the Electron Ion Collider

Authors: C. Fanelli, Z. Papandreou, K. Suresh, J. K. Adkins, Y. Akiba, A. Albataineh, M. Amaryan, I. C. Arsene, C. Ayerbe Gayoso, J. Bae, X. Bai, M. D. Baker, M. Bashkanov, R. Bellwied, F. Benmokhtar, V. Berdnikov, J. C. Bernauer, F. Bock, W. Boeglin, M. Borysova, E. Brash, P. Brindza, W. J. Briscoe, M. Brooks, S. Bueltmann , et al. (258 additional authors not shown)

Abstract: The Electron-Ion Collider (EIC) is a cutting-edge accelerator facility that will study the nature of the "glue" that binds the building blocks of the visible matter in the universe. The proposed experiment will be realized at Brookhaven National Laboratory in approximately 10 years from now, with detector design and R&D currently ongoing. Notably, EIC is one of the first large-scale facilities to… ▽ More The Electron-Ion Collider (EIC) is a cutting-edge accelerator facility that will study the nature of the "glue" that binds the building blocks of the visible matter in the universe. The proposed experiment will be realized at Brookhaven National Laboratory in approximately 10 years from now, with detector design and R&D currently ongoing. Notably, EIC is one of the first large-scale facilities to leverage Artificial Intelligence (AI) already starting from the design and R&D phases. The EIC Comprehensive Chromodynamics Experiment (ECCE) is a consortium that proposed a detector design based on a 1.5T solenoid. The EIC detector proposal review concluded that the ECCE design will serve as the reference design for an EIC detector. Herein we describe a comprehensive optimization of the ECCE tracker using AI. The work required a complex parametrization of the simulated detector system. Our approach dealt with an optimization problem in a multidimensional design space driven by multiple objectives that encode the detector performance, while satisfying several mechanical constraints. We describe our strategy and show results obtained for the ECCE tracking system. The AI-assisted design is agnostic to the simulation framework and can be extended to other sub-detectors or to a system of sub-detectors to further optimize the performance of the EIC detector. △ Less

Submitted 19 May, 2022; v1 submitted 18 May, 2022; originally announced May 2022.

Comments: 16 pages, 18 figures, 2 appendices, 3 tables

Avoiding the bullies: The resilience of cooperation among unequals

Authors: Michael Foley, Rory Smead, Patrick Forber, Christoph Riedl

Abstract: Can egalitarian norms or conventions survive the presence of dominant individuals who are ensured of victory in conflicts? We investigate the interaction of power asymmetry and partner choice in games of conflict over a contested resource. We introduce three models to study the emergence and resilience of cooperation among unequals when interaction is random, when individuals can choose their part… ▽ More Can egalitarian norms or conventions survive the presence of dominant individuals who are ensured of victory in conflicts? We investigate the interaction of power asymmetry and partner choice in games of conflict over a contested resource. We introduce three models to study the emergence and resilience of cooperation among unequals when interaction is random, when individuals can choose their partners, and where power asymmetries dynamically depend on accumulated payoffs. We find that the ability to avoid bullies with higher competitive ability afforded by partner choice mostly restores cooperative conventions and that the competitive hierarchy never forms. Partner choice counteracts the hyper dominance of bullies who are isolated in the network and eliminates the need for others to coordinate in a coalition. When competitive ability dynamically depends on cumulative payoffs, complex cycles of coupled network-strategy-rank changes emerge. Effective collaborators gain popularity (and thus power), adopt aggressive behavior, get isolated, and ultimately lose power. Neither the network nor behavior converge to a stable equilibrium. Despite the instability of power dynamics, the cooperative convention in the population remains stable overall and long-term inequality is completely eliminated. The interaction between partner choice and dynamic power asymmetry is crucial for these results: without partner choice, bullies cannot be isolated, and without dynamic power asymmetry, bullies do not lose their power even when isolated. We analytically identify a single critical point that marks a phase transition in all three iterations of our models. This critical point is where the first individual breaks from the convention and cycles start to emerge. △ Less

Submitted 17 April, 2021; originally announced April 2021.

Journal ref: PLoS Computational Biology 17(4): e1008847, 2021

Optimal design of experiments to identify latent behavioral types

Authors: Stefano Balietti, Brennan Klein, Christoph Riedl

Abstract: Bayesian optimal experiments that maximize the information gained from collected data are critical to efficiently identify behavioral models. We extend a seminal method for designing Bayesian optimal experiments by introducing two computational improvements that make the procedure tractable: (1) a search algorithm from artificial intelligence that efficiently explores the space of possible design… ▽ More Bayesian optimal experiments that maximize the information gained from collected data are critical to efficiently identify behavioral models. We extend a seminal method for designing Bayesian optimal experiments by introducing two computational improvements that make the procedure tractable: (1) a search algorithm from artificial intelligence that efficiently explores the space of possible design parameters, and (2) a sampling procedure which evaluates each design parameter combination more efficiently. We apply our procedure to a game of imperfect information to evaluate and quantify the computational improvements. We then collect data across five different experimental designs to compare the ability of the optimal experimental design to discriminate among competing behavioral models against the experimental designs chosen by a "wisdom of experts" prediction experiment. We find that data from the experiment suggested by the optimal design approach requires significantly less data to distinguish behavioral models (i.e., test hypotheses) than data from the experiment suggested by experts. Substantively, we find that reinforcement learning best explains human decision-making in the imperfect information game and that behavior is not adequately described by the Bayesian Nash equilibrium. Our procedure is general and computationally efficient and can be applied to dynamically optimize online experiments. △ Less

Submitted 6 August, 2020; v1 submitted 10 July, 2018; originally announced July 2018.

Journal ref: Exp. econ. 24 (2021) 772-799

And Now for Something Completely Different: Visual Novelty in an Online Network of Designers

Authors: Johannes Wachs, Bálint Daróczy, Anikó Hannák, Katinka Páll, Christoph Riedl

Abstract: Novelty is a key ingredient of innovation but quantifying it is difficult. This is especially true for visual work like graphic design. Using designs shared on an online social network of professional digital designers, we measure visual novelty using statistical learning methods to compare an images features with those of images that have been created before. We then relate social network positio… ▽ More Novelty is a key ingredient of innovation but quantifying it is difficult. This is especially true for visual work like graphic design. Using designs shared on an online social network of professional digital designers, we measure visual novelty using statistical learning methods to compare an images features with those of images that have been created before. We then relate social network position to the novelty of the designers images. We find that on this professional platform, users with dense local networks tend to produce more novel but generally less successful images, with important exceptions. Namely, users making novel images while embedded in cohesive local networks are more successful. △ Less

Submitted 23 April, 2018; v1 submitted 16 April, 2018; originally announced April 2018.

Comments: accepted to 10th International ACM Web Science Conference, 2018, May 27-30, Amsterdam, The Netherlands, 11 pages, 6 figures, 60 references

Conflict and Convention in Dynamic Networks

Authors: Michael Foley, Patrick Forber, Rory Smead, Christoph Riedl

Abstract: An important way to resolve games of conflict (snowdrift, hawk-dove, chicken) involves adopting a convention: a correlated equilibrium that avoids any conflict between aggressive strategies. Dynamic networks allow individuals to resolve conflict via their network connections rather than changing their strategy. Exploring how behavioral strategies coevolve with social networks reveals new dynamics… ▽ More An important way to resolve games of conflict (snowdrift, hawk-dove, chicken) involves adopting a convention: a correlated equilibrium that avoids any conflict between aggressive strategies. Dynamic networks allow individuals to resolve conflict via their network connections rather than changing their strategy. Exploring how behavioral strategies coevolve with social networks reveals new dynamics that can help explain the origins and robustness of conventions. Here we model the emergence of conventions as correlated equilibria in dynamic networks. Our results show that networks have the tendency to break the symmetry between the two conventional solutions in a strongly biased way. Rather than the correlated equilibrium associated with ownership norms (play aggressive at home, not away), we usually see the opposite host-guest norm (play aggressive away, not at home) evolve on dynamic networks, a phenomenon common to human interaction. We also show that learning to avoid conflict can produce realistic network structures in a way different than preferential attachment models. △ Less

Submitted 22 February, 2018; originally announced February 2018.

MSC Class: 91-XX

Detecting Figures and Part Labels in Patents: Competition-Based Development of Image Processing Algorithms

Authors: Christoph Riedl, Richard Zanibbi, Marti A. Hearst, Siyu Zhu, Michael Menietti, Jason Crusan, Ivan Metelsky, Karim R. Lakhani

Abstract: We report the findings of a month-long online competition in which participants developed algorithms for augmenting the digital version of patent documents published by the United States Patent and Trademark Office (USPTO). The goal was to detect figures and part labels in U.S. patent drawing pages. The challenge drew 232 teams of two, of which 70 teams (30%) submitted solutions. Collectively, tea… ▽ More We report the findings of a month-long online competition in which participants developed algorithms for augmenting the digital version of patent documents published by the United States Patent and Trademark Office (USPTO). The goal was to detect figures and part labels in U.S. patent drawing pages. The challenge drew 232 teams of two, of which 70 teams (30%) submitted solutions. Collectively, teams submitted 1,797 solutions that were compiled on the competition servers. Participants reported spending an average of 63 hours developing their solutions, resulting in a total of 5,591 hours of development time. A manually labeled dataset of 306 patents was used for training, online system tests, and evaluation. The design and performance of the top-5 systems are presented, along with a system developed after the competition which illustrates that winning teams produced near state-of-the-art results under strict time and computation constraints. For the 1st place system, the harmonic mean of recall and precision (f-measure) was 88.57% for figure region detection, 78.81% for figure regions with correctly recognized figure titles, and 70.98% for part label detection and character recognition. Data and software from the competition are available through the online UCI Machine Learning repository to inspire follow-on work by the image processing community. △ Less

Submitted 11 November, 2014; v1 submitted 24 October, 2014; originally announced October 2014.

The Effects of Prediction Market Design and Price Elasticity on Trading Performance of Users: An Experimental Analysis

Authors: Ivo Blohm, Christoph Riedl, Johann Füller, Orhan Köroglu, Jan Marco Leimeister, Helmut Krcmar

Abstract: We employ a 2x3 factorial experiment to study two central factors in the design of prediction markets (PMs) for idea evaluation: the overall design of the PM, and the elasticity of market prices set by a market maker. The results show that 'multi-market designs' on which each contract is traded on a separate PM lead to significantly higher trading performance than 'single-markets' that handle all… ▽ More We employ a 2x3 factorial experiment to study two central factors in the design of prediction markets (PMs) for idea evaluation: the overall design of the PM, and the elasticity of market prices set by a market maker. The results show that 'multi-market designs' on which each contract is traded on a separate PM lead to significantly higher trading performance than 'single-markets' that handle all contracts one on PM. Price elasticity has no direct effect on trading performance, but a significant interaction effect with market design implies that the performance difference between the market designs is highest in settings of moderate price elasticity. We contribute to the emerging research stream of PM design through an unprecedented experiment which compares current market designs. △ Less

Submitted 16 April, 2012; originally announced April 2012.

Comments: Presented at Collective Intelligence conference, 2012 (arXiv:1204.2991)

Report number: CollectiveIntelligence/2012/77