Modeling cooperative and competitive decision-making in the Tiger Task

Abstract

The mathematical models underlying reinforcement learning helpus understand howagents navigate the world and maximize future reward. Partially observable Markov Decision Processes (POMDPs) –an extension of classic RL –allow for action planning in uncertainenvironments. In this study we set out to investigate human decision-making under these circumstances in the context of cooperation and competitionusing the iconic Tiger Task(TT)in single-player and cooperative and competitive multi-player versions. The task mimics the setting of a game show,in which the participant has to choose between two doors hiding eithera tiger (-100 points) or a treasure (+10 points)or taking a probabilistic hint about the tiger location (-1 point). In addition to the probabilistic location hints, the multi-player TTalso includesprobabilistic information about the other player’s actions.POMDPshave been successfully used in simulations of the single-player TT. A critical feature are the beliefs (probability distributions) about current position in the state space. However, here we leverage interactivePOMDPs(I-POMDPs) for the modeling choicedata from the cooperative and competitive multi-player TT. I-POMDPs construct a model of the other player’s beliefs, which are incorporatedinto the own valuation process. We demonstrate using hierarchical logistic regression modeling that the cooperative context elicits better choices and more accurate predictions of the other player’s actions. Furthermore, we show that participantsgenerate Bayesian beliefs to guide their actions. Critically, including the social information in the belief updating improves model performanceunderliningthat participants use this informationin theirbeliefcomputations. In the next step we will useI-POMDPs that explicitly modelother playersas an intentional agentsto investigate the generationof mental modelsand Theory of Mind in cooperative and competitive decision-making in humans.

Publication
4th Multi-disciplinary Conference on Reinforcement Learning and Decision Making (RLDM 2019)