NeuroRelay

We tend to be creatures of habit. In fact, the human brain has a learning system that is devoted to guiding us through routine, or habitual, behaviors. At the same time, the brain has a separate goal-directed system for the actions we undertake only after careful consideration of the consequences. We switch between the two systems as needed. But how does the brain know which system to give control to at any given moment?
Researchers from California Institute of Technology say that they have identified the mechanism in the human brain that balances decisions between habitual behaviors and spontaneous action. The brain has a learning system devoted to guiding people through routine (habitual behaviors), as well as a separate goal-directed system for actions undertaken only after careful consideration of consequences, they said. The results appear in the current issue of the journal Neuron and the paper is titled “Neural computations underlying arbitration between model-based and model-free learning”.
Scientists are reporting that for the first time they’ve pinpointed areas of the brain – the inferior lateral prefrontal cortex and frontopolar cortex – that seem to serve as an “arbitrator” between the two decision-making systems. Those brain areas weigh the reliability of the predictions each of those systems makes and then allocates control accordingly, principle investigator John O’Doherty said. Understanding the location of the arbitrator and how it works may eventually lead to better treatment for brain disorders, such as drug addiction, and psychiatric disorders such as obsessive-compulsive disorder. The arbitrator ensures that the system making the most reliable predictions at any moment exerts the greatest degree of control over behavior and it is basically making decisions about decisions, the researchers said.

In the study, participants played a decision-making game on a computer while connected to a functional magnetic resonance imaging scanner that monitored their brain activity. Participants were instructed to try to make optimal choices in order to gather coins of a certain color, which were redeemable for money. During a pre-training period, the subjects familiarized themselves with the game—moving through a series of on-screen rooms, each of which held different numbers of red, yellow, or blue coins. During the actual game, the participants were told which coins would be redeemable each round and given a choice to navigate right or left at two stages, knowing that they would collect only the coins in their final room. Sometimes all of the coins were redeemable, making the task more habitual than goal-directed. By altering the probability of getting from one room to another, the researchers were able to further test the extent of participants’ habitual and goal-directed behavior while monitoring corresponding changes in their brain activity.
Researchers compared the fMRI data and choices made by the subjects against several computational models they constructed to account for behavior, and the model that most accurately matched the experimental data involved the two brain systems making separate predictions about which action to take in a given situation. Receiving signals from those systems, the arbitrator kept track of the reliability of the predictions by measuring the difference between the predicted and actual outcomes for each system. It then used those reliability estimates to determine how much control each system should exert over the individual’s behavior. In this model, the arbitrator ensures that the system making the most reliable predictions at any moment exerts the greatest degree of control over behavior.