An interesting paper came across my radar week – “Placebo effects in cognitive training” published in PNAS (with a paywall, unfortunately). In this paper, the authors investigate the effect that people’s expectations might have on the results of brain-training experiments. Every research study needs to recruit people to participate, and a common way is to put up posters around campus (because many brain-training studies are done at universities). The authors of this new paper wanted to investigate if they could influence the results of a brain-training experiment by simply changing how the experiment was described. So in half of the posters, they described the study using very positive language – “Brain Training & Cognitive Enhancement – numerous studies have shown working memory training can increase fluid intelligence.” In the other half of the posters, they described the study using neutral language – “Email Today & Participate in a Study – need credits? Sign up today and earn up to 5 credits.” People responding to both posters then did a baseline assessment of intelligence, then did a short amount of working memory training, then took the intelligence test again. People who responded to the poster with positive language – “working memory can increase fluid intelligence” – showed an improvement in intelligence, but people who responded to the poster with neutral language – “earn up to 5 credits” – did not.
The authors made a reasonable interpretation – there may be differences between people who come into a brain-training experiment expecting to benefit, and people who come in without that expectation. If brain-training experiments promise benefits, then there may be a placebo effect – people expecting benefits will get them, in the same way that patients in pain who take a sugar pill can experience pain relief. The authors go on to suggest the scientists running trials of brain training be careful in how they recruit participants – it’s possible that the expectations set in recruitment can have an effect on the results. That’s an interesting result, although not a surprising one, and it’s always helpful to advance our understanding of how to run clinical trials well.
But the authors go on to make some very broad and critical comments about brain training, which are not at all justified by their results. They argue that their study (with 50 undergraduates, all in their late teens or early twenties, all recruited at a single university) has “broad implications for the advancement of science of human cognition” and that “the brain-training industry may be advised to temper their claims until the role of placebo effects is better understood.” And some of the commentary about this article has gone even further. ArsTechnica, a website where I usually find quite thoughtful articles on science and technology, said that the study suggested that the “Billion-dollar brain training industry a sham—nothing but placebo.”
These criticisms aren’t supported by the data in the study. Here’s why: the best, gold-standard studies of brain training use a control group that engages in an another activity that could improve brain function. For example, in Posit Science’s IMPACT trial, half the participants did brain training from Posit Science, and half studied educational DVD’s and took quizzes. So regardless of the recruitment fliers, both groups had the same expectation of benefit. If the recruitment materials implied benefit, then both groups might have experienced a placebo effect. But the brain-training group in IMPACT still showed a four-times larger gain in memory than the control group. So the results were clearly from the brain training, not from the recruitment posters.
By happy coincidence, a very nice paper illustrating this exact point was published last week. Researchers from the University of Rochester, in a paper entitled “Cognitive and Neural Effects of Vision-Based Speed-of-Processing Training in Older Adults with Amnestic Mild Cognitive Impairment: A Pilot Study” and published in the Journal of the American Geriatric Society (also, unfortunately behind a paywall) randomly assigned people to do brain training (from Posit Science) or mentally stimulating activities (games and puzzles). They showed that brain training led to significantly greater improvements in processing speed, attention, working memory, and activities of daily living; as well as showing direct effects on brain activity in the central executive network and the default mode network. By using a very strong active control, this study clearly shows that it is the brain training that drives the substantial results – not a non-specific effect of recruitment materials, or something else.
Throwing out all results of brain training studies based on the over-sensationalizing of narrow results about recruitment materials is clearly an overreaction. Our field should focus on learning from what well-designed and well-executed randomized controlled trials show us, where real benefits have been shown over and over again.
