Subscribe to the OSS Weekly Newsletter!

Register for the OSS 25th Anniversary Event

Mindfulness Meditation Often Fails the Scientific Test

When studying mindfulness meditation, we should be mindful of the kind of control group we use

Mindfulness meditation is primed to save all of us from anxiety and depression, it would seem. It’s also big business: in 2015, this growing industry made nearly USD 1 billion. The money comes from speaking engagements, workshops, books, as well as apps meant to guide you through the process. The popular Headspace app claims 70 million downloads; the #1 meditation app, Calm, over 150 million! Given the state of our world, the number of people seeking some relief from everyday stress shouldn’t be surprising.

But does it work? Mindfulness meditation has been the subject of increasing scrutiny by scientists. We have seen an exponential growth in the number of studies published about it, and 2023 alone saw the publication of a considerable number of studies of studies on mindfulness, attempts at summarizing what we have learned so far on how mindfulness might help with anxiety, depression, ADHD, pain, sleep, and cognition, in all age groups.

On the surface, it all looks promising. Beneath it, however, we find so many problems that the best conclusion I can come to is that something is better than nothing.

The small benefit that keeps disappearing

Mindfulness meditation has its roots in the Buddhist traditions of Asia. Its entry into the West is widely credited to Jon Kabat-Zinn, who was introduced to the technique by a Zen missionary who spoke at the Massachusetts Institute of Technology when Kabat-Zinn was earning his doctorate in molecular biology. He went on to study meditation and, in 1979, he founded a stress reduction centre at the University of Massachusetts Medical School, where he developed his now-famous eight-week mindfulness-based stress reduction course. It’s important to point out that mindfulness meditation is usually stripped of the spirituality and ethical philosophy within which it used to be practiced.

The goal of mindfulness meditation is not to empty your mind; rather, it’s about training yourself to pay attention to the present moment in a non-judgmental way. The breath is often used as an anchor. You are taught to pick a spot where you feel your breath, for example your nose or your chest. You pay attention to your breath going in and out. Worries, ideas, and regrets will inevitably pop into focus and distract you. You simply recognize that this is happening and you bring your attention back to the breath. It all sounds very easy but in practice it can be frustrating, especially given the number of distractions we are surrounded by these days.

The definition of mindfulness appears straightforward; in reality, though, it is akin to trying to grab hold of a wet fish, which makes studying the practice challenging. In his best-selling book on the broader subject, How to Meditate, psychotherapist Lawrence LeShan pointed out that meditating can be done in a variety of ways, including arranging flowers, practicing martial arts, and religious singing. With such a sweeping toolkit, it’s hard to know exactly where mindfulness begins and ends. As reported by Brian Resnick for Vox, one scientific study had children colouring for 15 minutes, which the researchers said was a “mindfulness-based” activity. Mindfulness can also be one part of a larger therapy, as in Acceptance and Commitment Therapy. Whatever benefits are measured there, are they really caused by the mindfulness component or by the other ones? Many scientists are worried that this imprecision makes comparisons between studies very difficult.

That hasn’t stopped the explosion of systematic reviews and meta-analyses on the subject, however. Many of them report benefits for the therapy, but the devil hides in the details. When testing mindfulness in a randomized controlled trial, each participant ends up in one of two groups: the mindfulness group or a control group. Sometimes, the control group is an active control, meaning that its participants are given some sort of intervention. It may be psychoeducation or even listening to a guided meditation but being instructed to count the verbs instead of engaging with it. But often, the control group is a passive control. Participants may be put on a waiting list to receive the mindfulness training much later. Thus, while the people in the mindfulness group are doing something to deal with their anxiety, depression or insomnia, the people in the passive group do nothing. Is it any wonder, then, that mindfulness is almost always superior to passive control?

The kicker is that mindfulness is rarely shown to be better than an active control. Many authors of systematic reviews of the evidence will excitedly report that mindfulness showed a small but significant benefit over people in the control group, before quietly adding that there was no difference when mindfulness was strictly compared to active control groups. The conclusion here is that doing something—anything, it would seem—is better than doing nothing. I have seen this frequently when so-called complementary or alternative medical interventions are touted as being scientifically proven: often, they were compared to people receiving nothing. If you’re in pain or mildly anxious, seeing someone, even someone whose intervention is based on magical beliefs, will feel better than being ignored.

Even when researchers put mindfulness’ best foot forward, the effect it seems to have is small. Despite an accumulation of ho-hum results, I read many discussion sections of papers and quite a few editorials spinning essentially negative results into enthusiasm for future research. This is a good example of the white hat bias, which is when information is distorted because the goal is seen as righteous. It would be great if mindfulness meditation could credibly treat anxiety, insomnia, and pain perception! Thus, many scientists in the field seem intent on proving that mindfulness is good. This attitude is not limited to this practice; rather, the image we have of scientists trying to disprove their hypothesis is, in my experience, rarely seen in practice. Scientists want to prove that cool and interesting things exist, even when the evidence before them is far from convincing.

Some will argue that mindfulness meditation has worked for them. It may be because they have been practicing it for years; studies tend to be much shorter, so if the benefits really take time to manifest, they are unlikely to be unearthed by current research. It may also be because mindfulness can work, just that there is apparently nothing magical or specific about it. A similar relief might be felt by practicing other relaxing activities. They may also report a deeper change in themselves, perhaps even a sort of mystical experience or a spiritual awakening. This kind of outcome is extremely hard to measure in the context of a study and unlikely to surface when said study wraps up after a measly two months.

And these studies have so many problems which showcase how hard it is to investigate the human mind. Sometimes, researchers will report that mindfulness improved a composite score in participants, meaning an aggregate of different measures like anxiety, depression, and stress. For example, a meta-analysis showed that mindfulness really worked for adolescents in high school when looking at that composite score… but the benefits disappeared when looking only at anxiety or depression. Also, even something as straightforward as anxiety can be measured using many different questionnaires, but are they really all measuring the same thing? Then there is who you test. Can you compare children to teenagers? Can teenagers and young adults be combined? What if age makes a difference in how you respond to a mindfulness intervention?

Beyond these problems, there is the crucial difference between efficacy and effectiveness, two words which are not synonymous in science. The efficacy of mindfulness is when the intervention is done under ideal conditions; its effectiveness is how it fares in the real world. Those world-famous apps to help you meditate? Maybe they work a little bit, but they certainly don’t work if you don’t use them. When deployed in the context of a study, a quarter of participants abandon them during the study itself. (This number is similar to what is seen with mental health apps in general.) The dropout can be partly avoided by paying the participant, but it’s unlikely that Headspace will start paying its customers to continue using the app. Outside of research projects in which people are told to use a meditation app, the numbers are even worse. Thousands of people who chose to get a one-year membership to the Calm app were followed in time, and over half of them had abandoned the app within a year. For those who stuck with it, the average duration of meditating was less than four minutes a day. This reminds me of the time before the Apple Watch, when dedicated fitness trackers very often ended up in a drawer. That tracker may have been efficacious in a trial, but it has zero effectiveness when you don’t wear it.

If you still want to try mindfulness meditation to see if it makes a difference in your life, you should also be aware of its darker side. The harms may be subtle, but we should be mindful of them.

Happiness is an inside job

Meditating is pretty harmless for most people, but scholars have raised a concern that its side effects are often not systematically reported in the literature. There have been examples of people experiencing panic attacks, traumatic flashbacks, depersonalization, disorientation, even psychosis following mindfulness. These undesirable consequences appear to be rare, and it’s possible they affect very specific types of people who should be told to stay away from it, but given how poorly documented these effects are, it is impossible to truly estimate how common they are.

Despite the lukewarm literature on the benefits of mindfulness, there is something unsettling about how quickly it was adopted by employers and schools. Part of the reason why is that it displaces societal problems onto the individual. The message often conveyed is that if people are stressed out or depressed or anxious about the world around them, it’s because they’re not meditating enough. We already see this with pollution and climate change: how easily corporations will put the onus on consumers to stop littering and to recycle. As Professor Ronald Purser phrased it in a scathing essay, “instead of setting practitioners free, [mindfulness meditation] helps them adjust to the very conditions that caused their problems.” Why should people in power improve the system we live in when they can simply tell us to meditate and get used to it?

The bottom line, though, is that I am not against mindfulness meditation. I am against its promotion as a coping tool by organizations that refuse to change and against its proselytizing as a revolution-in-the-making, when the mountains of scientific evidence so far show that, in the short term at least, its benefits are no different than those of similar stress-relieving activities and its potential harms are poorly understood. For us to gain a better comprehension of what mindfulness can do, we will need much longer studies with active control groups and multi-year follow-ups to see if people keep up with the meditation after the study ends and if it pays dividends then.

If you want to give mindfulness a try, simply focus on your breath and keep your expectations low.

Take-home message:
- Mindfulness meditation is a practice which teaches you to focus on the present moment and redirect your attention to it without judgement when you get distracted
- When studied in randomized clinical trials for a variety of issues like anxiety, depression, and stress, mindfulness meditation often shows a small benefit over not doing anything, but that benefit commonly disappears when it is compared to doing something other than meditation
- Mindfulness meditation appears to be safe for most people, although reports of complications like traumatic flashbacks and psychosis have led to calls for scientists to better report side effects in studies of this form of meditation


@CrackedScience

Back to top