Science and Pseudoscience in Clinical Psychology
To say that sometimes my reading list slips off into the odd is an understatement. Sometimes I’m reading some really clinical research based books. I’m trying to make sure that I’m really understanding a topic and I realize that I may be reading more about psychology than most practitioners, but I believe that there are key insights to be gained. This time I’m looking at the Science and Pseudoscience in Clinical Psychology. Last time I was looking at The Heart and Soul of Change: Delivering what Works in Therapy. This follows on the heels of The Cult of Personality Testing where I got to see some of the underbelly of the psychological assessment part of the industry. I needed to learn more about just how bad the work on personality testing was – and to pick up a broader perspective of the issues in the industry.
The book is organized into chapters with specific focuses. As a collection of works from various authors, there’s not a great deal of tie-in between chapters so here I’ll address some general issues with pseudoscience in clinical psychology and from there I’ll walk through the topics of the chapters by indicating the chapter subject in headings.
Pseudoscience
Mutual fund managers have been shown to be no better at predicting the market than an index fund. (See Do Active Mutual Fund Managers ‘Beat the Market’?) When there is ambiguous or inconsistent feedback one can easily delude themselves into believing that it’s their skill that allowed them to earn money in the market. This is true even when a more detailed look at the evidence shows that few (if any) will beat the market.
Thinking, Fast and Slow calls this a zero-validity environment. That is there is no reasonable way that they can make the kind of predictions necessary to accurately forecast – and therefore capitalize on – the opportunity.
The problem isn’t that the problem is unsolvable, it’s that people believe they’ve solved it. Whether it’s the Rorschach Inkblot test or the Thematic Apperception Test (TAT) there’s no reference by which to measure it and so the clinician can’t get reliable clues or feedback from their clinical practice in sufficient quantity and reliability to determine that the results are little better than chance.
Worse yet, despite having a battery of tools – some effective and some not – most of the social issues that we’re most concerned with – such as abusive parents – will demonstrate no mental defects that are detectable on the instruments that we have – including the Rorschach which has a noted bias to overpathologize subjects.
Science and Pseudoscience in Psychology says “Munchausen mothers force doctors to impose treatments on their children by interpreting ‘borderline’ medical conditions as problems needing intervention.” Clinicians have ineffective tools to determine whether the medical condition is real or “borderline” and whether the treatment is appropriate or not. Diagnosis of ADHD and the subsequent prescription of stimulants to children and adolescents has made stimulants more prescribed for these individuals than antibiotics.
Depression effects approximately 8% of the population today. (CDC, 2011) From 2005-2008 1 in 10 (10%) of Americans 12 and older took an antidepressant. And yet, only 43% of the studies of antidepressants sent to the FDA showed any statistically significant benefit compared to a placebo.
In short, clinicians don’t know what works and what doesn’t. They’ll prescribe a SSRI (Selective Serotonin Reuptake Inhibitor) because they don’t know what else to do to make the depression better.
If the result of science is the development of techniques which have reliable and predictable outcomes, then there is still a great deal of pseudoscience in psychology. Whether it’s anti-depressants that have dubious effectiveness or testing techniques that have minimal external and internal validity, pseudoscience is still rampant.
Controversial and Questionable Assessment Techniques
Do you ever look at your horoscope? Do you ever wonder how the horoscope seems to have something applicable to you? Do you feel like it is a truth? Or perhaps you feel, as I do, that it’s an interesting diversion that sometimes causes me to remember to live out my values. Either way, horoscopes are perceived by many to have some things that resonate. However, so do cold readings – that is a technique that psychics and fortune-tellers use to encourage their subjects to believe they know more than they actually do.
They throw out a topic and try to get the subject to fill in the information. The subject fills in information and the psychic moves forward in that line of thinking. If they get something wrong, they redirect and try to get the subject to believe that they have some extrasensory understanding of the situation.
It turns out that personality tests are much the same thing. Subjects were randomly assigned to two groups. One group received a “Barnum” profile which was designed to “give a little something to everyone” or an Adult Child of an Alcoholic (ACOA) profile. The Barnum profile – named after P.T. Barnum – was general in nature and not specific to any one person. Of the ACOA in the study, 79% of them said the Barnum profile described them very well or better. Of the Non-ACOA participants 70% the Barnum profile described them well. Of those receiving the ACOA profile, 71% of the ACOA participants rated the profile very well or better. Of the non-ACOA subjects in the ACOA profile group 63% said it described them very well.
There are a few conclusions to draw from this. First, people are eager to say that a personality profile matches them. Second, ACOA subjects liked a fictional, non-specific profile 8% more than one which was founded on the characteristics found in most ACOA subjects. In short our ability to discern whether a profile is accurate or is just interesting is questionable.
I’ve so extensively covered the issues with testing in The Cult of Personality that I won’t revisit the additional criticism leveled here, except to say that every so called projective test suffers from reliability problems.
Expert Testimony
Perhaps the most challenging aspect of expert testimony is what constitutes an expert. If so many people believe they’re in the top of their field how do you know who really is – and who is not? In the context of legal proceedings the answer is supposed to be that they rely on sound scientific principles under their opinions and beliefs. While they’re not required to initially disclose how they formed their opinions, they may be required to divulge this information on cross examination.
The standard for evidence – at a federal level – are the Federal Rules of Evidence and the major decisions in the cases summarized as Daubert, Joiner, and Kumho. Daubert sets the four major factors for evaluation of a trial judge which are:
- The theory or technique is scientific knowledge that is testable;
- The theory or technique has been subjected to peer review;
- The rate of potential error is known; and
- The theory or technique has gained general acceptance in the field.
Daubert says that the expert’s opinion must rest on a reliable foundation and must be relevant to the task at hand.
Joiner affirms the weaker decision in Daubert and says “trial courts have broad discretion to reject proffered expert opinions if they are inadequately supported by the data.” In short, if the expert can’t provide that they have a reasonable foundation, it can be rejected.
Kumho further broadens the trial judge’s ability by allowing the trial judge to reject testimony that is considered “technical” as well as scientific.
In the end, experts are supposed to be able to provide a reliable foundation for their testimony. Sadly in many cases, particularly where protective tests are used no reliable foundation exists (based on the research data) and yet the experts are allowed to proffer their beliefs. Though “It is incumbent on expert witnesses to acknowledge the limits of their competence and the evidential bases of their opinions, defend the data on which they rely to support their conclusions, and, to the extent possible, buttress their opinions with rigorous research findings” few actually do.
Science in Psychotherapy
We’d like to think that in psychotherapy – like medicine – that psychologists would be seeking out new research on treatment techniques and efficacy of the treatments they’re already using to continuously improve their practice. However, as was mentioned earlier, doctors with less experience and more training are more likely to use ECT appropriately. In 2000 a survey of 891 psychologists indicated that 47% never use evidence-based psychotherapy treatment manuals in their practice. In other words, it appears that efforts to encourage appropriate continuing education are currently insufficient to keep practitioners up with the latest scientific findings.
Like mentioned in The Heart and Soul of Change effectiveness is determined to a significant degree on patient-therapist alliance. The main three themes of which are defined as:
- the collaborative nature of the relationship;
- the affective bond between patient and therapist; and
- the patient’s and therapist’s ability to agree on treatment goals
The factors that impact the results of therapy are most notably alliance and secondarily the effect of the therapist. However, knowing someone is good and proving that they’re good are two different things.
Novel Unsupported Therapies
I firmly believe that modern medicine and psychotherapy don’t have people “all figured out.” The gaps that exist in our knowledge may be shrinking but with each discovery we realize how little we actually know. It’s no surprise then that people would seek therapies that offer answers that modern medicine can’t provide. The New Age movement (or so it is sometimes called) is focused around non-traditional attempts to provide remedies. Some of these rely upon old eastern medicine practices and some are focused on the holistic person more so than western medicine but the ability to determine the efficacy is dubious.
Some of the items discussed in this category are scientific in their origins – like Hubbard’s E-meter – but their ability to indicate anything useful or to treat a patient are unclear. For instance, techniques like the Emotional Freedom Technique (EFT) often known as tapping were studied and were found to have no therapeutic value. However, some of these ideas persist in practice and in the minds of adults.
Another example comes from Harvard Psychology Professor Richard J. McNally who noted, “The notion that traumatic events can be repressed and later recovered is the most pernicious bit of folklore ever to infect psychology and psychiatry. It has provided the theoretical basis for ‘recovered memory therapy’—the worst catastrophe to befall the mental health field since the lobotomy era.” Recovered memory “therapies” have harmed a great many people when the memories that were “recovered” were subsequently discovered to be false – as we’ll see in the next chapter.
Constructing the Past
Have you ever wondered why you can’t spontaneously remember everything from the moment of your birth? If our memories really work like a video cassette recorder shouldn’t everything that we have experienced be available to us? As it turns out, not only does our memory not work like a tape recorder but we also didn’t have the ability to record events in a way that can be recalled for the first 24 months of life. The structural changes in the brain which are required to store and recall complex events doesn’t typically develop until about the 24 month mark. And yet, recovered memories are reported to come as early as in utero – or from prior lives. This of course accepts that reincarnation is true which while may be acceptable to the Dalai Lama – it’s not been proven. (Then again no one has proven that God exists either.)
So what happens when you introduce hypnosis into the mix of memory “recovery”. The answer seems to be confabulation. There is research that asserts that the recall ability of someone in hypnosis is equivalent. What does happen, however, is that subtle hints provided by the hypnotist are expanded upon. In general, ideas introduced into our memories through others – whether accurate or inaccurate — tend to take on an air of accuracy.
Consider the research where family members were told to “remember” a story about an adult when they were a child which was false. After the story was told to the adult they began to believe it as truth – and remember it. They couldn’t in fact distinguish between the memory that was told to them by their family from a real memory. We often can’t remember the source of our memories.
Self-Help Therapy
It’s 1969 and George Miller has been elected president of the American Psychological Association and in his presidential address he encourages the members to “give psychology away.” Psychologists are investigating how to convert their treatment programs into self-help systems that would allow anyone to leverage the power of psychology to improve their lives. The self-help movement didn’t start with Miller’s address. There were books available since the 1950s including Norman Vincent Peale’s The Power of Positive Thinking that were already creating the self-help category. Miller’s address simply encouraged psychologists to bring their training to bear on the market.
Unfortunately, there were numerous problems that occurred in the market. First, the claims made by books and programs weren’t regulated or monitored by anyone and thus outlandish claims were printed on some materials. What’s worse even if the materials were based on sound, well-researched psychological treatments their conversion to self-help form seemed to render them ineffective – or worse. So there was little possibility to ensure that the self-help being produced would create any positive effect.
Trauma-Related Stress Disorders
There’s a strong desire to help folks who have been through trauma. If you’ve never been close to the death of someone or to seeing some of the inhumanities of man then it’s hard to explain the profound loss you feel. Over the years several techniques have developed for the treatment of trauma related stress disorders. I discussed one called Critical Incident Stress Debriefing (CISD) in my review of Redirect. CISD was found to be less effective than nonintervention or an alternative intervention. While an opportunity to free journal write about an incident seemed to be helpful, CISD seemed to create problems.
Other treatments, like Anxiety Management Training (AMT) does have research to support that it is an effective therapy. It is a set of cognitive and behavioral strategies designed to reduce symptoms of anxiety, irritability, and hyperarousal.
Alcohol Use Disorders
Alcoholics Anonymous (AA) is perhaps the most widely known programs for alcoholics and addiction resistance in general. Though the research is mixed on its effectiveness – in part due to the anonymous and distributed nature of the group – there’s some agreement that it’s a cost effective resource since there are no dues or fees and the groups are largely self-supporting. It’s believed that AA participation reduces overall health care costs.
However, AA isn’t the only game in town. There are numerous boot camps and programs like Drug Abuse Resistance Education (DARE), Scared Straight, and locally developed programs. Most of these programs, particularly DARE and Scared Straight-like programs are actually harmful. Specifically, The US Surgeon General has classified DARE as a potentially harmful treatment. So despite the relative ease of implementation, the affinity from the communities where it is tried, and the satisfaction of the officers participating in the program, it doesn’t work.
However, there are less popular approaches – such as controlled drinking – that are effective despite being shunned by the community or members of stricter programs. Controlled drinking in particular seeks to teach folks the skills to only drink in moderation and there by seeks to minimize the trap that total abstinence folks are put into. By making alcohol a shameful activity the first drink leads to a downward spiral of shame and coping – which controlled drinking avoids.
Herbs and Antidepressants
If I were to tell you that depression was one of the most widespread psychological disorders, it’s unlikely that you’d be surprised. If I told you that St. John’s Wort (hypericum) was as effective as some (if not most) of the antidepressants on the market – at a substantially lower cost – you might be surprised.
It’s rare that herbal remedies have the level of research to support their efficacy in clinical settings. It’s similarly rare for an entire class of medicines (antidepressants) to have such a strong placebo effect that the impacts of the drugs are often difficult to distinguish from placebo. So it seems that if you’re facing depression, a good place to start might be to get some St. John’s Wort.
However, there are some supplements, like Ginkgo Biloba that while they have medicinal uses, those uses have been harder to quantify. Sold as an aid to slow down the aging process, it’s effectiveness at improving standard cognitive tests hasn’t been exciting. Some newer approaches to measurement are showing some promise.
Attention-Deficit/Hyperactivity Disorder (ADHD)
ADHD is being diagnosed in record numbers. While at some level I believe we all have some level of ADHD, DSM-V has established criteria for diagnosing the condition. Historically the treatment was stimulants. While some non-stimulant treatments have become available for treating ADHD, they’re in the significant minority. For the most part psychology based treatments haven’t shown great promise at mitigating the disorder.
However, Behavioral Therapy (BT), particularly as it relates to educating the adults working with ADHD impacted children have been effective at managing the day-to-day symptoms of ADHD but unfortunately the effects are not lasting. It appears that BT can’t spare an ADHD child from the long-term negative outcomes associated with the disorder.
Autism Spectrum Disorders
Perhaps the greatest medical sham that’s been foisted on the public in the last 50 years is the one perpetrated by Andrew Wakefield. In an article (since retracted) in the prestigious medical journal Lancet, Wakefield and his colleagues asserted a causal relationship between vaccines and autism. Unknown to Wakefield’s colleagues he had a conflict of interest in the study. The investigation about the article concluded that Wakefield falsified the medical histories of 12 of the children in the study. Wakefield lost his medical license. However, unfortunately the stigma has remained. Untold numbers of children aren’t receiving vaccinations for important diseases because of Wakefield’s article.
Numerous therapies have been suggested for Autism spectrum disorders. Many like Dolphin-Assisted Therapy (DAT) have no scientific basis and seem to have been by perhaps well-meaning people trying to find a way to improve the situation using unsupported approaches.
Attachment Therapy
Sometimes things go wrong in medicine. Sometimes treatments don’t go as planned or aren’t sufficient to resolve the issues in the body of the patient and the patient dies. However, in psychotherapy it’s quite rare for things to go so wrong that a death is the result. However, Attachment Therapy (AT) has managed to have deaths and injuries as a result of the procedures and the “extensions” that some of the therapists have applied to AT.
Whether it’s rebirthing or another offshoot from AT that caused the issues, at the heart of the debate is the approach in AT of restraining the patient. While Attachment Disorder (AD) is real and a treatment is needed, AT seems to be risky. Several proponents including Zaslow and Cline have surrendered their licenses due to injuries to patients as the result of the treatment.
Antisocial Behavior
Persistent and serious forms of antisocial behavior are estimated to be perpetrated by 5-10% of children in developed Western countries. That’s a lot of kids who are struggling to adapt to society. Other estimates place the referrals to treatment are 33-50% from antisocial kids. The financial impact on society is staggering. There are numerous factors that lead to antisocial behavior in children and adolescents and not a lot of clear resolutions. (See Our Kids for more on the impacts of under-supported children.)
Wrapping It Up
There is too much pseudoscience in clinical psychology and not enough real science. Perhaps if you read Science and Pseudoscience in Clinical Psychology you’ll be able to spot the difference, and get to better outcomes.