Research which aims to understand and prevent adverse effects of psychological therapies – yes, really

by Julia Evans on March 8, 2012

Contents:  b) Comments on this misuse of research a) Information received – a description of research which is in progress.

a) Comments on the misuse of research

Over the years, there have been many regx2 posts on research which is politically driven. Political because usually the answer to my informant’s question ‘who paid for it?’, can be traced back to the Government. This research is not value free – it is based in political expediency and how to justify it.

I find it useful to use the experimenter’s errors, named Type I and Type II errors, to weigh up this research. Over 40 years ago, I was made to submit my experimental findings, vaguely in the area known as psychology, to both tests.  The following is from www. Experiment-resources.com which has updated my memory of two types to three and is available here.

Selected quotes:

With any scientific process, there is no such ideal as total proof or total rejection, and researchers must, by necessity, work upon probabilities. That means that, whatever level of proof was reached, there is still the possibility that the results may be wrong.

This could take the form of a false rejection, or acceptance, of the null hypothesis.

HOW DOES THIS TRANSLATE TO SCIENCE

TYPE I ERROR

A Type I error is often referred to as a ‘false positive’, and is the process of incorrectly rejecting the null hypothesis in favor of the alternative.

TYPE II ERROR

A Type II error is the opposite of a Type I error and is the false acceptance of the null hypothesis.

REPLICATION

This is the reason why scientific experiments must be replicable, and other scientists must be able to follow the exact methodology.

TYPE III ERRORS

Many statisticians are now adopting a third type of error, a type III, which is where the null hypothesis was rejected for the wrong reason.

In an experiment, a researcher might postulate a hypothesis and perform research. After analyzing the results statistically, the null is rejected.

The problem is, that there may be some relationship between the variables, but it could be for a different reason than stated in the hypothesis. An unknown process may underlie the relationship.

CONCLUSION

Both Type I errors and Type II errors are factors that every scientist and researcher must take into account.

Whilst replication can minimize the chances of an inaccurate result, this is one of the major reasons why research should be replicatable.

Many scientists do not accept quasi-experiments, because they are difficult to replicate and analyze. End quote.

 

I suggest that this proposed study contains all three errors and is not replicable. For starters, the experiment’s design does not establish the null hypothesis. Even if they manipulate their data to show therapy is dangerous – what does this mean? Individual recipients and practitioners are unique human beings and they vary. It is not possible to apply the categorised results anywhere away from the unique population of experimental subjects.  The subjects being experimented on are unique. Type III errors are rife.  I suggest the relationship between the variables is the experimenter’s interest in making abuse appear and the volunteers compliance with this interest.

 

Some erroneous assumptions in which, I suggest, this Government-funded research is based.

 

a) ‘Many people with mental health difficulties are helped by psychological therapies.’

JE:  How do they know?

What is their definition of mental health difficulties? How do the experimenters make sure that their definition of ‘mental health difficulties’ is reproducible, that is, the limits are so defined that everyone looking at the research data or involved in replicating the research, uses the same criteria of ‘mental health difficulties’. Or is this driven by the Government’s drive to get those who are economically dependent on the Government, off long-term sickness benefits. Thus, the definition is anyone who is not working a 35 hour week.  This is a criterion which works and is reproducible.

What do they mean ‘helped by psychological therapies’? So, does this mean that on the IAPT   programme, or similar, when fed-up recipients tick all the yes boxes to escape, that this means they have been ‘helped’? [N.B. evaluation sheets are given out at the end of each session so the assumption is that change can be seen after a single session.] Or is the definition of ‘being helped’ that the sufferers are back in regular full-time employment?

b)  ‘but there is some evidence from research studies and individual clients that people can occasionally feel worse after therapy.’  I bet Professor Fonagy and the unit at UCL are not far from this claim of evidence for harm.  Many people have written debunking their claims & I will dig out the references, once again, on request.  So this assertion is not well founded. Further, let us look at the claim that ‘people occasionally feel worse.’ What does this mean? How is it measured? And what external benchmarks are there for this ‘scientific’ term of feel worse.

This also brings forward the question ‘what is the purpose of psychological therapies’? I propose that this research project implies a fundamental misunderstanding of human-being-ness. There is no such thing as a standard human being and no such standard measurement of feeling worse or better and ….  In fact, there is no such thing as a standard human being.  We are all unique and different. As I wrote to the Government in several of my replies to their CON-sultations, if I chose to be a miserable bugger, of what business is that to the Government or high-minded academic researchers? And how do you measure whether I am worse or better on the standard measure of miserableness.  And it may be that psychological therapy would make me worse. But then, I like being miserable so this is an excellent result. (Must calm down – blood pressure must have gone up with all those ‘ands’) And what is the objective of psychological treatment, is a valid question to which ‘producing emotionally-standard human beings’ is not the answer.

 

To conclude, for this rant: This research is a waste of money because

– it is badly set up with the experimenters making many fallacious judgments, such as, it is possible to produce meaningful results given the definition of ‘worse’ will vary within the research team and from one experimental subject to the next. It is what could be described as adding one pear to one orange and making two what?

– it is impossible to put all psychological therapy into one framework and state they all produce the same Government-standard measure of Wellbeing – see HPO2001.

– those who volunteer are a skewed sample. Why are they volunteering? What is in it for them? Maybe they enjoy stuffing the blame for their condition into someone else. Why not? It is a good game which gets you Government funding and lots of attention.

My conclusion is that this is yet another academic institution which has sold out.  They are prepared to research anything in order to please the Government and/or secure their mortgages. They also support the contention that a completely risk-free talking therapy with consistent results of standard wellbeing is possible. So that is their wages paid for the next 3 years and then they can go on to managing ‘Happiness Factories’ to produce Lord Layard’s ‘economically happy’ people. Does anyone read ‘Animal Farm’ or ‘Brave New World’?

Recommended further reading:

You are recommended to read a previous post:  ‘Simplification is necessary in research. But it has limitations in providing a grand theory of everything.’ 3rd March 2012

Also the post ‘The Government as Sadeian experimenter by Julia Evans on August 17, 2011’ shows how, if the experimenter sets the stage up for abuse to appear, it will. What does this prove? Nothing except the experimenter’s interest in abuse, is verified.

 

b) Information received and downloaded from here. Thank you to my informant.

Quote:

 AdEPT – Supporting Safe Talking Treatments

AdEPT:Understanding and Preventing the Adverse Effects of Psychological Therapies

If you have experienced therapy as a therapist or client, that you feel has ‘gone wrong’ or been harmful, we are keen to hear from you. It doesn’t matter whether you have used NHS services or not but you need to live in England and be aged 18 years or older in order to take part.

We can accept completed questionnaires up until May 2012

There are different ways to take part in our study, filling in a questionnaire, being interviewed, and/or taking part in a focus group – everyone has to initially complete a questionnaire. [JE: Invitation omitted]

Project Title

Understanding and Preventing Adverse Effects of Psychological Therapies (AdEPT)

Many people with mental health difficulties are helped by psychological therapies (“talking treatments”), but there is some evidence from research studies and individual clients that people can occasionally feel worse after therapy. We do not know how often this is because of the treatment as people could have become more distressed anyway, for example, after stressful life events. The AdEPT study is aimed at understanding and preventing these adverse effects (feeling worse) following psychological therapy.

There are several strands to the project:

  • Reviewing the existing evidence about the nature and extent of the problem of adverse effects within psychological therapy.
  • Analysing existing datasets to determine what kind of people in what type of therapy, with what kind of therapists are most likely to experience adverse effects within psychological therapies.
  • Undertaking in-depth interviews with clients and therapists where therapy has gone wrong to explore the process of failing or harmful therapy and generate understanding of what may have prevented the problems.

All of this work will lead to the main objective of the project: to develop and test practical support tools for clients, therapists and service managers to reduce adverse effect and prevent harm within psychological therapy.

Funder

This research is being funded by the NIHR Research for Patient Benefit (RfPB). It is due to last for three years from January 2011 to December 2013.

Project Team

The Project team is made up of people with diverse backgrounds, including people who have used psychological therapy services, therapists and academic researchers. We believe this to be important to ensure that differing perspectives are represented throughout the study.

Mental Health Group, Health Services Research
School of Health and Related Research (ScHARR)
University of Sheffield