5 May 2017
Did we really say that?
Keith R Laws, Professor of Psychology University of Hertfordshire
Other Contributors:
Peter J McKenna, Psychiatrist
Sameer Jauhar, Psychiatrist
In their article Turkington & Lebert (2017) ask “What is the state of the current evidence base?” regarding psychological interventions for psychosis. In addressing this question, they cite a meta-analysis by ourselves (Jauhar et al 2014) and we would like to draw attention to significant misrepresentations and/or misinterpretations of our findings.
As the central platform in their case for Cognitive Behavioural Therapy (CBT), Turkington & Lebert (p.17) cite our meta-analysis - asserting that “Current meta-analyses are fairly consistent in their findings. CBT v. treatment as usual (TAU) delivers a moderate effect size benefit on overall symptoms, positive symptoms, negative symptoms and functioning (Jauhar 2014)”. Interpretations of effect sizes as ‘small’, ’medium’ or ‘large’ can be misleading and it is better to look at the actual effect sizes and preferably with some context. Cohen (1988), with some caveats, suggested classifying small, medium and large effect sizes as 0.2, 0.5 and 0.8 respectively. The Cochrane handbook also points to a range (<0.40 = small, 0.40 to 0.70 = moderate, >0.70 = large). The effect size reported by Jauhar et al (2014) for overall symptoms was 0.33, and so is not accurately described as medium (or ‘moderate’). To add further context, in 20 blind trials the effect size more than halved at 0.15. Indeed, the conclusion of our abstract asserts that: “Cognitive–behavioural therapy has a therapeutic effect on schizophrenic symptoms in the ‘small’ range. This reduces further when sources of bias, particularly masking, are controlled for.” (p.20)
Concerning the claim of consistent moderate effect sizes, the reality is that only one meta-analysis during the past decade has reported a mean effect size that reaches what might be described as medium/moderate (0.40 Wykes et al 2008); however, every other recent meta-analysis of total symptoms - regardless of study quality or study inclusion/exclusion - has reported smaller effect sizes (0.27 NICE 2009; 0.08 Lynch et al, 2010; 0.10 Sarin et al 2011; 0.04 Newton-Howes et al 2011; 0.33 Jauhar et al 2014; 0.16 Turner et al 2014).
Turning to the claim that CBT delivers a moderate effect size benefit for positive and negative symptoms, the reference to Jauhar et al again stretches the notion of moderate. In our meta-analyses, the ESs for positive and negative symptoms were .31, and .17 respectively – both again would conventionally be described as small. To contextualise these effects, they were further reduced to .08 and .04 respectively in trials using blind assessment of outcomes. Other recent meta-analyses of negative symptoms (Velthorst et al 2014; Turner et al 2014) reported nonsignificant ESs of .08 and 0.04 respectively.
A further citation of our meta-analysis in the same paragraph is also misleading - Turkington & Lebert say that “In head to head comparison of CBT v another psychological treatment with patients already stabilised on medication, CBT has a small but statistically significant benefit over all head-to-head comparators (Jauhar 2014)”. In fact, we reported that CBT was not significantly better than other psychological interventions in reducing total symptoms, positive symptoms or negative symptoms. Cochrane had likewise stated in their review (Jones et al, 2012) that “Trial-based evidence suggests no clear and convincing advantage for cognitive behavioural therapy over other - and sometime much less sophisticated - therapies for people with schizophrenia” (p.2).
In summary, in their references to our meta-analysis,Turkington & Lebert misrepresent our findings. At best, reasonable conclusions from our meta-analytic evidence would be that: a) CBT has small effect on total symptoms of schizophrenia and that these became smaller still in blinded studies; b) that CBT has a small effect on positive symptoms but this becomes non-significant in blind trials; and c) in the case of negative symptoms, both blinded and non-blinded trials point to small and nonsignificant effect sizes.
References
Jauhar, S., McKenna, P. J., Radua, J., Fung, E., Salvador, R., & Laws, K. R. (2014). Cognitive- behavioural therapy for the symptoms of schizophrenia: systematic review and meta-analysis with examination of potential bias. The British Journal of Psychiatry, 204(1), 20-29.
Jones C, Cormac I, Silveira da Mota Neto J, Campbell C. Cognitive behaviour therapy for schizophrenia. Cochrane Database Syst Rev 2004; 4: CD000524.
Lynch, D., Laws, K. R., & McKenna, P. J. (2010). Cognitive behavioural therapy for major psychiatric disorder: does it really work? A meta-analytical review of well-controlled trials. Psychological medicine, 40(01), 9-24.
National Institute of Health and Clinical Excellence. Schizophrenia: Core Interventions in the Treatment and Management of Schizophrenia in Adults in Primary and Secondary Care (Update). NICE, 2009.
Newton‐Howes, G., & Wood, R. (2013). Cognitive behavioural therapy and the psychopathology of schizophrenia: Systematic review and meta‐analysis. Psychology and Psychotherapy: Theory, Research and Practice, 86(2), 127-138.
Sarin, F., Wallin, L., & Widerlöv, B. (2011). Cognitive behavior therapy for schizophrenia: a meta-analytical review of randomized controlled trials. Nordic journal of psychiatry, 65(3), 162-174.
Turner, D. T., van der Gaag, M., Karyotaki, E., & Cuijpers, P. (2014). Psychological interventions for psychosis: a meta-analysis of comparative outcome studies. American Journal of Psychiatry, 171(5), 523-538.
Velthorst, E., Koeter, M., van der Gaag, M., Nieman, D. H., Fett, A. K., Smit, F., … & de Haan, L. (2015). Adapted cognitive–behavioural therapy required for targeting negative symptoms in schizophrenia: meta-analysis and meta-regression. Psychological medicine, 45(03), 453-465.
Wykes, T., Huddy, V., Cellard, C., McGurk, S. R., & Czobor, P. (2011). A meta-analysis of cognitive remediation for schizophrenia: methodology and effect sizes. American Journal of Psychiatry, 168(5), 472-485.