
ICATA Journal
Measuring the Effectiveness of Cognitive Analytic Therapy: An Evaluation of Using Psychological Outcome Measures in a Personality Disorder Service
Cite as: Dawson, M., Rowell, K., Johnstone, L., Pampoulov, P., & Rushbrook, S. (2023). Measuring the Effectiveness of Cognitive Analytic Therapy: An Evaluation of Using Psychological Outcome Measures in a Personality Disorder Service. International Journal of Cognitive Analytic Therapy and Relational Mental Health, 5(1), 57-69. https://www.internationalcat.org/volume-5-4
Int. Journal of CAT & RMH Vol. 5, 2023 / ISSN2059-9919
Abstract
The present study aimed to evaluate the impact of Cognitive Analytic Therapy (CAT) on a range of psychological outcomes for adults with personality disorder. CAT is a time- limited psychotherapy with a focus on relational patterns and a collaborative alliance between the client and therapist. The data was analysed from a non-randomised sample of twenty-one clients open to a tertiary Personality Disorder Service in a mental health trust in Dorset. These clients were seen for CAT between November 2017 and November 2019. Outcome measures were collected at three time points; prior to starting therapy, at the mid- point of therapy and at the end of therapy. Data from the following outcome measures was collated; The Structured Clinical Interview for DSM-5 (SCID-5), The Post-Traumatic Stress Disorder Checklist for DSM-5 (PCL-5), The Dissociative Experiences Scale (DES), The Inventory of Interpersonal Problems (IIP-32), Clinical Outcomes in Routine Evaluation (CORE-34), The Acceptance and Action Questionnaire (AAQ-2), The Brief Over control Scale (BOS) and The Symptom Checklist (SCL-90). Paired samples t-tests found that there were significant effects for all outcome measures, suggesting that CAT can be effective in reducing measures of psychological distress and improving psychological wellbeing in clients with personality disorder. Clinical implications and recommendations for future research are discussed.
Key words: Cognitive Analytic Therapy, Outcome, Measurement, Evaluation
Introduction
CAT is a time-limited psychotherapy that integrates cognitive and analytic models. It takes a relational approach to therapeutic change, with the collaboration between the therapist and the client playing a central role. CAT derives from Object Relations Theory (Ogden, 1983) and Kelly’s Personal Construct Theory (Kelly, 1955) which suggest that early reciprocal interactions with significant others influence an individual’s representation of themselves, the world and others. These templates are internalised as reciprocal roles. The objectives of CAT are for the client to identify their target problems and the target problem procedures that keep them stuck in the same patterns of behaviour. Another objective of CAT is to look at the underlying causes of the problems in terms of their early experiences, and to discover that choices and doing things differently (‘exits’) are possible through recognising their target procedures, reformulating and revising these with different exits. Traps, snags and dilemmas (‘target problem procedures’) are problematic patterns of social interaction and restricted or damaging self-care (Ryle & Kerr, 2002). CAT targets intrapsychic and interpersonal problems by implementing a phased approach to therapeutic change of reformulation, recognition and revision. In the middle stages of therapy, tools are matched to demands of the phases of therapy (Ryle & Kerr, 2002). These can include the psychotherapy file (Ryle & Kerr, 2002), the six-part story (Dent-Brown, 2001) and compassion-focused ideas.
Emerging evidence suggests that CAT can be utilised to support individuals with a range of physical and emotional conditions. Baronian and Leggett (2020) explored the effectiveness of 8-session CAT for clients with persistent pain. There were large pre–post effects on self-efficacy (d = 1.13) and wellbeing (d =1.50). Reliable change and clinical significance analyses found clinically meaningful results. Many of the clients (67.9%) showed reliable improvement following the CAT intervention. Evans et al. (2017) found that clients with bipolar disorder reflected that the recognition of patterns of mood variability was the most common helpful event during CAT. However, there were no major differences between psychometric outcomes when compared to the treatment as usual (TAU) group. Taylor et al. (2019) reported that clients with non-affective psychosis demonstrated improvement in perceived recovery and personality integration but that there was limited evidence of change in psychotic symptoms. A recent systematic review by Hallam et al. (2020) explored the acceptability, effectiveness, and durability of CAT. Twenty five studies provided pre and post CAT treatment outcomes and were aggregated across three outcome comparisons of depression, functioning, and interpersonal problems. CAT produced large pre-post reductions in depression symptoms (ES = 1.05, 95% CI 0.80–1.29, N = 586), large pre–post improvements in global functioning (ES = 0.86; 95% CI 0.71–1.01, N = 628), and moderate to large improvements in interpersonal problems (ES = 0.74, 95% CI 0.51–0.97, N = 460). At follow-up, all these effects were maintained or improved upon. These findings show positive early indicators of CAT utility for a range of conditions, yet research remains in its infancy.
Some studies have explored the utility of CAT when working with clients with personality disorder. A bibliometric review of CAT found that most of the research has focused on the use of CAT with clients with emotionally unstable personality disorder (35%) (Gimeno & Chiclana, 2016). A systematic review by Calvert and Kellett (2014) explored the effectiveness of CAT and reported that CAT was central to the treatment of personality disorder in 44% of the studies. A randomised controlled trial (RCT) by Clarke, Thomas and James (2013) followed up on the utility of CAT specifically for clients with a personality disorder. The RCT was conducted within the service from which the current dataset was taken and was a robust, high quality study exploring the effectiveness of CAT compared with a TAU condition. CAT participants showed significant improvements in interpersonal functioning and significant reductions in symptomatic distress, in comparison with TAU participants. At post- therapy, CAT participants no longer met symptomatic criteria for personal- ity disorder (33%), whereas all TAU participants remained symptomatic. In another study, CAT and manualised good clinical care (GCC) were compared in an RCT of 86 out-patients, aged 15–18 years, who met two to nine of the DSM–IV criteria for borderline personality disorder. There was no significant difference between the outcomes of the treatment groups at 24 months, but results suggested that patients allocated to CAT improved more rapidly. The current literature suggest that CAT is a promising therapy for complex presentations, but that further research contributions are required to be able to generalise these outcomes more widely.
This paper aimed to examine the psychometric outcomes following a CAT intervention in a psychological therapies service. It is hoped that the findings will contribute to the emerging evidence base for using this type of psychotherapy for clients with longstanding mental health difficulties.
Method
The service
The Intensive Psychological Therapies Service (IPTS) in Poole offers psychological therapies including CAT, Dialectical Behavioural Therapy (DBT), Radically-Open Dialectical Behavioural Therapy (RO-DBT) and Eye Movement Desensitisation and Reprocessing (EMDR). CAT has been offered at IPTS since the mid 1990s. The use of CAT has increased in this time as the number of accredited and trainee CAT therapists has grown.
CAT Assessment
Clients were referred to IPTS via their community mental health team (CMHT) and were invited to attend an assessment by a member of the IPTS multi-disciplinary team who may or may not have been trained in CAT. The assessment included the completion of psychometric tests and engaging in a clinical interview. The Structured Clinical Interview for DSM-5 (SCID-5) is a semi-structured interview guide administered by a clinician or trained mental health professional. The interview can be used to help professionals make DSM-5 diagnoses. Clients completed the SCID-5 again after CAT. The assessment also gathers information about the client’s history, risk issues, current functioning and goals for therapy. The assessments were discussed in a Clinical Decisions Meeting or CDM (with two Clinical Psychologists present) before agreeing to refer the clients for CAT. The clinician who carried out the assessment was from a Mental Health Nursing or an Occupational Therapy background and was trained in at least one core therapy, not necessarily CAT. The two Clinical Psychologists present in the CDM were trained in a number of core therapies, including CAT. The decision about the client’s suitability for CAT is based on the client’s identified goals being relational and them having an ability to manage emotions, whilst exploring their past and current problems.
Sample and Data Collection
21 clients were assessed as suitable for CAT and completed pre-, mid- and post-therapy outcome measures. When people were referred from a CMHT, they would be assessed as suitable for CAT rather than for the alternative therapies using the following criteria:
-
Are they able to relate their past experiences to their current problems in relating to themselves and others?
-
Are they able to manage emotional dysregulation without significant harm to self or others?
Clients started and ended therapy between November 2017 and November 2019 and there were no therapy dropouts from any of the 21 clients. The last CAT client was discharged after their follow-up session in February 2020. 14 clients had 24 sessions of therapy, four had 20 sessions and three had 16 sessions. In comparison, 20 clients started DBT, 38 clients started RO-DBT, and 2 clients started EMDR during the same time period.
Therapeutic Content
Therapy followed the basic 16 to 24 session CAT model, involving an initial focus on reformulation, identifying target problem procedures and underlying reciprocal roles. After the reformulation phase of CAT, the therapist would take the case for supervision and discussion in the service’s clinical decisions meeting. At this stage, the total number of CAT sessions to be offered was agreed, based on their engagement and the complexity of their presentation. Further sessions involved development of a reformulation and map (previously called a Sequential Diagrammatic Reformulation or SDR), a focus on developing recognition of underlying patterns and procedures, exploration of potential exits, an emphasis on dynamics within the therapeutic relationship and work on the ending of therapy. Each client was offered a follow-up appointment with their therapist three months after the completion of CAT.
Therapist Accreditation
All therapists had completed an Association for Cognitive Analytic Therapy (ACAT) accredited practitioner training course or were in the process of training and were supervised by an ACAT accredited practitioner. Each case was discussed in weekly supervision with an ACAT accredited supervisor.
Outcome measures
Clients consented to complete a series of outcome measures to assess psychological symptoms, wellbeing and functioning, intrapersonal and interpersonal difficulties. Measures were taken at the start, middle and end of CAT. A list of all outcome measures can be found in Table 1 below.
Table 1: List and descriptions of all the outcome measures used in this service evaluation

Ethical Consideration
This evaluation was an attempt to explore whether CAT was effective in improving psychological wellbeing and reducing emotional distress. Patient consent to complete outcome measures was given at the start of therapy. The need to submit to a formal ethics committee was considered but following advice from the Trust Clinical Audit Team; this was not required, due to patient consent being given, meeting the NHS trust criteria for client confidentiality and anonymisation for research purposes.
Results
Paired sample t-tests were run using SPSS (Version 26) for the outcomes measures that clients completed pre- and post-CAT. The analysis accounted for missing values across some of the outcome measures – please see Graphs 1-7. Although a strong effort was made to collect psychometric scores at both pre- and post-therapy, some data from participants were missing due to unreturned or incomplete psychometric tests. The authors recoded the missing variables in SPSS before running the analyses. This provided accurate results, only accounting for variables which were available to the authors. Most analyses completed accounted for the whole sample (n=21). However, the number of participants that were included in each analysis can be found in the descriptive statistics (see Table 2 below).
Table 2: Paired-sample statistics for each of the outcome measures. N=number; SD=Standard Deviation; SEM=Standard Error Mean

Results showed that there was a statistically significant decrease in clinical symptoms between pre- and post-therapy and this was observed across all the outcome measures described in this paper. For further details on scores, please refer to Graphs 1-7.
Graph 1: Average scores for pre- and post-CAT on the Clinical Outcomes in Routine Evaluation (CORE-34) outcome measure.

Graph 2: Average scores at pre- and post-CAT on the Brief Overcontrol Scale (BOS) outcome measure. (author’s note: these scores represent an improvement).

Graph 3: Average scores at pre- and post-CAT on the PTSD Checklist for the DSM-5 (PCL-5) outcome measure.

Graph 4: Average scores at pre- and post-CAT on the Dissociative Experiences Scale (DES) outcome measure.

Graph 5: Average scores at pre- and post-CAT on the Inventory of Interpersonal Problems (IIP-32) outcome measure.

Graph 6: Average scores at pre- and post-CAT on the Acceptance and Action Questionnaire (AAQ-II) outcome measure.

Graph 7: Average scores at pre- and post-CAT on the Symptom Checklist (SCL-90) outcome measure.

Discussion
Summary
The statistically significant changes in pre- and post-therapy scores across all psychometric measures indicated that there was a substantial improvement in clinical symptoms throughout the therapy period of CAT. More specifically, these changes demonstrated a decrease in pathological symptomatology as well as an increase in psychological flexibility. This demonstrates that CAT could have a positive impact on multiple aspects of psychological wellbeing in clients with complex presentations, like personality disorder. The results suggest that CAT might have played a role in facilitating improvements in interpersonal difficulties, as measured by the IIP-32, and this may have been anticipated due to the relational focus of CAT. However, it is encouraging to reflect that CAT appeared to give rise to benefits across a range of other psychological domains, including dissociation (as shown by the DES), psychological flexibly and experiential avoidance (as shown in the AAQ), and over controlled and obsessional characteristics (as shown in the BOS). These are promising outcomes that support further research into this area.
Strengths and limitations
There are some limitations to this service evaluation. The small sample size reduces statistical power to determine an effect size and lowers reproducibility. Additionally, the lack of control group and uneven group sizes makes it difficult to generalise the results. There was variation in the number of CAT sessions that the participants attended (e.g. 16, 20 or 24 sessions) but analysis to compare the effect of treatment length was not conducted. Subsequently, any between-group differences cannot be inferred.
It is important to note that even though a statistically significant reduction in clinical symptoms was observed, there would need to be additional studies to compare CAT to other therapies and reproduction of the current results, to definitively indicate that it was CAT that led to symptom reduction.
The service evaluation may have benefited from describing demographic characteristics, such as ethnicity and age, to help recognise patterns of service access and qualitative data would allow assessment of whether the change that was measured was consistent with the lived experience of the therapy.
Clinical implications
These findings suggest that CAT can be utilised effectively in therapeutic work with clients with personality disorder. It is hoped that this will add to the emerging evidence for offering CAT and help to develop knowledge and awareness about this form of psychotherapy. The results also suggest that CAT can be effective in supporting improvements across a range of psychological constructs, not just relational factors, as may be anticipated from a relational psychotherapy.
The authors who were delivering CAT as a therapy hoped that in starting this piece of work they would find that CAT was beneficial to clients with significant interpersonal difficulties. These outcomes further strengthened their belief that CAT is an effective time-limited intervention for people with longstanding mental health difficulties.
Recommendations for Future Service Evaluation/Research
The results of the present study invite recommendations for future service evaluation/research:
-
To continue to gather outcome data to add to the evidence base and to replicate the service evaluation with a larger client sample.
-
To look at clinical and significant change between pre- and mid- therapy scores on the outcome measures.
-
To gather qualitative information from clients to allow for a richer and deeper understanding of the client’s experience of change during CAT.
-
To gather service demographics to explore the characteristics of the clients accessing IPTS and to compare against local and national data. This will support social inclusion and facilitate service development in accordance with the needs of client population.
-
To ensure that the client sample is representative of the wider population and that more detailed demographic data were provided (e.g. gender, age, ethnicity, etc.)
-
Using a randomised controlled trial methodology to compare CAT to a control group to demonstrate the causal link between CAT and and decrease in clinical symptoms.
References
aronian, R., & Leggett, S. J. (2020). Brief cognitive analytic therapy for adults with chronic pain: a preliminary evaluation of treatment outcome. British Journal of Pain, 14(1), 57-67.
https://doi.org/10.1177/2049463719858119
Calvert, R. & Kellett, S. (2014). Cognitive analytic therapy: a review of the outcome evidence base for treatment. Psychology and Psychotherapy, 87(3): 253-277.
https://doi.org/10.1111/papt.12020
Chanen, A.M., Jackson, H.J., McCutcheon, L.K., et al. (2008). Early intervention for adolescents with borderline personality disorder using cognitive analytic therapy: randomised controlled trial. The British Journal of Psychiatry, 193, 477-484.
https://doi.org/10.1192/bjp.bp.107.048934
Clarke, S., Thomas, P. & James, K. (2013). Cognitive analytic therapy for personality disorder: randomised controlled trial. The British Journal of Psychiatry, 202, 129-134.
https://doi.org/10.1192/bjp.bp.112.108670
Dent-Brown, K. (2001). Story as therapeutic tool: The Six-Part Story Method. Context, 55, 22-23.
Evans, M., Kellett, S., Heyland, S., Hall, J., & Majid, S. (2017). Cognitive analytic therapy for bipolar disorder: A pilot randomised controlled trial. Clinical Psychology & Psychotherapy, 24, 22-35.
https://doi.org/10.1002/cpp.2065
Gimeno, E. & Chiclana, C. (2016). Cognitive analytic therapy: A bibliometric review. European Psychiatry, 33,
https://doi.org/10.1016/j.eurpsy.2016.01.581
Hallam, C., Simmonds-Buckley, M., Kellett. S., Greenhill. B. & Jones, A. (2020). The acceptability, effectiveness, and durability of cognitive analytic therapy: Systematic review and meta analysis. Psychology and Psychotherapy.
https://doi.org/10.1111/papt.12286
Kelly, G. A. (1955). The psychology of personal constructs. Vol. 1. A theory of personality. Vol. 2. Clinical diagnosis and psychotherapy. W. W. Norton.
Ogden, T. (1983). The concept of internal object relations. International Journal of Psycho-Analysis, 64, 227-241.
Ryle, A., & Kerr, I. B. (2002). Introducing cognitive analytic therapy: Principles and practice. Oxford, UK: Wiley-Blackwell.
https://doi.org/10.1002/9780470713587
Taylor, P., Perry, A., Hutton, P., Tan, R., Fisher, N., Focone, C., Griffiths, D., & Seddon, C. (2018). Cognitive analytic therapy for psychosis: A case series. Psychology and Psychotherapy: Theory, Research and Practice,1-20.
https://doi.org/10.1111/papt.12183