This article was published in Counselling and Psychotherapy Research (CPR) 2023;00:1–9
(A free PDF can be downloaded at the link above)
Daryl Mahon, Outcomes Matter, Wicklow, Ireland
Takuya Minami, University of Massachusetts-Boston, Boston, Massachusetts, USA
(G.S.) Jeb Brown, The Center for Clinical Informatics/ ACORN, Salt Lake City, Utah, USA
Abstract
This article investigates the percentage of variance in psychotherapy outcomes explained by the case-mix variables for individual cases, by the therapist (therapist effects), and what additional variance is explained by the clinic with which the therapist is affiliated. While there has been substantial recent research regarding therapist effects, very little has been published regarding clinic-level effects after controlling for therapist effects. The study utilised the largest sample reported to date, using data from 28 clinics with a minimum of 2,000 cases in a clinical range of severity of symptoms with pre-post change scores on an outcome questionnaire. Only cases treated by a therapist with at least 30 cases were included. These selection criteria resulted in a case count of 156,258 clients treated by 874 therapists located at 28 clinics. After controlling for differences in case mix using diagnosis and intake score (severity of symptoms), the resulting analyses indicated that 5.21% of the variance in treatment outcome was explained by the therapist, while another 1.13% of the variance was associated with the clinic. Findings are discussed with implications for practice and policy.
KEYWORDS
clinic effects, expertise, feedback-informed treatment, psychotherapy outcome variables, therapist effects
REFERENCES
Anderson, T., Ogles, B. M., Patterson, C. L., Lambert, M. J., & Vermeersch, D. A. (2009). Therapist effects: Facilitative interpersonal skills as a predictor of therapist success. Journal of Clinical Psychology, 65, 755–768. https://doi.org/10.1002/jclp.20583
Anderson, T., & Perlman, M. R. (2022). Therapist and client facilitative interpersonal skills in psychotherapy. In J. N. Fuertes (Ed.), The other side of psychotherapy: Understanding clients’ experiences and contributions in treatment (pp. 99–124). American Psychological Association. https://doi.org/10.1037/0000303-005
Baldwin, S. A., & Imel, Z. E. (2013). Therapist effects: Findings and methods. In M. J. Lambert (Ed.), Bergin and Garfield's handbook of psychotherapy and behavior change (6th ed., pp. 258–297). Wiley.
Baldwin, S. A., Wampold, B. E., & Imel, Z. E. (2007). Untangling the alliance-outcome correlation: Exploring the relative importance of therapist and patient variability in the alliance. Journal of Consulting and Clinical Psychology, 75(6), 842–852. https://doi. org/10.1037/0022-006X.75.6.842
Becker, B. J. (1988). Synthesizing standardized mean-change measures. The British Journal of Mathematical and Statistical Psychology, 41, 257–278. https://doi.org/10.1111/j.2044-8317.1988. tb00901.x
Bohart, A. C. (2013). Greaves Wade a. the client in psychotherapy. In M. J. Lambert (Ed.), Bergin and Garfield's handbook of psychotherapy and behavior change (6th ed.). Wiley.
Bovendeerd, B., De Jong, K., De Groot, E., Moerbeek, M., & De Keijser, J. (2022). Enhancing the effect of psychotherapy through systematic client feedback in outpatient mental healthcare: A cluster randomized trial. Psychotherapy Research, 32(6), 710–722. https://doi. org/10.1080/10503307.2021.2015637
Brown, G. S., Cazauvieilh, C., & Simon, A. (2021). Therapist engagement in feedback informed care and patient outcomes. [Web article]. Retrieved from http://www.societyforpsychotherapy. org/therapist-engagement-in-feedback-informed-care-and-patie nt-outcomes
Brown, G. S., Simon, A. E., Cameron, J., & Minami, T. (2015). A collaborative outcome resource network (ACORN): Tools for increasing the value of psychotherapy. Psychotherapy, 52(4), 412–421. https://doi. org/10.1037/pst0000033
Brown, G. S., Simon, A., & Foster, L. (2021). Clinical impact of school-based interventions. [Web article]. Retrieved from http://www.societyfor psychotherapy.org/clinical-impact-of-school-based-interventions
Brown, G. S., Simon, A. E., & Minami, T. (2015). Are you any good…as a therapist? [Web article]. Retrieved from http://www.societyfor psychotherapy.org/are-you-any-good-as-a-therapist
Brydges, C. R. (2019). Effect size guidelines, sample size calculations, and statistical power in gerontology. Innovation in Aging, 3(4), igz036. https://doi.org/10.1093/geroni/igz036
Chow, D. L., Miller, S. D., Seidel, J. A., Kane, R. T., Thornton, J. A., & Andrews, W. P. (2015). The role of deliberate practice in the development of highly effective psychotherapists. Psychotherapy, 52(3), 337–345. https://doi.org/10.1037/pst0000015
de Jong, K., Conijn, J. M., Gallagher, R., Reshetnikova, A. S., Heij, M., & Lutz, M. C. (2021). Using progress feedback to improve outcomes and reduce drop-out, treatment duration, and deterioration: A multilevel meta-analysis. Clinical Psychology Review, 85, 102002. https://doi.org/10.1016/j.cpr.2021.102002
Deisenhofer, A. K., Rubel, J. A., Bennemann, B., Aderka, I. M., & Lutz, W. (2022). Are some therapists better at facilitating and consolidating sudden gains than others? Psychotherapy Research: Journal of the Society for Psychotherapy Research, 32(3), 343–357. https://doi. org/10.1080/10503307.2021.1921302
Del Re, A. C., Flückiger, C., Horvath, A. O., Symonds, D., & Wampold, B. E. (2012). Therapist effects in the therapeutic alliance-outcome relationship: A restricted-maximum likelihood meta-analysis. Clinical Psychology Review, 32(7), 642–649. https://doi.org/10.1016/j. cpr.2012.07.002
Delgadillo, J., Asaria, M., Ali, S., & Gilbody, S. (2016). On poverty, politics and psychology: The socioeconomic gradient of mental healthcare utilisation and outcomes. The British Journal of Psychiatry: the Journal of Mental Science, 209(5), 429–430. https://doi.org/10.1192/bjp. bp.115.171017
Duncan, B. L. (2014). The heart and soul of change. In B. L. Duncan (Ed.), On becoming a better therapist: Evidence-based practice one client at a time (pp. 147–173). American Psychological Association. https:// doi.org/10.1037/14392-006
Eubanks, C. F., Muran, J. C., & Safran, J. D. (2018). Alliance rupture repair: A meta-analysis. Psychotherapy (Chicago, Ill.), 55(4), 508–519. https://doi.org/10.1037/pst0000
Finegan, M., Firth, N., & Delgadillo, J. (2020). Adverse impact of neighbourhood socioeconomic deprivation on psychological treatment outcomes: The role of area-level income and crime. Psychotherapy Research: Journal of the Society for Psychotherapy Research, 30(4), 546–554. https://doi.org/10.1080/10503307.2019.1649500
Firth, N., Saxon, D., Stiles, W. B., & Barkham, M. (2019). Therapist and clinic effects in psychotherapy: A three-level model of outcome variability. Journal of Consulting and Clinical Psychology, 87(4), 345– 356. https://doi.org/10.1037/ccp0000388
Firth, N., Saxon, D., Stiles, W. B., & Barkham, M. (2020). Therapist effects vary significantly across psychological treatment care sectors. Clinical Psychology & Psychotherapy, 27, 770–778. https://doi. org/10.1002/cpp.2461
Flückiger, C., Del Re, A. C., Wampold, B. E., & Horvath, A. O. (2018). The alliance in adult psychotherapy: A meta-analytic synthesis. Psychotherapy (Chicago, Ill.), 55(4), 316–340. https://doi. org/10.1037/pst0000172
Goldberg, S. B., Babins-Wagner, R., Rousmaniere, T., Berzins, S., Hoyt, W. T., Whipple, J. L., Miller, S. D., & Wampold, B. E. (2016). Creating a climate for therapist improvement: A case study of an agency focused on outcomes and deliberate practice. Psychotherapy (Chicago, Ill.), 53(3), 367–375. https://doi.org/10.1037/pst0000060
Goldberg, S. B., Rousmaniere, T., Miller, S. D., Whipple, J., Nielsen, S. L., Hoyt, W. T., & Wampold, B. E. (2016). Do psychotherapists improve with time and experience? A longitudinal analysis of outcomes in a clinical setting. Journal of Counseling Psychology, 63(1), 1–11. https:// doi.org/10.1037/cou0000131
Hansen, N. B., Lambert, M. J., & Forman, E. M. (2002). The psychotherapy dose-response effect and its implications for treatment delivery services. Clinical Psychology: Science and Practice, 9, 329–343. https://doi.org/10.1093/clipsy.9.3.329
Heinonen, E., & Nissen-Lie, A. H. (2020). The professional and personal characteristics of effective psychotherapists: A systematic review. Psychotherapy Research, 30(4), 417–432. https://doi. org/10.1080/10503307.2019.1620366
Johns, R. G., Barkham, M., Kellett, S., & Saxon, D. (2019). A systematic review of therapist effects: A critical narrative update and refinement to review. Clinical Psychology Review, 67, 78–93. https://doi. org/10.1016/j.cpr.2018.08.004
Kraus, D. R., Bentley, J. H., Alexander, P. C., Boswell, J. F., Constantino, M. J., Baxter, E. E., & Castonguay, L. G. (2016). Predicting therapist effectiveness from their own practice-based evidence. Journal of Consulting and Clinical Psychology, 84(6), 473–483. https://doi. org/10.1037/ccp0000083
Kraus, D. R., Castonguay, L., Boswell, J. F., Nordberg, S. S., & Hayes, J. A. (2011). Therapist effectiveness: Implications for accountability and patient care. Psychotherapy Research, 21(3), 267–276.
Lambert, M. J., & Norcross, J. C. (1992). Handbook of psychotherapy integration. Psychotherapy outcome research: Implications for integrative and eclectic therapists içinde. 94–129.
Lambert, M. J., Whipple, J. L., & Kleinstäuber, M. (2018). Collecting and delivering progress feedback: A meta-analysis of routine outcome monitoring. Psychotherapy, 55(4), 520–537. https://doi. org/10.1037/pst0000167
Lambert, W., Minami, T., Hamilton, E., Mc Culloch, J., Peters, J., Selway, M., … Brown, J. (2009). Outcomes measurement 2.0: Emerging technologies for managing treatment outcomes in behavioral healthcare. Retrieved from: https://acorncollaboration.org/outcomes-20
Lutz, W., & Barkham, M. (2015). In R. L. Cautin & S. O. Lilienfeld (Eds.), Therapist effects. In the encyclopedia of clinical psychology. WielyBlackwell. https://doi.org/10.1002/9781118625392.wbecp109
Mahon, D. (2022). A scoping review of deliberate practice in the acquisition of therapeutic skills and practices. Counselling and Psychotherapy Research, 00, 1–17. https://doi.org/10.1002/capr.12601
Nissen-Lie, H. A., Rønnestad, M. H., Høglend, P. A., Havik, O. E., Solbakken, O. A., Stiles, T. C., & Monsen, J. T. (2017). Love Yourself as a Person, Doubt Yourself as a Therapist? Clinical psychology & psychotherapy, 24(1), 48–60. https://doi.org/10.1002/cpp.1977
Norcross, J. C., & Lambert, M. J. (2018). Psychotherapy relationships that work III. Psychotherapy, 55(4), 303–315. https://doi.org/10.1037/ pst0000193
Okiishi, J., Lambert, M. J., Nielsen, S. L., & Ogles, B. M. (2003). Waiting for supershrink: An empirical analysis of therapist effects. Clinical Psychology & Psychotherapy, 10, 361–373. https://doi.org/10.1002/ cpp.383
Pybis, J., Saxon, D., Hill, A., & Barkham, M. (2017). The comparative effectiveness and efficiency of cognitive behaviour therapy and generic counselling in the treatment of depression: Evidence from the 2nd U.K. National Audit of Psychological Therapies. BMC Psychiatry, 17(1), 215. https://doi.org/10.1186/s1288 8-017-1370-7
Rodenbush, S. W., & Bryk, S. A. (2002). Hierarchial Linhera models-applications and data analysis methods. University of Chicago.
Saxon, D., & Barkham, M. (2012). Patterns of therapist variability: Therapist effects and the contribution of patient severity and risk. Journal of Consulting and Clinical Psychology, 80(4), 535–546. https://doi.org/10.1037/a0028898
Singer, J. D. (1998). Using SAS PROC MIXED to fit multilevel models, hierarchical models, and individual growth models. Journal of Educational and Behavioral Statistics, 23(4), 323–355. https://doi. org/10.3102/10769986023004323
Stiles, W. B., Barkham, M., & Wheeler, S. (2015). Duration of psychological therapy: Relation to recovery and improvement rates in U.K. routine practice. [corrected]. The British Journal of Psychiatry: the Journal of Mental Science, 207(2), 115–122. https://doi.org/10.1192/ bjp.bp.114.145565
Wampold, B. E. (2015). How important are the common factors in psychotherapy? An update. World Psychiatry: Official Journal of the World Psychiatric Association (WPA), 14(3), 270–277. https://doi. org/10.1002/wps.20238
Wampold, B. E., & Brown, G. S. (J.). (2005). Estimating variability in outcomes attributable to therapists: A naturalistic study of outcomes in managed care. Journal of Consulting and Clinical Psychology, 73(5), 914–923. https://doi.org/10.1037/0022-006X.73.5.914
Wampold, B. E., & Imel, Z. E. (2015). The great psychotherapy debate (2nd ed.). Routledge/Taylor & Francis Group.
Wampold, B. E., & Owen, J. (2021). Therapist effects: history, methods, magnitude, and characteristics of effective therapists. In M. Barkham, W. Lutz, & L. G. Castonguay (Eds.), Bergin and Garfield's handbook of psychotherapy and behavior change (pp. 301–330). Wiley.
Ziem, M., & Hoyer, J. (2020). Modest, yet progressive: Effective therapists tend to rate therapeutic change less positively than their patients. Psychotherapy Research, 30(4), 433–446. https://doi. org/10.1080/10503307.2019.1631502