Outcome Measures Suck

patients! I know admitting this may be shocking, almost sacrilegious to some evidence-based clinicians, and it may leave me open to some criticism. But before you do, hear me out as to why I think outcome measures don't help most patients or therapists that use them.

Outcome measures are simply something that quantifies and records a dependent variable, such as pain or function, that is being affected by an independent variable such as treatment and interventions. There are literally hundreds of outcome measures available in musculoskeletal medicine and they come in all different variations and varieties

Some outcome measures are performance-based such as the 30 second Sit to Stand Test. Some are observer reported such as the Functional Movement Screen. But many are patient self-reported outcome measures, often abbreviated to PROMs that are either area-specific such as the Shoulder Pain And Disability Index or generic such as the EQ-5D.

There is no doubt that outcome measures have helped healthcare move away from eminence based practice which relied upon biased and unreliable observations and personal opinions of what clinicians thought did or did not work, towards more reliable and unbiased ways of recording what does or doesn't work. However, I think many outcome measures in clinical practice today are not used well and offer very little for patients and clinicians.

The Early Days

From the start of my career as a physio, I was taught to use all varieties and variations of outcome measures with my patients to 'prove' the effectiveness of my interventions and justify what I was doing. And that's exactly what I did for many many years, but these days I hardly use any outcome measures.

You may be thinking, how do I know if what I am doing is effective or working. You may also be thinking that I'm a huge hypocrite for always going on about the need for more evidence in physiotherapy, yet here I am saying I don't collect evidence of my interventions with my patients.

Well, your right this is contradictory behaviour, and if I am being honest this is not the first time I have been a big old hot mess of contradiction. I do think the physio profession desperately needs more evidence to demonstrate what it does or doesn't do is effective, I just don't think my day to day collection of outcome measures with my patients helps achieve this in any way shape or form.

I don't think most outcome measures are useful for most clinicians in normal everyday clinical encounters for a number of reasons. First, a lot of the data collected with many outcome measures is rarely collated, analysed, or used for anything useful. Secondly, most outcome measures create frustration and unnecessary barriers between patients and clinicians. And lastly, outcome measures don't actually measure what many think they are measuring.

Wasted Data

Unless involved in research or a clinical trial most data obtained on outcome measures is not used for anything meaningful. Most data collected in outcome measures is collected by clinicians on orders of their managers, insurance companies, or other professional organisations often for the sake of paperwork purposes, legal documentation, or just to be seen to be doing the right thing.

Outcome measure scores mean very little to patients and most outcome scores are not used to check if patients are actually getting better. Instead, a lot of outcome measure data is used to rate and assess if clinicians are effective rather than if patients are improving.

This is not what outcome measures are designed for. I have been appraised many times throughout my career by managers checking my patient's outcome scores. I have even been given specific outcome measure targets to achieve in some clinics I've worked at.

There's an old saying that goes "as soon as a measure becomes a target, it ceases to be a good measure". This is because when a measure becomes a target our biases and errors increase during the recording of the measurements as we attempt to obtain the target.

This affects both the reliability and accuracy of the outcome measure and gives us skewed perceptions and results of treatments and clinicians' effectiveness. There is also another saying that goes "what is mismeasured is mismanaged". Meaning that misusing outcome measures to rate clinicians rather than patients often leads to patients being mismanaged.

They Don't Do What You Think They Do

Many clinicians collect outcome measures in the mistaken belief that they prove the 'effectiveness' of their treatments and interventions. They don't. Outcome measures measure patients' outcomes not the effects of treatments or interventions given to them. That sounds bloody confusing so let me clarify it a little more.

Basically, there are many things that can affect a patient's outcome other than just the treatment they receive. Things like time, beliefs, expectations, reassurance, placebo, and many other factors which I have discussed before. Outcome measures therefore also 'measure' the effects of these things as well, not just the effects of treatments received.

A common misunderstanding made by many clinicians is that when an outcome measure score improves or worsens it must mean their treatment was the cause of this. It doesn't. Just because a patient improves, or deteriorates after treatment it doesn't mean it was because of the treatment, as patients can and do improve and deteriorate despite the treatment they receive. (ref)

Many also misinterpret patients' responses recorded in outcome measures. For example, the Pain Self Efficacy Questionnaire (PSEQ) is often used to assess patients' fear of movement when doing tasks when in pain. However, the Pain Self Efficacy Questionnaire only measures a patient's confidence to do the tasks described in the questionnaire, and this doesn't transfer well to other tasks.

Just because a patient scores high in self-efficacy to do household chores on the PSEQ when they have back pain, this doesn't mean they will have high levels of self-efficacy to do kettlebell swings or bird-dog exercises during their rehab. It's important to remember that outcome measures only measure what the outcome measure is measuring, they do not inform us about other things.

Another factor to consider is that patients may fill out outcome measures based on information they have to recall from the past which may be poor and inaccurate. They may also answer questions in a specific way due to concerns about the effects or impact their answers may have on future treatments or interactions with clinicians.

Unnecessary Barriers

Outcome measures also create a lot of unnecessary barriers to patient-clinician communication and therapeutic relationship building. I often see a lot of confusion, frustration, and apathy in patients as they struggle to fill out questionnaires and answer a long list of boring meaningless questions, or try to give a simple single word or numerical score on something that is complex and complicated.

For example, does the number 7 really explain and describe the severity, intensity or frequency of an individual's experience of back pain? Does 'sometimes' or 'occasionally' on a tick box really convey the frustration and anxiety someone has with their sleep being disturbed?

Some outcome measures can also take 15-20 minutes to complete, and often the questions mean very little to those filling them out. A classic example I come across is with the Oxford Shoulder Score (OSS) which I have to use a lot in my current role. There is a question in this about the level of difficulty combing or washing hair which always gets bald men rather annoyed and perplexed!

Outcome measures can not and do not capture the complexity or uniqueness of individuals' pain and disability well, yet many clinicians use these measures to identify patients' problems and disabilities instead of actually asking them what their actual problems and disabilities are.

Just because someone says they have a lot of difficulty washing their hair, or a lot of back pain when doing household chores on a questionnaire, this does not mean these are meaningful goals for them to achieve. They may be far more concerned about not being able to play with their kids, sleep through the night, or do their job far more than they are concerned about their doing their hair or household chores.

Not Completely Useless

However, although I've been very critical of outcome measures and think they are not used well and offer little for most, I don't want you to think I am completely dismissing them, or that I don't collect any information from my patients at all.

As I have said I think most outcome measures are often best suited for research purposes and clinical trials. However, there is one outcome measure that I use regularly with a lot of my patients which is validated, quick, and simple, but more importantly, doesn't create any unnecessary barriers or frustrations with my patients, and this is the Patient-Specific Functional Score (PSFS).

The PSFS involves asking the patient what are 1 to 3 things that are important and meaningful to them that they are having difficulty with currently because of their pain or issues now. This could be anything from brushing their hair to playing tiddlywinks, it doesn't matter, it's up to the patient to decide, not a bloody questionnaire.

They then rate how difficult it is for them to do these things on a simple scale from 0 to 10, with 0 being impossible, and 10 being no problem at all. These tasks are then reassessed at various time points throughout their treatment to see how progress is or is not going. I find this outcome measure to be one of the most simple, meaningful, and least obtrusive ones out there and one that I think should be used often in clinical practice.

Summary

So there you go that's my quick look at why I think many outcome measures suck in physio. To summarise...

* Don't use outcome measures to measure a clinician's effectiveness

* Don't think outcome measures only measure a treatment's effectiveness

* Don't ask bald men to fill out the Oxford Shoulder Score

* Do use the Patient-Specific Functional Score more

As always thanks for reading

Adam