Brant Oliver, PhD, on patient-reported outcomes in MS

brant_oliver_dartmouth_institute.jpg

Brant Oliver, PhD, MS, MPH, APRN-BC, is a health care improvement and implementation scientist, educator, and board-certified family and psychiatric nurse practitioner (FNP-BC, PMHNP-BC). Dr. Oliver's work focuses on applied health care improvement science research, with a focus on "3C" (complex, costly, and chronic) conditions including MS, IBD, CF, RA, and others; coproduction; learning health systems; and shared decision making. He is Associate Professor at the Geisel School of Medicine at Dartmouth and also has been in clinical practice since 2003, working primarily as a certified MS specialist (MSCN) and a MS neurobehavioral nurse practitioner.  

Which patient-reported outcomes (PROs) are commonly assessed in the care of patients with multiple sclerosis (MS)?

BRANT OLIVER, PhD: The current reality is that PROs are not being used routinely in MS clinical care in most settings. They are most commonly being used in research settings, either as part of clinical trials of new treatments, or in epidemiologic studies looking at the prevalence, incidence, and severity of certain symptoms or functional impairments in MS at the population level. A good example is the NARCOMS registry, which has been used to conduct  high-quality epidemiologic studies of MS-related symptoms using self-report questionnaires.

We are starting to see some use of MS PROs in select clinical settings. Oftentimes these are measures of highly prevalent comorbid conditions that have an impact on quality of life, treatment adherence (depression severity and anxiety), and self-efficacy, which is a measure of perceived coping ability related to self-management of a chronic illness. Also, measures of patient experience are being employed in hospital-based clinics, such as the CG-CAHPS satisfaction measures.

The PROMIS (Patient-Reported Outcomes Measurement Information System) battery is a promising set of PRO measures that are being utilized across a number of conditions and that contain some MS-specific measures, such as the Fatigue MS measure. Fatigue is the most common and disabling symptom in MS, so having a good measure of fatigue is of critical importance. Having a sense of a patient’s experience with fatigue over time can also be very helpful from a clinical perspective. The PROMIS measure performs much better than its predecessor, the Modified Fatigue Impact Scale (MFIS). Another strength of the PROMIS measures is that they tend to do well correlating with other conditions. In certain research and improvement settings, PROMIS can help us understand the burden a particular disease or population places on total health system resources. For example, is fatigue burden in MS similar in severity to that in another 3C condition, such as inflammatory bowel disease?

There are other PRO measures that have not been used in MS but which I think we will see used in the future. One of these is called Collaborate, which falls into the category of patient-reported experience measures (PREMs). Collaborate measures the degree of shared decision-making that occurs in a clinical encounter, which becomes especially important as the complexity of disease-modifying treatment decisions increases as more disease-modifying therapies are introduced, each with their own profile of risks and benefits. I anticipate that the ability of clinics to facilitate effective shared decision-making will be of increasing interest to clinicians, patients, and also payers in the future.

What are some of the potential benefits of using PRO measures in terms of clinician/patient communication and clinical decision-making?

BRANT OLIVER, PhD: I think the most important benefit of measuring PROs is that it can make care more patient-centered. The big push for PROs in the first place was to incorporate more of the patient’s voice into their own care. Standard clinical measures, such as magnetic resonance imaging (MRI) or relapse rates, can be very precise and provide a good clinical picture of how the disease is behaving. But if we rely solely on that kind of information, we can lose sight of the person who is experiencing the disease.

PROs, be they just regular PROs (instruments that gather qualitative and/or quantitative patient-reported information) or PROMs (validated and standardized PRO measures) or PREMs (standardized and validated patient-reported experience measures), all aim to improve the ability to get a better story of the person who's experiencing the illness and how that condition affects them. We can also gain insight on what's most important to them regarding how their illness is managed, and what their experience of care is.

Sometimes the PRO picture and the clinical picture don't necessarily align. I'll tell a story about one a person, who's given me permission to share the story. She is an established professional, and her PRO measures concerning her quality of life, support from her workplace environment, and her ability to function in society are very high. She's very high functioning according to the PRO picture.  However,  her clinical measures suggest that she is significantly impaired. Her MRI burden is quite high, and her disability due to the MS, her EDSS (Expanded Disability Status Scale), is also quite high. So there are 2 very different stories provided by 2 different sets of data. The inclusion of PROs data, in connection with clinical data, can provide a more holistic view of this person, who is functioning well despite significant disease burden.

As a clinician working with patients with MS, this gives me much more to work with in trying to help people, not only treating the disease, but also trying to see where the person is most in need of help. A person could be very well controlled on MRI in terms of relapse rate, but have fatigue severity that is through the roof. If I weren't paying closer attention to that with a validated scale that measures fatigue severity over time, like the PROMIS Fatigue MS, I may not have a sense of how well treatment is helping that person's fatigue longitudinally, beyond what I can glean from the history and clinical examination.

PROs can also be helpful in terms of conditions that are harder to quantify. Depression is a significant cause of disability and a significant factor contributing to poor treatment adherence in MS. Also, the suicide rate in persons with MS is much higher than that in the general population. But there is no blood test for depression severity, and oftentimes a patient’s report on exam can be incomplete or misleading. Accompanying the mental status exam and the clinical evaluation with a validated depression severity measure (such as the PHQ9 [Patient Health Questionnaire-9], CES-D [Center for Epidemiologic Studies Depression Scale], or others) can help to determine whether treatments are having an initial effect, and this can also help with self-monitoring and treatment adherence.  

That brings us to the second area where PROs show real promise in MS care: the learning health system (LHS) approach, which uses feed forward information to either predict the needs of people before they come to the office or optimize decision-making at the point of care (as discussed in our review article published in JCOM).¹

PROs can play a significant role in the LHS approach in 2 ways. For example, if I have a good sense before a visit that the person has stable symptoms but is having a lot of trouble coping with MS, that may change how I allocate resources and focus the time at the visit to target the primary concern of the person before the visit even occurs. Over time I'd be able to monitor if those coping scores increase in response to the work we’re doing with the person.

The second part of this approach is a feedback mechanism. These measures can be aggregated at the population level and fed back to clinical MS centers to help clinicians who are trying to improve overall population health outcomes of people with MS to see how well they're doing on things like fatigue management, depression management, coping ability, and pain control, which can't be tested for with biologic methods. So in addition to looking at the MRI scan trends across the population and relapse rate annualized for the population over time, I'd also be able to see fatigue severity and depression severity at a clinic population level and assess the change in these severity levels over time, which I could then use to inform efforts to change how my system (clinic) provides MS care.

This capability of PROs could be of significant interest to payers, especially those who are looking to optimize the value of care that is being provided, given the high cost of MS care. MS treatments are very expensive, and MS care is multidisciplinary and requires a lot of resources. Payers, I think, will have an increasing interest in assessing the value of high-cost care. PROs could help to demonstrate that value, especially if they can help quantify that outcomes can be improved over time at the population level.

The third area where PROs are beneficial is population health research, especially in terms of the major improvement movements going on in the country, such as those advocated for by the Institute for Healthcare Improvement via the Triple Aim: improve population health outcomes, improve the experience of care, and minimize per capita cost. PROs can really help with the first 2 of those Triple Aim categories.

What are some of the factors that limit the use of PRO measures in MS research and clinical practice?

BRANT OLIVER, PhD: In our JCOM paper,¹ we outlined major barriers or constraints or challenges to implementing PRO measures. One is time and technology in clinical care. It's busy. It's fast. A PRO measure requires patients to complete a questionnaire. And to do that, you need some time and space during the clinical visit, or before the visit, to collect this data. In the absence of appropriate technology, this is done using paper-based methods, which can be challenging. Technology can help. Examples include smart phone app-based collection systems, tablet PCs, or an online mechanism. But there are barriers to technology, too. Is the technology good enough? Is the interface with the people using it working well? Do you have an equal amount of access to and understanding of the technology from the people across your population? Is it only being used by a certain subset that is wealthier and more educated, or maybe who has English as a first language? Such constraints can be overcome, but they’re present in any situation where you are gathering PRO data.

Software constraints also extend to the electronic health record (EHR) capability of the clinics. If you can collect PRO data, does the EHR have the capability to present that data in a useful way during a 20- or 30-minute clinical visit? In other words, does it improve the quality of the visit, or does it generate just one more distraction during the visit? Some approaches, such as feed forward dashboards, provide easy visual displays that can quickly and easily add information at the point of care and can get around this limitation. However, a lot of MS care centers in private practice settings may not have the informatics capability of a large academic center.

 Academic centers often have more EHR capability, but can also  move much slower in implementation than more nimble private practice or community clinics. With many competing demands, it can sometimes take months to build one of these surveys into an EHR, for a task that takes just a day or so. But because other departments are in the queue for these changes, implementation often takes longer than we would intuitively think.

A second barrier is the burden on the individuals themselves. As much as we feel that PROs can make a difference, and the growing literature suggests that they really can, this is counterbalanced with the burden they may place on the patient. For example, a person may arrive at a clinic visit already with a lot on their mind, and then they get a questionnaire to complete in the waiting room before their visit (or perhaps they get asked to do it at home before the visit). The patient, rather than perceiving this as beneficial, may instead perceive it as an extra task or stressor. Explaining the purpose of the PRO measures, encouraging their use, and then actually using the results meaningfully during the visit can substantially improve their perceived value.

It’s important to limit the total amount of PROs used. In research, as well as in clinical settings, too many measures can lead to too little data. It's similar to an overly complex treatment plan: the chance that a person will adhere to all of it decreases with each level of complexity added. This can lead to decreased engagement by patients, and gradually they'll move away from participating in the PROs in general. Research and clinical efforts around PROs strive to get to a parsimonious set of critical measures, which will minimize the burden, but maximize the potential benefit.

 A third challenge is interpretability. This gets into the psychometric properties of the PRO measures and setting appropriate clinical thresholds for what constitutes a “positive” or “actionable” result. What is a clinically significant level of fatigue severity on the PROMIS MS Fatigue? We know that the mean range of that instrument is somewhere between 50 and 60. Does that mean that scores above 60 suggest higher than average fatigue levels, or is it more complicated than that? In many cases, the instruments don't have extensive population level research for the populations in which they are used. Setting clinical thresholds can be difficult, and I suspect that this will be a major area of research in the coming years.

Another example is setting appropriate depression thresholds. We know that on the CES-D, for example, the positive threshold for active depression in a patient with MS may be different from that in a patient without MS, because fatigue is a contributing factor to the neurovegetative symptoms that are scored on the CES-D.

So even if we can incorporate PROs into general practice and minimize burden, we also have to pay attention to factors that limit their interpretability and employ them in a way whereby they provide clinically meaningful results that can help inform care. The good news is,  for many PROs, even with these limitations, this can be done.

Following directly from this is error risk. Many PRO measures were designed for large-sample epidemiologic research, not for a smaller sample clinical practice, which can lead to a higher error risk, especially when following single individuals over time. Adjustments in how these scales are interpreted are required in many cases.

The limitations of many of the respondents completing questionnaires needs to be taken into account. Fatigue and cognitive impairment are very common in MS. Respondents may be selecting answers that are not entirely accurate if they're getting fatigued while doing the surveys, or if they have a comprehension deficit or a short-term memory deficit.

            A final challenge is demonstrating, just like any treatment or assessment approach, that the benefit outweighs the burden. PRO measures that are fairly reliable, validated, and quick and easy to complete suddenly become quite valuable because their benefit far outweighs the potential constraints. I think as PROs and the ways of assessing them continue to develop, that will become a bigger issue. The question will be, okay, you can do this, but what is the value added by doing this? That added value could be demonstrated in terms of better clinical outcomes, reduced costs, reduced hospitalizations, better treatment adherence, and so on.

How do you see the use of patient-reported outcomes in MS care evolving over time?

BRANT OLIVER, PhD: I tend to be optimistic regarding PROs. I think they will become part of the new reality, especially in LHS-oriented models of care. As mentioned, LHSs use feed forward data to predict the needs of people or optimize shared decision-making. I think we're going to see more shared decision-making rather than less over time in complex chronic illnesses care.

In environments where there's an increasing focus on value for high-cost, such as complex chronic conditions like MS and inflammatory bowel disease, systems will have to justify the high cost of treatment. PROs will be a critical piece in making that value assessment, especially since that value assessment is coming from the voice of patients rather than only from clinicians or other stakeholders. Arguably, patients may be able to contribute the most to making the value proposition because it is their outcomes and their experiences that matter to payers. Because experience matters (and not just outcomes), I think we'll see more PREMs used over time, including in MS.

I also think over time we'll see studies working to optimize PROs in the clinical environment and for improvement and research. An early example of that is the MS Continuous Quality Improvement (MS-CQI) Collaborative. It is a prospective randomized multicenter study using patient-reported and clinical data in a LHS approach to study population health outcomes and the effect of quality improvement interventions on those outcomes. It's also optimizing how these PRO measures are actually used at the point of care. Studies like MS-CQI will help to better articulate how, when, and for what purpose PROs should best be used and also when they should be avoided.

I think over time the predictive analytics component of PROs will be emphasized. There is a big push in the MS field in developing biomarkers to help predict disease progression over time. From the predictive analytics or machine learning perspective, imagine a situation where PROs could be used to predict a person's trajectory over time: if they were more likely to be lost to follow-up, to be hospitalized, or to have a relapse in the next year or 2. Reliable PROs generating population health data  at scale can inform the development of risk and outcome predictive models.  

Dr. Oliver discloses that he has received research grant support (MS-CQI research study mentioned in interview). He is the principal investigator and developer of the MS-CQI study, which is grant funded by Biogen under a Sponsored Research Agreement.

brant_oliver_dartmouth_institute.jpg

Brant Oliver, PhD, MS, MPH, APRN-BC, is a health care improvement and implementation scientist, educator, and board-certified family and psychiatric nurse practitioner (FNP-BC, PMHNP-BC). Dr. Oliver's work focuses on applied health care improvement science research, with a focus on "3C" (complex, costly, and chronic) conditions including MS, IBD, CF, RA, and others; coproduction; learning health systems; and shared decision making. He is Associate Professor at the Geisel School of Medicine at Dartmouth and also has been in clinical practice since 2003, working primarily as a certified MS specialist (MSCN) and a MS neurobehavioral nurse practitioner.  

Which patient-reported outcomes (PROs) are commonly assessed in the care of patients with multiple sclerosis (MS)?

BRANT OLIVER, PhD: The current reality is that PROs are not being used routinely in MS clinical care in most settings. They are most commonly being used in research settings, either as part of clinical trials of new treatments, or in epidemiologic studies looking at the prevalence, incidence, and severity of certain symptoms or functional impairments in MS at the population level. A good example is the NARCOMS registry, which has been used to conduct  high-quality epidemiologic studies of MS-related symptoms using self-report questionnaires.

We are starting to see some use of MS PROs in select clinical settings. Oftentimes these are measures of highly prevalent comorbid conditions that have an impact on quality of life, treatment adherence (depression severity and anxiety), and self-efficacy, which is a measure of perceived coping ability related to self-management of a chronic illness. Also, measures of patient experience are being employed in hospital-based clinics, such as the CG-CAHPS satisfaction measures.

The PROMIS (Patient-Reported Outcomes Measurement Information System) battery is a promising set of PRO measures that are being utilized across a number of conditions and that contain some MS-specific measures, such as the Fatigue MS measure. Fatigue is the most common and disabling symptom in MS, so having a good measure of fatigue is of critical importance. Having a sense of a patient’s experience with fatigue over time can also be very helpful from a clinical perspective. The PROMIS measure performs much better than its predecessor, the Modified Fatigue Impact Scale (MFIS). Another strength of the PROMIS measures is that they tend to do well correlating with other conditions. In certain research and improvement settings, PROMIS can help us understand the burden a particular disease or population places on total health system resources. For example, is fatigue burden in MS similar in severity to that in another 3C condition, such as inflammatory bowel disease?

There are other PRO measures that have not been used in MS but which I think we will see used in the future. One of these is called Collaborate, which falls into the category of patient-reported experience measures (PREMs). Collaborate measures the degree of shared decision-making that occurs in a clinical encounter, which becomes especially important as the complexity of disease-modifying treatment decisions increases as more disease-modifying therapies are introduced, each with their own profile of risks and benefits. I anticipate that the ability of clinics to facilitate effective shared decision-making will be of increasing interest to clinicians, patients, and also payers in the future.

What are some of the potential benefits of using PRO measures in terms of clinician/patient communication and clinical decision-making?

BRANT OLIVER, PhD: I think the most important benefit of measuring PROs is that it can make care more patient-centered. The big push for PROs in the first place was to incorporate more of the patient’s voice into their own care. Standard clinical measures, such as magnetic resonance imaging (MRI) or relapse rates, can be very precise and provide a good clinical picture of how the disease is behaving. But if we rely solely on that kind of information, we can lose sight of the person who is experiencing the disease.

PROs, be they just regular PROs (instruments that gather qualitative and/or quantitative patient-reported information) or PROMs (validated and standardized PRO measures) or PREMs (standardized and validated patient-reported experience measures), all aim to improve the ability to get a better story of the person who's experiencing the illness and how that condition affects them. We can also gain insight on what's most important to them regarding how their illness is managed, and what their experience of care is.

Sometimes the PRO picture and the clinical picture don't necessarily align. I'll tell a story about one a person, who's given me permission to share the story. She is an established professional, and her PRO measures concerning her quality of life, support from her workplace environment, and her ability to function in society are very high. She's very high functioning according to the PRO picture.  However,  her clinical measures suggest that she is significantly impaired. Her MRI burden is quite high, and her disability due to the MS, her EDSS (Expanded Disability Status Scale), is also quite high. So there are 2 very different stories provided by 2 different sets of data. The inclusion of PROs data, in connection with clinical data, can provide a more holistic view of this person, who is functioning well despite significant disease burden.

As a clinician working with patients with MS, this gives me much more to work with in trying to help people, not only treating the disease, but also trying to see where the person is most in need of help. A person could be very well controlled on MRI in terms of relapse rate, but have fatigue severity that is through the roof. If I weren't paying closer attention to that with a validated scale that measures fatigue severity over time, like the PROMIS Fatigue MS, I may not have a sense of how well treatment is helping that person's fatigue longitudinally, beyond what I can glean from the history and clinical examination.

PROs can also be helpful in terms of conditions that are harder to quantify. Depression is a significant cause of disability and a significant factor contributing to poor treatment adherence in MS. Also, the suicide rate in persons with MS is much higher than that in the general population. But there is no blood test for depression severity, and oftentimes a patient’s report on exam can be incomplete or misleading. Accompanying the mental status exam and the clinical evaluation with a validated depression severity measure (such as the PHQ9 [Patient Health Questionnaire-9], CES-D [Center for Epidemiologic Studies Depression Scale], or others) can help to determine whether treatments are having an initial effect, and this can also help with self-monitoring and treatment adherence.  

That brings us to the second area where PROs show real promise in MS care: the learning health system (LHS) approach, which uses feed forward information to either predict the needs of people before they come to the office or optimize decision-making at the point of care (as discussed in our review article published in JCOM).¹

PROs can play a significant role in the LHS approach in 2 ways. For example, if I have a good sense before a visit that the person has stable symptoms but is having a lot of trouble coping with MS, that may change how I allocate resources and focus the time at the visit to target the primary concern of the person before the visit even occurs. Over time I'd be able to monitor if those coping scores increase in response to the work we’re doing with the person.

The second part of this approach is a feedback mechanism. These measures can be aggregated at the population level and fed back to clinical MS centers to help clinicians who are trying to improve overall population health outcomes of people with MS to see how well they're doing on things like fatigue management, depression management, coping ability, and pain control, which can't be tested for with biologic methods. So in addition to looking at the MRI scan trends across the population and relapse rate annualized for the population over time, I'd also be able to see fatigue severity and depression severity at a clinic population level and assess the change in these severity levels over time, which I could then use to inform efforts to change how my system (clinic) provides MS care.

This capability of PROs could be of significant interest to payers, especially those who are looking to optimize the value of care that is being provided, given the high cost of MS care. MS treatments are very expensive, and MS care is multidisciplinary and requires a lot of resources. Payers, I think, will have an increasing interest in assessing the value of high-cost care. PROs could help to demonstrate that value, especially if they can help quantify that outcomes can be improved over time at the population level.

The third area where PROs are beneficial is population health research, especially in terms of the major improvement movements going on in the country, such as those advocated for by the Institute for Healthcare Improvement via the Triple Aim: improve population health outcomes, improve the experience of care, and minimize per capita cost. PROs can really help with the first 2 of those Triple Aim categories.

What are some of the factors that limit the use of PRO measures in MS research and clinical practice?

BRANT OLIVER, PhD: In our JCOM paper,¹ we outlined major barriers or constraints or challenges to implementing PRO measures. One is time and technology in clinical care. It's busy. It's fast. A PRO measure requires patients to complete a questionnaire. And to do that, you need some time and space during the clinical visit, or before the visit, to collect this data. In the absence of appropriate technology, this is done using paper-based methods, which can be challenging. Technology can help. Examples include smart phone app-based collection systems, tablet PCs, or an online mechanism. But there are barriers to technology, too. Is the technology good enough? Is the interface with the people using it working well? Do you have an equal amount of access to and understanding of the technology from the people across your population? Is it only being used by a certain subset that is wealthier and more educated, or maybe who has English as a first language? Such constraints can be overcome, but they’re present in any situation where you are gathering PRO data.

Software constraints also extend to the electronic health record (EHR) capability of the clinics. If you can collect PRO data, does the EHR have the capability to present that data in a useful way during a 20- or 30-minute clinical visit? In other words, does it improve the quality of the visit, or does it generate just one more distraction during the visit? Some approaches, such as feed forward dashboards, provide easy visual displays that can quickly and easily add information at the point of care and can get around this limitation. However, a lot of MS care centers in private practice settings may not have the informatics capability of a large academic center.

 Academic centers often have more EHR capability, but can also  move much slower in implementation than more nimble private practice or community clinics. With many competing demands, it can sometimes take months to build one of these surveys into an EHR, for a task that takes just a day or so. But because other departments are in the queue for these changes, implementation often takes longer than we would intuitively think.

A second barrier is the burden on the individuals themselves. As much as we feel that PROs can make a difference, and the growing literature suggests that they really can, this is counterbalanced with the burden they may place on the patient. For example, a person may arrive at a clinic visit already with a lot on their mind, and then they get a questionnaire to complete in the waiting room before their visit (or perhaps they get asked to do it at home before the visit). The patient, rather than perceiving this as beneficial, may instead perceive it as an extra task or stressor. Explaining the purpose of the PRO measures, encouraging their use, and then actually using the results meaningfully during the visit can substantially improve their perceived value.

It’s important to limit the total amount of PROs used. In research, as well as in clinical settings, too many measures can lead to too little data. It's similar to an overly complex treatment plan: the chance that a person will adhere to all of it decreases with each level of complexity added. This can lead to decreased engagement by patients, and gradually they'll move away from participating in the PROs in general. Research and clinical efforts around PROs strive to get to a parsimonious set of critical measures, which will minimize the burden, but maximize the potential benefit.

 A third challenge is interpretability. This gets into the psychometric properties of the PRO measures and setting appropriate clinical thresholds for what constitutes a “positive” or “actionable” result. What is a clinically significant level of fatigue severity on the PROMIS MS Fatigue? We know that the mean range of that instrument is somewhere between 50 and 60. Does that mean that scores above 60 suggest higher than average fatigue levels, or is it more complicated than that? In many cases, the instruments don't have extensive population level research for the populations in which they are used. Setting clinical thresholds can be difficult, and I suspect that this will be a major area of research in the coming years.

Another example is setting appropriate depression thresholds. We know that on the CES-D, for example, the positive threshold for active depression in a patient with MS may be different from that in a patient without MS, because fatigue is a contributing factor to the neurovegetative symptoms that are scored on the CES-D.

So even if we can incorporate PROs into general practice and minimize burden, we also have to pay attention to factors that limit their interpretability and employ them in a way whereby they provide clinically meaningful results that can help inform care. The good news is,  for many PROs, even with these limitations, this can be done.

Following directly from this is error risk. Many PRO measures were designed for large-sample epidemiologic research, not for a smaller sample clinical practice, which can lead to a higher error risk, especially when following single individuals over time. Adjustments in how these scales are interpreted are required in many cases.

The limitations of many of the respondents completing questionnaires needs to be taken into account. Fatigue and cognitive impairment are very common in MS. Respondents may be selecting answers that are not entirely accurate if they're getting fatigued while doing the surveys, or if they have a comprehension deficit or a short-term memory deficit.

            A final challenge is demonstrating, just like any treatment or assessment approach, that the benefit outweighs the burden. PRO measures that are fairly reliable, validated, and quick and easy to complete suddenly become quite valuable because their benefit far outweighs the potential constraints. I think as PROs and the ways of assessing them continue to develop, that will become a bigger issue. The question will be, okay, you can do this, but what is the value added by doing this? That added value could be demonstrated in terms of better clinical outcomes, reduced costs, reduced hospitalizations, better treatment adherence, and so on.

How do you see the use of patient-reported outcomes in MS care evolving over time?

BRANT OLIVER, PhD: I tend to be optimistic regarding PROs. I think they will become part of the new reality, especially in LHS-oriented models of care. As mentioned, LHSs use feed forward data to predict the needs of people or optimize shared decision-making. I think we're going to see more shared decision-making rather than less over time in complex chronic illnesses care.

In environments where there's an increasing focus on value for high-cost, such as complex chronic conditions like MS and inflammatory bowel disease, systems will have to justify the high cost of treatment. PROs will be a critical piece in making that value assessment, especially since that value assessment is coming from the voice of patients rather than only from clinicians or other stakeholders. Arguably, patients may be able to contribute the most to making the value proposition because it is their outcomes and their experiences that matter to payers. Because experience matters (and not just outcomes), I think we'll see more PREMs used over time, including in MS.

I also think over time we'll see studies working to optimize PROs in the clinical environment and for improvement and research. An early example of that is the MS Continuous Quality Improvement (MS-CQI) Collaborative. It is a prospective randomized multicenter study using patient-reported and clinical data in a LHS approach to study population health outcomes and the effect of quality improvement interventions on those outcomes. It's also optimizing how these PRO measures are actually used at the point of care. Studies like MS-CQI will help to better articulate how, when, and for what purpose PROs should best be used and also when they should be avoided.

I think over time the predictive analytics component of PROs will be emphasized. There is a big push in the MS field in developing biomarkers to help predict disease progression over time. From the predictive analytics or machine learning perspective, imagine a situation where PROs could be used to predict a person's trajectory over time: if they were more likely to be lost to follow-up, to be hospitalized, or to have a relapse in the next year or 2. Reliable PROs generating population health data  at scale can inform the development of risk and outcome predictive models.  

Dr. Oliver discloses that he has received research grant support (MS-CQI research study mentioned in interview). He is the principal investigator and developer of the MS-CQI study, which is grant funded by Biogen under a Sponsored Research Agreement.

brant_oliver_dartmouth_institute.jpg

Brant Oliver, PhD, MS, MPH, APRN-BC, is a health care improvement and implementation scientist, educator, and board-certified family and psychiatric nurse practitioner (FNP-BC, PMHNP-BC). Dr. Oliver's work focuses on applied health care improvement science research, with a focus on "3C" (complex, costly, and chronic) conditions including MS, IBD, CF, RA, and others; coproduction; learning health systems; and shared decision making. He is Associate Professor at the Geisel School of Medicine at Dartmouth and also has been in clinical practice since 2003, working primarily as a certified MS specialist (MSCN) and a MS neurobehavioral nurse practitioner.  

Which patient-reported outcomes (PROs) are commonly assessed in the care of patients with multiple sclerosis (MS)?

BRANT OLIVER, PhD: The current reality is that PROs are not being used routinely in MS clinical care in most settings. They are most commonly being used in research settings, either as part of clinical trials of new treatments, or in epidemiologic studies looking at the prevalence, incidence, and severity of certain symptoms or functional impairments in MS at the population level. A good example is the NARCOMS registry, which has been used to conduct  high-quality epidemiologic studies of MS-related symptoms using self-report questionnaires.

We are starting to see some use of MS PROs in select clinical settings. Oftentimes these are measures of highly prevalent comorbid conditions that have an impact on quality of life, treatment adherence (depression severity and anxiety), and self-efficacy, which is a measure of perceived coping ability related to self-management of a chronic illness. Also, measures of patient experience are being employed in hospital-based clinics, such as the CG-CAHPS satisfaction measures.

The PROMIS (Patient-Reported Outcomes Measurement Information System) battery is a promising set of PRO measures that are being utilized across a number of conditions and that contain some MS-specific measures, such as the Fatigue MS measure. Fatigue is the most common and disabling symptom in MS, so having a good measure of fatigue is of critical importance. Having a sense of a patient’s experience with fatigue over time can also be very helpful from a clinical perspective. The PROMIS measure performs much better than its predecessor, the Modified Fatigue Impact Scale (MFIS). Another strength of the PROMIS measures is that they tend to do well correlating with other conditions. In certain research and improvement settings, PROMIS can help us understand the burden a particular disease or population places on total health system resources. For example, is fatigue burden in MS similar in severity to that in another 3C condition, such as inflammatory bowel disease?

There are other PRO measures that have not been used in MS but which I think we will see used in the future. One of these is called Collaborate, which falls into the category of patient-reported experience measures (PREMs). Collaborate measures the degree of shared decision-making that occurs in a clinical encounter, which becomes especially important as the complexity of disease-modifying treatment decisions increases as more disease-modifying therapies are introduced, each with their own profile of risks and benefits. I anticipate that the ability of clinics to facilitate effective shared decision-making will be of increasing interest to clinicians, patients, and also payers in the future.

What are some of the potential benefits of using PRO measures in terms of clinician/patient communication and clinical decision-making?

BRANT OLIVER, PhD: I think the most important benefit of measuring PROs is that it can make care more patient-centered. The big push for PROs in the first place was to incorporate more of the patient’s voice into their own care. Standard clinical measures, such as magnetic resonance imaging (MRI) or relapse rates, can be very precise and provide a good clinical picture of how the disease is behaving. But if we rely solely on that kind of information, we can lose sight of the person who is experiencing the disease.

PROs, be they just regular PROs (instruments that gather qualitative and/or quantitative patient-reported information) or PROMs (validated and standardized PRO measures) or PREMs (standardized and validated patient-reported experience measures), all aim to improve the ability to get a better story of the person who's experiencing the illness and how that condition affects them. We can also gain insight on what's most important to them regarding how their illness is managed, and what their experience of care is.

Sometimes the PRO picture and the clinical picture don't necessarily align. I'll tell a story about one a person, who's given me permission to share the story. She is an established professional, and her PRO measures concerning her quality of life, support from her workplace environment, and her ability to function in society are very high. She's very high functioning according to the PRO picture.  However,  her clinical measures suggest that she is significantly impaired. Her MRI burden is quite high, and her disability due to the MS, her EDSS (Expanded Disability Status Scale), is also quite high. So there are 2 very different stories provided by 2 different sets of data. The inclusion of PROs data, in connection with clinical data, can provide a more holistic view of this person, who is functioning well despite significant disease burden.

As a clinician working with patients with MS, this gives me much more to work with in trying to help people, not only treating the disease, but also trying to see where the person is most in need of help. A person could be very well controlled on MRI in terms of relapse rate, but have fatigue severity that is through the roof. If I weren't paying closer attention to that with a validated scale that measures fatigue severity over time, like the PROMIS Fatigue MS, I may not have a sense of how well treatment is helping that person's fatigue longitudinally, beyond what I can glean from the history and clinical examination.

PROs can also be helpful in terms of conditions that are harder to quantify. Depression is a significant cause of disability and a significant factor contributing to poor treatment adherence in MS. Also, the suicide rate in persons with MS is much higher than that in the general population. But there is no blood test for depression severity, and oftentimes a patient’s report on exam can be incomplete or misleading. Accompanying the mental status exam and the clinical evaluation with a validated depression severity measure (such as the PHQ9 [Patient Health Questionnaire-9], CES-D [Center for Epidemiologic Studies Depression Scale], or others) can help to determine whether treatments are having an initial effect, and this can also help with self-monitoring and treatment adherence.  

That brings us to the second area where PROs show real promise in MS care: the learning health system (LHS) approach, which uses feed forward information to either predict the needs of people before they come to the office or optimize decision-making at the point of care (as discussed in our review article published in JCOM).¹

PROs can play a significant role in the LHS approach in 2 ways. For example, if I have a good sense before a visit that the person has stable symptoms but is having a lot of trouble coping with MS, that may change how I allocate resources and focus the time at the visit to target the primary concern of the person before the visit even occurs. Over time I'd be able to monitor if those coping scores increase in response to the work we’re doing with the person.

The second part of this approach is a feedback mechanism. These measures can be aggregated at the population level and fed back to clinical MS centers to help clinicians who are trying to improve overall population health outcomes of people with MS to see how well they're doing on things like fatigue management, depression management, coping ability, and pain control, which can't be tested for with biologic methods. So in addition to looking at the MRI scan trends across the population and relapse rate annualized for the population over time, I'd also be able to see fatigue severity and depression severity at a clinic population level and assess the change in these severity levels over time, which I could then use to inform efforts to change how my system (clinic) provides MS care.

This capability of PROs could be of significant interest to payers, especially those who are looking to optimize the value of care that is being provided, given the high cost of MS care. MS treatments are very expensive, and MS care is multidisciplinary and requires a lot of resources. Payers, I think, will have an increasing interest in assessing the value of high-cost care. PROs could help to demonstrate that value, especially if they can help quantify that outcomes can be improved over time at the population level.

The third area where PROs are beneficial is population health research, especially in terms of the major improvement movements going on in the country, such as those advocated for by the Institute for Healthcare Improvement via the Triple Aim: improve population health outcomes, improve the experience of care, and minimize per capita cost. PROs can really help with the first 2 of those Triple Aim categories.

What are some of the factors that limit the use of PRO measures in MS research and clinical practice?

BRANT OLIVER, PhD: In our JCOM paper,¹ we outlined major barriers or constraints or challenges to implementing PRO measures. One is time and technology in clinical care. It's busy. It's fast. A PRO measure requires patients to complete a questionnaire. And to do that, you need some time and space during the clinical visit, or before the visit, to collect this data. In the absence of appropriate technology, this is done using paper-based methods, which can be challenging. Technology can help. Examples include smart phone app-based collection systems, tablet PCs, or an online mechanism. But there are barriers to technology, too. Is the technology good enough? Is the interface with the people using it working well? Do you have an equal amount of access to and understanding of the technology from the people across your population? Is it only being used by a certain subset that is wealthier and more educated, or maybe who has English as a first language? Such constraints can be overcome, but they’re present in any situation where you are gathering PRO data.

Software constraints also extend to the electronic health record (EHR) capability of the clinics. If you can collect PRO data, does the EHR have the capability to present that data in a useful way during a 20- or 30-minute clinical visit? In other words, does it improve the quality of the visit, or does it generate just one more distraction during the visit? Some approaches, such as feed forward dashboards, provide easy visual displays that can quickly and easily add information at the point of care and can get around this limitation. However, a lot of MS care centers in private practice settings may not have the informatics capability of a large academic center.

 Academic centers often have more EHR capability, but can also  move much slower in implementation than more nimble private practice or community clinics. With many competing demands, it can sometimes take months to build one of these surveys into an EHR, for a task that takes just a day or so. But because other departments are in the queue for these changes, implementation often takes longer than we would intuitively think.

A second barrier is the burden on the individuals themselves. As much as we feel that PROs can make a difference, and the growing literature suggests that they really can, this is counterbalanced with the burden they may place on the patient. For example, a person may arrive at a clinic visit already with a lot on their mind, and then they get a questionnaire to complete in the waiting room before their visit (or perhaps they get asked to do it at home before the visit). The patient, rather than perceiving this as beneficial, may instead perceive it as an extra task or stressor. Explaining the purpose of the PRO measures, encouraging their use, and then actually using the results meaningfully during the visit can substantially improve their perceived value.

It’s important to limit the total amount of PROs used. In research, as well as in clinical settings, too many measures can lead to too little data. It's similar to an overly complex treatment plan: the chance that a person will adhere to all of it decreases with each level of complexity added. This can lead to decreased engagement by patients, and gradually they'll move away from participating in the PROs in general. Research and clinical efforts around PROs strive to get to a parsimonious set of critical measures, which will minimize the burden, but maximize the potential benefit.

 A third challenge is interpretability. This gets into the psychometric properties of the PRO measures and setting appropriate clinical thresholds for what constitutes a “positive” or “actionable” result. What is a clinically significant level of fatigue severity on the PROMIS MS Fatigue? We know that the mean range of that instrument is somewhere between 50 and 60. Does that mean that scores above 60 suggest higher than average fatigue levels, or is it more complicated than that? In many cases, the instruments don't have extensive population level research for the populations in which they are used. Setting clinical thresholds can be difficult, and I suspect that this will be a major area of research in the coming years.

Another example is setting appropriate depression thresholds. We know that on the CES-D, for example, the positive threshold for active depression in a patient with MS may be different from that in a patient without MS, because fatigue is a contributing factor to the neurovegetative symptoms that are scored on the CES-D.

So even if we can incorporate PROs into general practice and minimize burden, we also have to pay attention to factors that limit their interpretability and employ them in a way whereby they provide clinically meaningful results that can help inform care. The good news is,  for many PROs, even with these limitations, this can be done.

Following directly from this is error risk. Many PRO measures were designed for large-sample epidemiologic research, not for a smaller sample clinical practice, which can lead to a higher error risk, especially when following single individuals over time. Adjustments in how these scales are interpreted are required in many cases.

The limitations of many of the respondents completing questionnaires needs to be taken into account. Fatigue and cognitive impairment are very common in MS. Respondents may be selecting answers that are not entirely accurate if they're getting fatigued while doing the surveys, or if they have a comprehension deficit or a short-term memory deficit.

            A final challenge is demonstrating, just like any treatment or assessment approach, that the benefit outweighs the burden. PRO measures that are fairly reliable, validated, and quick and easy to complete suddenly become quite valuable because their benefit far outweighs the potential constraints. I think as PROs and the ways of assessing them continue to develop, that will become a bigger issue. The question will be, okay, you can do this, but what is the value added by doing this? That added value could be demonstrated in terms of better clinical outcomes, reduced costs, reduced hospitalizations, better treatment adherence, and so on.

How do you see the use of patient-reported outcomes in MS care evolving over time?

BRANT OLIVER, PhD: I tend to be optimistic regarding PROs. I think they will become part of the new reality, especially in LHS-oriented models of care. As mentioned, LHSs use feed forward data to predict the needs of people or optimize shared decision-making. I think we're going to see more shared decision-making rather than less over time in complex chronic illnesses care.

In environments where there's an increasing focus on value for high-cost, such as complex chronic conditions like MS and inflammatory bowel disease, systems will have to justify the high cost of treatment. PROs will be a critical piece in making that value assessment, especially since that value assessment is coming from the voice of patients rather than only from clinicians or other stakeholders. Arguably, patients may be able to contribute the most to making the value proposition because it is their outcomes and their experiences that matter to payers. Because experience matters (and not just outcomes), I think we'll see more PREMs used over time, including in MS.

I also think over time we'll see studies working to optimize PROs in the clinical environment and for improvement and research. An early example of that is the MS Continuous Quality Improvement (MS-CQI) Collaborative. It is a prospective randomized multicenter study using patient-reported and clinical data in a LHS approach to study population health outcomes and the effect of quality improvement interventions on those outcomes. It's also optimizing how these PRO measures are actually used at the point of care. Studies like MS-CQI will help to better articulate how, when, and for what purpose PROs should best be used and also when they should be avoided.

I think over time the predictive analytics component of PROs will be emphasized. There is a big push in the MS field in developing biomarkers to help predict disease progression over time. From the predictive analytics or machine learning perspective, imagine a situation where PROs could be used to predict a person's trajectory over time: if they were more likely to be lost to follow-up, to be hospitalized, or to have a relapse in the next year or 2. Reliable PROs generating population health data  at scale can inform the development of risk and outcome predictive models.  

Dr. Oliver discloses that he has received research grant support (MS-CQI research study mentioned in interview). He is the principal investigator and developer of the MS-CQI study, which is grant funded by Biogen under a Sponsored Research Agreement.