Validity / Reliability / Usefulness As Hunger Vital Sign Expands

Find all supporting materials at the Hunger Vital Sign explainer series website.

This episode features an interview with Richard Sheward, Director of Innovative Partnerships at Children's HealthWatch. 

Citation for the Hunger Vital Sign tool and link to the original research:

Hager, E. R., Quigg, A. M., Black, M. M., Coleman, S. M., Heeren, T., Rose-Jacobs, R., Cook, J. T., Ettinger de Cuba, S. E., Casey, P. H., Chilton, M., Cutts, D. B., Meyers A. F., Frank, D. A. (2010). Development and Validity of a 2-Item Screen to Identify Families at Risk for Food Insecurity. Pediatrics, 126(1), 26-32. doi:10.1542/peds.2009-3146.

Audio Editing and Post-Production Provided By Evergreen Audio

Transcript

LABUN:

Welcome to episode number six in our series of short explainers for the Hunger Vital Sign tool. I’m your host, Helen Labun. And to help with the explanations, we have a guest expert from the organization that created the Hunger Vital Sign. 

SHEWARD:

I'm Richard Sheward, Director of Innovative Partnerships at Children's HealthWatch

LABUN:

From the previous episodes we know we have a screening tool that’s valid, reliable, and appears to be useful for many health care practices and health providers. As we discussed in the episode on reliability, these attributes are the first step towards having what can be considered a standardized tool. 

SHEWARD:

One way that standardization happens is when you have a previously validated test or tool, in this case the Hunger Vital Sign. . . that is continually administered in the same manner and shows consistently reliable results again and again

LABUN:

Standardization can refer to having the policy frameworks in place to use the same reliable tool, or it can mean this tool is so commonly used that we would colloquially refer to it as “standard practice”. Hunger Vital Sign is in a bit of an in-between place. It’s a formally-defined way to measure food insecurity, but as of this recording most health care practices choose to adopt it without a national policy instructing them to do so, and where external policies exist they tend to be tied to certain patient groups, such as Medicaid beneficiaries. For the purpose of this series, then, we’ll consider that akin to the colloquial understanding of standard practice.  

Other tools can appear outside of the policy realm to help this type of expansion. For example, most electronic health records, or EHRs, have now added a field to allow for documenting food insecurity screening results. This is a recent change that could have major implications.

SHEWARD:

If you have the same tools embedded into the variety of EHRs that are used out there, then you have continuity of care. If a patient moves to a new provider or moves to a new location the pieces aren't missing.

LABUN:

And if you can document across practices, then it becomes possible to start using the information across broader regions. Last episode we talked about a single practice reviewing its patient population data to find gaps in access and programs, but we can think bigger than that.

SHEWARD:

The benefit of having standards in regard to tool selection and recommendations is that you are then able to compare results across populations, across states, across counties even, you know, census tracks. And so the ability to compare from a public policy and public health perspective is really important.  

LABUN:

But to reach this larger potential, the way in which a tool like Hunger Vital Sign expands – whether through practice-by-practice adoption or official policy - becomes an issue. 

A standardized tool in the sense of being reliable and well documented, but with practices left to decide on their own how to implement it, has a definite appeal  - especially given that the context in which Hunger Vital Sign is used, that pathway of next steps after the screening, may change from place to place. 

But this type of expansion has downsides, too. Even if you’ve documented the correct way to perform the screening, without a mechanism for replicating that in every place it can become a bit of a game of telephone, with modifications along the way. Remember the example of changing the answer options and possibly missing a large group of patients. Other changes may happen based on the goals of the health care practice, for example if they’re trying to build their own general screening tool that covers a range of programs they offer, or needs they see in their community, they might bundle a lot of questions together into one composite screener. 

SHEWARD:

I think in an ideal scenario, if a composite tool is constructed that you would be able to perform new analyses to ensure that validation is still there, that it's still a reliable tool, whether it's through psychometric testing or other means. I think in reality what the field often relies on is subject matter expertise. And I think, in the still nascent realm of bringing social risks into care, that that's sufficient. In the case of the Hunger Vital Sign being incorporated into tools such as the CMS Accountable Health Communities screening tool, the integrity of the Hunger Vital Sign we believe is maintained. Our researchers were consulted as part of the development process and creating that 10 item, five domain tool.

LABUN:

The original researchers were consulted – plus, the efficacy of Hunger Vital Sign relies on encouraging ongoing research beyond Children’s HealthWatch to test new applications. The research group wants a certain level of informed tinkering and sharing results back out to the broader research community. 

SHEWARD:

One thing to give CMS credit for is, you know, they offer guidance on using their screening tool for Accountable Health Communities in settings that are not part of the CMS model.  

LABUN:

The Accountable Health Communities model doesn’t only test the reliability of the screening tool itself, it also tests best practices for on-the-ground implementation and those next steps elements of connecting patients with resources to address food insecurity if that’s a concern. However, as the previous episodes have illustrated, testing out new applications or iterations of a tool like Hunger Vital Sign is a fairly involved process – and getting it wrong can have serious consequences for patients and the relationship of trust they’re building with health care providers. There’s a balance to reach. 

SHEWARD:

There’s sort of the ideal scenario where a provider is using the Hunger Vital Sign exactly as it was validated, in every single instance, whether that's in New York or California or Texas or Tennessee. Then there's the reality and the constraints that different providers or institutions face and the patients that they serve and questions might arise about, do we use one question or do we have to use both questions? Does the response options, you know, need to be the three-part response -- often, sometimes, never true? Or can it be, yes, no. Does the timeframe need to be in the last 12 months, or can it be in the last seven days? And I think the more that you modify the tool as it's been validated the less accurate and the less useful it is. If you think of it like a sharp blade, you go from having a highly sharpened scalpel to, you know, maybe a rusty butter knife in its worst case scenario. I think there's a middle ground somewhere, but the more that you are able to maintain the integrity of the tool that's been validated, the better patients will be served, the better the outcome will be ultimately for everyone.

LABUN:

We can also go too far into testing, and re-testing, what makes the perfect screening tool. Yes, it’s important to get it as right as possible, but it’s also important to get past screening and into doing something about the problem we’re trying to detect. Hunger Vital Sign research has been going on since the late 1990s. Since the year I graduated from high school, to be exact. The year Backstreet Boys and Third Eye Blind were topping the pop charts, and we did not yet know what it would literally mean to party like it was 1999. We’re a full generation past that time. And the trends in poor health outcomes connected to diet have only gotten worse – a lot worse. 

Here's another concern. Remember that Hunger Vital Sign had a head start with the work the U.S. Dept of Agriculture had done to define a detailed monitoring tool, the Household Food Security Survey or HFSS.

SHEWARD:

The reason why we were able to develop this shortened screener for food insecurity was because we were able to rely on that gold standard. In the case of housing instability, for example, there is actually no agreed upon official definition of the circumstances that would define how housing instability is it moving multiple times, is it overcrowding? Is it the particular conditions of the housing, whether there are pests or mold or hazards? What role does homelessness play? Or being behind on rent or mortgage payments? There are a number of circumstances that factor into housing and stability, but there's no technical agreed upon definition. We believe that investments in future research to create a robust standard diagnostic tool is warranted. And using the U S D A's HFSS as a model agencies like HUD have taken steps in that direction to create a way to truly measure one's risk of housing instability or homelessness.

LABUN:

If work in food insecurity screening can serve as a model for other issue areas, then presumably work on the next steps after screening can become a model too. Let’s check in one last time on that original goal statement for Hunger Vital Sign.

SHEWARD:

I hope that health care’s interest in identifying and addressing food and security continues to mature and become more refined. And that researchers and clinicians work closely with policymakers to continue to codify, systematize, institutionalize this process to ultimately promote health and wellbeing for all families. And that our structural social safety nets are adequate and robust enough to address food insecurity when it's identified.

LABUN:

Those bigger goals can’t be reached unless use of the tool expands, in a way that balances maintaining validity and reliability with the practical considerations of implementing a screening system that’s immediately useful to the practice and, most essentially, to the patient who is there being screened; something that’s responsive to our lofty future goals and also something useful at the time when patients are engaging with the health system. It’s a tall order and our next set of episodes will look at how using Hunger Vital Sign as the starting point it’s meant to be is playing out in health care.