A Review of Common Limitations in Rural-Related Studies in the Peer-Reviewed Literature

Introduction Roughly a fifth of the U.S. population lives in rural communities, according to the 2020 U.S. Census.1Health disparities between rural and urban communities are persistent and exist across the lifespan.2 For example, gaps in mortality rates between rural and urban communities are sizable and have increased over time. 3-5 Similarly, rural residents continue to experience a disproportionate burden of other negative health outcomes, such as cardiovascular, respiratory, mental health, and cancer-related outcomes.6-9 Underlying social and economic factors such as limited health care access and transportation options contribute to rural health disparities.2 High-quality data, research, and evidence are crucial to understanding and improving health outcomes in rural communities. Health-related data provide valuable information for promoting the health of individuals and populations.10,11 Derived from various sources (e.g., vital records, health care claims, and surveys), 12 data commonly underpin decision making regarding population health, health care delivery, funding eligibility and allocation, reporting, and more. Despite their importance, there are known gaps in data, data limitations, and analytic challenges when conducting rural health research. Frequently used datasets, such as the American Community Survey and mortality data from the Centers for Disease Control and Prevention (CDC), may provide national, state, and large metropolitan estimates, but estimates for smaller and rural regions are often limited.13-15 Additional challenges such as underreporting, small sample sizes, lack of resources for collecting data, and reporting delays further undermine the quality of available data.13,14,16 Such challenges can hinder efforts to understand rural health issues and effectively target resources and interventions in rural communities. The need for reliable and timely data received renewed attention during the COVID-19 pandemic as well. 17 Applying definitions of “rural” creates further challenges to using health-related data in rural communities.2,15,18 Estimates suggest there are over 30 different rural definitions used by the federal government.19,20 A recent review similarly indicated that rural definitions and the geographic levels of analysis may not be consistent across the body of health services research.21 Rural definitions can influence the direction of research, policy, and practice.20,22 Discrepancies between rural definitions, for example, can impact estimates of rural population sizes and the scale of disparities observed in rural areas.22 While challenges specific to rural definitions have increasingly been documented, a systematic assessment of the broader challenges associated with using health-related data in rural communities is lacking. Understanding the range of limitations inherent to rural health data is critical in identifying strategies to overcome them. This study sought to quantify and characterize limitations commonly reported in rural-related studies, with a focus on studies using secondary, quantitative data. We systematically reviewed a subset of the literature published in select peer-reviewed journals between 2019-2022. Study characteristics, secondary data sources, and author-reported limitations were summarized. Findings provide a thorough description of data-related challenges, which could be used by researchers, policymakers, and health data creators to improve rural health data and inform future initiatives relying on those data, including those at the national, state, local, or tribal levels.