INTRODUCTION: Wearable technology is a quickly evolving field, and new devices with new features to measure/estimate physiological variables are being released constantly. Despite their use, the validity of the devices are largely unknown to the users or researchers, and the quality of the studies that do test validity and reliability vary widely. PURPOSE: Therefore, the purpose of this systematic review was to review the current validity and reliability literature concerning consumer-grade wearable technology measurements/estimates of physiological variables during exercise. Additionally, we sought to perform risk of bias assessments utilizing the novel WEArable technology Risk of Bias and Objectivity Tool (WEAR-BOT). METHODS: This review was conducted following PRISMA guidelines, searching 3 databases: Google Scholar, Scopus, and SPORTDiscus. After screening, 46 papers were identified that met the pre-determined criteria. Then data was extracted and risk of bias assessment performed by independent researchers. Descriptive statistics, weighted averages of mean absolute percentage error (MAPE) and Pearson correlations were calculated. Sample size statistics were performed utilizing the lower 95% confidence interval of the weighted correlation average. RESULTS: Of the 46 papers reviewed, 44 performed validity testing, while 9 performed reliability. The weighted average for MAPE was 12.48% for heart rate (HR) and 30.70% for energy expenditure (EE). The weighted average for Pearson correlations was 0.737 for HR and 0.672 for EE. Risk of bias assessment of validity studies resulted in 30/44 studies being classified as having a “High Risk of Bias”, and 14/44 having “Some Risk of Bias”. None had a “Low Risk of Bias”, according to the novel WEAR-BOT. For reliability studies, 7/9 were classified as “High Risk of Bias”, 2 as “Some Risk of Bias”, and 0 as “Low Risk of Bias”. CONCLUSION: The risk of bias assessment and descriptive statistics paint a troubling picture of the overall state of validity and reliability studies. Statistical analyses, methods, and reporting vary excessively. This review and associated WEAR-BOT analysis can be used by researchers to help standardize methodology, analytics, and reporting of validation and reliability studies of consumer-grade wearable technology.
Carrier, Bryson; Bunn, Jennifer; Reece, Joel D.; Aguilar, Charli D.; Eschbach, Chris; and Navalta, James W.
"The Risk of Bias in Validity and Reliability Studies Testing Physiological Variables using Consumer-Grade Wearable Technology: A Systematic Review and WEAR-BOT Analysis,"
International Journal of Exercise Science: Conference Proceedings: Vol. 14:
3, Article 6.
Available at: https://digitalcommons.wku.edu/ijesab/vol14/iss3/6