BACKGROUND: The concept of mental toughness (MT) has been extensively investigated in various high-stress domains, including sports and the military. It is associated with superior performance in challenging environments. Presently, the Sports Mental Toughness Questionnaire (SMTQ) and Military Training Mental Toughness Inventory (MTMTI) are administered to assess MT in firefighters. Previous research suggested these instruments may lack specificity for firefighting. Therefore, this study aimed to investigate the concurrent validity of SMTQ and MTMTI in the context of firefighters. In addition, to determine whether the SMTQ and MTMTI could validly assess MT in firefighters. METHODS: Sixty-two male firefighters from two southern Florida Fire departments participated in a two-day data collection process. They completed the SMTQ (self-assessment), while a colleague and an officer rated their MT using the MTMTI. Mean MTMTI scores per day for both assessors were computed, and both surveys were converted into z-scores for statistical analysis. This analysis involved Pearson correlation (r), Cohen’s Kappa (k), and two-way random Intraclass correlation coefficient (ICC2k) using R statistical packages in Jamovi version 2.4.8 (p < 0.05). RESULTS: On Day 1, inventory scores were negatively correlated (r = -0.09, p = 0.5), while on Day 2, they were positively correlated (r = 0.1, p = 0.5). The reliability agreement on Day 1 was extremely poor (ICC2k < 0.001, 95%CI [-0.53, 0.35]). On Day 2, reliability improved but remained relatively low (ICC2k = 0.13, 95%CI [-0.33, 0.43]). Cohen’s kappa for both days indicated no agreement between the inventory scores (k < 0.001). CONCLUSION: The results suggest weak and non-significant correlations between the MT inventory scores on both Day 1 and Day 2, along with poor reliability and agreement between the measurements. These findings indicate a need for caution when interpreting and relying on these inventory scores for making inferences. Further development of firefighting-specific MT assessment tools may be warranted.

This document is currently not available here.