Validation of a Newly Developed Assessment Tool for Point-of-Care Ultrasound of the Thorax in Healthy Volunteers (VALPOCUS)

Abstract

Objectives: Point-of-care ultrasound (POCUS) has become an integral part of emergency, in tensive care, and perioperative medicine. However, the training and subsequent evaluation of POCUS users are still not standardized. The aim of the study was to develop and validate an assessment tool for POCUS users. Methods: After reviewing the existing literature and a multi-stage expert survey (Delphi method), consensus on twelve items for the assessment tool was reached. To validate the assessment tool, a group of volunteer doctors and medical students performed a POCUS examination using simple linear probe and more complex sector probe techniques. The examination was evaluated by two independent assessors using the created assessment tool. Then, four experts evaluated anonymized recordings of the examinations. We tested the reliability and validity, including internal consistency. Results: A total of 70 examinations were included. Of these, 19 examinations were carried out by physicians and 51 by medical students. A high inter-rater reliability (Cohen’s kappa 0.78 (linear weighted; SEM 0.37; p < 0.001) and Krippendorff’s alpha 0.895) was shown for the evaluation tool. To improve discriminative power and strengthen reliability, the assessment tool was modified using Cronbach’s alpha. Modification resulted in the removal of three items (patient positioning, ultrasound mode selection, and probe selection) from the tool. The mean values of instrument and expert ratings were now 2.62% apart (46.90% instrument vs. 44.29% expert). Pearson’s correlation coefficient between tool and expert ratings showed moderate to high validity (r = 0.69; p < 0.001). Conclusions: The new assessment tool is highly reliable and a valid tool for assessing POCUS skills. It holds strong potential for integration into medical education and training to objectify ultrasound skills. Further studies are required to investigate discriminatory power and transferability to other POCUS algorithms.