Developments

Description: symptomcheckR is an R package that can be used to analyze the performance of symptom checkers across various metrics and create publication-ready figures with one command. It builds on our previously developed methods for evaluating and reporting the performance of different symptom checkers in a standardized and more reliable way. Being the first software of this kind, we aim to move the field towards not only proposing improved methods, but also making them easy to implement.

Download:

GitHub: https://github.com/ma-kopka/symptomcheckR

CRAN: https://cran.r-project.org/web/packages/symptomcheckR/index.html

Journal Publication & Tutorial: https://doi.org/10.1186/s44247-024-00096-7

Description: The RepVig Framework is a standardized method to develop representative clinical vignettes that can be used in digital health and medical education. In contrast to traditional vignettes, these vignettes have high external validity (i.e., they represent real-world data). By standardizing the vignette creation processed instead of using a fixed vignette set, these vignettes are not part of the training data of LLMs and other AI applications, making them a suitable method to evaluate such tools.

Download:

Journal Publication: https://www.nature.com/articles/s41598-024-83844-z

Worksheet: https://static-content.springer.com/esm/art%3A10.1038%2Fs41598-024-83844-z/MediaObjects/41598_2024_83844_MOESM1_ESM.docx

Description: The G-MAUQ and G-MAUQ-S are two validated questionnaires to measure the usability of mHealth applications in German language across three factors: (1) Ease of use, (2) Interface and Satisfaction, and (3) Usefulness. We specifically developed a short version, the G-MAUQ-S, for circumstances with limited time.

Download:

Journal Publication: https://doi.org/10.1016/j.smhl.2024.100517

Questionnaire G-MAUQ: https://ars.els-cdn.com/content/image/1-s2.0-S2352648324000734-mmc2.pdf

Questionnaire G-MAUQ-S: https://ars.els-cdn.com/content/image/1-s2.0-S2352648324000734-mmc3.pdf

Description: The SCARF introduces a framework for how to evaluate symptom-assessment applications. Additionally, it provides a reporting checklist to help standardize reported information to make evaluation and benchmarking studies more transparent and comparable.

Download:

Journal Publication: https://doi.org/10.2196/76168

SCARF Checklist: https://jmir.org/api/download?alt_name=humanfactors_v13i1e76168_app1.docx&filename=a2276b79-f324-11f0-8ba3-dbf6c25344a9.docx

Description: The Trust in AI Generated Health Advice Scale (TAIGHA) and Short Scale (TAIGHA-S) are two validated questionnaires to measure state trust in health advice generated by AI applications using two dimensions: (1) trust and (2) distrust. We specifically developed a short version, the TAIGHA-S, for circumstances with limited time.

Download:

Journal Publication: https://arxiv.org/abs/2512.14278