•  
  •  
 

DOI

https://doi.org/10.1016/j.jds.2025.06.005

First Page

198

Last Page

205

Abstract

Background/purpose The importance of oral health is globally recognized, which has increased the demand for qualified dental hygienists. This study assessed the performance of multimodal large language models (LLMs), on the Japanese National Examination for Dental Hygienists, focusing on their ability to answer visually-based questions and evaluating image-recognition capabilities. Materials and methods The 34th Japanese National Examination for Dental Hygienists (March 2025) supplied 213 multiple-choice questions (74 text-only, 139 visually-based). Five multimodal LLMs were tested: OpenAI o3-mini-high (o3-mh), ChatGPT-4.5 Preview (GPT-4.5), Gemini 2.0 Flash Thinking Experimental (Gemini 2.0), Gemini 2.5 Pro Experimental (Gemini 2.5), and Claude 3.7 Sonnet (Claude 3.7). Performance was evaluated by comparing LLM answers to official correct answers. Cochran's Q test and McNemar's tests with Bonferroni correction were used for statistical analysis. Results Gemini 2.5 achieved the highest overall correct response rate (85.0 %), followed by Claude 3.7 (77.5 %), o3-mh (77.0 %), GPT-4.5 (76.1 %), and Gemini 2.0 (75.1 %). For text-only questions, Claude 3.7 (91.9 %) performed best. On visually-based questions, Gemini 2.5 was superior (82.0 %), while other models scored around 70–73 %. Gemini 2.5 significantly outperformed, GPT-4.5 and Gemini 2.0 overall, and GPT-4.5 and Claude 3.7 on visually-based questions. Conclusion Multimodal LLMs, particularly Gemini 2.5, demonstrate significant proficiency on the Japanese National Examination for Dental Hygienists, including questions with visual elements. These findings suggest a growing potential for LLMs as educational tools in dental hygiene. However, current limitations in accuracy and reliability necessitate further refinement and cautious integration into educational and clinical settings.

Publication Date

1-1-2026

Download

Share

COinS