Keywords
Artificial intelligence
Medical education
Semiology teaching
Technology in medicine
Large language models
How to Cite
Copyright © | All rights reserved.
The journal holds the exclusive copyright for the publication of this article under the terms of Brazilian law 9610/98.
Partial reproduction
The use of parts of the texts, figures and questionnaire of the article is free, being mandatory the citation of the authors and journal.
Total reproduction
It is expressly prohibited and must be authorized by the journal.
Abstract
Objective: To evaluate the reliability and performance of current artificial intelligence (AI) tools that use large language models to describe signs and symptoms in medical semiology and compare them with each other. Methods: Common descriptors in medical practice and medical semiology teaching were selected, covering the main body systems such as the cardiovascular, respiratory, digestive, and neurological systems. Free and widely used language models such as Chat GPT (GPT-4), Gemini 1.5, and Microsoft Copilot were used for queries, using the command "Define [term]." Responses were assessed as non-existent, incomplete, or complete, comparing them to specialized literature. Questionable cases were reviewed by a third observer. Results: Of the 300 queries performed, 56.4% were considered complete. The respiratory system showed the highest accuracy (87.5%). The artificial intelligence virtual assistant Copilot stood out with 60% complete responses. Conclusion: AI has demonstrated potential in learning medical semiology, although it needs improvements for greater accuracy.
References
AMISHA M, et al. Overview of artificial intelligence in medicine. J Family Med Prim Care, 2019; 8: 2328-31.
ALZAABI A, MASTERS K. Assessing medical students' readiness for artificial intelligence after pre-clinical training. BMC Med Educ, 2025; 25(1): 824.
ARMSTRONG P. Bloom’s taxonomy. In: Vanderbilt University Center for Teaching (Ed.). A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom’s Taxonomy of Educational Objectives, 2010.
ARORA A. The promise of large language models in health care. The Lancet, 2023; 401(10377): 641.
AYUMBA EM. Medical Semiology Teaching Based on Intelligent eLearning. Stud Health Technol Inform, 2023; 302: 482-483.
BAJWA J, et al. Artificial intelligence in healthcare: transforming the practice of medicine. Future Healthcare Journal, 2021; 8(2): e188–94.
BEARMAN M, et al. Discourses of artificial intelligence in higher education: a critical literature review. High Educ, 2022.
BRASIL. Presidência da República. Lei Geral de Proteção de Dados Pessoais (LGPD). Lei nº 13.709, de 14 de agosto de 2018. Disponível em: https://www.planalto.gov.br/ccivil_03/_ato2015-2018/2018/lei/l13709.html. Acessado em: 28/03/2024.
DA VINHA RICIERI D da V, et al. Erros comuns de docentes sem letramento em Inteligência Artificial: uma revisão integrativa para o ensino superior. Peer Review, 2024; 6(7): 284–300.
DA VINHA RICIERI D, et al. Roteiro de comandos de ajuste fino para ChatGPT. SCIAS - Educação, Comunicação e Tecnologia, 2024; 6(1): 107–138.
DAVENPORT T, KALAKOTA R. The potential for artificial intelligence in healthcare. Future Healthcare Journal, 2019; 6(2): 94-98.
DENNICK R. Constructivism: reflections on twenty five years teaching the constructivist approach in medical education. Int J Med Educ. 2016 Jun 25; 7:200-5.
EMANUEL EJ, WACHTER RM. Artificial Intelligence in Health Care: Will the Value Match the Hype? JAMA, 2019; 321(23): 2281–2282.
GOH PS, SANDARS J. A vision of the use of technology in medical education after the COVID-19 pandemic. MedEdPublish, 2020; 9: 49.
GORDON M, et al. A scoping review of artificial intelligence in medical education: BEME Guide No. 84. Medical Teacher, 2024; 46(4): 446-470.
GRIPAY B, et al. Benefits of semiology taught using near-peer tutoring are sustainable. BMC Med Educ, 2022; 22: 26.
GUAN J. Artificial intelligence in healthcare and medicine: promises, ethical challenges and governance. Chinese Medical Sciences Journal, 2019; 34(2): 76-83.
GUERRERO-ROLDÁN, et al. Experiences in the use of an adaptive intelligent system to enhance online learners' performance: a case study in Economics and Business courses. Int J Educ Technol High Educ 18, 36 (2021).
HAMET P, TREMBLAY J. Artificial intelligence in medicine. Metabolism, 2017; 69: S36-S40.
HE J, et al. The practical implementation of artificial intelligence technologies in medicine. Nature Medicine, 2019; 25(1): 30-36.
HEN J, et al. How Many Demonstrations Do You Need for In-context Learning? arXiv, 2023; n. 2303.08119v3: 1-11.
HORTON R. Offline: No dark sarcasm in the classroom. The Lancet, 2022; 400: 1666.
HUMBLE N, MOZELIUS P. The threat, hype, and promise of artificial intelligence in education. Discover Artificial Intelligence, 2022; 2(1): 22.
JIANG F, et al. Artificial intelligence in healthcare: past, present and future. Stroke and Vascular Neurology, 2017; 2(4): 230-243.
KHOR W, NGIAM KY. Big data and machine learning algorithms for health-care delivery. The Lancet Oncology, 2019; 20(5): e262-273.
KOLACHALAMA VB, GARG PS. Machine learning and medical education. NPJ Digital Medicine, 2018; 1(1): 54.
LASO GUZMÁN FJ. Relevance of medical semiology in the technological era. Med Clin (Barc), 2017; 148(12): 559-561.
LGPD: a Lei Geral de Proteção de Dados Pessoais e a atuação do profissional de medicina / Conselho Federal de Medicina. Brasília: CFM, 2022. Disponível em: https://www.flip3d.com.br/pub/cfm/index9/?numero=38&edicao=5305#page/1. Acessado em: 28/03/2024.
MCCOY LG, et al. What do medical students actually need to know about artificial intelligence? NPJ Digital Medicine, 2020; 3(1): 86.
NGIAM KY, KHOR W. Big data and machine learning algorithms for health-care delivery. The Lancet Oncology, 2019; 20(5): e262-273.
PARANJAPE K, et al. Introducing artificial intelligence training in medical education. JMIR Medical Education, 2019; 5(2): e16048.
PATEL VL, et al. The coming of age of artificial intelligence in medicine. Artificial Intelligence in Medicine, 2009; 46(1): 5-17.
Porto, C. Semiologia Médica. 8. ed. Rio de Janeiro: Guanabara Koogan, 2019.
PSYCHÉ V, RUER P. L’apprentissage adaptatif intelligent. Revue Le Tableau, 2019; 8(4).
RAJPURKAR P, et al. "AI in medical imaging: from prediction to diagnosis". Nature Medicine, 2022; 28: 31–38.
RUIZ JG, et al. The impact of e-learning in medical education. Academic Medicine, 2006; 81(3): 207-212.
STEINHUBL S, TOPOL EJ. Digital medicine, on its way to being just plain medicine. NPJ Digital Medicine, 2018; 1(1): 20175.
SARWAR H, et al. Self-Reported Effectiveness of e-Learning Classes during COVID-19 Pandemic: A Nation-Wide Survey of Pakistani Undergraduate Dentistry Students. Eur J Dent. 2020 Dec; 14(S 01): S34-S43.
TANG KS, et al. Augmented reality in medical education: a systematic review. Canadian Medical Education Journal, 2020; 11(1): e81.
TEIXEIRA JAC. Semiologia médica: principais epônimos, sintomas, manobras e sinais. 1 ed. Rio de Janeiro. Editora UFF, 2024; 506 p.
TOPOL EJ. High-performance medicine: the convergence of human and artificial intelligence. Nature Medicine, 2019; 25(1): 44-56.
TRIANTAFYLLOU SA. Constructivist learning environments. In: Proceedings of the 5th International Conference on Advanced Research in Teaching and Education, 2022.
WHO. Role of primary care in the COVID-19 response. World Health Organization, 2021.
YU KH, et al. Artificial intelligence in healthcare. Nature Biomedical Engineering, 2018; 2(10): 719-731.