Keywords
Academic performance
Medical school
How to Cite
Copyright © | All rights reserved.
The journal holds the exclusive copyright for the publication of this article under the terms of Brazilian law 9610/98.
Partial reproduction
The use of parts of the texts, figures and questionnaire of the article is free, being mandatory the citation of the authors and journal.
Total reproduction
It is expressly prohibited and must be authorized by the journal.
Abstract
Objective: To compare the performance of medical students in open-book tests composed of open-ended questions (OBTOEQ), describing variables related to planning, execution, test duration, response type, and the cognitive domain of the questions. Methods: This retrospective observational study compared student performance in OBTOEQ administered without supervision (2023.1) and with supervision and restricted access to information sources (2024.1) in a medical course at a private Brazilian institution. The following exploratory variables were considered: time available to complete the tests, type of response, and cognitive domain of the questions according to Bloom’s Taxonomy. Statistical analysis was performed using SPSS 26.0 and Excel 365, adopting a significance level of 5% (p ≤ 0.05). The Mann-Whitney test was used to compare the groups. Results: A total of 1,160 OBTOEQ-2023 and 1,146 OBTOEQ-2024 were analyzed. Most supervised tests showed lower mean scores (p < 0.05), despite the longer test duration and the predominance of lower cognitive domains. Long-answer responses predominated in the supervised OBTOEQ. Conclusion: Differences in performance were observed between the assessment contexts, indicating that the application conditions influence the interpretation of results and the reliability of students’ individual performance.
References
ABUZIED AIH, NABAG WO. Structured viva validity, reliability, and acceptability as an assessment tool in health professions education: a systematic review and meta-analysis. BMC Med Educ., 2023; 23(1): 531.
AHMED Y, et al. Evolving assessment in medical education: exploring the role of open-book examinations. J Adv Med Educ Prof., 2024; 12(4): 215-25.
ALGUACIL M, et al. Academic dishonesty and monitoring in online exams: a randomized field experiment. J Comput High Educ., 2024; 36: 835–51.
BEARMAN M, et al. Ensuring academic integrity and assessment security with redesigned online delivery. Melbourne: Deakin University; 2020. Disponível em: http://dteach.deakin.edu.au/2020/03/23/academic-integrity-online/. Acessado em: 6 de junho de 2026.
BLADT F, et al. Medical students' perception of changes in assessments implemented during the COVID-19 pandemic. BMC Med Educ., 2022; 22(1): 844.
BULJAN I, et al. Cognitive levels in testing knowledge in evidence-based medicine: a cross-sectional study. BMC Med Educ., 2021; 21(1): 25.
CRUESS RL, et al. Amending Miller’s Pyramid to include professional identity formation. Acad Med., 2016; 91(2): 180–5.
DAVIES DJ, et al. Assessment of factual recall and higher-order cognitive domains in an open-book medical school examination. Adv Health Sci Educ Theory Pract., 2022; 27(1):147-65.
DAWSON P, et al. Beyond open book versus closed book: a taxonomy of restrictions in online examinations. Assess Eval High Edu., 2024; 49(2): 262–74.
DENEEN C. Assessment considerations in moving from closed-book to open-book exams. Melbourne: Melbourne Centre for the Study of Higher Education, The University of Melbourne; 2020 May. Disponível em: https://melbourne-cshe.unimelb.edu.au/__data/assets/pdf_file/0010/3341944/closed-book-to-open-book-exam_final.pdf. Acessado em: 6 de junho de 2026.
DURNING SJ, et al. Comparing open-book and closed-book examinations: a systematic review. Acad Med., 2016; 91(4): 583-99.
ELDAKHAKHNY B, ELSAMANOUDY AZ. Discrimination power of short essay questions versus multiple choice questions as an assessment tool in clinical biochemistry. Cureus., 2023; 15(2): e35427.
ELSALEM L, et al. Remote E-exams during COVID-19 pandemic: a cross-sectional study of students' preferences and academic dishonesty in faculties of medical sciences. Ann Med Surg., 2021; 62: 326-33.
ELSAMANOUDY A, et al. Evaluation of modified essay questions (MEQs) as an assessment tool in third-year medical students’ modular summative assessment. BMC Med Educ., 2024; 24:1445.
ELSHAMA SS. How to use and apply assessment tools in medical education? Iberoam J Med., 2020; 2(4): 351-9.
ER HM, et al. Remote online open book examinations: through the lenses of faculty and students in health professions programmes. BMC Med Educ., 2023; 23(1): 397.
FERRAZ AP, BELHOT RV. Taxonomia de Bloom: revisão teórica e apresentação das adequações do instrumento para definição de objetivos instrucionais. Gest Prod., 2010; 17(2): 421-31.
GAMAGE KA, et al. Online delivery and assessment during COVID-19: safeguarding academic integrity. Educ Sci., 2020; 10(11): 301.
GARG M, GAEL A. A systematic literature review on online assessment security: Current challenges and integrity strategies. Comput Secur., 2022; 113: 102544.
GUANGUL FM, et al. Challenges of remote assessment in higher education in the context of COVID-19: a case study of Middle East College. Educ Assess Eval Acc., 2020; 32(4): 519-35.
HOLDEN OL, et al. Academic integrity in online assessment: a research review. Front Educ., 202; 6: 639814.
JERVIS CG, BROWN LR. The prospects of sitting ‘end of year’ open book exams in the light of COVID-19: a medical student’s perspective. Med Teach., 2020; 42(7): 830–31.
Mak-VAN DER VOSSEN MC, et al. A road map for attending to medical students' professionalism lapses. Acad Med., 2019; 94(4): 570-8.
MATHIESON G, et al. Could the future of medical school examinations be open-book: a medical student's perspective? Med Educ Online., 2020; 25(1): 1787308.
MEE J, et al. An experimental comparison of multiple-choice and short-answer questions on a high-stakes test for medical students. Adv Health Sci Educ Theory Pract., 2024; 29(3):783-801.
MILLER GE. The assessment of clinical skills/competence/performance. Acad Med.,1990 Sep; 65(9): S63-7.
MUKURUNGE E, et al. Assessment approaches in undergraduate health professions education: towards the development of feasible assessment approaches for low-resource settings. BMC Med Educ., 2024; 24(1): 318.
NEWTON, PM, ESSEX, K. How common is cheating in online exams and did it increase during the COVID-19 pandemic? A systematic review. J Acad Ethics., 2024; 22:323–43.
NG CK. Evaluation of academic integrity of online open book assessments implemented in an undergraduate medical radiation science course during COVID-19 pandemic. J Med Imaging Radiat Sci., 2020; 51(4): 610-16.
PANÚNCIO-PINTO MP, TRONCON LEA. Avaliação do estudante: aspectos gerais. Medicina (Ribeirão Preto)., 2014; 47(3): 314-23.
PIENAAR L, et al. Health professions education and unprofessional behaviour in the global south: a scoping review of conceptions, theoretical frameworks, and prevalence. BMC Med Educ., 2025; 25(1): 1733.
RADEMAKERS J, et al. Progress testing with short answer questions. Med Teach., 2005; 27(7): 578-82.
SABRINA F, et al. Ensuring academic integrity in online assessments: a literature review and recommendations. Int J Inf Educ Technol., 2022; 12(1): 60-70.
SARKAR S, et al. Online open-book examination of undergraduate medical students: a pilot study of a novel assessment method used during the coronavirus disease 2019 pandemic. J Laryngol Otol., 2021;135(4): 288-92.
SEFER CC, et al. Avaliação do desempenho de estudantes de Medicina em Neurologia utilizando metodologias ativas. REAS [Internet]. 2019[citado 14 mar 2026]; 11(9):e293.
SUSNJAK T, MCINTOSH TR. ChatGPT: the end of online exam integrity? Educ Sci., 2024; 14(6): 656.
TUAH NA, NAING L. Is online assessment in higher education institutions during COVID-19 pandemic reliable? Siriraj Med J., 2020; 73(1): 61–8.
VAZQUEZ JJ, et al. Can we stay one step ahead of cheaters? A field experiment in proctoring online open book exams. J Behav Exp Econ., 2021;90: 101653.