Suggested format for university assessment in undergraduate competency based curriculum in otorhinolaryngology

Sir,

The competency based medical education emphasizes role of formative and summative assessments in academic progression of learners. The university examination represents summative assessment in which distribution of marks becomes important for overall evaluation of students’ performance in examinations. The competency based curriculum (CBC) began with 2019-20 batch that has already completed phase-I training and is studying in phase-II presently. The phase-III, part-1 training for this batch would begin in February 2022.¹ The details of otorhinolaryngology (ORL) curriculum have already been published by erstwhile Medical Council of India (MCI).² There shall be formative and summative assessment examinations during the 12 months period (duration of phase-III, part-1 has been reduced from 13 months to 12 months as per the advisory from National Medical Commission).¹ The suggested break-up of the internal assessment (IA) marks in ORL is published earlier.³ Here, we further suggest scheme of theory paper and clinical examination, and division of university assessment (UA) marks in ORL. The CBC document has assigned total 200 marks in ORL i.e. 100 marks for theory and 100 marks for practical/clinical examination including viva. The IA marks will not contribute to the university marks, but will be displayed as a separate head in the marksheet.

Theory examination (UA)

There will be one theory paper of 100 marks. It would contain multiple choice questions (MCQs), not exceeding 20% of marks, structured long answer questions (LAQs), short answer questions (SAQs) (example 2), and at least one compulsory question on attitude, ethics and communication (AETCOM).⁴ The MCQs should be scenario based (example 3) and ‘all of above’ and ‘none of above’ should be avoided as options. Tables 1 and 2 show two different suggested formats for the structure of theory paper in ORL. There are more questions in format-2, which will allow more coverage of the syllabus. We suggest that the paper should be divided into 3 sections. Section A would contain 20 item-MCQ, section B will have LAQs/ SAQs and section C will be like section B but with a question on AETCOM (example 4). The LAQs should be structured essay type of questions based on a case or a problem (example 1). The questions should be framed in such a way that these assess the clinical reasoning and knowledge application skills of the students. We suggest that the total duration of the paper should be three hours.

Example 1: Structured LAQ (problem based)

A 35 years old female attended ear, nose and throat outpatient department (ENT OPD) with complaints of gradually diminishing hearing in both ears for last 2 years which has aggravated during her recent pregnancy. What is your most probable diagnosis? What is the differential diagnosis? What will be the expected findings on pure tone audiogram and tympanogram? How will you manage this patient? (1+3+2+4=10 marks).

Example 2: SAQ

What are the risk factors for congenital deafness? How will you screen a new born for deafness? (2+3=5 marks).

Example 3: MCQ item

A two years old child is very slow in developing language. On otoscopic examination bilateral tympanic membrane is dull in appearance. Next investigation of choice is: (a) pure tone audiogram (b) tympanogram (c) X-ray mastoids (d) nystagmogram.

Example 4: AETCOM

What does confidentiality entail? What are the instances in ENT practice in which confidentiality of patient information may be breached? (2+3=5 marks).

The number and type of questions to be asked from a particular topic and the allotted marks is calculated by blueprinting as done with format-1 in the present work.⁵ For blueprinting we assign the impact score and frequency score to each content area and calculate the weightage to be given. The impact score (I) is based on the importance of topic. Impact scores 1, 2 and 3 are given for ‘nice to know’, ‘desirable to know’ and ‘must know’ content areas respectively. Frequency score (F) is based on frequency of asking a particular question. Frequency scores 1, 2 and 3 are given to ‘less’, ‘moderate’ and ‘frequently’ asked questions respectively. Weightage of each content area (W) is calculated by (I×F)/T, where T is total summation of all (I×F). Hence, by blueprinting we can decide the number of questions to be asked from a particular topic depending on the importance of the topic and also distribute marks accordingly (Table 3). Here we have demonstrated the blueprinting for format-1 only.

Practical/clinical examination (UA)

The university practical examination in ORL will be of 100 marks and will include viva also. Table 4 shows suggested break up of 100 marks. The weightage of viva marks is to be decided by the institutions/universities and hence would vary between institutes and subjects. We feel that 20 marks (out of 100) are adequate for viva. As seen in Table 4, approximately 45 minutes will be required to assess each student, though this may vary from one institute to other. It will be feasible as the number of examiners is now six (3 internals and 3 externals) for assessing 150 students.

*It is important to device subject specific AETCOM test items/OSCE

It is essential to assess the AETCOM skills of students during clinical/practical examination. This can be done through objective structured clinical examination (OSCE) station with a checklist (e.g. a skill station in which the student may administer informed consent to a standardized patient).⁶ A similar scheme of theory and practical assessment can be followed in send off or preliminary examination so that students get prior exposure to pattern of university examination. However, formative and internal assessments may have different patterns depending on the coverage of course and time available. Nevertheless, the principles would remain same.

REFERENCES

1. 1. Advisory for reopening the medical colleges. Available at: https://www.nmc.org.in/MCIRest/open/getDocument?path=/Documents/Public/Portal/LatestNews/Advisory-external%20Examiner-04.08.2020%20.pdf. Accessed on 14 February 2021.
2. 2. Medical Council of India. Competency based Undergraduate curriculum for the Indian Medical Graduate. 2018;3:89-101.
3. 3. Khilnani AK, Khilnani G, Thaddanee R. Understanding and analyzing competency based undergraduate curriculum in otorhinolaryngology for its effective implementation. Int J Otorhinolaryngol Head Neck Surg. 2021;7:409-16.
4. 4. AETCOM Module. Available at: AETCOM_book.pdf (nmc.org.in). Accessed on 13 February 2021.
5. 5. Gujarathi AP, Dhakne-Palwe S, Patil RN, Borade-Gedam P, Mahale MS, Gosavi SV, et al. Preparation of Blue Prints for Formative Theory Assessment of Undergraduate Medical Students in Community Medicine. MVP J Med Sci. 2015;2(2):100-3.
6. 6. Assessment Module for undergraduate medical education 2019. Available at: https://www.nmc.org.in/wp-content/uploads/2020/01/Module_Competence_based_02.09.2019.pdf. Accessed on 14 February 2021.