Publication:

Are we assessing them right? A study on the content validity of undergraduate pathology question papers and role of blueprinting in assessment

Background Assessment forms a very essential and critical part of the learning process in medical education. Written examinations are widely used to test the knowledge component of learning. For a written examination to be valid, it should match the contents of the course and should provide proportional weightage to each of the content. A periodic evaluation of question papers to examine the weightage of each topic and the use of blue printing is necessary to ensure the validity of the written assessments.

Methods The ten year question papers (2007-2016) of II MBBS summative written examinations in Pathology conducted by Rajasthan University of Health Sciences (RUHS) were analysed and the appropriateness of weightage given to content areas was examined in relation to the syllabus.

Results The syllabus distribution in Paper 1 and Paper 2 was highly skewed with extreme concentration of topics in Paper 1 (due to clubbing of both general and systemic pathology) which led to under-representation of many topics despite their high Impact and Frequency. There was also evidence of paper setter’s bias leading to disproportionate (over/under) representation of many topics. An ideal blueprint for paper 1 and 2 was prepared with appropriate syllabus distribution and allotment of marks as per weightage of each topic according to its impact and frequency score.

Conclusions Frequent analysis should be carried out to provide feedback and to ensure that the assessments are aligned with the learning objectives. A blueprint is a vital component and helps us to plan written assessments in a rational and balanced manner.