Objective structured practical examination as a formative assessment tool compared to traditional method for third year M.B.B.S. community medicine students

INTRODUCTION

Evaluation is considered as an important step in the medical curriculum to assess the clinical and practical competencies of students. But the system of evaluation in medicine needs improvement from subjective methods to objective ones. The Medical Council of India emphasizes the importance of accurately assessing the competency of medical students.¹

Historically, it has originated from medical education, and is now being adopted by many disciplines of health care education.² As a faculty of my institution, I was exposed to the Medical Education Technologies as a part of Medical Council of India workshop. During the workshop, I was introduced to the concept of objective structured practical examination (OSPE), which inspired me to conduct this study on students.³

The OSPE is now an accepted tool in the assessment of practical skills in both pre- and para-clinical subjects.⁴ The objective of OSPE is to reduce the chances of examiner bias and assess the students skills. Though this concept was introduced by Ronald Harden more than thirty five years ago; still it is not implemented in many of the medical colleges in India.²

The objective structured clinical examination (OSCE)/OSPE is a method of objectively testing the clinical and practical knowledge and skills acquired during the medical curriculum. This method was standardized by the All India Institute of Medical Sciences.⁵

The present study is an attempt to evaluate the effectiveness of OSPE as a formative assessment tool compared to traditional methods for M.B.B.S. students and to know the perceptions of teachers and students regarding OSCE/OSPE as a tool for assessment and to know how it can be made more effective.

Objectives

The objectives of this study was to evaluate the effectiveness of OSPE as a formative assessment tool as compared to traditional methods for M.B.B.S. students and to know the perception of participating students towards OSPE.

METHODS

This cross-sectional comparative study was conducted at Department of Community Medicine, KAMSRC, LB Nagar, Hyderabad, Telangana, India, for a period of 2 months from March to April 2017. In the first week of postings the topics for the viva voce examination and OSPE was announced. During second week, the viva voce examination was conducted (Table 1).

During third week the OSPE was conducted among the students who have attended the viva voce. Students belonging to 6th Semester were included in the study. Out of the 6 batches attending clinical posting in community medicine, 2 batches were selected randomly by lottery method. All the five faculties who were trained in medical education technology conducted the study. All the participating faculty and students were sensitized regarding traditional method of examination and OSPE.

Inclusion criteria

Students of 6^th semester who were willing to participate in the study.

Exclusion criteria

Students who were not willing to participate in the study.

Ethical clearance

From the institutional ethical committee.

Data collection

In this study a total of 64 students gave consent and 56 participated in the examination, 8 did not attend. The entire 56 MBBS students first underwent a viva voce examination. The same 5 trained faculties who were involved in viva voce took OSPE. After undergoing traditional method of examination, all the participating students underwent OSPE and the marks were recorded. A model viva voce question bank was prepared and questions were asked from the same. The question bank did not have questions asked in OSPE, other questions revolved around the theme used in OSPE. Students were given 5 minutes at each station and each station had a structured question which had to be answered by them. At the end of every 5 minutes, the participant moved to the successive station till completion of all 08 stations which include 2 rest stations after station 2 and 5.Thus the evaluation is done for a total of 30 marks. All participants were given the same set of questions.

The total marks obtained out of 30 marks in traditional practical examination method (viva voce) were compared with marks obtained in OSPE. The students were given a pre-validated feedback questionnaire before leaving out. All the data collected was entered and analyzed with MS excel software 2007. Analysis was done using paired t-test. The p<0.05 considered significant.

RESULTS

In this study a total of 64 students gave consent and 56 participated in both the rounds of traditional method of assessment (viva voce) and OSPE. The mean marks obtained by students by traditional methods were 15.00±3.49 as compared to score in OSPE was 17.78±2.69, which was statistically significant (p<0.05) (Table 2).

In this study, 76% of the students agreed that OSPE is fairer in comparison to traditional practical examination. 71% students opined that OSPE covers wide range of knowledge and 84% students agreed that variability of examiner can be removed to a large extent by OSPE. 91% students agreed that OSPE is easier to pass and score better when compared to traditional practical method. 80% of the students opined that OSPE is a lengthy process. 86% students felt that OSPE is a better form of assessment and learning. 94% felt that it covers all the domains of assessment. Only 38% students felt that viva voce should be continued in assessment process. 88% students felt that attitude of examiners during OSPE was better than traditional practical examination. 86% students had an opinion that the emotional stress in OSPE was less as compared to traditional practical examination (Table 3).

DISCUSSION

In the present study, Students scored higher marks in OSPE (21.06±1.06) than in traditional method(15.0±3.49) of examination, this similarity was found in a study done by Trivedi et al and Nigam R et al, who concluded that by using OSPE as an assessment tool, the students get a chance to score better.^6,7

This study was also aimed to assess the student’s perception towards OSPE. For this, students feedback was taken after OSPE, 76% of the students agreed that OSPE is fairer in comparison to traditional practical examination. In a study conducted by Relwani where more than 70% students strongly agreed that OSPE is fairer method which is similar to present study.⁸ As shown in Table 3, 71% students opined that OSPE covers wide range of knowledge. Present finding was similar to study conducted by Wadde et al where 54.5% students agreed that OSPE covered a wide range of knowledge when compared to TPE.²

In this study, 84% students agreed that variability of examiner can be removed to a large extent by OSPE, this similarity was found in a study done by Relwani, where 90.5% students strongly agreed that examiner bias can be removed to a large extent by OSPE.⁸

In the present study, 91% students agreed that OSPE is easier to pass and score better when compared to traditional practical method. In a study done by Shivani et al where 62.7% students agreed that OSPE is easier to pass and score better which is similar to present study.⁹

In this study, 80% of the students opined that OSPE is a lengthy process, this similarity is observed in a study done by Relwani et al where 71.9% agreed that OSPE is a lengthy process compared to TPE.⁸

In the present study, 94% students felt that it covers all the domains of assessment, this similarity was found in a study conducted by Shaifali et al where they conclude that OSPE proved to be a more reliable tool for assessment of student’s cognitive, psychomotor and affective domains.¹⁰

In this study, only 38% students felt that viva voce should be continued in assessment process, this similarity was found in a study done by Relwani et al where only 10.5% students agreed that viva voce should be continued in assessment process.⁸

In the present study, 88% students felt that attitude of examiners during OSPE was better than traditional practical examination, similar results were found in a study done by Wadde et al.²

In this study, 86% students had a opinion that the emotional stress in OSPE was less as compared to traditional practical examination. In a study done by Rao et al, where 74% students agreed that OSPE was less stress full compared to TPE which is similar to present study.¹¹

CONCLUSION

Based on the observation, Students scored higher marks in OSPE (21.06±1.06) than in traditional method (15.0±3.49) of examination. Majority of the students agreed that OSPE is fairer in comparison to traditional practical examination and opined that OSPE covers wide range of knowledge and they agreed that variability of examiner can be removed to a large extent by OSPE and it is easier to pass and score better when compared to traditional practical method. Majority of the students felt that OSPE is a better form of assessment and learning, covers all the domains of assessment and attitude of examiners during OSPE was better than traditional practical examination. The students had an opinion that the emotional stress in OSPE was less as compared to traditional practical examination and it was found to be more effective assessment tool than traditional method.

ACKNOWLEDGEMENTS

Authors would like to thank the faculty members of Department of Community Medicine and students of 6th semester for their active participation and co-operation in the study.

Funding: No funding sources

Conflict of interest: None declared

Ethical approval: The study was approved by the Institutional Ethics Committee

REFERENCES

1. 1. Medical Council of India. Regulation of graduate medical education, 2012. Available at: http://www.mciindia.org. Accessed on 9 April 2017.
2. 2. Wadde SK, Deshpande RH, Madole MB, Pathan FJ. Assessment of III MBBS students using OSPE/OSCE in community medicine: Teachers and students perceptions. Sch J App Med Sci S. 2013;1:348-53.
3. 3. Ananthakrishnan N. Objective structured clinical/practical examination (OSCE/OSPE). J Postgrad Med 1993;39(2):82-4.
4. 4. Kundu D. Objective structured practical examination in biochemistry: An experience in Medical College, Kolkata. J Nat Sci Biol Med. 2013;4(1):103-7.
5. 5. Radhika G, Dara AK, Varalaxmi KP, Bhavani C. Perceptions of the introduction of objective structured practical examination (OSPE)/objective structured clinical examination (OSCE): a pilot study carried out in Government Medical College, Ananthapuramu, Andhra Pradesh, India. J NTR Univ Health Sci. 2015;4:145-9.
6. 6. Trivedi RS, Diwan JS, Shah CJ, Jani RD, Anand AK. The influence of objectively structured practical examination (OSPE) on scoring pattern. Int J Biomed Adv Res. 2014;5:87-9.
7. 7. Nigam R, Mahawar P. Critical analysis of performance of M.B.B.S. students using OSPE &TDPE-a comparative study. National J Community Med. 2011;2(3):322-4.
8. 8. Relwani NR, Wadke RA, Anjenaya S, Sawardekar PN. Effectiveness of objective structured practical examination as a formative assessment tool as compared to traditional method for M.B.B.S. students. Int J Community Med Public Health. 2016;3:3526-32.
9. 9. Jaswal S, Jugesh C, Kaur J, Gupta S, Singh T. Assessment for learning with Objectively Structured Practical Examination in Biochemistry. Int J Bas App Med Res. 2015;5(4):784-90.
10. 10. Shaifali I, Ahsan M, Mallick A. A study on objective structured practical examination (OSPE) as a tool for assessment of medical students. Indian J Bas App Med Res. 2016;5(2):784-90.
11. 11. Rao RD, Babu PU, Chakravarthy KCV, Ramya N. Objective structured practical examination (OSPE) as a tool in formative assessment of II MBBS students, in pathology. Int J Res Med Sci 2018;6:221-4.