Objective Structured Practical Examination: our experience in Pharmacology at Goa Medical College, Bambolim-Goa, India

INTRODUCTION

Practical examination is an important component of evaluation in the medical curriculum.¹ In conventional practical examination in pharmacology students are more involved in writing the answers like table work, prescription writing, prescription criticism, therapeutic problem and spots. Presently there are a number of newer Drug Delivery Systems (DDS) available for use in clinical practice. Whether students i.e. future doctors have really understood the method of using these devices may be difficult to judge in this conventional practical examination format. Explaining the procedure in detail to simulated patients by the students and observation of this by faculty enhances understanding of concepts better and probably improves logical reasoning.² The method of OSPE like the objective structured clinical examination (OSCE) tests in the students what they can do rather than what they know. According to Roy V et al this method is now believed to meet the deficiencies of the conventional system of practical examination.³

The term OSPE is derived from OSCE in 1975 which was later extended to practical examination and was modified by Harden and Gleeson.^4,5 According to Ravi P. Shankar et al, OSPE appears to be a reliable device which has a good capacity to differentiate between different categories of students. It also tests the mental attendance and the students’ attitude towards patients.⁶ Moreover, because OSPE questions are structured with marks to each point or instructions all the objectives of teaching-learning can be tested and each aspect can be assigned the desired weightage.⁷ It also helps to minimize or prevent patient and examiner variability thus improving the validity of the examination.^8,9 That is the reason why Gitanjali B is of the opinion that since OSPE system is more objective it can be a good substitute for the conventional method.¹⁰ In conventional practical examination, the assessment is made on the basis of global performance rather than the candidate’s individual competency. Some of the problems involved in conventional practical examination include patient and examiner variability significantly affecting the score.³

METHODS

This feedback form evaluation study was done in the Department of Pharmacology and Therapeutics, Goa Medical College, Bambolim-Goa, India. We have conducted part of the practical examination using OSPE system of evaluation for 146 students of II MBBS in subject of Pharmacology. All students were divided in batches of 25 each. We introduced a setup of five stations with same set of questions viz; dosage form and drug advice at each station. Every student had to spend two minutes each for dosage form and drug advice at their respective station.

An attempt was made to make the OSPE atmosphere as congenial as possible with the aim to familiarize the students with this new system of examination. All the students had to attempt same questions to avoid bias of evaluation. During the OSPE session students were not allowed to communicate with each other in order to prevent leaking out of questions. The OSPE questions were designed to test the cognitive aspect of learning mainly knowledge about the drugs and drug delivery device i.e. dosage form and instructions to be given to the patients. Questions were randomly changed each day and the entire schedule lasted for six consecutive days.

In case of dosage form students had to explain and demonstrate how to use the given dosage form (insulin syringe, metered dose inhaler, etc.) to the simulated patients. As far as drug advice is concerned students were expected to give instructions to the patient how to take drugs especially with relation to food (Appendix 1 and 2). During the OSPE, faculty functioned as observer, watching every student’s approach towards patient and their communication skills. Checklist was provided to all the observers with marks for each point/ instruction to be told by the student to the patients. Based on this the final score was calculated. At the end of OSPE examination we requested the students to fill the feedback forms to seek their opinion and suggestions to make OSPE more acceptable. We also requested faculty involved in OSPE evaluation to give their experience and suggestions to improve it further.

Table work, prescription writing, prescription criticism, therapeutic problem, spots and theory viva have been routinely practiced in pharmacology practical examination for many years in most medical colleges of our country. They are relatively subjective, unstructured, can have errors of bias, ambiguity and obsolescence. It is for this reason we introduced OSPE in pharmacology in our Goa Medical College. We conducted this study to test student’s actual knowledge which is essential to be told to the patients, to analyze their feedback on OSPE and to see the impact and experience of both students and the faculty with OSPE system of evaluation.

RESULTS

All the students (100%, n=146) were comfortable while answering the OSPE, while majority of students i.e. 93.15% (n=136) think OSPE is better than traditional viva or table work. Further, all the students believed that OSPE tests the practical skills and liked to have the same in future examination as the evaluation system (Table 1).

Out of 146 students who answered the OSPE, 43.83% (n=64) of students were satisfied with the setup whereas 28.77% (n=42) did not give any comments. 15.75% (n=23) wanted more revision as they faced some difficulty. 4.12 % (n=6) of students mentioned that, given a choice, they would prefer to choose dosage form and drug advice from the given set of questions, rather than observer choosing the questions for them during the OSPE. 3.42% (n=5) of students opined that marks allotted for OSPE are less and that we should emphasize for more focus on demonstration on dummies during lectures. Although majority of students could answer comfortably within allotted time, only 0.69% (n=1) of students found that time allotted for OSPE was insufficient (Table 2).

While analyzing the feedback by the faculty, we found that, all the faculty members (100%) were comfortable while listening to students and they accept that OSPE is better than viva or table work. In addition, since it tests practical skills better all the faculty believed that, the entire practical exam can be conducted in OSPE setup (Table 3).

Following OSPE, all the faculties were satisfied and did not recommend any change in present pattern, whereas 25% of faculty thinks this pattern will help the students to improve their clinical and communication skills. 50% of faculty proposed need of incorporating injection technique on dummies which will help the students to learn how to give injections better (Table 4).

DISCUSSION

In an attempt to improve the practical evaluation in the subject of pharmacology, OSPE was introduced along with the other conventional assessment pattern like table work, prescription writing, prescription criticism, therapeutic problems and spots for the II MBBS students. Assessing teaching-learning outcomes by the students in any subject is a complex task. It requires the evaluation of multiple domains: theoretical, practical, and clinical knowledge. In general, theoretical knowledge is tested by a written examination system constituted by Short Answer Questions (SAQ), Short Notes (SN) and/or Multiple Choice Questions (MCQs).

According to Yaqinuddin A et al, OSPE remains the most efficient tool to assess the practical knowledge in a system where basic knowledge is integrated with the clinical knowledge. However, this contention only holds true if the OSPE process revolves around structured objectives.¹¹ It is general belief that good students would do well in any form of evaluation,¹² but in OSPE setup, the student has to address the subject politely, make him comfortable, explain the procedure, etc. which is also keenly observed by the observer while evaluating students in OSPE. Further to evaluate application of clinical knowledge we had also used drug advice in our OSPE set up as used by Nayar U et al. in their study to evaluate the cognitive aspect of learning.⁷

In our study we found that, the entire faculty involved in conducting OSPE was in favor of having entire practical exam in OSPE setup. This feedback is highly significant and comparable to the findings by B. H. Verhoeven, et al, that OSPE if properly structured, along with a short written component can replace the current clinical/practical examination exercise taught in the preclinical years and it can also improve reliability.¹³According to Joshi A et al, enriching the learning environment through incorporation of a variety of teaching and learning strategies both in and out of classroom should yield enhanced learning. These findings can be considered significant as all the students in our study showed interest in having OSPE in future exam because they were comfortable while answering this new pattern of evaluation that we incorporated in practical evaluation system in our institute.¹⁴

As compared to a study by Yaqinuddin A et al, wherein 74% students agreed to have multiple modes of assessment to improve their knowledge and skill, in our study, we found all the students (100%) accepting OSPE because it helped them to improve their practical skills and application.¹¹ Similar findings have been reported in their research by R. G. Menezes et al, wherein they opined that that OSPE is an acceptable tool in Forensic Medicine at the undergraduate level.¹⁵ Besides these 50% of faculty suggested including injection techniques on dummies so that students are at ease when they start actual practice in clinical settings. According to N Ananthakrishnan and M. V. Natu et al, OSCE and OSPE have a uniform evaluation and normal distribution of grading.^9,16

In our study, 43.83% students liked the pattern of OSPE and disagreed for any change. All the students and faculty considered OSPE better than traditional viva or other conventional pattern of practical examination in their feedback. This findings match with the study by Menezes et al, and Nayar et al, wherein they found that OSPE has been accepted and appreciated by teachers as well as students as a distinct improvement when compared with earlier pattern of practical examination.^7,15 In our study only one students was unhappy with time allotted for OSPE. Yaqinuddin A et al, have reported similar findings in their study.¹¹

CONCLUSION

For any educational program to succeed, attention should be paid, not only to imparting knowledge but also to changing the attitudes of students and faculty towards implementing the programme. Whenever the conventional method of teaching is replaced by addition of practical classes, problem based learning or OSPE etc., we need to see that whatever we teach students in theory and practical has potential clinical application in the future.

It might be difficult to measure practical knowledge and applications by written practical examinations. In such situation OSPE can be structured and standardized which may prove effective to evaluate the clinical applications. Students were satisfied with the OSPE setup and majority of students thought OSPE is better than traditional viva or table work. Further, all the students and faculty believed that OSPE tests the practical skills and wished to have in future examination as part of evaluation.

ACKNOWLEDGEMENTS

We acknowledge all the students and faculty for giving their feedback for this particular study.

Appendix 2

Drug advice 1: A patient with type II diabetes mellitus FBSL 200 mg% and PPBSL 300 mg% is started on glimepiride, metformin and voglibose. Explain the patient regarding dose and schedule.

Drug advice 2: A 40 year old patient with hypothyroidism has to be prescribed l-thyroxine. Give appropriate advice.

Instructions Marks (2)

Start with a low dose of 50 µgm per ½ day

Tablet to be taken on empty stomach ¾ 30 minutes before meals

Monitor TSH levels 6-8 weeks after initiation of treatment. Adjust dose of ¾ thyroxine in increments of 25µgm until plasma TSH is normal

Total 2

Funding: No funding sources

Conflict of interest: None declared

Ethical approval: The study was approved by the institutional ethics committee, Goa Medical College

REFERENCES

1. 1. Roy V, Tekur U, Prabhu S. A comparative study of two evaluation techniques in pharmacology practicals: conventional practical examination versus objective structured practical examination. Indian J Pharmacol. 2004;36(6):385-9.
2. 2. Shivananda Kalludi, Ravindra Maradi, Murali Dhar, Raghavendra Babu, Manjunath Sathyanarayanachar. Introducing physiology practical demonstration in course curriculum for the first year nursing students. J Clin Diagn Res. 2013 Sep;7(9):1989-91.
3. 3. Hasan S, Malik S, Hamad A, Khan H, Bilal M. Conventional/traditional practical examination (CPE/TDPE) versus objective structured clinical evaluation (OSPE)/semi objective structured clinical evaluation (SOSPE). Pak J Physiol. 2009;5(1):58-64.
4. 4. Harden RM, Gleeson FA. Assessment of clinical competencies using an objective structured clinical examination (OSCE) In: ASME Medical Education Booklet no. 8. Dundee: ASME; 1979;64:123-125.
5. 5. Harden RM, Stevenson M, Wilson DW, Wilson GM. Assessment of clinical competencies using objective structured clinical examination. Br Med J. 1975;5955(1):447-51.
6. 6. Ravi P. Shankar, Arun K. Dubey, Pranaya Mishra, Vibhavri Y. Deshpande, T. S. Chandrasekhar, P. G. Shivanda. Student attitudes towards communication skills training in a medical college in Western Nepal. Educ Health Change Learn Pract. 2006;19(1):71-84.
7. 7. Nayar U, Malik SL, Bijlani RL. Objective structured practical examination: a new concept in assessment of laboratory exercises in preclinical sciences. Med Educ. 1986 May;20(3):204-9.
8. 8. Huseyin Cahit Taskiran. A new competency level system for practical and procedural skills in an undergraduate curriculum. Foundation for Advancement of International Medical Education and Research (FAIMER) Institute, 2003. Available at: http://www.faimer.org/education/fellows/abstracts/03taskiran.pdf. Accessed 30 June 2009.
9. 9. Ananthakrishnan N. Objective structured clinical/practical examination (OSCE/OSPE). JPGM. 1993;39(2):82-4.
10. 10. Gitanjali B. The other side of OSPE. Indian J Pharmacol. 2004;36:388-9.
11. 11. Yaqinuddin A, Zafar M, Ikram MF, Ganguly P. What is an objective structured practical examination in anatomy? Anat Sci Educ. 2013 Mar-Apr;6(2):125-33.
12. 12. Newble DI, Swanson DB. Psychometric characteristics of the objective structured clinical examination. Med Educ. 1988;22:325-34.
13. 13. Verhoeven BH, Hamers JGHC, Scherpbier AJJA, Hoogenboom RJI, Van der Vleuten CPM. The effect on reliability of adding a separate written assessment component to an objective structured clinical examination. Med Educ. 2000;34:525-9.
14. 14. Joshi A, Trivedi M. Innovations in pharmacology teaching. Int J Pharmaceut Biomed Res. 2010;1(2):62-4.
15. 15. Menezes Ritesh G, Nayak Vinod C, Binu VS, Kanchan Tanuj, Jagadish Rao P, Baral Prakash, et al. Objective structured practical examination in forensic medicine: students’ point of view. J Forensic Legal Med. 2011;18:347-9.
16. 16. Natu MV, Singh Tejinder. Objective structured practical examination (OSPE) in pharmacology-students’ point of view. Indian J Pharmacol. 1994;26:188-9.