Objective: To assess the effectiveness of high-fidelity simulation-based medical education (HF-SBME) in teaching and learning respiratory clinical examination in medical students.
Study Design: Quasi-experimental pilot study.
Place and Duration of Study: The Aga Khan University, Karachi, from November 2018 to January 2020.
Methodology: This study was conducted amongst third year medical students at the University. Students were assigned to intervention (IG) or control groups (CG). The IG underwent training for the respiratory clinical examination on a high-fidelity simulator mannequin, while the CG received the conventional practice session on standardised patients. Students were assessed on their respiratory clinical examination skills in five domains, and each domain was scored between 1-3 points (poor=1, fair=2, good=3) for a maximum composite score of 15. Feedback on use of SBME was also obtained from students.
Results: There were no statistically significant differences in demographics for the CG (n=41) and IG (n=40). Composite score for control and intervention groups was not significantly different (CG: 12.9 ± 1.89 vs. IG: 12.0 ± 2.35; p=0.067). However, a greater percentage of CG students were rated good in all five domains, with the difference being statistically significant for ability to correlate findings with clinical history (CG: 87.8% vs. IG: 67.5%; p=0.028).
Conclusion: Although medical students perceived HF-SBME as a beneficial teaching modality, it did not translate into improved performance. More research is required to determine the utility of HF-SBME in a developing country, like Pakistan.
Key Words: Simulation, High-fidelity, Medical education, Developing country, Clinical skills.
Simulation-based medical education (SBME) is well established in education systems of developed countries. High-fidelity simulators have been successfully used to teach and assess clinical and basic skills in the respiratory and cardiovascular systems.1-3
In developing countries, the lack of advanced facilities, expensive equipment and experienced operators represent significant barriers to the assimilation of SBME as a routine part of medical education.4
Pakistan, a developing country, relies primarily on didactic lectures, simulated (standardised) patients and small-group clinical sessions for clinical skills education in medical school.5 However, students report dissatisfaction with current methods of teaching clinical skills at the patient’s bedside.6 There have been limited studies exploring the effectiveness of SBME. One study used an intermediate-fidelity simulator to teach normal vaginal delivery; and reported significantly higher scores using SBME as compared to traditional methods.7
In Pakistan, there is mainly use of low- or intermediate-fidelity simulators; and therefore, a lack of research on the effectiveness of high-fidelity simulators for SBME. The Center for Innovation in Medical Education (CIME) at the Aga Khan University (AKU) is the first simulation centre in South-Asia to be accredited by the American-based Society for Simulation in Healthcare.
This study was designed to determine the effectiveness of high-fidelity SBME in teaching/learning respiratory clinical examination to third-year medical students of AKU, Karachi, and also to assess the attitude of medical students towards SBME.
A quasi-experimental pilot study was conducted among third-year medical students at the Aga Khan University (AKU). This study was a pilot project within this clerkship to offer respiratory clinical skills learning on a high-fidelity mannequin. It was conducted from November 2018 to January 2020 with approval from the Ethical Review Committee of the University.
The sample size was calculated for comparing mean performance scores between two groups (ratio of intervention group to control group 1:1). With an assumed mean difference of 2.5 points and standard deviation of 3, and at 95% power and 95% CI, the sample size was calculated to be 38 each for the control and intervention arm. The sample size was inflated to include minimum 40 participants in each arm.
The study participants were assigned in a 1:1 allocation ratio to either the control or the intervention group, based on the pre-assigned clinical rotation groups to evaluate the effectiveness of SBME. The clinical rotation groups are randomly formed by the curriculum office and not made on the basis of any socio-demographic or academic criteria. Thus, using existing clinical rotation groupings ensured random allocation to control and intervention groups while permitting logistical convenience.
The control group received the usual medical education, i.e. an interactive discussion session elaborating the theoretical basis and steps of clinical examination, common pathologic presentations of the respiratory system, correlation of clinical findings, and hands-on practice on standardised patients.
The students of the intervention group received the same training by the same faculty members as the control group and underwent an additional training session on a high-fidelity simulator mannequin to practice techniques of respiratory system clinical examination. This was a 30-minute session that included auscultation of normal and pathological findings (normal vesicular breathing; added lung sounds, wheezing, rhonchi and crepitations) that was followed by a discussion of clinical relevance to underlying disease. This simulation session was conducted by a senior instructor, an assistant professor, and associate professor of family medicine at AKU.
All the participants of the study were individually assessed by formative examination at the end of the family medicine clerkship. The students were given a brief clinical scenario and were required to perform the relevant clinical examination. Examiners included an associate professor of family medicine, an assistant professor of family medicine and a senior instructor at AKU. The examiners objectively assessed the students (via a checklist) in the domains of (a) organisation, (b) correctness of technique, (c) ability to identify abnormal findings, (d) ability to correlate findings with clinical history, and (e) a global rating. This global rating was an overall assessment of the students’ performance of the clinical examination. To eliminate bias, the same examiners assessed both the control and intervention groups, keeping examination conditions uniform as well, i.e. exam sequence and time allocation. Each domain was scored on a 3-point Likert scale (1=poor, 2=fair, 3=good). Equal weightage was given to all domains and a composite score was built by adding the individual scores of each domain (maximum score of 15).
Examiners also recorded socio-demographic characteristics of the students, such as age, gender, and high-school (A-Levels/Other) on the scoring sheet. The students of the intervention group completed a self-administered anonymous feedback form to assess attitude towards SBME (Adapted and modified from Joseph et al., India, 2016).8
Family medicine clerkship scores were then obtained for both groups. The clerkship score is a sum of student scores obtained during ongoing observation during the clerkship by faculty members and the performance in the end-of-rotation summative OSCE.
The performance in each domain was first compared by applying the Pearson Chi-square test. For each domain, the categories poor and fair were grouped into a single category. The mean composite scores of the formative examination (/15) of both groups were compared. Then family medicine clerkship scores of the intervention and control group were compared for significant differences by applying the independent sample t-test. A p-value of less than 0.05 was considered statistically significant. All analysis were performed using SPSS version 22.
A total of 81 students, 41 in control and 40 in the intervention group, were included in this study. Mean student age was 20.0 years (SD: 0.7), and 48 (59.3%) of students were males. Most students had studied in A-Levels (n=68; 84%). No statistical difference across socio-demographic characteristics of control and intervention groups was seen.
On comparison of mean composite score, there was no difference between the control and intervention group (control group: 12.9 ± 1.89 vs. Intervention Group: 12.0 ± 2.35; p = 0.067).
The percentage of students performing poorly across domains was low (A: 1.2%; B: 11.1%; C: 14.8%; D: 16.0%; and E: 3.7%), the categories into poor/fair were merged. A greater percentage of students in the control group were rated good in each of the five domains. The difference in percentages was statistically significant for domain D, i.e. the ability to correlate findings with clinical history (control: 87.8% vs. Intervention: 67.5%; p = 0.028, Figure 1).
Figure 1: Percentage of students rated good in each domain.Figure 2: Students attitudes towards SBME.
On comparison of the family medicine clerkship score, the control group obtained a significantly higher mean clerkship score (79.0% ± 4.15%) as compared to the intervention group (72.0% ± 5.93%) in the family medicine rotation (p <0.001).
Students’ attitude towards SBME was largely positive, with majority strongly agreeing/ agreeing that SBME was beneficial especially in increasing student confidence in clinical skills and providing a safe environment (Figure 2).
Although a regular feature in the developed world, SBME is only recently gaining interest in Pakistan, a developing country. Surprisingly, these results showed that the control group performed better than the intervention group in a demographically well-balanced cohort.
Although the findings of this study may seem counter-intuitive, they need not necessarily be taken at face value. The higher end-of-clerkship scores obtained by the control group indicate that this group may worked harder than the intervention group, as they perceived themselves as disadvantaged by missing out on the simulation session using the high-fidelity simulator mannequin.
This is not the first study to show less-than-favorable results for the use of high-fidelity simulation. Although several studies have shown the advantage high-fidelity simulation over low-fidelity simulation,9-11 increasing evidence shows that both are essentially comparable in terms of knowledge acquisition and skill.12-17 Indeed, medical students training with high-fidelity simulation may sometimes have significantly poorer performance than those training with low-fidelity simulation.18 One disadvantage of high-fidelity simulation is misplaced overconfidence in self-ability and skill.18,19 The overconfidence that accompanies training on high-fidelity simulators is something of a hazard, with possible negative impact on patient healthcare that overconfidence in abilities may have.18,20
Nevertheless, attitudes towards SBME amongst the students in the intervention group were generally positive and similar to those seen in the study by Joseph et al.8 Most student-perceived benefits of SBME lay in its capacity for improving clinical skills and increasing patient safety and comfort. The chief drawback perceived was deterioration of soft skills, such as empathy and communication skills. Around 57.5% students believed that SBME could reduce students’ communication skills, similar to a study by Joseph et al.8 Moreover, 45% felt that the use of SBME could reduce students’ empathy towards patients, also reported by Joseph et al.8
In a developing country, like Pakistan, cost is an important factor when considering incorporation of SBME into medical curricula. Therefore, judicious planning is required prior to acquisition and use of such technology.
There are a few limitations of this study. Being a single centre study, the findings of our study may not be generalisable to other medical institutions across Pakistan. Moreover, this study used only a respiratory system simulation mannequin, and thus findings may not necessarily be generalisable to other simulations. Lastly, student clinical examination assessment was subjective to the examiner, and more objective methods of assessment could have been employed. There is a need for multi-centre randomised control trials conducted with a wider variety of simulation exposures to provide deeper insight.
Though medical students demonstrate positive attitudes towards simulation-based education, high-fidelity simulation may not necessarily bring about significant increases in performance. Thus, keeping in mind the costs involved, the utility of high-fidelity simulation in a developing country like Pakistan is questionable.
Ethical approval was received from the Ethics Review Committee of the Aga Khan University, Pakistan (Reference Number: 2019-1856-5300).
Informed consent was obtained by means of a consent form administered to participants prior to their inclusion in the study. It explained the nature of the study, the extent of their involvement in the study, risks and benefits, and use of their academic performance data and survey responses for research purposes and publication.
CONFLICT OF INTEREST:
The authors report no conflict of interest.
SRS: Conceptualized and designed the study, executed its methodology, and coordinated study proceedings.
RSM, MI: Performed data synthesis and analysis.
RSM, SRS: Wrote the manuscript.
All authors critically reviewed the manuscript and approved the final version.