Assessing Depression in the Mental State Examination: OSCE Scores and Student Satisfaction with Simulated Versus Real Patients

By Anum Haider¹, Syeda Kauser Ali²

Affiliations

1. Department of Psychiatry, Dow Medical College, Dow University of Health Sciences, Karachi, Pakistan
2. Institute of Medical Education, Jinnah Sindh Medical University, Karachi, Pakistan

doi: 10.29271/jcpsp.2025.10.1289

ABSTRACT
Objective: To compare the Objective Structured Clinical Examination (OSCE) performance and learner satisfaction among undergraduate medical students trained in depression assessment during the Mental State Examination (MSE), using simulated patients (SPs) versus real patients (RPs).
Study Design: A quasi-experimental study.
Place and Duration of the Study: Department of Psychiatry, Civil Hospital, Karachi, Pakistan, from June to November 2023.
Methodology: Fourth-year medical students on psychiatry clinical postings were divided into two groups: Control group and Intervention group. A demonstration session on assessing depression, led by the facilitator, was followed by a practice session. The Control group practised on real patients, while the Intervention group practised on simulated patients. Students’ satisfaction with learning experience was recorded using a validated, self-rated scale, Escala de Satisfacao com as Experiencias Clinicas Simuladas (ESECS). Clinical skills were assessed during the OSCE at the end of the clinical posting using a rating scale. Data were analysed using SPSS, Chi-square test, independent sample t-test, and Pearson correlation for descriptive statistics.
Results: A total of 80 students participated in this study. The two groups demonstrated comparable performance on the OSCE station (p = 0.43) and almost similar levels of learner satisfaction (p = 0.06), indicating no statistically significant differences.
Conclusion: Both real and simulated patients contributed equally to the acquisition of clinical skills for assessing depression by medical students. The selection of teaching resources, therefore, depends on patient availability and specific clinical needs of the students.

Key Words: Simulated Patients, Mental state examination, Depression, Medical students, Objective structured clinical examination, Learner satisfaction.

INTRODUCTION

Psychiatric disorders are a major contributor to the global burden of disease, yet patients with such conditions remain stigmatised.¹ The stigma attached to mental illness, together with inconsistent access to care, not only worsens outcomes for psychiatric patients but also delays recovery from physical disorders.² Recognising these challenges, the World Health Organization, in its 2021 Mental Health Plan, called for improved education, training, and community mental health services.³ Similarly, the Pakistan Medical and Dental Council has highlighted the need for undergraduate medical students to acquire communication skills, empathy, and clinical competence in all disciplines, including psychiatry.⁴

Learning psychiatry, however, is uniquely challenging. Undergraduate students often show limited interest in the subject due to cultural attitudes, stigma, and a lack of confidence in engaging with psychiatric patients.⁵ Real patients (RPs) may be uncooperative, unpredictable, or extremely unwell to allow repeated assessments; thereby, limiting their availability for large student groups and restricting the opportunities for structured teaching and fair assessment.⁶ Moreover, inconsistent clinical exposure further compromises the reliability of high- stakes assessments, such as the Objective Structured Clinical Examination (OSCE).

Simulated patients (SPs) have emerged as a promising educational strategy in medical education.⁷ SPs provide standardised, reproducible clinical scenarios that allow students to practice skills in a safe and supportive environment, without burdening real patients.⁸ In psychiatry, SPs are particularly valuable for teaching the Mental State Examination (MSE), which requires careful observation of behaviour, emotional state, and thought processes, along with sensitive communication.⁹ Simulation offers consistency, ensures all learners face the same level of complexity, and facilitates structured feedback and debriefing. Previous literature from medicine and nursing supports the value of simulation-based learning in enhancing clinical competence, decision-making, and patient-centred communication. Importantly, simulation also improves learner satisfaction, which is a key driver of motivation, professional development, and academic performance.¹⁰

In Pakistan, large class sizes in public-sector medical colleges make it especially difficult to provide equitable exposure to psychiatric patients. Simulation-based approaches can help overcome these constraints and complement RPs teaching. However, evidence regarding the effectiveness of SPs in undergraduate psychiatry education, particularly in the assessment of depression through the MSE, remains limited.

The present study, therefore, aimed to compare undergraduate students’ OSCE scores and learner satisfaction as primary and secondary outcomes when trained in the assessment of depression using SPs versus RPs. Hence, a significant difference would be observed between the mean OSCE and satisfaction scores of students learning the MSE with SPs and those with RPs during their psychiatry posting in a public-sector teaching hospital.

METHODOLOGY

This single-centre, quasi-experimental study was conducted in the Department of Psychiatry, Civil Hospital, Karachi, Pakistan, from 1^st June to 30^th November 2023. Two consecutive groups of fourth-year MBBS students, each posted for one month, were included. Ethical approval was obtained from the Institutional Review Board of Dow University of Health Sciences, Karachi, Pakistan (IRB-2807/DUHS/Approval/2022/62). All consenting students attending the full teaching programme were eligible; absentees or those missing the OSCE were excluded. A total of 80 students participated, which was considered sufficient for a pilot study. Students posted in the first month (Control group) were taught with RPs, and those in the second month (Intervention group) were taught with SPs.

Teaching comprised three components. First, the principal investigator (PI) delivered a 30–45-minute demonstration of the MSE in depression, using role play with a resident. Second, students practised in groups: the Control group with nine stable, cooperative RPs (mean age 32.5 ± 12 years), and the Intervention group with eight trained SPs (mean age 24.6 ± 1.9 years). The SPs were psychology interns trained over two days as per ASPE¹¹ guidelines, using culturally adapted scripts and videos of RPs. RPs and SPs were matched for socioeconomic background; RPs were compensated with free medication, while SPs were awarded certificates. Finally, each session concluded with a 10–15-minute structured debriefing led by the PI. The sample collection procedure is summarised in Table I.

Table I: Procedure of sample collection.
 

Study Groups	Demonstration Class (30–45 mins)	Patient Interaction	Debriefing Session (10–15 minutes)	OSCE Exam	Online Satisfaction Survey
Control group	Yes	Real patient interaction	Yes	Yes	Yes
Intervention group	Yes	Simulated patient interaction	Yes	Yes	Yes

Table II: Descriptive statistics.

Variables	Characteristics		p-values*
	(Control Group)	(Intervention Group)
Gender       Male       Female	11 (27.5%) 29 (72.5%)	10 (25%) 30 (75%)
Schooling       Cambridge       Intermediate Board	22 (55%) 18 (45%)	14 (35%) 26 (65%)
Electives in psychiatry       No       Yes	40 (97.5%)	39 (97.5%) 01 (2.5%)
Total OSCE score (out of 45)	24.3 ± 4.7	23.6 ± 3.9	0.43
Total satisfaction score (out of 140)	111.25 ± 11.7	116.7 ± 13.85	0.06
Overall OSCE performance (1-5 scale)	3.10 ± 0.778	2.9 ± 0.67
Overall learning experience satisfaction score (1-10 scale)	8.25 ± 1.104	8.37 ± 1.29
p-value was calculated by applying an independent t-test.

Table III: Correlation of the OSCE performance and learner satisfaction with total satisfaction scores.

Variables	Groups	r	p-values
OSCE performance vs. total OSCE score	NC	0.66	<0.001
	NI	0.815	<0.001
Learner satisfaction vs. total satisfaction score	NC	0.67	<0.001
	NI	0.63	<0.001
*p-value was calculated by Pearson's correlation. r: Pearson's correlation.

At the end of the postings, students undertook an OSCE within one week of the practice session. Five stations were used, one of which assessed depression via the MSE. Two trained examiners independently scored each station using a 9-item, 5-point entrustment scale (1 = below expectation, 5 = above expectation) and a global rating scale. Different SPs were employed for the OSCE to avoid bias. Faculty training standardised scoring and removed ambiguity.

Learner satisfaction was measured anonymously using a modified 14-item English version of the Escala de Satisfacao com as Experiencias Clinicas Simuladas (ESECS).¹² Items were rated on a 10-point scale, and internal consistency was satisfactory (Cronbach’s alpha = 0.795). Additional questions addressed gender, schooling type, and prior psychiatric exposure.

Normality was assessed with the Shapiro–Wilk test. Descriptive statistics (mean ± SD, frequencies, and percentages) were calculated. Independent t-tests compared the OSCE and satisfaction scores between the groups, while the Chi-square test assessed the associations between gender and schooling. The Pearson correlation examined the relationship between the OSCE and satisfaction scores. A p-value of <0.05 was considered statistically significant.

RESULTS

A total of 80 fourth-year medical students participated, with no attrition (40 in each group). Female participants predominated (n = 59; 73.8%). Eleven men (27.5%) were in the Control group and 10 (25%) in the Intervention group. Over half of the students (55%) had a Pakistani schooling background. Cambridge schooling was more frequent in the Control group (n = 22, 55%), while the Pakistani Intermediate Board predominated in the Intervention group (n = 26, 65%). Only one student in the Intervention group had completed a psychiatry elective. The details are shown in Table II.

The adapted 14-item ESECS scale showed good reliability (Cronbach’s alpha = 0.795). Both groups reported high satisfaction, with similar mean scores (Control group: 8.25 ± 1.10; Intervention group: 8.37 ± 1.29). The total satisfaction scores were slightly higher in the Intervention group (117 ± 13.8) than in the Control group (111 ± 11.7); however, this difference was not statistically significant (p = 0.06).

The mean OSCE scores were comparable between the groups (Control group: 24.3 ± 4.7; Intervention group: 23.6 ± 3.9), with no significant difference (p = 0.43). However, when analysed collectively, the female participants scored significantly higher OSCE marks than the male participants (p = 0.032).

Significant positive correlations were observed between the groups. The OSCE performance correlated strongly with the total OSCE scores (Control group: r = 0.66; Intervention group: r = 0.815; both p <0.001). Similarly, the overall learner satisfaction scores correlated with the total satisfaction scores (Control group: r = 0.67; Intervention group: r = 0.63; both p <0.001), as shown in Table III.

DISCUSSION

This study evaluated fourth-year undergraduate medical students learning to assess depression during the MSE through either RPs or SPs. The OSCE performance and satisfaction scores were compared between the two groups. In this curriculum, the psychiatry posting is limited to the fourth year, whereas international literature reports simulation-based studies across the third, fourth, and final years in other disciplines.¹³

No statistically significant differences were found between RPs and SPs groups in the OSCE performance or the learner satisfaction (p >0.05), suggesting that both approaches are equally effective for teaching the MSE. Although not formally measured, students appeared more engaged with RPs, possibly due to hearing their life stories. Limited clinical exposure, examination anxiety, and underlying student depression may explain the absence of score differences despite varied learning opportunities.¹⁴ Anxiety during assessments, including OSCE, has been well documented in medical students.¹⁴

A gender difference was observed: the female students achieved significantly higher OSCE scores (p = 0.032), consistent with the previous studies showing that female students often outperform males.¹⁵ However, no significant differences were noted in the satisfaction scores or outcomes by schooling system, indicating that other factors such as mentorship, teaching environment, and student motivation may play stronger roles.¹⁶ Prior psychiatric electives did not show measurable influence due to limited variation in this sample.

The OSCE and satisfaction scores showed a weak, non- significant correlation in both groups. High satisfaction did not necessarily translate into higher exam performance, likely due to inadequate practice, exam anxiety, or overconfidence.¹⁷ Repeated deliberate practice and constructive feedback are essential for meaningful improvement, as highlighted in the previous studies.¹⁸ These findings reinforce the need for multiple practice sessions and faculty-led debriefing to consolidate skills.

A validated simulation-based satisfaction tool¹² was adapted to measure satisfaction in both groups. Other studies, such as Ziaee et al.¹⁹ have used modified job satisfaction questionnaires in RPs settings. Higher satisfaction levels correlated with better performance, consistent with the findings by Wu et al.²⁰ who reported that motivation and positive engagement improve outcomes.

Variation in the OSCE scores is a recognised issue. Zehra et al.²¹ identified rater and demographic influences on scores. Although not directly assessed in this study, standard- setting and examiner training remain important for fairness. The OSCE remains a well-established assessment tool in medicine.²² Combining checklists and global ratings, as in this study, aligns with the recommendations by Abass and Ahmed,²³ who found both methods to be valid.

During data collection, RPs often shared personal stressors, which distracted students from structured tasks, whereas SPs maintained focus. This suggests that early training with SPs provides structured skill development, while RPs later add complexity and authenticity. Both approaches are essential, but their timing and frequency should be optimised.

Teaching institutes should integrate simulation-based psychiatry teaching into the undergraduate curriculum, with efforts to develop SP banks for psychiatric conditions. Exposure to SPs may also help reduce stigma by increasing familiarity with psychiatric disorders. Stress management programmes and mock exams should be introduced to reduce exam anxiety, and examiner training should ensure reliable assessment.

Effective psychiatry education requires an enabling environment, adequate timing and frequency of practice sessions, and clearly defined expectations of competence. Structured feedback, mentoring, and resources for stress and time management further support student learning. Faculty development and examiner training are also essential to ensure reliable teaching and fair assessment.

Limitations of this study include a small sample size, single- centre design, the absence of randomisation, the inability to use a crossover design, lack of SPs’ feedback to students, and possible confounding factors from higher student engagement with RPs. These factors limit generalisability. Larger, multicenter, and randomised studies are recommended.

CONCLUSION

No significant differences were found in the short-term out- comes of OSCE performance or learner satisfaction, indicating that both approaches hold comparable educational value. Incorporating both RPs and SPs is recommended to balance structured practice with clinical authenticity.

ETHICAL APPROVAL:
Ethical approval was obtained from the Institutional Review Board of Dow University of Health Sciences, Karachi, Pakistan (IRB-2807/DUHS/Approval/2022/62).

PARTICIPANTS’ CONSENT:
Informed consent was obtained from all the study participants.

COMPETING INTEREST:
The authors declared no conflict of interest.

AUTHORS’ CONTRIBUTION:
AH: Conception, design, acquisition, data analysis, and writing of the manuscript.
SKA: Data analysis, review, and interpretation.
Both authors approved the final version of the manuscript to be published.

REFERENCES

1. Pescosolido BA, Halpern-Manners A, Luo L, Perry B. Trends in public stigma of mental illness in the US, 1996–2018. JAMA Netw Open 2021; 4(12):e2140202. doi: 10.1001/jamanetworkopen.2021.40202.
2. Ohrnberger J, Fichera E, Sutton M. The relationship between physical and mental health: A mediation analysis. Soc Sci Med 2017; 195:42-9. doi: 10.1016/j.socscimed. 2017.11.008.
3. World Health Organization. Comprehensive mental health action plan 2013–2030. Geneva: WHO; 2021. Available from: httpss://www.who.int/publications/i/item/97892400 31029.
4. Pakistan Medical and Dental Council. Guidelines for Undergraduate medical education curriculum. Islamabad: PM&DC; 2024. Available from: httpss://pmdc.pk/Documents/ Others/PM&DC% 20GUIDELINES%20FOR%20UG%20M.EDUCATION-06242024035407.pdf.
5. Sharmin T, Azim E, Choudhury S, Kamrun S. Reasons of absenteeism among undergraduate medical students: A review. Anwer Khan Mod Med Coll J 2017; 8(1):60-6. doi: 10.3329/akmmcj.v8i1.31662.
6. Dickinson BL, Lackey W, Sheakley M, Miller L, Jevert S, Shattuck B. Involving a real patient in the design and implementation of case-based learning to engage learners. Adv Physiol Educ 2018; 42(1):118-22. doi: 10.1152/advan. 00174.2017.
7. Hajira B, Rana A, Naeem F, Qammer Y. Virtual skills teaching of nursing students during COVID-19: A problem- solving approach. J Coll Physicians Surg Pak 2022; 32(4):548-9. doi: 10.29271/jcpsp.2022.04.548.
8. Dave S. Simulation in psychiatric teaching. Adv Psychiatr Treat 2012; 18(4):292-8. doi: 10.1192/apt.bp.110.008482
9. Martin A, Krause R, Jacobs A, Chilton J, Amsalem D. The mental status exam through video clips of simulated psychiatric patients: An online educational resource. Acad Psychiatry 2020; 44(2):179-83. doi: 10.1007/s40596-019- 01140-9.
10. Bird GC, Kozakowski SM, Mullins A, Campbell E. Impact of educational format on learner commitment to change and satisfaction. J Contin Educ Health Prof 2020; 40(3):207-10. doi: 10.1097/CEH.0000000000000301.
11. Lewis KL, Bohnert CA, Gammon WL, Holzer H, Lyman L, Smith C, et al. The association of standardized patient educators (ASPE) standards of best practice (SOBP). Adv Simul 2017; 2:10. doi: 10.1186/s41077-017-0043-4.
12. Baptista RC, Martins JC, Pereira MF, Mazzo A. Students' satisfaction with simulated clinical experiences: Validation of an assessment scale. Rev Lat Am Enfermagem 2014; 22(5):709-15. doi: 10.1590/0104-1169.3295.2471.
13. Siemerkus J, Petrescu AS, Kochli L, Stephan KE, Schmidt H. Using standardized patients for undergraduate clinical skills training in an introductory course to psychiatry. BMC Med Educ 2023; 23(1):159. doi: 10.1186/s12909-023-04107-5.
14. Steiner-Hofbauer V, Holzinger A. How to cope with the challenges of medical education? Stress, depression, and coping in undergraduate medical students. Acad Psychiatry 2020; 44(4):380-7. doi: 10.1007/s40596-020-01193-1.
15. Klein R, Ufere NN, Rao SR, Koch J, Volerman A, Snyder ED, et al. Association of gender with learner assessment in graduate medical education. JAMA Netw Open 2020; 3(7):e2010888. doi: 10.1001/jamanetworkopen.2020.10 888.
16. Alyahyan E, Dustegor D. Predicting academic success in higher education: Literature review and best practices. Int J Educ Technol High Educ 2020; 17:3. doi: 10.1186/s41 239-020-0177-7.
17. Guraya SY, Guraya SS, Habib F, AlQuiliti KW, Khoshhal KI. Medical students' perception of test anxiety triggered by different assessment modalities. Med Teach 2018; 40 (Sup1):S49-55. doi: 10.1080/0142159X.2018.1465178.
18. Aoki S, Shikama Y, Yasui K, Moroi Y, Sakamoto N, Suenaga H, et al. Optimising simulated interviews and feedback to maximise medical students’ self-efficacy in real time. BMC Med Educ 2022; 22(1):438. doi: 10.1186/s12909-022- 03512-6.
19. Ziaee V, Ahmadinejad Z, Morravedji AR. An evaluation on medical students’ satisfaction with clinical education and its effective factors. Med Educ Online 2004; 9(1):4365. doi: 10.3402/meo.v9i.4365.
20. Wu H, Li S, Zheng J, Guo J. Medical students’ motivation and academic performance: the mediating roles of self-efficacy and learning engagement. Med Educ Online 2020; 25(1):1742964. doi: 10.1080/10872981.2020.1742964.
21. Zehra T, Aaraj S, Naeem H, Altaf HN, Aamir R, Aitazaz M. OSCE grading: A cross-sectional study on inter-observer variability in assessment. Rawal Med J 2022; 47(3):723-7.
22. Memon S, Shaikh SH. Comparison of performance on written and OSCE assessment during end semester paediatric examination. Pak J Med Sci 2020; 36(4):711-6. doi: 10.12669/pjms.36.4.2026.
23. Abass MO, Ahmed ME. Comparison of a task-specific checklist and end exam global rating scale for scoring the objective structured clinical examination used to evaluate sixth-year medical students in surgery at Shendi University, Sudan. Eas J Humanit Cult Stud 2020; 6(2):257-61. doi: 10.36349/EASJEHL.2020.v03i06.010.