Unit: Medical Education
Program: Medicine (MD)
Degree: Doctor, Medical
Date: Tue Oct 13, 2009 - 7:14:41 am

1) List your program's student learning outcomes (SLOs).

I.                    Life-Long Learning Skills: Graduates will be life-long learners.

II.                 The Biological Sciences: Graduates will understand the biological sciences underlying clinical medicine.

III.               The Care of Patients: Graduates will be able to care for their patients with increasing responsibility, and seeking and responding to appropriate evaluation and supervision.

IV.              Oral and Written Communication Skills: Graduates will be able to communicate effectively with patients, families and other healthcare providers.

V.                 Populational and Community Health: Graduates will appreciate the epidemiology of disease and the role of the physician in public health and global health issues, particularly those important to Hawaii and the Asia-Pacific region.

VI.              Professionalism: Graduates will be professional and ethical, demonstrate an enthusiasm for medicine, and value honor, integrity, altruism, respect, accountability, excellence, scholarship, and leadership while delivering compassionate care to their patients.

VII.            Personal Health and Well-Being: Graduates will maintain personal health and well-being.

2) Where are your program's SLOs published?

Department Website URL: http://jabsom.hawaii.edu/JABSOM/admissions/objectives.php?l1=mdp&l2=currS
Student Handbook. URL, if available online: http://jbows.jabsom.hawaii.edu/view/url/M82C/54080/54502
Information Sheet, Flyer, or Brochure URL, if available online:
UHM Catalog. Page Number:
Course Syllabi. URL, if available online: NA
Other: On framed posters displayed throughout the medical school and in clinical departments
Other: On pocket-sized cards distributed to all faculty

3) Upload your program's current curriculum map(s) as a PDF.

Curriculum Map File(s) from 2009:

4) What percentage of courses have the course SLOs explicitly stated on the course syllabus, department website, or other publicly available document? (Check one)


5) State the SLO(s) that was Assessed, Targeted, or Studied

All SLOs were assessed and reviewed as part of a national accreditation process that took place November 2008-January 2009.  The medical school received full 8-year accreditation status based on this comprehensive evaluation of curriculum, faculty and physical infrastructure, student services, assessment and outcomes.

6) State the Assessment Question(s) and/or Goal(s) of Assessment Activity

The goal of these ongoing assessment activities is to determine if medical students are achieving these objectives. 

7) State the Type(s) of Evidence Gathered

Student achievement of the JABSOM Graduation Objectives is monitored by the JABSOM Curriculum Committee through the review and analysis of specific, measurable outcomes both internal and external.  They include but are not limited to:

  • Student Performance on Selected Examinations

  • USMLE Step 1 Examination Scores

  • USMLE Step 2 Clinical Knowledge Examination Scores

  • USMLE Step 2 Clinical Skills Examination Scores

  • USMLE Step 3 Examination Scores

  • AAMC Graduation Questionnaire

  • Student Academic Progress and Graduation Rates

  • JABSOM NRMP Results

  • JABSOM Intern Survey

  • JABSOM Program Director Survey

  • JABSOM Alumni Survey

8) State How the Evidence was Interpreted, Evaluated, or Analyzed

The medical school uses a number of processes to interpret, evaluate and analyze assessment data.

Results of USMLE and other national exams

USMLE. The USMLE Step 1 and Step 2 exam results are sent to the Office of Student Affairs (OSA) by the NBME. The results are reviewed by OSA, the curriculum committee, subcommittees, department chairs, the medical school Executive Committee, and the faculty senate. Unsatisfactory performance is reviewed by the Evaluation Review and Remediation Committee (ERRC) and by the Student Standing and Promotion Committee as needed. 

Comprehensive Basic Science Exam. To help students gauge their progress in learning the basic sciences and becoming familiar with the depth and breadth of questions on a national standardized exam, the NBME Comprehensive Basic Science Examination is administered to all students at the beginning and end of their second year of medical school. The exam, first given in the 2003 – 2004 academic year, is optional at the beginning of the year and required at the end of the year. Results are reviewed by OSA and the curriculum committee. Unsatisfactory performance on the required end-of-year exam also is reviewed by the ERRC. Students whose performance indicates a need for improvement are counseled by an OSA learning specialist.

NBME Clinical Discipline Shelf Exams. The relevant NBME Clinical Discipline Shelf Exams are administered to students at the end of the internal medicine, ob/gyn, psychiatry, and surgery clerkships. Exam results are reviewed by the respective clerkship directors and faculty as well as by OSA. The ERRC also reviews unsatisfactory performances and recommends plans for remediation.

Results of internally-developed examinations

Student scores on internally developed examinations. Written knowledge exams, consisting of essay and/or multiple choice questions, are prepared, administered, and graded by course and clerkship directors and their faculties. Similarly, preclinical laboratory exams are prepared, administered, collected, and graded by basic science instructors. Relevant course directors, faculty, and the Office of Student Affairs review all results. Student achievement of course objectives is formally assessed in the Curriculum Committee’s Curriculum Review Process.  The ERRC reviews the exams of students with unsatisfactory performance and recommends plans for remediation.

Performance-based assessment of clinical skills (e.g., OSCEs).  The OSCE. Students are required to pass a six-station multidisciplinary OSCE before starting the third-year clerkships and a ten-station multidisciplinary OSCE (BIOM 531: Comprehensive OSCE) at the end of the clerkship year. OSCEs are administered by the Center for Clinical Skills (CCS), a part of the JABSOM Office of Medical Education. CCS collects and grades the exams. The results are reviewed by the clinical skills committee, OSA, curriculum committee, and the clerkship directors committee.

Clinical Skills Exams. In addition, standardized patient clinical skills exams are included in the evaluation of student performance in the first- and second-year clinical skills course, the third-year family medicine, internal medicine, and pediatric clerkships, and the fourth-year geriatric course. CCS also administers and grades these exams, which then are reviewed at the relevant course or departmental level.  

Oral Exams. The pediatric and psychiatry clerkships design and administer oral exams to students as part of their evaluations. These exams assess clinical knowledge, clinical reasoning, and problem-solving. The clerkship directors and faculty collect and review the results, which inform the clerkships about each student’s ability to integrate knowledge and skills. 

Triple Jump. The Triple Jump, an oral and written assessment of students’ knowledge, self-directed learning, critical appraisal, and problem-solving, is administered individually to all first-year students. Students are rated on both their knowledge and skills. Ratings then are collected and reviewed by the course director. 

Other measures of curriculum effectiveness

Student responses on the AAMC Medical School Graduation Questionnaire.  JABSOM students complete the AAMC Medical School Graduation Questionnaire online. A paper copy of the comparison between JABSOM student and national responses is sent to OSA. The results are reviewed annually by OSA and the curriculum committee. 

Student evaluation of courses and clerkships.  As part if JABSOM’s larger program evaluation process, the Office of Medical Education (OME) designs a set of standardized surveys with core items based on the school’s graduation objectives. Within this set are surveys that are administered to students at the end of every course and clerkship. OME collects the surveys, analyzes the data, and forwards written reports of the findings to respective course and clerkship directors, the chair of the curriculum committee, and the director of the Office of Medical Education. A summary of the findings is reported to the curriculum committee. Relevant course and clerkship directors are invited to attend the curriculum committee meetings at which their survey findings are presented and discussed. The curriculum committee makes recommendations to the course and clerkship directors as needed. The directors, in turn, are responsible for addressing the recommendations and reporting the results of actions taken to the curriculum committee. 

Student advancement and graduation rates.  OSA monitors students’ academic progress and program completion rates. This information is reported to the curriculum committee.

NRMP results.  NRMP results are sent to OSA. Both OSA and the curriculum committee review these results. Information about the students’ ranking of the residency program with which they matched was made available through 2001. In 2001, more than half of JABSOM students matched with their first choice and 92 percent matched with one of their first three choices.

Specialty choice of graduates.  As part of JABSOM’s curriculum evaluation process, OME sends surveys to alumni six years after graduation. Once every four years, the Alumni Survey includes demographic items such as licensure, specialty choice, specialty certification, practice location and practice type. The findings, which are included in the written report of the Alumni Survey, inform the chair of the curriculum committee, the director of OME, and the chairs of the preclinical education and clerkship directors committees about JABSOM graduates’ practice selections. 

Assessment of residency performance of graduates.  JABSOM’s curriculum evaluation process also includes a residency program director survey, sent at the end of graduates’ first year of residency to their program directors.  This survey asks program directors to rate the performance of JABSOM graduates relative to their peers.  A written report of the survey findings is given to the chair of the curriculum committee, the director of OME, and the chairs of the preclinical education and clerkship directors committees. A written and oral summary of the report is given to the curriculum committee. 

Licensure and specialty certification rates of graduates.  The Alumni Survey, sent by OME to JABSOM alumni six years after graduation, includes an item about licensure and an item about specialty certification once every four years. The licensure and specialty certification rates are included in the written report given to the chair of the curriculum committee, the director of OME, and the chairs of the preclinical education and clerkship directors committees. They also are included in the summary report given to the curriculum committee, where they are reviewed and discussed.

Practice location and practice type of graduates.  Every fourth year, the Alumni Survey also includes items about practice location and practice type. Graduates’ responses are included in the written report given to the chair of the curriculum committee, the director of OME, and the chairs of the preclinical education and clerkship directors committees. They also are included in the summary report given to the curriculum committee, where they are reviewed and discussed.

Graduate evaluation of preparation for internship.  JABSOM sends an Intern Survey to graduates near the end of their first year of residency. The survey focuses on graduates’ perceptions of how well JABSOM prepared them for residency. Together, the findings of this survey and the Residency Program Director Survey provide information about the effectiveness of JABSOM’s curriculum from two perspectives, that of the postgraduate learner and that of the postgraduate teacher. A written report of the findings is sent to the chair of the curriculum committee, the director of OME, and the chairs of the preclinical education and clerkship directors committees. A written and oral summary of the report is given to the curriculum committee for review and discussion. 

Graduate evaluation of preparation for practice.  JABSOM sends an Alumni Survey to graduates six years after graduation when most are thought to have completed their postgraduate training. This survey focuses on graduates’ perceptions of their preparation for medical practice, thereby providing a long view of the effectiveness of the JABSOM curriculum. A written report of the findings is sent to the chair of the Curriculum Committee, the director of OME, and the chairs of the preclinical education and clerkship directors committees. A written and oral summary of the report is given to the curriculum committee for review and discussion.  

Student evaluation of teachers.  Preclinical students use a standardized form to evaluate their tutors and lecturers at mid-course and again at the end of the course. The evaluations consist of both ratings and narrative responses. At the end of every clerkship, third-year students complete an evaluation form for all faculty members with whom they have had learning experiences. Fourth-year students in electives, geriatrics, and emergency medicine evaluate their preceptors at the end of the course.  

Faculty peer review of courses and clerkships.  Each course and clerkship is systematically reviewed every three years by a set of two faculty members designated by the curriculum committee. The review process is outlined in a handbook which is distributed to the directors of the course or clerkship and to the faculty reviewers. Directors and reviewers are encouraged to work collaboratively during the review process in pursuit of innovative curriculum improvement. Faculty reviewers prepare a report of the findings, which they present to the curriculum committee with the relevant course or clerkship directors in attendance. Following a discussion, recommendations are given to the directors as needed. Course directors are required to respond to the recommendations with a written and oral report to the curriculum committee, thereby completing the communication loop.

How the information is used for curriculum review and change

The student and program evaluation data are used to evaluate student acquisition of the knowledge, skills, behaviors, and attitudes described in JABSOM’s educational objectives. The multi-trait, multi-method data collected from students, faculty, graduates, and residency program directors inform educational leaders about the effectiveness of the overall curriculum as well as each individual course and clerkship. The information guides decision-making at all curricular levels and allows the school to monitor the effects of changes that are made to the curriculum on the basis of these decisions. The data collected is presented and reviewed by the JABSOM Curriculum Committee and other major stakeholders in the educational program.

9) State How Many Pieces of Evidence Were Collected

As noted above, multiple “pieces of evidence” were collected through a number of means.  Examinations are both course-specific as well as strategically placed throughout the curriculum at “milestone” points.  Program (curriculum) surveys were administered to all enrolled medical students at multiple points throughout their curricular experience.

10) Summarize the Actual Results

JABSOM students perform well on the various steps of the national licensing examinations (USMLE Step 1, Step 2 CK, and Step 2 CS exam).  Students have been successful in the residency matching process, with many students matching with competitive residency programs both in Hawaii and across the mainland. The results of the AAMC Graduation Questionnaire support the effectiveness of the curriculum.  JABSOM graduates report near the end of their internship that they are well-prepared for graduate training.  Finally, residency program directors relate that in comparison to others in their intern class, JABSOM graduates are competent physicians who demonstrate strong professional behavior.

11) Briefly Describe the Distribution and Discussion of Results

Results were discussed at the Curriculum Committee and appropriate subcommittees as part of formal committee processes.  Course directors, administrators, and key content coordinators received detailed copies of the results.  Summaries of the results were shared with the larger faculty and students.  The national accreditation body for medical schools also received both detailed data and summaries.

12) Describe Conclusions and Discoveries

Both internal groups and national accreditation organization concluded that medical student SLOs are being achieved.  Quality improvement efforts continue in all areas.

13) Use of Results/Program Modifications: State How the Program Used the Results --or-- Explain Planned Use of Results

Assessment outcomes, and national accreditation results, affirmed that the medical student curriculum is achieving its intended goals.  Continuing quality improvement efforts are ongoing in all areas of the curriculum with the goal of maintaining and/or improving outcomes even further.

14) Reflect on the Assessment Process

What went well?  Assessment needs to be done at multiple levels and on a continuous basis.  Central institutional oversight through a Curriculum Committee and Office of Medical Education ensured maintaining of momentum and follow-through on school-wide assessment efforts.What would we do differently next time?  Assessment at a large, continuous scale requires resources and institutional commitment. 

15) Other Important Information

16) FOR DISTANCE PROGRAMS ONLY: Explain how your program/department has adapted its assessment of student learning in the on-campus program to assess student learning in the distance education program.

17) FOR DISTANCE PROGRAMS ONLY: Summarize the actual student learning assessment results that compare the achievement of students in the on-campus program to students in the distance education program.