methods

The development and implementation of surveys to assess learning within the College of Education (COE) for students in the final semester of their programs and for mentor teachers who supervise teacher candidates in their final semester of student teaching are detailed in this poster. Outlined through the use of text and data visualization are: (a) the utilization of surveys as indirect evidence of student learning, (b) key considerations and principles when developing new survey instruments and survey scales, (c) recommendations for successful administration of surveys, (d) methods for interpreting survey results, and (e) suggested ways of using survey results for program improvement and assessment reporting.

Program Completion and Mentor Surveys as Indirect Evidence of Learning: From Development to Use

The development and implementation of surveys to assess learning within the College of Education (COE) for students in …

The Tropical Medicine Curriculum Committee reviewed the organization and content of the Tropical Medicine MS and PhD core course series (TRMD 604 & 605). The following data relating to Student Learning Outcome #1 (Tropical Medicine Knowledge Base) were assessed: (1) student comments in program review interviews by an external committee and in course evaluations; (2) faculty review of student performance on qualifying examinations over the past 5 years; (3) faculty interviews of current graduate students regarding specific core content and overall fulfillment of the program’s student learning outcomes; and (4) overall demonstration of student proficiency in the various disciplines of Tropical Medicine. The major point emerging from this assessment was that the core Tropical Medicine content, particularly in immunology and virology, was not being adequately covered in the current two-semester Infectious Disease Microbiology course format. Based on the above data, it was decided that the core courses required expansion from two to three semester courses to (1) increase and improve coverage of the major topics in virology, an area of research emphasis in the program, and (2) to provide a background in immunology which is more focused and relevant to infectious disease pathogenesis, immunity, and host:pathogen interactions.

Assessment & Revision of Core Curriculum in Tropical Medicine

The Tropical Medicine Curriculum Committee reviewed the organization and content of the Tropical Medicine MS and PhD core …

Due to the growing diversity of the United States population, various legal mandates and accrediting bodies require doctors to receive training that will allow them to provide optimal care to patients regardless of their cultural backgrounds. More prevalent in medical specialties such as family medicine and psychiatry, the inclusion of cultural competency or cross-cultural care issues in surgery is an emerging area of recognized need. A nationwide survey on residents’ perceptions of their preparedness to provide cross-cultural care revealed that surgical residents view cultural knowledge as important, but were hampered in acquiring these skills due to a lack of time and absence of formal training. The Accreditation Council for Graduate Medical Education requires residency programs to incorporate cultural training into their curricula. Of the six competencies, cultural competency is addressed under both Professionalism and Interpersonal and Communication Skills. With the assistance of the UH Department of Family Medicine and Community Health, the UH Department of Surgery’s general surgery residency program has developed and is currently piloting a cultural standardized patient exam as a means to both train and evaluate its residents. The case scenario involves a surgeon attempting to obtain informed consent from an elderly Samoan male who needs to have his leg amputated or face certain death. All Program Year-1 (PGY-1) residents (n=13) participated in the exam and were evaluated by one of four faculty preceptors using a standardized tool. Using this as the baseline, an educational training intervention is being planned, which will be followed by a post-test.

The Development of a Cultural Standardized Patient Exam for a General Surgery Residency Program

Due to the growing diversity of the United States population, various legal mandates and accrediting bodies require doctors …

Anthropology is a four-field discipline that encompasses historical, humanistic, biological, linguistic, and psychological approaches to a holistic study of humankind. This disciplinary diversity poses problems for assessment. Quite distinct methods are used in each subfield, and assessment must be targeted to the specific courses in which particular methods are emphasized and mastered. During this review period a pilot assessment activity focused on one of the most important methodological approaches in cultural anthropology (one of the four fields) to help determine effective techniques that might apply to assessments of the other major methods categories. The goal of the activity was to transcend course specific and instructor specific evaluation (as provided already by exams and papers) and allow faculty specialists in each subfield to participate in the focused assessments. In this case members of the “cultural caucus” (one of the three faculty subgroups in the department) focused on linguistic anthropology methodology.

Pilot Test of an Assessment Method

Anthropology is a four-field discipline that encompasses historical, humanistic, biological, linguistic, and psychological approaches to a holistic study …

New LCME accreditation standards will require medical schools to monitor curriculum and content and to develop a curriculum inventory over the next 1--2 years. There is little literature on the use of curriculum mapping to evaluate Problem-Based Learning (PBL) curriculum. Equipping medical students with knowledge and skills to care for our rapidly aging population is critical. We describe how developing a curriculum map for geriatric medicine core competencies during preclinical education helped identify areas to target revision of curricula. While curriculum mapping will be necessary in meeting AAMC inventory requirements in the future, the Curriculum Mapping of Geriatric Medicine Core Competencies at JABSOM has already proven to be an effective strategy. During our analysis, we found that we covered many LCME survey topics important for accreditation (e.g. care of the disabled, abuse, end--of--life care, health care systems). Through examination of the preclinical curriculum, we identified areas of deficiency, and also that the sequence was not ideal. Greater emphasis on geriatric physiology should occur earlier in the preclinical years, and coverage of optimal medication management should occur in later courses. Greater discussion regarding “Hospital Care for Elders,” including the important topics of patient safety and discharge planning towards the end of the second year, would also help better prepare students for their clinical years. Revisions to the curriculum and examinations are planned. We also plan to expand our analysis to include interprofessional education and cultural competence.

Curriculum Mapping of Geriatric Medicine Core Competencies in the Preclinical Problem-Based Learning Curriculum at the John A Burns School of Medicine, University of Hawai′i

New LCME accreditation standards will require medical schools to monitor curriculum and content and to develop a curriculum …

Oral proficiency in Mandarin is one of the core language skills targeted by the Chinese program learning outcomes. This skill is primarily assessed through ACTFL Oral Proficiency Interviews (OPI), a nationally recognized language proficiency test. Given the volunteer nature of the test participation, only BA Chinese major students with more confidence have participated in it. Even though the results have shown that seniors from the Chinese B.A. program is performing above expectation, the program is not satisfied with the interview participation rate. These students contributed the reasons of low level of participation to the lack of confidence and practice. To address this need, the program coordinator collaborated with faculty and re-engineered Chinese 411, a speaking capstone course to prepare students for the OPI test in the Fall 2013. A rubric was developed for students to understand the expectations so that they can prepare with guidance. During this course, students were trained with a series of Advanced Level oral performance tasks: self-introduction, description, narration in different time frames in paragraphs, and report of current event as well as various role-plays. After each module, they produced recordings of performance in simulated interviews similar to OPI. Two assistants outside of the class independently evaluated 6 to 7 oral performance recordings and provided feedback. The result of this curricular change is that students have increased confidence in oral performance and willingness to participate in the OPI.

Improving Oral Proficiency in Chinese Using Simulated OPI Assessment

Oral proficiency in Mandarin is one of the core language skills targeted by the Chinese program learning outcomes. …

Mānoa Works is designed to increase the college retention and success of need-based students through part-time work experience with campus employers. Students in the Federal Work Study program are provided with additional support to secure a campus-based job in positions that promote student success, recruitment, campus engagement and outreach. In support of students’ development and learning, career and professional development training was designed to increase their awareness and preparedness with the use of Peer Trainers from our MCC student assistant pool.

Mānoa Works

Mānoa Works is designed to increase the college retention and success of need-based students through part-time work experience …

The Hawai‘i Rehabilitation Counseling program (HRC) at the University of Hawaiʻi at Mānoa (UHM) prepares graduate students to become successful practitioners and administrators in the field of vocational rehabilitation. This 48-credit distance education program leads to a Master of Science degree in Kinesiology & Rehabilitation Science, with a specialization in Rehabilitation Counseling. The HRC Program includes three clinical courses (Practicum, Internship I, and Internship II) that prepare students for participation in the field of vocational rehabilitation counseling. Although a form has been used in the past to assess students’ performance in these courses, the current program director found that the assessment form did not tie directly to accreditation standards and specific student learning outcomes. This poster reflects the process used to evaluate current practices; involve current teaching faculty and community site supervisors to review, revise, and refine the program map, student learning outcomes, and present student evaluation form; and through this process, develop an applicable and relevant clinical student evaluation form. The assessment process and subsequent product will enable students to be more appropriately evaluated in their clinical courses.

Moving Forward with Fieldwork Assessment!

The Hawai‘i Rehabilitation Counseling program (HRC) at the University of Hawaiʻi at Mānoa (UHM) prepares graduate students to …

The University of Hawaiʻi at Mānoa’s highest attrition rate occurs between students’ first and second years. This issue may be attributed to students succumbing to the sophomore slump, demonstrated by decreases in grades, motivation, and outlook on college experiences. To address this issue, a campus-wide program comprised of members from both academic and student affairs was established in Spring 2010 and was named the Mānoa Sophomore Experience (MSE). To better understand our student population and how to address their needs, MSE members collaboratively developed, analyzed, and applied the results of three assessment resources: 1) Survey (601 student respondents), 2) Focus groups with students, faculty, and staff (3 separate sessions), 3) Written reflection assignment (over 500 student respondents). Each of the assessment resources examined first- and second-year students and their challenges faced, interest and likelihood of attending MSE activities, level of use of campus resources, what they enjoyed most about attending UHM, and suggestions for MSE programming. Using the data collected from assessment, MSE successfully developed and executed programming for UHM’s first- and second-year students. This poster outlines the assessment methods, sample student programming, and assessment results.

Manoa Sophomore Experience: An Example of Successful Data-Driven Programming

The University of Hawaiʻi at Mānoa’s highest attrition rate occurs between students’ first and second years. This issue …

The Sociology Department at the University of Hawaiʻi at Mānoa (UHM) made significant changes to program assessment during Spring and Fall 2014 as a result of strong leadership from the undergraduate advising staff combined with increased communication with faculty and the direct participation of instructors. This poster offers three key pieces of information: the methods we used during Fall semester to increase faculty buy-in; an overview of results from our pilot undergraduate exit survey; and our program’s initial utilization of results. Key to increasing faculty participation in program assessment were strong leadership and improved intra-departmental communications. Our revised SLOs—now in alignment with University ILOs—allowed us to target our required methods and theory courses to assess student performances. Instructors reported data on students’ final class assignments based on rubrics developed specifically for that purpose. Streamlined data collection led to an increase in the accuracy of responses, a response rate of 100% from instructors, and data on the performances of 78 students. Results from the exit survey of students who had petitioned to graduate during Spring and Summer 2015 were reported to the Department, resulting in several actions. First, there will be ‘strategy workshops’ in which instructors share teaching practices that have helped students improve their written and oral communication skills. Second, instructors were urged to include more opportunities for students to discuss and apply research methods and more opportunities for students to hone their oral presentation skills.

UH Manoa Sociology Department: Undergraduate Assessment in Action

The Sociology Department at the University of Hawaiʻi at Mānoa (UHM) made significant changes to program assessment during …

Pacific Island Studies at the University of Hawaiʻi at Mānoa has 6 teaching faculty and 20 majors enrolled each semester, including 85% underserved students. Writing and critical thinking skills are very important to us. In May 2014 faculty collected student course assignments and evaluated them collaboratively using a rough rubric. The results showed that 50% students were not meeting our expectations. Subsequently, we revised SLO4 and explored external learning opportunities (e.g., Eng 100, Writing Center, our own writing intensive courses) and discovered that they are not sufficient in helping students to achieve our expectations. As a result, we developed the Yeah, WRITE! program to enhance disciplinary writing across the curriculum. The program consists of a series of in-class workshops built around reading and note-taking skills, citation, information literacy, and stages of the writing processes for various genres. Further, we used students work to develop and refine an evaluation rubric and aligned the Yeah, WRITE curriculum with the rubric. We also implemented writing intensive designations in all major core courses. Preliminary data showed that the program is successful: students were happy to learn diverse approaches to the writing process; grateful for handouts and models useful for other classes; relieved to have tools to address writer’s block. We plan to systematically assess SLO 4 every two years to monitor the effect of the new program. The poster shares the program curriculum, strategies to engage faculty, and invite collaboration with campus committees in refining and utilizing the Yeah, WRITE! program.

Yeah, WRITE! New Disciplinary Writing Curriculum As Outcome of Assessment Cycle

Pacific Island Studies at the University of Hawaiʻi at Mānoa has 6 teaching faculty and 20 majors enrolled …

In Spring 2016, the General Education Office identified a need to improve its indirect assessment efforts. Working with the co-creators of the Student Assessment of their Learning Gains (SALG) instrument, the office developed and piloted an assessment plan beginning with the Hawaiian, Asian, and Pacific Focus requirement. Through the pilot, the General Education Office aimed to increase familiarity with SALG in order to educate others about its value in improving the quality of instruction; recruit faculty to pilot the use of SALG in HAP-designated courses; and identify how meaningful, usable assessment data could be extracted from SALG to address programmatic and learning needs within the General Education program. This poster provides information on the SALG instrument, outlines the process taken and challenges involved in implementing a new assessment plan, and identifies important next steps in the process.

Adopting a New Assessment Approach: Using SALG to Evaluate General Education Learning Outcomes

In Spring 2016, the General Education Office identified a need to improve its indirect assessment efforts. Working with …

This poster shares results of a survey conducted in Summer 2016 to individuals eligible to participate in the Come Back to Mānoa program. In addition to an evaluation of the program itself, the results provide an overview of why seniors left UH Mānoa so close to finishing their degree and their reasons for returning. This poster adds to the extant research, which largely concentrates on first year and sophomore students, by bringing light to senior attrition and how to support those students to persist to degree (Hunt et al., 2012). Practical applications for utilizing the information obtained from the survey in order to help improve the Come Back to Mānoa program are discussed.

Assessing the Come Back to Manoa Program: Why Seniors Leave and How to Help them Graduate

This poster shares results of a survey conducted in Summer 2016 to individuals eligible to participate in the …

Exploring possibilities for, and effects of, Integrated Performance Assessments (IPAs) in the Department of East Asian Languages and Literatures, this poster reports an action study using IPAs in an undergraduate beginning Mandarin program. The poster first features a review of IPAs, followed by an overview of curriculum redesign and IPA test reconstruction. The poster then presents a concrete model for IPAs in a Chinese 101 and 102 Beginning Mandarin. Successful teaching activities and assessment task samples will be presented. The effects of this adaptation are demonstrated using quantitative and qualitative data, including oral assessment videos, writing samples, supplementary listening and reading materials, rubrics for scoring, test scores, student self-reflections, and more. The data show that students (1) generally favored using IPAs, (2) took initiative to review the IPA rubrics and to reliably engage in filling out "can-do" checklists, (3) demonstrated a positive correlation between IPAs and traditional test scores. The poster concludes that IPAs can be equally successful, and can offer more, in university foreign language classes.

Implementation of Integrated Performance Assessments (IPA) in Beginning Level Chinese Language Classes

Exploring possibilities for, and effects of, Integrated Performance Assessments (IPAs) in the Department of East Asian Languages and …

Department of Mathematics Program Assessment via Exam

Department of Mathematics Program Assessment via Exam

Department of Mathematics Program Assessment via Exam by Heiner Dovermann

This poster outlines the process through which faculty examined how the embedded assignments, field components, and signature assignment across a three-course sequence aligned. The course sequence of the three Learner in the Environment courses are outlined. Faculty utilized the federal funded Collaboration for Effective Educator Development, Accountability and Reform (CEEEDAR) Center tools to ensure that across the three courses students engaged in meaningful course and field assignments aligned with current research. Scoring criteria are provided along with initial and post-implementation ratings and a sample report. The courses were examined using the Culturally Responsive Teaching Practices, the Classroom Management, and the High Leverage Practices rubrics. Finally, from identified areas in need of additional clarity or additional coverage, this poster highlights how results from the report were utilized toward syllabi revision, which increased the number of elements addressed across the course sequence.

Course Series Alignment: Examining a Three Course Sequence

This poster outlines the process through which faculty examined how the embedded assignments, field components, and signature assignment …

For the BS in Animal Sciences supervisor evaluations from the capstone field experience provide a valuable external check on student achievement of learning objectives. A summary of five years of evaluations of ethical behavior and oral and written communication confirms student success.

Using Field Supervisor Evaluations for Assessment

For the BS in Animal Sciences supervisor evaluations from the capstone field experience provide a valuable external check …

The Marine Biology Graduate Program has established an ad-hoc committee to review existing protocols of the comprehensive exam and is collating the most effective elements and best practices from each of academic units across the program. This committee will make recommendations to the MBGP faculty at large for discussion, consideration, and ultimately implementation.

Revising the oral comprehensive exam in the UHM Marine Biology Graduate Program

The Marine Biology Graduate Program has established an ad-hoc committee to review existing protocols of the comprehensive exam …

The poster describes the steps and resulting standardized student admission procedure for the Communication Sciences and Disorders’ Master program. The process features: (1) admission interview questions crafted based on agreed-on desirable student qualities and (2) the evaluation rubric stemmed from the department core values.

Standardizing admission process to defy the “crystal ball” in student selection to Communication Sciences and Disorders’ Master Program

The poster describes the steps and resulting standardized student admission procedure for the Communication Sciences and Disorders’ Master …

Less than 75% of Biological Engineering students have been assessed as meeting student learning outcome targets in the identifying, formulating, and solving engineering problems. The program faculty have collaborated to adopt a developmental approach to assessment, rubric development, curriculum mapping, and instruction, including a signature assignment at the junior level.

Scaffolding and Assessing Engineering Design: Effecting Program Change from Course Innovations

Less than 75% of Biological Engineering students have been assessed as meeting student learning outcome targets in the …

This project was to increase the learning and the assessment capabilities in two core Child and Family courses.

Strengthening the Child and Family Specialization Curriculum through Developing Standard, Explicit, and Sustainable Assessments and Outcomes

This project was to increase the learning and the assessment capabilities in two core Child and Family courses. By …

In 2018, the Communication program in the School of Communications conducted a program assessment of undergraduate senior capstone ePortfolios to evaluate the evidence of quality learning within the Communication major. The poster presents our assessment results and pre- and post- evaluation activities toward curriculum improvement.

Advancing Learning in the Communication BA through ePortfolios

In 2018, the Communication program in the School of Communications conducted a program assessment of undergraduate senior capstone …

The poster will describe the process that the dietetics program faculty and staff underwent to develop a systematic and streamlined process for assessing student learning of 22 accreditation-required outcomes. The process includes collaborative curriculum mapping, assignment refinement, rubric development, and the establishment of an electronic professional portfolio system.

Streamlining student learning outcome assessment through ePortfolios in Dietetics Program

The poster will describe the process that the dietetics program faculty and staff underwent to develop a systematic …

Choose a Method to Collect Data/Evidence

Last Updated: 4 March 2024. Click here to view archived versions of this page. The data or evidence in a …