2009

A successful Animal Science program produces graduates that are well-prepared to assume professional roles in the work force or to continue into post-graduate programs. This preparation requires skills and competencies as well as knowledge. Among others, CTAHR Skills and Competencies include analytical/problem solving skills, personal characteristics, human relations skills, and leadership skills. Animal Science Student Learning Outcomes include applying knowledge to appropriate husbandry, developing problem-solving skills for lifetime learning, good citizenship in both personal and professional habits, and exploring the relationship between applied animal biology and society. In their final year, Animal Science students complete a required internship in an area of interest. Supervisor evaluations of these internships provide feedback on the skills and preparation of our students. Supervisors evaluate students in the areas of work performance, professional relationships, professional role, and general/overall. Mean scores for 59 students over the last five years ranged from 86% to 96% with an overall mean of 92%. The only score that was significantly different from the overall mean was 86% for “Initiative to identify needs and proposed solutions.” This appears to be related to a lack of student confidence in their abilities. Of particular interest are the score of 95% for “Would employ student in the future if an opportunity developed,” and the number of students (currently 33%) that end their internships with an offer of employment. Supervisor evaluations provide an objective assessment of student skills and competencies and preparation for a professional role and indicate that Animal Science students are generally well-prepared.

Using Internship Supervisor Evaluations for Program Assessment

A successful Animal Science program produces graduates that are well-prepared to assume professional roles in the work force …

The Tropical Medicine Curriculum Committee reviewed the organization and content of the Tropical Medicine MS and PhD core course series (TRMD 604 & 605). The following data relating to Student Learning Outcome #1 (Tropical Medicine Knowledge Base) were assessed: (1) student comments in program review interviews by an external committee and in course evaluations; (2) faculty review of student performance on qualifying examinations over the past 5 years; (3) faculty interviews of current graduate students regarding specific core content and overall fulfillment of the program’s student learning outcomes; and (4) overall demonstration of student proficiency in the various disciplines of Tropical Medicine. The major point emerging from this assessment was that the core Tropical Medicine content, particularly in immunology and virology, was not being adequately covered in the current two-semester Infectious Disease Microbiology course format. Based on the above data, it was decided that the core courses required expansion from two to three semester courses to (1) increase and improve coverage of the major topics in virology, an area of research emphasis in the program, and (2) to provide a background in immunology which is more focused and relevant to infectious disease pathogenesis, immunity, and host:pathogen interactions.

Assessment & Revision of Core Curriculum in Tropical Medicine

The Tropical Medicine Curriculum Committee reviewed the organization and content of the Tropical Medicine MS and PhD core …

In February 2009, the TIM School, through its Faculty Senate, began a process of program assessment with the guidance and assistance of the Manoa Assessment Office. A team of TIM faculty attended the WASC-sponsored retreat on program assessment, and developed a set of student learning objectives for the TIM program. These objectives were further developed and finalized by the TIM Faculty Senate in March 2009. They then served as the basis for the development of a curriculum map for the Hospitality emphasis courses in April 2009, and for the Tourism/Transportation emphasis courses in October 2009. The TIM School poster will include an overview of this process, as well as its planned next steps and intended uses of the assessment process outcomes.

TIM School’s Assessment Process

In February 2009, the TIM School, through its Faculty Senate, began a process of program assessment with the …

Due to the growing diversity of the United States population, various legal mandates and accrediting bodies require doctors to receive training that will allow them to provide optimal care to patients regardless of their cultural backgrounds. More prevalent in medical specialties such as family medicine and psychiatry, the inclusion of cultural competency or cross-cultural care issues in surgery is an emerging area of recognized need. A nationwide survey on residents’ perceptions of their preparedness to provide cross-cultural care revealed that surgical residents view cultural knowledge as important, but were hampered in acquiring these skills due to a lack of time and absence of formal training. The Accreditation Council for Graduate Medical Education requires residency programs to incorporate cultural training into their curricula. Of the six competencies, cultural competency is addressed under both Professionalism and Interpersonal and Communication Skills. With the assistance of the UH Department of Family Medicine and Community Health, the UH Department of Surgery’s general surgery residency program has developed and is currently piloting a cultural standardized patient exam as a means to both train and evaluate its residents. The case scenario involves a surgeon attempting to obtain informed consent from an elderly Samoan male who needs to have his leg amputated or face certain death. All Program Year-1 (PGY-1) residents (n=13) participated in the exam and were evaluated by one of four faculty preceptors using a standardized tool. Using this as the baseline, an educational training intervention is being planned, which will be followed by a post-test.

The Development of a Cultural Standardized Patient Exam for a General Surgery Residency Program

Due to the growing diversity of the United States population, various legal mandates and accrediting bodies require doctors …

Two years ago, the staff of the Office of Student Life and Development (SLD) began the process of identifying the learning outcomes that guide and inform our in- and out-of-classroom instruction of students. The SLD staff provides advising to UHM’s six Chartered Student Organizations and their programs and teaches introductory and upper division college credit leadership courses, offered through the College of Engineering, Interdisciplinary Studies, and the College of Education. The focus of this poster presentation is two activity sheets designed by the presenters to enable SLD staff to introduce their students, in the organizations which they advise and the classes they teach, to three foundational learning outcomes to be assessed this academic year: self awareness, building relationships, and empowerment of others. The first sheet asks students to rate their leadership knowledge and skill development – emerging, developing, and advanced – in the three learning outcome areas. The second sheet, Strategies for Growth, asks students to select one or more learning outcomes, articulate specific, action-oriented, and time-framed goals, and to consider how the leadership opportunities offered through their organizations and courses could assist them in progressing to the advanced leadership skill level. Recommended processes for staff implementation include individual advising and group discussion and problem-solving activities.

LEVELS OF LEADERSHIP: Developing Strategies for Improving Competency in Three Core Learning Outcomes

Two years ago, the staff of the Office of Student Life and Development (SLD) began the process of …

Each year, the Office of Student Housing Services conducts an assessment to measure our resident’s perspective of various aspects of our housing program. The information obtained is then used to prioritize planned improvements to our housing program; assess progress toward departmental and University goals and objectives; plan training programs; improve business services; develop budgets; and enhance communication strategies with various stakeholder groups.

Student Housing Services Annual Assessment, 2009-2010

Each year, the Office of Student Housing Services conducts an assessment to measure our resident’s perspective of various …

The Department of Psychology has adopted the learning goals and outcomes proposed by the American Psychological Association Task Force. We selected five major goals and outcomes to be assessed with surveys distributed to our undergraduates at the time they declare their major in psychology and again when graduating with their B.A. degree. The surveys serve as a means to determine whether the Student Learning Outcomes in our Psychology courses were achieved. The five major goals that are assessed include: 1) knowledge about psychological concepts and theory; 2) knowledge of basic research methods; 3) use of critical and creative thinking skills in solving problems; 4) understanding of how psychological concepts are used in everyday life; and 5) developing communication skills including writing, interpersonal and oral communication. Student rate the extent to which these goals were achieved at the time of graduation and their ratings are statistically compare to their earlier scores made at the time of major declaration. Analysis made of the rating scores obtained in the Fall 2008 to Spring 2009 academic year indicated significant improvement in attaining goals 1 and 2 (p

Principles That Guide Assessment

The Department of Psychology has adopted the learning goals and outcomes proposed by the American Psychological Association Task …

Students who utilize PAC's services learn their roles and responsibilities in the advising relationship; create a comprehensive plan that accounts for their academic and career endeavors; understand the steps along the pathway to professional school; learn about the available resources for applications, academic advising, campus engagement, and career development; perform a self-assessment that helps them identify their goals and determine their fit to an intended field of student. Utilization of PAC resources has continued to rise each year. Students have been very satisfied by the advising performance of PAC's undergraduate peer advisors.

Pre-Health / Pre-Law Advising Center

Students who utilize PAC’s services learn their roles and responsibilities in the advising relationship; create a comprehensive plan …

The Global Environmental Science (GES) program offers a B.S. degree through the Department of Oceanography in the School of Ocean and Earth Science and Technology. A defining characteristic of the program is the requirement for all majors to conduct a year-long senior research project working one-on-one with a faculty mentor. The research experience is directly integrated into the curriculum as follows: 1) students are introduced to research opportunities through OCN 100 Seminar in Global Environmental Science, 2) the GES curriculum is tailored to the student’s specific interests through selection of 4 upper division electives that complement and support their choice of research topic, 3) students earn OCN 499 Undergraduate Thesis credits while conducting their research, and 4) OCN 490 Communication of Research Results provides instruction and experience in written and oral presentations related to thesis results. Graduation requirements include submission of a written thesis and a public oral presentation of the research findings. This integrated capstone research experience provides a locus for assessment of nearly all GES program learning outcomes. Written theses are read and assessed by faculty mentors and the program chair. Oral presentations are assessed by the faculty mentor and the program steering committee, including the program chair. To date, assessments have been conducted through informal evaluations and group discussion. Development of assessment rubrics are under consideration.

The Global Environmental Science Senior Research Thesis: A Locus for Program Level Learning Outcome Assessment

The Global Environmental Science (GES) program offers a B.S. degree through the Department of Oceanography in the School …

The poster gives the Assessment Tools that we use in our department to assess our department Program Objectives and Outcomes. We have both Internal and External Assessment Tools and within each category, we have either Direct or Indirect tools. The Internal tools are those assessed within our department and External are those assessed outside of our department. Direct assessments are those assessed by sources other than students, and Indirect are those assessed by the students or the Alumni. In addition to the Surveys that are direct questions such as “how well a particular Outcome is achieved?”, we have developed Rubrics based on those Outcomes which are basically metrics that dissects our Outcomes into a number of Concepts which are broken, each, into a number of Performance Criteria, which, in turn, are measured, each, by various levels of achievements such as 1 being Worse and 4 being Best (i.e., the Rubrics).

ABET (Accreditation Board of Engineering and Technology) Assessment Tools

The poster gives the Assessment Tools that we use in our department to assess our department Program Objectives …

The LIS Program believes that assessment is the key to improving student learning and refining our teaching. The Program’s long-term strategic plan includes assessment based on oral comprehensive examinations that are part of the student’s culminating experience. It has also initiated a multi-year plan to assess samples of student work for major course assignments across the curriculum. By analyzing the evidence collected from these sources, the faculty identifies areas for needed improvement and implements data-informed modifications and changes to the Program’s learning experiences.

Student Achievement Is Our Goal!

The LIS Program believes that assessment is the key to improving student learning and refining our teaching. The …

Anthropology is a four-field discipline that encompasses historical, humanistic, biological, linguistic, and psychological approaches to a holistic study of humankind. This disciplinary diversity poses problems for assessment. Quite distinct methods are used in each subfield, and assessment must be targeted to the specific courses in which particular methods are emphasized and mastered. During this review period a pilot assessment activity focused on one of the most important methodological approaches in cultural anthropology (one of the four fields) to help determine effective techniques that might apply to assessments of the other major methods categories. The goal of the activity was to transcend course specific and instructor specific evaluation (as provided already by exams and papers) and allow faculty specialists in each subfield to participate in the focused assessments. In this case members of the “cultural caucus” (one of the three faculty subgroups in the department) focused on linguistic anthropology methodology.

Pilot Test of an Assessment Method

Anthropology is a four-field discipline that encompasses historical, humanistic, biological, linguistic, and psychological approaches to a holistic study …

In Spring 2008, the English Department began assessment of student writing for the University’s Foundations Requirement in Written Communication, which must be fulfilled by all UHM undergraduates. Assessment of FW was to be based on four SLOs devised by the UHM Foundations Board, and approved by the UHM General Education Committee in Fall 2007. As presently articulated, the four SLOs are the following: 1: Compose a text to achieve a specific purpose and respond adeptly to an identifiable audience. 2: Provide evidence of effective strategies for generating, revising, editing, and proofreading a text in order to produce finished prose. 3: Compose a text that makes use of source material that is relevant and reliable and that is integrated in accordance with an appropriate style guide. 4: Compose writing that expresses the writer's viewpoint and is supplemented by outside sources. In Spring 2008, relevant student writing (208 essays) was gathered from the four courses that satisfy the FW requirement: English 100 (Composition I—which makes up about 90 % of FW sections), English 100A (Composition I for Honors students), English 190 (Composition I for transfer students), and ELI 100 (Expository Writing: A Guided Approach—for students whose first language is not English) for assessment of SLO #1, and a report on levels of student success was generated for a Spring 2008 English Department Colloquium and discussion. In Spring 2009, 80 essays were randomly selected from papers submitted by FW instructors that were to be assessed for level of achievement in meeting SLO#3 (Information Literacy): “Students will be able to compose a text that makes use of source material that is relevant and credible and that is integrated in accordance with an appropriate style guide.” A team of six faculty scorers read the papers, with two raters scoring them independently along a 4-point scale according to the following traits: (1) making use of source material, (2) relevancy of sources, (3) credibility of sources, (4) style integration. Overall, student preparation for future writing tasks involving outside sources was measured as follows: “well-prepared” (6%), “prepared” (48%), “partially prepared” (26%), “not prepared” (21%). About 84% of students were at least partially prepared to make use of relevant and credible sources, but only about 65% were at least partially prepared to meet information literacy expectations in their future writing (“adherence to citation rules”). A full report on the results of this SLO Assessment, which has been drafted by the Review Team, will be disseminated to English and E.L.I. faculty and will be discussed at an English Department meeting on December 3.Meanwhile, assessment of SLO#4 and SLO#2 is anticipated in 2009-2010 and 2010-2011, respectively. A rubric for assessing SLO#4 will be discussed at the December 3 meeting and further discussion of student achievement of this Student Learning Outcome, as well as SLO#3, will follow at a January 21 meeting. Additionally, all Spring 2010 English 100, 100A, and 190 instructors have been directed to include all four SLOs on their Spring 2009 syllabi, with the expectation that classroom activities and paper assignments will engage students in achieving these four Student Learning Outcomes.

How Well are First-Year Students Composing? Assessing the Foundations in Written Communication Program

In Spring 2008, the English Department began assessment of student writing for the University’s Foundations Requirement in Written …

From fall 2007 to spring 2009, the UH Writing Mentors Program has reached approximately 1,300 students across 70 sections of English 100. Writing Mentors, who are graduate students in English, attend class and hold individual writing conferences with all students outside of class. For many first-year students, these mentors are the only university representative who learns their name, background, interests, academic goals, challenges in transitioning to college, and strengths and weaknesses as a writer. The initiative has received rankings of “satisfied” or “very satisfied” from 89% of students, 94% of mentors, and 98% of instructors; furthermore 85% of first-year students surveyed claimed that their mentor helped them in their transition to college. Program administrators have engaged in multiple forms of assessment including the following: a large-scale scoring of first-year student writing that demonstrated mentored students out-performing their non-mentored counterparts in statistically significantly ways in the categories of content, organization, language & style, and meta-cognition/ reflective ability; standardized logs tracking every mentor-student conference; analysis of longitudinal data on how mentored versus non-mentored students perform as writers and students post-English 100; interviews with focus groups of mentors, students, and instructors; and written end-of-semester evaluations from all participants. Our poster will summarize key results of these assessment activities and highlight the ways in which they have led to a) programmatic improvements each semester; b) peer-reviewed publications in the fields of Composition Studies and the Scholarship of Teaching and Learning (SoTL), each of which underscore the role of mentoring in student retention; c) arguments for continued funding.

Increasing First-Year Students’ (Writing) Success: An Assessment of the UH Writing Mentors Program

From fall 2007 to spring 2009, the UH Writing Mentors Program has reached approximately 1,300 students across 70 …

The learning objectives of the undergraduate program in economics include economic literacy, critical thinking, quantitative reasoning, and ability to effectively communicate results of economic research and analysis to colleagues and decision-makers. In the fall of 2006, UHM Department of Economics has adopted an assessment plan for the economics undergraduate program, in order to evaluate our progress in achieving these objectives. The undergraduate assessment was first implemented in AY 2006-2007, and has been carried out every academic year ever since. In this presentation, we describe the development of the assessment tools, the process’s preliminary findings, as well as difficulties that are being faced in the assessment process.

UHM Department of Economics’ Assessment Experience

The learning objectives of the undergraduate program in economics include economic literacy, critical thinking, quantitative reasoning, and ability …

Shidler College of Business assesses student learning in each of six programs: Bachelor of Business Administration, Master of Business Administration, Master of Accounting, Master of Financial Engineering, Master of Human Resource Management and PhD in International Management. Undergraduate and Master’s degree programs all make use of course-embedded assessment. Assessment of PhD student learning includes faculty reviews of students’ research and evaluation of students’ teaching. Each semester, the Shidler College assessment committee evaluates assessment results for undergraduate and master’s degree programs and makes recommendations to the college Curriculum and Programs Committee. In addition, each department reviews assessment outcomes of major-specific objectives for undergraduate students.

Assessment at The Shidler College of Business

Shidler College of Business assesses student learning in each of six programs: Bachelor of Business Administration, Master of …

The poster describes the Department of Civil and Environmental Engineering’s process of assessing program outcomes. The steps in the assessment process are first described followed by a list of program outcomes. Only direct modes of assessment are used. The frequency of assessment is tabulated. The poster then expands on the use of performance appraisal in select courses, which represents a relatively new mode of assessment that the department has recently adopted. Performance appraisal is performed once every three years per outcome and is advantageous in that it can be used as a vehicle to involve more faculty members and facilitate faculty buy-in to the assessment process. Detailed is the schedule involved in the performance appraisal assessment process along with a sample scorecard and a sample evaluation of one outcome. The sample scorecard contains the concepts that were evaluated along with the performance criteria. After one cycle of assessment of using performance appraisal, the Department is interested to know whether the changes implemented will result in any program improvement. It is envisioned this continual process of assessment will lead to continuous quality improvement.

Assessment of Program Outcomes

The poster describes the Department of Civil and Environmental Engineering’s process of assessing program outcomes. The steps in …

The Department of Botany faculty members have completed the initial steps of developing a program assessment plan for our three undergraduate degrees: BA, Botany; BS, Botany; and BS, Ethnobotany. We first devised explicit and realistic Student Learning Outcomes (SLOs) for all students completing any of our degrees. We then considered the extent to which our existing classes addressed individual SLOs and scored each class for each SLO in one of 5 overlapping categories: 1) does not address SLO; 2) provides introductory material that relates to SLO; 3) provides learning material that reinforces a subject; 4) provides students with the opportunity to master a subject; and/or 5) provides an assessment of student performance. This scoring process resulted in a curriculum map for each degree. The entire process has caused us to consider revising some courses, eliminating some courses, and designing some entirely new courses. The next step is to develop and implement tools for assessing the success of achieving programmatic learning outcomes.

Program Assessment – Department of Botany

The Department of Botany faculty members have completed the initial steps of developing a program assessment plan for …

The B.S. degree in Marine Biology from the University of Hawaii at Manoa College of Natural Sciences is an interdisciplinary academic program that offers unique opportunities for hands on learning. Extensive field experiences are integrated with traditional classroom and laboratory courses, giving full exposure to the theoretical/practical aspects of marine biology. The program’s student learning outcomes (SLOs) are aimed at preparing students for either future graduate school experiences or entry into the private sector. The students will be able to apply the scientific processes, to communicate about biological sciences through writing and oral communicating, and to recall foundational biological information that is necessary for pursuing post-baccalaureate schools or entering a career in the biological sciences. Certain ways that are proposed to collect assessment are to check students’ laboratory notebooks and reports, observe students as they perform laboratory techniques, evaluate students’ oral presentations, evaluate students’ research proposal and exams, and evaluate student portfolios and their senior capstone experience. Future steps we are proposing are to have more specific program SLOs for the marine biology majors. Presently, the current program SLOs highlight the student’s basic biological science foundation experiences. We plan to start assessing the senior students in the marine biology program through analysis of the experiences in their required directed research. We also plan to implement assessment through the students’ capstone course, which is also required. Finally, we intend to broaden the scope of the curriculum map with the inclusion of more courses with their new specific program SLOs.

B.S. Marine Biology Proposed Program SLOs and Assessment

The B.S. degree in Marine Biology from the University of Hawaii at Manoa College of Natural Sciences is …

The student-athlete experience at the University of Hawai‘i is unique! One significant challenge for our student-athletes is traveling more than 2,500 miles with each trip to a competition. The distance extends the time away from class and places additional pressure on student-athlete academic performance. A significant goal of our strategic plan is “To provide each student-athlete with the best possible opportunities for academic success, leading to graduation.” We use the National Collegiate Athletics Association’s (NCAA) Academic Progress Rate and Graduation Success Rate to assess the academic success, retention, and progress toward graduation of our student-athletes. Data assessment has led to the implementation of policies and strategies to enhance student-athlete academic performance, retention and graduation and a system has been initiated to evaluate the performance of head coaches on these important measures.

Student-Athletics Academic Performance

The student-athlete experience at the University of Hawai‘i is unique! One significant challenge for our student-athletes is traveling …

The Assessment Office supports programs as they engage in program-level assessment of student learning. Program assessment is undertaken for the purpose of improving student learning and guiding program development. It is faculty-driven and faculty-supervised. It involves establishing student learning outcomes, measuring/observing and documenting the extent to which outcomes are achieved, and finding ways to improve programs so effectiveness is increased. This poster describes how the Assessment Office assessed its Program Outcome #3: “Academic degree programs complete the assessment cycle, which includes faculty members using assessment results to improve student learning.” Data were collected through the Annual Assessment Reports submitted by program in Fall 2009. Results suggest that while the number of programs engaged in assessment activities has increased since 2008, not all programs have completed an assessment cycle. The Assessment Office used the results to develop the spring 2010 workshop series, improve the on-line Annual Assessment Report system, and schedule consultations with programs.

Annual Reports: Assessment Status Across Campus

The Assessment Office supports programs as they engage in program-level assessment of student learning. Program assessment is undertaken …

In the last few years, the need has increased at UHM for students to more actively engage in educational planning amidst many changes, including the decline of state resources for education and the economy at large. Providing advising and related academic services for over 5,000 A&S undergraduate majors, the Colleges of Arts and Sciences Student Academic Services (CASSAS) has been responding to these changes by determining its goals for assessment before revising programs. As such, this poster session represents the process by which CASSAS faculty have developed assessable student learning outcomes and advising programs in line with the department’s mission and objectives. The process begins with the distillation of student learning outcomes from a broader set of initial outcomes devised during a CASSAS retreat. With the establishment of understandable and assessable student learning outcomes, CASSAS faculty proceeded to build a learning map which charted when, how and through what potential or modified programs students would be given the opportunity to practice the skills that would lead them to master the established outcomes. The beginnings of an assessment rubric are also built into the learning map to identify different stages of knowledge and skill mastery as indicated by the outcomes. Finally, the poster provides artifacts that will be used to assess the quality of student’s internalization of CASSAS’s learning outcomes at various stages of their educational development. The extent to which students achieve the indicators of CASSAS student learning outcomes correlates to the validity of CASSAS advising programs.

Assessment for Advising

In the last few years, the need has increased at UHM for students to more actively engage in …