2008

This poster reviews Mathematics’ assessment activities. It describes the program goals, assessment plan, and current activities; presents a curriculum map; and includes samples of syllabi, which incorporate program goals. It also offers thoughts on systematic data collection and lessons learned.

An Overview of Assessment in the Mathematics Department

This poster reviews Mathematics’ assessment activities. It describes the program goals, assessment plan, and current activities; presents a …

The Department of Electrical Engineering has put in place an assessment process that (1) has educational objectives and outcomes; (2) a process to assess the achievement of these objectives and outcomes; and (3) a system to improve the program based on the assess data. The organization of the system, the constituents, and the process are presented.

Assessment Processes for the Department of Electrical Engineering

The Department of Electrical Engineering has put in place an assessment process that (1) has educational objectives and …

The logic model is a tool to assist faculty and program heads map their planning and assessment activities. It is a carefully laid out one-page document showing the whole as well as the parts developed in a logical sequence. The format encourages planners to focus on overall goal achievement as they carry out activities. This presentation provides a template of a logic model that can be adapted and used by a range of programs. Each component of the logic model matrix (inputs, activities, outputs, intermediate outcomes, long term outcomes, assessment methods, results, and use of results) is defined and described. A set of seven sample logic models from the University Health Services programs is presented.

Logic Models as a Tool to Plan and Manage Assessment

The logic model is a tool to assist faculty and program heads map their planning and assessment activities. …

This study evaluates team teaching models of a new curriculum adopted by the University of Hawai`i's journalism program in 2004. In this new media convergence curriculum, students learn all media formats — print, video and online — and do not specialize only in traditional media, such as newspaper or TV journalism. Instructors team teach so that students will learn a holistic approach to storytelling in which knowledge and skills from the different specialties are integrated. The assessment of the models focused on the first three cohorts of students, from 2004 to 2007. In the models, degrees of faculty collaboration varied depending on course level and student learning outcomes. Students in first-year basic journalism classes benefited from a lower level of faculty collaboration than those in second-year classes where advanced multimedia skills and holistic, integrated thinking were needed — i.e. an ability to "see the big picture." The study suggests that team teaching and teaching media convergence go hand-in-hand.

Assessing a New Journalism Curriculum: Evaluating Team-Teaching Models in a Media Convergence Curriculum 2004-2007

This study evaluates team teaching models of a new curriculum adopted by the University of Hawai`i’s journalism program …

Over the past several years the Department of Speech has collected data from Speech majors prior to their Fall or Spring graduation. Graduating majors complete an assessment instrument consisting of 50 multiple choice items culled from exams from all courses required to be taken by Speech majors. Questions are approximately evenly distributed across the 6 required courses. Scores are compared to those obtained from a sample of SP 151 students, i.e., largely, but not entirely, first year students who have had little to no experience in Speech courses other than SP 151. Graduating majors also complete a self-report survey assessing their perceptions of the department, the faculty, and their level of learning and achievement in the major.

Assessment of Retention of Content from Required Courses Using Comparison Groups

Over the past several years the Department of Speech has collected data from Speech majors prior to their …

Outreach College serves as an advocate for the needs of the nontraditional, adult and distance learning students, i.e., those who are unable to attend the regular day program that Mānoa provides. This population includes both the working adult seeking a degree and the professional looking to upgrade his or her skills. A few months ago, a task force made up of faculty and staff at Outreach College was formed to investigate ways to better service this nontraditional student population. The name “Transitions” was given to this project since we were specifically looking at adults interested in starting or returning to college and specifically, UH Mānoa. We invited two focus groups made up of representatives from various student support groups on campus and a third group composed of students who had taken noncredit courses from us in the past, asking them the same questions: 1) who are these nontraditional students; 2) what are the challenges they face in returning to school, and 3) how can we help them overcome these challenges. The poster will outline the formation of our focus groups from the beginning to the action plan resulting from these sessions. Also included will be mention of things that worked for us in this endeavor.

Growing an Assessment: Focus Groups

Outreach College serves as an advocate for the needs of the nontraditional, adult and distance learning students, i.e., …

Practicum assessment utilizes an approach that helps ensure the masters of public health (MPH) students learn to take action and apply their knowledge and skills while participating in practicum projects in community organizations. The main tools of assessment include: supervision by a field preceptor, observation of the student in the field, completion of objectives and deliverables designated on the department-wide evaluation form, and a three-part critical thinking reflective paper written by the student. The practicum experience has resulted in students making significant service-learning contributions to community agencies, while these community agencies are preparing MPH students to enter the public health workforce.

Practicum Assessment from the Department of Public Health Sciences

Practicum assessment utilizes an approach that helps ensure the masters of public health (MPH) students learn to take …

First-Year Programs (FYP) utilizes multiple approaches to assess student learning outcomes and program success. Institutional data are used to measure retention rates for students participating in Access to College Excellence Learning Communities (ACE); National Student Clearing House (NSCH) data are gathered to measure student transfer rates. Qualitative data and feedback is collected through focus groups and surveys. In addition, ACE students complete two surveys measuring student expectations, engagement, and institutional commitment. Fall 2007 student engagement survey results indicate ACE students felt significantly more informed about core graduation requirements, major requirements, and registration procedures. Students also felt significantly more connected to the university community. NSCH data indicates that a large proportion of ACE students that did not continue at UHM transferred to other institutions after their first year. FYP will expand its evaluation by collecting Drop-Failure-Withdraw (DFW) rates for classes offered as part of an ACE learning community.

First-Year Success: Evaluating a Peer-Led Learning Community Program

First-Year Programs (FYP) utilizes multiple approaches to assess student learning outcomes and program success. Institutional data are used …

Active and interested faculty members met in a half a day workshop to develop a "curriculum concept map" for the Kamakakuokalani Center for Hawaiian Studies. This exercise coupled the techniques of traditional curriculum mapping with those of conceptual mapping. Thus, at the end of this exercise faculty were able to develop in a graphic manner, the broad organizational concepts that link the Center's areas of concentration together. An underlying advantage was the sequencing of classes by area and in typical curriculum mapping tradition, sequence level of mastery by course.

Kamakakuokalani Curriculum Concept Map Exercise

Active and interested faculty members met in a half a day workshop to develop a “curriculum concept map” …

This poster presents an in-process plan for assessing student learning outcomes (SLOs) in ELI 83, the English Language Institute’s (ELI’s) advanced writing course for graduate students who have English as a second language. This plan is an initial effort to develop and assess SLOs for a single course, which will then lead to similar evaluation processes in other ELI courses. Guiding principles in this effort include (1) involving local assessment users (ELI teachers, administrators, and staff) and incorporating their input throughout the process, (2) providing multiple kinds of useful information to these same users to guide discussion and decisions related to evolution of the course, and (3) maximizing the validity of the assessment through use of a step-by-step piloting process. The poster shows how these guiding principles were built into SLO assessment, describes our progress up to this point, and shares practical insights from our experiences so far.

Outcomes Assessment of, by, and for the People: SLOs in the English Language Institute

This poster presents an in-process plan for assessing student learning outcomes (SLOs) in ELI 83, the English Language …

The philosophy undergraduate program has a relatively small number of student learning outcomes and most of these refer to acquired skills rather than to content knowledge. Mapping courses to these was a pretty uncontroversial and painless process. Development of assessment tools for the skills is a bit more challenging but we are making progress.

Keeping Program Assessment Simple

The Philosophy undergraduate program has a relatively small number of student learning outcomes and most of these refer …

The College of LLL (CLLL) formed an Evaluation Resource Team (LLL-ERT) to (a) create evaluation capacity and a useful and sustainable evaluation system; (b) conduct useful program evaluation practice to meet department internal needs as well accreditation demands; and (c) facilitate departments to take actions based on evidence. During the 2008-09 academic year, CLLL is collecting student opinions from graduating students via an online student exit survey in order to illuminate and improve various elements of CLLL programs. Program elements include curriculum, advising, intake and exit processes, learning environment and resources, program delivery, and student learning and teaching. Each department prioritized evaluation foci (i.e., two-year language requirement, BA, MA, and/or PhD), and is creating department-specific questions. A set of College-wide questions, based on faculty and student opinions, was created so that data can be accumulated in the coming years. The poster summarizes the mission of the LLL-ERT and provides an overview and plan for the CLLL evaluation project.

Enhancing Evaluation of CLLL Programs: Developing an Online Student Exit Survey System

The College of LLL (CLLL) formed an Evaluation Resource Team (LLL-ERT) to (a) create evaluation capacity and a …

The task of creating an assessment plan in a fine arts department that focuses largely on studio work can be a daunting one. This poster will provide a sketch of how the music department is successfully embarking on their assessment journey.

Creating a Successful Assessment Plan in the Fine Arts: Assessment Trials, Tribulations, and Successes of a Music Department

The task of creating an assessment plan in a fine arts department that focuses largely on studio work …

Palliative medicine is a highly complex field aimed to prevent and relieve suffering, and improve the quality of life for people with serious, progressive, and terminal illness and their families. Palliative medicine requires integrating medical knowledge, attitudes, clinical and communication skills, personal insight, reflective ability, self awareness, and narrative competence. For this reason, teaching, learning, and assessing learners in palliative medicine is also complex. At JABSOM, fourth year medical students complete a four week long mandatory rotation in geriatric and palliative medicine. The rotation combines small group interactive sessions in the classroom with clinical experiences in a variety of settings. Common challenges for learners include overcoming negative attitudes regarding serious illness and death and integrating psychological and social issues to the practice of medicine. We present a series of templates developed to aid ongoing assessment, evaluation, and support of the learners during the rotation.

Assessment in Palliative Medicine

Palliative medicine is a highly complex field aimed to prevent and relieve suffering, and improve the quality of …

In the Spring of 2008, all students in Foundations in Writing (FW) courses were asked to select and submit the piece of writing that best exemplified the following Student Learning Outcome: “Students will be able to compose a text that seeks to achieve a specific purpose and responds adeptly to an identifiable audience.” Students were also asked to compose a 30-minute in-class reflection on their essay’s purpose and audience as a means of assessing students’ meta-cognitive understandings of their essay’s rhetorical situation. Employing a stratified-random sampling design, 208 (or 50%) of the essays submitted by first-year students were selected for scoring across the five FW course types: English 100 Mentored, English 100 Non-Mentored, English 101 Lab, English 100A, and English Language Institute 100. Given the wide variety of essay prompts, scorers assumed the intended audience to be a critically-informed reader. Essays were scored independently by two raters along a 4-point scale according to the following primary analytic traits: 1) content, 2) organization, 3) language and style, and 4) mechanics. In a separate session, in-class reflective essays were scored holistically on a 4-point scale. Overall, students scored at the “prepared” or “well prepared” levels at the following rates: content (72%), organization (57%), language and style (74%), mechanics (68%), and reflective pieces (48%). Students in mentored sections out-performed their non-mentored counterparts in all categories, scoring significantly higher on content, organization, and reflective pieces. Results point to potential areas for programmatic improvement and also indicate that the Writing Mentors Program is improving the quality of writing for first-year students.

How Well are First-Year Students Composing? Assessing the Foundations in Written Communication Program

In the Spring of 2008, all students in Foundations in Writing (FW) courses were asked to select and …

The poster gives the Assessment Tools that we use in our department to assess our department Program Objectives and Outcomes. We have both Internal and External Assessment Tools and within each category, we have either Direct or Indirect tools. The Internal tools are those assessed within our department and External are those assessed outside of our department. Direct assessments are those assessed by sources other than students, and Indirect are those assessed by the students or the Alumni. In addition to the Surveys that are direct questions such as “how well a particular Outcome is achieved?”, we have developed Rubrics based on those Outcomes which are basically metrics that dissects our Outcomes into a number of Concepts which are broken, each, into a number of Performance Criteria, which, in turn, are measured, each, by various levels of achievements such as 1 being Worse and 4 being Best (i.e., the Rubrics).

ABET (Accreditation Board of Engineering and Technology) Assessment Tools

The poster gives the Assessment Tools that we use in our department to assess our department Program Objectives and …

Several studies have revealed that successful mentoring affects college student retention. Also, research on attrition shows that the first year is a critical stage in students' decisions to persist or leave college. Nora & Crisp have recently identified four essential elements to mentoring: psychological/emotional support; support for setting goals and choosing a career path; academic subject knowledge support aimed at advancing a student's knowledge relevant to their chosen field; and specification of a role model. In the 2007-2008 Academic Year, over 400 first-year students in English 100 at UHM were mentored by MA or PhD students in English, and analysis of their standardized end-of-term evaluations revealed that students identified each of the elements above in varying degrees, with “academic subject support knowledge” identified 79% of the time and with at least one of these elements identified 85% of the time. These results suggest that mentoring when delivered as part of a course holds great potential for boosting student retention at UHM.

Assessing Mentoring in First Year Composition: A Tool to Boost Retention?

Several studies have revealed that successful mentoring affects college student retention. Also, research on attrition shows that the …

The poster describes the Department of Civil and Environmental Engineering’s process of assessing program outcomes. The steps in the assessment process are first described followed by a list of program outcomes. Only direct modes of assessment are used. The frequency of assessment is tabulated. The poster then expands on the use of performance appraisal in select courses, which represents a relatively new mode of assessment that the department has recently adopted. Performance appraisal is performed once every three years per outcome and is advantageous in that it can be used as a vehicle to involve more faculty members and facilitate faculty buy-in to the assessment process. Detailed is the schedule involved in the performance appraisal assessment process along with a sample scorecard and a sample evaluation of one outcome. The sample scorecard contains the concepts that were evaluated along with the performance criteria. After one cycle of assessment of using performance appraisal, the Department is interested to know whether the changes implemented will result in any program improvement. It is envisioned this continual process of assessment will lead to continuous quality improvement.

Assessment of Program Outcomes

The poster describes the Department of Civil and Environmental Engineering’s process of assessing program outcomes. The steps in …

Empirical data in relation to students' satisfaction with tutoring sessions were gathered with an open-ended survey from one semester. Themes were extracted to generate representative items. These items now comprise an assessment form with outcomes based on students' expressed needs. This is an example of formative assessment using both qualitative and quantitative data

Empirically Based Assessment

Empirical data in relation to students’ satisfaction with tutoring sessions were gathered with an open-ended survey from one …

LILO (Learning Information Literacy Online) is a project of the UH Libraries Information Literacy Committee (UHLILC) for the UH System. LILO is a productivity application that guides students’ research process using library and Web resources. Students create and add to an online Research Journal, documenting their research on a topic over a period of weeks. Assessment is central to LILO. Most questions in LILO can be assessed using over 200 4-level embedded rubrics. See How Others Answered This Question links above text input boxes provide actual student responses modeling each rubric level, and defining what constitutes excellent, satisfactory, partial, or incomplete answers. LILO questions are mapped to national information literacy standards and outcomes developed by the Association for College and Research Libraries of the American Library Association. Instructors participating in LILO may now request an Information Literacy Assessment Report rating each student's response for any or all LILO modules used, along with a discussion section describing how students did overall.

Assessing Information Literacy Online

LILO (Learning Information Literacy Online) is a project of the UH Libraries Information Literacy Committee (UHLILC) for the …