surveys

First-Year Programs (FYP) utilizes multiple approaches to assess student learning outcomes and program success. Institutional data are used to measure retention rates for students participating in Access to College Excellence Learning Communities (ACE); National Student Clearing House (NSCH) data are gathered to measure student transfer rates. Qualitative data and feedback is collected through focus groups and surveys. In addition, ACE students complete two surveys measuring student expectations, engagement, and institutional commitment. Fall 2007 student engagement survey results indicate ACE students felt significantly more informed about core graduation requirements, major requirements, and registration procedures. Students also felt significantly more connected to the university community. NSCH data indicates that a large proportion of ACE students that did not continue at UHM transferred to other institutions after their first year. FYP will expand its evaluation by collecting Drop-Failure-Withdraw (DFW) rates for classes offered as part of an ACE learning community.

First-Year Success: Evaluating a Peer-Led Learning Community Program

First-Year Programs (FYP) utilizes multiple approaches to assess student learning outcomes and program success. Institutional data are used …

The poster gives the Assessment Tools that we use in our department to assess our department Program Objectives and Outcomes. We have both Internal and External Assessment Tools and within each category, we have either Direct or Indirect tools. The Internal tools are those assessed within our department and External are those assessed outside of our department. Direct assessments are those assessed by sources other than students, and Indirect are those assessed by the students or the Alumni. In addition to the Surveys that are direct questions such as “how well a particular Outcome is achieved?”, we have developed Rubrics based on those Outcomes which are basically metrics that dissects our Outcomes into a number of Concepts which are broken, each, into a number of Performance Criteria, which, in turn, are measured, each, by various levels of achievements such as 1 being Worse and 4 being Best (i.e., the Rubrics).

ABET (Accreditation Board of Engineering and Technology) Assessment Tools

The poster gives the Assessment Tools that we use in our department to assess our department Program Objectives …

Each year, the Office of Student Housing Services conducts an assessment to measure our resident’s perspective of various aspects of our housing program. The information obtained is then used to prioritize planned improvements to our housing program; assess progress toward departmental and University goals and objectives; plan training programs; improve business services; develop budgets; and enhance communication strategies with various stakeholder groups.

Student Housing Services Annual Assessment, 2009-2010

Each year, the Office of Student Housing Services conducts an assessment to measure our resident’s perspective of various …

Two years ago, the staff of the Office of Student Life and Development (SLD) began the process of identifying the learning outcomes that guide and inform our in- and out-of-classroom instruction of students. The SLD staff provides advising to UHM’s six Chartered Student Organizations and their programs and teaches introductory and upper division college credit leadership courses, offered through the College of Engineering, Interdisciplinary Studies, and the College of Education. The focus of this poster presentation is two activity sheets designed by the presenters to enable SLD staff to introduce their students, in the organizations which they advise and the classes they teach, to three foundational learning outcomes to be assessed this academic year: self awareness, building relationships, and empowerment of others. The first sheet asks students to rate their leadership knowledge and skill development – emerging, developing, and advanced – in the three learning outcome areas. The second sheet, Strategies for Growth, asks students to select one or more learning outcomes, articulate specific, action-oriented, and time-framed goals, and to consider how the leadership opportunities offered through their organizations and courses could assist them in progressing to the advanced leadership skill level. Recommended processes for staff implementation include individual advising and group discussion and problem-solving activities.

LEVELS OF LEADERSHIP: Developing Strategies for Improving Competency in Three Core Learning Outcomes

Two years ago, the staff of the Office of Student Life and Development (SLD) began the process of …

Due to the growing diversity of the United States population, various legal mandates and accrediting bodies require doctors to receive training that will allow them to provide optimal care to patients regardless of their cultural backgrounds. More prevalent in medical specialties such as family medicine and psychiatry, the inclusion of cultural competency or cross-cultural care issues in surgery is an emerging area of recognized need. A nationwide survey on residents’ perceptions of their preparedness to provide cross-cultural care revealed that surgical residents view cultural knowledge as important, but were hampered in acquiring these skills due to a lack of time and absence of formal training. The Accreditation Council for Graduate Medical Education requires residency programs to incorporate cultural training into their curricula. Of the six competencies, cultural competency is addressed under both Professionalism and Interpersonal and Communication Skills. With the assistance of the UH Department of Family Medicine and Community Health, the UH Department of Surgery’s general surgery residency program has developed and is currently piloting a cultural standardized patient exam as a means to both train and evaluate its residents. The case scenario involves a surgeon attempting to obtain informed consent from an elderly Samoan male who needs to have his leg amputated or face certain death. All Program Year-1 (PGY-1) residents (n=13) participated in the exam and were evaluated by one of four faculty preceptors using a standardized tool. Using this as the baseline, an educational training intervention is being planned, which will be followed by a post-test.

The Development of a Cultural Standardized Patient Exam for a General Surgery Residency Program

Due to the growing diversity of the United States population, various legal mandates and accrediting bodies require doctors …

Empirical data in relation to students' satisfaction with tutoring sessions were gathered with an open-ended survey from one semester. Themes were extracted to generate representative items. These items now comprise an assessment form with outcomes based on students' expressed needs. This is an example of formative assessment using both qualitative and quantitative data

Empirically Based Assessment

Empirical data in relation to students’ satisfaction with tutoring sessions were gathered with an open-ended survey from one …

Several studies have revealed that successful mentoring affects college student retention. Also, research on attrition shows that the first year is a critical stage in students' decisions to persist or leave college. Nora & Crisp have recently identified four essential elements to mentoring: psychological/emotional support; support for setting goals and choosing a career path; academic subject knowledge support aimed at advancing a student's knowledge relevant to their chosen field; and specification of a role model. In the 2007-2008 Academic Year, over 400 first-year students in English 100 at UHM were mentored by MA or PhD students in English, and analysis of their standardized end-of-term evaluations revealed that students identified each of the elements above in varying degrees, with “academic subject support knowledge” identified 79% of the time and with at least one of these elements identified 85% of the time. These results suggest that mentoring when delivered as part of a course holds great potential for boosting student retention at UHM.

Assessing Mentoring in First Year Composition: A Tool to Boost Retention?

Several studies have revealed that successful mentoring affects college student retention. Also, research on attrition shows that the …

The poster gives the Assessment Tools that we use in our department to assess our department Program Objectives and Outcomes. We have both Internal and External Assessment Tools and within each category, we have either Direct or Indirect tools. The Internal tools are those assessed within our department and External are those assessed outside of our department. Direct assessments are those assessed by sources other than students, and Indirect are those assessed by the students or the Alumni. In addition to the Surveys that are direct questions such as “how well a particular Outcome is achieved?”, we have developed Rubrics based on those Outcomes which are basically metrics that dissects our Outcomes into a number of Concepts which are broken, each, into a number of Performance Criteria, which, in turn, are measured, each, by various levels of achievements such as 1 being Worse and 4 being Best (i.e., the Rubrics).

ABET (Accreditation Board of Engineering and Technology) Assessment Tools

The poster gives the Assessment Tools that we use in our department to assess our department Program Objectives and …

The College of LLL (CLLL) formed an Evaluation Resource Team (LLL-ERT) to (a) create evaluation capacity and a useful and sustainable evaluation system; (b) conduct useful program evaluation practice to meet department internal needs as well accreditation demands; and (c) facilitate departments to take actions based on evidence. During the 2008-09 academic year, CLLL is collecting student opinions from graduating students via an online student exit survey in order to illuminate and improve various elements of CLLL programs. Program elements include curriculum, advising, intake and exit processes, learning environment and resources, program delivery, and student learning and teaching. Each department prioritized evaluation foci (i.e., two-year language requirement, BA, MA, and/or PhD), and is creating department-specific questions. A set of College-wide questions, based on faculty and student opinions, was created so that data can be accumulated in the coming years. The poster summarizes the mission of the LLL-ERT and provides an overview and plan for the CLLL evaluation project.

Enhancing Evaluation of CLLL Programs: Developing an Online Student Exit Survey System

The College of LLL (CLLL) formed an Evaluation Resource Team (LLL-ERT) to (a) create evaluation capacity and a …

In Spring 2013, the Matsunaga Institute for Peace and Conflict Resolution (MIPCR) began a program review process by completing a curriculum map of the Graduate Certificate in Conflict Resolution (GCCR). The 15-credit interdisciplinary certificate is designed to address a set of program-level student learning outcomes (SLO). A curriculum assessment would determine if the GCCR’s Peace and Conflict Education (PACE) courses, as well as GCCR-approved courses from other programs, aligned with these outcomes. It would also reveal whether students had adequate opportunities to achieve the outcomes. This initiative would open dialogue among faculty, administrators, and students to discuss student success and program improvement. Instructors of PACE and GCCR-approved courses received a survey listing each program-level SLO and were asked to indicate the degree to which their course addressed each outcome. Responses to the surveys were recorded in a curriculum map and presented to a volunteer review committee for interpretation and discussion. This committee made recommendations for the program, including clarifying program goals and positioning to guide program-level SLOs, and further clarifying SLOs to ensure collective understanding by faculty members. Committee members also noted that there are limited opportunities for students to focus on professional ethics. It was further recommended that faculty and staff ownership of the review process be fostered through transparency and collaboration. Challenges of meeting SLOs through interdisciplinary programming were also revealed. The MIPCR Curriculum Committee discussed the results and recommendations and concluded that a more complete curriculum map and continued discussion was needed before making any curricular decisions.

Starting the Curricular Conversation through Mapping: A Curriculum Review of the Graduate Certificate in Conflict Resolution

In Spring 2013, the Matsunaga Institute for Peace and Conflict Resolution (MIPCR) began a program review process by …

Over the past several years the Department of Speech has collected data from Speech majors prior to their Fall or Spring graduation. Graduating majors complete an assessment instrument consisting of 50 multiple choice items culled from exams from all courses required to be taken by Speech majors. Questions are approximately evenly distributed across the 6 required courses. Scores are compared to those obtained from a sample of SP 151 students, i.e., largely, but not entirely, first year students who have had little to no experience in Speech courses other than SP 151. Graduating majors also complete a self-report survey assessing their perceptions of the department, the faculty, and their level of learning and achievement in the major.

Assessment of Retention of Content from Required Courses Using Comparison Groups

Over the past several years the Department of Speech has collected data from Speech majors prior to their …

Using Multiple Assessment Strands

Shidler College of Business -800 declared undergraduate majors and 550 graduate students.-12 academic degree programs plus executive programs …

Assessment Tools for Affirmation and Improvement

The Myron B. Thompson School of Social Work -Approximately 20 students graduate with a Bachelor’s degree in Social …

Assessing Civic and Community Engagement in Ethnic Studies

Ethnic Studies was born out of community demand and community engagement has had a central place in the …

Cross-Cultural Classroom: Assessing Student Opinion of Online, Case-Based Learning Modules, Utilizing MRI and XR Technology, in American and Turkish Medical Students

The objective of this study was to develop online case-based anatomy learning modules and assess student perceptions of …

Incorporating Learning Outcome Assessments into Noncredit Education: A Novel Approach

Quantitative assessments in noncredit education typically evaluate participant’s satisfaction or experiences with courses and the instructor. Learning outcomes …

UH Online 5-Week Professional Development (PD) Program Evaluation and Iterative Improvements

The annual University of Hawaiʻi (UH) Online 5-Week Professional Development (PD) Program was created in 2018 in alignment …

Developing Learning Assessment Surveys

This workshop introduces attendees to survey questions that reveal respondents’ perceptions of learning, and it provides information on …

Access to College Excellence (ACE) at UHM provides a series of first year programs that serve approximately 25% of incoming freshmen a year. A major ACE program is Learning Communities, in which ACE staff helps students to register a collection of courses in a cluster (e.g., general education, pre-business) and an Access to College Community freshman seminar course: CAS 110. All the students who registered for the courses in one cluster form a cohort—a learning community. CAS 110 enhances the community by helping students reaching the following learning outcomes: (1) building effective study and time management skills; (2) setting goals for success; (3) learning about the academic resources available; (4) learning core and major graduation requirements at UHM. This poster introduces the ACE learning communities and how it assesses the outcomes by using a pre- and post-survey method. The poster will present quantitative results from the closed-ended questions and qualitative results from the open-ended questions. The presenter will also present the impact of ACE Learning Communities by comparing the retention rates between the participants and non-participants, and between the less-engaged and more-engaged participants.

Assessment of ACE Learning Communities 2013

Access to College Excellence (ACE) at UHM provides a series of first year programs that serve approximately 25% …

Access to College Excellence (ACE) at UHM provides a series of first year programs that serve approximately 25% of incoming freshmen a year. A major ACE program is Learning Communities, in which ACE staff helps students to register a collection of courses in a cluster (e.g., general education, pre-business) and an Access to College Community freshman seminar course: CAS 110. All the students who registered for the courses in one cluster form a cohort—a learning community. CAS 110 enhances the community by helping students reaching the following learning outcomes: (1) building effective study and time management skills; (2) setting goals for success; (3) learning about the academic resources available; (4) learning core and major graduation requirements at UHM. This poster introduces the ACE learning communities and how it assesses the outcomes by using a pre- and post-survey method. The poster will present quantitative results from the closed-ended questions and qualitative results from the open-ended questions. The presenter will also present the impact of ACE Learning Communities by comparing the retention rates between the participants and non-participants, and between the less-engaged and more-engaged participants.

Learning Through Community: Best Practices and Assessment

Access to College Excellence (ACE) at UHM provides a series of first year programs that serve approximately 25% …

Five faculty in Department of Microbiology run five programs (BS/BA Microbiology; BS Molecular Cell Biology (MCB); MS/PhD Microbiology). One of three faculty members lost in 2104 had run assessment for many years. To 2011 this comprised submitting paper form to Assessment Office. Replaced by selecting ‘radio buttons’ and typing in online version. Faculty member new to assessment took over BS/BA Microbiology assessment in 2012. No BS/BA assessment activities due to time constraints, inexperience; 2012 and 2013 reports used as templates in 2013 and 2014, respectively. No BS/BA assessment undertaken. MCB assessment activities run by second faculty passed in 2014 to faculty reporting on BS/BA Microbiology. First assessment for latter degrees undertaken late in 2014, i.e., faculty provided Student Learning Outcomes (SLO) for their courses. SLOs for all required and elective BS/BA Microbiology courses mapped to revised Program Learning Outcomes through collaboration with Assessment Office in Spring 2015.

Assessment in the Department of Microbiology: Zero to…

Five faculty in Department of Microbiology run five programs (BS/BA Microbiology; BS Molecular Cell Biology (MCB); MS/PhD Microbiology). …

In Fall 2014 and Spring 2015, I surveyed the reactions of N = 58 undergraduate students in three sections of a business course, who were invited to pilot-test a web-based, 3-D, interactive game. Entitled “Marketing Mayhem,” this UK-based game is designed to assess business students’ knowledge and application of ethical principles. I conducted descriptive analysis of the quantitative and qualitative data and found some mixed results on the suitability, usability and effectiveness of this 3-D game (as a tool for assessing of business ethics principles). Students suggested several improvements to the content and/or format of the game to make it more user-friendly and more tailored to US business ethics and related regulations. Implications for SLO assessment and curriculum development, as well as for business ethic game development were discussed.

Assessing Students’ Business Ethics Using a Web-based 3-D Game

In Fall 2014 and Spring 2015, I surveyed the reactions of N = 58 undergraduate students in three …

This poster highlights the work of University of Hawaii at Manoa 2013 Information Literacy Working Group. Tasked by the Manoa Faculty Senate, the General Education Committee established this working group that (1) reviewed definitions of information literacy, (2) reviewed academic department’s undergraduate Student Learning Outcomes, (3) reviewed courses’ information literacy aspects, and (4) surveyed writing intensive instructors on information literacy. The poster will present the curriculum map created by the group. The map included courses for the 2012-2013 academic year that had (1) a library instruction workshop, (2) Student Learning Outcome addressing some aspect of information literacy, or (3) writing intensive designation. The poster will also share the results of an information literacy survey sent to a stratified sample of instructors who taught writing intensive courses during the 2012-2013 year.  Observations and recommendations of the working group will also be presented.

Information Literacy Across the Curriculum

This poster highlights the work of University of Hawaii at Manoa 2013 Information Literacy Working Group. Tasked by …

The Learning Assistance Center (LAC) offers academic support, e.g., tutoring and supplemental instruction (SI) study groups, for students in traditionally challenging and targeted gateway courses. Unlike academic degree programs, the LAC, a co-curricular program, cannot utilize direct assessments of student work to determine program effectiveness; therefore, assessment of LAC programming must rely on triangulated data from multiple sources to determine its impact on student learning and success. After five years of data strongly suggesting that the LAC has had a positive impact on students who use our services, the LAC is poised to expand its programming to help a wider variety of students achieve their academic goals.

Assessment: Informing Co-curricular Programming from Past to Present to Future

The Learning Assistance Center (LAC) offers academic support, e.g., tutoring and supplemental instruction (SI) study groups, for students …

Institutions that understand what helps and hinders student learning will know how to improve student proficiency in core academic areas that are important for college and workforce success. Research supports learning as a holistic (Pascarella and Terenzini, 2005), complex, and domain-specific process (Beyer et al., 2007). Institutions are encouraged to employ principles of learning that are domain-independent, experience-independent, and culturally relevant (Ambrose et al., 2010). It is important that institutions continue to examine a variety of (new) learning tools and innovative pedagogical approaches in order to improve or evolve undergraduate teaching methods, curriculum, and experiences. This study provides important descriptive evidence for understanding undergraduate student learning across campus and over time. The results provide specific ways faculty and administrators may teach and implement services to better serve students.

Factors and Experiences that Help and Hinder Undergraduate Student Learning

Institutions that understand what helps and hinders student learning will know how to improve student proficiency in core …

The online teaching award began at Manoa during the 2013-2014 academic year. This poster will present the positive and negative aspects of student nominated awards, present student data, summarize the results of the program, and present what is needed for the program to continue, as well as the future value.

Student nominated online teaching awards at Manoa: What student satisfaction surveys tell us

The online teaching award began at Manoa during the 2013-2014 academic year. This poster will present the positive …

The Come Back to Mānoa program, established in Summer 2014 by the University of Hawaiʻi at Mānoa and housed administratively in Outreach College, assists undergraduate seniors who stopped attending college to return and graduate. This poster provides an overview of the program’s development, including creating mission and vision statements and objectives; outreach efforts and participation data; and plans to implement an assessment survey of respondents. In addition to assessing outcomes, the survey evaluated why students left using broad categories from the UHM 2012 Leavers Survey and adding a few more reasons based on relevant literature. Most research on student attrition concentrates on first year and sophomore students, but very little literature focuses on senior attrition and how we might help those students persist to degree (Hunt et al., 2012). The information obtained from the survey is used to improve the Come Back to Mānoa program and contributes to the academic literature. Initial challenges and strategies are also presented.

Developing and Assessing the Come Back to Manoa Program: Why Seniors Leave and How to Help them Graduate

The Come Back to Mānoa program, established in Summer 2014 by the University of Hawaiʻi at Mānoa and …

Alumni and Employer Surveys for Program Improvement

This workshop shows how programs can conduct alumni and employer surveys for improvement. For the last 2 years, …

Student Surveys: Perceptions of Achievement

In this workshop, the presenters will take participants through a step-by-step procedure to create surveys about student learning …

Focus Group, Interview, or Survey: Which is Right for Your Academic or Co-Curricular Program?

This workshop will describe the purpose, strengths, and weaknesses of each data collection method. For Academic Programs: PowerPoint …

Designing Effective Surveys

This workshop will introduce you to the basics of survey design, from planning to question construction and layout. …

Using Surveys for Understanding and Improving University Educational Programs

This workshop provides advice on using surveys in university departments/programs, focusing on assessment project planning, survey development, and …

Analyzing Open-Ended Survey Responses – Where to start?

In this workshop you will learn how to carry out a preliminary analysis of open-ended survey responses; how …