Unit: Office of the Vice Chancellor for Academic Affairs
Program: Mānoa Advising Center
Date: Mon Sep 29, 2014 - 12:25:49 pm

1) Below are your program's student outcomes (SOs). Please add or update as needed.

1.  Students can identify major options

2.  Students can learn, identify, and understand General Education, graduation and program requirements using advising combined with supplemental services (MyUH, STAR, catalog, registration guide and program sheets.)

3.  Students can identify campus resources available to them and use them, as necessary (Career Center, CSDC, LAC, tutoring, etc.)

2) Your program's SOs are published as follows. Please update as needed.

Program's Website. URL: http://manoa.hawaii.edu/undergrad/mac/
Student Handbook. URL, if available online:
Information Sheet, Flyer, or Brochure. URL, if available online:
UHM Catalog. Page Number:
Other: Advising syllabus (on MAC website)
Other:

3) Provide the program's activity map or other graphic that illustrates how program activities/services align with program student outcomes. Please upload it as a PDF.

Activity Map File(s) from 2014:

4) Did your program engage in any program assessment activities between June 1, 2013 and September 30, 2014? (e.g., establishing/revising outcomes, aligning activities to outcomes, collecting evidence, interpreting evidence, using results, revising the assessment plan, creating surveys, etc.)

Yes
No (skip to question 14)

5) For the period between June 1, 2013 and September 30, 2014: State the assessment question(s) and/or assessment goals. Include the student outcomes that were targeted, if applicable.

We continued to use our SurveyMonkey mandatory advising assessment survey for another semester to see whether responses continued to be the same or yield different responses. Our goals were to assess whether our mandatory advising efforts are successful or not. We met with Monica Stitt-Bergh to analyze the responses. 

6) State the type(s) of evidence gathered to answer the assessment question and/or meet the assessment goals that were given in Question #5.

In order to complete their mandatory advising requirement (and have their registration hold lifted), all students were required to not only complete a mandatory advising option, but also complete a mandatory advising survey. 

In addition, we continue to administer walk-in evaluations for our MAC advisor and peer advisor mandatory advising in order to monitor the perceived quality of advising as well as obtain suggestions for improvement. 

7) State how many persons submitted evidence that was evaluated. If applicable, please include the sampling technique used.

In Fall 2013, 1089 students (out of 1116) with mandatory advising completed the survey. This reflects a 97.58% completion rate. The 27 students who did not complete either completed it after January 31, 2014 or did not register in the Spring.

In Spring 2014, 838 students (out of 907) with mandatory advising completed the survey. This reflects a 92.39% completion rate. The lower number of students in Spring is attributable to the fact that students in the Student Support Services and the Student Athlete Academic Services, due to their nationally regulated requirements, were considered to have fulfilled their mandatory advising.

Mandatory advising has only been required of incoming freshmen and sophomore students for their first four semesters. In Fall 2014, in an effort to spread mandatory advising to all of MAC's students, we are requiring freshmen and sophomore transfer students to complete mandatory advising (as well as the survey.) As a result, the numbers should rise again.

8) Who interpreted or analyzed the evidence that was collected? Check all that apply.

Program faculty/staff member(s)
Faculty/staff committee
Ad hoc faculty/staff group
Director or department chairperson
Persons or organization outside the university
Students (graduate or undergraduate)
Dean or Associate Dean
Advisory Board
Other: Monica Stitt-Bergh also assisted us

9) How did he/she/they evaluate, analyze, or interpret the evidence? Check all that apply.

Compiled survey results
Used quantitative methods on student data (e.g., grades, participation rates) or other numeric data
Used qualitative methods on interview, focus group, or other open-ended response data
Scored exams/tests/quizzes
Used a rubric or scoring guide
Used professional judgment (no rubric or scoring guide used)
External organization/person analyzed data (e.g., Social Science Research Institute)
Other:

10) For the assessment questions/goals stated in Question #5, summarize the actual results.

We discovered that findings were fairly consistent with the previous semester. Students report having learned more about majors, resources and resource offices. This held true for most MAC mandatory advising options as well as non-MAC mandatory advising options, which would show that the options that we have identified for our students are effective in meeting our SLOs.

11) What was learned from the results?

There were two interesting findings. 

One had to do with the fact that students reported already having known quite a bit about resources such as MyUH portal, the catalog, STAR and program sheets. This could also have something to do with the fact that many students come in for advising during the summer, before their mandatory advising. 

Also, the sophomore experience options (on-line assignment as well as peer advising) both rated the lowest in terms of alignment with the SLOs. Upon reflection, we realized that this had to do with the fact that these options were meant to focus on different content entirely.

12) State how the program used the results or plans to use the results. Please be specific.

We adjusted the assessment questions to better reflect our goals. 

This semester, we are trying to promote one specific part of STAR - the "What If" function, which is the best tool to help students select a major. We are curious to see if we can make a noticeable difference by integrating this into the advising, assignments and workshops that MAC runs.

Also, we decided to utilize skip logic in our mandatory advising survey so that students who select the sophomore experience option can reply to questions that assess their specific option's learning outcomes.

13) Reflect on the assessment process. Is there anything related to assessment procedures your program would do differently next time? What went well?

Yes. This process (and Monica) have assisted us in recognizing and affirming that assessment is an important tool to help us refine our efforts with mandatory advising. Outcomes can help us refocus our efforts in areas that may need more attention. Our campaign to assess whether students are learning about the "What If" option in STAR, is a good example of this.

Also, the fact that students report coming to mandatory advising with quite a bit of knowledge made us wonder if we are overlooking the advising they receive when they first arrive at UHM.

In addition to online mandatory advising assessment, we still ask students to fill in paper evlauations when they come in for appointments. We do this to assess the perceived quality of the advising the students are receiving as well as obtain valuable suggestions and feedback. In the past, we did not have students fill in evaluations during our first-come, first-served walk-in advising period because of the busy nature of this time. However, with the more manageable numbers, we have decided to start administering evaluations year-round.  

We feel that the assessment was helpful. Until now, we have offered mandatory advising only to our incoming freshmen and sophomore students. Since we now have more manageable numbers, we have decided to expand our mandatory advising requirement to our freshmen and sophomore transfer students as well. They will be required to complete a mandatory advising option and also the mandatory advising survey. It will be interesting to be able to compare the transfer students' responses with the incoming student responses.

14) If the program did not engage in assessment activities, please explain.
Or, if the program did engage in assessment activities, please add any other important information here.