Program: Native Hawaiian Student Services
Date: Fri Oct 12, 2012 - 10:18:36 am
1) Below are your program's student outcomes (SOs). Please add or update as needed.
Student Learning Outcome 1: Create and Promote a Hawaiian Place of Learning
Hawaiian places of learning are intentional and organic spaces that give power to Hawaiian perspective; welcome multiple Hawaiian identities; promote collaboration, networking and organizing; encourage mentorship; & serve as pu‘uhonua for Hawaiian students and community.
Student Learning Outcome 2: Promote and Advocate Hawaiian Language, Studies & Education
In addition to the recruitment of students to our HSHK majors, NHSS promotes and advocates for the teaching and learning of Hawaiian knowledge to be used as the medium for students to better understand themselves (as a Hawaiian), their education, their Kuleana, their community, and their nation.
Student Learning Outcome 3: Recognize our Kuleana to Honor the Traditions of our Indigenous People and to Promote Social Justice for Native Hawaiians
NHSS empowers students and advocates for social justice by supporting student organizing efforts; helping students recognize the intersectionalities of oppression; promoting community access and human service; and investing in student leadership development.
Student Learning Outcome 4: Educate a Highly Knowledgeable, Skilled, Flexible, World-Class Native Hawaiian Community/Nation/Transformative Force
NHSS hopes to produce a highly knowledgeable, skilled, flexible, world-class Native Hawaiian student community by teaching students how to learn; providing students with transformative skills; providing opportunities for student professional development; and fostering an entrepreneurial spirit in our students.
Organizational Learning Outcome 5: Learning Organization
NHSS strives to be an entity that responds to research trends on effectively serving Hawaiian students, community, and nation.
2) Your program's SOs are published as follows. Please update as needed.
Program's Website. URL: facebook.com/nativehawaiianstudentservices/info
Student Handbook. URL, if available online: Information Sheet, Flyer, or Brochure. URL, if available online: UHM Catalog. Page Number: Other: Posted in our two Student Resource Centers at QL 104 and KAMA 211 Other: Included in the Hawai‘inuiākea program literature, including our World Indigenous Nations Higher Education Consortium (WINHEC) accreditation report and presentation materials. We also use it to guide all of our grant writing and research endeavors, so it appears in those types of publications. 3) Provide the program's activity map or other graphic that illustrates how program activities/services align with program student outcomes. Please upload it as a PDF.
4) Did your program engage in any program assessment activities between June 1, 2011 and September 30, 2012? (e.g., establishing/revising outcomes, aligning activities to outcomes, collecting evidence, interpreting evidence, using results, revising the assessment plan, creating surveys, etc.)
Yes
No (skip to question 14)5) For the period June 1, 2011 to September 30, 2012: State the assessment question(s) and/or assessment goals. Include the student outcomes that were targeted, if applicable.
Our assessment activities and assessment evidence collected spans all of our 5 Student and Organizational Learning Outcomes. Ultimately, our assessment efforts attempt to better understand the ways in which (and the degree to which) students are engaging in the activities we design for them, and whether that engagement is impactful. We try to measure impact around our work approach: the ways students better understand or think differently about issues like Hawaiian identity, self-determination, kuleana (privilege and responsibility), nationhood, and Hawaiian “culture” and language.
Our assessment activities are usually designed around the program activity. For our high touch-programs like our Internship Program (12+ weeks) and our Summer Enrichment Program (residential 2 weeks), we collect student work and do in-depth interviews. These high-touch programs also involve multiple Student Learning Outcomes, which we try to elicit through the different evaluation activities.
6) State the type(s) of evidence gathered to answer the assessment question and/or meet the assessment goals that were given in Question #5.
● Sign-in logs: student spaces, activities/programs, events, tutoring, etc.
● Staff advising logs
● Final student presentations for the Internship Program & Summer Enrichment Program
● Student artifacts (e.g. presentations, portfolios, journals, etc.)
● Student Participant Interviews
○ 7 Summer Internship Program participant interviews from Summer 2011 cohort and 4 participant interviews from the first Summer 2012 cohort (ongoing)
○ 9 Summer Enrichment Program participant interviews from Summer 2012 cohort
○ 1 focus group of 8 Hawaiian students who attended NAISA 2012 conference
● Tracking database for participants in Internship Program and Summer Enrichment Program to track their institutional enrollment, persistence, and graduation data.
● Evaluation Reception hosted by Native Hawaiian Student Services inviting back students in our high-touch programs like the internship program and Summer Enrichment Program. Students developed critical life maps describing their experience in our programs, did a self-guided interview, and completed a survey.
7) State how many persons submitted evidence that was evaluated. If applicable, please include the sampling technique used.
All 10 Native Hawaiian Student Services staff submitted evidence that was evaluated by the two unit assessment leads, Dr. E. Kahunawai Wright (Director, Native Hawaiian Student Services) and Nalani Balutski (Research & Evaluation Coordinator).
The other staff members who submitted evidence were: C. Lehua Nishimura (Hawaiian Studies Academic Advisor), C. Ululani Oliva (Graduate Support Specialist), S. Kamuela KaʻAhanui (Hawaiian Language Academic Advisor), Pearl Wu (Hulili Program Coordinator), Ileana Ruelas (Outreach Coordinator), J. Mahealani Quirk (Bridge Coordinator), and J. Kalika Wong (Kakoʻo Coordinator).
8) Who interpreted or analyzed the evidence that was collected? Check all that apply.
Program faculty/staff member(s) Faculty/staff committee Ad hoc faculty/staff group Director or department chairperson Persons or organization outside the university Students (graduate or undergraduate) Dean or Associate Dean Advisory Board Other: 9) How did he/she/they evaluate, analyze, or interpret the evidence? Check all that apply.
Compiled survey results Used quantitative methods on student data (e.g., grades, participation rates) or other numeric data Used qualitative methods on interview, focus group, or other open-ended response data Scored exams/tests/quizzes Used a rubric or scoring guide Used professional judgment (no rubric or scoring guide used) External organization/person analyzed data (e.g., Social Science Research Institute) Other: 10) For the assessment questions/goals stated in Question #5, summarize the actual results.
Our work is driven by our core values which was the foundation for our 4 Student Learning Outcomes and our 1 Organizational Learning Outcome. To demonstrate how our work aligns with our core values and our Learning Outcomes, we included exemplars of our work and corresponding assessment pieces that best address these values. The numbers below provide a snapshot of students who utilize some of our various resources and services between June 1, 2011 and September 30, 2012.
SLO #1: Create and promote a Hawaiian place of learning
Resource Centers
NHSS oversees two Resource Centers, which provide about 1,300 square feet of student space where students study, access computers, launa (relax), or seek other NHSS services like tutoring and career/personal/academic counseling. We use Survey Monkey as an electronic sign in for our two spaces. All students who visit the spaces sign in. We analyze the data at the end of reporting periods and do pivot tables on frequencies and periodically will query the UH System ODS database to learn more about our student visitors (we ask for their UH username and reason for visit), such as major and academic level. During this reporting period, there were 9,613 sign ins at our Resource Centers, of which 2,362 were at our Kamakakūokalani space, and 7,251 at our Queen Liliʻuokalani space.
Mālama Outreach Program
During the Fall 2011, NHSS launched a pilot program called Mālama. Mālama is designed to provide career and college readiness support to high school and UH Mānoa college students through ongoing interactive activities. Mālama fosters kuaʻana-kaikaina (elder sibling, younger sibling) relationships between high school students and current UH Mānoa students and between current UH Mānoa students and NHSS (as well as our partners). As part of the Mālama program, students interact with NHSS in three ways:
· UH Mānoa campus visits: NHSS partners with targeted high schools (Nānākuli, Waiʻanae, Hālau Kū Māna, Kailua, and ʻĀnuenue) and each high school brings 10-25 students to be paired with a current UH Mānoa student majoring in a similar field of study.
· Community service days: NHSS intentionally employs the ʻāina, the land, as the teacher and context for learning because it is culturally relevant. All participating high schools, including students and teachers, are invited to attend.
· Career exploration events: NHSS partners with Kamehameha Schools Career and Post-High Counseling to engage students with professionals in a spectrum of vocational fields to invest in the practice of connecting higher education with career pathways.
SLO #2: Promote and advocate Hawaiian language, studies and education
Student Professional Development
NHSS is committed to increasing training and leadership development opportunities for (1) Hawaiian Language and Hawaiian Studies undergraduate and graduate students, and (2) Native Hawaiian students, regardless of their major. During this reporting period, NHSS supported 12 students to attend two professional conferences: 4 to attend NASPA (Student Affairs Professionals) in Phoenix, Arizona in March 2012 and 8 to attend NAISA (Native American Indigenous Studies Association) in New London, Connecticut in June 2012.
To better support students, NHSS has begun to organize trips to professional conferences where NHSS staff attend with students and organized mentorship activities at the conference. For most participants, it was their first time presenting at national professional conferences, so, mentorship was vital to helping students navigate, discuss, engage, and reflect on their learning. For the NAISA event, NHSS invited Hawaiian Studies & Hawaiian Language undergraduate and graduate students to submit a research paper to NHSS, and we submitted a panel presentation with 3 students and 1 staff per panel (2 other students were on different panels). The panels were accepted, and NHSS worked with students to further develop their papers and presentations. At NAISA, NHSS staff conducted a short focus group where students shared their reflections on their learning and development at the conference and with presenting at a national conference. Students reported that the professional development experience did the following:
● Solidified the importance of professional development, research, and presenting
● Built relationships and networks
● Built confidence in their own work
● Connected them to Indigenous peoples beyond Hawai‘i
Here are some quotes from that conversation:
● “I think this is a good way to see what other people are doing and to see...what kind of possibilities your research has....and have a chance to present our work.” (SLO #4)
● “I’m going to take away from this experience...the professional development experiences that will be able to correlate into my future plans...I don’t think I would have been able to acquire them elsewhere because this conference was specific for indigenous peoples....Being here with other indigenous peoples, and learning how to come to resolutions between each other, I think that’s very valuable.” (SLO #3)
● “I’m really grateful for this opportunity...I feel like I really got a lot out of it. It was great to be at this conference in particular because of the relevance of all of the presenters’ topics to stuff I’m stuff that I generally study in school. Not only that, but the high percentage of Hawaiian presenters. Sadly I had to come this far to see what they were working on, but I was really inspired to see where some of these students, especially some of the PhD students, are taking Hawaiian research.” (SLO #2, 4)
● “It’s nice to come to events like this and be reminded that we’re not the only ones struggling with the things we’re struggling with, and not the only ones who are committed to fighting against these struggles, and standing up for our people while doing this really important work.” (SLO #3)
● “The validation and confirmation from colleagues and even just other scholars that I’ve never met before, that I am on the right track and they’re interested in the work that I’m doing...to be given that confirmation and validation from people I respect is very motivating to know that I can go home and continue to work, continue down the path I’m on.” (SLO #4)
NHSS plans to do a follow-up survey or interview at the end of the year, where we can gauge the impact that professional development complemented their classroom learning.
Student Counseling/Advising
NHSS created an internal log/survey that captures real-time advising sessions by our 6 staff members that meet/advise/counsel students (technically 4 staff, because 2 staff members were just recently hired, but they will start to add their data to our general log soon). The log captures student UH username as well as the nature of NHSS staff member interaction with them (what kind of advising: career, personal, academic, etc.). During the reporting period, there were 1,149 advising sessions with students conducted by NHSS staff. We believe this number is highly undercounted, as staff have not fully implemented this advising log. Our next steps include figuring out either an alternative way of capturing this advising information, or getting staff buy-in to complete them regularly and in a timely manner.
SLO #3: Recognize our kuleana to honor the traditions of Hawai‘i’s indigenous people and to promote social justice for Native Hawaiians
Summer Enrichment Program
The Summer Enrichment Program (SEP) is an intensive two-week, residence-based experience in which students take culturally-grounded classes, participate in community-based huaka‘i, learn about campus resources, and actively engage in discussions with Native Hawaiian scholars in different fields. The program is designed to provide culturally-grounded readiness activities to Native Hawaiian students entering the University of Hawai‘i at Mānoa from UH System community colleges.
During this reporting period, 10 students participated in the 2012 Summer Enrichment Program. Students in the program complete assignments which are posted on a closed blog site, write journals, complete critical life maps, and also prepare a capstone presentation about their learning and the ways their experience in the program has helped them better prepare to transition to UHM and solidify their academic and career plans.
NHSS Research & Evaluation Coordinator conducted one-on-one interviews with 9 of the 10 participants in Summer 2012 program. Analysis of this interview data is ongoing.
SLO #4: Educate a highly knowledgeable, skilled, flexible, established Native Hawaiian workforce
Internships
The goal of the NHSS Internship Program is to provide students with opportunities for hands-on meaningful workplace experience, to increase opportunities for gainful employment and/or pursuit of a higher degree. During this reporting period, there were 26 students who participated in our Internship Program, an increase from the 14 we served last year. NHSS received a new grant from the Office of Hawaiian Affairs to increase the number of Hawaiian students who participate in our Internship program. Students work with our Internship Coordinator to select an internship site, depending on their career interest and academic discipline. Students complete approximately 240-300 internship hours at their site and also attend mandatory classes with our Internship Coordinator, that focus on a myriad of student development areas. Students learn skills like resume writing and interviewing skills, but also engage in identity exploration activities, aligned with our NHSS approach and learning outcomes. There are several evaluation components of this Internship Program: (1) Internship site evaluation survey of student, (2) Student evaluation of internship site, (3) Student Portfolio, (4) Student Capstone Presentation, (5) an Internship database tracking student enrollment and graduation and (5) Intern Exit Interview. For the purposes of this report, we only focused on the Intern Exit Interview. NHSS staff conducted interviews of 7 students who participated in the Summer 2011 internship cohort. The interview protocol examined the ways in which students’ experience in the Internship program helped them reflect/interrogate/examine their Hawaiian identity, sense of clarity on career/life goals, provided transferrable skills, as well as a series of other elements. Students reported that the Internship program helped them with the following goals:
● Feel more confident, mature, and comfortable with self
● Raise critical questoins of life purpose and found that inquiry necessary for their growth
● Reaffirm goals, skills, and interests
● Clarify or consider their role in broader society and “find self”
● Answer questions like “Where do I want to go?”
● Identify next steps for their educational and career journeys
● Better understand “what it means to be Hawaiian” for themselves
Intern Exit Interviews for the Summer 2012 cohorts are ongoing, thus far, we have conducted interviews with 4 of the interns, but have not analyzed the data yet.
Tutoring
NHSS offers tutoring in Math, English and Hawaiian Language. Our student tutors have students complete a tutoring log through Google Docs with their UH username, the class they need help with, as well as their contact information. This information is used to track how many students receive tutoring from our NHSS Tutoring program. During this reporting period, 304 tutoring sessions were conducted by our student tutors. Additionally, NHSS staff can periodically query pre-and-post GPA for students in the courses theyʻre seeking tutoring. This information is required for one of our federal grants that support the tutoring program (reported on every December).
Organizational LO #5: Learning Organization
NHSS staff is actively involved in knowledge generation and dissemination in the fields of student services, student development, and institutional research on Native Hawaiians in the UH System. Our staff serve on a variety of UH committees including M.A. and doctoral student committees. Here’s a preview of some of our work:
· Community Organizations & Local Professional Organizations
o Papa Kū Māna Local School Board (Hālau Kū Māna Public Charter School)
o Hawaiʻi People’s Fund
o Hawaiʻi-Pacific Evaluation Association Board
o Protect Kahoʻolawe ʻOhana
· Professional Memberships
o American Educational Research Association
o American Evaluation Association
o Association for Institutional Research
o Hawaiʻi-Pacific Evaluation Association
o Hawaiʻi Public Health Association
o Student Affairs Administrators in Higher Education (NASPA)
o National Association for Social Workers
o Native American Indigenous Studies Association
o Native Hawaiian Education Association
· Original Scholarship (in 2011-2012 reporting period)
o 7 completed conference presentations, 2 accepted and pending conference presentations, and 3 conference presentations pending notification
o 2 book chapters
o 4 Native Hawaiian Student Profiles
o 1 journal article
· Grant Writing
o 1 new accepted from the Office of Hawaiian Affairs, 1 pending
11) What was learned from the results?
In review of our learning goals and associated data collected, these are the general trends or themes that surfaced:
· The Resource Centers continue to serve as significant points for Native Hawaiian students to utilize different resources;
· The Internship Program is successful in helping students develop their self-awareness and align life and career goals;
· The Summer Enrichment Program is successful in helping students feel more confident about transferring to UHM;
· NHSS Graduate Student Travel Scholarship Program is successful in providing valuable research and professional development opportunities to Hawaiian Studies & Hawaiian Language graduate students;
· Over the past year, the number of professional development opportunities for NHSS staff has grown significantly;
· Over the past year, the number of original scholarship by NHSS staff has grown exponentially.
· NHSS needs to continue to maintain (and possibly create new) spaces for Native Hawaiian students on our campus.
· NHSS needs to create mechanisms to measure student learning for its academic services (i.e. academic advising, graduate support).
· NHSS needs to expand the availability of professional development opportunities for Native Hawaiian graduate students.
· NHSS needs to expand the availability of research development opportunities for Native Hawaiian undergraduate and graduate students.
12) State how the program used the results or plans to use the results. Please be specific.
· In our OVCAA Assessment Report from last year, we identified two goals that we have made significant progress on: NHSS needs to expand the availability of (1) professional development opportunities for Native Hawaiian graduate students, and (2) research development opportunities for Native Hawaiian undergraduate and graduate students. In this current reporting period, we worked hard to meet this goal with extramural funds that supported student professional development, mainly travel to national conferences to present student research. We hope to continue this work and build in more mentorship opportunities to develop students’ research potential.
· Over the last several years, our assessment data has informed new and improved programming and grant writing initiatives. As such, we will continue to use our assessment findings to further develop the scope of our programs and services.
· Our unit’s research work has made inlays in our Hawaiian community, and we hope to further develop that work through the creation of a Research Center. Our current Research & Evaluation Coordinator is on soft-money that will be expiring in September 2013. We plan to institutionalize or find other monies to support this position and further develop the position’s scope.
· Our school received provisional accreditation with the World Indigenous Nations Higher Education Consortium (WINHEC). We have a site visit team coming to review our work in Spring 2013, where we hope to use our assessment findings as demonstration of our student’s learning and growth.
13) Reflect on the assessment process. Is there anything related to assessment procedures your program would do differently next time? What went well?
What’s Working Well
● Our new Program Learning Outcomes and Experience Map are much more reflective of our diverse programs and services. The new assessment documents are straight forward enough where it allows us to utilize quantitative measures like survey data, but it also gives us the flexibility to integrate more qualitative measures from our interview data.
● Student interviews from our Summer Enrichment Program and Internship Program continue to provide a unique opportunity to get feedback on the studentʻs overall learning and the ways those experiences have supported their academic journey at UH Mānoa.
● The Student Reception we hosted on September 10 was a great opportunity to engage our past participants in an evening of catching up, but also a way for us to gather important evaluation data. We tried to make it a fun experience and integrated interviews, surveys and other fun activities like life map drawing into the event. We would like to integrate this type of reception-style into more of our future event planning.
● We are in the process of implementing a new evaluation activity: Photo Voice. We hosted a group of students on a 3-day Kahoolawe huakaʻi (field trip, community service engagement). The NHSS staff wanted an evaluation exercise that could be done after the event took place but that wasnʻt too intrusive because some participants had a sacred experience that they perhaps would not want interrogated. We came up with a Photo Voice activity where the hundreds of pictures taken during the huakaʻi were posted to our NHSS Facebook page. The Kahoolawe huakaʻi students and staff received an email inviting them to view the pictures, and select 3 photos that best capture the experience that resonated the most with them from Kahoʻolawe. Students then wrote reflections on those photos around thematic areas aligned to our NHSS program learning outcomes.
What We Plan to Work On/Next Steps
● We need to check-in with our staff more often on our assessment processes to make sure they are working effectively and to adjust if necessary. We are considering a “monthly highlights” form to make it easier for staff to keep track of their work and progress, and also as a benchmark reporting period.
● Advising Logs: Our new logs worked well in the sense that we didn’t have any standard form before these logs. The non-confidential data is captured using Google Docs forms so we can upload and access data in real-time, which works well for some staff, but also allows the flexibility to be inputted retroactively too. It appears that some staff keep up well with inputting their information, while others are not yet comfortable with the process. We plan on getting more buy-in for this process.
● Satisfaction Surveys: We plan on creating satisfaction surveys that are sent anonymously to students who meet with our staff, namely for advising and other high-touch services. We hope to have this piloted and ready to implement by Spring 2013.
● Portfolio & Presentation Rubrics: Our Internship Program & Summer Enrichment Program students already create great student work like portfolios, papers, and other pieces. The students in these programs also do a public presentation at the end of their program where they talk about their learning, growth, and their overall reflection of the experience. It may be appropriate to develop a rubric where other NHSS staff attending the Presentation Day can assess students’ presentations based on its alignment with our NHSS overall program learning outcomes and the Internship/Summer Enrichment program outcomes. We hope to implement this by Summer 2013.
● We have to revisit how we capture all of our workshops and events participants, as our current log that captures this data has not been updated regularly by staff. Our next step would be to create a more formal schedule with reporting periods so staff can input retroactive data into our databases by certain reporting deadlines.