Unit: Hawaiian Knowledge, Hawai'inuiakea School of
Program: Native Hawaiian Student Services
Date: Wed Oct 21, 2015 - 2:08:53 pm

1) Below are your program's student outcomes (SOs). Please add or update as needed.

The core values, which drive and shape our student support activities stems from our mission and informed by our collective experience and initial assessment work.  In Summer 2013, NHSS drafted a new program logic model (see attachment).  The document is still in working draft form, as we have not yet finalized our eloquent “long-term” impact and vision statements.  Additionally, we feel that there is one more piece missing in this overall logic model that still needs to be represented here (as illustrated in the large gap in the second column).   In Summer 2014, the previous Director of Native Hawaiian Student Services vacated the position, and a new Director, Dr. Willy Kauai, was hired in January 2015. These administrative changes has meant that our unit has not revisited our mission and vision documents to finalize our Student Learning Outcomes and overall logic model. We are confident that this next year will help us fill in that missing piece with the assessment data we collect and our ongoing conversations about our work.   

(1)  Haumāna develop a sense of place & belonging at UH Mānoa.

(2)  Haumāna build networks of mentorship support with peers.

(3)  Haumāna develop a critical thinking epistemology using Hawaiian identity and history as the foundation. 

(4)  Haumāna recognize their place within the lāhui.

(5)  Haumāna develop transferrable academic & career skills.

(6)  Haumāna develop positive attitudes about their career or graduate school potential.

(7)  Lāhui Hawai‘i develop a sense of place & belonging at UH Mānoa.

(8)  NHSS develop and maintain a contributing presence in the Hawaiian community.

Importantly, NHSS receives approximately $14,000 in institutional monies, which is supposed to provide support for our programs and services for the 3,000 Native Hawaiian students at UHM. The rest of our unit is supported by federal grants, which are assessed yearly through the US Department of Education. 

2) Your program's SOs are published as follows. Please update as needed.

Program's Website. URL: facebook.com/nativehawaiianstudentservices/info
Student Handbook. URL, if available online:
Information Sheet, Flyer, or Brochure. URL, if available online:
UHM Catalog. Page Number:
Other: facebook.com/nativehawaiianstudentservices/info
Other:

3) Provide the program's activity map or other graphic that illustrates how program activities/services align with program student outcomes. Please upload it as a PDF.

Activity Map File(s) from 2015:

4) Did your program engage in any program assessment activities between June 1, 2014 and September 30, 2015? (e.g., establishing/revising outcomes, aligning activities to outcomes, collecting evidence, interpreting evidence, using results, revising the assessment plan, creating surveys, etc.)

Yes
No (skip to question 14)

5) For the period between June 1, 2014 and September 30, 2015: State the assessment question(s) and/or assessment goals. Include the student outcomes that were targeted, if applicable.

Targeted Outcomes:

  1. Haumāna develop a sense of place & belonging at UH Mānoa.
  2. Haumāna build networks of mentorship support with peers.
  3. Haumāna develop positive attitudes about their career or graduate school potential.

Assessment Questions:

  • In what ways did the Hūlili Professional Development Program provide Native Hawaiian students meaningful research experience and development?  How did students describe the impact of such development on their future educational, career or life goals?  
  • In what ways did the Hūlili Summer Bridge Program prepare Native Hawaiian students for their transfer to UH Mānoa? 
  • To what extent did Hūlili Transfer Center visitors feel satisfied, supported, and well resourced when visiting the NHSS spaces?

6) State the type(s) of evidence gathered to answer the assessment question and/or meet the assessment goals that were given in Question #5.

This past year, we engaged in the following assessment activities:

  • Hūlili Summer Bridge Program Survey
  • Hūlili Professional Development Program Survey
  • Hūlili Tutoring Survey
  • Hūlili Transfer Center Survey
  • Hūlili Summer Bridge Program Exit Interviews
  • Hūlili Professional Development Program Exit Interviews

 

7) State how many persons submitted evidence that was evaluated. If applicable, please include the sampling technique used.

Assessment Activity # of participants
Hūlili Summer Bridge Program Survey 67 survey invitations, 22 responses
Hūlili Professional Development Program Survey 80 survey invitations, 24 responses
Hūlili Tutoring Survey 54 survey invitations, 5 responses
Hūlili Transfer Center Survey 377 survey invitations, 115 responses
Hūlili Summer Bridge Program Exit Interviews 26 focus group interview participants, all program participants interviewed
Hūlili Professional Development Program Exit Interviews 50+ exit interviews, all program participants interviewed

 

8) Who interpreted or analyzed the evidence that was collected? Check all that apply.

Program faculty/staff member(s)
Faculty/staff committee
Ad hoc faculty/staff group
Director or department chairperson
Persons or organization outside the university
Students (graduate or undergraduate)
Dean or Associate Dean
Advisory Board
Other:

9) How did he/she/they evaluate, analyze, or interpret the evidence? Check all that apply.

Compiled survey results
Used quantitative methods on student data (e.g., grades, participation rates) or other numeric data
Used qualitative methods on interview, focus group, or other open-ended response data
Scored exams/tests/quizzes
Used a rubric or scoring guide
Used professional judgment (no rubric or scoring guide used)
External organization/person analyzed data (e.g., Social Science Research Institute)
Other:

10) For the assessment questions/goals stated in Question #5, summarize the actual results.

Because our Hūlili grant just ended on September 30, 2015, we only closed the surveys for all Hūlili programs in the last two weeks. Thus, we are still going through the data and compiling the qualitative data, but here are the results so far.  To the extent possible, we wanted to honor the voices of our participants, and thus kept their actual quotes here. We will provide more summarized results in the next report. 

Hūlili Summer Bridge Program Survey

Students described their experience in the Summer Bridge Program as providing them with resources, know-how, confidence and preparation needed to be successful at UHM. These are aligned to many of the NHSS SLOs, but namely #1 and #2. 

  • “It helped me open my eyes to the endless possibilities that I can do during college to prepare myself after I graduate as well as ways to support myself while I still was in college.” SLO #6
  • “The Bridge helped me become more prepared to be a student at UHM by providing me the resources that I would need in order to be successful.” SLO #1
  • “It rooted me a little more in my identity as a Hawaiian which help strengthen my confidence in pursuing my education.” SLO #3
  • “Bridge has helped me to be more confident in myself and all that I can do. Bridge has helped me realize that I am more than what I seem and that ultimately I can do whatever I set my mind to.” SLO #6
  • “Don’t be afraid to step out of your comfort zone because when you let yourself be vulnerable you get rewarded in the most amazing and unexpected ways.” SLO #1
  • “Bridge helped me realize UHM is a lot more involved with helping students achieve academic success than I originally thought.” SLO #1
  • “It made me realize that there is an abundance of resources out there and that people are willing to help you reach your goals.” SLO #1
  • “It made me see that there is a program to turn to—when the stresses of college life become somewhat overwhelming.” SLO #1
  • “Bridge affected me in such a positive way. I never knew what was out there for me until I was a part of this program. It helped me get out of my comfort zone, look for resources, see all the resource that are out there and available to eligible students. It helped me see that it is okay to seek assistance and help. When I first started college, I had the most difficult time applying for scholarships because I never knew where to start but bridge helped me realize how much resources are out there and the workshops helped me also with so much different things.” SLO #1 & 6
  • “Bridge has widened my perceptions of UHM. Before I thought that UHM wasn’t out to help students but now I’ve come to realize at least through the NHSS that there are so many resources and opportunities for students like myself.” SLO #1
  • “It made it easier to deal with UHM. Coming from a community college unsure where to get help.” SLO #1
  • “The program impacted me directly because without it I would not have been the first in my family to graduate with a BA from any university.” SLO #8
  • “Mahalo for such a great experience! I wouldn’t be who I am or where I am without your services. It helped me learn so much about myself, different programs and resources. I liked the workshops that were out there for us, it was very educational and very useful.” SLO #1
  • “I continued and always will use the NHSS services after my Bridge experience. The reason why is because it is such a safe and friendly environment.” SLO #1
  • “I would love to continue to be a part of and give back to a program that has offered me so many amazing experiences and opportunities.” SLO #1

Hūlili Professional Development Program Survey

91.3% of survey respondents said the program exceeded their expectations and 95.7% said interactions with staff were very good.

  • “The opportunity to interact with professionals on the national and international level, network, etc.”
  • “It is a platform to grow and to learn. You become fully aware of your potential and you’re able to use this program to ones full advantage. The experience also helps to guide you and take your time in the program to develop the skills necessary to succeed. It is indeed a very communal and nurturing environment. This is what I like most about the experience. It was a familiar place, like home, like family. The support that I felt was appreciated so much and will always enjoy working with this program.”
  • “The experience to network with highly respected researchers in my field.”
  • “Being able to meet other indigenous professionals in the same field as me was extremely beneficial.”
  • “Exposure, networking, and first hand field work experience.”
  • “One of the best highlights of my PD experience was having the opportunity to network with other students and professionals at the conference I attended.”
  • “My PD experience enhanced the quality of my research. It allowed me to effectively honor those who provided insight into my research. Simply said, PD gave me the financial support to be able to make these connections and go about my research in a truly ‘Hawaiian’ way.”
  • “The main lesson I learned…was that I am on the right path with my personal and professional goals. Sometimes it’s hard to get that validation in Hawaii because there aren’t many Hawaiians in the field so being able to travel and meet other native people is a great opportunity.”
  • “The main lessons I took away from my PD experience were how to incorporate my culture and traditions into my career, especially as a Native Hawaiian.”
  • “It has expanded my network of peer and professional mentors.”
  • “Broadened my network and has strengthened my resolve.”
  • “The PD experience has helped me reach my goals, because through it, I was able to cultivate more leadership skills that I feel I can apply to my life as a student here at UH Manoa.”
  • “It has allowed me to network and gain irreplaceable contacts within my field of study. They have opened up opportunities that I would not have had if their services weren’t there.”
  • “PD has helped me refocus my educational goals and expand my perspectives. Not to mention being able to establish a broader network of people in the field.”
  • “Provided the support and confidence I needed to holomua.”
  • “I think it has helped me to embrace my decision to really pursue my PhD. Without this chance I might have still be in question. They, alongside the program, gave me the confidence to say that I can do this no matter what.”
  • “PD has definitely gave more of a drive to persevere to my goals regardless of the obstacles I may encounter along the way.”
  • “I think it’s helped energize me and helped me create more enthusiasm to complete my project and graduate!”
  • “Helped me expand my horizons beyond traditional jobs.”
  • “I believe the experience has strengthened multiple areas of my personal, professional, and academic life.”
  • “My scholarly drive to be a better and more efficient researcher is greater than ever.”
  • “I am now working full-time in the field that I have been putting into practice the ideas and theories that I learned from the conference.”
  • “Less stress and minimal anxiety due to the exposure, mentorship/partnership, field work, and guidance I have gained through my PD experience making me more confident and determined to finish strong in my endeavors.”
  • “As a result of the conference I attended, I was elected into a leadership position that will help me pursue my goals.”
  • “I feel like I’m more vested in the filed and it’s more of a reality than just the theoretical situations I experience regularly in class.”

11) What was learned from the results?

Ultimately, we learned that students had very transformational things to say about their experiences in our programs. Our SLO #1 about creating a sense of belonging for students on our campus is key. The assessment data from each of our activities highlighted different ways in which we were able to create this sense of beloging for students, namely through the interactions with staff and faculty, connecting them to resources, supporting them in their discipline, as well as making them more confident and prepared to navigate college life were all cited as examples of how this sense of belonging was created and maintained.

The Hūlili Professional Development Program has done an exceptional job at offering a unique research and conference travel program for Native Hawaiian students at UHM.  Students have said that NHSS "made it easy" to apply for funding, where alternative funding sources (GSO and departmental funding) is bureaucratic and defeating.  Participants talked about how these research support allowed them to network with people in their field, reinforce why theyʻre in school (the pragmatic, seeing their academics in action), and provided them with confidence to continue in the field or further their education.  

Our Summer Bridge Program is one of our more in-depth programming, as students live on campus and are with us full-time for six weeks (including weekends). Thus, the assessment data from the focus groups highlight how NHSS was able to interrogate many of our NHSS Student Learning Outcomes for these students. Namely, students talked about their interrogation of their Hawaiian identity (in positive ways related to their higher education goals), becoming more prepared and confident for their college journey, being connected to resources on campus, and creating a strong peer support group. 

12) State how the program used the results or plans to use the results. Please be specific.

This past May and June, NHSS wrote a new federal grant called Kekaulike, which is a collaborative grant with the University of Hawaii Maui College (UHMC). NHSS used assessment results from years previous to identify our emerging best practices, based on the results and data shared by students. Namely, the Kekaulike grant has three main activities (with related subactivities):

  1. Lāhui Hawaiʻi Research Center: annual conference, professional development, think tanks and workshops, Lāhui Hawaiʻi journal
  2. Maui to Mānoa Transfer Pipeline: Summer Bridge Program, online pre-transfer course and learning community, UH Maui faculty and staff advanced degree fellowship
  3. Maui Place-Based Practicum & Field Schools: Mauiakama Hawaiian language immersion camp, field schools, practicum/internship program

Furthermore, Hūlili was focused on Native Hawaiian transfer students, and thus we were able to use what we learned about supporting transfer students to conceptualize this new project, also for Native Hawaiian transfer students, but now from UHMC. The grant was one of our most well-conceptualized project designs to date, and we also used some assessment data in the actual grant narrative. We just found out a few weeks ago that we were awarded the grant, which started on October 1, 2015. Because our unit receives almost no institutional funds, it is important to engage in grant writing to support the work of our unit that provides programs and services to the 3,000 Native Hawaiian students on this campus. 

13) Reflect on the assessment process. Is there anything related to assessment procedures your program would do differently next time? What went well?

This past year, we had a new coordinator do some of the exit interviews for our Professional Development Program. Although this was good to get more staff and faculty engaged in the process, there was not enough training of the staff member in proper qualitative research protocols. Thus, there was not good protocols observed to document the student interviews for future analysis. Therefore, this will be improved in the future so that qualitative data analysis is done well for report purposes. 

Also, our unit is working on refining our internal data collection processes and finalizing an internal comprehensive program database so that we can easily query the database to do analyses and future assessment efforts, like surveying program participants. This past summer, one of our Graduate Research Assistants made a program database just for our Hūlili grant, which enabled him to survey those student participants. In the future, we want a comprehensive database for all grants, programs and services in our unit. 

14) If the program did not engage in assessment activities, please explain.
Or, if the program did engage in assessment activities, please add any other important information here.