Program: Native Hawaiian Student Services
Date: Thu Oct 13, 2011 - 6:11:49 pm
1) List your program's student outcomes.
Learning Goal 1- Self Appraisal: Students can identify and evaluate personal qualities, values, attitudes, strengths, interests, skills and acquired learning and use this self-appraisal to recognize their unique career, educational, and life planning processes.
Learning Goal 2 – Exploration, Assessment and Decision Making: Students can develop, respond, explore, assess, re-examine, and act upon their progress toward academic, career, and life goals to make informed decisions.
Learning Goal 3 – Personal Integration and Community Engagement: Students can engage in and reflect upon their participation in college and community activities to grow in self-understanding and acceptance, confidence, Hawaiian identities, and appreciation and respect for diverse cultures.
Learning Goal 4 – Adapting to Change: Students can adjust, adapt, respond to, and be resourceful in new situations and in rapidly changing conditions.
Learning Goal 5 – Learning Organization: NHSS strives to be an entity that responds to research trends on effectively serving students.
2) Where are your program's student outcomes published? Mark all that apply and include URLs when appropriate.
Program's Website. URL: Student Handbook. URL, if available online: Information Sheet, Flyer, or Brochure. URL, if available online: UHM Catalog. Page Number: 
Other: Student space in Kamakakuokalani 211 Other: Student space in Queen Liliuokalani Center for Student Services 1043) Provide the program's activity map or other graphic that illustrates how program activities/services align with program student outcomes. Please upload it as a PDF.
- File (10/13/2011)
4) For the period June 1, 2010 to September 30, 2011: State the assessment question(s) and/or assessment goals. Include the student outcomes that were targeted, if applicable.
-
Learning Goal 1: Self Appraisal
Student will be able to (SWiBAT) access, register and complete the HLPE. -
Learning Goal 3: Personal Integration and Community Engagement
SWiBAT articulate how their experiences have developed their self-awareness. -
Learning Goal 3: Personal Integration and Community Engagement
SWiBAT articulate how their academic goals fit in their career and life goal. -
Learning Goal 5: Learning Organization
Each NHSS staff member will participate in 1 professional development activity per year.
Native Hawaiian Student Services Explanatory Data Display of Exemplars
|
Learning Goals & Associated Competencies |
Competency Overview |
Data Description |
Data Source(s) |
Interpretation |
|
Learning Goal 1: Self Appraisal SWiBAT recognize the Resource Centers as a place to access different resources . |
The Resource Centers is a centerpiece of NHSS as they provide NH students with a safe place to gather, access resources, study and network. |
Survey asks students to log-in with their UH username and indicate the purpose of their visit (advising, hang out, study, tutoring, workshop, use computer, etc.). Also tracks which Resource Center location. |
Resource Center sign-in Survey Monkey |
From June 1, 2010 – September 30, 2011: -There were 5,970 sign-ins to the 2 spaces. -78.0% of visits were by NH students. 39.1% were HSHK majors, 60.8% were other majors. -736 unique students (1-118 sign-ins each) -The majority of students came in to utilize the space to study, launa (relax, hang out) or use computers (>45%). |
|
Learning Goal 1: Self Appraisal SWiBAT articulate the purpose of transferring to UHM. |
The KAP Summer Enrichment program is a 4 week residence-based experience designed to provide culturally grounded academic enrichment to Native Hawaiian students entering the University of Hawai‘i at Mānoa from system community colleges. |
To date, conducted in-depth interviews with 7 of the 10 students who participated in the 2011 Summer Enrichment program. Interviewer asked, among other things, how program prepared student to transfer to UHM and made student more comfortable and confident about being a UHM transfer student. |
7 student interviews (conducted by staff member external to the summer enrichment program) |
Students reported that summer program: -Made them more confident about transferring -Made UHM seem less intimidating and scary -Provided them a support network -Made them change the way they view the university, professors, and college courses -Made them eager to learn & appreciate research -Made them more open-minded -Made them want to get involved in community |
|
Learning Goal 1: Self Appraisal
|
The Hawaiian Language Placement Exam (HLPE) is administered to students who have prior Hawaiian language experience or students who have not recently engaged in Hawaiian but are considering taking advanced Hawaiian language courses. |
HLPE log tracks students who register and complete the HLPE, including student demographic information, date and time of HLPE, and HLPE score (determining what Hawaiian language level course student should take). |
HLPE Logs |
From June 1, 2010 – September 30, 2011: - 39 students registered for the HLPE - 37 students completed the HLPE with scores ranging from HAW 101 – HAW 302. |
|
Learning Goal 3: SWiBAT articulate how their experiences have developed their self-awareness. |
The KAP Internship program provides a number of different skills and professional development workshops for its students. Part of this curriculum for the Summer ’11 participants involved many opportunities for self-reflection and self-awareness as a way to complement career exploration. |
To date, conducted in-depth interviews with 7 of the 14 students who participated in the internship program in Summer 2011. Interviewer asked, among other things, how program helped student better understand self, think differently, more confident, more prepared, and more comfortable with themselves and their academic and life journey. |
7 student interviews (conducted by staff member external to the internship program) |
Students reported that internship program: -Made them more confident & mature -Made them more comfortable with self -Helped raise questions of life purpose -Was a reaffirmation of goals, skills & likes -Helped them see/think of their role in society -Helped them “find self” -Helped answer “where do I want to go” and what are the “steps to get there” -Helped them understand “what it means to be Hawaiian” for themselves |
|
Learning Goal 3: SWiBAT articulate how their academic goals fit in their career and life goal. |
In May 2011, NHSS supported 4 Hawaiian Studies & Hawaiian Language MA students to attend the Native American Indian Studies Association (NAISA) conference in Sacramento, California. |
NHSS staff member conducted a focus group session with the 4 Hawaiian Studies & Hawaiian Language MA students who participated in travel to the NAISA conference. Interviewer asked students about their overall impressions of the conference, the importance of attending conferences, and the impact of the travel experience on their academic and career growth. |
1 focus group of 4 student participants |
Students reported that the travel experience: -Helped identify gaps in their own research -Solidified the importance of professional development, research & presenting -Helped build relationships & networks -Built confidence in their own works -Inspired them to consider PhD programs -Was beneficial in connecting to other indigenous people beyond Hawaii |
|
Learning Goal 5: Each NHSS staff member will participate in 1 professional development activity per year. |
Over the past year, NHSS staff have been invited to speak at conferences locally, nationally and internationally. Staff have also attended many conferences outside of Hawaii (mostly on extramural funds) covering a wide range of disciplines including educational research, indigenous education, evaluation and ethnic studies. |
NHSS developed a Google Documents folder maintained by staff. We created a form entitled “Professional Development Log” which is basically a survey that asked staff members to fill in the date, event title, host organization, and type of event for each professional development activity they completed. Additionally, staff members also took notes on each entry and indicated things like if they received a special type of certification or what they learned at that professional development activity. The log was started in January 2011, so we may be missing data from June ’10-Jan ’11. |
Professional Development log |
Staff Conference Presentation Highlights: -Native Hawaiian Education Association annual conference keynote presentation -Title III Director’s Meeting Empowering Institutions conference, Washington D.C. -Critical Ethnic Studies, UC Riverside -World Indigenous Nations Higher Education Consortium conference presentation, Cusco, Peru Staff Conference Attendance Highlights: -NASPA 2010 conference, Portland, OR -Native American and Indigenous Studies Association 2010 -American Evaluation Association 2010 -Transatlantic Dialogue on Culture in University Setting, Luxembourg, Germany -American Educational Research Association
-World Indigenous Nations Higher Education Consortium, Norway |
5) State the type(s) of evidence gathered to answer the assessment question and/or meet the assessment goals that were given in Question 4.
- Sign-in logs: student spaces, activities/programs, events, tutoring, etc.
- Basic surveys (primarily satisfaction-oriented) from NHSS activities
- Staff advising logs
- Final student presentations: Internship Program, Summer Enrichment Program
- Kuder Career Assessments
- Student artifacts (e.g. presentations, portfolios, journals, etc.)
-
Student Participant Interviews
- 7 Summer Internship Program participant interviews from Summer 2011 cohort
- 7 Summer Enrichment Program participant interviews from Summer 2011 cohort
- 1 focus group of 4 Hawaiian Studies/Hawaiian Language MA students who attended NAISA conference
- External evaluation report for Kōkua a Puni program (which included student focus groups, other stakeholder interviews)
6) State how many persons submitted evidence that was evaluated. If applicable, please include the sampling technique used.
All 8 NHSS staff members submitted evidence: E. Kahunawai Wright (Director of NHSS), B.J. Nalani Balutski (Research & Evaluation Coordinator), C. Lehua Nishimura (Counseling Coordinator), C. Ululani Oliva (Graduate Support Specialist), S. Kamuela KaAhanui (Academic Advisor), N. Mehana Hind (Academic Advisor, now vacant), Pearl Wu (Enrichment Coordinator), Ileana Ruelas (Outreach Coordinator).
7) Who interpreted or analyzed the evidence that was collected. Check all that apply.
Program faculty/staff Faculty/staff committee Ad hoc faculty/staff group Director or department chairperson Persons or organization outside the university Students (graduate or undergraduate) Dean Other 8) How did he/she/they evaluate, analyze, or interpret the evidence? Check all that apply.
Compiled survey results Used qualitative methods on interview, focus group, open-ended response data Scored exams/tests/quizzes Used a rubric or scoring guide Used professional judgment (no rubric or scoring guide used) External organization/person analyzed data (e.g., Social Science Research Institute) Other: 9) For the assessment questions/goals stated in Question 4, summarize the actual results.
The numbers below provide a snapshot of students who utilize some of our various resources and services:
- Resource Centers: 5,970 sign-ins (of which 736 were unique student visitors)
- Internships: 14
- HLPE: 37
- Tutoring: 95
- HSHK Graduate Student Travel Scholarship: 4
- Summer Enrichment Program: 10
In review of our learning goals and associated data collected, these are the general trends or themes which surfaced:
- The Resource Centers continue to serve as significant points for Native Hawaiian students to utilize different resources;
- Over the past year, NHSSS has been more successful at attracting more non-HSHK majors into our Resource Centers (60.9% of visits this past year were non-HSHK majors);
- The Internship Program is successful in helping students develop their self-awareness and align life and career goals;
- The Summer Enrichment Program is successful in helping students feel more confident about transferring to UHM;
- NHSS Graduate Student Travel Scholarship Program is successful in providing valuable research and professional development opportunities to Hawaiian Studies & Hawaiian Language graduate students;
- Over the past year, the number of professional development opportunities for NHSS staff has grown significantly.
The NHSS Kōkua a Puni program (funded by USDOE, Title III) just finished its fourth year of a 5-year grant. An external evaluation report was completed in September 2010 by Dr. Anna Ortiz from California State University Long Beach. Some highlights from the evaluation report:
- “It is difficult to overstate the importance of the two resource centers to the work and success of the grant. Combined, these centers add 1,300 square feet of additional space dedicated to the success of Native Hawaiian students.
- “Perhaps more profound, were the efforts KAP staff expended to develop advocacy within students by helping them to define and connect to their sense of Hawaiian identity.”
- “Kokua a Puni has excelled with the development of its data infrastructure and building its technology capacity. Smartly, the grant supports the research analyst position, which has primary responsibility for implementing grant activities related to research, data management, reporting, and technology initiatives.”
- “Kokua a Puni is more than meeting the expectations of the grant. In almost every category, KAP has exceeded grant proposal goals. The KAP staff have been smart about every decision they have made in regards to activities, use of grant funds, and taking advantage of opportunities. When staff was hired later than anticipated, the cost savings immediately supported expanded services and student access to the summer and internship programs, and additional tutorial offerings. They have sought and secured additional funding for program activities and infrastructure. They are taking advantage of every opportunity to partner with units and extend their involvement in university governance structures…Native Hawaiian students benefit from each of these efforts. They are a true learning organization.”
10) What was learned from the results?
- NHSS needs to continue to maintain (and possibly create new) spaces for Native Hawaiian students on our campus.
- NHSS needs to create mechanisms to measure student learning for its academic services (i.e. academic advising, graduate support).
- NHSS needs to expand the availability of professional development opportunities for Native Hawaiian graduate students.
11) State how the program used the results or plans to use the results. Please be specific.
- Our extramural Kōkua a Puni program (Title III funding) is now in its final year of funding. Over the next year, NHSS has to work on ways to institutionalize the positions and programming that will expire with the Title III monies that are in its final year of funding.
- Over the next several months, we will be brainstorming the next iteration of the Kōkua a Puni (Title III) program to submit for the next round of funding in the spring of 2012. Kokua a Puni provides a large base of fiscal and personnel (3 of our 8 staff members are funded by this grant) support, so it is imperative to use our assessment data gathered to inform the next iteration of Title III programming. In this way, our next grant proposal will be driven by our current assessment data.
- Hawai‘inuiākea is currently preparing our eligibility application for World Indigenous Nations Higher Education Consortium (WINHEC) accreditation. NHSS assessment data will be an integral piece of our narrative about our efforts in supporting Native Hawaiian students on our campus.
12) Reflect on the assessment process. Is there anything related to assessment procedures your program would do differently next time? What went well?
Is there anything related to assessment procedures your program would do differently next time?
- We need to check-in with our staff more often on our assessment logs and processes to make sure they are working effectively and to adjust if necessary. We are considering a “monthly highlights” form to make it easier for staff to keep track of their work and progress, and also as a benchmark reporting period.
- We will be revisiting our experience map, learning goals, outcomes, and assessment tools. We have a half-day “Assessment Talk Story” session scheduled for October 20, 2011 where we will start to discuss these things and possibly revisit and revise them.
What went well?
- Our new logs worked well in the sense that we didn’t have any standard form before these logs. The non-confidential data is captured using Google Docs forms so we can upload and access data in real-time, which works well for some staff, but also allows the flexibility to be inputted retroactively too.
- We conducted interviews of students who participated in a few of our programs, namely our Summer Enrichment Program and Internship Program. Considering this was our first time doing these student interviews, we thought we came up with great questions and a great process for transcribing the interviews. We just recently completed these interviews, so our next steps will be to analyze the interview data and disseminate the information to the program staff. We are excited to analyze, share and use the results to inform future efforts of our unit.