Unit: Kawaihuelani Center for Hawaiian Language
Program: Hawaiian (MA)
Degree: Master's
Date: Mon Nov 16, 2015 - 3:11:18 pm

1) Institutional Learning Objectives (ILOs) and Program Student Learning Outcomes (SLOs)

1) Below are your program's student learning outcomes (SLOs). Please update as needed.

Upon completion of a M.A in Hawaiian, our students should be able to . . .

Reading

Demonstrate comprehension of traditional literary texts.

Listening

Demonstrate comprehension of native speaker dialog

Speaking

Offer a quality* public presentation in Hawaiian

*Quality defined as proper use of the Hawaiian language and demonstration of Hawaiian concepts (i.e., welina, hua ʻōlelo, pilina ʻōlelo, ʻōlelo noʻeau, kūkulu manaʻo, kuanaʻike)

Writing

Demonstrate competence in formal writing skills that have practical/contemporary application

Culture

Demonstrate the ability to apply cultural norms in a range of communicative events

Research

Construct a culturally sensitive research project that utilizes/analyzes relevant existing resources and contributes to the overall Hawaiian knowledge base

2) Your program's SLOs are published as follows. Please update as needed.

Department Website URL: http://manoa.hawaii.edu/hshk/degrees-programs/graduate-degrees/
Student Handbook. URL, if available online: via Native Hawaiian Student Services website http://manoa.hawaii.edu/nhss/wp-content/uploads/2014/09/2014-Kawaihuelani-Grad-Student-Handbook.pdf
Information Sheet, Flyer, or Brochure URL, if available online:
UHM Catalog. Page Number:
Course Syllabi. URL, if available online:
Other: website for Native Hawaiian Student Services http://manoa.hawaii.edu/nhss/academicadvising/kawaihuelani-majors/undergraduate-program-requirements/
Other:

3) Please review, add, replace, or delete the existing curriculum map.

Curriculum Map File(s) from 2015:

4) For your program, the percentage of courses that have course SLOs explicitly stated on the syllabus, a website, or other publicly available document is as follows. Please update as needed.

0%
1-50%
51-80%
81-99%
100%

5) Did your program engage in any program learning assessment activities between June 1, 2014 and September 30, 2015?

Yes
No (skip to question 16)

6) What best describes the program-level learning assessment activities that took place for the period June 1, 2014 to September 30, 2015? (Check all that apply.)

Create/modify/discuss program learning assessment procedures (e.g., SLOs, curriculum map, mechanism to collect student work, rubric, survey)
Collect/evaluate student work/performance to determine SLO achievement
Collect/analyze student self-reports of SLO achievement via surveys, interviews, or focus groups
Use assessment results to make programmatic decisions (e.g., change course content or pedagogy, design new course, hiring)
Investigate curriculum coherence. This includes investigating how well courses address the SLOs, course sequencing and adequacy, the effect of pre-requisites on learning achievement.
Investigate other pressing issue related to student learning achievement for the program (explain in question 7)
Other:

7) Briefly explain the assessment activities that took place in the last 18 months.

Our ongoing assessment activity that we have continued from last academic year is the assessment of our MA program SLOs through evaluation of final theses/Plan B papers and oral defenses of our graduate students through the use of faculty-developed rubrics. Our program was interested in seeing how well and to what extent our students were meeting the MA program SLOs at the end of their graduate studies.

Kawaihuelani also spent time investigating our overall curriculum coherence by analyzing the alignment of course SLOs developed in previous assessment activities with course sequencing, prerequisites, and requirements and then using our findings along with our other assessment results and evidence to make appropriate programmatic changes to improve student learning and success. These changes included modifying two of our courses by adjusting their pre-requisites to align with standard course SLOs developed by our graduate faculty, making one of these courses a required course for all our graduate students, and developing and offering a new course that provide students with additional time to practice some of the skills outlined in our program SLOs that our rubric assessment results told us need more focused attention.

8) What types of evidence did the program use as part of the assessment activities checked in question 6? (Check all that apply.)

Direct evidence of student learning (student work products)


Artistic exhibition/performance
Assignment/exam/paper completed as part of regular coursework and used for program-level assessment
Capstone work product (e.g., written project or non-thesis paper)
Exam created by an external organization (e.g., professional association for licensure)
Exit exam created by the program
IRB approval of research
Oral performance (oral defense, oral presentation, conference presentation)
Portfolio of student work
Publication or grant proposal
Qualifying exam or comprehensive exam for program-level assessment in addition to individual student evaluation (graduate level only)
Supervisor or employer evaluation of student performance outside the classroom (internship, clinical, practicum)
Thesis or dissertation used for program-level assessment in addition to individual student evaluation
Other 1:
Other 2:

Indirect evidence of student learning


Alumni survey that contains self-reports of SLO achievement
Employer meetings/discussions/survey/interview of student SLO achievement
Interviews or focus groups that contain self-reports of SLO achievement
Student reflective writing assignment (essay, journal entry, self-assessment) on their SLO achievement.
Student surveys that contain self-reports of SLO achievement
Other 1:
Other 2:

Program evidence related to learning and assessment
(more applicable when the program focused on the use of results or assessment procedure/tools in this reporting period instead of data collection)


Assessment-related such as assessment plan, SLOs, curriculum map, etc.
Program or course materials (syllabi, assignments, requirements, etc.)
Other 1:
Other 2:

9) State the number of students (or persons) who submitted evidence that was evaluated. If applicable, please include the sampling technique used.

In terms of our rubric assessment of student final theses/Plan B papers and oral defenses/presentations, we collected evidence from one MA students who successfully completed and defended her research during the last academic year, graduating with a master’s degree in Hawaiian in Spring 2015. Faculty sitting on this student’s committee met to review and discuss both the paper and the defense before agreeing as a committee on levels of performance for each product overall and in terms of each SLO.

In terms of our investigation of our overall graduate curriculum coherence, we looked at programmatic evidence like program and specific course SLOs, program requirements, our MA program curriculum map, and course proposals and syllabi for specific courses (HAW 601, 604, 605, 654). 

10) Who interpreted or analyzed the evidence that was collected? (Check all that apply.)

Course instructor(s)
Faculty committee
Ad hoc faculty group
Department chairperson
Persons or organization outside the university
Faculty advisor
Advisors (in student support services)
Students (graduate or undergraduate)
Dean/Director
Other:

11) How did they evaluate, analyze, or interpret the evidence? (Check all that apply.)

Used a rubric or scoring guide
Scored exams/tests/quizzes
Used professional judgment (no rubric or scoring guide used)
Compiled survey results
Used qualitative methods on interview, focus group, open-ended response data
External organization/person analyzed data (e.g., external organization administered and scored the nursing licensing exam)
Other:

12) Summarize the results of the assessment activities checked in question 6. For example, report the percent of students who achieved each SLO.

Tables 1 and 2 below provide a summary of the results of our assessment of this student’s Plan B final paper and oral defense.

Table 1: Rubric Results for Hawaiian M.A. Student Plan B Non-Thesis (How many students were rated at each level? N=1)

Student Learning Objectives

Exemplary/ Distinguished

Competent/ Proficient

Developing/Not Yet Competent

Writing: Formal Research Paper

 

x

 

Culture: Worldview, Cultural Norms

 

x

 

Research: Culturally sensitive research project

 

x

 

Reading

 

x

 

OVERALL

 

x

 

 

Table 2: Rubric Results for Hawaiian M.A. Student Plan B Public Presentation (How many students were rated at each level? N=1)

Student Learning Objectives

Exemplary/ Distinguished

Competent/ Proficient

Developing/Not Yet Competent

Speaking: Public Presentation

 

x

 

Listening Comprehension

 

x

 

Culture: Worldview, Cultural Norms

 

x

 

OVERALL

 

x

 

 

This student was defined as competent and proficient in the eyes of her committee members in all areas being assessed: writing, research, speaking, listening, culture, and overall performance. These results are right in line with the levels of our previous students who have gone through our rubric assessment process. This collection of assessment data over the past three years confirms that our MA graduates in Hawaiian demonstrated quality research skills as well as Hawaiian language and culture proficiency. We are excited that their contributions to our community after graduation will have a positive impact on the understanding of Hawaiian and the development of the teaching and learning of Hawaiian.

After reviewing course SLOs for HAW 604 & 605 and their relationship to their current course pre-requisites and our current program requirements, we identified the following areas that needed addressing:

  • We realized that content from HAW 604 (Writing a Hawaiian Master's Proposal) is critical to the achievement of successful academic outcomes in HAW 605, in that, students identify their research topic and get approval via their proposal in HAW 604 so that they can begin their actual research in HAW 605. This sequence ensures that work done in HAW 605 leads directly to successful completion of their final Plan A or B. However, at the time of our assessment, students could take HAW 605 without taking HAW 604 first. Furthermore, students could choose not to take HAW 604 at all because it was not required for all students.

  • We realized that content from the only prerequisite for HAW 604 (HAW 601, related to Hawaiian literature) was not critical to the achievement of course SLOs for HAW 604.

  • We realized that given the focus of HAW 604 on proposal writing, students should wait until at least their second semester before enrolling when they have had time to become acclimated to our graduate program and our expectations, however, at the time, there was nothing informing our students of this in the course catalog.

13) What best describes how the program used the results? (Check all that apply.)

Assessment procedure changes (SLOs, curriculum map, rubrics, evidence collected, sampling, communications with faculty, etc.)
Course changes (course content, pedagogy, courses offered, new course, pre-requisites, requirements)
Personnel or resource allocation changes
Program policy changes (e.g., admissions requirements, student probation policies, common course evaluation form)
Students' out-of-course experience changes (advising, co-curricular experiences, program website, program handbook, brown-bag lunches, workshops)
Celebration of student success!
Results indicated no action needed because students met expectations
Use is pending (typical reasons: insufficient number of students in population, evidence not evaluated or interpreted yet, faculty discussions continue)
Other:

14) Please briefly describe how the program used the results.

In terms of our rubric assessment results, an area we have been watching closely over the last several years has been the level of student attainment of our cultural SLO in both their written work as well as their oral defense. Even though the trend of their level of attainment of this SLO has been going up since first implementing this assessment activity in 2012 (none of the students scored exemplary in this area for their written work and only one performed at this high level in their oral defense in 2012 while two years later, one of our students scored exemplary in this area for their final paper and two scored exemplary in this area for their oral defense in 2014), our faculty still recognized that we needed to be providing students with more opportunities to be introduced to and practice different ways to express Hawaiian cultural values and worldviews through their language. Therefore, we decided to offer a new graduate course in ʻōlelo Niʻihau, the dialect of the Hawaiian language from the island of Niʻihau spoken and perpetuated primarily by native speakers from this small, isolated Hawaiian community. HAW 654 engages students in advanced analysis and research of ʻōlelo Niʻihau, through face-to-face conversations with Ni‘ihau Native speakers, listening to audio recordings, and watching video recordings of Ni‘ihau Native speakers. It is the first of its kind ever to be offered at the University of Hawai‘i and in the world. Furthermore, it was developed and is taught by a mānaleo or Native speaker of ʻōlelo Niʻihau who is a professor in our department. The addition of this course no only gives Hawaiian language students the opportunity to access new knowledge, create new ways of expressing themselves, and open up new avenues for their graduate research in the field of Hawaiian language, but it also allows them to learn to appropriately interact with Native speakers, many of whom are kūpuna, whose style of speech reflects Hawaiian worldview and cultural values and whose depth of cultural knowledge deeply connects to our ancestors. For example, students in this course help to plan, develop, and coordinate Lā Mānaleo (Native Hawaiian Language Speaker Day), a day for the Ni‘ihau Native speakers to come and share their knowledge of language and culture and to interact with all Hawaiian language students and faculty, by assisting in the hosting and interviewing of the guest speakers, thus allowing them an opportunity to practice their speaking, listening comprehension, and interviewing skills in ʻōlelo Niʻihau with actual Native speakers, which requires competent cultural engagement skills. We are excited about this new addition to our coursework and look forward to seeing how its addition will impact student learning and achievement of our program SLOs, especially those in the areas of speaking, listening, and culture.

Based on our investigation of our overall graduate curriculum coherence, we made the following programmatic changes to support student achievement and success.

  • We modified HAW 604 by deleting HAW 601 as a prerequisite and adding a line to the course description that encourages majors not to enroll in this course until after they have completed at least one semester in the program.
  • We made HAW 604 a required course for all of our MA students because it focuses on the development of research proposals, a requirement for all students in our program anyway; therefore we decided they should all benefit from the direct guidance provided in HAW 604.
  • Modify HAW 605 by adding HAW 604 as a prerequisite.

15) Beyond the results, were there additional conclusions or discoveries? This can include insights about assessment procedures, teaching and learning, and great achievements regarding program assessment in this reporting period.

Since we have been using the rubric assessment process for our theses and oral defenses for about three years, it is our hope that this academic year our faculty will begin to revisit the rubrics and score sheets and possibly make some revisions to the assessment documents and process based on three years of data and experience.

16) If the program did not engage in assessment activities, please explain.