Unit: Kawaihuelani Center for Hawaiian Language
Program: Hawaiian (MA)
Degree: Master's
Date: Wed Oct 19, 2011 - 8:15:05 am

1) Below are your program student learning outcomes (SLOs). Please update as needed.

Upon completion of a M.A in Hawaiian, our students should be able to . . .

Reading

Demonstrate comprehension of traditional literary texts.

Listening

Demonstrate comprehension of native speaker dialog

Speaking

Offer a quality* public presentation in Hawaiian

*Quality defined as proper use of the Hawaiian language and demonstration of Hawaiian concepts (i.e., welina, hua ʻōlelo, pilina ʻōlelo, ʻōlelo noʻeau, kūkulu manaʻo, kuanaʻike)

Writing

Demonstrate competence in formal writing skills that have practical/contemporary application

Culture

Demonstrate the ability to apply cultural norms in a range of communicative events

Research

Construct a culturally sensitive research project that utilizes/analyzes relevant existing resources and contributes to the overall Hawaiian knowledge base

2) Your program's SLOs are published as follows. Please update as needed.

Department Website URL: website still under construction
Student Handbook. URL, if available online: NA
Information Sheet, Flyer, or Brochure URL, if available online:
UHM Catalog. Page Number:
Course Syllabi. URL, if available online:
Other:
Other:

3) Below is the link(s) to your program's curriculum map(s). If we do not have your curriculum map, please upload it as a PDF.

Curriculum Map File(s) from 2011:

4) For your program, the percentage of courses that have course SLOs explicitly stated on the syllabus, a website, or other publicly available document is as follows. Please update as needed.

0%
1-50%
51-80%
81-99%
100%

5) For the period June 1, 2010 to September 30, 2011: State the assessment question(s) and/or assessment goals. Include the SLOs that were targeted, if applicable.

The two main program SLOs our program targeted during this period focused on verbal and non-verbal language as well as application of cultural knowledge and worldview.

Upon completion of a M.A. in Hawaiian, students should be able to . . .

  • Offer a quality* public presentation in Hawaiian
  • Demonstrate the ability to apply cultural norms in a range of communicative events

Our program was interested in seeing how well and to what extent our students were meeting these select program SLOs.

6) State the type(s) of evidence gathered to answer the assessment question and/or meet the assessment goals that were given in Question #5.

Our program identified an existing Hawaiian language activity, Mūkīkī Wai Nā Mamo o Mānoa, that our students already participate in as a place to collect evidence to answer our assessment question above. It is a yearly multifaceted Hawaiian language celebration/competition developed in the 2004-2005 academic year by several of our Kawaihuelani faculty members. It was created as a venue to showcase students’ achievements in Hawaiian language acquisition of traditional performance forms as well as contemporary communication mediums such as theatre, film and multi-media presentations.  The annual competition strategically includes appropriate categories for university students from various levels of language learning (beginning, intermediate, advanced and graduate level).  Students aspire to reach higher levels of language fluency and competency as they strengthen their language skills through participating in the festival. 

Haʻi ʻŌlelo Hōʻeuʻeu (persuasive speeches) presented by our HAW 602 graduate students were assessed.

7) State how many persons submitted evidence that was evaluated. If applicable, please include the sampling technique used.

Of the 571 unduplicated students enrolled in our Hawaiian language courses in the Spring 2011 semester, 115 participated in Mūkīkī Wai this year. This comes out to a 17% participation rate.

See pie charts that illustrate these overall participation numbers as well as the break down for each level (first through fourth year, and HAW 602).

Of the 115 students who participated, 74 submitted evidence that was evaluated for program assessment. These 74 students made up 39 individual/group entries in 6 categories enrolled in 200-600 level courses.

Of the 74 students who were assessed, 6 were graduate students all from HAW 602 Kākāʻōlelo (Oratory).

We acknowledge upfront that these numbers are probably not substantial enough to make generalizations about how our students are doing in general at meeting these two program SLOs, however they do provide a glimpse into our program that will hopefully motive our colleagues to enter their students in Mūkīkī Wai in the near future in order to validate these initial findings with more substantial numbers.

8) Who interpreted or analyzed the evidence that was collected? (Check all that apply.)

Course instructor(s)
Faculty committee
Ad hoc faculty group
Department chairperson
Persons or organization outside the university
Faculty advisor
Advisors (in student support services)
Students (graduate or undergraduate)
Dean/Director
Other:

9) How did they evaluate, analyze, or interpret the evidence? (Check all that apply.)

Used a rubric or scoring guide
Scored exams/tests/quizzes
Used professional judgment (no rubric or scoring guide used)
Compiled survey results
Used qualitative methods on interview, focus group, open-ended response data
External organization/person analyzed data (e.g., external organization administered and scored the nursing licensing exam)
Other:

10) For the assessment question(s) and/or assessment goal(s) stated in Question #5:
Summarize the actual results.

See bar charts that illustrate our results graphically (200-600 level). Charts are summarized below as well.

For our graduate students, 100% of them were identified as meeting expectations in terms of language. In terms of the cultural knowledge and worldview SLO, 83% of students were identified as meeting expectations while 17% were identified as exceeding. No students were identified as approaching or not meeting expectations for both SLOs.

11) State how the program used the results or plans to use the results. Please be specific.

This assessment activity was a pilot to test out our new rubrics and get familiar with the data collection, analysis, and reporting process. While the results are encouraging and we should be proud of how well our students fulfilled the target SLOs, the results from this pilot will be shared with our colleagues at an upcoming faculty meeting as more of a motivation to become involved. We hope to spark their curiosity to look closer and seek more data to substantial (or refute) these initial findings by encouraging them to participate in Mūkīkī Wai (encouraging their students to enter, serving as evaluators, etc.) as well as the assessment activities of the program as a whole. We do not expect to use these initial results to make dramatic changes or improvements to our program. We need to wait until we have data from a more substantial sampling before we can move forward in this way, which we hope to have as a result of motivating our colleagues to participate in Mūkīkī Wai 2012.

12) Beyond the results, were there additional conclusions or discoveries?
This can include insights about assessment procedures, teaching and learning, program aspects and so on.

After this pilot activity, several lessons were learned and areas were identified for improvement in terms of the process and tools themselves. We list a few of them below, which we hope to address in our second implementation at the upcoming Mūkīkī Wai Nā Mamo o Mānoa in Spring 2012.

  • As mentioned above, we hope that sharing the initial results from the pilot will motive more instructors to enter their students.
  • Share rubrics with instructors well before the event in order to familiarize them with the tools and generate feedback so that adjustments can be made in time to share the tools with their students as well as the evaluators who will use them on the day of the event.
  • Meet with the evaluators ahead of time to familiarize them with the rubrics and score sheets as well as engage them in norming conversations.

13) Other important information.
Please note: If the program did not engage in assessment, please explain. If the program created an assessment plan for next year, please give an overview.

On September 14, 2011, our graduate faculty participated in an evaluation/assessment retreat/workshop with the assistance of the UHM Assessment Office in order to not only review our curriculum as a whole but also to prepare for and make decisions about our upcoming UHM program review and accreditation application to WINHEC. The goals for the retreat were:

  • to review the program review and accreditation process in order to increase participation by all faculty and their students;
  • to re-familiarize ourselves with our Graduate Program Learning Objectives and Curriculum Map, making any changes or adjustments as needed;
  • to develop standard Course Learning Objectives for several of our required courses (HAW 601, 603, 614, 652) that all professors agree upon;
  • to identify significant assignments from specific courses/experiences/events at both BA and MA level that can be used to assess whether our students are in fact meeting our Program SLOs so we can start collecting this data in Spring 2012;
  • to decide upon appropriate tools to develop after the retreat to use to assess these collected assignments (rubrics, etc.).

Here are the outcomes of this retreat.

  • Standardized course SLOs were developed for HAW 601, 603, 614, and 652 and were agreed upon by all professors present.
  • It was determined that we needed to change course numbers for two of our required courses to better reflect the order students should be taking these required courses, thus assisting students when registering. Based on course content, student learning objectives, and assignments, we realized that our students should take existing HAW 614 before existing HAW 603, therefore we are planning to change their course numbers to reflect this new sequencing: existing HAW 614 à new HAW 604; existing HAW 603 à new HAW 605.
  • Our graduate faculty also agreed that a minor change to our graduate requirements was needed to provide students will more support in completing their Plan A thesis or Plan B non-thesis/project. We will be requesting that HAW 604 become a required course, listed alongside HAW 605 & HAW 612 in which students will then choose two of the three for a total of 6 credits.
  • Two assignments/artifacts were identified as appropriate for program assessment purposes using a rubric designed by our professors used by the student’s committee: for PLAN A, the final thesis & defense; for PLAN B, the final written product & defense/oral presentation.

Next Steps

  • Follow up with faculty about the decisions they made in terms of assignments/artifacts to collect, the rubric, and the process, such as:
    • Who will work with Maya to develop the rubrics?
    • In terms of process . . .

DEFENSE: When will committee use rubric to assess defense? When committee deliberates? Will each member of the committee fill out their own rubric and turn it in to Maya OR will the committee discuss, find consensus, and just turn in just one rubric to Maya?

PAPER: When will committee use rubric to assess final paper? After each member receives the final version? Will each member of the committee fill out the rubric individually at their own time and send to Maya by graduation OR will the committee meet again to discuss the paper in terms of the rubric & program SLOs, find consensus and turn in just one rubric to Maya by graduation?