Program: Hawaiian (MA)
Degree: Master's
Date: Tue Oct 16, 2012 - 8:07:59 am
1) Below are your program's student learning outcomes (SLOs). Please update as needed.
Upon completion of a M.A in Hawaiian, our students should be able to . . .
|
Reading |
Demonstrate comprehension of traditional literary texts. |
|
Listening |
Demonstrate comprehension of native speaker dialog |
|
Speaking |
Offer a quality* public presentation in Hawaiian *Quality defined as proper use of the Hawaiian language and demonstration of Hawaiian concepts (i.e., welina, hua ʻōlelo, pilina ʻōlelo, ʻōlelo noʻeau, kūkulu manaʻo, kuanaʻike) |
|
Writing |
Demonstrate competence in formal writing skills that have practical/contemporary application |
|
Culture |
Demonstrate the ability to apply cultural norms in a range of communicative events |
|
Research |
Construct a culturally sensitive research project that utilizes/analyzes relevant existing resources and contributes to the overall Hawaiian knowledge base |
2) Your program's SLOs are published as follows. Please update as needed.
Department Website URL: website still under construction Student Handbook. URL, if available online: NA
Information Sheet, Flyer, or Brochure URL, if available online: UHM Catalog. Page Number: Course Syllabi. URL, if available online: Other: Other: 3) Select one option:
- File (03/16/2020)
4) For your program, the percentage of courses that have course SLOs explicitly stated on the syllabus, a website, or other publicly available document is as follows. Please update as needed.
0%
1-50% 51-80% 81-99% 100%5) Did your program engage in any program assessment activities between June 1, 2011 and September 30, 2012? (e.g., establishing/revising outcomes, aligning the curriculum to outcomes, collecting evidence, interpreting evidence, using results, revising the assessment plan, creating surveys or tests, etc.)
Yes No (skip to question 14)6) For the period June 1, 2011 to September 30, 2012: State the assessment question(s) and/or assessment goals. Include the SLOs that were targeted, if applicable.
Kawaihuelani conducted two separate program assessment activities related to our graduate program this past Spring 2012 semester. Our first was a continuation of our assessment of two MA program SLOs at our annual Mūkīkī Wai Nā Mamo o Mānoa Hawaiian language competition/showcase that focused on verbal and non-verbal language as well as application of cultural knowledge and worldview. Our program was interested in seeing how well and to what extent our graduate students were meeting these select program SLOs listed below at the mid point of their program.
Upon completion of a M.A. in Hawaiian, students should be able to . . .
· Offer a quality* public presentation in Hawaiian
· Demonstrate the ability to apply cultural norms in a range of communicative events
Our second assessment activity that was developed and implemented for the first time in Spring 2012 was an assessment of all MA program SLOs through evaluation of final theses/Plan B papers and oral defenses of our graduate students. Again, our program was interested in seeing how well and to what extent our students were meeting the MA program SLOs listed below at the end of their studies.
Upon completion of a M.A. in Hawaiian, students should be able to . . .
· Demonstrate comprehension of literary texts.
· Demonstrate comprehension of native speaker dialog.
· Offer a quality public presentation in Hawaiian.
· Demonstrate competence in formal writing skills that have practical/contemporary application.
· Demonstrate the ability to apply cultural norms in a range of communicative events.
· Construct a culturally sensitive research project that utilizes/analyzes relevant existing resources and contributes to the overall Hawaiian knowledge base.
7) State the type(s) of evidence gathered to answer the assessment question and/or meet the assessment goals that were given in Question #6.
MŪKĪKĪ WAI
We continued what we began in 2011 by collecting evidence from Mūkīkī Wai Nā Mamo o Mānoa, an existing Hawaiian language activity that our students already participate in to answer our assessment question above. It is a yearly multifaceted Hawaiian language celebration/competition developed in the 2004-2005 academic year by several of our Kawaihuelani faculty members. It was created as a venue to showcase students’ achievements in Hawaiian language acquisition of traditional performance forms as well as contemporary communication mediums such as theatre, film and multi-media presentations. The annual competition strategically includes appropriate categories for university students from various levels of language learning (beginning, intermediate, advanced and graduate level). Students aspire to reach higher levels of language fluency and competency as they strengthen their language skills through participating in the festival. Haʻi ʻŌlelo Hōʻeuʻeu (persuasive speeches) presented by our HAW 602 graduate students were assessed in terms of the two program SLOs listed in #6 above.
MA THESIS/DEFENSE
One of the outcomes of our assessment workshop back in September 2011 was the decision by the graduate faculty to collect and evaluate two student assignments/artifacts against MA program SLOs using a rubric designed by our professors and used by the students’ MA committeea.
1. For students completing a PLAN A, the final thesis and oral defense would be evaluated.
2. For students completing a PLAN B, the final written paper and oral defense would be evaluated.
The primary reason for this decision was because the MA students assessed during Mūkīkī Wai were assessed against two of the five MA program SLOs (speaking and culture); therefore, Kawaihuelani needed an additional activity that targeted the other SLOs (reading, writing, listening, and research). Graduate faculty worked throughout the Spring 2012 semester developing rubrics for the thesis and Plan B final written product and the oral defense, accompanying score sheets, and an implementation process for the collection and review of these pieces of student work.
8) State how many persons submitted evidence that was evaluated. If applicable, please include the sampling technique used.
MŪKĪKĪ WAI
Of the 599 unduplicated students enrolled in our Hawaiian language courses in the Spring 2012 semester, 95 participated in Mūkīkī Wai this year. This comes out to a 16% participation rate.
See pie charts attached that illustrate these overall participation numbers as well as the break down for each level (first through fourth year, and graduate level).
Of the 95 students who participated, 82 submitted evidence that was evaluated for program assessment. These 82 students made up 34 individual/group entries in 4 categories enrolled in 200-600 level courses.
Of the 82 students who were assessed, 7 were graduate students all from HAW 602 Kākāʻōlelo (Oratory).
We acknowledge upfront that these numbers are probably not substantial enough to make generalizations about how our students are doing in general at meeting these two program SLOs, however they do provide a glimpse into our program that can serve as a reference point to compare data from other complementary assessment activities to like our rubric evaluation of MA Theses/Defenses, and these number will hopefully motive our colleagues to enter their students in Mūkīkī Wai in the near future in order to validate these initial findings with more substantial numbers.
MA THESIS/DEFENSE
Four students successfully completed and defended their research in Spring 2012, graduating with master’s degrees in Hawaiian, and one student defended her research and will graduate this academic year. Therefore, 4 theses/Plan B papers and 5 oral defenses were assessed. Faculty sitting on these five students’ committees met in early Summer 2012 to review and discuss both the papers and the defenses before agreeing as a committee on levels of performance for each product in terms of each SLO and overall.
9) Who interpreted or analyzed the evidence that was collected? (Check all that apply.)
Course instructor(s) Faculty committee Ad hoc faculty group Department chairperson Persons or organization outside the university Faculty advisor Advisors (in student support services) Students (graduate or undergraduate) Dean/Director Other: 10) How did they evaluate, analyze, or interpret the evidence? (Check all that apply.)
Used a rubric or scoring guide Scored exams/tests/quizzes Used professional judgment (no rubric or scoring guide used) Compiled survey results Used qualitative methods on interview, focus group, open-ended response data External organization/person analyzed data (e.g., external organization administered and scored the nursing licensing exam) Other: 11) For the assessment question(s) and/or assessment goal(s) stated in Question #6:
Summarize the actual results.
MŪKĪKĪ WAI
See bar charts attached that illustrate our results graphically (charts are summarized below).
For our graduate students, 100% of them were identified as meeting expectations in terms of the cultural knowledge and worldview SLO. In terms of language, 71% of students were identified as meeting expectations while 14% were identified as exceeding and 14% as approaching. No students were identified as not meeting expectations for both SLOs.
MA THESIS/DEFENSE
Tables 1 and 2 below provide a summary of the results of our assessment of student theses/Plan B final papers and oral defenses.
Table 1: Rubric Results for Hawaiian M.A. Student Plan A Theses or Plan B Non-Thesis (How many students were rated at each level? N=4)
|
Student Learning Objectives |
Exemplary/ Distinguished |
Competent/ Proficient |
|
Writing: Formal Research Paper |
2 |
2 |
|
Culture: Worldview, Cultural Norms |
0 |
4 |
|
Research: Culturally sensitive research project |
2 |
2 |
|
Reading |
3 |
1 |
|
OVERALL |
2 |
2 |
Table 2: Rubric Results for Hawaiian M.A. Student Plan A Defense or Plan B Public Presentation (How many students were rated at each level? N=5)
|
Student Learning Objectives |
Exemplary/ Distinguished |
Competent/ Proficient |
|
Speaking: Public Presentation |
2 |
3 |
|
Listening Comprehension |
3 |
2 |
|
Culture: Worldview, Cultural Norms |
1 |
4 |
|
OVERALL |
3 |
2 |
All five students were defined as competent or exemplary in the eyes of their committee members, spread evenly between the two levels for writing, research, speaking, listening, and overall performance. As these results confirm, this group of MA graduates in Hawaiian demonstrated a high quality of research. We are encouraged that their contributions will have a positive impact on the understanding of Hawaiian and the development of the teaching and learning of Hawaiian. An area that caught our attention was the culture SLO. None of the students scored exemplary in this area for their written work and only 1 of the 4 performed at this high level in their oral defense.
12) State how the program used the results or plans to use the results. Please be specific.
MŪKĪKĪ WAI
The results from this second implementation will be shared with our colleagues as a motivation to become involved. We hope to spark their curiosity to look closer and seek more data to substantiate (or refute) these findings by encouraging them to participate in Mūkīkī Wai (encouraging their students to enter, serving as evaluators, etc.) as well as the assessment activities of the program as a whole. We do not expect to use these results to make any dramatic changes or improvements to our program. We need to wait until we have data from a more substantial sampling before we can move forward in this way, which we hope to have as a result of motivating our colleagues to participate in Mūkīkī Wai 2013.
MA THESIS/DEFENSE
As stated in question #11 above, none of the students scored exemplary in this area for their written work and only 1 of the 4 performed at this high level in their oral defense. While we understand that reaching an exemplary level for culture in relation to language is difficult for any second language learner, especially at the master’s level, these results signal to us that more attention should be spent at the graduate level discussing, researching, and applying different vehicles of transmission (e.g., grammatical patterns, vocabulary, presentation of ideas, and cultural norms) in written and oral activities as a way to express Hawaiian worldview. Since 2011, the graduate faculty has been discussing different ways to approach this in our courses and this new data reaffirms that this should be something we continue to talk about and propose solutions to.
In addition to providing the graduate faculty with evidence of student achievement and challenges that will inform improvements to the program curriculum, this assessment activity will benefit our students as they complete the program by giving them insight into what is expected of them and what they should strive for, thus ensuring more widespread success among our majors. Now that these rubrics had been created, we plan to share them with all current and incoming M.A. students via the Kawaihuelani Graduate Student Handbook so they are not only familiar with the Program SLOs but are also clear about what elements make up each one and what they look like at different levels of proficiency.
13) Beyond the results, were there additional conclusions or discoveries?
This can include insights about assessment procedures, teaching and learning, program aspects and so on.
MŪKĪKĪ WAI
After our first pilot activity in Spring 2011, several lessons were learned and areas were identified for improvement in terms of the process and tools themselves. We hoped to address them in our second implementation this past Spring 2012, but we were unable to get to all of them, so we plan to do our best to address those that we did not get to in the upcoming Mūkīkī Wai Nā Mamo o Mānoa in Spring 2013, including…
- We hope that sharing the initial results from the pilot will motive more instructors to enter their students, especially those who teach HAW 202, 402, and our upper division content area courses because their students are those we are targeting specifically in to our program SLOs.
- Revise the rubrics for the Haku Puke category so this data can be used as well.
- Share rubrics with instructors well before the event in order to familiarize them with the tools and generate feedback so that adjustments can be made in time to share the tools with their students as well as the evaluators who will use them on the day of the event.
- Meet with the evaluators ahead of time to familiarize them with the rubrics and scores as well as engage them in norming conversations.
NOTE: The faculty member who was in charge of this particular assessment activity moved to another department on campus, so we will have to see what we are able to accomplish without her assistance next semester.
MA THESIS/DEFENSE
The process of developing the Thesis/Plan B and oral defense rubrics engaged our faculty in meaningful discussions about expectations for our master’s students, what student work at each level should look like, and what components, skills make up the essence of each Program SLO. These discussions were extremely valuable, since graduate faculty had never spent an extended period of time discussing these topics in such depth before. One professor took our draft rubrics to his HAW 602 oratory class to test with his students and get their feedback. He expressed that they helped him to conceptualize and breakdown expectations he had for his students but was never able to articulate in such detail.
14) If the program did not engage in assessment activities, please explain.
Or, if the program did engage in assessment activities, please add any other important information here.
In addition to Mūkīkī Wai, we plan to also continue the MA Thesis/defense assessment activities described above whenever a new graduate student is ready to defend and graduate. Results of these assessments will be included in next year’s annual assessment report.