Program: Hawaiian (BA)
Degree: Bachelor's
Date: Mon Oct 24, 2011 - 2:20:49 pm
1) Below are your program student learning outcomes (SLOs). Please update as needed.
Kawaihuelani has two sets of program curriculum.
1) Upon completion of two years of Hawaiian language (HAW 101-202), our students should be able to . . .
Speaking:
|
Listening:
|
Reading:
|
Writing:
|
|
Cultures, Comparisons, Connections, Communities:
|
2) Upon completion of a B.A. in Hawaiian Language, our students should be able to . . .
|
Speaking |
|
|
Listening |
|
|
Reading |
|
|
Writing |
|
|
Cultures, Comparisons, Connections, & Communities |
|
2) Your program's SLOs are published as follows. Please update as needed.
Department Website URL: website still under construction Student Handbook. URL, if available online: NA
Information Sheet, Flyer, or Brochure URL, if available online: UHM Catalog. Page Number: Course Syllabi. URL, if available online: Other: Other: 3) Below is the link(s) to your program's curriculum map(s). If we do not have your curriculum map, please upload it as a PDF.
- File (03/16/2020)
4) For your program, the percentage of courses that have course SLOs explicitly stated on the syllabus, a website, or other publicly available document is as follows. Please update as needed.
0% 1-50%
51-80% 81-99% 100%5) For the period June 1, 2010 to September 30, 2011: State the assessment question(s) and/or assessment goals. Include the SLOs that were targeted, if applicable.
The two main program SLOs our program targeted during this period focused on verbal and non-verbal language as well as application of cultural knowledge and worldview.
Upon completion of a B.A. in Hawaiian, students should be able to . . .
- Present information, concepts, and ideas to an audience of listeners at an advanced level of fluency and accuracy.
- Expand and broaden appropriate application of Hawaiian language and culture appropriately in authentic settings within and beyond the classroom in order to connect to other disciplines, contexts, and domains, access information, and interact with others in the Hawaiian language community.
Our program was interested in seeing how well and to what extent our students were meeting these select program SLOs. While we focused our attention on our fourth year students for whom these program SLOs were intended, we also looked at data related to our second and third years students as a way to see how they were approaching these final outcomes and how they compared to our more advanced students.
6) State the type(s) of evidence gathered to answer the assessment question and/or meet the assessment goals that were given in Question #5.
Our program identified an existing Hawaiian language activity, Mūkīkī Wai Nā Mamo o Mānoa, that our students already participate in as a place to collect evidence to answer our assessment question above. It is a yearly multifaceted Hawaiian language celebration/competition developed in the 2004-2005 academic year by several of our Kawaihuelani faculty members. It was created as a venue to showcase students’ achievements in Hawaiian language acquisition of traditional performance forms as well as contemporary communication mediums such as theatre, film and multi-media presentations. The annual competition strategically includes appropriate categories for university students from various levels of language learning (beginning, intermediate, advanced and graduate level). Students aspire to reach higher levels of language fluency and competency as they strengthen their language skills through participating in the festival.
Student work entered in the following categories of the competition were assessed: Haʻi ʻŌlelo (multimedia speech presentations), Hana Keaka (theater), Haku Puke (book composition), Haku Wikiō (video production), Haku Mele (song composition), and Haʻi ʻŌlelo Hōʻeuʻeu (persuasive speeches presented by our graduate students).
7) State how many persons submitted evidence that was evaluated. If applicable, please include the sampling technique used.
Of the 571 unduplicated students enrolled in our Hawaiian language courses in the Spring 2011 semester, 115 participated in Mūkīkī Wai this year. This comes out to a 17% participation rate.
Of the 115 students who participated, 74 submitted evidence that was evaluated for program assessment. These 74 students made up 39 individual/group entries in 6 categories enrolled in 200-600 level courses.
Of the 74 students who were assessed, 38 were from second year, 16 were from third year, 14 were from fourth year, and 6 were graduate students.
We acknowledge upfront that these numbers are probably not substantial enough to make generalizations about how our students are doing in general at meeting these two program SLOs, however they do provide a glimpse into our program that will hopefully motive our colleagues to enter their students in Mūkīkī Wai in the near future in order to validate these initial findings with more substantial numbers.
8) Who interpreted or analyzed the evidence that was collected? (Check all that apply.)
Course instructor(s) Faculty committee Ad hoc faculty group Department chairperson Persons or organization outside the university Faculty advisor Advisors (in student support services) Students (graduate or undergraduate) Dean/Director Other: 9) How did they evaluate, analyze, or interpret the evidence? (Check all that apply.)
Used a rubric or scoring guide Scored exams/tests/quizzes Used professional judgment (no rubric or scoring guide used) Compiled survey results Used qualitative methods on interview, focus group, open-ended response data External organization/person analyzed data (e.g., external organization administered and scored the nursing licensing exam) Other: 10) For the assessment question(s) and/or assessment goal(s) stated in Question #5:
Summarize the actual results.
See bar charts that illustrate our results graphically (charts are summarized below).
In terms of the cultural knowledge and worldview SLO, 54% of students were identified as meeting expectations, while 26% were exceeding and 21% were approaching. No students were identified as not meeting expectations for this SLO. In terms of the verbal and non-verbal language SLO, 54% were identified as meeting, 18% were exceeding, 26% were approaching, and 3% were not meeting.
When looking more closely at the breakdown of results by level, our second and third year students did very well when assess against our program SLOs even though they are not at the end of their program yet. While the majority was assessed at either approaching or meeting, there were several who were assessed at the exceeding level, but this was mostly in terms of the cultural knowledge and worldview SLOs. No second year students were identified as exceeding in terms of the language SLO and only 13% of our third year students identified as exceeding in terms of the language SLO, which is to be expected. What was not expected was how well our fourth year students performed: none were identified as not meeting and the majority was either meeting or exceeding for both SLOs. This is great news! The only area of our results that is a little puzzling is that our second year students appeared to perform better than our third year students in terms of both culture and language. It should be considered, however, than they were entered in different categories, and we also noticed that the rubric for one of the third year categories (Haku Puke) was somewhat flawed and could have affected their results. More investigation needs to take place to account for this finding as well as a larger sample size.
11) State how the program used the results or plans to use the results. Please be specific.
This assessment activity was a pilot to test out our new rubrics and get familiar with the data collection, analysis, and reporting process. While the results are encouraging and we should be proud of how well our students fulfilled the target SLOs, the results from this pilot will be shared with our colleagues at an upcoming faculty meeting as more of a motivation to become involved. We hope to spark their curiosity to look closer and seek more data to substantial (or refute) these initial findings by encouraging them to participate in Mūkīkī Wai (encouraging their students to enter, serving as evaluators, etc.) as well as the assessment activities of the program as a whole. We do not expect to use these initial results to make dramatic changes or improvements to our program. We need to wait until we have data from a more substantial sampling before we can move forward in this way, which we hope to have as a result of motivating our colleagues to participate in Mūkīkī Wai 2012.
12) Beyond the results, were there additional conclusions or discoveries?
This can include insights about assessment procedures, teaching and learning, program aspects and so on.
After this pilot activity, several lessons were learned and areas were identified for improvement in terms of the process and tools themselves. We list a few of them below, which we hope to address in our second implementation at the upcoming Mūkīkī Wai Nā Mamo o Mānoa in Spring 2012.
- As mentioned above, we hope that sharing the initial results from the pilot will motive more instructors to enter their students, especially those who teach HAW 202, 402, and our upper division content area courses because their students are those we are targeting specifically in to our program SLOs.
- Revise the rubrics for the Haku Mele and Haku Puke categories.
- Share rubrics with instructors well before the event in order to familiarize them with the tools and generate feedback so that adjustments can be made in time to share the tools with their students as well as the evaluators who will use them on the day of the event.
- Meet with the evaluators ahead of time to familiarize them with the rubrics and scores heets as well as engage them in norming conversations.
13) Other important information.
Please note: If the program did not engage in assessment, please explain. If the program created an assessment plan for next year, please give an overview.
On September 15, 2011, our undergraduate faculty participated in an evaluation/assessment retreat/workshop with the assistance of the UHM Assessment Office in order to not only review our curriculum as a whole but also to prepare for and make decisions about our upcoming UHM program review and accreditation application to WINHEC. The goals for the retreat were:
- to review the program review and accreditation process in order to increase participation by all faculty and their students;
- to re-familiarize ourselves with our Undergraduate Program Learning Objectives and Curriculum Map, making any changes or adjustments as needed;
- to develop standard Course Learning Objectives for several of our required courses (HAW 202 & 402) that all instructors agree upon;
- to identify significant assignments from specific courses/experiences/events at both BA and MA level that can be used to assess whether our students are in fact meeting our Program SLOs so we can start collecting this data in Spring 2012;
- to decide upon appropriate tools to develop after the retreat to use to assess these collected assignments (rubrics, etc.).
Here are the outcomes of this retreat.
- Standardized course SLOs were developed for HAW 202 & 402 and were agreed upon by all instructors present.
- Two assignments/artifacts were identified for both HAW 202 (demonstration speech & exit interview) & 402 (oral presentation & research paper) that we can collect and evaluate for program assessment purposes using a rubric designed by our instructors.
Next Steps
-
Follow up with faculty about the decisions they made in terms of assignments/artifacts to collect, the rubric, and the process, such as:
- Since many of our HAW 202 teachers did not attend this portion of the retreat, we need to vet the decisions made at the meeting to ensure that demonstration speeches are in fact something all HAW 202 teachers already assign.
- For HAW 202, we need to flesh out what we mean by an exit interview (who conducts, how many students, purpose to test language competency OR to tease our perceptions and opinions about experience in the program OR both)
- How do these artifacts meet all our Program SLOs?
- Who will work with Maya to develop the rubrics and the exit interview questions?
- Who will use the rubrics to do the assessment (the instructor, a committee of instructors, internal and external people)?
- How many of each artifact do we need? From how many different sections?