Program: Japanese (BA)
Degree: Bachelor's
Date: Sun Nov 11, 2018 - 5:14:46 pm
1) Program Student Learning Outcomes (SLOs) and Institutional Learning Objectives (ILOs)
1. [SLO1: ORAL] Engage in oral communication in Japanese in various social contexts, in linguistically and culturally appropriate ways.
(1b. Specialized study in an academic field, 2c. Communicate and report, 3a. Continuous learning and personal growth, 3b. Respect for people and cultures, in particular Hawaiian culture)
2. [SLO2: READING] Read and comprehend texts written in Japanese from a variety of genres and contexts (e.g., newspapers, essay collections, novels).
(1b. Specialized study in an academic field, 2a. Think critically and creatively, 3a. Continuous learning and personal growth, 3b. Respect for people and cultures, in particular Hawaiian culture)
3. [SLO3: WRITING] Apply critical thinking and rhetorical skills to produce coherent written works and presentations in both English and Japanese.
(1b. Specialized study in an academic field, 2a. Think critically and creatively, 2b. Conduct research, 3a. Continuous learning and personal growth)
4. [SLO4: RESEARCH] Use a variety of Japanese reference works and sources, including dictionaries and encyclopedias both in book form and on the internet.
(1b. Specialized study in an academic field, 2a. Think critically and creatively, 3a. Continuous learning and personal growth)
5. [SLO5: RESEARCH] Conduct independent research on topics in Japanese literature and/or linguistics, and effectively communicate the results.
(1b. Specialized study in an academic field, 2a. Think critically and creatively, 2b. Conduct research, 2c. Communicate and report)
6. [SLO6: LINGUISTICS] Demonstrate an understanding of phonology, morphology, syntax, and semantics through analysis of words, phrases, and clauses from authentic Japanese samples.
(1b. Specialized study in an academic field)
7. [SLO7: LITERATURE] Identify and describe major authors, works, features, forms, and styles of Japanese literature, both premodern and modern.
(1b. Specialized study in an academic field, 3b. Respect for people and cultures, in particular Hawaiian culture)
8. [SLO8: LITERATURE] Analyze and interpret works of Japanese poetry, prose, and drama, read both in translation and in the original Japanese, using terms appropriate to each genre.
(1b. Specialized study in an academic field, 2a. Think critically and creatively, 3b. Respect for people and cultures, in particular Hawaiian culture)
9. [SLO9: LITERATURE] Situate and evaluate Japanese literature in its social, historical, intellectual, and religious contexts.
(1b. Specialized study in an academic field, 2a. Think critically and creatively, 3b. Respect for people and cultures, in particular Hawaiian culture)
2) Your program's SLOs are published as follows. Please update asneeded.
Department Website URL: http://www.hawaii.edu/eall/download/JPN%20BA%20curriculum%20map.pdf
Student Handbook. URL, if available online: Information Sheet, Flyer, or Brochure URL, if available online: UHM Catalog. Page Number: Course Syllabi. URL, if available online: Other: Other: 3) Please review, add, replace, or delete the existing curriculum map.
- File (03/16/2020)
4) For your program, the percentage of courses that have course SLOs explicitly stated on the syllabus, a website, or other publicly available document is as follows. Please update as needed.
0% 1-50% 51-80%
81-99% 100%5) Does the program have learning achievement results for its program SLOs? (Example of achievement results: "80% of students met expectations on SLO 1.")(check one):
No Yes, on some(1-50%) of the program SLOs Yes, on most(51-99%) of the program SLOs Yes, on all(100%) of the program SLOs6) Did your program engage in any program learning assessment activities between June 1, 2015 and October 31, 2018?
Yes No (skip to question 17)7) What best describes the program-level learning assessment activities that took place for the period June 1, 2015 to October 31, 2018? (Check all that apply.)
Create/modify/discuss program learning assessment procedures (e.g., SLOs, curriculum map, mechanism to collect student work, rubric, survey) Collect/evaluate student work/performance to determine SLO achievement Collect/analyze student self-reports of SLO achievement via surveys, interviews, or focus groups Use assessment results to make programmatic decisions (e.g., change course content or pedagogy, design new course, hiring) No (skip to question 17) Investigate other pressing issue related to student learning achievement for the program (explain in question 7) Other: Investigate curriculum coherence8) Briefly explain the assessment activities that took place.
The Japanese Language Committee analyzed the results of the exit surveys over the past three years. This data was used to formulate new major requirements that went into effect in Fall 2018. It was also used in planning a new first- and second-year Japanese language curriculum, which was rolled out in Fall 2017.
We have also reviewed the results from a standard language proficiency test, the Japanese Computerized Adaptive Test (J-CAT), which was administered to students in JPN 407D and JPN 407E in AY2016-17. These results were used to gauge the effectiveness of our language programs.
In addition, we have analyzed student oral performance, graded according to a rubric, in JPN 402, a required Japanese language course.
Finally, we reviewed student grades in JPN 350 and EALL 271-272, courses where students gain the skills in knowledge mentioned in SLO 6 and 7.
9) What types of evidence did the program use as part of the assessment activities checked in question 7? (Check all that apply.)
Artistic exhibition/performance Assignment/exam/paper completed as part of regular coursework and used for program-level assessment Capstone work product (e.g., written project or non-thesis paper) Exam created by an external organization (e.g., professional association for licensure) Exit exam created by the program IRB approval of research Oral performance (oral defense, oral presentation, conference presentation) Portfolio of student work Publication or grant proposal Qualifying exam or comprehensive exam for program-level assessment in addition to individual student evaluation (graduate level only) Supervisor or employer evaluation of student performance outside the classroom (internship, clinical, practicum) Thesis or dissertation used for program-level assessment in addition to individual student evaluation Alumni survey that contains self-reports of SLO achievement Employer meetings/discussions/survey/interview of student SLO achievement Interviews or focus groups that contain self-reports of SLO achievement Student reflective writing assignment (essay, journal entry, self-assessment) on their SLO achievement. Student surveys that contain self-reports of SLO achievement Assessment-related such as assessment plan, SLOs, curriculum map, etc. Program or course materials (syllabi, assignments, requirements, etc.) Other 1: Other 2: 10) State the number of students (or persons) who submitted evidence that was evaluated. If applicable, please include the sampling technique used.
118 out of 125 seniors graduating between AY2015-16 and AY2017-18 responded to the exit survey.
43 students taking JPN 407D and E, required courses for the major, took the J-CAT exam in AY 2016-17. We used this standardized exam to measure the reading proficiency of our students.
We looked at oral presentation grades for 29 students in Japanese 402, the last course in the language sequence to have a strong oral component.
11) Who interpreted or analyzed the evidence that was collected? (Check all that apply.)
Course instructor(s) Faculty committee Ad hoc faculty group Department chairperson Persons or organization outside the university Faculty advisor Advisors (in student support services) Students (graduate or undergraduate) Dean/Director Other: 12) How did they evaluate, analyze, or interpret the evidence? (Check all that apply.)
Used a rubric or scoring guide Scored exams/tests/quizzes Used professional judgment (no rubric or scoring guide used) Compiled survey results Used qualitative methods on interview, focus group, open-ended response data External organization/person analyzed data (e.g., external organization administered and scored the nursing licensing exam) Other: 13) Summarize the results of the assessment activities checked in question 7. For example, report the percentage of students who achieved each SLO.
SLO1, Engage in oral communication in Japanese in culturally appropriate ways.
Our primary instrument for measuring achievement of this SLO is the oral presentation given in JPN 402, the last course in the language sequence with a significant oral component. The final oral presentation in this course is graded on a rubric that emphasizes the comprehensibility and clarity of content, as well as the appropriateness of language used. In the Spring 2018, 28 out of 29 students received a B or better for this presentation, with 22 out of 29 receiving an A- or better. In assessing this data, the JLC concluded that the results indicated that, in relation to presentational skills, 97% of students are achieving the expected level of oral proficiency for students completing four years in our language program.
On the exit surveys, 97 out of 118 students (82%) felt that they had achieved this outcome adequately to very well, with the largest number rating themselves a “4.”
[READING]
SLO2, Read and comprehend texts written in Japanese from a variety of genres and contexts(e.g., newspapers, essay collections, novels).
We measured this outcome using the Japanese Computerized Adaptive Test (J-CAT) that was administered to students taking JPN 407B, 407D, and 407E (the highest level required language courses) in AY 2016-17. Based on the reading section scores, the JLC concluded that 14% of our advanced students were approaching expectations, 23% met expectations, and 63% exceeded expectations.
On the exit surveys, 105 out of 118 students (89%) felt that they had achieved this outcome adequately to very well.
[WRITING]
SLO3, Apply critical thinking and rhetorical skills to produce coherent written works and presentations in both English and Japanese.
On the exit surveys, 108 out of 118 students (92%) felt that they had achieved this outcome adequately to very well.
[RESEARCH]
SLO4 Use a variety of Japanese reference works and, including dictionaries and encyclopedias both in book form and on the internet
On the exit surveys, 108 out of 118 students (92%) felt that they had achieved this outcome adequately to very well.
[RESEARCH]
SLO5, Conduct independent research on topics in Japanese literature and/or linguistics, and effectively communicate the results.
On the exit surveys, 104 out of 118 students (88%) felt that they had achieved this outcome adequately to very well.
[LINGUISTICS]
SLO6, Demonstrate an understanding of phonology, morphology, syntax, and semantics through analysis of words, phrases, and clauses from authentic Japanese samples.
Using data from JPN 350, the Japanese linguistics survey course that covers the listed areas, we determined that 74% of majors had met or exceeded expectations.
On the exit surveys, 105 out of 118 students (89%) felt that they had achieved this outcome adequately to very well.
[LITERATURE]
SLO7, Identify and describe major authors, works, features, forms, and styles of Japanese literature, both premodern and modern.
Reviewing data from JPN 271 and 272, Japanese literature survey courses, we determined that 86% of concentrators had met or exceeded expectations.
On the exit surveys, 96 out of 118 students (81%) felt that they had achieved this outcome adequately to very well.
[LITERATURE]
SLO 8, Analyze and interpret works of Japanese poetry, prose, and drama, read both in translation and in the original Japanese, using terms appropriate to each genre.
On the exit surveys, 95 out of 118 students (80%) felt that they had achieved this outcome adequately to very well.
[LITERATURE]
SLO 9, Situate and evaluate Japanese literature in its social, historical, intellectual, and religious contexts.
On the exit surveys, 102 out of 118 students (86%) felt that they had achieved this outcome adequately to very well.
14) What best describes how the program used the results? (Check all that apply.)
Assessment procedure changes (SLOs, curriculum map, rubrics, evidence collected, sampling, communications with faculty, etc.) Course changes (course content, pedagogy, courses offered, new course, pre-requisites, requirements) Personnel or resource allocation changes Program policy changes (e.g., admissions requirements, student probation policies, common course evaluation form) Students' out-of-course experience changes (advising, co-curricular experiences, program website, program handbook, brown-bag lunches, workshops) Celebration of student success! Results indicated no action needed because students met expectations Use is pending (typical reasons: insufficient number of students in population, evidence not evaluated or interpreted yet, faculty discussions continue) Other: 15) Please briefly describe how the program used the results.
Overall, all of the assessment data indicated that the program was meeting its goals. We undertook two new assessment activities this cycle: 1)Having students in the highest required level of Japanese take the Japanese Computerized Adaptive Test (J-CAT). 2) Examining the oral presentation scores from JPN 402, the highest-level language course with a strong oral component. These assessment activities were used to evaluate the reading and oral proficiencies of our student. They showed that students were, on the whole, meeting or exceeding the SLOs in these areas. The data from exit surveys was also strong, with student self-evaluations showing that a large majority of our graduates felt that they had attained the entire range of learning objectives. This was underlined by the many positive comments in the surveys.
In the context of these positive overall results, there was a pattern, in the exit surveys, of students desiring a stronger oral component in their language training. To further respond to such wishes, the Japanese language program is designing and implementing an entirely new 1st- and 2nd-year curriculum emphasizing communicative competence. We began a gradual roll-out of this new program starting in Fall 2017; we will expect to see some results from the new program reflected in the next 3-year assessment.
We also observed in the exit surveys comments from students who felt that some advanced language and literature courses were not being offered at adequate frequency. While this situation ultimately needs to be addressed through restoring lost faculty positions, we have tried to reduce the impact on students by instituting new major requirements, starting in Fall 2018, that introduce more flexibility in upper-level course elections.
16) Beyond the results, were there additional conclusions or discoveries? This can include insights about assessment procedures, teaching and learning, and great achievements regarding program assessment in this reporting period.
We would like to no more about how assessment results are used beyond the department level. For example, who reads them at the level of the college and UHM? Are they used in the allocation of resources? Is assessment for accreditation purposes?
17) If the program did not engage in assessment activities, please justify.
Assessment activities and results have been detailed above.