Unit: East Asian Languages & Literatures
Program: East Asian Lang & Lit: Korean (MA)
Degree: Master's
Date: Fri Sep 27, 2013 - 4:17:43 pm

1) Below are your program's student learning outcomes (SLOs). Please update as needed.

[Korean Language & Linguistics]

1. Knowledge Base of Language and Linguistics   

 (a) Demonstrate familiarity with general topics and concepts fundamental to the field of Korean linguistics, including, historical linguistics, phonology, pedagogy, and syntax, and sociolinguistics. 

(b) Demonstrate comprehensive knowledge of key issues and theoretical approaches in the area of their specialization  

2. Utilization of Research    

(a) Review critically the purposes, approaches, designs, and analyses of published research in the field.  

(b) Conduct research (i.e., systematic and principled investigation) of the field and present the work orally and as a research paper.    

 3. Professionalism  

 (a) Write a research abstract and submit it to the (major) conferences in the field.  

 (b) Communicate effectively and productively with a group of peers/colleagues regarding research, teaching or other professional matters.   

 (c) Design and teach Korean language skill courses. 

[MA in Korean Language Flagship]     

1. Knowledge Base of Language and Linguistics   

 (a) Demonstrate familiarity with domain-specific terminology required for professionals, and its semantic association and collocational characteristics.      

(b) Understand ethonolinguistic, sociolinguistic, and pragmatic features of on- and off-line Korean materials.      

(c) Demonstrate familiarity with both South Korean and North Korean via extensive reading and listening of authentic materials.    

(d) Analyze the characteristics of Korean academic discourse at various levels.    

(e) Demonstrate understanding of the language, history, culture, and society of Korea from a plethora of on- and off-line professional and academic texts.  

2. Utilization of Research    

 (a) Review critically the purposes, approaches, designs, and analyses of published research in the field.  

 (b) Conduct research (i.e., systematic and principled investigation) of the field and present the work orally and as a research paper.

 3. Task Performance and Professionalism  

  (a) Communicate effectively and productively with a group of peers/colleagues regarding research, teaching or other professional matters.  

 (b) Understand domain-specific lectures or participate in conferences and communicate effectively and productively with colleagues regarding the content of those lectures and conferences.  

 (c) Prepare a professional portfolio for future careers and search for internships and jobs.     

[M.A. in Korean Literature]    

[In addition to the B.A. SLOs,] recipients of the M.A. in Korean literature should be able to: 

1. Read primary Korean literary texts of all major periods with appropriate academic tools. 

2. Read annotations in standard literary editions, use standard reference works, and read secondary sources in Korean with authority. 

3. Narrate a basic literary history of Korea. 

4. Demonstrate an ability to apply major Western and traditional East Asian theoretical and critical approaches to works of Korean literature. 

5. Conduct original research in his/her area of specialization. 

2) Your program's SLOs are published as follows. Please update as needed.

Department Website URL: www.hawaii.edu/eall
Student Handbook. URL, if available online:
Information Sheet, Flyer, or Brochure URL, if available online:
UHM Catalog. Page Number:
Course Syllabi. URL, if available online: http://www.hawaii.edu/eall/kor/courses.html
Other:
Other:

3) Select one option:

Curriculum Map File(s) from 2013:

4) For your program, the percentage of courses that have course SLOs explicitly stated on the syllabus, a website, or other publicly available document is as follows. Please update as needed.

0%
1-50%
51-80%
81-99%
100%

5) Did your program engage in any program assessment activities between June 1, 2012 and September 30, 2013? (e.g., establishing/revising outcomes, aligning the curriculum to outcomes, collecting evidence, interpreting evidence, using results, revising the assessment plan, creating surveys or tests, etc.)

Yes
No (skip to question 14)

6) For the period June 1, 2012 to September 30, 2013: State the assessment question(s) and/or assessment goals. Include the SLOs that were targeted, if applicable.

[MA in Korean Language Flagship]

Assessment questions:

1)    What are the students’ current language and content proficiency levels measured by the assessment battery?

2)    Are these assessment tools appropriate to test students’ knowledge of language and content?

3)    How can we utilize the results of assessment?

Goals of assessment activities

1)    Appropriate diagnose of the students proficiency levels

2)    Improvement of the assessment battery and applying the results

3)    Provision of exact information to the instructors regarding students’ current proficiency level and knowledge about content areas

4)    Confirmation of students’ achievement or requirement mandated by the program

With these assessment questions and goals in mind, Korean Language Flagship MA program utilizes various assessment tools (see question #7 for details). The results of these tools are used for student advising (or counseling) and program revision.  

All SLOs were targeted, studied, and assessed throughout the program (tests, homework, task performances, journals, student conferences, term papers and scholarly paper) 

7) State the type(s) of evidence gathered to answer the assessment question and/or meet the assessment goals that were given in Question #6.

No specific questions were asked. However, curriculum committees in each academic program regularly review data such as enrollment figures, results of diagnostic and comprehensive examinations, and student surveys to assess how effectively it meets its objectives. This includes revising curriculum, course requirements, exams, etc.

[MA in Korean Language Flagship]

[Flagship]

1. Plan B only: Oral defense and scholarly paper

Students are required to conduct research in their domain (e.g., political science, sociology, computer engineering, law, business, etc.) during the overseas program. They choose a topic in their chosen field, and conduct research using materials in Korean. Students are encouraged to make use of information obtained from Korea University (KU) subject-matter courses and through an internship in Korea. During the fourth semester of the program (the second semester at KU), students make a progress report once a month and receive feedback and comments from KU faculty. At the end of the semester, students make an oral presentation and submit a paper on their research finding. The oral defenses of the scholarly papers are conducted in front of the UHM and KU Flagship Faculty.

2. Participation in the College of LLL student exit survey (satisfaction with the program, learning outcomes, plans after graduation, suggestions for program improvement, etc.). 

We received reports on response data from the Korean Flagship MA for Professionals exit survey for students graduating in summer/fall of 2010, and spring of 2011. A total of 9 students were invited to respond; 8 students completed the survey (response rate=89%).

3. Internship log in Korea

Supervisors or mentors in students’ internship sites provide a monthly log which is reviewed by KU and UHM Flagship faculty.

4. Site visit to the overseas program in KU

Once a year, UHM faculty visits the overseas program in Korea to evaluate the program and students’ progress.

5. Preliminary conference with students

In the beginning of an academic year, faculty members have an individual conference with each student to accommodate students’ needs and suggestions.

6. Student Monthly journal

Every Month, students write a journal to reflect their performance during the past month.

7. Meeting (communication) with tutors

The program tutor coordinator meets with language tutors and students to talk about students’ progress, study content, and suggestions regarding students and the program.

8. Student performance

Weekly students’ performance data (e.g., individual presentation, discussion, debate, survey, and negotiation) are collected and reviewed by faculty. Students’ final presentations of each semester are also major assessment tools of how they are progressing along the SLOs.

9. Research papers

Students write research papers regarding topics or issues of their own majors or interest fields. These papers are also very effective assessments along with other evidence.

10. Professional-level proficiency in Korean

Students must demonstrate professional-level proficiency (ILR Level 3, Interagency Language Roundtable scales) in Korean at the end of the program by oral proficiency interviews and other ILR-based test batteries (Reading, Listening and Writing). Flagship also administers institutional TOPIK (Test of Proficiency in Korean) to measure students’ general proficiency.

8) State how many persons submitted evidence that was evaluated. If applicable, please include the sampling technique used.

[MA in Korean Language Flagship]

1. Plan B only: Oral defense and scholarly paper

All graduates of Spring/Summer 2012 (9 students) and Spring/Summer 2013 (5 students).

2. Participation in the College of LLL student exit survey (satisfaction with the program, learning outcomes, plans after graduation, suggestions for program improvement, etc.). 

We received reports on response data from the Korean Flagship MA for Professionals exit survey for students graduating in spring/summer of 2012, and spring of 2013. A total of 14 students were invited to respond; 8 students completed the survey (response rate=57%).

3. Internship log in Korea

All supervisors or mentors in students’ internship sites

4. Site visit to the overseas program in KU

This summer three visited Korea University for this.

5. Preliminary conference with students

Three faculty members and all the students in the program.

6. Student Monthly journal

All students

7. Student performance

All students

8. Research papers

All students

9. Professional-level proficiency in Korean

All students

10. Professional-level proficiency in Korean

14 graduates were tested on Institutional ACTFL OPI (speaking) and KLFC (Korean Language Flagship Center) proficiency test (ILR-based, Reading and Listening).

Speaking (OPI): Advanced High (2); Superior (12)

Reading (ILR): Level 3 (10); Level 2 (4)

Listening (ILR): Level 3 (12); Level 2 (2)

9) Who interpreted or analyzed the evidence that was collected? (Check all that apply.)

Course instructor(s)
Faculty committee
Ad hoc faculty group
Department chairperson
Persons or organization outside the university
Faculty advisor
Advisors (in student support services)
Students (graduate or undergraduate)
Dean/Director
Other:

10) How did they evaluate, analyze, or interpret the evidence? (Check all that apply.)

Used a rubric or scoring guide
Scored exams/tests/quizzes
Used professional judgment (no rubric or scoring guide used)
Compiled survey results
Used qualitative methods on interview, focus group, open-ended response data
External organization/person analyzed data (e.g., external organization administered and scored the nursing licensing exam)
Other:

11) For the assessment question(s) and/or assessment goal(s) stated in Question #6:
Summarize the actual results.

1. Plan B only: Oral defense and scholarly paper

14 students graduated the program during the report period (June 1, 2012 ~ September 30, 2013). All of them successfully defend their research results and argument of the scholarly paper.

2. Participation in the College of LLL student exit survey

As for questions about satisfaction with the program, most students responded satisfied except for satisfaction with the student morale, adequacy of departmental information provided, and research opportunities in and outside of class which was mostly ‘Neutral.’ or ‘somewhat dissatisfied.’  Because of the rigorousness of the program, students improved their proficiency levels. It seems that they need more guidance regarding their progress and achievement. The curriculum requires ample opportunities for research.  Students don’t seem to feel that the research they conduct every week for classes is more or less homework, not ‘real’ research like research for their scholarly papers. Students responded that they want more variety in courses and flexibility to pursue academic interest within individual courses. Accordingly, the program included more elective courses since then, so we hope this issue will be resolved in the future. Regarding program’s financial support, students are overall satisfied except for teaching opportunities. The program secured one TA-ship and another teaching opportunity in a local high school since they graduated.  Students seem to be dissatisfied with classroom facilities and the computer lab. Finally, even though students had faced a few issues mentioned above, most students answered that they achieved the SLOs very well.                  

3. Professional-level proficiency in Korean

The following is the results of the 14 graduates’ language proficiency levels tested by Institutional ACTFL OPI (speaking) and KLFC (Korean Language Flagship Center) proficiency test (ILR-based, Reading and Listening).

Speaking (OPI): Advanced High (2); Superior (12)

Reading (ILR): Level 3 (10); Level 2 (4)

Listening (ILR): Level 3 (12); Level 2 (2)

12) State how the program used the results or plans to use the results. Please be specific.

The following program modifications were made based on information gathered about our students' learning experiences and outcomes:

1. We will make the information about our graduate courses more available to our students (e.g., post them on the department website)

 2.  In the Korean Linguistics section, we are discussing major changes in the graduation requirements for MA in Korean Linguistics (e.g., course requirements, the elimination of comprehension exams, requirement on scholarly paper, etc.)

The 2012-2013 survey data were distributed among EALL graduate faculty members. We discussed the results at a recent EALL graduate faculty meeting. Other types of data on student learning are normally discussed in individual sections.

[MA in Korean Language Flagship]

All other evidence mentioned above (Question 7) is collected and reviewed by KU and UHM Flagship faculty or committee (consisting of UHM Flagship faculty), and the results are utilized to improve the program.   

13) Beyond the results, were there additional conclusions or discoveries?
This can include insights about assessment procedures, teaching and learning, program aspects and so on.

We will develop strategies for improving the response rate of the exit survey.

14) If the program did not engage in assessment activities, please explain.
Or, if the program did engage in assessment activities, please add any other important information here.

In order to better respond to the internal and external needs for assessment, the EALL Assessment Committee has been formed.