Degree Program Assessment Reports

1. Demonstrate critical thinking and awareness of issues within the context of their professional work and social practice.

(1b. Specialized study in an academic field, 2a. Think critically and creatively)

2. Demonstrate an understanding of the value bases of their professional work.

(1b. Specialized study in an academic field, 2c. Communicate and report)

3. Understand and interpret the history of second and foreign language study and its contemporary issues.

(1b. Specialized study in an academic field)

4. Critically evaluate and make use of research into the learning, use, structure, and/or pedagogy of second languages.

(1b. Specialized study in an academic field, 2a. Think critically and creatively)

5. Develop and apply sound frameworks in the assessment and evaluation of institutions and agents involved in second language instruction, planning, and policy.

(1b. Specialized study in an academic field)

6. Show an understanding of local language issues of Hawai'i and the Pacific in their professional work.

(1b. Specialized study in an academic field, 1c. Understand Hawaiian culture and history, 3b. Respect for people and cultures, in particular Hawaiian culture)

7. Be able to support language minority students' development of academic and/or professional literacies.

(1b. Specialized study in an academic field, 1c. Understand Hawaiian culture and history)

Curriculum Map File(s) from 2018:

• File (03/16/2020)

No
Yes, on some(1-50%) of the program SLOs
Yes, on most(51-99%) of the program SLOs
Yes, on all(100%) of the program SLOs

Create/modify/discuss program learning assessment procedures (e.g., SLOs, curriculum map, mechanism to collect student work, rubric, survey)
Collect/evaluate student work/performance to determine SLO achievement
Collect/analyze student self-reports of SLO achievement via surveys, interviews, or focus groups
Use assessment results to make programmatic decisions (e.g., change course content or pedagogy, design new course, hiring)
No (skip to question 17)
Investigate other pressing issue related to student learning achievement for the program (explain in question 7)
Other:

Artistic exhibition/performance
Assignment/exam/paper completed as part of regular coursework and used for program-level assessment
Capstone work product (e.g., written project or non-thesis paper)
Exam created by an external organization (e.g., professional association for licensure)
Exit exam created by the program
IRB approval of research
Oral performance (oral defense, oral presentation, conference presentation)
Portfolio of student work
Publication or grant proposal
Qualifying exam or comprehensive exam for program-level assessment in addition to individual student evaluation (graduate level only)
Supervisor or employer evaluation of student performance outside the classroom (internship, clinical, practicum)
Thesis or dissertation used for program-level assessment in addition to individual student evaluation
Alumni survey that contains self-reports of SLO achievement
Employer meetings/discussions/survey/interview of student SLO achievement
Interviews or focus groups that contain self-reports of SLO achievement
Student reflective writing assignment (essay, journal entry, self-assessment) on their SLO achievement.
Student surveys that contain self-reports of SLO achievement
Assessment-related such as assessment plan, SLOs, curriculum map, etc.
Program or course materials (syllabi, assignments, requirements, etc.)
Other 1: Information collected from faculty/GA/staff workshops.
Other 2: Other data collected from LLL exit survey. Advisor reflections from student-advising sessions. Other evidence that arose from BA Committee meetings.

Course instructor(s)
Faculty committee
Ad hoc faculty group
Department chairperson
Persons or organization outside the university
Faculty advisor
Advisors (in student support services)
Students (graduate or undergraduate)
Dean/Director
Other: Associate Dean of the College of LLL, SLS BA Committee members.

·       Curriculum map.  The SLS BA Committee reviewed the SLO areas receiving coverage for the four courses that were identified by the Manoa Assessment Committee as being potentially problematic (SLS 312, 313, 408 and 480-alpha).  The number of SLO areas covered were reduced for SLS 312, 313, and 408.  However, because SLS 480-alpha are variable-topics courses, covering a very wide range of topics in second language pedagogy, research, use, analysis, and learning, it was agree that this course should, indeed, include coverage of all 7 SLO areas. Additionally, the committee added new SLS courses to the curriculum map.

·       Actions from GA/Faculty/Staff workshops.  The SLS BA Committee discussed the information that was garnered from the GA/faculty/staff workshops, identified three main areas for making improvements, and began the process of making those improvements.

o   A Laulima project site was created to provide downloadable instructor resources for the BA program. Multiple syllabi for all courses in the BA program were included, and some courses included complete sets of lesson plans and other materials.  This site will continue to be developed in the coming years.

o   A new GA orientation packet, in electronic format, was developed and included in the Laulima project site.  (The packet includes information that would also be useful for faculty who are teaching in the BA program for the first time.)  The packet will be piloted in Spring 2019 and Fall 2019, and revised as needed.

o   Each SLS course has been assigned a faculty resource person (with that faculty person’s agreement).  The roles and duties of faculty resource people was reviewed, and the Undergraduate Chair began working with each faculty member in regard to these roles and duties.

·       Data from College of LLL exit survey.  Every semester, the College of Languages, Linguistics, and Literatures conducts an exit survey for each of its programs, and provides compiled results to each department.  The surveys include Likert-scale items and also student comments.  The BA Coordinator and SLS Chair reviewed the results and followed up, either individually or via the BA Committee in three main areas: student advising, opportunities for students to gain relevant experience, and the identification of a course that was in need of committee review.  Survey data also included graduating students’ self-reporting of their attainment of the program level outcomes.

o   Student advising.  One of the questions in the LLL exit survey ask “Was advising in the department helpful?”  On a 1-5 Likert scale, where 5 was “Definitely Yes” and 1 was “Definitely No”, the mean was 4.9375 (with 45 out of 48 students giving a score of 5, and 3 out of 48 giving a score of 4).  Based on this information, it was clear that SLS undergraduate advisors should continue to do their advising in much the same way.  (Note that this involves spending 30-60 minutes for each student during mandatory advising sessions, taking time to get to know each student, including but not limited to their academic and career goals, and also being available outside of the mandatory advising period.  While this is a huge investment of human resources, it was clear from the survey results that it is time well spent.)

o   Opportunities for students to gain relevant experience.  One comment that was frequently and regularly voiced by graduating students was a desire to have more opportunities to gain field-related work experience during their time as an undergraduate.  The department had anticipated this need, and developed an undergraduate practicum course.  However, the frequency of comments about this need reinforced the importance of offering the practicum course on a yearly basis.  Additionally, while the practicum had initially been created and offered as a 480-alpha (variable topics) course, the decision was made to create a separate course just for the practicum (SLS 475).  Further, the Undergraduate Coordinator created a Faculty Sponsored Study Tour (FaSST) through the Study Abroad Center, providing an opportunity for additional SLS majors to gain experience through internships in Ho Chi Minh City, Vietnam.  The first FaSST Vietnam Internship was successfully run in Summer 2018.  Advisors also started more actively encouraging students to pursue local volunteer and paid work opportunities, including at public schools, Hawaii Literacy, Project SHINE (Students Helping in Naturalization of Elders), and others.

o   Identifying a potential problem area with SLS 441. SLS 441 is a required course for both the BA in SLS and the MA in SLS.  SLS advisors had regularly been hearing from undergraduate students that it was the most time-consuming, challenging course in the major.  It is also the one course that regularly has students who get grades of C- or lower, and thus have to repeat the course in order to graduate (it should be noted that it is a relatively low number of students who have to repeat, but the fact that there are regularly students who have to repeat drew it to the attention of the advisors).  A significant number of somewhat negative student comments about SLS 441 on exit interviews caught the eye of the LLL Associate Dean, SLS Department Chair, and SLS Undergraduate Coordinator, and reinforced the advisors’ anecdotal information. The BA Committee discussed these concerns and decided that they would review the course and explore possibilities for either recommending changes to the course or developing an alternative course to fulfill the same requirement for the major. This review will begin in AY 2018-19.

o   Students’ self-reported attainment of program level outcomes.  The LLL exit survey listed each of the seven program level learning outcomes and asked, “Self-assessment of learning outcomes. To what extent can you do the following?”  A 1-5 Likert scale was used, where 5 was “Very well” and 1 was “Not at all”.  Means for the seven outcomes were:

1.     4.10

2.     4.27

3.     4.24

4.     4.06

5.     4.10

6.     4.00

7.     4.24

In short, students voiced a high degree of confidence that they achieved the outcomes.

·       Student handbook.  Based on discussions between the two undergraduate advisors, based on their reflections on advising sessions (and also data in the LLL exit survey), it was clear that students would benefit from having a student handbook, in electronic format, that provided important information about the program, careers in the SLS, how to be successful and get the most out of the major (and their university experience), and other important information and reminders.  The two advisors revised the SLS BA new student orientation session to include more of this information, and in addition, the Undergraduate Coordinator worked with an alumnus to create a table of contents for the student handbook.  (In upcoming semesters, the handbook itself will be fleshed out and made available to students.)

·       Other review of the SLS BA curriculum. 

o   The SLS Department Chair, who had taught SLS 302 several years earlier, discussed how the original intent for course content was a balance of psycholinguistics and sociolinguistics, but the course in its current form was over-relying on a textbook that primarily focused on psycholinguistics, and the course seemed to be far more heavily weighted in that area.  He worked with GAs who were currently teaching the course to come up with additional course materials that provided a sociolinguistic perspective.  However, after discussion with the faculty resource person for SLS 302, it was agreed that all the psycholinguistic course materials (theories, etc.) were important for students to be informed about.  The BA Committee agreed that the curriculum should be revised so that a new course, providing sociolinguistic perspectives, would be developed and become one of the required courses for the major, and that SLS 302 would retain its primary focus on psycholinguistics.  Development of this new course would begin in AY 2018-19 and continue into AY 2019-20.

o   Additionally, the Department Chair explored with SLS faculty, the Director of the AH/LLL Advising Center, and SLS undergraduate advisors about potential areas for new lower-division SLS courses.  SLS 150 was revised so that it could get a General Education “FGC” designation.  Two additional courses were developed and approved (SLS 130: Introduction to Pidgin in Hawai`i, and SLS 218: Introduction to Second Language Learning & Technology)

 

This is previously mentioned in quite a bit of detail under Item 13.

Not applicable