Unit: Second Language Studies
Program: Second Language Studies (BA)
Degree: Bachelor's
Date: Fri Nov 16, 2018 - 4:30:56 pm

1) Program Student Learning Outcomes (SLOs) and Institutional Learning Objectives (ILOs)

1. Demonstrate critical thinking and awareness of issues within the context of their professional work and social practice.

(1b. Specialized study in an academic field, 2a. Think critically and creatively)

2. Demonstrate an understanding of the value bases of their professional work.

(1b. Specialized study in an academic field, 2c. Communicate and report)

3. Understand and interpret the history of second and foreign language study and its contemporary issues.

(1b. Specialized study in an academic field)

4. Critically evaluate and make use of research into the learning, use, structure, and/or pedagogy of second languages.

(1b. Specialized study in an academic field, 2a. Think critically and creatively)

5. Develop and apply sound frameworks in the assessment and evaluation of institutions and agents involved in second language instruction, planning, and policy.

(1b. Specialized study in an academic field)

6. Show an understanding of local language issues of Hawai'i and the Pacific in their professional work.

(1b. Specialized study in an academic field, 1c. Understand Hawaiian culture and history, 3b. Respect for people and cultures, in particular Hawaiian culture)

7. Be able to support language minority students' development of academic and/or professional literacies.

(1b. Specialized study in an academic field, 1c. Understand Hawaiian culture and history)

2) Your program's SLOs are published as follows. Please update as needed.

Department Website URL: http://www.hawaii.edu/sls/ba/?page_id=246
Student Handbook. URL, if available online:
Information Sheet, Flyer, or Brochure URL, if available online:
UHM Catalog. Page Number:
Course Syllabi. URL, if available online:
Other: SLS 485 (Professionalism in SLS) spends one class session reviewing the program level SLOs and helping students to articulate the ways in which they meet those SLOs.
Other:

3) Please review, add, replace, or delete the existing curriculum map.

Curriculum Map File(s) from 2018:

4) For your program, the percentage of courses that have course SLOs explicitly stated on the syllabus, a website, or other publicly available document is as follows. Please update as needed.

0%
1-50%
51-80%
81-99%
100%

5) Does the program have learning achievement results for its program SLOs? (Example of achievement results: "80% of students met expectations on SLO 1.")(check one):

No
Yes, on some(1-50%) of the program SLOs
Yes, on most(51-99%) of the program SLOs
Yes, on all(100%) of the program SLOs

6) Did your program engage in any program learning assessment activities between June 1, 2015 and October 31, 2018?

Yes
No (skip to question 17)

7) What best describes the program-level learning assessment activities that took place for the period June 1, 2015 to October 31, 2018? (Check all that apply.)

Create/modify/discuss program learning assessment procedures (e.g., SLOs, curriculum map, mechanism to collect student work, rubric, survey)
Collect/evaluate student work/performance to determine SLO achievement
Collect/analyze student self-reports of SLO achievement via surveys, interviews, or focus groups
Use assessment results to make programmatic decisions (e.g., change course content or pedagogy, design new course, hiring)
No (skip to question 17)
Investigate other pressing issue related to student learning achievement for the program (explain in question 7)
Other:

8) Briefly explain the assessment activities that took place.

  • Curriculum map.  The Manoa Assessment Committee provided feedback about our 2015 Assessment Report, which included a suggestion that “A good practice is to mark only the SLOs that a course covers in-depth and in which students have ample time to think, practice, and get feedback. In other words, it is usually not be possible in a single course to help students make progress or develop their knowledge and skill in all 7 areas. “
 The SLS BA Committee reviewed the curriculum map and made relevant adjustments.  Additionally, new SLS courses have been developed in the past year, and those were integrated into the curriculum map.
  • Actions from GA/Faculty/Staff workshops.  In December 2015, the BA program in SLS held a Saturday workshop to discuss curricular coherence, orientation for new teachers (especially GAs), and instructor information and resources related to teaching in the BA program.  Follow-up actions were taken by the SLS BA Committee.
  • Data from College of LLL exit survey.  Every semester, the College of Languages, Linguistics, and Literatures conducts an exit survey for each of its programs, and provides compiled results to each department.  The BA Coordinator and SLS Chair reviewed the results and followed up, either individually or via the BA Committee in three main areas: student advising, opportunities for students to gain relevant experience, and the identification of a course that was in need of committee review.  Survey data also included graduating students’ self-reporting of their attainment of the program level outcomes.
  • Student handbook.  The two advisors in the BA program reflected on advising sessions and discussed the need for having important information readily available to students, including the idea of creating a student handbook.
  • Other review of the SLS BA curriculum.  The SLS Department Chair, BA Coordinator, and later BA Committee also discussed the need to review SLS 302 to ensure it had a balance of psycholinguistic and sociolinguistic viewpoints on second language learning.  Additionally, the Department Chair explored with SLS faculty and advisors about potential areas for new lower-division SLS courses.

9) What types of evidence did the program use as part of the assessment activities checked in question 7? (Check all that apply.)

Artistic exhibition/performance
Assignment/exam/paper completed as part of regular coursework and used for program-level assessment
Capstone work product (e.g., written project or non-thesis paper)
Exam created by an external organization (e.g., professional association for licensure)
Exit exam created by the program
IRB approval of research
Oral performance (oral defense, oral presentation, conference presentation)
Portfolio of student work
Publication or grant proposal
Qualifying exam or comprehensive exam for program-level assessment in addition to individual student evaluation (graduate level only)
Supervisor or employer evaluation of student performance outside the classroom (internship, clinical, practicum)
Thesis or dissertation used for program-level assessment in addition to individual student evaluation
Alumni survey that contains self-reports of SLO achievement
Employer meetings/discussions/survey/interview of student SLO achievement
Interviews or focus groups that contain self-reports of SLO achievement
Student reflective writing assignment (essay, journal entry, self-assessment) on their SLO achievement.
Student surveys that contain self-reports of SLO achievement
Assessment-related such as assessment plan, SLOs, curriculum map, etc.
Program or course materials (syllabi, assignments, requirements, etc.)
Other 1: Information collected from faculty/GA/staff workshops.
Other 2: Other data collected from LLL exit survey. Advisor reflections from student-advising sessions. Other evidence that arose from BA Committee meetings.

10) State the number of students (or persons) who submitted evidence that was evaluated. If applicable, please include the sampling technique used.

  • Curriculum map.  Six BA committee members (comprised of two faculty, two advisors, and two PhD students who serve as GAs to teach courses in the BA program) discussed changes to existing courses and addition of new courses in the curriculum map.
  • Actions from GA/Faculty/Staff workshop.  16 faculty, GAs and staff participated in a Saturday meeting to provide history and information about the BA program in SLS and to discuss curricular coherence.  The six members of the BA committee discussed the information resulting from this workshop, and relevant changes to be made to the program.
  • College of LLL Exit Survey.  Data from the survey was compiled by the Associate Dean of LLL and her staff.  51 students responded to the surveys, out of a total of 62 students (82% response rate). 
  • Student handbook. Both undergraduate advisors provided anecdotal information based on reflections on student advising sessions.
  • Other review of the SLS BA curriculum.  Regarding SLS 302, the SLS Chair had been involved with the original of the course and teaching it, and provided evidence about the original intent for course content.

11) Who interpreted or analyzed the evidence that was collected? (Check all that apply.)

Course instructor(s)
Faculty committee
Ad hoc faculty group
Department chairperson
Persons or organization outside the university
Faculty advisor
Advisors (in student support services)
Students (graduate or undergraduate)
Dean/Director
Other: Associate Dean of the College of LLL, SLS BA Committee members.

12) How did they evaluate, analyze, or interpret the evidence? (Check all that apply.)

Used a rubric or scoring guide
Scored exams/tests/quizzes
Used professional judgment (no rubric or scoring guide used)
Compiled survey results
Used qualitative methods on interview, focus group, open-ended response data
External organization/person analyzed data (e.g., external organization administered and scored the nursing licensing exam)
Other:

13) Summarize the results of the assessment activities checked in question 7. For example, report the percentage of students who achieved each SLO.

·       Curriculum map.  The SLS BA Committee reviewed the SLO areas receiving coverage for the four courses that were identified by the Manoa Assessment Committee as being potentially problematic (SLS 312, 313, 408 and 480-alpha).  The number of SLO areas covered were reduced for SLS 312, 313, and 408.  However, because SLS 480-alpha are variable-topics courses, covering a very wide range of topics in second language pedagogy, research, use, analysis, and learning, it was agree that this course should, indeed, include coverage of all 7 SLO areas. Additionally, the committee added new SLS courses to the curriculum map.

·       Actions from GA/Faculty/Staff workshops.  The SLS BA Committee discussed the information that was garnered from the GA/faculty/staff workshops, identified three main areas for making improvements, and began the process of making those improvements.

o   A Laulima project site was created to provide downloadable instructor resources for the BA program. Multiple syllabi for all courses in the BA program were included, and some courses included complete sets of lesson plans and other materials.  This site will continue to be developed in the coming years.

o   A new GA orientation packet, in electronic format, was developed and included in the Laulima project site.  (The packet includes information that would also be useful for faculty who are teaching in the BA program for the first time.)  The packet will be piloted in Spring 2019 and Fall 2019, and revised as needed.

o   Each SLS course has been assigned a faculty resource person (with that faculty person’s agreement).  The roles and duties of faculty resource people was reviewed, and the Undergraduate Chair began working with each faculty member in regard to these roles and duties.

·       Data from College of LLL exit survey.  Every semester, the College of Languages, Linguistics, and Literatures conducts an exit survey for each of its programs, and provides compiled results to each department.  The surveys include Likert-scale items and also student comments.  The BA Coordinator and SLS Chair reviewed the results and followed up, either individually or via the BA Committee in three main areas: student advising, opportunities for students to gain relevant experience, and the identification of a course that was in need of committee review.  Survey data also included graduating students’ self-reporting of their attainment of the program level outcomes.

o   Student advising.  One of the questions in the LLL exit survey ask “Was advising in the department helpful?”  On a 1-5 Likert scale, where 5 was “Definitely Yes” and 1 was “Definitely No”, the mean was 4.9375 (with 45 out of 48 students giving a score of 5, and 3 out of 48 giving a score of 4).  Based on this information, it was clear that SLS undergraduate advisors should continue to do their advising in much the same way.  (Note that this involves spending 30-60 minutes for each student during mandatory advising sessions, taking time to get to know each student, including but not limited to their academic and career goals, and also being available outside of the mandatory advising period.  While this is a huge investment of human resources, it was clear from the survey results that it is time well spent.)

o   Opportunities for students to gain relevant experience.  One comment that was frequently and regularly voiced by graduating students was a desire to have more opportunities to gain field-related work experience during their time as an undergraduate.  The department had anticipated this need, and developed an undergraduate practicum course.  However, the frequency of comments about this need reinforced the importance of offering the practicum course on a yearly basis.  Additionally, while the practicum had initially been created and offered as a 480-alpha (variable topics) course, the decision was made to create a separate course just for the practicum (SLS 475).  Further, the Undergraduate Coordinator created a Faculty Sponsored Study Tour (FaSST) through the Study Abroad Center, providing an opportunity for additional SLS majors to gain experience through internships in Ho Chi Minh City, Vietnam.  The first FaSST Vietnam Internship was successfully run in Summer 2018.  Advisors also started more actively encouraging students to pursue local volunteer and paid work opportunities, including at public schools, Hawaii Literacy, Project SHINE (Students Helping in Naturalization of Elders), and others.

o   Identifying a potential problem area with SLS 441. SLS 441 is a required course for both the BA in SLS and the MA in SLS.  SLS advisors had regularly been hearing from undergraduate students that it was the most time-consuming, challenging course in the major.  It is also the one course that regularly has students who get grades of C- or lower, and thus have to repeat the course in order to graduate (it should be noted that it is a relatively low number of students who have to repeat, but the fact that there are regularly students who have to repeat drew it to the attention of the advisors).  A significant number of somewhat negative student comments about SLS 441 on exit interviews caught the eye of the LLL Associate Dean, SLS Department Chair, and SLS Undergraduate Coordinator, and reinforced the advisors’ anecdotal information. The BA Committee discussed these concerns and decided that they would review the course and explore possibilities for either recommending changes to the course or developing an alternative course to fulfill the same requirement for the major. This review will begin in AY 2018-19.

o   Students’ self-reported attainment of program level outcomes.  The LLL exit survey listed each of the seven program level learning outcomes and asked, “Self-assessment of learning outcomes. To what extent can you do the following?”  A 1-5 Likert scale was used, where 5 was “Very well” and 1 was “Not at all”.  Means for the seven outcomes were:

1.     4.10

2.     4.27

3.     4.24

4.     4.06

5.     4.10

6.     4.00

7.     4.24

In short, students voiced a high degree of confidence that they achieved the outcomes.

·       Student handbook.  Based on discussions between the two undergraduate advisors, based on their reflections on advising sessions (and also data in the LLL exit survey), it was clear that students would benefit from having a student handbook, in electronic format, that provided important information about the program, careers in the SLS, how to be successful and get the most out of the major (and their university experience), and other important information and reminders.  The two advisors revised the SLS BA new student orientation session to include more of this information, and in addition, the Undergraduate Coordinator worked with an alumnus to create a table of contents for the student handbook.  (In upcoming semesters, the handbook itself will be fleshed out and made available to students.)

·       Other review of the SLS BA curriculum. 

o   The SLS Department Chair, who had taught SLS 302 several years earlier, discussed how the original intent for course content was a balance of psycholinguistics and sociolinguistics, but the course in its current form was over-relying on a textbook that primarily focused on psycholinguistics, and the course seemed to be far more heavily weighted in that area.  He worked with GAs who were currently teaching the course to come up with additional course materials that provided a sociolinguistic perspective.  However, after discussion with the faculty resource person for SLS 302, it was agreed that all the psycholinguistic course materials (theories, etc.) were important for students to be informed about.  The BA Committee agreed that the curriculum should be revised so that a new course, providing sociolinguistic perspectives, would be developed and become one of the required courses for the major, and that SLS 302 would retain its primary focus on psycholinguistics.  Development of this new course would begin in AY 2018-19 and continue into AY 2019-20.

o   Additionally, the Department Chair explored with SLS faculty, the Director of the AH/LLL Advising Center, and SLS undergraduate advisors about potential areas for new lower-division SLS courses.  SLS 150 was revised so that it could get a General Education “FGC” designation.  Two additional courses were developed and approved (SLS 130: Introduction to Pidgin in Hawai`i, and SLS 218: Introduction to Second Language Learning & Technology)

 

14) What best describes how the program used the results? (Check all that apply.)

Assessment procedure changes (SLOs, curriculum map, rubrics, evidence collected, sampling, communications with faculty, etc.)
Course changes (course content, pedagogy, courses offered, new course, pre-requisites, requirements)
Personnel or resource allocation changes
Program policy changes (e.g., admissions requirements, student probation policies, common course evaluation form)
Students' out-of-course experience changes (advising, co-curricular experiences, program website, program handbook, brown-bag lunches, workshops)
Celebration of student success!
Results indicated no action needed because students met expectations
Use is pending (typical reasons: insufficient number of students in population, evidence not evaluated or interpreted yet, faculty discussions continue)
Other:

15) Please briefly describe how the program used the results.

This is previously mentioned in quite a bit of detail under Item 13.

16) Beyond the results, were there additional conclusions or discoveries? This can include insights about assessment procedures, teaching and learning, and great achievements regarding program assessment in this reporting period.

 

There are a wide array of ways that program development can result from “assessment” or “evaluation” activities beyond direct assessment of student learning outcomes.

17) If the program did not engage in assessment activities, please justify.

Not applicable