Degree Program Assessment Reports

1. Demonstrate critical thinking and awareness of issues within the context of their professional work and social practice.

(1b. Specialized study in an academic field, 2a. Think critically and creatively)

2. Demonstrate an understanding of the value bases of their professional work.

(1b. Specialized study in an academic field, 2c. Communicate and report)

3. Understand and interpret the history of second and foreign language study and its contemporary issues.

(1b. Specialized study in an academic field)

4. Critically evaluate and make use of research into the learning, use, structure, and/or pedagogy of second languages.

(1b. Specialized study in an academic field, 2a. Think critically and creatively)

5. Develop and apply sound frameworks in the assessment and evaluation of institutions and agents involved in second language instruction, planning, and policy.

(1b. Specialized study in an academic field)

6. Show an understanding of local language issues of Hawai'i and the Pacific in their professional work.

(1b. Specialized study in an academic field, 1c. Understand Hawaiian culture and history, 3b. Respect for people and cultures, in particular Hawaiian culture)

7. Be able to support language minority students' development of academic and/or professional literacies.

(1b. Specialized study in an academic field, 1c. Understand Hawaiian culture and history)

Curriculum Map File(s) from 2020:

• File (11/21/2020)

No
Yes, on some(1-50%) of the program SLOs
Yes, on most(51-99%) of the program SLOs
Yes, on all(100%) of the program SLOs

Create/modify/discuss program learning assessment procedures (e.g., SLOs, curriculum map, mechanism to collect student work, rubric, survey)
Collect/evaluate student work/performance to determine SLO achievement
Collect/analyze student self-reports of SLO achievement via surveys, interviews, or focus groups
Use assessment results to make programmatic decisions (e.g., change course content or pedagogy, design new course, hiring)
Investigate other pressing issue related to student learning achievement for the program (explain in question 8)
Other: Investigate and improve upon curriculum coherence

Artistic exhibition/performance
Assignment/exam/paper completed as part of regular coursework and used for program-level assessment
Capstone work product (e.g., written project or non-thesis paper)
Exam created by an external organization (e.g., professional association for licensure)
Exit exam created by the program
IRB approval of research
Oral performance (oral defense, oral presentation, conference presentation)
Portfolio of student work
Publication or grant proposal
Qualifying exam or comprehensive exam for program-level assessment in addition to individual student evaluation (graduate level only)
Supervisor or employer evaluation of student performance outside the classroom (internship, clinical, practicum)
Thesis or dissertation used for program-level assessment in addition to individual student evaluation
Alumni survey that contains self-reports of SLO achievement
Employer meetings/discussions/survey/interview of student SLO achievement
Interviews or focus groups that contain self-reports of SLO achievement
Student reflective writing assignment (essay, journal entry, self-assessment) on their SLO achievement.
Student surveys that contain self-reports of SLO achievement
Assessment-related such as assessment plan, SLOs, curriculum map, etc.
Program or course materials (syllabi, assignments, requirements, etc.)
Other 1: Other data collected from College of LLL exit surveys
Other 2: Advisor reflections from student-advising sessions and meetings or correspondence with prospective students. Plus Other 3: Other evidence that arose from BA Committee meetings

Course instructor(s)
Faculty committee
Ad hoc faculty group
Department chairperson
Persons or organization outside the university
Faculty advisor
Advisors (in student support services)
Students (graduate or undergraduate)
Dean/Director
Other: Associate Dean of the College of LLL; SLS BA Committee members

• Curriculum map.  The SLS BA Committee reviewed the syllabi for each new course and filled in the curriculum map to indicate which of the program-level SLOs each course was introduced (I), reinforced and practiced (R), mastered (M), and/or assessed (A).  This also included LING 320, which was found to fulfill the requirements of the SLS 441 core course.

 

• Data from College of LLL exit survey.  Every semester, the College of Languages, Linguistics, and Literatures conducts an exit survey for each of its programs, and provides compiled results to each department.  The surveys include Likert-scale items and also student comments.  The BA Coordinator and SLS Chair reviewed the results and followed up, either individually or via the BA Committee in two main areas: student advising and opportunities for students to gain relevant experience.  Survey data also included graduating students’ self-reporting of their attainment of the program level outcomes.
• Student advising.  One of the questions in the LLL exit survey ask “Was advising in the department helpful?”  On a 1-5 Likert scale, where 5 was “Definitely Yes” and 1 was “Definitely No”, the mean was 4.88 (with 30 out of 34 students giving a score of 5, and 4 out of 34 giving a score of 4. No one gave scores of 3, 2 or 1).  Based on this information, it was clear that SLS undergraduate advisors should continue to do their advising in much the same way.  (Note that this involves spending 30-60 minutes for each student during mandatory advising sessions, taking time to get to know each student, including but not limited to their academic and career goals, and also being available outside of the mandatory advising period.  While this is a huge investment of human resources, it was clear from the survey results that it continues to be time well spent.)
• Opportunities for students to gain relevant experience.  One comment that was frequently and regularly voiced by graduating students was a desire to have more opportunities to gain field-related work experience during their time as an undergraduate.  The department had anticipated this need, and developed an undergraduate practicum course.  Unfortunately, the practicum was not offered in either Fall 2019 or Fall 2020 (due to the BA Coordinator was on professional improvement leave in Fall 2019, and due to constraints resulting from the COVID-19 pandemic in Fall 2020). Further, because of the pandemic, there were few if any opportunities to teach via the other programs that normally provide our students with those opportunities.  Members of the BA committee shared information about opportunities to gain experience teaching online, which the advisors shared with students who were interested.
•  Even though concerns about SLS 441 were addressed by the creation of SLS 301 and the addition of LING 320 as alternatives, some students still mentioned concerns about the rigor of SLS 441 (some of those who took 441 instead of one of the alternative courses). Because 441 is a required course for MA in SLS, students who were pursuing the BAM pathway, and others who planned to continue on with an MA in SLS, were advised to take 441, but to think carefully about how heavy a course load they take during the semester that they take that course.
• Students’ self-reported attainment of program level outcomes.  The LLL exit survey listed each of the seven program level learning outcomes and asked, “Self-assessment of learning outcomes. To what extent can you do the following?”  A 1-5 Likert scale was used, where 5 was “Very well” and 1 was “Not at all”.  Means for the seven outcomes were:

1.     4.55

2.     4.48

3.     4.30

4.     4.31

5.     4.24

6.     4.36

7.     4.18

In short, students voiced a high degree of confidence that they achieved the outcomes.

 

• Creation of two minors in SLS.  It was not infrequent for SLS advisors to meet or receive email from students who were interested in taking some SLS courses, but did not have time to pursue SLS as a second major. Similarly, the advisors frequently heard from peers that an SLS minor would likely be popular with students in a wide range of majors (not only students major in a language, but also including fields such as Education, Travel Industry Management, and Business). Because the SLS major was still quite new, the BA Committee felt it was important to first ensure that the major was well organized and running smoothly, before adding a minor. The committee felt that, by 2019, the time had come to revisit the idea of minors in SLS. Again, based on anecdotal information and professional judgment, the Committee developed two minors: one with a specific focus on “Second Language Teaching” and a second one with a more general focus on “Second Language Studies”, for those students who were interested in aspects of SLS other than language teaching.  The minors were proposed and have been approved, effective Fall 2020.  Given the heavy advising load for the two major advisors, it was decided that the Educational Specialist (APT) and Undergraduate Chair (faculty) would serve as advisors for students pursuing either of the two minors.

 

• Strengthening of the role of faculty resource people.  In response to a recognition that we needed better oversight of courses in the undergraduate curriculum taught by graduate assistants (PhD students in SLS), in 2018 the BA committee updated its comprehensive list of faculty members who are responsible for specific courses in their areas of expertise. In most cases, the faculty member has taught the course and developed materials for it. When a graduate student is assigned to teach an undergraduate course, they are directed to meet with the faculty resource person responsible for that course well in advance of the semester. During this meeting, they should discuss the syllabus, textbooks and other materials, and approaches to teaching. In cases where the course has General Education designation(s), the discussion should also include a check to make sure that the GA’s syllabus meets the relevant GE hallmarks. During the semester the GA is teaching the course, they should check in regularly with the faculty resource person; the faculty resource person should schedule at least one date when they can observe the GA’s teaching. When they are teaching a course that will be assigned to a GA in the future, faculty are also encouraged to offer the GA the chance to take SLS 799 (Apprenticeship in College Teaching) so the GA gets additional firsthand experience with the course.

 

• Creation of SLS 250.  Incoming freshmen usually have to wait 3-4 semesters before they are able to begin taking the upper division SLS courses that count toward the major, so SLS has created four lower division courses that provide an introduction to an interesting range of topics in the field.  However, some SLS faculty felt that there was a place to offer additional courses in SLS, many of which could be of particular interest to students (e.g., a course on language-learning strategies could help students be more successful with the HSL requirement, and enjoy their language classes more). To accomplish this, and provide flexibility for a wider range of course options, the SLS Department Chair created SLS 250 as a variable-topics course, sought feedback from the SLS BA Committee, and then submitted the course via a UHM-1 Form, which was approved effective Fall 2021.

 

• Alternative courses to fulfill the SLS 441 core requirement.  As mentioned in the 2018 Assessment Report, SLS 441 is considered by almost all undergraduate students to be the most time-consuming, challenging course in the SLS major, most likely because 441 was originally created as an MA-level course. As a result, most SLS majors postponed taking it until the end of their undergraduate studies.  However, many members of the BA Committee felt strongly that was important to be exposed to basic linguistic concepts early in the major, rather than at the end. Thus, after careful discussion, the BA Committee decided that the department needed to create a 300-level, more undergraduate-friendly version of a course that introduced these concepts. SLS 301 was developed by one of the SLS faculty, and piloted as SLS 480N (variable topics in Second Language Analysis). After successful piloting and some minor revisions, the course was proposed via a UHM-1 Form, and approved effective Fall 2020.  During the process of submitting the UHM-1 Form, we discovered that LING 320 (General Linguistics) covered similar topics and could also be used as an alternative to SLS 441.  Both courses were added to the SLS Program Sheet (effective Fall 2020) and the curriculum map.

 

• Creation of SLS 304 (The Sociolinguistics of Multilingualism).  In the 2018 Assessment Report, one of the items related to curriculum review was to review SLS 302 to ensure that there was a balanced coverage of both psycholinguistics and sociolinguistics.  After discussion with the faculty resource people for SLS 302, the BA Committee agreed that all the psycholinguistic concepts were important for students to be aware of, and that the curriculum should be revised to include a new course that covered important sociolinguistic concepts and perspectives. A course was developed by one of the SLS faculty, and is currently being piloted as SLS 480U (variable topics in Second Language Use).  A UHM-1 form has been submitted for the course as SLS 304, and we expect it to be approved effective Fall 2021.  Because of the importance of this course, the BA Committee unanimously agreed that it should be a core requirement for the major (and also one of the requirements for the general SLS minor). After careful deliberation, the committee agreed that SLS 304 would replace SLS 408 (Multilingual Education), which will be moved out of the core requirements and will serve as one of the options for SLS electives.

 

• Hire of an APT Educational Specialist to support the administration of the BA program in SLS and the English Language Institute.  When the major in SLS was first proposed, we were advised of the budgetary situation, and the importance of using existing human resources to run the program.  To accomplish this, the Department decided to shift the duties of the ELI’s Director and Coordinator of Curriculum and Teacher Development.  The ELI Director’s duties were adjusted so that 20-25% of his time was devoted to overseeing operations in the ELI, and 75-80% of his time was devoted to serving as Undergraduate Coordinator and Advisor in the BA program in SLS.  The duties of ELI Coordinator of Curriculum and Teacher Development were adjusted so that 75-80% of her time was devoted to the hands-on aspects of administering the ELI (her title changed to “Associate Director” of the ELI), and 20-25% of her time was devoted to serving as an advisor and occasional in the BA program in SLS.  Shortly thereafter, both of these people were occasionally called upon to teach courses in the BA program in SLS (primarily SLS 485, the capstone course, and SLS 475, the practicum).  When the Department of SLS underwent an external review, one of the recommendations of the reviewers was to hire an additional person to support the administrative needs of both the ELI and the BA program in SLS. A few years later, the SLS Chair, SLS Undergraduate Coordinator/ELI Director, and the ELI Associate Director/SLS Undergraduate Advisor worked with the HR Officer and Head Fiscal Officer in the College of Languages, Linguistics, & Literature to develop a position description for an Educational Specialist (APT Band A) to serve in this role.  The position was filled in November of 2019, and the Educational Specialist has taken on several projects and duties to assist the successful maintenance and growth of the BA program in SLS, including improvements to the website, PR materials, and serving as the primary advisor for students pursuing one of the newly created minors in SLS.

 

These have been described in quite a bit of detail under Item 13.

N/A