Unit: Second Language Studies
Program: Second Language Studies (MA)
Degree: Master's
Date: Thu Nov 15, 2018 - 5:00:38 pm

1) Program Student Learning Outcomes (SLOs) and Institutional Learning Objectives (ILOs)

1. Knowledge Base of Second Language Studies: Our graduates will develop familiarity with topics and concepts fundamental to the broad knowledge base of the field of Second Language Studies, including: (a) the scope of issues and methods in applied linguistics, (b) linguistic analysis, (c) second language acquisition, and (c) sociolinguistics. They will also understand how their own interests in SLS relate to the larger academic, educational, and sociopolitical contexts of the discipline.

(1. Demonstrate comprehensive knowledge in one or more general subject areas related to, but not confined to, a specific area of interest., 3. Apply research methodology and/or scholarly inquiry techniques specific to one’s field of study., 4. Critically analyze, synthesize, and utilize information and data related to one’s field of study.)

2. Utilization of Research: Our graduates will be able to access, understand, and critically evaluate the current SLS research literature and engage in systematic investigation of topics and concepts in the SLS knowledge base to inform their own and others' professional practices.

(2. Demonstrate understanding of research methodology and techniques specific to one’s field of study., 3. Apply research methodology and/or scholarly inquiry techniques specific to one’s field of study., 4. Critically analyze, synthesize, and utilize information and data related to one’s field of study., 5. Proficiently communicate and disseminate information in a manner relevant to the field and intended audience.)

3. Professionalism: Our graduates will acquire the disposition to continue professional development for the duration of their careers, seeking increased knowledge of themselves and the discipline while remaining flexible and open to change. To do so, they will acquire the skills to communicate and interact effectively with their colleagues, in order to promote effective and ethical professional environments. In addition, our graduates will be able to communicate skillfully about their SLS work, both orally (e.g., at work or professional meetings) and in writing (e.g., through in-house reports and/or articles in professional newsletters and journals).

(5. Proficiently communicate and disseminate information in a manner relevant to the field and intended audience., 6. Conduct research or projects as a responsible and ethical professional, including consideration of and respect for other cultural perspectives., 7. Interact professionally with others.)

2) Your program's SLOs are published as follows. Please update as needed.

Department Website URL: hawaii.edu/sls
Student Handbook. URL, if available online: https://www.hawaii.edu/sls/wp-content/uploads/2018/05/180522_SLS-Academic-Handbook_rvsdNov2016.pdf
Information Sheet, Flyer, or Brochure URL, if available online:
UHM Catalog. Page Number: http://www.catalog.hawaii.edu/schoolscolleges/arts-sciences/departments/sls.htm
Course Syllabi. URL, if available online:
Other:
Other:

3) Please review, add, replace, or delete the existing curriculum map.

Curriculum Map File(s) from 2018:

4) For your program, the percentage of courses that have course SLOs explicitly stated on the syllabus, a website, or other publicly available document is as follows. Please update as needed.

0%
1-50%
51-80%
81-99%
100%

5) Does the program have learning achievement results for its program SLOs? (Example of achievement results: "80% of students met expectations on SLO 1.")(check one):

No
Yes, on some(1-50%) of the program SLOs
Yes, on most(51-99%) of the program SLOs
Yes, on all(100%) of the program SLOs

6) Did your program engage in any program learning assessment activities between June 1, 2015 and October 31, 2018?

Yes
No (skip to question 17)

7) What best describes the program-level learning assessment activities that took place for the period June 1, 2015 to October 31, 2018? (Check all that apply.)

Create/modify/discuss program learning assessment procedures (e.g., SLOs, curriculum map, mechanism to collect student work, rubric, survey)
Collect/evaluate student work/performance to determine SLO achievement
Collect/analyze student self-reports of SLO achievement via surveys, interviews, or focus groups
Use assessment results to make programmatic decisions (e.g., change course content or pedagogy, design new course, hiring)
No (skip to question 17)
Investigate other pressing issue related to student learning achievement for the program (explain in question 7)
Other:

8) Briefly explain the assessment activities that took place.

The MA program naturally uses a range of assessment techniques. Major assessments related to student learning outcomes (and the educational factors that contribute to them) include:

1. Course grades, course papers and projects

2. Student presentations, through weekly seminars and occasional presentations

3.  Scholarly Paper / Thesis: For the MA program there is a scholarly work requirement which is intended to reflect each student’s ability to: (a) engage in thoroughgoing research that is relevant to the field of SLS; (b) persist in longterm scholarly projects, from inception to dissemination; and (c) produce high quality publishable writing. These assessments offer valuable insights into the extent to which students have achieved primary learning outcomes, such as familiarity with the broad content that describes particular domains of SLS, understanding and competent implementation of relevant research methods, and professional level abilities to communicate about their work. Both of the assessment formats involve multiple stages of (a) research conceptualization (formalized for the Thesis in a proposal, and its defense), (b) research, (c) writing, (d) feedback, and (d) completion of a final version (with formal public review in the case of the thesis, and internal two reader review in the case of the scholarly paper).

4. Graduating student survey: The College of LLL exit survey has subsumed our previous internal survey of graduating students. Within the redesign, questions have been generated specifically for the MA program, and these questions target both the levels of learning in key outcomes areas and the perception of professional value of these outcomes. In addition to these department-internal questions, SLS stakeholders have advised the College of LLL on the design of general questions to ask of all graduating students in the college. The Department has benefited from the recent administration of this survey again in the past year.

5. Professional activities review: We normally gather data on current and former students' publication and presentation activities over the preceding year. These professional activities provide one key indicator of the extent to which our students in the three graduate programs are developing and maintaining professional profiles that are of recognized value by the second language studies disciplines. Because of lack of staff and professorial faculty over the past two academic years, we have not analyzed these data nor been able to present them systematically.

6. Alumni survey and review. The new Departmental website has begun to show alumni places / institutions ofemployment. However systematic collection and analysis of pertinent data is beyond Departmental capacity under current funding conditions. The alumni data that is supplied by UH Foundation is not fully current or correct (despite the fact that this is a UHF responsibility) so it will take some time before this can be brought back to an adequate degree of usability.

9) What types of evidence did the program use as part of the assessment activities checked in question 7? (Check all that apply.)

Artistic exhibition/performance
Assignment/exam/paper completed as part of regular coursework and used for program-level assessment
Capstone work product (e.g., written project or non-thesis paper)
Exam created by an external organization (e.g., professional association for licensure)
Exit exam created by the program
IRB approval of research
Oral performance (oral defense, oral presentation, conference presentation)
Portfolio of student work
Publication or grant proposal
Qualifying exam or comprehensive exam for program-level assessment in addition to individual student evaluation (graduate level only)
Supervisor or employer evaluation of student performance outside the classroom (internship, clinical, practicum)
Thesis or dissertation used for program-level assessment in addition to individual student evaluation
Alumni survey that contains self-reports of SLO achievement
Employer meetings/discussions/survey/interview of student SLO achievement
Interviews or focus groups that contain self-reports of SLO achievement
Student reflective writing assignment (essay, journal entry, self-assessment) on their SLO achievement.
Student surveys that contain self-reports of SLO achievement
Assessment-related such as assessment plan, SLOs, curriculum map, etc.
Program or course materials (syllabi, assignments, requirements, etc.)
Other 1:
Other 2:

10) State the number of students (or persons) who submitted evidence that was evaluated. If applicable, please include the sampling technique used.

All 12 faculty instructors are involved in this process. Efforts are made to keep a high rate of response in regard to students involved in this process. We have 40 MA students per cohort. In particular, course level evaluation exercises draw on materials normally collected byinstructors in the course of their pedagogical activities, thus ensuring a high rate of return. Key course evaluation activities, especially end of course evaluation data is conducted in person rather than remotely— generally high rates of return are obtained. 

11) Who interpreted or analyzed the evidence that was collected? (Check all that apply.)

Course instructor(s)
Faculty committee
Ad hoc faculty group
Department chairperson
Persons or organization outside the university
Faculty advisor
Advisors (in student support services)
Students (graduate or undergraduate)
Dean/Director
Other:

12) How did they evaluate, analyze, or interpret the evidence? (Check all that apply.)

Used a rubric or scoring guide
Scored exams/tests/quizzes
Used professional judgment (no rubric or scoring guide used)
Compiled survey results
Used qualitative methods on interview, focus group, open-ended response data
External organization/person analyzed data (e.g., external organization administered and scored the nursing licensing exam)
Other:

13) Summarize the results of the assessment activities checked in question 7. For example, report the percentage of students who achieved each SLO.

The results of our assessment efforts in the MA program continue to be generally positive, indicating that the program is functioning well. Where we find areas that need improvement, we make changes to improve the MA program curriculum. Our rate of return for exit surveys is between 90-100%. We estimate that 90-100% achieve each SLO. Besides formal connections in the assessment evaluation cycle, informal lines of communication are open between assessment specialists and instructors.

14) What best describes how the program used the results? (Check all that apply.)

Assessment procedure changes (SLOs, curriculum map, rubrics, evidence collected, sampling, communications with faculty, etc.)
Course changes (course content, pedagogy, courses offered, new course, pre-requisites, requirements)
Personnel or resource allocation changes
Program policy changes (e.g., admissions requirements, student probation policies, common course evaluation form)
Students' out-of-course experience changes (advising, co-curricular experiences, program website, program handbook, brown-bag lunches, workshops)
Celebration of student success!
Results indicated no action needed because students met expectations
Use is pending (typical reasons: insufficient number of students in population, evidence not evaluated or interpreted yet, faculty discussions continue)
Other:

15) Please briefly describe how the program used the results.

Within the SLS Department, there is a considerable range of assessment-based information about MA student

learning. We provide here again a few notes on its use:

 

1. Individual Students: Students naturally, and rightly, interpret class-based and project-based assessment

(including feedback and grades) as an indication of the extent to which their work is approximating the academic

and professional norms of the SLS discipline and the high expectations of the SLS department. Via assessment,

students come to realize their accomplishments as well as gaps in their ongoing development, and they are

enabled to focus their energies on closely articulated learning targets that make sense for the individualized paths

that they take through their degree program.

 

2. Individual Faculty: Individual faculty members of SLS interpret class-based assessment data (from conventional exams, term papers, and project-based work, etc.) as an indication of the extent to which their courses are effectively fostering student learning towards specific targeted outcomes. Additionally, faculty interpret end-of-semester course evaluation data as an important indicator of the aspects of course design and delivery which are

functioning as intended and those which may be in need of adjustment. Interpretations of assessment data here

are about course and instructor contribution to learning outcomes, rather than merely about student achievement of learning outcomes.

 

3. Administrators in SLS: The Graduate Chair interprets course grades, scholarly paper/thesis, and other requirement completion data, in adjudicating final graduation decisions about individual students.

 

4. Scholarly Paper and Thesis Committees: These small committees of faculty members (usually of size 2 and 3

respectively, though occasional MA theses committees may have an additional member) utilize the major scholarly

work requirement as a means for: (a) promoting a professional-grade research, writing, feedback, and final product cycle; and (b) ensuring that SLS students graduate with professional capabilities sufficient to their individualized needs and reflective of the high standards of the department. The committees work to determine the extent to which students’ research and writing reflect and approximate to professional disciplinary standards for publishable and worthwhile scholarly research on topics of importance to second language studies.

 

5. Graduate Faculty: The graduate faculty of SLS, working in informal groups most likely centered on an individual

faculty advisor, in specific cases led by the Graduate Chair, to identify individual students who may not be meeting

expectations, with the intent of providing feedback to those students. Any apparent patterns of progress or lack of

progress towards key learning outcomes which appear to have broad rather than individual sources will naturally

lead to larger-scale program and course changes through regular curriculum review processes.

16) Beyond the results, were there additional conclusions or discoveries? This can include insights about assessment procedures, teaching and learning, and great achievements regarding program assessment in this reporting period.

The highly flexible 600 and 700 level course structure is consistently responsive to ongoing assessments of the success of the MA curriculum as it stands, and even a core introductory course such as SLS 600 has been offered in quite distinctively different manifestations in recent years. Most notably, we have assessed the MA curriculum as in need of alternative delivery modes, and have moved to offer the introductory course as well as another core course (SLS 650, Second Language Acquisition) in an online-only option (or section) as well as in a conventional face-to-face option. 

Some students have reported an interest in being able to teach K-12 in the state of Hawaii and elsewhere but find that their MA in SLS does little in that regard. We have been encouraged to communicate with the College of Education to see how we can better work together, but the numerous requirements in place for teacher licensure make it difficult to imagine producing a viable option for students in the near future.

17) If the program did not engage in assessment activities, please justify.

na