Unit: Second Language Studies
Program: Second Language Studies (BA)
Degree: Bachelor's
Date: Fri Oct 11, 2013 - 3:22:44 pm

1) Below are your program's student learning outcomes (SLOs). Please update as needed.

Upon graduating from the BA in SLS, students will:

  1. Manifest the skills, understandings, and dispositions necessary to be exceptional language professionals.
  2. Demonstrate critical thinking and awareness of issues within the context of their professional work and social practice.
  3. Demonstrate an understanding of the value bases of their professional work.
  4. Interpret the history of second and foreign language study and its contemporary issues.
  5. Critically evaluate and make use of research into the learning, use, structure, and pedagogy of second languages.
  6. Develop and apply sound frameworks in the assessment and evaluation of institutions and agents involved in second language instruction, planning, and policy.
  7. Show an understanding of local language issues of Hawai'i and the Pacific in their professional work.
  8. Be able to support language minority students’ development of academic and/or professional literacies.
  9. Be able to improve the quality of teaching and learning of second, foreign, and heritage languages, in the state of Hawai'i, domestically, and abroad.

Additional program goal: Upon graduating from the BA in SLS, students will be prepared to apply for admission to graduate programs in second language studies, applied linguistics, or related fields.

2) Your program's SLOs are published as follows. Please update as needed.

Department Website URL: http://web41.its.hawaii.edu/www.hawaii.edu/sls/ba/?page_id=246 (and also http://www.hawaii.edu/sls/sls/programs/bachelors/ )
Student Handbook. URL, if available online: http://www.hawaii.edu/sls/sls/resources/additional-resources/sls-student-handbook/bachelors-degrees-in-second-language-studies-sls/
Information Sheet, Flyer, or Brochure URL, if available online:
UHM Catalog. Page Number:
Course Syllabi. URL, if available online:
Other:
Other:

3) Select one option:

Curriculum Map File(s) from 2013:

4) For your program, the percentage of courses that have course SLOs explicitly stated on the syllabus, a website, or other publicly available document is as follows. Please update as needed.

0%
1-50%
51-80%
81-99%
100%

5) Did your program engage in any program assessment activities between June 1, 2012 and September 30, 2013? (e.g., establishing/revising outcomes, aligning the curriculum to outcomes, collecting evidence, interpreting evidence, using results, revising the assessment plan, creating surveys or tests, etc.)

Yes
No (skip to question 14)

6) For the period June 1, 2012 to September 30, 2013: State the assessment question(s) and/or assessment goals. Include the SLOs that were targeted, if applicable.

  1. LLL Survey of Students Graduating with a BA in SLS.   The College of Languages, Linguistics, & Literature (LLL) encourages all students in its programs to complete exit surveys when they graduate.  Among the information sought, of particular relevance to SLS is students’ self-assessment of the program-level student learning outcomes, as well as their satisfaction with the program and our courses and recommendations for the program.
  1. Requirements for 5-year Provisional Review.  Because the BA in SLS is a new major (as of Fall 2011), we are currently under “provisional” status, and will be up for review in Spring 2016.  It is important to have a clear understanding of what the requirements are for the proposal to be moved from “provisional” to “established” status, and to develop a timeline to ensure success.
  1. SLO Assessment Plan – Where to Gather Data.  When SLS originally wrote the proposal to start a major, our curriculum map showed that all SLOs would be mastered and assessed via materials included in students’ professional portfolios, developed in the SLS capstone course.  After offering the capstone course for three semesters, we realized that items that were logical requirements for inclusion in the portfolios covered many of the SLOs, but not all.  It was important to map SLOs and portfolio items, to see where the gaps existed, and to discuss how best to fill those gaps.

7) State the type(s) of evidence gathered to answer the assessment question and/or meet the assessment goals that were given in Question #6.

  1. LLL Survey of Students Graduating with a BA in SLS.   A 19-question survey about the SLS major was administered by the College of LLL.  All students graduating with a BA in SLS in Fall 2012 and Spring/Summer 2013 were highly encouraged (multiple times) to take the survey.  Questions on the survey were jointly designed by LLL and SLS (starting from LLL templates, and adding in mutually agreed upon additional items).  SLS asked to have one question asking students the degree to which they felt they fulfilled each of the SLOs. The survey includes both Likert-scale items and open ended comments and recommendations.
  1. Requirements for 5-year Provisional Review.  Information was gathered from the OVCAA at UH Manoa related to the requirements for provisional review, were divided into relevant sections, and a tentative, first-draft timeline was created to ensure completion of all required information by the time our proposal for “established” status would be due.
  1. SLO Assessment Plan – Where to Gather Data.  An assessment map was developed, with SLOs on one axis and capstone portfolio requirements on the other axis.  The SLS BA Committee mapped SLOs to portfolio items, to help with creating a plan to assess SLOs and to identify gaps where the capstone portfolio items did not provide data that could be used in assessing fulfillment of an SLO.

8) State how many persons submitted evidence that was evaluated. If applicable, please include the sampling technique used.

  1. LLL Survey of Students Graduating with a BA in SLS.  Out of 33 graduating seniors, 12 completed the survey (3 out of 10 from Fall 2012 graduates; 9 out of 23 from Spring/Summer 2013 graduates).
  1. Requirements for 5-year Provisional Review.  N/A
  1. SLO Assessment Plan – Where to Gather Data.  N/A

9) Who interpreted or analyzed the evidence that was collected? (Check all that apply.)

Course instructor(s)
Faculty committee
Ad hoc faculty group
Department chairperson
Persons or organization outside the university
Faculty advisor
Advisors (in student support services)
Students (graduate or undergraduate)
Dean/Director
Other: Most of the analysis was done by our BA committee, which is comprised of two advisors (one of whom serves as BA Coordinator), two faculty members, and two PhD students who serve as graduate-assistant instructors of SLS BA courses. Additionally, the gathering of information about 5-year provisional review and planning of a timeline was done by a PhD student, with help from the BA Coordinator.

10) How did they evaluate, analyze, or interpret the evidence? (Check all that apply.)

Used a rubric or scoring guide
Scored exams/tests/quizzes
Used professional judgment (no rubric or scoring guide used)
Compiled survey results
Used qualitative methods on interview, focus group, open-ended response data
External organization/person analyzed data (e.g., external organization administered and scored the nursing licensing exam)
Other: Discussion among the BA Committee members.

11) For the assessment question(s) and/or assessment goal(s) stated in Question #6:
Summarize the actual results.

  1. LLL Survey of Students Graduating with a BA in SLS.
  • Students’ opinions of their ability to fulfill the SLOs were high.  Using a Likert scale from 1-5, where 1=Not at all and 5=Very well, mean scores for the question, “To what extent can you do the following?” were:
    • SLO 1 = 3.92
    • SLO 2 = 4.33
    • SLO 3 = 4.08
    • SLO 4 = 4.08
    • SLO 5 = 4.17
    • SLO 6 = 3.75
    • SLO 7 = 4.08
    • SLO 8 = 3.92
    • SLO 9 = 4.17
    • Additional goal = 4.33
  • Overall satisfaction with the program.  Out of 10 questions, the vast majority of responses were “very satisfied”, followed next by “very satisfied”, with a few neutral and very few that were somewhat or very dissatisfied.  The one category that had the lowest satisfaction level was “Professional training opportunities”.
  • Satisfaction with courses in the program.  Out of 6 questions, the vast majority of responses were “very satisfied”, followed next by “very satisfied”, with a few neutral and only two who voiced views of somewhat or very dissatisfied.
  • Satisfaction with the capstone course.  Out of 5 questions, the vast majority of responses were “very satisfied”, followed next by “very satisfied”, with a few neutral and very few who voiced views of somewhat or very dissatisfied.
  • Comments and recommendations.  The vast majority of comments were very complimentary of the program, but one recommendation that recurred frequently was the need for a practicum or internship program.
  1. Requirements for 5-year Provisional Review.  Four areas for inclusion in the proposal for moving from “provisional” to “established” status were found:
  • Assessment of program-level student learning outcomes.
  • Review of program quality.
  • Review of program efficiency.
  • Review of program design and organization.

A very tentative plan was drafted, starting with Fall 2013 and continuing through the program review process (Spring 2016).

  1. SLO Assessment Plan – Where to Gather Data.  Capstone portfolio items were mapped to SLOs as follows:

Capstone Item

SLO

1

2

3

4

5

6

7

8

9

Philosophy statement

√

√

√

√

*

*

√

Professional CV

√

Materials/Lesson Plans

√

√

√

*

√

Representative Academic Paper

*

√

√

•

•

√

Cover letter

√

√

√

Tailored CV

√

√

√ = data that can clearly be used to assess the SLO;
* = occasionally students’ choice of data can be used to assess SLO

Gaps were found to exist for SLOs 6, 7 and 8.  The BA Committee discussed which courses involved assignments that could be used as data to assess those SLOs.  Potential courses were identified.  Further, it was found that, if we were to add an “annotated rationale” to the “Materials/Lesson Plans” requirement for the capstone portfolio, this item would add clear supporting data for assessing SLOs 4 and 5.

12) State how the program used the results or plans to use the results. Please be specific.

  1. LLL Survey of Students Graduating with a BA in SLS.
  • Because overall satisfaction with the program seems quite high (overall, advising, courses, and capstone course), no specific changes will be made at this time at the organizational and programmatic level.
  • Several students who made that recommendation for a “practicum” component.  SLS anticipated students’ desire to have an experiential learning component to the program (note that our 2012 assessment report includes an extended study of experiential learning options, involving practicum, internships, and service learning). Because some SLS majors want to pursue careers other than language teaching, we have decided to begin work toward developing an elective internship program. A newly filled S2 “programs specialist” position in SLS will include, among several other duties, the development of an internship program. We hope to have this person begin in Spring 2014 with developing a network of potential employers and designing options for internships with an attached “practicum-like” course.
  1. Requirements for 5-year Provisional Review.  The primary goal of presenting the requirements for our 5-year provisional review, along with a draft timeline, was to raise awareness among members of the BA committee (and potentially, other faculty whose expertise may be needed) regarding the degree of involvement needed to complete the 5-year provisional review, as well as the need for getting started.  Revising the timeline, assigning duties, and operationalizing 1-3 SLOs are steps expected in AY 2013-14.
  1. SLO Assessment Plan – Where to Gather Data.  Instructors of SLS 485 (the capstone course) agreed to add the “annotated rationale” requirement to the minimum requirements for the “Materials/Lesson Plans” portfolio item.  Further discussion and planning will be forthcoming in the next academic year regarding artifacts that can be collected to assess those SLOs where gaps were found (SLOs 6, 7 and 8).  The curriculum map was revised to include modifications based on these identified gaps.

13) Beyond the results, were there additional conclusions or discoveries?
This can include insights about assessment procedures, teaching and learning, program aspects and so on.

We would like to learn more about ways of making assessment and evaluation an important  part of our standard practices, given the constraints we face (particularly limited human resources).  In particular, given that our program faces the 5-year provisional review, and human resources are quite limited, we have honest concerns about our ability to effectively complete this review.

14) If the program did not engage in assessment activities, please explain.
Or, if the program did engage in assessment activities, please add any other important information here.

Additional activity, not mentioned above, includes:

  • SLO 8 was revised based on SLS faculty’s knowledge of language-learning opportunities and practices in K-12 and post-secondary education, and job opportunities and expectations.
  • SLO 9 was revised based on SLS faculty’s understanding of SLOs and their assessability.
  • The capstone course was given its own course number (changed from SLS 480P to SLS 485) to remove any potential confusion between the capstone course and other 480-alpha courses.  (Confusion was found to exist for students, Arts & Sciences advisors, and the Registrar’s office.)
  • The curriculum map was revised to include two recently developed introductory courses in SLS.
  • To date, all students’ portfolio materials from three semesters of capstone courses (Fall 2011, Fall 2012, and Spring 2013) have so far been collected for future assessment purposes.
  • The department has requested students’ permission to share electronic portfolios for recruiting purposes and for the program’s website (portfolios are used only from those students who signed a permission form).
  • A Laulima developmental site has been created for use by faculty, advisors, and GA instructors in the BA program in SLS.
  • Initial discussion has begun within our BA Committee about how to map UH institutional learning outcomes (ILOs) to our program-level SLOs.