Program: Second Language Studies (PhD)
Degree: Doctorate
Date: Thu Sep 23, 2010 - 10:32:37 am
1) Below are the program student learning outcomes submitted last year. Please add/delete/modify as needed.
SLS knowledge base - SLS PHD graduates will develop advanced understanding in three of the four broad areas of SLS:
(a) Pedagogy
(b) Use
(c) Learning
(d) Analysis
Mastery and application of appropriate research methods in SLS
SLS PHD graduates will develop mastery of research methods appropriate to their specific area of expertise in SLS.
SLS PHD graduates will demonstrate a commitment to professional engagement and will be recognized for excellence in their selected areas of SLS:
(a) Presenting at conferences
(b) Publishing research
(c) Teaching in area of expertise
(d) Assuming a leadership role in an area of expertise
SLS PHD graduates should be prepared for a variety of academic and professional career paths, and in particular for tenure-track college positions.
2) As of last year, your program's SLOs were published as follows. Please update as needed.
Department Website URL: http://www.hawaii.edu/sls/?link=view_handbook§ion=11 Student Handbook. URL, if available online: http://www.hawaii.edu/sls/?link=view_handbook§ion=11 Information Sheet, Flyer, or Brochure URL, if available online: http://www.hawaii.edu/sls/?link=view_handbook§ion=11
UHM Catalog. Page Number: Course Syllabi. URL, if available online: Other: Other: 3) Below is the link to your program's curriculum map (if submitted in 2009). If it has changed or if we do not have your program's curriculum map, please upload it as a PDF.
- File (03/16/2020)
4) The percentage of courses in 2009 that had course SLOs explicitly stated on the syllabus, a website, or other publicly available document is indicated below. Please update as needed.
0% 1-50% 51-80%
81-99% 100%5) State the assessment question(s) and/or goals of the assessment activity. Include the SLOs that were targeted, if applicable.
As usual (see previous reports), we wanted to monitor our PhD students progress and satisfaction with the program as well as the accomplishments of our students, faculty, and alumni.
6) State the type(s) of evidence gathered.
As usual (see previous reports), we monitored students' progress while in the program using course evaluation procedures based on quizzes, examinations, class presentations, and final projects. We also continued to monitor the activities of our faculty, students, and the alumni of our program through their conributions to the Department of SLS Newsletter.
More specifically, at the PhD program level, a diversity of assessments were used by diverse users for diverse purposes. For example, major assessments related to student learning outcomes (and the educational factors that contribute to them) include:
Dissertation : For the PhD program there is a scholarly work requirement which is intended to reflect each individual student’s ability to: (a) engage in thorough-going research that is relevant to the field of SLS; (b) persist in long-term scholarly projects, from inception to dissemination; and (c) produce high-quality publishable writing. These assessments offer valuable insights into the extent to which students have achieved primary learning outcomes, such as familiarity with the broad content that describes particular domains of SLS, skill with research methods, and professional-level abilities to communicate about their work. Each of these assessments involves multiple stages of proposal, research, writing, feedback, and completion. In addition, the Dissertation assessments also includes multiple public presentation and defense components, including a formal comprehensive exam and proposal process.
2. Graduating students survey: Historically, the Graduate Chair has regularly distributed a graduating students survey to ascertain students’ end-of-study perspectives on their own learning outcomes (both achievement and perceived value). This survey is currently merged with a larger initiative in the College of LLL to encourage the collection of student data. Within the redesign, questions have been generated specifically for the PhD program, and these questions target both the levels of learning in key outcomes areas and the perception of professional value of these outcomes. In addition to these department-internal questions, SLS stakeholders have advised the College of LLL on the design of general questions to ask of all graduating students in the college. Web-based data collection using this survey began in Spring 2009 and was ongoing through Fall 2009 and all of 2010. Our efforts to foster increasingly high response rates appear to have failed so far, but we intend to keep working on this issue. In sum, we actively participated in the LLL evaluation questionnaire project because it was designed to evaluate LLL programs from the point of view of our graduates. We discussed the results at a graduate faculty meeting in terms of what they mean to our department and what changes we should consider making. We also presented the results to our graduate students in a brown bag session in September 2010 and solicited further feedback from our active graduate students.
3. Professional activities review: On an annual basis, current and former students report on their publication and presentation activities over the preceding year. These activities are posted in the annual SLS Letter, which is published to the Departmental web site and disseminated to faculty, students, alumni, and other interested parties (e.g., the dean of the College of LLL). These professional activities provide one key indicator of the extent to which our students in the three graduate programs are developing and maintaining professional profiles that are of recognized value by the second language studies disciplines. Publication and presentation venue are periodically reviewed for quality, and suggestions are made for ways of enhancing/encouraging improvements.
4. Alumni survey and review: The SLS Department maintains contact with graduates in order to monitor post-graduate activities. Due to rising postal costs and to enhance efficiency, the alumni files are currently in the process of being converted to electronic records. Graduates are encouraged to announce their professional activities to be posted in the annual SLS Letter, available on the Department’s website http://www.hawaii.edu/sls/?link=view_sls_letter&id=11
5. Annual doctoral student progress report: This year, we instituted a new process that requires all doctoral students in our program who are not ABD to report on and summarize their progress in the doctoral program. Those were due in August 2010. The graduate facutly plan to meet to reveiw those reports in the coming months.
7) Who interpreted or analyzed the evidence that was collected?
Course instructor(s) Faculty committee Ad hoc faculty group Department chairperson Persons or organization outside the university Faculty advisor Advisors (in student support services) Students (graduate or undergraduate) Dean/Director Other: 8) How did they evaluate, analyze, or interpret the evidence?
Used a rubric or scoring guide Scored exams/tests/quizzes Used professional judgment (no rubric or scoring guide used) Compiled survey results Used qualitative methods on interview, focus group, open-ended response data External organization/person analyzed data (e.g., external organization administered and scored the nursing licensing exam) Other: 9) State how many persons submitted evidence that was evaluated.
If applicable, please include the sampling technique used.
We have no idea what this question means.
If we must give an answer let it be: Many hundred persons were involved in dozens of processes.
10) Summarize the actual results.
By and large, the results of our assessment efforts in the PhD program are positive, indicating that the program is functioning well. Where we have found areas that need improvement, our departments' history indicates that we will take the issues seriously and make changes to improve our PhD program curriculum.
11) How did your program use the results? --or-- Explain planned use of results.
Please be specific.
Within the SLS Department, diverse users interpret diverse assessment-based information about student learning for diverse purposes. Following is a brief summary of the primary intended users of assessments:
1. Individual Students: Students interpret class-based and project-based assessment (including feedback and grades) as an indication of the extent to which their work is approximating the academic and professional norms of the SLS discipline and the high expectations of the SLS department. Via assessment, students come to realize their accomplishments as well as gaps in their on-going development, and they are enabled to focus their energies on closely articulated learning targets that make sense for the individualized paths that they take through the three graduate programs in SLS.
2. Individual Faculty: Individual faculty members of SLS interpret class-based and project-based assessment data as an indication of the extent to which their courses are effectively fostering student learning towards specific targeted outcomes. Additionally, faculty interpret mid-semester and end-of-semester course evaluation data as an important indicator of the aspects of course design and delivery which are functioning as intended and those which may be in need of adjustment. Interpretations of assessment data here are about course and instructor contribution to learning outcomes, rather than merely about student achievement of learning outcomes.
3. Staff and Administration of SLS: The Department Chair, the Graduate Chair, and the Graduate Advisor interpret course grades, scholarly paper / thesis outcomes, and other requirement completion data for adjudicating final graduation decisions about individual students.
4. Dissertation Committees: These committees of 5 faculty utilize major scholarly work requirements as a means for: (a) promoting a professional-grade research, writing, feedback, and final product cycle; and (b) ensuring that SLS students graduate with professional capabilities sufficient to their individualized needs and reflective of the high standards of the department. The committees work in unison to interpret the extent to which students’ research and writing reflect professional disciplinary standards for publishable and worthwhile scholarly research on topics of importance to second language studies.
5. The entire Graduate Faculty: The graduate faculty of SLS meets on a regular basis (once a semester) to review overall student progress in each of the three programs. As a group, the faculty reviews the key sources of assessment data (focusing on students’ progress towards degree completion, scholarly productivity, and attitudes towards courses and requirements) with an eye towards: (a) identifying individual students who may not be meeting expectations, with the intent of providing feedback to those students; and (b) any apparent patterns of progress or lack of progress towards key learning outcomes, such that needed larger-scale program and course changes may be identified. In addition, the results of the Fall 2009 and Spring/Fall 2010 LLL Program Graduate Survey were analyzed, statistically for the quantitative questions and thematically for the qualitative questions. These results were presented and discussed at the September 2010 meeting of our Graduate Faculty and were presented and discussed at a brown bag session led by the SLS Department Chair and Graduate Chair.
6. The Departmental Assessment Committee: The three-member assessment committee reviews and interprets all forms of assessment activities for three basic purposes: (a) to make recommendations on revisions/additions to existing assessment practices, where needed for acquiring more valid and/or useful data; (b) to make recommendations regarding areas in need of attention in program/curriculum/course design to the Departmental Personnel and Policy Committee; and (c) to construct annual reports about assessment activities for audiences outside of the SLS Department (e.g., the Associate Dean and Dean of the College of LLL, the VCAA).
7. The Departmental Personnel and Policy Committee: As the primary decision-making body of the Department of SLS, the DPPC regularly receives reports on priority assessment findings and adjudicates needed responses. Generally speaking, the DPPC interprets assessment data in terms of their purview, including: (a) potential areas of expertise in need of enhancement (e.g., through the hiring of new faculty or instructors); (b) structural changes that might enhance student learning and outcomes achievement (e.g., adaptations to degree-program requirements or course offerings); and (c) recommendations for new assessment data to be collected as needed (e.g., the collection of students’ professional publication and presentation data, for tracking of scholarly productivity).
12) Beyond the results, were there additional conclusions or discoveries? This can include insights about assessment procedures, teaching and learning, program aspects and so on.
For more than twenty years, we have included assessment and evaluation procedures and processes in our regular our departmental activities. The conclusions and discoveries have been numerous and useful. Here we will supply but a few examples:
For one example, based on student interest, enrollment patterns, and data on demand in the profession, three new courses have been proposed for addition to the available SLS course offerings in the past four years, in order to enhance the particular area of Language Assessment, Measurement, and Program Evaluation. SLS 631 (Second Language Program Evaluation), SLS 674 (Second Language Survey Design), and SLS 676 (Second Language Interpretive Qualitative Inquiry) have been added to respond to large demand for applied course work in these areas.
For a second example, the academic representatives of the SLS student organization (SLSSA) took it upon themselves to solicit feedback from students regarding course offerings for the 2009-2010 academic year, such that faculty and the Department Chair will be provided with one empirical basis for deciding upon which courses to offer.
For a third example, the Graduate Faculty instituted a policy requiring all doctoral students in our program who are not ABD to report on and summarize their progress in the doctoral program. Those were due in August 2010. We are planning to meet and discuss these reports and the requirement/process in an upcoming Graduate Faculty meeting.
13) Other important information:
Diverse users make diverse decisions on the basis of multiple assessment indicators for diverse purposes, as described above. Generally speaking, there are multiple types of assessment-based decisions occurring at any given time, from individual to full-departmental levels. The principle decision-making body for curriculum and administrative matters is the Departmental Personnel and Policy Committee, who meet on a monthly basis. Voting membership on this committee includes students, staff, and faculty representation. The review of assessment data plays a regular part in Graduate Faculty and DPPC meetings.