Program: Hawai'i English Language Program
Date: Fri Oct 14, 2011 - 12:08:40 am
1) List your program's student outcomes.
HELP does not award an academic degree; instead, incoming students are tested, evaluated, and placed at one of four levels (100, 200, 300, or 400) based on their English language competency. Students then enroll at courses in their level until they are able to demonstrate that level’s required language skills (i.e. student learning outcomes) and can progress into the subsequent level. There is no prescribed length of time that students must study at each level or at HELP in toto. Students can enter the program at the beginning of any one of seven terms throughout the course of the year and stay for as many terms as they wish. Students cannot progress to the next level until they have demonstrated the required SLOs at their current level. Upon completion of the 400 (final) level at HELP, a student is equipped with the language and academic skills necessary to enter and succeed at the University of Hawai‘i (or other American college/university).The SLOs for each level in the HELP program are:
Upon completion of the 100 LEVEL, students will be able to:
[Students ready to enter the 200 level can:]
1. Express themselves and their opinions at basic levels of both speaking and writing in everyday
English.
2. Identify the main ideas from simple readings and listening materials.
3. Produce basic English grammar structures accurately in speech and writing.
Upon completion of the 200 LEVEL, students will be able to:
[Students ready to enter the 300 level can:]
1. Identify the main ideas and key supporting details from modified text and audio passages.
2. Demonstrate knowledge of effective strategies for comprehending meaning, such as predicting,
guessing
meaning, skimming, scanning, and knowledge of text organization.
3. Express and support their opinion on various topics from their own point of view with ease (written
and oral).
Upon completion of the 300 LEVEL, students will be able to:
[Students ready to enter the 400 level can:]
1. Identify main ideas and supporting ideas (details) in authentic aural materials.
2. Express a clear opinion supported by details in a group discussion or oral presentation.
3. Identify main ideas, major supporting ideas and text organization in various types of authentic
readings.
4. Write multi-paragraph essays that are well-developed and well-organized.
Upon completion of the 400 LEVEL, students will be able to:
Students ready to begin study in the ELI level can:]
1. Identify main ideas and key details of academic aural materials such as lectures and academic
discussions.
2. Orally express opinions and provide detailed support appropriately under time pressure.
3. Demonstrate understanding of and critically evaluate authentic academic reading materials on a
variety of topics.
4. Demonstrate understanding of the style and conventions of an academic research paper (e.g.
paragraph/essay organizations, references, and citations).
2) Where are your program's student outcomes published? Mark all that apply and include URLs when appropriate.
Program's Website. URL: Student Handbook. URL, if available online: Information Sheet, Flyer, or Brochure. URL, if available online: UHM Catalog. Page Number: Other: 
Other: Program [Level] SLOs were drafted (January 2010) and are still being revised/refined. Therefore, they have not yet been published en masse to students. They are shared regularly with faculty and staff as part of New Teacher Orientation and at faculty meetings. They also inform the development of all course objectives, which are published to students in required course syllabi for all classes.3) Provide the program's activity map or other graphic that illustrates how program activities/services align with program student outcomes. Please upload it as a PDF.
- File (10/13/2011)
4) For the period June 1, 2010 to September 30, 2011: State the assessment question(s) and/or assessment goals. Include the student outcomes that were targeted, if applicable.
In our last annual report, the potential timeframe for assessment at HELP was:
|
Fall 2010 |
Address problem areas in Curriculum Map through refinement of current courses, development of new courses, and development and publication of Grammar Ladder |
|
2011 |
Publish SLOs to students Draft rubrics for evaluating the SLOs at the 100-level Identify and collect evidence of SLOs from 100-level courses (HELP will use a course-embedded assessment approach) Form a faculty team with expertise at the 100-level and assess each outcome Use assessment results to make recommendations for improvement at the 100-level |
|
2012 |
REPEAT process for 200-level SLOs |
|
2013 |
REPEAT process for 300-level SLOs |
|
2014 |
REPEAT process for 400-level SLOs |
With significant changes in the staff and faculty over the summer, the assessment has been reevaluated. After discussion with the Department of SLS, the most immediate need is refining the Curriculum Map and SLOs at the 400-level as a major part of HELP’s mission is to prepare students for college studies. It is, therefore, essential that the curriculum at the highest level leads to a smooth transition into the courses in the English Language Institute at UHM, preferably at the higher of the two levels of instruction. In the Fall Term 2011, a pair of M.A. graduate students/teachers are conducting a research project to evaluate the 400-level SLOs. As such, the timeline is being reversed in the order of priority to the following:
|
Fall 2011 |
Participate in research project on evaluating 400-level SLOs Rewrite all level objectives to utilize action verbs as recommended in the revised Bloom’s Taxonomy |
|
Spring 2012 |
Publish SLOs to students Draft rubrics for evaluating the SLOs at the 400-level Identify and collect evidence of SLOs from 400-level courses (HELP will use a course-embedded assessment approach) Form a faculty team with expertise at the 400-level and assess each outcome Begin work on revising course objectives to include assessment measures. Use assessment results to make recommendations for improvement at the 400-level |
|
Spring 2012 |
REPEAT process for 300-level SLOs |
|
Fall 2012 |
REPEAT process for 200-level SLOs |
|
Fall 2012 |
REPEAT process for 100-level SLOs |
5) State the type(s) of evidence gathered to answer the assessment question and/or meet the assessment goals that were given in Question 4.
The evaluation project on the 400-level SLOs will be collecting data from focus groups with 400-level students, interviews with HELP and ELI administrators, online surveys with HELP alumni, and document analysis of ELI placement tests and their results. The final evaluation report is tentatively planned for submission on December 9, 2011.
6) State how many persons submitted evidence that was evaluated. If applicable, please include the sampling technique used.
In process, so not yet available.
7) Who interpreted or analyzed the evidence that was collected. Check all that apply.
Program faculty/staff Faculty/staff committee Ad hoc faculty/staff group Director or department chairperson Persons or organization outside the university Students (graduate or undergraduate) Dean Other 8) How did he/she/they evaluate, analyze, or interpret the evidence? Check all that apply.
Compiled survey results Used qualitative methods on interview, focus group, open-ended response data Scored exams/tests/quizzes Used a rubric or scoring guide Used professional judgment (no rubric or scoring guide used) External organization/person analyzed data (e.g., Social Science Research Institute) Other: 9) For the assessment questions/goals stated in Question 4, summarize the actual results.
In process, so not yet available.
10) What was learned from the results?
In process, so not yet available.
11) State how the program used the results or plans to use the results. Please be specific.
HELP plans to use the results of the research project to better articulate the 400-level SLOs, the curriculum and its map, the objectives of each course, and to better advise students of their options for and pathways to higher education.
12) Reflect on the assessment process. Is there anything related to assessment procedures your program would do differently next time? What went well?
Not lose staff during the process. But, of course, this is out of our hands. We are doing the best given the reality of staff changes to assure that the process of continuous improvement goes forward.
13) Other important information
While we are committed to continuous improvement of our services and programs, HELP has not been able to move forward as quickly as we would like in our program-level assessment plan.
Due to the unique nature of the HELP program as a research and teaching unit of the Department of Second Language Studies, most of our faculty members are MA and PhD level graduate students. While this makes for a dynamic curriculum incorporating the latest research in second language acquisition, it also creates a challenge when seeking faculty to lead and participate in assessment projects that span several semesters. While they are teaching at HELP, graduate assistants are also taking coursework towards their own degrees and may not have the time to assume additional projects. Furthermore, there is naturally a high turnover as GAs move through the program and graduate.
Given these realities, it has been difficult for us to identify faculty who can commit to the long-term process of program evaluation. In addition, the departure in August 2011 of our designated person assigned to oversee program-wide assessment has served to interrupt some ongoing initiatives. At the same time, this change of personnel has allowed us to take a fresh look at the process, resulting in the re-evaluation of our needs as detailed in item #4.
We are already in the process of making curricular changes at all levels to begin Spring 2012, which may be helpful in better defining the curriculum map and meeting our mission goals. One of the key aspects of the new curriculum mapping exercise is that all students will be required to take a reading and writing course each term to ensure that they receive instruction in these essential skills for college success.