Unit: Languages, Linguistics, & Literature, College of
Program: English Language Institute
Date: Fri Oct 09, 2015 - 12:51:58 pm

1) Below are your program's student outcomes (SOs). Please add or update as needed.

Student Learning Outcomes

ELI Listening & Speaking Curriculum Area

After successful completion of ELI Listening & Speaking courses, students will be able to:

  • demonstrate effective use of strategies for: comprehending advanced academic lectures in English; critically evaluating speakers’ perspectives, techniques, and arguments; and incorporating information from academic lectures into their overall studies
  • make academic presentations (individually or in group or panel contexts) with a high degree of formal accuracy and cultural and stylistic appropriacy
  • autonomously lead academic discussions using academic English, and demonstrate effective use of advanced strategies for participation in academic discussions with expert users of English
  • self-assess their strengths in terms of listening/speaking abilities, identify areas for continued development, and state a range of strategies to address those areas

Student Learning Outcomes

ELI Reading Curriculum Area

After successful completion of ELI Reading courses, students will be able to:

  • select reading and note-taking strategies appropriately for a range of different academic reading tasks, in accordance with courses they are enrolled in as well as their own purposes for reading advanced academic English texts
  • evaluate authors’ messages, perspectives, techniques, and arguments
  • evaluate print and web-based sources
  • self-assess their strengths in terms of reading abilities, identify areas for continued development, and state a range of strategies to address those areas

Student Learning Outcomes

ELI Writing Curriculum Area – Undergraduate Level

After successful completion of ELI Writing courses, undergraduate students will be able to:

  • fluently generate sufficient written text, at the brainstorming and drafting stages of the writing process, in response to a writing assignment
  • Compose college writing that achieves a specific purpose and responds adeptly to an identifiable audience.
  • Provide evidence of effective strategies for generating, revising, editing, and proofreading a text in order to produce finished prose.
  • Compose an argument that makes use of source material that is relevant and credible and that is integrated in accordance with an appropriate style guide.

Working definitions for SLO #2 (by committee representing ELI and the English Department – the two programs offering FW):

  • Revising: making global changes (e.g., adding, deleting, or moving content; rewriting for a different audience; rewriting in a different tone)

  • Editing: making changes at the sentence level, including changes in sentence style, syntax, phrasing

  • Proofreading: correcting grammar, punctuation and mechanics, spelling, formatting

Student Learning Outcomes

ELI Writing Curriculum Area – Graduate Level

After successful completion of ELI Writing courses, graduate students will be able to:

  • fluently generate sufficient written text, at the brainstorming and drafting stages of the writing process, in response to a writing assignment
  • analyze discipline and genre-specific academic English writing conventions and effectively apply that knowledge to graduate level writing tasks
  • self-assess their strengths as academic writers, as well as areas for continued development, incorporating personal reflection and feedback from others
  • appropriately incorporate a variety of reliable information sources that are relevant for doing graduate-level research in their academic writing

 

2) Your program's SOs are published as follows. Please update as needed.

Program's Website. URL: http://www.hawaii.edu/eli/students/SLO.html
Student Handbook. URL, if available online:
Information Sheet, Flyer, or Brochure. URL, if available online:
UHM Catalog. Page Number:
Other:
Other:

3) Provide the program's activity map or other graphic that illustrates how program activities/services align with program student outcomes. Please upload it as a PDF.

Activity Map File(s) from 2015:

4) Did your program engage in any program assessment activities between June 1, 2014 and September 30, 2015? (e.g., establishing/revising outcomes, aligning activities to outcomes, collecting evidence, interpreting evidence, using results, revising the assessment plan, creating surveys, etc.)

Yes
No (skip to question 14)

5) For the period between June 1, 2014 and September 30, 2015: State the assessment question(s) and/or assessment goals. Include the student outcomes that were targeted, if applicable.

SKIP

6) State the type(s) of evidence gathered to answer the assessment question and/or meet the assessment goals that were given in Question #5.

7) State how many persons submitted evidence that was evaluated. If applicable, please include the sampling technique used.

8) Who interpreted or analyzed the evidence that was collected? Check all that apply.

Program faculty/staff member(s)
Faculty/staff committee
Ad hoc faculty/staff group
Director or department chairperson
Persons or organization outside the university
Students (graduate or undergraduate)
Dean or Associate Dean
Advisory Board
Other:

9) How did he/she/they evaluate, analyze, or interpret the evidence? Check all that apply.

Compiled survey results
Used quantitative methods on student data (e.g., grades, participation rates) or other numeric data
Used qualitative methods on interview, focus group, or other open-ended response data
Scored exams/tests/quizzes
Used a rubric or scoring guide
Used professional judgment (no rubric or scoring guide used)
External organization/person analyzed data (e.g., Social Science Research Institute)
Other:

10) For the assessment questions/goals stated in Question #5, summarize the actual results.

11) What was learned from the results?

12) State how the program used the results or plans to use the results. Please be specific.

13) Reflect on the assessment process. Is there anything related to assessment procedures your program would do differently next time? What went well?

14) If the program did not engage in assessment activities, please explain.
Or, if the program did engage in assessment activities, please add any other important information here.

The ELI and the Department of Second Language Studies (where the ELI is housed) have long been interested in program evaluation and assessment.  For years, we have encouraged faculty and graduate students in SLS to conduct research in the ELI, particularly in areas related to curriculum, student achievement, and other aspects of our program’s ongoing evolution and development.  Relevant information from these projects has been discussed by ELI administrators and in curriculum area meetings, with actions and reforms taken as appropriate.  This was possible largely because the ELI had two full-time administrators, which made it possible to handle day-to-day operations of the program as well as engage in long-term planning and improvement.

Unfortunately, our current program- and department-level resources are not adequate to support sustained comprehensive assessment of SLOs.  Specifically, in January 2011 (when the BA program in SLS was approved and moved from the Interdisciplinary Studies program to being housed within SLS), 1.00 FTE of these two administrators’ time was stripped from the ELI and reallocated to the very important jobs of coordinating the BA program and providing advising to its students.  Since that time, the ELI has had no choice but to run on “maintenance mode” (i.e., we are now able to handle only the day-to-day operations of the program). Further exacerbating matters is the popularity of the BA program in SLS, with roughly double the number of declared majors than was anticipated; this places even more demand on the limited time of the two administrators whose energies were previously devoted primarily to the ELI.

Despite these constraints, the ELI continued to promote research, preferably according to the program’s research agenda.  A few graduate-student research projects were conducted in the ELI (and other graduate students proposed projects that, although not relevant to the ELI or its mission, required solicitation of participants from ELI courses -- each of these proposals requires that the ELI Director review a project description, research instruments, consent forms, and participant-solicitation materials).  The following is a brief summary of relevant research conducted in the ELI:

  • Piloting of new diagnostic materials for use in ELI reading courses.  The ELI’s Reading Lead Teacher (a graduate assistant) developed a set of three diagnostic measures to give instructors a better view of students’ reading abilities (based on research into the constructs related to academic reading).  An oral reading test proved to be the most statistically reliable of the three measures.  As a result of this study, the Reading Lead Teacher developed a new, computerized version of the oral reading test that could be administered and recorded in a computer lab.  Further piloting of the other two measures was to be done in future semesters by continuing graduate assistants in the ELI.
  • Research into the role of combining “extensive reading” (which involves student choice of texts to read for pleasure) with written tasks in the ELI’s intermediate writing course.  The ELI’s Writing Lead Teacher conducted this project over several semesters as her dissertation research.  Findings were that the treatment group (who engaged in extensive reading with follow-up writing tasks designed to help the students connect their reading and writing) made greater gains in content, organization, vocabulary, language use, and mechanics than the control group.  As a result of the positive results from this study (as well as from non-research use of extensive reading in the reading curriculum area), the ELI’s Reading and Writing Lead Teachers were able to get funding from the Student Activity and Program Fee Board (SAPFB) to purchase more than 500 books for specially designated “ELI shelves” in Hamilton Library, to further support extensive reading for both the reading and writing curriculum areas.
  • One of the ELI’s reading instructors conducted a partial needs analysis, with the goal of enhancing instruction and activities for developing students’ critical thinking abilities. While there were some flaws with this study, it did illuminate a need to revisit the course-level SLOs in the reading curriculum area to incorporate recent advances in software that allows students to read and annotate texts digitally.  Further, this study led the instructor to propose a special section of ELI 82 that revolves around developing students’ critical thinking skills while meeting the course SLOs.
  • One full-time member of UHM’s Center for Language and Technology (CLT) conducted a study to evaluate blended language learning in the ELI.  He researched the history of online and hybrid courses in the ELI, interviewed current and past instructors of both online, hybrid and face-to-face courses, and interviewed ELI administrators. Based on his findings, the CLT developed a new workshop series called “Exploring Blended Learning”.  This workshop was originally designed to be just for ELI instructors, but due to the volume of requests, the CLT opened the workshop to instructors of other languages within the College of LLL.

In Spring 2015, Priscilla Faucette (Associate Director of the ELI) met with Monica Stitt-Bergh of the Manoa Assessment Office to discuss ways of assessing ELI SLOs at the end of each semester, bearing in mind the constraints faced by the ELI (as mentioned above).  Suggestions and comments from Stitt-Bergh included:

  • Acknowledgment that, since Faucette would be on Study Leave throughout Fall 2015 semester, any assessment activities should be postponed to Spring 2016.
  • The ELI’s curriculum map should be revised by December 2017, using the term “significant emphasis” in relevant courses rather than the “IRM” scheme, because Stitt-Bergh said “The IRM scheme doesn’t work that well with only two courses” [in a curriculum area sequence].
  • In Spring 2016, choose one curriculum area to assess, find a common “exit” task for that curriculum area, and decide which of the curriculum-area SLOs are covered by that task. Assign a person to lead the process.  Develop a rubric. Hold meetings for rubric review and revision, rater training, and rating of “exit” tasks.  Assign a person to aggregate data and writing a report of the results.
  • In Fall 2016, discuss the results with relevant stakeholders, identifying strengths and weakness of both student performance and the rubric and assessment procedures.  Create an improvement plan if needed, and decide whether or not the same SLO should be assessed again.
  • Implement the improvement plan (if needed).

As mentioned above, the ELI aims to begin implementing Stitt-Bergh’s recommendations in Spring 2016.  (ELI administrators are still concerned, however, about finding enough time to implement this process.)

 

Other Notes

Unlike curricular programs (from which students graduate with a degree or certificate), the ELI is a co-curricular, non-degree program. Therefore, there is not a fixed set of courses that must be successfully completed by all students. Students’ course requirements are determined by a battery of placement tests taken at the time they enter UH Manoa, and students’ placements range from being completely exempt (no ELI courses required) to needing six courses across all three curriculum areas, with the majority of students falling somewhere in between. Thus, it is important to note that exempting out of a curriculum area based on the placement test does not guarantee that a student has mastered all the outcomes of that area.

The ELI is one of two ESL programs that are housed within the Department of Second Language Studies.  The other is the Hawai`i English Language Program (HELP).  The ELI is for students who have matriculated at UH Manoa, whereas HELP is an intensive English program for students whose English is not yet high enough for admission to university.  Many students who complete HELP are admitted to UH Manoa and place into ELI courses.  Because of our efforts to coordinate the SLOs of our two programs, the transition from HELP to ELI is usually as seamless as possible, and we aim, when resources allow, to make assessment an important part of that process.  Thus, our assessment efforts are designed not only to serve within the ELI, but across our two programs.