Unit: Languages, Linguistics, & Literature, College of
Program: Hawai'i English Language Program
Date: Fri Oct 11, 2013 - 3:40:34 pm

1) Below are your program's student outcomes (SOs). Please add or update as needed.

Program Learning Outcomes (PLOs)

While the HELP Mission Statement and program goals list several stakeholders, the primary focus is on the second language learner, many of whom are interested in future university studies while others are interested only in personal development. For both types of students, we provide the following Program Learning Outcomes:

 1. Effective Written Communications Skills

Students demonstrate the ability to write well-organized, well-developed, and well-supported paragraphs and essays of varying lengths and rhetorical styles at each level of the program.

 2. Active Reading Skills

Students demonstrate the application of reading strategies to extract meaning and ideas from a variety of texts in different genres and at different levels of complexity as experienced at each level of the program.

 3. Active Listening Skills

Students demonstrate the ability to apply listening strategies to comprehend, interpret, and respond to discourse that is either non-participative or participative in developing stages at each level of the program.

 4. Effective Oral Communication Skills  

Students produce spoken language to express themselves and their relationship with others and the world in a variety of activities, such as conversations, discussions, presentations and debates, each of which demands increasing fluency at each level of the program.

 5. Grammar Competency 

Students demonstrate a solid foundation in the form, meaning, and use of grammatical structures in speaking, reading, and writing.

 6. Personal and Social Responsibility     

Students demonstrate personal development in becoming autonomous lifelong learners who engage in self-assessment, critical thinking, ethical reflection, cooperative and collaborative teamwork, and participation in activities with the larger community both on and off campus.

2) Your program's SOs are published as follows. Please update as needed.

Program's Website. URL: http://manoa.hawaii.edu/eslhelp/wordpress/?page_id=28#OurMission
Student Handbook. URL, if available online:
Information Sheet, Flyer, or Brochure. URL, if available online:
UHM Catalog. Page Number:
Other:
Other:

3) Provide the program's activity map or other graphic that illustrates how program activities/services align with program student outcomes. Please upload it as a PDF.

Activity Map File(s) from 2013:

4) Did your program engage in any program assessment activities between June 1, 2012 and September 30, 2013? (e.g., establishing/revising outcomes, aligning activities to outcomes, collecting evidence, interpreting evidence, using results, revising the assessment plan, creating surveys, etc.)

Yes
No (skip to question 14)

5) For the period June 1, 2012 to September 30, 2013: State the assessment question(s) and/or assessment goals. Include the student outcomes that were targeted, if applicable.

1.  Along with the revision of the Program Learning Outcomes, it was necessary to better align the Student Learning Outcomes for each proficiency level of the program. Each of the four levels  (100 –Beginning, 200 – Low Intermediate, 300 – High Intermediate, and 400 – Advanced) will attain the Program Outcomes in different degrees corresponding to the language proficiency appropriate for the level. The level outcomes developed in 2009-2010 showed a progression of skills from beginner to advanced, but they appeared to be discrete objectives more than outcomes.

Program Learning Outcomes by Level

Upon completion of the 100 LEVEL, students will be able to:

  1. Express themselves and their opinions at basic levels of both speaking and writing in everyday English.
  2. Identify the main ideas from simple readings and listening materials.
  3. Produce basic English grammar structures accurately in speech and writing.

Upon completion of the 200 LEVEL, students will be able to:

  1. Summarize and articulate the main ideas with supporting details from text and audio passages.
  2. Apply their knowledge of general vocabulary to effectively carry out strategies for language learning, such as guessing meaning, inferencing, and negotiating meaning.
  3. Express their opinion and make an argument with supporting details (written and oral).
  4. Students will be able to present their ideas on various topics from their own point of view with ease, so that they are ready to begin presenting their ideas academically in the 300 level.

Upon completion of the 300 LEVEL, students will be able to:

  1. Identify main ideas and supporting ideas (details) in authentic aural materials.
  2. Express a clear opinion supported by details in a group discussion or oral presentation.
  3. Identify main ideas, major supporting ideas and text organization in various types of authentic readings.
  4. Write multi-paragraph essays that are well developed and well organized.

Upon completion of the 400 LEVEL, students will be able to:

  1. Identify main ideas and key details of academic aural materials such as lectures and academic discussions.
  2. Orally express opinions and provide detailed support appropriately under time pressure.
  3. Demonstrate understanding of authentic academic materials on a variety of topics, and critically evaluate these readings.
  4. Demonstrate understanding of the style and conventions of a research paper (e.g. paragraph/essay organizations, references, and citations).

Question: How can the Student Level Outcomes be worded to align with the Program Learning Outcomes and show the essence of what students should take away from each level?

2.  The next level of student learning outcomes is for individual courses. Here, too, we found too many discrete objectives. There were more objectives listed for each skill than teachers could possibly cover in an eight-week term and students could possibly attain.

Question: Can the number of outcomes be reduced to the most essential ones and be worded without excessively discrete behaviors?

3.  Beginning Fall Term 1 2012, we decided that the rating of students as developing, competent, or exemplary should be provided to the students themselves rather than used solely for program assessment. By giving students their rating for each course outcome and the reasons for those ratings, students should have a better understanding of their proficiency than one based merely on their course grade, which is based on other factors.

Question: What is the correlation of the course grade and outcome assessments?

4.  As a result of the feedback from the Manoa Assessment Office to our 2012 Annual Assessment Report, we carefully read the university’s Institutional Learning Outcomes for undergraduates and added a sixth Program Learning Outcome on Personal and Social Responsibility to align ourselves better with the university as a whole.

Questions: Where else besides the PLOs should this outcome appear in order to be assessed?  How can values be assessed?

6) State the type(s) of evidence gathered to answer the assessment question and/or meet the assessment goals that were given in Question #5.

1.   As reported last year, a research study was conducted by two graduate students/instructors on the alignment of the 400-Level SLOs with those of the English Language Institute (ELI) into which most of our graduates enter after acceptance into undergraduate or graduate programs. The latest iteration of these SLOs resulted from that study. The SLOs at the lower levels should follow from the 400-level ones. In the Spring Term 2 2013, an attempt was made to better define the level outcomes with small groups of teachers working on specific levels. Unfortunately, teachers at the 300 and 400 levels were unable to come to a consensus on the wording of the outcomes. Determining the essence of the learning students should obtain at each level has proven to be a difficult exercise. The following drafts of revised outcomes for the two lowest levels do not show much better understanding of outcomes

100 Level

     When you complete the 100 Level (ready to enter the 200 Level), you should be able to:

  1. Express yourself and your opinion at a paragraph level in narrative and cause/effect rhetorical styles.
  2. Demonstrate the capacity to use reading strategies to comprehend adapted reading passages contained within high frequency vocabulary.
  3. Demonstrate capability to use listening strategies to comprehend spoken English in life-skill situations.
  4. Express yourself and your opinions in daily life skill situations at the sentence level.
  5. Show knowledge of the parts of speech and demonstrate proper use of this knowledge.
  6. Demonstrate receptive knowledge of 200 high-frequency words.

200 Level

When you complete the 200 Level (ready to enter the 300 Level), you should be able to:

  1. Write well-organized paragraphs on various topics with clear topic, supporting, and concluding sentences using specific examples.
  2. Identify the main ideas and key supporting details in modified reading passages.
  3. Identify the main ideas and key supporting details in modified audio materials.
  4. Express and support your opinions on various everyday topics in class discussions and short presentations.
  5. Identify the parts of speech of sentential elements and accurately use the past, present, and future progressives, the Second conditional, basic modals, and the present perfect.
  6. Self-assess your academic achievement and play an active role in cooperative and collaborative teamwork in class.

2.  At the end of each term, each teacher completes an evaluation of the course outcomes. Based on this feedback and conferences with each teacher, the Assistant Director has been revising course outcomes to reduce their number and revise the wording where needed. In addition, we realized that since we now have two-term core courses, there needs to be separate sets of outcomes for each term.

3.  Beginning Fall Term 1 2012, the teachers have been completing Final Student Assessment Reports at the end of each term for each of their students. These reports are digitally stored.

4.  Beginning Fall Term 1 2013, Service Learning became a mandatory component of the program in order to provide opportunities for students to 1) fulfill the sixth Program Learning Outcome, 2) practice English off campus, 3) meet and interact with native English speakers, and learn/observe American culture in real-life situations. Students are required to volunteer for a minimum of six hours of Service Learning each eight-week term. For each service experience, students are required to turn in a time log, which is signed by the event or agency volunteer coordinator. In addition, students must write a reflection on their experiences.

7) State how many persons submitted evidence that was evaluated. If applicable, please include the sampling technique used.

1.  For the Level Outcomes, the following teams of teachers submitted drafts:

                  100 level = 3 teachers

             200 level = 3 teachers

2.  As part of the End-of-Term reports, each of the 17 teachers in the 8-week Fall Terms I and II 2012, __ in the 3-week Spring Terms I and II 2013, __ in the 4-week Summer Terms I and II 2013, ___ in the 3-week Summer III Term 2013 completed evaluations of the course learning outcomes. The comments served to define the outcomes appropriate for each course, especially those that are two-term courses.

3.  Another end-of-term evaluation is the Student Final Assessment Report, which each teacher completes for each of their students. The number of teachers who complied with this requirement each term is the same as reported above in #2.

4.  The current Fall Term 1 2013 has 106 students, all of whom participated in a variety of service learning events or projects. We anticipate receiving completed and verified time logs and reflections from all of the students,

8) Who interpreted or analyzed the evidence that was collected? Check all that apply.

Program faculty/staff member(s)
Faculty/staff committee
Ad hoc faculty/staff group
Director or department chairperson
Persons or organization outside the university
Students (graduate or undergraduate)
Dean or Associate Dean
Advisory Board
Other:

9) How did he/she/they evaluate, analyze, or interpret the evidence? Check all that apply.

Compiled survey results
Used quantitative methods on student data (e.g., grades, participation rates) or other numeric data
Used qualitative methods on interview, focus group, or other open-ended response data
Scored exams/tests/quizzes
Used a rubric or scoring guide
Used professional judgment (no rubric or scoring guide used)
External organization/person analyzed data (e.g., Social Science Research Institute)
Other:

10) For the assessment questions/goals stated in Question #5, summarize the actual results.

  1. Further work on Student Level Outcomes will be done in Fall Term 2 2013. More than half of the current faculty have not been involved with the process of rewriting the level outcomes. With better coaching, it is hoped that fresh eyes will be able to produce better results.
     
  2. Work on the revision of the course outcomes began in Summer 2013, and the results of this project will be evaluated early next year.
     
  3. We have collected Student Assessment Reports for four full eight-week terms and three shorter summer terms.  However, we have not yet evaluated the data. We intend to do this by the end of the year.
     
  4. Student Service logs are being collected from the students. We will examine the actual number of service hours performed. We have already received positive reports from students about the program.

11) What was learned from the results?

This is yet to be determined.

12) State how the program used the results or plans to use the results. Please be specific.

All of the results of our assessment questions will aid us in strengthening our curriculum design, course offerings, formative and summative class assessments and, ultimately, our program assessment.

As for the Service Learning requirement, we are looking into implementing a virtual badge award system to further incentivize student participation. Students would receive badges of different colors corresponding to the number of hours of service as in the following example:

                  Green =  minimum 6 hours
                  Silver  =  7 to 21 hours
                  Gold   =   22+ hours 
 

13) Reflect on the assessment process. Is there anything related to assessment procedures your program would do differently next time? What went well?

Finalizing the Level Outcomes is taking much longer than anticipated due to the apparent inability of team members to meet. In hindsight, there was not enough guidance given on how to write good outcomes. It is our intention to rectify this situation with a workshop and setting a deadline for the submission of the outcomes.

What did go well is the students’ positive reaction to the Final Assessment Reports. Most students want and need detailed feedback on their learning. Students who intend to enter degree programs at community colleges or 4-year universities need to set realistic goals for their progression through the levels of our program. The assessment reports help them see their strengths and weaknesses in all language skills.

14) If the program did not engage in assessment activities, please explain.
Or, if the program did engage in assessment activities, please add any other important information here.

1.  As it is not apparent in either the PLOs or Curriculum Map, it should be noted that HELP is a non-credit, S-Funded ESL Program. Students are placed into levels according to the results of placement tests – one a standardized placement test developed at the University of Michigan and two created in house. It is rare for a student to enter at the 100 level and exit at the 400 level. As such, it is often the case that a student enters the program lacking mastery of learning outcomes at the level one below placement. For these reasons, we are unable to label any course for mastery.

2.  The addition of PLO #6 has created a bit of a problem as to its placement in the SLOs for the levels and the courses. An attempt was made during the summer to add cooperative and collaborative teamwork to lower level course SLOs, but it was viewed as a misplaced item that should be a component of the participation grade.  The description of this on a course syllabus will be the subject for faculty discussion before the next term.

The real problem with the PLO is the concept of learner autonomy, which is difficult for international students to grasp. Instructors would like to see this developed in our students, but without more research and discussion on creating a mechanism for training students, full implementation of this component of the PLO will not be possible. This could be a possible research study for DSLS graduate students.