Program: Sociology (BA)
Degree: Bachelor's
Date: Thu Oct 11, 2012 - 2:09:43 pm
1) Below are your program's student learning outcomes (SLOs). Please update as needed.
In the 2012-2013 academic year we continued the process of revising our department SLOs that began in 2010-2011. Our goal for the year was to refine one of our proposed SLOs, receive department approval for the refined SLO, and design and test a pilot questionnaire to measure undergraduate students’ learning based on the revised SLO. We accomplished all three tasks and the new SLO is as follows:
“Students will have clear and effective verbal and written communication skills.”
2) Your program's SLOs are published as follows. Please update as needed.
Department Website URL: Student Handbook. URL, if available online: Information Sheet, Flyer, or Brochure URL, if available online: UHM Catalog. Page Number: 
Course Syllabi. URL, if available online: http://socialsciences.people.hawaii.edu/esyllabi/?subject=soc Other: Other: 3) Select one option:
- File (03/16/2020)
4) For your program, the percentage of courses that have course SLOs explicitly stated on the syllabus, a website, or other publicly available document is as follows. Please update as needed.
0% 1-50% 51-80%
81-99% 100%5) Did your program engage in any program assessment activities between June 1, 2011 and September 30, 2012? (e.g., establishing/revising outcomes, aligning the curriculum to outcomes, collecting evidence, interpreting evidence, using results, revising the assessment plan, creating surveys or tests, etc.)
Yes No (skip to question 14)6) For the period June 1, 2011 to September 30, 2012: State the assessment question(s) and/or assessment goals. Include the SLOs that were targeted, if applicable.
Our goal was to see if we could clarify and come to consensus about our written and verbal communication goals for sociology undergraduate students. We also wanted to see if we could come up with an easy to use, but accurate system to discover whether students were meeting our expectations for achieving clear and effective verbal and written communication skills.
To achieve these goals, we had five objectives during the 2011-2012 year:
1) To clarify and propose language for one of our assessment goals.
2) To obtain feedback from the department regarding the new wording and clarifications of the SLO.
3) To received departmental approval for the wording of this one SLO.
4) To design a questionnaire to assess students’ capabilities given the new SLO.
5) To pilot test the questionnaire.
7) State the type(s) of evidence gathered to answer the assessment question and/or meet the assessment goals that were given in Question #6.
We used a combined qualitative and quantitative questionnaire. We also relied on rubrics used by various department members as well as rubrics posted on the UH Assessment Office website as a base to begin our questionnaire construction.
8) State how many persons submitted evidence that was evaluated. If applicable, please include the sampling technique used.
Initially, to obtain feedback on the questionnaire as a method of assessing student learning, we sent the pilot questionnaire and proposed rubrics to all instructors and faculty in our department.
When we decided to use the questionnaire to assess student learning, we sent the questionnaire to all instructors/faculty members who had taught a 400 level undergraduate course during the spring 2012 semester. This included 13 faculty/instructors. Three e-mails were sent to request and then to remind the faculty to return their questionnaires. Eight faculty/instructors responded to the questionnaires, with five faculty/instructors failing to respond to our queries for information.
9) Who interpreted or analyzed the evidence that was collected? (Check all that apply.)
Course instructor(s) Faculty committee Ad hoc faculty group Department chairperson Persons or organization outside the university Faculty advisor Advisors (in student support services) Students (graduate or undergraduate) Dean/Director Other: 10) How did they evaluate, analyze, or interpret the evidence? (Check all that apply.)
Used a rubric or scoring guide Scored exams/tests/quizzes Used professional judgment (no rubric or scoring guide used) Compiled survey results Used qualitative methods on interview, focus group, open-ended response data External organization/person analyzed data (e.g., external organization administered and scored the nursing licensing exam) Other: 11) For the assessment question(s) and/or assessment goal(s) stated in Question #6:
Summarize the actual results.
Table 1 Evaluation on the Verbal Communication Skills of Students by 6 Instructors
|
Respondents |
Exemplary |
Accomplished |
Acceptable |
Below Acceptable/Needs Significant Improvement |
|
#1 |
0% |
25% |
50% |
25% |
|
#2 |
5% |
10% |
10% |
75% |
|
#3 |
50% |
25% |
25% |
0% |
|
#4 |
25% |
35% |
30% |
10% |
|
#5 |
30% |
40% |
10% |
20% |
|
#6 |
18% |
29% |
47% |
6% |
|
Average percentages |
21% |
27% |
29% |
23% |
Note: N=6
On the basis of the evaluation provided by the six responded instructors, the average percentages of our undergraduate students achieving the exemplary level, accomplished level, acceptable level, and below acceptable level of verbal communications skills are 21%, 27%, 29%, and 23%, respectively.
Table 2 Evaluation on the Written Communication Skills of Students by 8 Instructors
|
Respondents |
Exemplary |
Accomplished |
Acceptable |
Below Acceptable/Needs Significant Improvement |
|
#1 |
0% |
0% |
50% |
50% |
|
#2 |
10% |
10% |
40% |
40% |
|
#3 |
35% |
45% |
15% |
5% |
|
#4 |
39% |
22% |
39% |
9% |
|
#5 |
50% |
18% |
14% |
18% |
|
#6 |
23% |
30% |
25% |
22% |
|
#7 |
31% |
39% |
10% |
20% |
|
#8 |
29% |
29% |
35% |
6% |
|
Average percentages |
27% |
24% |
29% |
21% |
Note: N=8
On the basis of the evaluation provided by the eight responded instructors, the average percentages of our undergraduate students achieving the exemplary level, accomplished level, acceptable level, and below acceptable level of written communication skills are 27%, 24%, 29%, and 21%, respectively.
Oral Communication
Written comments were received from five instructors on their students’ abilities to make presentations during the Spring 2012 semester. While one instructor felt students improved in their oral communication skills over the course of the semester and one felt their students surpassed expectations, a number of instructors expressed concerns about their students’ verbal communication abilities. Two instructors felt their students evidenced insufficient knowledge of key sociological theories and theorists and that students’ materials were verbally presented at a superficial level only. One instructor commented that students appeared unfamiliar with key sociological concepts, and another commented that students had difficulty building arguments. Two instructors noted that students failed to show up for their presentations and did not attend class afterwards for the remainder of the semester.
Written Communication
Eight instructors submitted comments about their students’ written communication skills. While two instructors felt students evidenced improvement over the course of the semester, most instructors showed concern about the level of their students’ written work, and three felt that students were unable to communicate effectively in writing. Four instructors noted that students made basic structural errors in their writing, five noted an inability of students to cite materials effectively, and another three instructors noted that their students’ work showed basic grammatical errors. Two instructors expressed concern about their students’ poor reading skills and habits, two noted that students had difficulties building arguments in their written work, and a further two instructors noted that the students who needed the most help were either routinely absent or simply not turning work in. There were additional comments from three instructors about students’ unfamiliarity with key sociological concepts and theories. Suggestions for improvement include that all classes in the department have a writing component with feedback to the student on their abilities, and that instructors include in-class writing assignments that build students’ writing skills, first, in a judgment-free way and, once they offer their written arguments, the grammatical and structural components of writing can be discussed with the students.
12) State how the program used the results or plans to use the results. Please be specific.
We are preparing to distribute the findings to the department having first discussed these findings within our undergraduate studies committee (the USC). Our second step is to make some recommendations to the department based on our findings. Our third step is to incorporate department comments into our assessment process and our future plans to address our findings. We will be using our questionnaire again at the end of the semester and will continue to make modifications based on feedback from the department.
Other plans include:
1) Discover a way to publish our SLOs on our department website, or at least identify the barriers to publishing our SLOs on the department website and plan a way to overcome these barriers.
2) Continue to encourage instructors/faculty to include our SLOs in their course syllabi.
3) Plan a system of disseminating resources (e.g., rubrics, instructional resources, teaching approaches, questionnaires etc.) and for increasing communication among faculty/instructors about our collective progress regarding SLOs.
13) Beyond the results, were there additional conclusions or discoveries?
This can include insights about assessment procedures, teaching and learning, program aspects and so on.
We discovered that streamlining the SLO assessment process was much less painful and much more enjoyable and rewarding than we thought. We had developed a system a decade ago, although the insights gleaned from the process were rarely ever discussed or used to enhance student learning. This new system developed was easy to communicate to instructors/faculty as well as easy to interpret and communicate. Simplification and clarification of our assessment process, we found, helped increase USC members’ commitment to the process of assessment and gave us all important information that we can use in the future. We feel as though we accomplished quite a bit in a very short amount of time and our accomplishments have been promising, at least in terms of gaining some level of consensus among instructors/faculty and learning how well we are assisting students in meeting our learning objectives. Our major lesson learned from this process was that simplification of an assessment process can help, rather than hinder, the development of clear and effective SLOs.