Unit: Sociology
Program: Sociology (BA)
Degree: Bachelor's
Date: Wed Oct 15, 2014 - 12:11:00 pm

1) Institutional Learning Objectives (ILOs) and Program Student Learning Outcomes (SLOs)

1. Students will be able to utilize sociological theories to understand aspects of the social world.

(1a. General education, 1b. Specialized study in an academic field)

2. Students will be able to collect, analyze, and interpret data, and draw valid conclusions from their analyses.

(1b. Specialized study in an academic field, 2a. Think critically and creatively, 2b. Conduct research)

3. Students will have clear and effective verbal and written communication skills.

(2c. Communicate and report)

4.

2) Your program's SLOs are published as follows. Please update as needed.

Department Website URL:
Student Handbook. URL, if available online:
Information Sheet, Flyer, or Brochure URL, if available online:
UHM Catalog. Page Number:
Course Syllabi. URL, if available online:
Other: not published
Other:

3) Select one option:

Curriculum Map File(s) from 2014:

4) For your program, the percentage of courses that have course SLOs explicitly stated on the syllabus, a website, or other publicly available document is as follows. Please update as needed.

0%
1-50%
51-80%
81-99%
100%

5) Did your program engage in any program assessment activities between June 1, 2013 and September 30, 2014? (e.g., establishing/revising outcomes, aligning the curriculum to outcomes, collecting evidence, interpreting evidence, using results, revising the assessment plan, creating surveys or tests, etc.)

Yes
No (skip to question 14)

6) For the period between June 1, 2013 and September 30, 2014: State the assessment question(s) and/or assessment goals. Include the SLOs that were targeted, if applicable.

 

Direct evidence of student learning

The Undergraduate Studies Committee requested evaluative information from instructors of 400-level Sociology courses regarding the 3rd SLO:  Students will have clear and effective verbal and written communication skills. 

 

Indirect evidence of student learning

An Exit Survey, prepared by the advising staff, was pre-tested with the students who petitioned to graduate during Spring and Summer terms 2014.  The questionnaire contained items asking for students' evaluations relating to all of our SLOs. Those results are reported in Question 14 below.

7) State the type(s) of evidence gathered to answer the assessment question and/or meet the assessment goals that were given in Question #6.

Instructors were asked to evaluate students' performances by responding to close-ended and open-ended questions. The Chair of the Undergraduate Studies Committee contacted (by e-mail) the instructors of 10 of our 400-level Sociology courses, asking them to complete a brief questionnaire on Survey Monkey.  (The instructors of our online courses were not included.)  A link to the on-line survey was included in the e-mail. Nine of the 10 instructors responded.  Six instructors responded via Survey Monkey, and 3 sent their results directly to the Chair.   The questionnaire asked instructors to report the following information for the students in their classes.

 

1.  All questions in the following survey pertain to one 400 level course that you taught in the Spring of 2014. What was the enrollment of this one course?

 

2.  VERBAL COMMUNICATION SKILLS (Not applicable for on-line courses) In terms of the undergraduate students in your 400 level course this semester, please tell us what percent of students achieved exemplary, accomplished, average, or below acceptable levels of clear and effective verbal communication skills: [exemplary is A level, accomplished is B level, average is C level, and below acceptable is C- and below).

 

3.  In terms of clear and effective verbal communication skills, please tell us what specific challenges that you faced and/or any suggestions for student improvement that you want to share with the department this semester.

 

4.  WRITTEN COMMUNICATION SKILLS. The department currently has the following definition of clear and effective writing skills: Students’ written work is clear and the meaning of ideas is effectively conveyed. The author displays a wide vocabulary and proper choice of words and exhibits proper sentence structure, some sentence variety, and cohesion between sentences. There are few errors in grammar and the author observes conventions of standard written English. Q: Considering these dimensions of undergraduate students’ writing abilities, what percent of students achieved exemplary, accomplished, average, or below acceptable levels of clear and effective writing skills. [exemplary is A level, accomplished is B level, average is C level, and below acceptable is C- and below].

 

5.  In terms of clear and effective written communication, please tell us what specific challenges that you faced and/or any suggestions for student improvement that you want to share with the department this semester.

8) State how many persons submitted evidence that was evaluated. If applicable, please include the sampling technique used.

 

Six instructors evaluated the written and oral communication skills of students in nine 400-level Sociology courses, and reported their evaluations to the Undergraduate Studies Committee.

9) Who interpreted or analyzed the evidence that was collected? (Check all that apply.)

Course instructor(s)
Faculty committee
Ad hoc faculty group
Department chairperson
Persons or organization outside the university
Faculty advisor
Advisors (in student support services)
Students (graduate or undergraduate)
Dean/Director
Other:

10) How did they evaluate, analyze, or interpret the evidence? (Check all that apply.)

Used a rubric or scoring guide
Scored exams/tests/quizzes
Used professional judgment (no rubric or scoring guide used)
Compiled survey results
Used qualitative methods on interview, focus group, open-ended response data
External organization/person analyzed data (e.g., external organization administered and scored the nursing licensing exam)
Other:

11) For the assessment question(s) and/or assessment goal(s) stated in Question #6:
Summarize the actual results.

 

Combined, instructors submitted ratings for 195 students, but students' accomplishments could have been evaluated more than once, depending on how many 400-level sociology courses they were taking. 

 

Based on our assessment this year, we learned the following:

1.  Regarding verbal communication, we found that there was a fairly even distribution of students’ verbal clarity and effectiveness across the range of exemplary, accomplished, and acceptable categories. A clear majority (83%) of students were at or above acceptable levels of verbal skills. Specifically, instructors rated 34% of students as having verbal skills at the exemplary level, 27% at the accomplished level, 22% at the acceptable level, and 17% at the below acceptable level. 

 

Instructors' evaluations of the verbal communication skills of students in their classes were quite varied.  Instructors of two courses said that the verbal skills of all of the students in those classes were satisfactory, with 66% of the students in one class and 72% in the other class being at the exemplary level.  In contrast, one instructor said that the verbal communication abilities of 40% of the students in that class were less than acceptable, and only 10% were at the exemplary level. 

 

2.  Regarding writing clarity and effectiveness, we also found that students’ skills were fairly evenly distributed across the range of exemplary, accomplished, and acceptable.  Combining the instructors' responses, 76% of students were at or above the acceptable level in terms of writing skills. Specifically, 24% of students achieved at the exemplary level, 29% at the accomplished level, 26% at the acceptable level, and 21% were below the acceptable level.

 

Instructors' reports also varied considerably in their assessments of students' written communication skills.  The range of reported percentages of students whose work indicated a need for improvement was 0% to 40%, and the range of reported percentages of students whose written communication abilities were exemplary was 10% to 62%.

 

3.  There were 5 responses to question 3, which asked about challenges and/or suggestions for improvement with respect to verbal communication skills.  One Instructor observed, "many students have little experience speaking in public or speaking to groups" and one instructor observed that students were "more concerned with being effective and clear communicators when in front of their peers."  Two instructors commented that students were not prepared for class discussions.

 

In contrast, one instructor commented, "This class was excellent with discussions. There was a solid group of students coming to class prepared and participating with insightful comments on the readings."   That instructor stated that "small group discussions which joined into the larger class discussion at the end were most effective for getting shy students to participate regularly and build comfort with the rest of the class," adding, "The students who did poorly were frequently absent. " 

 

4.  There were 7 responses to question 5, which asked about challenges and/or suggestions for improvement with respect to written communication skills.  Two instructors indicated that students lacked preparation for the writing requirements of 400-level courses, with one saying " The 'average' students had poor research, organization, and writing skills - needs to be more solid preparation before they get to the 400 level."  Three instructors noted that editing and grammar were problems for some students, and one instructor observed that " Students really struggled with the research paper format this semester."  One instructor had the following suggestion for improvement: " I think we should direct students to the writing center and have them turn in a 'receipt' with their final drafts."

 

One instructor offered the following description of successfully encouraging students' development of their writing proficiency:  "... critical thinking skills began to emerge after [the]  first round of intensive editing ... and the peer-review. Many of these students commented on the usefulness of our feedback and about how reading each other's papers helped them see how to incorporate theory, formulate effective literature reviews, and advance their findings beyond the basic themes and into broader sociological connections. Students need to be encouraged to do less summary and more of their own thinking."  

12) State how the program used the results or plans to use the results. Please be specific.

Based on our findings, the Undergraduate Studies Committee (USC) will do the following:

·       The results of the assessment and the entire Exit Survey will be referred to the Curriculum Committee (CC) for their consideration.  The entire Exit Survey will also be distributed to the members of the Sociology Department.

·       The USC will recommend that the CC consider the selection of a single writing text, which would be included in all syllabi with writing requirements and used by students throughout their programs to help students master written communication skills.

·       The USC will recommend to the CC that SOC 321 (Survey of Sociological Theory) be added to the prerequisites for 400-level Sociology courses to help our students be better prepared for our advanced courses.

·       The USC will urge our Sociology course instructors to include more discussions and applications of research methods in their courses to help students master research methods.

·       The USC also also urge our Sociology course instructors to include more oral presentations in their courses to help our students hone their oral communication skills.

 

13) Beyond the results, were there additional conclusions or discoveries?
This can include insights about assessment procedures, teaching and learning, program aspects and so on.

The Udergraduate Studies Committee will endeavor to complete the following during the 2014-2015 academic year:

·       Less than 60% of our course syllabi contain explicit course Student Learning Objectives (SLOs).  To increase that unacceptably low percentage, the USC will distribute examples of course Student Learning Objectives, drawn from currently available Sociology syllabi, to facilitate the inclusion of course SLOs on syllabi.

·       The USC will revise the SLOs to bring them into better alignment with the ILOs that are in effect for the Manoa Campus and present them to the faculty for their review and approval.

·       The approved program SLOs will be published on the Sociology Department website. 

·       The USC will develop assessment procedures for the approved SLOs and pre-test them this academic year.

·       The USC will revise the Curriculum Map to bring it into agreement with the assessment procedures and present that to the faculty for their review and approval.

·       The USC will sponsor "Strategy Workshops" where instructors will share strategies that have effectively helped our students to hone their written and oral communication skills.

14) If the program did not engage in assessment activities, please explain.
Or, if the program did engage in assessment activities, please add any other important information here.

The program also pre-tested an undergraduate exit survey in spring 2014. Below is the executive summary.

Click here for the full report: UH Manoa Undergraduate Exit Survey Pre-Test Spring 2014

 

UH Mānoa Undergraduate Exit Survey Pre-Test Spring 2014

Executive Summary

·       Students self-reported high levels of confidence in multiple dimensions relating to conducting sociological research, understanding and incorporating sociological theory, and in communicating their findings in both written and oral formats.

·       These results indicate—from a student self-assessment perspective—that there is a good match between the expected learning outcomes articulated by the Sociology Department (see below) and the confidence that students have in their abilities as they exit the program.

 

Summary of student's responses

Sociological Theory

·       78% of students (18/23) reported that theory was included in most (5-9 courses) or all of their major courses

·       83% (19/23) reported that they were very or fairly confident in their ability to apply theory to everyday life situations (39%, 9 students, were very confident)

 

Research Methods

·       26% of students (6/23) reported that research methods were included in most (5-9 courses) or all of their major courses.

·       87% (20/23) said that they were very or fairly confident in their ability to design a research project (61%, 14 students, were very confident)

·       78% (18/23) said they were very or fairly confident in their ability to implement a research project (43%, 10 students, were very confident)

·       74% (17/23) said they were very or fairly confident in their ability to incorporate theory into their research project (52%, 12 students, were very confident)

 

Written Communication

·       100%  (23/23) of the students reported that most (5-9 courses) or all of their major courses included written assignments.

·       87% of students (20/23) reported that they were very or fairly confident in their ability to write reflection and response papers (78%, 18 students, were very confident)

·       83% of students (19/23) said they were very or fairly confident in their ability to write literature reviews and research papers (61%, 14 students, were very confident)

 

Oral Communication

·       10% of the students (2/20) reported that oral presentations were required in most (5-9 courses) or all of their major courses.

·       100% of the students (20/20) reported that they were very or fairly confident in their ability to articulate their thoughts orally in class  (65%, 13 students, were very confident)

·       100% of the students (20/20) reported that they were very or fairly confident in their ability to present research findings orally (60%, 12 students) were very confident)

·       90% (18/20) said they were very or fairly confident in their ability to make a formal presentation (55%, 11 students, were very confident)