Degree Program Assessment Reports : View Reports

Unit: Sociology
Program: Sociology (BA)
Degree: Bachelor's
Date: Tue Nov 17, 2020 - 2:45:24 pm

1) Program Student Learning Outcomes (SLOs) and Institutional Learning Objectives (ILOs)

1. Students will be able to evaluate sociological theories and use them to analyze and understand aspects of the social world.

(1a. General education, 1b. Specialized study in an academic field, 3a. Continuous learning and personal growth)

2. Students will be able to collect, analyze, and interpret data, and draw valid conclusions from their analyses.

(1b. Specialized study in an academic field, 2a. Think critically and creatively, 2b. Conduct research, 3a. Continuous learning and personal growth)

3. Students will have clear and effective verbal and written communication skills.

(2c. Communicate and report)

2) Your program's SLOs are published as follows. Please update as needed.

Department Website URL: http://www.sociology.hawaii.edu/undergraduate/index.html

Student Handbook. URL, if available online:

Information Sheet, Flyer, or Brochure URL, if available online:

UHM Catalog. Page Number:

Course Syllabi. URL, if available online:

Other:

3) Please review, add, replace, or delete the existing curriculum map.

Curriculum Map File(s) from 2020:

File (11/17/2020)

4) For your program, the percentage of courses that have course SLOs explicitly stated on the syllabus, a website, or other publicly available document is as follows. Please update as needed.

1-50%

51-80%

81-99%

100%

5) Does the program have learning achievement results for its program SLOs? (Example of achievement results: "80% of students met expectations on SLO 1.")(check one):

Yes, on some(1-50%) of the program SLOs

Yes, on most(51-99%) of the program SLOs

Yes, on all(100%) of the program SLOs

6) Did your program engage in any program learning assessment activities between November 1, 2018 and October 31, 2020?

Yes

No (skip to question 17)

7) What best describes the program-level learning assessment activities that took place for the period November 1, 2018 and October 31, 2020? (Check all that apply.)

Create/modify/discuss program learning assessment procedures (e.g., SLOs, curriculum map, mechanism to collect student work, rubric, survey)

Collect/evaluate student work/performance to determine SLO achievement

Collect/analyze student self-reports of SLO achievement via surveys, interviews, or focus groups

Use assessment results to make programmatic decisions (e.g., change course content or pedagogy, design new course, hiring)

Investigate other pressing issue related to student learning achievement for the program (explain in question 8)

Other:

8) Briefly explain the assessment activities that took place since November 2018.

Since November 2018, we have made sure that our SLOs remain aligned with the Manoa Institutional Learning Objectives.
Revising our Curriculum Map (fall of 2014). Since November 2018, we have updated our curriculum map to place our core courses on the top of the map.
Used Embedded Assessments and targeted our required theory and senior-level methods courses for evaluation of students' projects and written and oral presentations in those courses.
Relied on a basic assignment (or a signature assignment) to be used in all offerings of our theory course. The assignment was extracted from the final assignment that was used in the theory course in fall 2014.
In 2014 we developed rubrics (in consultation with the course instructors) to be used by the instructors of our targeted courses to assess students' performances on our SLOs. Since November 2018 we have continued to use the SLOs and have solicited and received feedback regarding the applicability and managability of the SLO rubrics and survey.
Used tally sheets for instructors to record their evaluations
Continued to use the Qualtrics Survey to gather the data from instructors in which they reported the results from their tally sheets, recorded the major problems students were having, and gave suggestions for programmatic changes to correct the problems.

These revisions were approved by the Sociology Faculty in November 2014 and have been in use since that time to report assessments.

The Qualtrics Exit Survey, which was developed during spring 2014, was revised to align with our revised SLOs, and students who petitioned to graduate during academic year GET DATA were asked to complete the survey. Those data were also analyzed for this report.

9) What types of evidence did the program use as part of the assessment activities checked in question 7? (Check all that apply.)

Artistic exhibition/performance

Assignment/exam/paper completed as part of regular coursework and used for program-level assessment

Capstone work product (e.g., written project or non-thesis paper)

Exam created by an external organization (e.g., professional association for licensure)

Exit exam created by the program

IRB approval of research

Oral performance (oral defense, oral presentation, conference presentation)

Portfolio of student work

Publication or grant proposal

Qualifying exam or comprehensive exam for program-level assessment in addition to individual student evaluation (graduate level only)

Supervisor or employer evaluation of student performance outside the classroom (internship, clinical, practicum)

Thesis or dissertation used for program-level assessment in addition to individual student evaluation

Alumni survey that contains self-reports of SLO achievement

Employer meetings/discussions/survey/interview of student SLO achievement

Interviews or focus groups that contain self-reports of SLO achievement

Student reflective writing assignment (essay, journal entry, self-assessment) on their SLO achievement.

Student surveys that contain self-reports of SLO achievement

Assessment-related such as assessment plan, SLOs, curriculum map, etc.

Program or course materials (syllabi, assignments, requirements, etc.)

Other 1:

Other 2:

10) State the number of students (or persons) who submitted evidence that was evaluated. If applicable, please include the sampling technique used.

Number of students who were assessed: 53

In the spring of 2019 we used the following sampling procedures:

1) we sent a link to a Qualtrics Survey to all of the instructors who teach our theory courses (Soc 321, which is required for the major) and to all of the instructors who teach our 400 level methods courses. This was a total population survey, meaning that we sent the survey to all of the instructors of our theory and methods courses to assess the work of students in their courses.

2) The instructors assessed a total of 53 embedded assignments for their students. For the theory course, 9 students' assignments were assess, remembering that our theory courses provide us inforamtion for our first SLO. The course enrolled 19 students, but because of COVID and life complications, 10 of the 19 students received an incomplete for the course and, thus, the instructor was not able to assess the signature assignment for these 10 students. For our second and third SLOs, however, we have 44 students' work that was assessed, which includes students in four of our methods courses.

11) Who interpreted or analyzed the evidence that was collected? (Check all that apply.)

Course instructor(s)

Faculty committee

Ad hoc faculty group

Department chairperson

Persons or organization outside the university

Faculty advisor

Advisors (in student support services)

Students (graduate or undergraduate)

Dean/Director

Other:

12) How did they evaluate, analyze, or interpret the evidence? (Check all that apply.)

Used a rubric or scoring guide

Scored exams/tests/quizzes

Used professional judgment (no rubric or scoring guide used)

Compiled survey results

Used qualitative methods on interview, focus group, open-ended response data

External organization/person analyzed data (e.g., external organization administered and scored the nursing licensing exam)

Other:

13) Summarize the results from the evaluation, analysis, interpretation of evidence (checked in question 12). For example, report the percentage of students who achieved each SLO.

Direct Evidence:

SLO1: Students will be able to evaluate sociological theories and use them to analyze and understand aspects of the social world. Many of the students in one theory class (10 students) were unable to finish the semester in a timely fashion due to COVID and life upheavals. N=9

Part A (evaluate theory): 100% (9 our of 9) of the students’ projects were judged to be adequate or excellent on this dimension, and 0% were judged to be inadequate.

Part B (apply theory): 100% (9 out of 9) of the students’ projects were judged to be adequate or excellent on this dimension, and 0% were judged to be inadequate.

SLO2: Students will be able to collect, analyze, and interpret data, and draw valid conclusions from their analyses. N= 44

Part A (collect, analyze, and interpret data): 98% (43 our of 44) of the students’ projects were judged to be adequate or excellent on this dimension, and 2% (1 out of 44) were judged to be inadequate.

Part B (draw valid conclusions): 98% (43 out of 44) of the students’ projects were judged to be adequate or excellent on this dimension, and 2% (1 out of 44) were judged to be inadequate.

SLO3: Students will have clear and effective verbal and written communication skills.

Written N=44; Verbal N=14

Part A (Written): 100% (44 out of 44) of the students’ projects were judged to be adequate or excellent on this dimension, and none were judged to be inadequate.

Part B (Verbal): 100% (14 out of 14) of the students’ presentations were judged to be adequate or excellent, and none were judged to be inadequate.

Students who petitioned to graduate in Fall, Spring, and Summer of the academic years 2018-2019 (n=82) and 2019-2020 (n=85) were asked to complete a Qualtrics exit survey. A total of 68 students participated in the survey, a response rate of 37%. Students rated their experiences as Sociology majors very highly. 97% of the respondents were either satisfied or very satisfied with their experiences as a Sociology major.

A high percentage of the respondents also reported that they felt fairly or highly confident in their abilities related to the multiple dimensions of the department’s Student Learning Outcomes (SLOs).

These results indicate -- from a student self-assessment perspective -- that there is a good match between the Student Learning Outcomes articulated by the Sociology Department and the confidence that students have in their abilities as they exit the program.

Indirect Assessment Evidence: Summary of Students' Responses to the Exit Surveys

SLO 1: Sociological Theory

70% of the students (47/67) reported that theory was included in most (5-9 courses) or all of their major courses
70% of the students (47/67) said they were fairly or highly confident in their ability to evaluate theories (25%, 17 students, were highly confident)
76% of the students (28/37 -- missing data for 2019-2020) reported that they were fairly or highly confident in their ability to use the theories to analyze and understand aspects of the social world (27%, 10 students, were highly confident)

SLO 2: Research Methods

12% of students (18/68) reported that research methods were included in most (5-9 courses) or all of their major courses.
84% (56/67) said that they were fairly or highly confident in their ability to collect, analyze, and interpret data (29%, 7 students, were highly confident)
81% (54/67) said they were fairly or highly confident in their ability to draw valid conclusions from their analyses (48%, 32 students, were highly confident)
75% (50/67) said they were fairly or highly confident in their ability to incorporate theory into their research projects (46%, 31 students, were highly confident)

SLO 3: Students will have clear and effective verbal and written communication skills.

Written Communication

97% of the students (11/63) reported that most (5-9 courses) or all of their major courses included written assignments.
87% of the students (58/63) reported that they were fairly or highly confident in their ability to write reflection papers (70%, 44 students, were highly confident)
89% of the students (57/64) reported that they were fairly or highly confident in their ability to write response papers (65%, 41 students, were highly confident)
83% of the students (52/63) said they were fairly or highly confident in their ability to write literature reviews and research papers (52%, 33 students, were highly confident)
89% of the students (56/63) said they were fairly or highly confident in their ability to write research papers (54%, 34 students, were highly confident)

Oral Communication

30% of the students (19/63) said that oral presentations were required in most or all of their major courses
84% of the students (52/62) were fairly or highly confident in their ability to articulate their thoughts orally in class (452%, 32 students, were highly confident)
82% of the students (51/62) were fairly or highly confident in their ability to present research findings orally (53%, 33 students, were highly confident)
81% of the students (50/62) were fairly or highly confident in their ability to make a formal oral presentation (48%, 30 students, were highly confident)

14) What best describes how the program used the results? (Check all that apply.)

Assessment procedure changes (SLOs, curriculum map, rubrics, evidence collected, sampling, communications with faculty, etc.)

Course changes (course content, pedagogy, courses offered, new course, pre-requisites, requirements)

Personnel or resource allocation changes

Program policy changes (e.g., admissions requirements, student probation policies, common course evaluation form)

Students' out-of-course experience changes (advising, co-curricular experiences, program website, program handbook, brown-bag lunches, workshops)

Celebration of student success!

Results indicated no action needed because students met expectations

Use is pending (typical reasons: insufficient number of students in population, evidence not evaluated or interpreted yet, faculty discussions continue)

Other: We will continue to evaluate the program based on our findings and discuss ways to continue the success of students.

15) Please briefly describe how the program used its findings/results.

We shared the results of the survey via email to the faculty and during a faculty meetings. The results demonstrate, first, that students had a difficult time during COVID which delayed their progress. Despite the difficulties, the students who did complete the spring 2020 semester did very well. We will discuss the results within the Undergraduate Studies Committee and then share the results with the rest of the faculty. It should be noted, however, that much of our faculty meeting time is currently being taken up with discussing the reccomendations made by the Manoa Budget Team (MBT). We have had meetings with the Dean of the College of Social Sciences and the Chancelor of UHM. The usual format for making committee reports has been suspended (temporarily) while the department responds to the proposals from the MBT and others. The fiscal challenges facing UHM have somewhat mitigated our usual ability to discuss our undergraduate program. I anticipate that we will have time in the spring of 2021 to discuss our undergraduate program's strengths and ways to build on those strengths.

16) Beyond the results, were there additional conclusions or discoveries? This can include insights about assessment procedures, teaching and learning, and great achievements regarding program assessment in this reporting period.

1) Program Student Learning Outcomes (SLOs) and Institutional Learning Objectives (ILOs)

1. Students will be able to evaluate sociological theories and use them to analyze and understand aspects of the social world.

2. Students will be able to collect, analyze, and interpret data, and draw valid conclusions from their analyses.

3. Students will have clear and effective verbal and written communication skills.

2) Your program's SLOs are published as follows. Please update as needed.

3) Please review, add, replace, or delete the existing curriculum map.

4) For your program, the percentage of courses that have course SLOs explicitly stated on the syllabus, a website, or other publicly available document is as follows. Please update as needed.

5) Does the program have learning achievement results for its program SLOs? (Example of achievement results: "80% of students met expectations on SLO 1.")(check one):

6) Did your program engage in any program learning assessment activities between November 1, 2018 and October 31, 2020?

7) What best describes the program-level learning assessment activities that took place for the period November 1, 2018 and October 31, 2020? (Check all that apply.)

8) Briefly explain the assessment activities that took place since November 2018.

9) What types of evidence did the program use as part of the assessment activities checked in question 7? (Check all that apply.)

10) State the number of students (or persons) who submitted evidence that was evaluated. If applicable, please include the sampling technique used.

11) Who interpreted or analyzed the evidence that was collected? (Check all that apply.)

12) How did they evaluate, analyze, or interpret the evidence? (Check all that apply.)

13) Summarize the results from the evaluation, analysis, interpretation of evidence (checked in question 12). For example, report the percentage of students who achieved each SLO.

14) What best describes how the program used the results? (Check all that apply.)

15) Please briefly describe how the program used its findings/results.

16) Beyond the results, were there additional conclusions or discoveries? This can include insights about assessment procedures, teaching and learning, and great achievements regarding program assessment in this reporting period.

17) If the program did not engage in assessment activities, please justify.