Degree Program Assessment Reports : View Reports

Unit: Social Work
Program: Social Work (BSW)
Degree: Bachelor's
Date: Fri Nov 20, 2020 - 3:30:48 pm

1) Program Student Learning Outcomes (SLOs) and Institutional Learning Objectives (ILOs)

1. Demonstrate ethical and professional behavior.

(1b. Specialized study in an academic field, 2a. Think critically and creatively, 2c. Communicate and report, 3a. Continuous learning and personal growth, 3b. Respect for people and cultures, in particular Hawaiian culture, 3c. Stewardship of the natural environment)

2. Engage diversity and difference in practice.

(1b. Specialized study in an academic field, 2a. Think critically and creatively, 2b. Conduct research, 2c. Communicate and report, 3a. Continuous learning and personal growth, 3b. Respect for people and cultures, in particular Hawaiian culture, 3c. Stewardship of the natural environment, 3d. Civic participation)

3. Advance human rights and social, economic, and environmental justice.

4. Engage in practice-informed research and research-informed practice.

(1b. Specialized study in an academic field, 2a. Think critically and creatively, 2b. Conduct research, 3a. Continuous learning and personal growth, 3b. Respect for people and cultures, in particular Hawaiian culture, 3d. Civic participation)

5. Engage in policy practice.

(1b. Specialized study in an academic field, 2a. Think critically and creatively, 3a. Continuous learning and personal growth, 3b. Respect for people and cultures, in particular Hawaiian culture, 3d. Civic participation)

6. Engage with individuals, families, groups, organizations, and communities.

(1b. Specialized study in an academic field, 2a. Think critically and creatively, 2c. Communicate and report, 3b. Respect for people and cultures, in particular Hawaiian culture)

7. Assess individuals, families, groups, organizations, and communities.

(1b. Specialized study in an academic field, 2a. Think critically and creatively, 2c. Communicate and report, 3b. Respect for people and cultures, in particular Hawaiian culture)

8. Intervene with individuals, families, groups, organizations, and communities.

(1b. Specialized study in an academic field, 2a. Think critically and creatively, 2c. Communicate and report, 3b. Respect for people and cultures, in particular Hawaiian culture)

9. Evaluate practice with individuals, families, groups, organizations, and communities.

(1b. Specialized study in an academic field, 2a. Think critically and creatively, 2b. Conduct research, 2c. Communicate and report, 3b. Respect for people and cultures, in particular Hawaiian culture)

10. Engage, honor, and respect indigenous culture towards decolonized professional practice.

(1b. Specialized study in an academic field, 1c. Understand Hawaiian culture and history, 2a. Think critically and creatively, 2c. Communicate and report, 3a. Continuous learning and personal growth, 3b. Respect for people and cultures, in particular Hawaiian culture, 3c. Stewardship of the natural environment, 3d. Civic participation)

2) Your program's SLOs are published as follows. Please update as needed.

Department Website URL: http://www.hawaii.edu/sswork/bsw.html

Student Handbook. URL, if available online: https://www.hawaii.edu/sswork/wp-content/uploads/BSW-Handbook-2019-Adjusted.pdf

Information Sheet, Flyer, or Brochure URL, if available online: http://www.hawaii.edu/sswork/bulletin.html

UHM Catalog. Page Number:

Course Syllabi. URL, if available online:

Other: New Student Orientation Materials

3) Please review, add, replace, or delete the existing curriculum map.

Curriculum Map File(s) from 2020:

File (11/16/2020)

4) For your program, the percentage of courses that have course SLOs explicitly stated on the syllabus, a website, or other publicly available document is as follows. Please update as needed.

1-50%

51-80%

81-99%

100%

5) Does the program have learning achievement results for its program SLOs? (Example of achievement results: "80% of students met expectations on SLO 1.")(check one):

Yes, on some(1-50%) of the program SLOs

Yes, on most(51-99%) of the program SLOs

Yes, on all(100%) of the program SLOs

6) Did your program engage in any program learning assessment activities between November 1, 2018 and October 31, 2020?

Yes

No (skip to question 17)

7) What best describes the program-level learning assessment activities that took place for the period November 1, 2018 and October 31, 2020? (Check all that apply.)

Create/modify/discuss program learning assessment procedures (e.g., SLOs, curriculum map, mechanism to collect student work, rubric, survey)

Collect/evaluate student work/performance to determine SLO achievement

Collect/analyze student self-reports of SLO achievement via surveys, interviews, or focus groups

Use assessment results to make programmatic decisions (e.g., change course content or pedagogy, design new course, hiring)

Investigate other pressing issue related to student learning achievement for the program (explain in question 8)

Other:

8) Briefly explain the assessment activities that took place since November 2018.

Our assessment goal continues to be to measure the degree to which our ten Program Learning Outcomes (competencies) have been met throughout the curriculum over the course of the academic year. We use both direct evidence (instructor evaluation of student products such as reports, presentations, classroom activities, and skill demonstration as measured through the use of the Instructor Evaluation Sheet (IES) and the Field Supervisory Evaluation form (FSE)) and indirect evidence (student self-report as measured through the use of the Student Self-Assessment Survey Instrument (SSASI)). The Ten Program SLOs are:

1. Demonstrate ethical and professional behavior.

2. Engage diversity and difference in practice.

3. Advance human rights and social, economic, and environmental justice.

4. Engage in practice-informed research and research-informed practice.

5. Engage in policy practice.

6. Engage with individuals, families, groups, organizations, and communities.

7. Assess individuals, families, groups, organizations, and communities

8. Intervene with individuals, families, groups, organizations, and communities.

9. Evaluate practice with individuals, families, groups, organizations, and communities.

10. Engage, honor, and respect indigenous culture towards decolonized professional practice

The BSW and MSW Programs continue to hold “teachering” sessions to address curriculum questions and common classroom issues. Topics have included curriculum rollout, vertical and horizontal sequencing, assignment design and implementation as well as classroom management. Due to the COVID-19 pandemic, we have also held training sessions on utilization of Zoom, creating engaging classroom experiences while online, and sent out notifications for trainings being held in UH Manoa community as well as the community at large. Also covered are SLOs and course assignment alignment. It is critical the both instructors and adjunct faculty (lecturers) recognize the need to connect assignments to stated objectives to measure competence. Because of this, the “teachering” sessions serve as both training site and site for program culture development that is competency based.

In the spring of 2017 the department produced and continues to use a video on assessing student competency and effective competency measurement that is shown as part of the biannual faculty orientation. The aim of the video is to: 1) clarify the IES as an assessment tool, 2) decouple competency assessment from grading, and 3) address some of the common pitfalls in the assessment process. We have found over the years that faculty will sometimes assess students against the understandable yet unrealistic standard of the seasoned social worker. Against such a standard, it is nearly impossible for a student to be anything but substandard. Instead, we want faculty to assess social work competencies based on where the student is at in their academic and professional development. The question is not “How does this student stack up against a real or imagined seasoned social worker?” but rather, “How is this student performing relative to their position as a junior (or senior) level student?” This distinction is critical and while still an "issue' has improved over the course of the last several years.

In addition to assessing competency development, the program also assesses key program functions including student services and the field experience. In the past we modified student advising in response to student feedback. Students were clear that they wanted more professional advising as they prepared for life after school. In 2019 and 2020 we created and held trainings on using STAR as the final arbiter of program completion. The training focused specifically on using Academic Essentials and STAR GPS as central to the advising process.

Two substantial changes (one enacted and the other under review) initiated in the time since our last report include the discontinuation of the Logic Requirement for our Knowledge Base and the potential reduction of total field hours to earn a bachelor’s degree in social work. For the first, the program decided to discontinue the logic requirement, believing that the FQ was sufficient for BSW Students. This came after considerable discussion and a thorough review of logic course syllabi. More importantly, the assessment of the logic course was inspired by ongoing concern raised by students (while in advising sessions) to the point that the program believed it best we consider and review the matter.

The second is ongoing with a faculty/student/community member committee formed to assess the total number of hours needed for the field experience. This came after students formally submitted their own student survey to the BSW Program Committee noting that the total number of hours required by our accrediting body (CSWE) was 400 hours, while our program required 530 hours. It should be noted that while on the high end, our required field hours fall within the accepted range and many MSW Programs that accept the BSW as the preferred degree require a minimum of 500 hours to be considered for advanced standing (4+1 pathway).

In the Summer of 2018, the BSW Program at the Myron B. Thompson School of Social Work was notified by the Council on Social Work Education (CSWE) Commission on Accreditation (COA) that it was reaccredited for the maximum allowable eight (8) years without condition or contingency. Later that same year, the BSW Program petitioned to make a “substantive change” to the program by developing and implementing a Distance Education option. This “substantive change” was approved by our accrediting body.

9) What types of evidence did the program use as part of the assessment activities checked in question 7? (Check all that apply.)

Artistic exhibition/performance

Assignment/exam/paper completed as part of regular coursework and used for program-level assessment

Capstone work product (e.g., written project or non-thesis paper)

Exam created by an external organization (e.g., professional association for licensure)

Exit exam created by the program

IRB approval of research

Oral performance (oral defense, oral presentation, conference presentation)

Portfolio of student work

Publication or grant proposal

Qualifying exam or comprehensive exam for program-level assessment in addition to individual student evaluation (graduate level only)

Supervisor or employer evaluation of student performance outside the classroom (internship, clinical, practicum)

Thesis or dissertation used for program-level assessment in addition to individual student evaluation

Alumni survey that contains self-reports of SLO achievement

Employer meetings/discussions/survey/interview of student SLO achievement

Interviews or focus groups that contain self-reports of SLO achievement

Student reflective writing assignment (essay, journal entry, self-assessment) on their SLO achievement.

Student surveys that contain self-reports of SLO achievement

Assessment-related such as assessment plan, SLOs, curriculum map, etc.

Program or course materials (syllabi, assignments, requirements, etc.)

Other 1:

Other 2:

10) State the number of students (or persons) who submitted evidence that was evaluated. If applicable, please include the sampling technique used.

Students are assessed for competency acquisition in both the classroom and field (2 points) as explained elsewhere in this assessment report. Additionally, we assess across at least two of four dimensions including 1) knowledge, 2) values, 3) cognitive/affective processes and 4) skills. For our accreditation, skill assessment is mandatory and must be assessed in the field experience with real clients or through simulation (due to the pandemic real clients may be difficult so siimulation is considered acceptable until further notice). 100% of the 166 students in the program were assessed in the classroom and the field for A.Y. 2019-2020.

The total number of students assessed was: 166

11) Who interpreted or analyzed the evidence that was collected? (Check all that apply.)

Course instructor(s)

Faculty committee

Ad hoc faculty group

Department chairperson

Persons or organization outside the university

Faculty advisor

Advisors (in student support services)

Students (graduate or undergraduate)

Dean/Director

Other:

12) How did they evaluate, analyze, or interpret the evidence? (Check all that apply.)

Used a rubric or scoring guide

Scored exams/tests/quizzes

Used professional judgment (no rubric or scoring guide used)

Compiled survey results

Used qualitative methods on interview, focus group, open-ended response data

External organization/person analyzed data (e.g., external organization administered and scored the nursing licensing exam)

Other:

13) Summarize the results from the evaluation, analysis, interpretation of evidence (checked in question 12). For example, report the percentage of students who achieved each SLO.

Our assessment goal continues to measure the degree to which our ten Program Learning Outcomes (competencies) have been met throughout the curriculum over the course of the academic year. We use both direct evidence (instructor evaluation of student products such as reports, presentations, classroom activities, and skill demonstration as measured through the use of the Instructor Evaluation Sheet (IES) and the Field Supervisory Evaluation form (FSE)) and indirect evidence (student self-report as measured through the use of the Student Self-Assessment Survey Instrument (SSASI)). The Ten Program SLOs are:

1. Demonstrate ethical and professional behavior.

2. Engage diversity and difference in practice.

3. Advance human rights and social, economic, and environmental justice.

4. Engage in practice-informed research and research-informed practice.

5. Engage in policy practice.

6. Engage with individuals, families, groups, organizations, and communities.

7. Assess individuals, families, groups, organizations, and communities

8. Intervene with individuals, families, groups, organizations, and communities.

9. Evaluate practice with individuals, families, groups, organizations, and communities.

10. Engage, honor, and respect indigenous culture towards decolonized professional practice

Benchmark and Rationale

The BSW Program, working with the Department Assessment Committee, established both a 5-point Likert scale as well as an acquisition benchmark for all social work competencies. Taken in aggregate, the expectation is that 80% of all students surveyed in the BSW Program will score a 4 or higher on a 5-point Likert scale. A score of 4 on our 5-point Likert scale represents near consistency of competency demonstration. The BSW Program recognizes that students have varying skills upon entry to the Program and throughout their tenure, and will continue to learn and grow long after their formal educational experience is complete. The near consistent demonstration of social work competencies is critical if our graduates hope to be legitimate helpers in the community.

CSWE identifies several dimensions critical to each competency, they are Knowledge, Values, Skills, and Cognitive & Affective Processes. The BSW program determined that the Skill dimension would be measured on the Field Supervisory Evaluation (FSE), while one of the other remaining dimensions would be measured on the Instructor Evaluation Sheet (IES) as determined by the BSW Faculty.

Two Measures for each Competency

The attainment of SW competencies through specific dimensions as outlined by CSWE is assessed using the following two measures:

Field Supervisory Evaluation (FSE). The FSE measures the acquisition of each of the nine social work competencies outlined by CSWE as well as the tenth competency designed by the DSW and implemented by the BSW Program through the Skill dimension. Full details of the implementation process are detailed below.
Instructor Evaluation Sheets (IES): the IES measures the acquisition of each of the nine social work competencies outlined by CSWE as well as the tenth competency designed by the DSW and implemented by the BSW Program through one of the remaining three dimensions (Knowledge, Values, Cognitive and Affective Processes).

Rating Scale

Using the IES or FSE form, classroom and field instructors rate students on their level of proficiency for each SW competency through specific and particular dimensions that are measured in their courses using 5-point scale developed and approved by the Assessment Committee:

Not Yet Emerging (This dimension of the competency has not yet emerged in the student’s repertoire.)
Emerging (This dimension of the competency is beginning to emerge in the student’s repertoire.)
Developing (The student is developing this dimension of the competency.)
Approaching Consistency (The student is approaching consistent demonstration of/engagement in this dimension of the competency; consistency is anticipated in the near future.)
Consistently Demonstrates (The student consistently demonstrates/engages in this dimension of the competency.)

Results

As stated earlier, the BSW program set a benchmark of 80% of all students scoring 4 or higher on the 5-point Likert-type scale for each of the 10 SLOs/Competencies. This benchmark was met for half the competencies:

SLO/Comp 1: 73%

SLO/Comp 2: 79%

SLO/Comp 3: 76%

SLO/Comp 4: 87%

SLO/Comp 5: 81%

SLO/Comp 6: 88%

SLO/Comp 7: 86%

SLO/Comp 8: 78%

SLO/Comp 9: 78%

SLO/Comp 10: 86%

Most of the scores falling below the established bechmarks are hovering fairly close. For instance a score of 4 or higher for SLO 2: Engage Diversity and Difference in Practice occured at a rate of 79%, while SLO 8 and 9 were scored at a rate of 78%. More concerning were the scores for SLO/Comps 1: Demonstrate Ethical and Professional Behavior, 73% and SLO/Comp 3: Advance Human Rights and Social, Economic, and Environmental Justice, 76%.

While a full analysis is pending, early indications show this may have been a scoring error, an ongoing concern for the last several years that we believed rectified with training sessions and an instructional video. Two points of note, there were discrepencies between sections of the same course, which might be a result of cohort effect, but just as likely a difference of scoring philosophy on the part of the instructor (see above). Additionally, there was a significant discrepancy between the scores assigned in the classroom and those assigned by field instructors. Again, this will require further analysis by the DSW assessment committee.

14) What best describes how the program used the results? (Check all that apply.)

Assessment procedure changes (SLOs, curriculum map, rubrics, evidence collected, sampling, communications with faculty, etc.)

Course changes (course content, pedagogy, courses offered, new course, pre-requisites, requirements)

Personnel or resource allocation changes

Program policy changes (e.g., admissions requirements, student probation policies, common course evaluation form)

Students' out-of-course experience changes (advising, co-curricular experiences, program website, program handbook, brown-bag lunches, workshops)

Celebration of student success!

Results indicated no action needed because students met expectations

Use is pending (typical reasons: insufficient number of students in population, evidence not evaluated or interpreted yet, faculty discussions continue)

Other:

15) Please briefly describe how the program used its findings/results.

In the past results were used to modify both the implicit and explicit curriculum as well as the rubric used to assess collected data. Current results from assessment activities for 2019-2020 are pending a larger discussion with program faculty.

16) Beyond the results, were there additional conclusions or discoveries? This can include insights about assessment procedures, teaching and learning, and great achievements regarding program assessment in this reporting period.

Overall, the BSW Program has been satisfied with our assessment sampling efforts, the rubric used for analysis, and the results (see number #10 for specific examples. Full analysis of the data is pending.

Beyond the results, two substantial changes (one enacted and the other under review) initiated in the time since our last report include the discontinuation of the Logic Requirement for our Knowledge base and the potential reduction of total field hours to earn a bachelor’s degree in social work. For the first, the program decided to discontinue the logic requirement, believing that the FQ was sufficient. This came after considerable discussion and a thorough review of logic course syllabi. More importantly, the assessment of the logic course was inspired by ongoing concern raised by students (while in advising sessions) to the point that the program believed it best we consider and review the matter.

17) If the program did not engage in assessment activities, please justify.

Not Applicable