Program: Sociology (PhD)
Degree: Doctorate
Date: Fri Nov 20, 2020 - 10:48:54 am
1) Program Student Learning Outcomes (SLOs) and Institutional Learning Objectives (ILOs)
1. Demonstrate understanding of a broad range of sociological theories and be able to apply them to frame a research problem and to conduct a research project.
(1. Demonstrate comprehensive knowledge in one or more general subject areas related to, but not confined to, a specific area of interest.)
2. Demonstrate understanding of a range of quantitative and qualitative methods for conducting sociological research. �
(1. Demonstrate comprehensive knowledge in one or more general subject areas related to, but not confined to, a specific area of interest., 2. Demonstrate understanding of research methodology and techniques specific to one’s field of study.)
3. Apply principles to protect human subjects in a sociological research project.� �
(6. Conduct research or projects as a responsible and ethical professional, including consideration of and respect for other cultural perspectives.)
4. Acquire a professional level of knowledge in selected subfields of sociology in order to be equipped to teach courses on the subject and to develop research projects that will advance the field in the particular area.
(2. Demonstrate understanding of research methodology and techniques specific to one’s field of study., 3. Apply research methodology and/or scholarly inquiry techniques specific to one’s field of study.)
5. Demonstrate ability to carry out a research project that will contribute new knowledge to the field using appropriate quantitative and/or qualitative methods. �
(3. Apply research methodology and/or scholarly inquiry techniques specific to one’s field of study., 4. Critically analyze, synthesize, and utilize information and data related to one’s field of study.)
6. Demonstrate ability to analyze research data and to write clearly and effectively for a professional academic audience
(4. Critically analyze, synthesize, and utilize information and data related to one’s field of study., 5. Proficiently communicate and disseminate information in a manner relevant to the field and intended audience.)
7. Demonstrate ability to present research findings clearly and effectively in a professional setting, such as a classroom or academic conference.
(5. Proficiently communicate and disseminate information in a manner relevant to the field and intended audience., 7. Interact professionally with others.)
2) Your program's SLOs are published as follows. Please update as needed.
Department Website URL: https://sociology.manoa.hawaii.edu/phd-program/
Student Handbook. URL, if available online: Information Sheet, Flyer, or Brochure URL, if available online: http://www.sociology.hawaii.edu/graduate/phd.html UHM Catalog. Page Number: Course Syllabi. URL, if available online: http://socialsciences.people.hawaii.edu/esyllabi/index.cfm Other: 3) Please review, add, replace, or delete the existing curriculum map.
- File (10/15/2020)
4) For your program, the percentage of courses that have course SLOs explicitly stated on the syllabus, a website, or other publicly available document is as follows. Please update as needed.
0% 1-50% 51-80%
81-99% 100%5) Does the program have learning achievement results for its program SLOs? (Example of achievement results: "80% of students met expectations on SLO 1.")(check one):
No Yes, on some(1-50%) of the program SLOs Yes, on most(51-99%) of the program SLOs Yes, on all(100%) of the program SLOs6) Did your program engage in any program learning assessment activities between November 1, 2018 and October 31, 2020?
Yes No (skip to question 17)7) What best describes the program-level learning assessment activities that took place for the period November 1, 2018 and October 31, 2020? (Check all that apply.)
Create/modify/discuss program learning assessment procedures (e.g., SLOs, curriculum map, mechanism to collect student work, rubric, survey) Collect/evaluate student work/performance to determine SLO achievement Collect/analyze student self-reports of SLO achievement via surveys, interviews, or focus groups Use assessment results to make programmatic decisions (e.g., change course content or pedagogy, design new course, hiring) Investigate other pressing issue related to student learning achievement for the program (explain in question 8) Other: 8) Briefly explain the assessment activities that took place since November 2018.
All sociology faculty continue to meet every Spring semester to conduct an Annual Review of Graduate Students. This review evaluate MA and PhD students, placing them in three categories: "satisfactory progress", "warning", and "drastic action" based upon time in program and accomplishment of major progress milestones, including coursework requirements, Qualifying Review exam, Comprehensive Exam, Dissertation Proposal Defense, and Dissertation Defense. Signs of partial progress such as formation of Guidance or Thesis Committees may also affect the evaluation. The Chair of that student's guidance/thesis committee presents an evaluation of the student, and other relevant faculty also provide feedback, such as if the student was in a class or was a teaching assistant or graduate research assistant. The faculty then agree on an assessment for each student in the meeting. Then, the Chair of the Graduate program sends each student an Annual Review letter describing the student's review status, such as making "satisfactory progress" (which is the highest level) and suggestions for next steps, and in the case of "warning" or "drastic action", the goals that must be taken to achieve satisfactory progress.
Each SLO should be achieved at latest by a particular milestone as follows: Coursework: 1, 2, 3 Qualifying Review:1, 2, 5, 6. Comprehensive Exam: 1, 4. Dissertation Proposal Defense: 3, 5, 7 Dissertation: 6; Dissertation Defense: 7.
The faculty have decided to more directly incorporate discussion of SLOs into our Annual Review standards, and discussions will be ongoing on how best to do so. The Graduate Studies Committee met recently to discuss how to incorporate the SLOs into our Annual Review of Graduate Students meeting. The graduate studies committee wants to discuss in a faculty meeting how to evaluate SLO achievement levels and if we want to use the same categories as the undergraduate studies committee to be consistent. There was discussion about what "excellence" would actually mean and how to assess it reliably among the graduate studies committee members.
9) What types of evidence did the program use as part of the assessment activities checked in question 7? (Check all that apply.)
Artistic exhibition/performance Assignment/exam/paper completed as part of regular coursework and used for program-level assessment Capstone work product (e.g., written project or non-thesis paper) Exam created by an external organization (e.g., professional association for licensure) Exit exam created by the program IRB approval of research Oral performance (oral defense, oral presentation, conference presentation) Portfolio of student work Publication or grant proposal Qualifying exam or comprehensive exam for program-level assessment in addition to individual student evaluation (graduate level only) Supervisor or employer evaluation of student performance outside the classroom (internship, clinical, practicum) Thesis or dissertation used for program-level assessment in addition to individual student evaluation Alumni survey that contains self-reports of SLO achievement Employer meetings/discussions/survey/interview of student SLO achievement Interviews or focus groups that contain self-reports of SLO achievement Student reflective writing assignment (essay, journal entry, self-assessment) on their SLO achievement. Student surveys that contain self-reports of SLO achievement Assessment-related such as assessment plan, SLOs, curriculum map, etc. Program or course materials (syllabi, assignments, requirements, etc.) Other 1: Comprehensive exam reading lists Other 2: Dissertation proposal and oral defense10) State the number of students (or persons) who submitted evidence that was evaluated. If applicable, please include the sampling technique used.
Since milestone progress is used as a primary tool for assessment in our Annual Review of Graduate Students, the number of students providing materials of each type varies from semester to semester, depending on the trajectories of students in the program. Our annual review evaluates all graduate students every year. We had 30 PhD students enrolled in our graduate program in Spring 2019, Fall 2019, Spring 2020, and 26 PhD students enrolled in Fall 2020.
I was hoping to finalize other ways to collect evidence of SLO assessment, such as from Dissertation Chairs' evaluations of SLO achievement after students graduate, and graduate student self-evaluations of achieving the SLOs. The Graduate Studies Committee had a recent discussion in a meeting and was satisfied with our current Annual Reviews as assessement of our students and the GSC members were not in agreement about whether we should prioritize new assessment procedures at this time when we need to focus on student recruitment. I phased in as Graduate Chair this Fall 2020 semester, so I plan to find a way to better assess each SLO achievement as we do the Annual Review of Students and collect data each year to report results.
11) Who interpreted or analyzed the evidence that was collected? (Check all that apply.)
Course instructor(s) Faculty committee Ad hoc faculty group Department chairperson Persons or organization outside the university Faculty advisor Advisors (in student support services) Students (graduate or undergraduate) Dean/Director Other: 12) How did they evaluate, analyze, or interpret the evidence? (Check all that apply.)
Used a rubric or scoring guide Scored exams/tests/quizzes Used professional judgment (no rubric or scoring guide used) Compiled survey results Used qualitative methods on interview, focus group, open-ended response data External organization/person analyzed data (e.g., external organization administered and scored the nursing licensing exam) Other: Our annual graduate student paper competition motivates students to submit papers that demonstrate SLOs 1,2, 5, 613) Summarize the results from the evaluation, analysis, interpretation of evidence (checked in question 12). For example, report the percentage of students who achieved each SLO.
Our current evaluation of evidence is based on milestones and Annual Reviews of all graduate students. As mentioned earlier, the Annual Reviews of students use a rubric to assess progress and faculty use their professional judgment for each of the milestones in our PhD program. Based on the Annual Reviews, all 26 PhD students are making "satisfactory progress" (which is the highest level of evaluation) towards their degrees and making progress on their SLOs as expected.
To elaborate, based on recent evidence of a milestone for this Fall 2020 semester, all 4 PhD students who did the Qualifying Review passed, so 4/4 (100%) students succeeded and accomplished that milestone and met SLOs 1, 2, 5, 6. Students are required to take the QR by submitting a paper to their 3-person guidance committee in the fifth semester of enrollment in the program at the latest.
SLO 1 Demonstrate understanding of a broad range of sociological theories and be able to apply them to frame a research problem and to conduct a research project.
SLO 2 Demonstrate understanding of a range of quantitative and qualitative methods for conducting sociological research.
SLO 5 Demonstrate the ability to carry out a research project that will contribute new knowledge to the field using appropriate quantitative and/or qualitative methods.
SLO 6 Demonstrate ability to analyze research data and to write clearly and effectively for a professional academic audience.
The QR committee (3-person guidance committee) evaluates whether a student's research paper has met the following criteria at a sufficient level for graduate-level work in sociology to pass:
- understanding and creating theoretical and methodological material at the graduate level
- thinking analytically at the graduate level
- writing effectively at the graduate level
Also, 4 students passed the comprehensive exam in 2020 so they achieved SLOs 1 and 4. All students who took the comprehensive exams passed. The comprehensive exam is a week long take-home written exam and an oral exam to examine knowledge of 2 areas of research/teaching specialization. Passing the comprehensive examine achieves SLO 1 and especially SLO 4.
SLO 4 Acquire a professional level of knowledge in selected subfields of sociology in order to be equipped to teach courses on the subject and to develop research projects that will advance the field in the particular area.
Completing a dissertation and the oral defense of the findings achieve SLO 3, 5, 6, 7.
SLO 3 Apply principles to protect human subjects in a sociological research project
SLO 7 Demonstrate the ability to present research findings clearly and effectively in a professional setting, such as a classroom or academic conference.
A number of our students also regularly attend regional or national meetings, but we don't usually keep track of how many each year. Maybe that information could be included in a graduate student self-evaluation of SLOs in the future.
Since the last assessment report in 2018, we have had 6 PhD students successfully complete and defend their dissertations and graduate with their PhD degrees in 2019-2020. Based on their positive Annual Review letters, those 6 students met all of our program SLOs at some point during their degree at UHM.
In Spring 2020, 2 students completed their PhD degrees.
In Fall 2019, there were no graduating PhD students.
In Summer 2019, 2 students completed their PhD degrees.
In Spring 2019, 2 students completed their PhD degrees
In 2019, we had 4 PhD students graduate out of 30 enrolled PhD students 4/30, so at least (13%) achieved all SLOs by completing their degrees. In 2020, 2 students have completed their PhDs so far, so at least 2/26 (7%) enrolled students completed all SLOs. One more student will be graduating later this semester once the paperwork is completed.
As the new graduate chair, I will incorporate a more specific analysis of SLO achievement into the Annual Review of Graduate Students meeting, so I can report those results in our assessments in more detail. The department will need to decide on the SLOs levels of evaluation "not satisfactory" to "excellent" and if we want to use the same categories as the Undergraduate Studies Committee and decide on what each level means.
Overall, our department is very satisfied with these results and proud of our PhD students.
14) What best describes how the program used the results? (Check all that apply.)
Assessment procedure changes (SLOs, curriculum map, rubrics, evidence collected, sampling, communications with faculty, etc.) Course changes (course content, pedagogy, courses offered, new course, pre-requisites, requirements) Personnel or resource allocation changes Program policy changes (e.g., admissions requirements, student probation policies, common course evaluation form) Students' out-of-course experience changes (advising, co-curricular experiences, program website, program handbook, brown-bag lunches, workshops) Celebration of student success! Results indicated no action needed because students met expectations Use is pending (typical reasons: insufficient number of students in population, evidence not evaluated or interpreted yet, faculty discussions continue) Other: 15) Please briefly describe how the program used its findings/results.
For the first semester of the modified Qualifying Review, we used the findings of 100% success with all students who took the QR passing as positive feedback that the new process is working well so far. We are going to ask the graduate students for feedback on the process and their experience to see if the instructions for the new process can be improved or if they are sufficient. We are also going to get faculty feedback, too. The graduate studies committee members are very satisfied with the new QR process and the students' success this semester.
Based on the Annual Review of students, we are very proud of the students who have recently graduated from our PhD program and that we currently have all students "making satisfactory progress" (which is the highest level of evaluation) towards our PhD degree in sociology. We celebrated our students' success at the end-of-semester Departmental Potlucks in 2018 and 2019, but due to the pandemic, we'll schedule a virtual end-of-semester celebration.
16) Beyond the results, were there additional conclusions or discoveries? This can include insights about assessment procedures, teaching and learning, and great achievements regarding program assessment in this reporting period.
Beyond the results so far, we realize that there is room for improvement and we plan to decide on "how" to include assessment of SLOs more directly into our Annual Review of students without it extending the length of the meeting too much. The graduate studies committee wants to discuss further with the faculty and sociology chair about the categories of "unsatisfactory to excellence" to evaluate level of achievement of each SLO and what those levels of evaluation actually mean for achieving an SLO before implementing these new types of assessment and to be able to understand the results.
As a department, we want to continue to celebrate students' succcess and focus more on recruiting and retaining more students. For example, this semester we plan to create a new graduate program flyer that we can email and post to recruit more PhD students to our sociology department at UHM. Our graduate program size has been declining, which is a national trend in sociology, and could get worse in the near future due to the pandemic and economic downturn. We also want to work more on providing professional development, we have considered a proseminar for course credit, to help our graduate students be competitive in an unprecedented and challenging job market for academia. As Graduate Chair, I plan to conduct Zoom meetings about our graduate program and visiting online classes to invite our sociology majors to learn more about our graduate program and to encourage them to apply and answer any questions that they may have.