Departmental Assessment Update - Social Sciences Report

Department: Sociology
Program: MA, PhD
Level: Graduate

1. List in detail your graduate Student Learning Outcomes (SLOs) for each degree/certificate offered.

MA Degree (Plan A) 

 

1)  understanding of a broad range of sociological theories and methods, and commonly used statistical techniques

2)  ability to design a feasible research project to address a sociological problem or issue of theoretical interest

3)  understanding of principles of protection of human subjects and how to design sociological research that respects and protects human subjects.

4)  ability to carry out an independent research project to collect and analyze research data that addresses a sociological question

5) ability to interpret research results in relation to sociological theory, to draw reasonable inferences, and to report research results and conclusions accurately and effectively.

 

PhD Program

 

1)  high level understanding of a broad range of sociological theories and methods, and commonly used statistical techniques, and the ability to apply this knowledge to specific research problems.

2)  mastery of at least two specific subfields of sociology and a range of methods and statistical techniques appropriate to those subfields

3) ability to design a feasible research project to address a sociological problem or issue of theoretical interest

4) understanding of principles of protection of human subjects and how to design sociological research that respects and protects human subjects.

5) ability to carry out independent research to collect and analyze research data that addresses a sociological question and makes an original contribution to the field.

6) ability to interpret research results in relation to sociological theory, to draw reasonable inferences and interpretations, and to report research results and conclusions accurately and effectively.

7) ability to present research results in professional venues through presentation of papers at professional conferences and publication in appropriate professional outlets.

 

2. Where are these SLOs published (e.g., departmental web page)?

Both the MA and PhD SLOs will be published on the department website when the department has approved the draft. The draft iis the subject of a departmental meeting the first month of the Fall 2007 semester.

3. Explain how your SLOs map onto your curriculum, i.e., how does your program of graduate studies produce the specific SLOs in your students?

MA Program

 1.       Required graduate core course in theory (611 or 612) with undergraduate theory course (321 or equivalent) as prerequisite; required basic statistics course (476/L, with 605 optional), and required methods course (Soc. 475, 478, 604, 606, 608, 609, or 691 ).

2.      Detailed thesis proposal, which must be approved by student’s thesis committee.

3.       Soc. 606 includes certification in human subjects protection for investigators; students must obtain human subjects committee approval for all thesis research involving human subjects or data on living humans. Consideration of human subjects protection is concurrent with development of an acceptable thesis proposal.

4.      Thesis data collection, data analysis, and write-up

5.       Thesis write-up and oral defense.

PhD Program

 

The program uses a combination of formal coursework and a sequence of writing requirements under the guidance of the student’s committee to develop the learning outcomes in students.

 

1.       Core courses Sociology 611* and 612* cover general theories, and Sociology 613, 615, 616, 631, 632, 638, 661, 706, 711, 715, 716, 720, 721, 722, 723, 725, 730, 735, 741, 750, 751, 752, 753, and 754 cover the middle range theories and research findings of various subfields of the discipline. Sociology 476/L, 605/L* and 705 cover introductory and advanced statistics. Sociology 604, 606*, 608, 609, 691, and 701 offer various research methods. (Asterisked courses are required core courses). Ability to apply theory and methods to specific sociological problem is developed through seminars and research papers, and is evaluated in the Qualifying Review.

2.      Obtained through seminars, research papers, and preparation for comprehensive examinations in two subfields. These areas of knowledge and skills and tested through written and oral comprehensive exams.

3.       Developed through all methods and subfield seminars, through the Qualifying Review requirement, Research Papers, and the Dissertation Prospectus.  Qualifying Review requires submission of two papers that will demonstrate student’s ability to utilize sociological theory and methods to specific problems. Papers are first evaluated by the student’s own guidance committee and then submitted to the departmental QR panel convened each semester. For the Qualifying Review, Research Papers, and Dissertation Prospectus, students must develop their own research projects under committee guidance.

4.      All students receive full investigator training in human subjects protection in Soc. 606. They must apply to the Committee on Human Studies for their dissertation research if it involves human subjects, but are encouraged also to apply for approval or exemption for any original research projects they carry out for the Qualifying Review and Research Papers, since the results of these projects may later be submitted for publication.

5.       Developed through independent work in seminars and through research to fulfill the Qualifying Review, Research Papers, and Dissertation. Such research is developed and carried out under the careful guidance of the student’s committee.

6.      Developed as in 5., through independent work in seminars and through writing up research to fulfill the Qualifying Review, Research Papers, and Dissertation under the guidance of the student’s committee.

7.      Because students produce original research papers through the Qualifying Review and their Research Papers, they frequently present these papers at professional conferences and then go on to publish some of them. The process teaches students how to write in professional journal style and ensures that they also learn to present their work in a professional venue even before they get to the dissertation stage. Each year several students have their papers accepted at national professional conferences, and about a third of our doctoral students publish professionally before they complete the program.

 

4. What population(s) is covered by your assessment(s)?

All Plan A and PhD Students are covered by the assessments outlined above

5. Please list/describe all the assessment events and devices used to monitor graduate student progress through the program. Consider the following questions:

For the MA Program:

1.       Individual course grades, since students must maintain at least a B average to remain in good standing in the program. All 400, 600, and 700 level courses evaluate students on a combination of written work and oral performance in class.

2.  Supervision and evaluation of the thesis proposal, thesis, and oral defense of the thesis by a three-member committee. Most thesis proposals undergo at least two drafts before the committee approves them, and most theses require at least two or three drafts to meet committee standards. The oral defense of the thesis evaluates whether the student can organize a formal presentation of the work and can answer questions about it at an acceptable professional level. The completed theses are public products.

3.    Since most of our MA thesis projects involve human subjects, the research must also pass human subjects review before the student can carry it out. While this is not a direct measure of quality, it does require that the student present a clear description of a research project that provides appropriate protection of human subjects.

4.   Annual review of all graduate students, in which the entire faculty of the department assesses whether each student is progressing at an appropriate pace through the program. Students receive a letter each year indicating whether they are progressing appropriately and what milestones they are expected to achieve by the time of the next annual review.  Students who fail to meet annual goals despite two years of warnings are dropped from the program. (see attachments  B and C) In the 2005 review, of 26 MA students reviewed, 76.9% (20) were making satisfactory progress, 15.4% (4) received warnings of inadequate progress, 7.7% (2) received unsatisfactory progress ratings, and 2 students were dropped from the program. This compares favorably with the 2004 review of 23 MA students, in which 69.6% (16) were making satisfactory progress, 13% (3) received warnings, and 17.4% (4) received unsatisfactory progress ratings.

For the PhD Program:

1Individual course grades, since students must maintain at least a B average to remain in good standing in the program. All 400, 600, and 700 level courses evaluate students on a combination of written work and oral performance in class.

2Qualifying Review is a required portfolio process in lieu of a qualifying exam in theory and methods, which doctoral students must pass in order to be officially admitted to candidacy in the doctoral program. They must submit two papers, one of which should have been developed through independent work under the supervision of a faculty member. The two papers, taken together, should display the student’s ability to utilize sociological theory and methods to address a specific sociological problem or issue. The papers must first be submitted to the student’s guidance committee for review and approval before being submitted to the department for Qualifying Review along with a confidential evaluation by each committee member.  If the student’s committee does not find the two papers satisfactory in the fourth semester of doctoral work the review may be postponed to the fifth semester, and upon appeal can be postponed to the sixth semester. If the guidance committee does not find the work satisfactory in the sixth semester, it must still be submitted for qualifying review. A Qualifying Review panel of faculty is convened each semester to read and evaluate all papers submitted for QR. The QR committee evaluates the paper for Quality of Writing, Methodological and Logical Ability, and Analytical Skills and produces an overall evaluation on a scale from 5 (excellent) to 1 (unacceptable) (see attachment E). Both papers must achieve passing scores in order for the student to pass the QR. Students who fail the review may retake it one more time in the following semester. If they fail a second time they are dropped from the program. Since fall 2000, a total of 34 students have presented papers for the Qualifying Review. Three students failed the review, after which one successfully passed the following semester and two others left the program. One student was passed in 2004 with a score of 2 (marginal) on each paper, after which the department voted to raise the standard for passing to 3 (acceptable). Average scores for recent semesters are as follows: Spring 2003, 3.79; Fall 2003, 3.425; Spring 2004, 2.5 (including one failure and one passing at 2.0 level, average of all passing students was 2.97); Fall, 2004, 3.75; Spring 2005, 3.4 (includes one failure. Average of passing students was 3.7). Grading is quite strict. Although individual members of the QR panel may award a 5 (excellent) to individual papers, the highest average score any student has achieved in the past three years is a 4.7 on both papers.

3Annual review of all graduate students, in which the entire faculty of the department assesses whether each student is progressing at an appropriate pace through the program. Students receive a letter each year indicating whether they are progressing appropriately and what milestones they are expected to achieve by the time of the next annual review.  Students who fail to meet annual goals despite two years of warnings are dropped from the program. (see attachments  B and C)  In the 2005 annual review of 46 doctoral students, 86.9% (40) were progressing satisfactorily, 8.7% (4) were making inadequate progress, and 4% (2) received unsatisfactory ratings. This compares favorably with the 2004 review of 54 doctoral students, in which 81.5% (44) were making satisfactory progress, 9.25% (5) were warned of inadequate progress, and 9.25% (5) received unsatisfactory ratings. (Two of those students were subsequently dropped by the Graduate Division for failure complete the dissertation in time.)  

4. Written and Oral Comprehensive Exams. The written comprehensive on two subfields selected by the student is based on a reading list of 25 to 50 items in each field, compiled by the student in consultation with the committee. The exam is a one week take-home in which the student writes on three or four questions. Students may consult any written source but may not discuss the exam with any person while they are taking it. It is followed within two to three weeks by an oral examination before the committee. Since fall 2000, twenty doctoral students have passed their comprehensive exams in sociology.

5.  Since most of our students do research on human subjects, their research projects must be approved or declared exempt by the Social Science Committee on Human Subjects before the research can be conducted. This not only provides an external evaluation of the ethical standards of the research, but also requires that the students develop and articulate appropriate methodology to this external committee’s satisfaction.

6.  Supervision and evaluation by a five-member committee of the dissertation prospectus and dissertation, and public oral defense of the dissertation. Most dissertation prospectuses undergo at least two drafts before the committee approves them, and most dissertations require at least two or three drafts to meet committee standards. The oral defense of the dissertation evaluates whether the student can organize a formal presentation of the work and can answer questions about it at an acceptable professional level. The completed dissertations are public products, most of which are subsequently revised for publication either as journal articles, as monographs, or both.

 

Attachment 1. Scoring criteria and letter text for annual review of all graduate students

 

A (Satisfactory Progress)

In its annual review of graduate students, the department has found that you are making satisfactory progress toward your degree.  Within the next year, we expect that you will [insert next step in program].  Keep up the good work!

 

B  (Warning)

In its annual review of graduate students the department found that you are not currently making satisfactory progress toward your degree.  Within the next semester we expect you to [  insert next step in program]. If you are unable to show satisfactory progress toward your degree by the next annual review, your status in the program and any financial support you receive from the department may be placed in jeopardy.

 

C  (Drastic Action)

In its annual review of graduate students the department found that you have not made satisfactory progress toward your degree.  Please meet with your committee chair within the next two weeks to develop a plan and timetable for completion of [ insert next step in program].  If you are unable to show satisfactory progress toward your degree by the next annual review, you will be dropped from the program.

 

Attachment 2  Evaluation Sheet for Qualifying Review Committee

Evaluation Sheet For Qualifying Review Committee  Quality of Writing                                                                         Paper  1         Paper  2 How well is the paper written?Are the arguments clearly stated?Is the paper well organized?Are there many grammatical or typographical errors?        _______        _______ 

Methodological and Logical Ability

 Are the key research questions of the study    clearly stated?Given the research questions posed, does the     student deal with them in a logical fashion,    with the appropriate methodologies?Are the conclusions logically drawn from the analysis?                ________       _______ 

Analytical Skills

 Does the student reveal an analytic ability to critically     assess the relevant literature and to frame research     questions?Does the student exercise some sociological imagination     in probing into the research questions?Does the student draw interesting sociological/policy      interpretations from the analysis?                                            _________       ________  Overall Evaluation                                                                       __________       ________  Please use the following numerical scores for grading.  The overall evaluation does not need to be an average of the scores on Quality or Writing, Methodological and Logical Ability, and Analytical Skills.  Scores from all QR Committee members will be combined.  Papers must receive average scores of 3 in overall evaluation to pass the QR. 
  1. Excellent
  1. Good
  1. Acceptable
  1. Marginal
1.   Unacceptable

6. Please list/describe how your graduate students contribute to your discipline/academic area? Consider the following questions:

Graduate Students in the Department have been successful in having their papers acceted at local, regoinal and natoinal professional meetings in Sociology. In the most recent set of meetings (ASA in New York last week), 9 students had papers accepted for presentation. Our doctoral students are publishing on a fairly regular basis in academic journals in Hawai, nationally and internationally. Our data systems are not presently capturing those events however so exact counts cannot be given.

7. What attempts are made to monitor student post-graduate professional activities?

On at least a 5 year basis, attempts are made to contact our graduates and to collect CVs and responses to a prestructured questionnaire on  satisfaction and utility of their degrees. These data will form a part of the Graduate Studies program committee database but are presently announced at departmental gatherings.

8. How were the assessment data/results used to inform decisions concerning the curriculum and administration of the program?

The Grafduate Studies Committee are beginning a process of review in preparation fo our COPR next year and as an activity under the departmental 5-year plan.

9. Has the program developed learning outcomes? Please indicate yes or no.

not yet

10. Has the program published learning outcomes? Please indicate yes or no.

no

11. If so, please indicate how the program has published learning outcomes.

12. What evidence is used to determine achievement of student learning outcomes?

succes in the market place

13. Who interprets the evidence?

Chair and Graduate Studies committee

14. What is the process of interpreting the evidence?

15. Indicate the date of last program review.

1997 with the knowledge that we were to be reviewed inthe spring but apparently that has now been put off for a year