Unit: Communication and Info Sci (PhD, interdisciplinary)
Program: Communication & Info Sci (PhD)
Degree: Doctorate
Date: Sat Oct 17, 2009 - 7:51:22 am

1) List your program's student learning outcomes (SLOs).

The student is expected to spend two to three years, depending on the student’s background,

(A) obtaining comprehensive mastery of the methods and substance in the field of Communication and Information Sciences;

(B) developing the ability to productively synthesize diverse data, theories, and methods; and

(C) demonstrating the ability to conduct research prior to proposing a dissertation study. 

The student then focuses on

(D) proposing and conducting original research in his or her area, and

(E) writing and defending a dissertation on that research. 

2) Where are your program's SLOs published?

Department Website URL: http://www.hawaii.edu/cis/?page=policies
Student Handbook. URL, if available online: http://www.hawaii.edu/cis/documents/CIS_Policies_2008-08.pdf
Information Sheet, Flyer, or Brochure URL, if available online:
UHM Catalog. Page Number:
Course Syllabi. URL, if available online:
Other:
Other:

3) Upload your program's current curriculum map(s) as a PDF.

Curriculum Map File(s) from 2009:

4) What percentage of courses have the course SLOs explicitly stated on the course syllabus, department website, or other publicly available document? (Check one)

0%
1-50%
51-80%
81-99%
100%

5) State the SLO(s) that was Assessed, Targeted, or Studied

Timely student progress toward all SLOs except (C) was studied, since the independent measure of (C) was added too recently for this study. 

6) State the Assessment Question(s) and/or Goal(s) of Assessment Activity

The goal of the assessment activity was to determine why the average time to completion of a dissertation in our program was so high. 

7) State the Type(s) of Evidence Gathered

  • Number of attempts at comprehensive exams and dates of completion of each of the four required exams.
  • Date of successful proposal defense.
  • Date of successful final defense.
  • Date of graduation.
  • Comments were also gathered from students and faculty concerning their perceptions of the program.

8) State How the Evidence was Interpreted, Evaluated, or Analyzed

Minimum, average and maximum times to meet the milestones of the previous question were computed for (1) students who had graduated, (2) ABD students, and (3) students who were between exams and proposal. We then identified the periods that were taking the longest. We also summarized the gist of the comments from students and faculty. 

9) State How Many Pieces of Evidence Were Collected

The timeline for each student in the program from admission through completion of area exams was collected for each student, with results analyzed by individual student, year of admission, and specific area exam.  This involved at least four exam results for each student, often more. This was the entire population, not sampling. Dozens of comments were recorded in a face-to-face seminar discussion and an online discussion forum. This was a voluntary sample (those who elected to speak  up). 

10) Summarize the Actual Results

Times in months:

  • From entry to completion of exams: Minimum 8, Average 30, Maximumm 54
  • From completion of exams to successful proposal defense: Minimum 3, Average 23, Maximum 68
  • From proposal defense to final defense: Minimum 5, Average 13, Maximum 27

The most frequent comments about the program included

Advantages to Retain:

  • Taking courses and involvement with faculty in 4 programs
  • Availability of large cohort of faculty as committee members

Problems to Address:

  • Students not publishing enough before graduating
  • Takes too long to get through comprehensive exams
  • No engagement in research during comprehensive exams is required
  • Insufficient student-faculty contact during comprehensive exams

11) Briefly Describe the Distribution and Discussion of Results

The results and conclusions (next item) were discussed at meetings of the Executive Board, and at a faculty retreat. They were also disseminated to the CIS program email list.

12) Describe Conclusions and Discoveries

Two related areas of concern were identified. Students were spending an excessive amount of time in the exam phase of the program, usually several years. There was a lack of ongoing involvement of students in research and with faculty during this time. Subsequently students took too long to write their proposals, apparently because they first started thinking about research after finishing the exams. Completion of the proposed work was relatively rapid after a successful proposal defense (partly because we encourage pilot studies before the proposal defense). Therefore it should be possible to streamline completion if we keep students focused on research during the exam period so they are also preparing to write a proposal. This interpretation of quantitative results was consistent with student and faculty perception of the strong exam requirements coupled with no formal process for involvement in research early in the program. 

13) Use of Results/Program Modifications: State How the Program Used the Results --or-- Explain Planned Use of Results

The program was modified to reduce the number of primary exams required, and add instead a requirement to publish a research paper. Faculty mentoring of new students was also implemented: students are paired with a faculty mentor by the beginning of the second year for purposes of developing this research paper.  The deadline for the paper is the same as for finishing the exams.

Also, each student provides a written progress update to the chair in advance of the bi-annual Executive Committee meeting.  He/she is then informed of any board response, including the prospect of probation for missing program milestones spelled out in the Policies and Procedures document.

14) Reflect on the Assessment Process

The process of examining actual progress of students through the program was vital in identifying the cause of problems and proposing and choosing remedies.

The curriculum map concept does not fit our program well, as this flexible interdisciplinary PhD program is not based on a specified set of courses.  We will consider the possibility of refactoring our learning objectives to better to enable independent mapping of objectives to outcomes. 

15) Other Important Information

As an interdisciplinary program, CIS has only 4 courses of its own.  Most graduate coursework is offered through the sponsoring schools and departments.  Therefore, most syllabi are the products of those departments. This is why we have not indicated the percentage of courses stating SLOs. (We do not have the data and it is not clear whose SLOs should be posted: the department's or the CIS program's?) Approximately 32 courses from these departments are currently listed as recommended for exam preparation. The set of courses taken depends on the focus areas the student chooses.

 One item identified in last year’s assessment report was the need to communicate with alumni.  Efforts in this regard this year included a personal letter mailed to each alumni which explained the program changes outlined above, and the program’s first newsletter which included alumni news.  This will be a continuing effort now that the program has some staff support to make it possible.

16) FOR DISTANCE PROGRAMS ONLY: Explain how your program/department has adapted its assessment of student learning in the on-campus program to assess student learning in the distance education program.

17) FOR DISTANCE PROGRAMS ONLY: Summarize the actual student learning assessment results that compare the achievement of students in the on-campus program to students in the distance education program.