Program: Communication & Info Sci (PhD)
Degree: Doctorate
Date: Thu Oct 10, 2013 - 10:00:36 am
1) Below are your program's student learning outcomes (SLOs). Please update as needed.
(SLO1) Demonstrate understanding of research methods and subject knowledge in the field of Communication and Information Sciences
(SLO2) Synthesize diverse data, theories, and methods
(SLO3) Demonstrate the ability to conduct research
(SLO4) Propose and conduct original research
(SLO5) Develop and articulate a professional identity as a contributing member of a research community
2) Your program's SLOs are published as follows. Please update as needed.
Department Website URL: http://www.hawaii.edu/cis/?page=policies
Student Handbook. URL, if available online: Information Sheet, Flyer, or Brochure URL, if available online: UHM Catalog. Page Number: Course Syllabi. URL, if available online: Other: Other: 4) For your program, the percentage of courses that have course SLOs explicitly stated on the syllabus, a website, or other publicly available document is as follows. Please update as needed.
0% 1-50% 51-80% 81-99%
100%5) Did your program engage in any program assessment activities between June 1, 2012 and September 30, 2013? (e.g., establishing/revising outcomes, aligning the curriculum to outcomes, collecting evidence, interpreting evidence, using results, revising the assessment plan, creating surveys or tests, etc.)
Yes No (skip to question 14)6) For the period June 1, 2012 to September 30, 2013: State the assessment question(s) and/or assessment goals. Include the SLOs that were targeted, if applicable.
The CIS faculty and Executive Board reviewed and updated the program Student Learning Outcomes to reflect the importance of creating and articulating a professional identity as a member of a research community. We revised our Curriculum Map and program policies to reflect the updated SLOs, and created and published an assessment matrix that allows students at every point in the program to track their progress in terms of program and grad division expectations of quality and time to degree.
7) State the type(s) of evidence gathered to answer the assessment question and/or meet the assessment goals that were given in Question #6.
We collected both faculty concerns about dissertation proposal quality and student concerns about clear expectations. We created the assessment matrix so students can track whether they are exceeding, meeting, approaching or not meeting program standards at every step of the program, and in every assessed program element. We also instituted a similar metric for each committee member to assess dissertation proposals and final dissertations, that we will use to evolve program policy and assessment.
8) State how many persons submitted evidence that was evaluated. If applicable, please include the sampling technique used.
All five members of the CIS Executive Board, each of whom is also CIS faculty and active on exam and dissertation committees, contributed to the discussion. The CIS Chair discussed these issues with three other CIS faculty who are particularly active in teaching CIS courses, advising students and serving on committees. The Chair drafted revised SLOs, Curriculum Map and Assessment matrix, and corresponding CIS policy clarifications and presented the drafts to approximately 25 CIS students for comments and discussion in the mandatory CIS seminar. Perhaps a total of five students raised comments, shared anecdotes or emailed the chair afterwards with questions or comments.
9) Who interpreted or analyzed the evidence that was collected? (Check all that apply.)
Course instructor(s) Faculty committee Ad hoc faculty group Department chairperson Persons or organization outside the university Faculty advisor Advisors (in student support services) Students (graduate or undergraduate) Dean/Director Other: CIS Executive Board, consisting of the chairs of our 4 constituent units10) How did they evaluate, analyze, or interpret the evidence? (Check all that apply.)
Used a rubric or scoring guide Scored exams/tests/quizzes Used professional judgment (no rubric or scoring guide used) Compiled survey results Used qualitative methods on interview, focus group, open-ended response data External organization/person analyzed data (e.g., external organization administered and scored the nursing licensing exam) Other: 11) For the assessment question(s) and/or assessment goal(s) stated in Question #6:
Summarize the actual results.
We found that most students were enthusiastic in their support of more explicit milestones and assessment of their work. Faculty and the Executive Board felt that the existing SLOs did not fully capture the active engagement and articulation of a research identity they expect from all PhD students, and that they regularly observe in the best students. We found that students were understandably confused by the difference between grad division policies and faculty expectations concerning maximum time to degree.
12) State how the program used the results or plans to use the results. Please be specific.
Based on our program assessment, we implemented corresponding CIS program policy changes in June 2013, including the revised SLOs, updated curriculum map, and new assessment matrix. These documents have been shared with incoming students and faculty at CIS orientation 22 August 2013, and with all students in our CIS Town Hall meeting 28 August 2013. They have also been posted on our Website for prospective students to review.
13) Beyond the results, were there additional conclusions or discoveries?
This can include insights about assessment procedures, teaching and learning, program aspects and so on.
Program assessment requires an explicit articulation of how program pieces fit together, what students get from each step in the experience, and a willingness to change. We feel that our process allowed both students and longtime faculty members to question assumptions, contribute to the dialogue, and create more of a shared investment in, and commitment to, the goals and rationale of the CIS program.