Unit: Information & Computer Science
Program: Computer Science (PhD)
Degree: Doctorate
Date: Thu Sep 27, 2012 - 11:30:38 am

1) Below are your program's student learning outcomes (SLOs). Please update as needed.

These SLOs have been officially approved by the ICS faculty and are posted on our Web site. 

M.S. Program:

The ICS M.S. graduate program provides courses for advanced education in Computer Science and affords opportunities to conduct research. Our objective is to help students achieve a high level of professional competence and lifelong learning, with the following Student Learning Objectives:

  1.   Master core computer science theoretical concepts, practices and technologies;

  2.   Identify, formulate and solve problems employing knowledge within the discipline;

  3.   Contribute effectively to collaborative team oriented activities;

  4.   Communicate effectively about computer science topics using appropriate media;

  5.   Demonstrate advanced knowledge in an area of specialization within the discipline;

  6.   Engage in significant research in their area of specialization within the discipline and/or in projects that respond to community and industry needs.

Ph.D. Program:

The ICS Ph.D. graduate program provides advanced, individualized training in research in Computer Science, preparing students for research careers in academia and industry.  Beyond  those for the M.S. program, the Ph.D. program involves the three following Student Learning Objectives:

  1. Develop a research portfolio that demonstrates the capacity to carry out original research in the field;

  2. Become an expert in the area of specialization including mastery of the relevant research skills and methods, develop a research vision, and formulate a research plan that will lead to novel scientific contributions;

  3.  Execute a research plan and demonstrate original contributions to the field, as shown through findings and/or publications, culminating in a Ph.D. dissertation and oral defense.

2) Your program's SLOs are published as follows. Please update as needed.

Department Website URL: http://www.ics.hawaii.edu/academics/slos/ICS-GradPrograms-SLOs.pdf
Student Handbook. URL, if available online:
Information Sheet, Flyer, or Brochure URL, if available online: http://www.ics.hawaii.edu/academics/slos/ICS-GradPrograms-SLOs.pdf
UHM Catalog. Page Number:
Course Syllabi. URL, if available online:
Other:
Other:

3) Select one option:

Curriculum Map File(s) from 2012:

4) For your program, the percentage of courses that have course SLOs explicitly stated on the syllabus, a website, or other publicly available document is as follows. Please update as needed.

0%
1-50%
51-80%
81-99%
100%

5) Did your program engage in any program assessment activities between June 1, 2011 and September 30, 2012? (e.g., establishing/revising outcomes, aligning the curriculum to outcomes, collecting evidence, interpreting evidence, using results, revising the assessment plan, creating surveys or tests, etc.)

Yes
No (skip to question 14)

6) For the period June 1, 2011 to September 30, 2012: State the assessment question(s) and/or assessment goals. Include the SLOs that were targeted, if applicable.

* For this program, assessment is not done in regular courses since these courses are not required (or undertaking by the students as they are obtaining their M.S. degree "en route" to their Ph.D.).  The 3 Ph.D.-specific SLOs assessed are:

   - Develop a research portfolio that demonstrates the capacity to carry out original research in the field;
   - Become an expert in the area of specialization including mastery of the relevant research skills and methods, develop a research vision, and formulate a research plan that will lead to novel scientific contributions;
   - Execute a research plan and demonstrate original contributions to the field, as shown through findings and/or publications, culminating in a Ph.D. dissertation and oral defense.

These SLOs as assessed via our assessment grids, which are filled by ad-hoc committees of the faculty (PhD proposal defense, PhD final defense) or the graduate Chair (ICS690 seminar presentations). These grids are kept in paper format in the Graduate Chair's office.

* The Student Orientation assessment is filled by the students themselves and kept in the Graduate Chair's office in paper format.

7) State the type(s) of evidence gathered to answer the assessment question and/or meet the assessment goals that were given in Question #6.

* SLO Assessment grids
* Student orientation assessment

8) State how many persons submitted evidence that was evaluated. If applicable, please include the sampling technique used.

* 23 SLO assessment grid has been collected to date since June 1 2011
* 13 Graduate Student Orientation assessments were collected since June 2011, 26 in total.

9) Who interpreted or analyzed the evidence that was collected? (Check all that apply.)

Course instructor(s)
Faculty committee
Ad hoc faculty group
Department chairperson
Persons or organization outside the university
Faculty advisor
Advisors (in student support services)
Students (graduate or undergraduate)
Dean/Director
Other: Graduate Chair

10) How did they evaluate, analyze, or interpret the evidence? (Check all that apply.)

Used a rubric or scoring guide
Scored exams/tests/quizzes
Used professional judgment (no rubric or scoring guide used)
Compiled survey results
Used qualitative methods on interview, focus group, open-ended response data
External organization/person analyzed data (e.g., external organization administered and scored the nursing licensing exam)
Other:

11) For the assessment question(s) and/or assessment goal(s) stated in Question #6:
Summarize the actual results.

* Assessment Grids for thre 3 SLOs: 23 grids

Associating the following scores:
Unacceptable: 0
Marginal: 1
Acceptable: 2
Exceptional: 3
SLO1: Develop a research portfolio that demonstrates the capacity to carry out original research in the field
( 0 + 2 + 1 + 2 + 1 + 2 + 3 + 2 + 2 + 2) / 10 = 1.7
SLO2: Become an expert in the area of specialization including mastery of the relevant research skills and methods, develop a research vision, and formulate a research plan that will lead to novel scientific contributions
(2.5 + 3 + 3 + 3 + 3 + 3 + 3 + 2 + 2 + 2 + 1.5 + 3 + 2.5 + 3 + 2 + 3) / 16 = 2.6
SLO3: Execute a research plan and demonstrate original contributions to the field, as shown through findings and/or publications, culminating in a Ph.D. dissertation and oral defense
( 3 + 3 + 3  + 2.5 + 3 + 2 + 3 + 3) / 8 = 2.8
* For the Graduate Student Orientation Assessment, associating the following scores to the answers:
Totally Disagree: 0
Partially Disagree: 1
Neither Afree nor Disagree: 2
Partially Agree: 3
Totally Agree: 4
We obtain the following averages for both questions since June 2011:
Fall 2011: (8 students)
  "This orientation was useful to me": 3.88
  "I would recommend this orientation to incoming students": 4.00
Spring 2012: (0 students)
  No data was collected since only very few students were admitted that semester, and
  the orientation consisted solely in individual meetings with the Graduate Chair.
Fall 2012: (5 students)
  "This orientation was useful to me": 4.00
  "I would recommend this orientation to incoming students": 4.00

12) State how the program used the results or plans to use the results. Please be specific.

* The results for the first PhD SLO ("Develop a research portfolio that demonstrates the capacity to carry out original research in the field") is the one on which the overall score is the lowest, below the "Acceptable" rating. This is on that SLO that the program will focus its efforts in the future. We found that the main reason for these low scores is that this SLO entails writing a "literature review" and that our students almost never write a great literature review. This is mostly due to their lack of applied critical thinking. Critical thinking is something we would expect from our incoming students, but the assessment makes it clear that this expectation is unwarranted. As part of our ICS690 seminar series, we will create several sessions that focus on critical thinking for Computer Science research, and in particular focused on writing a literature review. 
* Regarding Graduate Student Orientation assessment, results over the last semester are so positive that the orientation seems in good shape. If the results were to become significantly worse, then the Graduate Committee would meet to propose a set of improvements.

13) Beyond the results, were there additional conclusions or discoveries?
This can include insights about assessment procedures, teaching and learning, program aspects and so on.

14) If the program did not engage in assessment activities, please explain.
Or, if the program did engage in assessment activities, please add any other important information here.