Unit: Education (multiple departments)
Program: Education (PhD)
Degree: Doctorate
Date: Fri Nov 20, 2009 - 2:11:58 pm

1) List your program's student learning outcomes (SLOs).

The objectives of the program are

1. Knowledge.  Students will become knowledgeable in the broad issues of education and in the skills, knowledge and problems within their specialization area.  Students will develop an inquiring and critical approach to issues and possible solutions to problems in education.  The body of knowledge and specific objectives are developed on an individual basis in collaboration among the student, his/her advisor, and committee members. The doctoral program is unique in that students are expected to develop their knowledge of the field to the degree that they can understand the issues and problems in the field of education, and their specialization, in order to develop an inquiry project that will generate new knowledge in education. 

2. Research. Students will develop competencies in the broad issues of conducting and evaluating research in education, and develop the skills needed to develop a research problem and questions, design a study about a significant issue, collect appropriate data, analyze results and write a dissertation.  Students’ dissertation research must be on an original problem and contribute to the knowledge base in education.  The results of students  research will be disseminated internationally, nationally and locally, where appropriate, so that it may be utilized for the improvement of education.

3. Caring and Professional Ethics.  Students will develop a deep respect for the public trust that is invested in them as future intellectual and social leaders in the field of education.  Research that they conduct, or to which they refer in making recommendations, will be carefully vetted for accuracy, fairness, and beneficence regarding the clients, recipients, participants and the broader public good.  The program supports an attitude or disposition of caring or consideration for all people, and especially for students at every level and from various walks of life. 

2) Where are your program's SLOs published?

Department Website URL:
Student Handbook. URL, if available online:
Information Sheet, Flyer, or Brochure URL, if available online:
UHM Catalog. Page Number:
Course Syllabi. URL, if available online:
Other:
Other:

3) Upload your program's current curriculum map(s) as a PDF.

Curriculum Map File(s) from 2009:

4) What percentage of courses have the course SLOs explicitly stated on the course syllabus, department website, or other publicly available document? (Check one)

0%
1-50%
51-80%
81-99%
100%

5) State the SLO(s) that was Assessed, Targeted, or Studied

The SLOs that were assessed were knowledge, ability to conduct and complete research for a dissertation, carrying out ethical research, and successfully completing all requirements for graduating with a PhD in Education.

6) State the Assessment Question(s) and/or Goal(s) of Assessment Activity

The program wanted to document the following:

How many students successfully passed the written comprehensive exam?

How many student successfully passed the oral comprehensive exam?

How many students successfully defended their dissertation proposal?

How many students successfully gained approval from the IRB for their research?

How many students successfully defended their disseration in the oral defense?

How many students successfully completed all requirements for the program, including turning in their final revised dissertation draft and graduated?

7) State the Type(s) of Evidence Gathered

PhD in Education Program Assessments

Title of Assessment

Form of Assessment

Time of Assessment

Who Assesses the Work?

1. Knowledge: Comprehensive Exam, Written

Written Comprehensive Exam

Upon completion of course work and committee meeting with doctoral student previous to exam to develop focus for questions

Specialization chair and doctoral committee

2. Knowledge: Comprehensive Exam, Oral

Oral Comprehensive Exam

Upon successful completion of written comprehensive exam

Specialization chair and doctoral committee

3. Research Skills: Oral defense of the dissertation proposal; acceptable written proposal

Dissertation Proposal, written and oral defense

After completion of course work and before beginning the data collection for the dissertation

Dissertation committee of 5, including outside full graduate faculty member to monitor the process

4. Dispositions: Research Ethics and protection of human subjects

IRB Approval of the dissertation and certificate of completion of IRB training in research ethics

At time of dissertation proposal and before student begins dissertation study and signs up for EDUC 800, Dissertation

IRB at the University of Hawaii, Manoa; Committee chair; and Graduate Division

5. Research Skills: Dissertation Oral Defense, acceptable written dissertation

Oral Defense of Dissertation

After the dissertation chair approves final copy of the dissertation and it is circulated among the committee

Dissertation committee. College faculty and students, and the public

6. Knowledge, Research Skill and Professionalism: Final copy of dissertation meets committee approval.

Graduate Division approval for graduation.  This includes committee signatures approving the final copy of the written dissertation and any needed revisions; a grade check documenting that all course and other  requirements of the program and the graduate division were met, and publication of the dissertation through ProQuest or another publisher

Two copies of the approved dissertation are turned into the graduate division, and the student meets any other requirements for graduation.

The committee reads, approves and signs  the final version of the dissertation.  The program chair and specialization coordinator conduct a grade check on all core and other course requirements.  The graduate division reviews the student’s record to ascertain that all University of Hawaii, Manoa, requirements have been met.

The College of Education doctoral program in collaboration with the Graduate Division monitors student progress throughout the program. Students grades are monitored during the course taking phase of the program, and students must pass a number of assessments as shown about along the way to graduation. Students are assigned interim advisors in the beginning of the program, and later chose a permanent advisor to chair their dissertation.

8) State How the Evidence was Interpreted, Evaluated, or Analyzed

This was described previously in # 7.

9) State How Many Pieces of Evidence Were Collected

Six types of evidence were collected, as described in #7.

10) Summarize the Actual Results

Items 1-4: Data for Students Still in the PhD program as of Spring 2009 (does not include graduates): Items 5 & 6: Data for Graduates

1. Knowledge: Written Comprehensive Exam

Academic year

Unacceptable

Acceptable

N %

N/%

2008-2009

10

2007-2008

05

2006-2007

05

2005-2006

03

2. Knowledge: Oral Comprehensive Exam

Academic year

Unacceptable

Acceptable

N/%

N/%

2008-2009

10

2007-2008

05

2006-2007

05

2005-2006

03

3. Research Skills: Oral Defense of the Dissertation Proposal

Academic year

Unacceptable

Acceptable

N/%

N/%

2008-2009

10

2007-2008

04

2006-2007

07

2005-2006

02

4. Dispositions: Research Ethics

Academic year

Unacceptable

Acceptable

N/%

N/%

2008-2009

06

2007-2008

04

2006-2007

07

2005-2006

02

Note. IRB approval is needed only for those students conducting research with human subjects or animals. A few students did not need IRB approval.

5. Research Skills: Oral Defense of the Dissertation (Graduates)

Academic year

Unacceptable

Acceptable

N/%

N/%

2008-2009

12*

2007-2008

20

2006-2007

16

2005-2006

15

6. Knowledge, Research Skill and Professionalism: Final approval of the dissertation, and all other requirements of program approved for graduation.

[Graduates: Evidence is also given in the Completers Table above]

Academic year

Unacceptable

Acceptable

N/%

N/%

2008-2009

12*

2007-2008

20

2006-2007

16

2005-2006

15

* Note:  The data for 2008-2009 does not include summer 2009 graduates.  The number of summer graduates was 7 in summer 2008, 16 in summer 2007, and 15 in summer 2006.

Candidate and Completer Chart for the PhD in Education Program

PHD in Education Program

Academic Year

(Provide for last 3 years)

Number of candidates enrolled in the program

Number of program completers

2008-2009

189[228 listed, 189 taking courses, Grad Div. enrollment report] 12 (summer results to be added)

2007-2008

169 20

2006-2007

149 16

11) Briefly Describe the Distribution and Discussion of Results

The results of this assessment was placed on the college of education website under the NCATE section.  Faculty and administrators have reviewed the report.  The assessment system was collaboratively developed by representative faculty within the program, and the administration for the College, especially the administrative committee for the program.  The information in this report contains the key elements of the NCATE accreditation report (which is public on the web), and the entire report contains many examples that are useful to faculty serving on doctoral committees as members or chairs.

12) Describe Conclusions and Discoveries

Student performance on the comprehensive exam, the dissertation proposal defense, and the oral defense of the dissertation combined with a careful reading of the written dissertation, is the basis for faculty recommendations, and suggestions for the improvement of student work. Indeed the preparation for taking the comprehensive exam, construction of exam questions, and also work leading up to the proposal and final dissertation draft benefits from constant faculty advisement and supervision.

Student grades and progress through the program is monitored by the program, and the graduate division. The Graduate Division monitors student's obtaining approval from the IRB for all dissertations involving the participation of Human Subjects.

In addition, problems or challenges students and faculty are facing in the program is discussed and acted upon in the Graduate Faculty of Education Administrative committee, and by the coordinators of each of the specializations.

Over the years, faculty have become concerned about the length of time that it takes students to progress through the comprehensive exam and the dissertation proposal, and currently many students are encouraged to defend their dissertation proposal before taking the comprehensive exam. This has helped students to focus and progress through this stage of the program.

Some students who have been engaged in action research, or a pilot study for their dissertation have applied for approval from the IRB with the supervision of their advisor, before obtaining approval from their full committee for the disseration proposal. This has also helped to speed up the progress to degree.

13) Use of Results/Program Modifications: State How the Program Used the Results --or-- Explain Planned Use of Results

The program is continually evaluated by faculty and by the graduate faculty of education (GFE) administrative committee for the PhD in Education Program.  Facaulty have sought to reduce the time between course completion and the dissertation by in many cases having the students develop their dissertation proposal and have it reviewed by the IRB before they take their comprehensive examination.  This has been hugely successful in helping students proceed through the program.  In addition the program has a stong research in education emphasis which helps provide a foundation for student research.  In addition, the program offers students who are interested in college teaching to do an internship in college teaching mentored by a professor.  The follow-up survey of graduates has also shown that students believe that the dissertation is the most important part of the program and that the program has helped them to advance in their careers or to seek a new career, especailly in college teaching. 

14) Reflect on the Assessment Process

The evaluation of the program was streamlined with the requirements of the graduate division and the type of monitoring and assessment that is important in graduate work and a high level of scholarship and research in the field.  Given more resources, even more could be done to evaluate the Phd in Education program using up to date evaluation strategies such as focus groups of faculty and students, and stakeholder interviews of administrators as well as other participants.  I also believe that the thrust of these types of endeavors should be called evaluation and assessment to show the true nature of this effort.

15) Other Important Information

The PhD in Education Program,  a college-wide PhD with seven specializations, has recently been accredited by the NCATE as part of the overall assessment of College of Education graduate programs.  As part of a research one University we consider doctoral research in education for educational leaders in schools, universities, including many teachers and university instructors, to be essential in improving public education at all levels.

16) FOR DISTANCE PROGRAMS ONLY: Explain how your program/department has adapted its assessment of student learning in the on-campus program to assess student learning in the distance education program.

17) FOR DISTANCE PROGRAMS ONLY: Summarize the actual student learning assessment results that compare the achievement of students in the on-campus program to students in the distance education program.