Unit: Educational Psychology
Program: Educational Psychology (PhD)
Degree: Doctorate
Date: Thu Oct 15, 2009 - 3:35:50 pm

1) List your program's student learning outcomes (SLOs).

1.    Educational Psychology graduate students are learning and development, inquiry methods, and student assessment.

2.    Educational Psychology graduate students have inquiry skills to conduct scholarly research effectively.

3.    Educational Psychology graduate students present scholarly research effectively.

  1. Educational Psychology graduate students model the ethical treatment of research participants.

2) Where are your program's SLOs published?

Department Website URL: http://www.coe.hawaii.edu/edep
Student Handbook. URL, if available online:
Information Sheet, Flyer, or Brochure URL, if available online: NA
UHM Catalog. Page Number:
Course Syllabi. URL, if available online:
Other:
Other:

3) Upload your program's current curriculum map(s) as a PDF.

Curriculum Map File(s) from 2009:

4) What percentage of courses have the course SLOs explicitly stated on the course syllabus, department website, or other publicly available document? (Check one)

0%
1-50%
51-80%
81-99%
100%

5) State the SLO(s) that was Assessed, Targeted, or Studied

1.    Educational Psychology graduate students are learning and development, inquiry methods, and student assessment.

2.    Educational Psychology graduate students have inquiry skills to conduct scholarly research effectively.

3.    Educational Psychology graduate students present scholarly research effectively.

  1. Educational Psychology graduate students model the ethical treatment of research participants.

6) State the Assessment Question(s) and/or Goal(s) of Assessment Activity

  1. We wanted to know whether the students had the requisite knowledge of learning and development, inquiry methods, and student assessment needed to (a) develop an appropriate research question and (b) contextualize their research question within existing research.
  2. We wanted to know whether students had the inquiry skills needed to (a) develop a research design that was a good fit with their research questions, (b) describe their methods clearly, such that others could replicate them, and (c) plan for and implement data analyses that would answer their research questions.
  3. We wanted to know whether students had the skills needed to make an effective research presentation.
  4. We wanted to know whether students modeled the ethical treatment of research participants by designing and conducting research that was consistent with the standards set forth by the University’s Committee on Human Studies.

7) State the Type(s) of Evidence Gathered

  1. The dissertation chairs rated their students’ dissertation proposals and final paper literature reviews using a rubric developed by the Department faculty.
  2. The dissertation chairs rated their students’ dissertation proposals and final paper method sections using a rubric developed by the Department faculty.
  3. The dissertation chairs rated their students’ dissertation research presentations using a rubric developed by the Department faculty.
  4. Students who conducted dissertation research involving human participants were required to submit an application to the Committee on Human Studies that described their use of human participants, the potential risks involved, and how they would guard against such risks. We collected information on whether these applications were approved by the Committee on Humans Studies.

8) State How the Evidence was Interpreted, Evaluated, or Analyzed

The Department staff analyzed the percentage of students who met, did not meet, or exceed expectations on each component of each of the five rubrics (Proposal Literature Review, Dissertation Literature Review, Proposal Method Section, Dissertation Method Section, and Research Presentation). The dissertation chairs informed the Department staff about whether students’ human studies applications were approved.

The Department faculty met to discuss these results and to determine areas or relative strength and weakness.

9) State How Many Pieces of Evidence Were Collected

We collected scores for all dissertation proposals, dissertations, and dissertation research presentations. We also collected information on whether all human studies applications had received approval.

Our data included scores for four dissertation proposals, four dissertations, and four research presentations. We also received information regarding whether three human studies applications received approval.

10) Summarize the Actual Results

  1. Regarding the proposal literature review, 50% of the candidates met and 50% exceeded expectations regarding the statement of the research question. Twenty five percent met, 50% exceeded, and 25% did not meet expectations regarding the contextualization of the research question within existing literature. Fifty percent met and 50% exceeded expectations for the organization of their proposals and for writing style and conventions.
  2. Regarding the proposal method section, 50% met and 50% exceeded expectations for the research design. All of the candidates exceeded expectations regarding the description of the research participants. Seventy five percent met and 25% exceeded expectations for the description of the procedures. Fifty percent met and 50% exceeded expectations for the data analysis.
  3. Regarding the dissertation literature review, all of the candidates exceeded expectations regarding the statement of the research question. Fifty percent met and 50% exceeded expectations regarding the (a) contextualization of the research question within existing literature, (b) organization of the paper, and (c) writing style and conventions. Twenty five percent met and 75% exceeded expectations for revision of the papers based on faculty feedback.
  4. Regarding the dissertation method section, 25% met and 75% exceeded expectations for the research design. All of the candidates exceeded expectations regarding the description of the research participants. Fifty percent met and 50% exceeded expectations for the description of the procedures and for the data analysis. All of the candidates exceeded expectations for revision of the paper based on faculty feedback.
  5. Regarding the research presentation, 50% of the candidates met and 50% exceeded expectations for the summary of the research and for the clarity and organization of the presentation. Twenty five percent met and 75% exceeded expectations regarding the speaker’s attention to the audience, including avoidance or jargon. Twenty five percent met and 75% exceeded expectations regarding use of appropriate visual aids. Twenty five percent met and 75% exceeded expectations regarding the presentation time limits.
  6. Regarding the humans subjects applications, all of students received approval from the Committee on Human Studies.

11) Briefly Describe the Distribution and Discussion of Results

The Department faculty received the results and discussed the findings in a series of faculty meetings. The results were also presented in our Department’s report to the National Council for Accreditation of Teacher Education (NCATE) and disseminated on our College’s internal website.

12) Describe Conclusions and Discoveries

  1. The faculty reviewed the results of the two assessments of candidates’ knowledge. These assessments focused on the literature reviews included in the candidates’ proposals and dissertations. The data suggested that in general, candidates met or exceeded expectations for all components of the proposal and dissertation literature reviews. There was only one candidate who did not meet expectations for one component of the literature review, the contextualization of the research question through critique and synthesis of the existing literature.
  2. The faculty reviewed the results of the three assessments of candidates’ skills. Two of these assessments focused on the method sections included in the candidates’ proposals and dissertations. The third assessment focused on students’ skill in making effective research presentations. The data suggested that all candidates met or exceeded expectations for their proposal and dissertation method sections.A relative strength of the methods sections was the candidates’ clear and thorough description of the participants of their studies. Although all candidates met expectations for the description of how their data would be analyzed to answer their research questions, fewer candidates exceeded expectations of this component in their proposals, compared to other components.The third assessment of candidates’ skills focused on students’ dissertation presentations. The data indicated that all candidates met or exceeded the expectations for all components of the assessment.
  1. Students’ modeling the ethical treatment of research participants was assessed by a review of their human studies applications. The data from this assessment suggested that all of the candidates were able to design research that was consistent with ethical values and principles involving the treatment of human participants.

13) Use of Results/Program Modifications: State How the Program Used the Results --or-- Explain Planned Use of Results

  1. The data suggested that in general, candidates met or exceeded expectations for all components of the proposal and dissertation literature reviews. The faculty will continue to provide the type and amount of assistance they have given to students in this area, both in advising and coursework. 
  1. Although all candidates met expectations for the description of how their data would be analyzed to answer their research questions, fewer candidates exceeded expectations of this component in their proposals, compared to other components. After discussing these findings, the faculty decided to emphasize data analysis in their advising. The faculty who teach EDEP 608: Introduction to Educational Research, the required course that focuses on the development of a research proposal, agreed to provide additional instruction on developing a plan for data analysis.
  1. Regarding research presentations and human studies applications, the faculty will continue to provide the type and amount of assistance they have given to students, both in advising and coursework.

14) Reflect on the Assessment Process

We decided to provide clearer instructions to students on the assessment processes. More detailed instructions are now published on our Department website and provided to students through orientations and advising sessions.

15) Other Important Information

16) FOR DISTANCE PROGRAMS ONLY: Explain how your program/department has adapted its assessment of student learning in the on-campus program to assess student learning in the distance education program.

17) FOR DISTANCE PROGRAMS ONLY: Summarize the actual student learning assessment results that compare the achievement of students in the on-campus program to students in the distance education program.