Unit: Educational Psychology
Program: Educational Psychology (MEd)
Degree: Master's
Date: Sun Sep 27, 2009 - 6:26:55 am

1) List your program's student learning outcomes (SLOs).

  1. Educational Psychology graduate students are knowledgeable about learning and development, inquiry methods, and student assessment.
  2. Educational Psychology graduate students have inquiry skills to conduct scholarly research effectively.
  3. Educational Psychology graduate students present scholarly research effectively.
  4. Educational Psychology graduate students model the ethical treatment of research participants.

2) Where are your program's SLOs published?

Department Website URL: http://www.coe.hawaii.edu/edep
Student Handbook. URL, if available online:
Information Sheet, Flyer, or Brochure URL, if available online: NA
UHM Catalog. Page Number:
Course Syllabi. URL, if available online:
Other:
Other:

3) Upload your program's current curriculum map(s) as a PDF.

Curriculum Map File(s) from 2009:

4) What percentage of courses have the course SLOs explicitly stated on the course syllabus, department website, or other publicly available document? (Check one)

0%
1-50%
51-80%
81-99%
100%

5) State the SLO(s) that was Assessed, Targeted, or Studied

  1. Educational Psychology graduate students are learning and development, inquiry methods, and student assessment.
  2. Educational Psychology graduate students have inquiry skills to conduct scholarly research effectively.
  3. Educational Psychology graduate students present scholarly research effectively.
  4. Educational Psychology graduate students model the ethical treatment of research participants.

6) State the Assessment Question(s) and/or Goal(s) of Assessment Activity

  1. We wanted to know whether the students had the requisite knowledge of learning and development, inquiry methods, and student assessment needed to (a) develop an appropriate research question and (b) contextualize their research question within existing research.
  2. We wanted to know whether students had the inquiry skills needed to (a) develop a research design that was a good fit with their research questions, (b) describe their methods clearly, such that others could replicate them, and (c) plan for and implement data analyses that would answer their research questions.
  3. We wanted to know whether students had the skills needed to make an effective research presentation.
  4. We wanted to know whether students modeled the ethical treatment of research participants by designing and conducting research that was consistent with the standards set forth by the University’s Committee on Human Studies.

7) State the Type(s) of Evidence Gathered

  1. The students’ advisors rated their students’ Plan B or thesis proposals and final paper literature reviews using a rubric developed by the Department faculty.
  2. The students’ advisors rated their students’ Plan B or thesis proposals and final paper method sections using a rubric developed by the Department faculty.
  3. The students’ advisors rated their students’ Plan B or thesis research presentations using a rubric developed by the Department faculty.
  4. Students who conducted research involving human participants for their Plan B or thesis research were required to submit an application to the Committee on Human Studies that described their use of human participants, the potential risks involved, and how they would guard against such risks. We collected information on whether these applications were approved by the Committee on Humans Studies.

8) State How the Evidence was Interpreted, Evaluated, or Analyzed

The Department staff analyzed the percentage of students who met, did not meet, or exceeded expectations on each component of each of the five rubrics (Proposal Literature Review, Plan B or Thesis Literature Review, Proposal Method Section, Plan B or Thesis Method Section, and Research Presentation). The students’ advisors informed the Department staff about whether students’ human studies applications were approved.

The Department faculty met to discuss these results and to determine areas or relative strength and weakness.

9) State How Many Pieces of Evidence Were Collected

We collected scores for all Plan B and thesis proposals, Plan B or thesis final papers, and students’ Plan B or thesis research presentations. We also collected information on whether all Plan B or thesis human studies applications had received approval.

 Our data included scores for eight Plan B or thesis proposals, nine Plan B or thesis final papers, and nine research presentations. We also received information regarding whether eight human studies applications received approval.

10) Summarize the Actual Results

  1. Regarding the proposal literature review, 87% of the candidates met and 13% exceeded expectations regarding the statement of the research question. Fifty percent met and 50% did not meet expectations regarding the contextualization of the research question within existing literature. Eighty seven percent met and 13% did not meet expectations for the organization of their proposals. Sixty two percent met and 38% exceeded expectations for writing style and conventions.
  2. Regarding the proposal method section, 74% met, 13% exceeded, and 13% did not meet expectations for the research design. Seventy five percent met and 25% exceeded expectations regarding the description of the research participants. Eighty seven percent met and 13% did not meet expectations for the description of the procedures. Seventy four percent met and 13% did not meet expectations for the data analysis.
  3. Regarding the Plan B or thesis literature review, 33% of the met and 67% exceeded expectations regarding the statement of the research question. Fifty six percent met, 33% exceeded, and 11% did not meet expectations regarding the contextualization of the research question within existing literature. Forty three percent met and 57% exceeded expectations for the organization of the papers. Sixty seven percent met and 33% exceeded expectations for writing style and conventions. Twenty two percent met and 78% exceeded expectations for revision of the papers based on faculty feedback.
  4. Regarding the Plan B or thesis method section, 56% met and 44% exceeded expectations for the research design. Thirty three percent met and 67% exceeded expectations regarding the description of the research participants. Forty four percent met and 56% exceeded expectations for the description of the procedures. Fifty six percent met and 44% exceeded expectations of the data analysis. Twenty two percent met and 78% exceeded expectations for revision of the paper based on faculty feedback.
  5. Regarding the research presentation, 44% of the candidates met and 56% exceeded expectations for the summary of the research. Forty four percent met and 56% exceeded expectations for the clarity and organization of the presentation. Twenty two percent met and 78% exceeded expectations regarding the speaker’s attention to the audience, including avoidance or jargon. Twenty two percent met, 67% exceeded, and 11% did not meet expectations regarding use of appropriate visual aids. All of the students exceeded expectations regarding the presentation time limits.
  6. Regarding the humans subjects applications, 100% of students, who were required to submit an application, received approval from the Committee on Human Studies.

11) Briefly Describe the Distribution and Discussion of Results

The Department faculty received the results and discussed the findings in a series of faculty meetings. The results were also presented in our Department’s report to the National Council for Accreditation of Teacher Education (NCATE) and disseminated on our College’s internal website.

12) Describe Conclusions and Discoveries

  1. The faculty reviewed the results of the two assessments of candidates’ knowledge. These assessments focused on the literature reviews included in the candidate’s Plan B or thesis proposals and final research papers. The data from both of these assessments suggested that compared to other components, fewer candidates exceeded expectations for contextualization of their research question through critique and synthesis of the existing literature.
  2. The faculty reviewed the results of the three assessments of candidates’ skills. Two of these assessments focused on the method sections included in the candidates’ Plan B or thesis proposals and final research papers. The data from these assessments suggested that compared to other components, an area for improvement was the descriptions of students’ data analyses. The data on the research presentations suggested that in general, candidates were successful in all components of the research presentation.
  3. Students’ modeling the ethical treatment of research participants is assessed by a review of their human studies applications. The data from this assessment suggested that all of the candidates, for whom this assessment was applicable, were able to design research that was consistent with ethical values and principles involving the treatment of human participants.

13) Use of Results/Program Modifications: State How the Program Used the Results --or-- Explain Planned Use of Results

1. Compared to other components of the proposal and final paper literature reviews, fewer candidates exceeded expectations for contextualization of their research question through critique and synthesis of the existing literature. To address this, the faculty will emphasize this aspect of the proposal in their advising of master’s students. In a required course, EDEP 611: Inquiry Approach to Educational Psychology, a major assignment is the development of a literature review. The faculty who teach EDEP 611 agreed to also emphasize the contextualization of the research question and to provide additional assistance in this area.

2. In addressing these findings related to students’ proposal and final paper method sections, the faculty felt that it would beneficial for students to enroll in additional coursework that focused on the specific type of data analysis that students would use for their research. As early as possible, advisors will encourage their advisees to identify whether they will be using mainly quantitative or qualitative methods. Faculty will advise students who will be using quantitative methods to enroll in an additional quantitative methods course (e.g., EDEP 601: Introduction to Quantitative Methods or EDEP 604: Multiple Regression). Those who will be using primarily qualitative methods will be encouraged to enroll in a qualitative methods course offered in either our or another department.

3. Regarding research presentations and human studies applications, the faculty will continue to provide the type and amount of assistance they have given to students, both in advising and coursework.

14) Reflect on the Assessment Process

We decided to provide clearer instructions to students on the assessment processes. More detailed instructions are now published on our Department website and provided to students through orientations and advising sessions.

15) Other Important Information

None.

16) FOR DISTANCE PROGRAMS ONLY: Explain how your program/department has adapted its assessment of student learning in the on-campus program to assess student learning in the distance education program.

17) FOR DISTANCE PROGRAMS ONLY: Summarize the actual student learning assessment results that compare the achievement of students in the on-campus program to students in the distance education program.