Program: Educational Psychology (MEd)
Degree: Master's
Date: Fri Oct 26, 2012 - 6:30:31 pm
1) Below are your program's student learning outcomes (SLOs). Please update as needed.
- Educational Psychology graduate students are knowledgeable about learning and development, inquiry methods, and student assessment.
- Educational Psychology graduate students have inquiry skills to conduct scholarly research effectively.
- Educational Psychology graduate students present scholarly research effectively.
- Educational Psychology graduate students model the ethical treatment of research participants.
2) Your program's SLOs are published as follows. Please update as needed.
Department Website URL: http://www.coe.hawaii.edu/edep
Student Handbook. URL, if available online: Information Sheet, Flyer, or Brochure URL, if available online: UHM Catalog. Page Number: Course Syllabi. URL, if available online: Other: Other: 3) Select one option:
- File (03/16/2020)
4) For your program, the percentage of courses that have course SLOs explicitly stated on the syllabus, a website, or other publicly available document is as follows. Please update as needed.
0% 1-50% 51-80% 81-99%
100%5) Did your program engage in any program assessment activities between June 1, 2011 and September 30, 2012? (e.g., establishing/revising outcomes, aligning the curriculum to outcomes, collecting evidence, interpreting evidence, using results, revising the assessment plan, creating surveys or tests, etc.)
Yes No (skip to question 14)6) For the period June 1, 2011 to September 30, 2012: State the assessment question(s) and/or assessment goals. Include the SLOs that were targeted, if applicable.
The program faculty wanted to know whether candidates:
1. Were knowledgeable about learning and development, inquiry methods, and student assessment (SLO 1).
2. Had the inquiry skills to conduct scholarly research effectively (SLO 2).
3. Could present scholarly research effectively (SLO 3).
4. Modeled the ethical treatment of research participants (SLO 4).
7) State the type(s) of evidence gathered to answer the assessment question and/or meet the assessment goals that were given in Question #6.
We collected candidates’ proposals and final MEd papers (Plan B or thesis) and analyzed the literature reviews and methods sections of those documents. We also collected documentation on whether or not students’ research had been approved by the UH Committee on Human Studies.
8) State how many persons submitted evidence that was evaluated. If applicable, please include the sampling technique used.
There were eleven candidates for whom evidence was evaluated. These were all candidates who completed proposals and/or final master's papers (Plan B or thesis).
9) Who interpreted or analyzed the evidence that was collected? (Check all that apply.)
Course instructor(s) Faculty committee Ad hoc faculty group Department chairperson Persons or organization outside the university Faculty advisor Advisors (in student support services) Students (graduate or undergraduate) Dean/Director Other: 10) How did they evaluate, analyze, or interpret the evidence? (Check all that apply.)
Used a rubric or scoring guide Scored exams/tests/quizzes Used professional judgment (no rubric or scoring guide used) Compiled survey results Used qualitative methods on interview, focus group, open-ended response data External organization/person analyzed data (e.g., external organization administered and scored the nursing licensing exam) Other: 11) For the assessment question(s) and/or assessment goal(s) stated in Question #6:
Summarize the actual results.
We wanted to know whether candidates were knowledgeable about learning and development, inquiry methods, and student assessment (SLO 1).
Six students completed their master’s Plan B or thesis proposals. Most of the students were rated as exemplary (60% or ratings) for their literature review, as opposed to satisfactory. The weakest components of their proposal literature reviews appeared to be in the areas of organization (67% of students rated as satisfactory) and writing style (83% of students rated as satisfactory.
Nine students completed their master’s Plan B or thesis final papers. The majority (80%) of the ratings of the five components of the final literature review were exemplary (36 of the 45 ratings). The weakest components were in the area of writing style, in which six of the nine students (67%) were rated as satisfactory rather than exemplary. The strongest components were statement of the problem, in which 100% were rated as exemplary. Of the nine students, 89% were rate as exemplary for contextualizing their research in the broader scholarly context and 100% were rated as exemplary for incorporating faculty feedback into their final literature review.
We wanted to know whether candidates had inquiry skills to conduct scholarly research effectively (SLO 2).
For the six candidates who wrote proposals, the majority (67%) of the 24 ratings was satisfactory rather than exemplary. The strongest area of the proposal method sections was the description of the participants, with 67% of students rated as exemplary. The weakest component was the research design, with 100% of students rated as satisfactory.
For the final method section, 80% of the 45 ratings was exemplary, as opposed to satisfactory. The weakest areas of the final method sections appeared to be in the areas of research design, data analysis, and revision with 56% of students rated as satisfactory in these areas, as opposed to exemplary.
We wanted to know whether candidates could present scholarly research effectively (SLO 3).
Eight students made their master’s presentations. The students were rated as exemplary on most components, as 90% of the 40 ratings were exemplary.
We wanted to know if candidates modeled the ethical treatment of research participants (SLO 4).
All five students (100%) who submitted human subjects applications received approval from the Committee on Human Studies. One person did not have to apply for human subject’s review.
12) State how the program used the results or plans to use the results. Please be specific.
The faculty will continue to assist students in academic writing in their courses and also by suggesting resources, such as the Writing Workshop and joining a writing group. As we have an increasing number of second language learners in our Department, the faculty needs to think about how to guide students to use the various resources available. For example, students are sometimes encouraged to use an editor, however faculty and students need to be aware that editors are not to do extensive revising. The faculty is considering adopting a policy around the use of professional editors.
One master’s student whose proposal was particularly problematic was a student who insisted on completing a Plan A project, although faculty members felt the student was more suited to a Plan B project. Whereas students had been told that they could choose to do a Plan A or B project, the faculty decided that students would have to find a faculty member would be willing to supervise a Plan A, if that is what they wanted to do. If they could not find someone who would supervise a Plan A project, then students would have to complete a Plan B. This would assure that students would only attempt the Plan A, if faculty felt they be successful.
The faculty had already made a change in advising to recommend that students take an additional research methods class (either statistics or qualitative analysis). Students who completed their papers this year did not receive this advising, but we anticipate the future students who take an additional class will be better prepared for their research design and data analysis.
Faculty members are pleased that students appear very prepared to make formal research presentations.
13) Beyond the results, were there additional conclusions or discoveries?
This can include insights about assessment procedures, teaching and learning, program aspects and so on.
The faculty felt that it would be better to design a rubric that better fits the Plan B proposals.
The faculty felt that because not all students are required to complete a human subjects application on their own, a better assessment of their understanding of the ethical treatment of human subjects can be completion of the CITI basic/ refresher online course.
The faculty felt that a rubric that has more levels will better capture variability in students’ performances and will consider revising our tools to include more levels.