Degree Program Assessment Reports : View Reports

Unit: Education (multiple departments)
Program: Professional Educational Practice (EdD)
Degree: Doctorate
Date: Tue Nov 08, 2011 - 12:40:58 pm

1) Below are your program student learning outcomes (SLOs). Please update as needed.

Outcome One: Leaders in professional educational practice work collaboratively to solve problems and implement plans of action
Outcome Two: Leaders in professional educational practice are able to apply research skills to bring about improvements in practice.
Outcome Three: Leaders in professional educational practice can reflect critically and ethically on matters of educational importance.
Outcome Four: Leaders in professional educational practice are able to take a broad, interdisciplinary perspective on a wide variety of educational issues

2) Your program's SLOs are published as follows. Please update as needed.

Department Website URL:

Student Handbook. URL, if available online:

Information Sheet, Flyer, or Brochure URL, if available online:

UHM Catalog. Page Number:

Course Syllabi. URL, if available online:

Other:

3) Below is the link(s) to your program's curriculum map(s). If we do not have your curriculum map, please upload it as a PDF.

Curriculum Map File(s) from 2011:

File (03/16/2020)

4) For your program, the percentage of courses that have course SLOs explicitly stated on the syllabus, a website, or other publicly available document is as follows. Please update as needed.

1-50%

51-80%

81-99%

100%

5) For the period June 1, 2010 to September 30, 2011: State the assessment question(s) and/or assessment goals. Include the SLOs that were targeted, if applicable.

This is a new program. Cohort I started in Fall 2011 so no assessment has been conducted. The following is the assessment plan.

Assessment Process and Timeline

Cohort I

December 2012. Assessment One will be conducted at the end of the semester in which students complete the Group Consultancy Project. Student-level assessments will be conducted on each of four program outcomes detailed in Rubric A (minimal, acceptable, and target). Data will be added to an online template on the College Portal that can be accessed by faculty, and summary information generated. Aggregated data will be reviewed and analyzed by faculty and used to make decisions about program improvements.
November/December 2013. Assessment Two will be completed in the second year of the program when students have prepared their proposals for conducting an individual research project. Student-level assessments will be conducted on each of four program outcomes detailed in Rubric B (minimal, acceptable, and target). Data will be added to an online template on the College Portal that can be accessed by faculty, and summary information generated. Aggregated data will be reviewed and analyzed by faculty and used to make decisions about program improvements.
July 2014. Assessment Three will be completed towards the end of the program once students have submitted their Individual Practitioner Research Project (capstone project) and made their presentations at the concluding Program Conference. Student-level assessments will be conducted on each of four program outcomes detailed in Rubric D(minimal, acceptable, and target). Data will be added to an online template on the College Portal that can be accessed by faculty, and summary information generated. Aggregated data will be reviewed and analyzed by faculty and used to make decisions about program improvements.
July 2014. A Student Self-Assessment will also be conducted at the end of the program. This self-assessment will provided students with an opportunity to reflect on their progress in reaching the four program goals.

Additional Assessments

Student evaluations of courses using CAFE system.
Evaluation of the value of the consultancy project for clients
Evaluation of the practical results of the individual research projects (capstone project)
Follow up evaluation to collect evidence of career advancement among graduates.

Transition Point Assessment for the EdD

The following table shows the transition assessment points consistent with the requirements for NCATE accreditation of advanced programs. There are three key assessments:

Assessment One: Faculty assessment of individual contributions to the group consultancy project at the end of the first year of the program. See Rubric A.
Assessment Two: Faculty assessment of the individual practitioner research proposal at the end of the second year of the program. See Rubric B.
Assessment Three: Committee assessment of individual practitioner research project at program completion (after submission of project report and conference presentation). See Rubric D.

Entry	Year One Assessment of Group Consultancy Project	Year Two Assessment of independent practitioner research proposal	Program Completion Assessment of completed practitioner research project
Doctoral Degree (EdD) in Professional Leadership in Education	Graduate school application Faculty assessment of EdD program applications
Knowledge Knowledge of research methods. Interdisciplinary knowledge of a variety of educational issues.		Assessment of literature review and contextual analysis.	Assessment of literature review
Skills Skill in applying research to achieve improvements in practice. Skill in applying problem solving strategies to practical educational problems.			Assessment of plan of action	Assessment of practitioner research project
Dispositions Ability to work collaboratively to solve problems. Ability to implement a plan of action. Capacity to reflect critically and ethically on matters of educational importance		Faculty assessment of group process. Assessment of ethical implications and individual reflections.		Assessment of project results

Formative Assessments of Program Effectiveness

A Program Advisory Committee will be established in the first semester of the program (Fall 2011). The advisory committee will be composed of two faculty, two field advisors, two students, and the program chair, who will also act as the PAC chair. This committee will discuss issues regarding the operation of the program and, as such, will provide important feedback that can be used in the formative evaluation of the program. The advisory committee will meet at least twice a semester to review data and develop plans to implement program improvements and to make revisions to and adapt assessment procedures.

Other assessments will be obtained from course evaluations and a concluding program survey that will be conducted during the program conference after the candidates have presented their projects. All courses taught during the summer will be assessed using CAFE. Student evaluations of field projects and seminars will also be conducted at the end of fall and spring semesters.

The Student Experience

Program policy calls for inclusion of outcomes in all course syllabi. Faculty will familiarize students the rubrics used for evaluation of student outcomes and copies will be included in the syllabi of the consultancy project course (EDUC 710) and for the individual research project (EDUC 720). At the completion of the program students will complete a self-assessment (concluding survey) of their achievement in relation to each of the program outcomes.

Assessment One: Assessment of Group Consultancy Project (EDUC 710)

Description of Project

In this project individuals are organized in research teams to explore problems of practice submitted by external state agencies such as school districts, post-secondary institutions, philanthropic organizations, and so on. The submissions, arising from “Requests for Assistance” (RFAs), will be screened for applicability and a final set prepared for the “consultancy” teams. The aim is for the each group to provide a contextual analysis of their assigned problem, to research the problem, to conduct data analysis (financial, operational, evaluative and demographic, as the case may require), to consider ethical implications, to provide program recommendations, and to offer strategies for implementation.

If a student fails to achieve “acceptable” in any one or more of the requirements, they will be asked to revise their work and resubmit. If, after resubmitting their work, they fail to achieve above minimal in any or all of the requirements, they may be offered the opportunity of an extended period of one semester to complete their work satisfactorily or of joining a later cohort. If after an extended period of one semester, a student fails to attain “acceptable” on all requirements, they will be dropped from the program.

Rubric A

Requirement	Minimal	Acceptable	Target
Quality of Writing	The report is poorly written, unorganized and contains spelling, punctuation, and grammatical errors.	The quality of writing is acceptable. The thesis is coherent and contains a few spelling, punctuation, and grammatical errors.	The quality of writing is above average. As a whole it is well organized, shows logical consistency, and is free of spelling, punctuation and grammatical errors.
Knowledge of research methods	The methodology section is disorganized and the methods selected are inappropriate for the project to be researched.	The methodology section is adequately organized and the methods are sufficient to provide an acceptable level of support for the thesis.	The methodology section is well organized. The methods chosen are appropriate for the study and have been implemented judiciously.
Ability to work collaboratively in solving problems	Individual contribution to the work of the group was inadequate.	Individual contribution to the work of the group was sufficient to	Individual made important contributions to the work of the group.
Capacity to reflect critically and ethically on matters of educational importance	The report failed to show evidence of critical reflection on ethical issues raised by the project.	Evidence of critical engagement with ethical and other issues.	Evidence that important ethical issues were raised and that problem solving occurred.

Assessment Two: Assessment of Individual Practitioner Research Proposal (EDUC 720)

Description of Project

Students will formulate an individual inquiry project that arises in the context of their own practice as professional educators. These projects will be discussed in conference with an interim advisor along with a professional mentor. The object of this work is to enable EdD candidates to demonstrate their analytical skills, research ability, professional knowledge, and understanding of context and culture in which the problem is embedded. In addition, it is a chance to apply their skills by implementing a well researched plan of action that is directed to an improvement in practice.

Rubric B

Requirement	Minimal	Acceptable	Target
Quality of writing	The proposal is poorly written, unorganized and contains spelling, punctuation, and grammatical errors.	The quality of writing is acceptable. The proposal is coherent and contain only a few spelling, punctuation, and grammatical errors.	The quality of writing is above average. As a whole the proposal is well organized, shows logical consistency, and is free of spelling, punctuation and grammatical errors.
Review of the literature	Number of sources insufficient. Review is poorly related to the project	The review is based on a sufficient number of sources and the relationship between the literature and proposed research is adequate.	The review section contains a synthesis of the material and provides a clear statement of the candidates position with respect to the literature.
Methods	The methods proposed in conducting the research are inadequate. Research questions are too broad and the proposed methods of data collection are unclear or vaguely expressed	The methods proposed in conducting the research are sufficient. Some further efforts should be made to connect data collection procedures to research questions.	Methods are clearly stated and appropriate to the research questions. Data collection procedures are well thought out and methods of analysis proposed.
Background issues	Inadequate discussion. Shows limited understanding of context.	The proposal shows an understanding of the social and historical background within which the problem arises.	The proposal provides a thorough analysis of the social historical context of the problem

Assessment Three: Assessment of Individual Practitioner Research Paper (EDUC 720)

Description of Project

A final report (approximately 70–100 pages) will be submitted at the end of the third spring semester and prior to the final conference at which a summary of the findings of the applied research project will be made. The report will include

Rubric C

Requirement	Minimal	Acceptable	Target
Quality of writing	The report is poorly written, unorganized and contains spelling, punctuation, and grammatical errors.	The quality of writing is acceptable. The report is coherent and contains only a few spelling, punctuation, and grammatical errors.	The report is well written. It conforms to APA style throughout. As a whole it is well organized, shows logical consistency, and is free of spelling, punctuation and grammatical errors.
Research Questions	Research question is poorly defined and inappropriately in relation to the action project. Poorly aligned with literature review.	Research questions are clearly articulated, but the relationship between questions, data collection, and analysis are unclear.	The project was clearly articulated and the research questions concisely stated.
Action Plan	Action plan was not well formulated and poorly implemented. Lack of data.	Action plan was carried out and actions taken were sufficient to answer research questions and support conclusions.	The action plan was well thought out, thoroughly organized, and effectively implemented with attention to research questions
Data Collection	Data sources are limited, insufficient information to support aims of project. Inadequate information about data collection procedures.	Data sources are appropriate and data collection methods sufficiently described	Appropriate methods were followed and the data collected provided valuable project information.
Data Analysis	Methods of analysis are not explained. Failure to include all data collected. Poor interpretation of data	Methods of analysis are appropriate, though other methods may have been employed to generate conclusions.	Analysis of data was insightful and provided useful implications for practice.
Findings	Findings are presented in an unorganized way. Little interpretation of data, and/or conclusions presented that are unrelated to data.	Findings are presented that connect with the data, but are incomplete.	Findings are well organized and consistent with research questions and data.

EDUC 730: Assessment of Program Outcomes

The following rubric will be used by the advisors to make a final and summative evaluation of the students work in the program based on their contributions to the consultancy project, their action research project, and final presentation.

Rubric D

Requirement	Minimal	Acceptable	Target
Collaborate to solve problems and implement action plans.
Apply research skills to bring about improvements in practice.
Reflect critically and ethically on matters of educational importance.
Take a broad, interdisciplinary perspective on a wide variety of educational issues.

CAFE

Student course evaluation items for courses taught in the EdD program

The following items constitute the EdD set and are to be used in all EdD Courses.

This is a minimum set; faculty members are free to add up to 10 more questions.

Multiple Choice Items

(Five part scale from strongly disagree to agree)

02 I gained a good understanding of concepts/principles in this field.

05 I developed the ability to solve real problems in this field.

018 I participated actively in class discussion.

117 The instructor was available for consultation with students.

124 The instructor uses class time well.

126 The objectives of the course were clearly explained.

186 Reading assignments are interesting and stimulating.

215 The instructional materials (e.g., tests, handouts, etc.) were relevant to course objectives.

218 I feel that this course challenged me intellectually.

286 The instructor was fair in grading and criteria of grades.

Open-Ended Items

304 What did you find most valuable and helpful about the course?

305 What did you find least valuable and helpful about the course?

311 Please list your suggestions for improving the course and/or the instructor's teaching style and methods.

333 My overall evaluation of this course is...

6) State the type(s) of evidence gathered to answer the assessment question and/or meet the assessment goals that were given in Question #5.

7) State how many persons submitted evidence that was evaluated. If applicable, please include the sampling technique used.

8) Who interpreted or analyzed the evidence that was collected? (Check all that apply.)

Course instructor(s)

Faculty committee

Ad hoc faculty group

Department chairperson

Persons or organization outside the university

Faculty advisor

Advisors (in student support services)

Students (graduate or undergraduate)

Dean/Director

Other:

9) How did they evaluate, analyze, or interpret the evidence? (Check all that apply.)

Used a rubric or scoring guide

Scored exams/tests/quizzes

Used professional judgment (no rubric or scoring guide used)

Compiled survey results

Used qualitative methods on interview, focus group, open-ended response data

External organization/person analyzed data (e.g., external organization administered and scored the nursing licensing exam)

Other:

10) For the assessment question(s) and/or assessment goal(s) stated in Question #5:
Summarize the actual results.

11) State how the program used the results or plans to use the results. Please be specific.

12) Beyond the results, were there additional conclusions or discoveries?
This can include insights about assessment procedures, teaching and learning, program aspects and so on.

13) Other important information.
Please note: If the program did not engage in assessment, please explain. If the program created an assessment plan for next year, please give an overview.

1) Below are your program student learning outcomes (SLOs). Please update as needed.

2) Your program's SLOs are published as follows. Please update as needed.

3) Below is the link(s) to your program's curriculum map(s). If we do not have your curriculum map, please upload it as a PDF.

4) For your program, the percentage of courses that have course SLOs explicitly stated on the syllabus, a website, or other publicly available document is as follows. Please update as needed.

5) For the period June 1, 2010 to September 30, 2011: State the assessment question(s) and/or assessment goals. Include the SLOs that were targeted, if applicable.

Entry

Year One

Assessment of Group Consultancy Project

Year Two

Assessment of independent practitioner research proposal

Program Completion

Assessment of completed practitioner research project

Requirement

Minimal

Acceptable

Target

6) State the type(s) of evidence gathered to answer the assessment question and/or meet the assessment goals that were given in Question #5.

7) State how many persons submitted evidence that was evaluated. If applicable, please include the sampling technique used.

8) Who interpreted or analyzed the evidence that was collected? (Check all that apply.)

9) How did they evaluate, analyze, or interpret the evidence? (Check all that apply.)

10) For the assessment question(s) and/or assessment goal(s) stated in Question #5:Summarize the actual results.

11) State how the program used the results or plans to use the results. Please be specific.

12) Beyond the results, were there additional conclusions or discoveries? This can include insights about assessment procedures, teaching and learning, program aspects and so on.

13) Other important information. Please note: If the program did not engage in assessment, please explain. If the program created an assessment plan for next year, please give an overview.

10) For the assessment question(s) and/or assessment goal(s) stated in Question #5:
Summarize the actual results.

12) Beyond the results, were there additional conclusions or discoveries?
This can include insights about assessment procedures, teaching and learning, program aspects and so on.

13) Other important information.
Please note: If the program did not engage in assessment, please explain. If the program created an assessment plan for next year, please give an overview.