Degree Program Assessment Reports : View Reports

Unit: Curriculum Studies
Program: Early Childhood Education (MEd)
Degree: Master's
Date: Mon Nov 16, 2015 - 11:44:42 am

1) Below are your program's student learning outcomes (SLOs). Please update as needed.

MEd Early Childhood Education Program SLOs

The MEd in Early Childhood Education (ECE) program includes a unique set of program assessments designed to measure student competency in meeting Five Core Program Standards. These assessments are based on the unit’s Conceptual Framework. They include measures of each candidate’s knowledge about child development and the field of early childhood education, ability to contribute to leadership in the profession as an effective early childhood educator, and disposition as a caring and ethical professional. These three foci form the conceptual framework of the College of Education.

The conceptual framework of the COE provides broad direction and focus for the program design. The MEd in ECE provides a narrower lens through which to interpret and manifest the conceptual framework. The two are directly linked through the mission, goals and objectives of the MEd in ECE program. They are linked to Program Standards, Student Learning Outcomes (SLOs), and Key Assessments.

The mission of the MEd in ECE is to develop depth of knowledge, collaboration skills, and the disposition to engage in leadership activities and advocacy in the field of early childhood education. Students develop the capacity to work collaboratively to design and implement high quality inclusive programs for young children and their families.

The goal of the MEd in ECE program is to provide students with a conceptual framework, skills, and knowledge that will enhance their effectiveness as early childhood educators. It is designed to develop master level competence related to five core program standards and additional program standards relevant to their interests. Student learning outcomes are embedded in each of the required courses and the Plan B capstone.

The five core program standards are embedded in mandatory coursework. SLOs are articulated below by standard:

Standard I: Child Development

MEd ECE graduates are knowledgeable about the developmental needs of young children from the prenatal period to eight years of age. As professionals who care about children achieving their maximum potential, they use that knowledge to effectively create programs that support children’s optimal development and to effectively develop strategies for families in an ethical and culturally sensitive manner.

SLOS for Standard 1

1.1 Students analyze, synthesize and utilize research based knowledge of: 1) the unique individual nature of early childhood development and the role of maturation, protection, and experience in the developmental domains, 2) the interactions between maturation and experience, 3) the interrelationships between domains; and, 4) the contexts of development.

1.2 Students apply knowledge of child development by contributing to improving the quality of programs so that they better support each child's growth and learning.

1.3 Students synthesize the research focused on a topic that impacts child well-being and development and can competently disseminate this information to policymakers, parents or practitioners in the field in one or more formats (e.g. a conference presentation, poster, web page, a brief or testimony for policymakers).

Standard II: The Field of Early Childhood Education and Care

MEd ECE graduates are knowledgeable about current issues and trends in early childhood care and education. As professionals who care about the larger needs of the community, they use that knowledge to effectively provide ethical and culturally sensitive leadership and advocacy with regard to policy and decision-making within their own programs and in government organizations.

SLOS for Standard 2

2.1 Students examine the professional literature on a broad base of current issues or trends in ECEC (e.g. governance and system building in the field, the form and purpose of programs and services for diverse young children and families, indicators of quality, credentials for the early childhood workforce, professional development, curriculum content and signature pedagogy, assessment and program evaluation, leadership).

2.2 Students collect or interpret information related to larger guiding influences that shape the profession (e.g., policies, initiatives and regulations from state or national agencies and professional organizations) and reflect a nuanced understanding of the impact of initiatives on practitioners and leaders from the perspective of diverse clients and stakeholders in a community.

2.3 Students synthesize the research on a current issue or trend and can competently disseminate this information to policymakers, parents or practitioners in the field in one or more formats (e.g. conference presentation, poster, web page, brief or testimony to policymakers).

2.4 Students analyze present policies, practices and programs that impact families with young children. They meaningfully engage in advocacy activities that support research based knowledge of effective ECEC programs and practices.

Standard III: Early Childhood Special Education

MEd ECE graduates are knowledgeable about children and families with special needs. As professionals who care about equity for all children and families, they are able to effectively use their knowledge to develop inclusive educational programs to meet individual and group needs in an ethical, caring, and culturally inclusive manner.

SLOS for Standard 3

3.1 Students review the literature on: 1) evolving trends in special education, 2) recommended practices regarding the needs of families with infants and young children with disabilities, 3) characteristics of infants and young children with disabilities, 4) legislative mandates for young children with special needs, 5) culturally inclusive assessment processes and procedures, 6) effective implementation of trends in the design and implementation of intervention and instruction in inclusive settings.

3.2 Students describe legislation that affects young children with special needs/disabilities and the services and programs in Hawai'i that result from federal legislation.

3.3 Students develop and implement an appropriate parent or professional development opportunity that focuses on one or more topics related to inclusion of children with special needs OR a curriculum modification for children with special needs in their care based on their knowledge of young children with special needs and research based practices that support their development and learning.

3.4 Students demonstrate the disposition to reflect on the effectiveness of present programs for infants and young children with disabilities and contribute to the development of culturally sensitive, inclusive settings that meet the needs of each child with a disability and his or her family.

Standard IV: Professionalism

MEd ECE candidates are knowledgeable about what it means to be a professional in the field of early childhood education. As professionals who care about the field, they work effectively in collaboration with families and other professionals to provide services in an ethical, caring and culturally sensitive manner. Candidates identify with and conduct themselves as members of the early childhood profession. They know and use ethical guidelines and other professional standards related to early childhood practice.

SLOS for Standard 4

4.1 Students describe the ethical and professional responsibilities of early childhood educators and the role of the NAEYC Code of Ethical Conduct and other ethical codes in guiding professional practice.

4.2 Students effectively analyze an ethical dilemma and engage in a methodical process to resolve it using the NAEYC Code of Ethical Conduct to guide their decision-making.

4.3 Students demonstrate professional competencies and dispositions in their interactions with children, families, colleagues, and the general public. They create and implement a professional development plan to intentionally develop their leadership capacity based on their strengths and areas for growth.

4.4 Students intentionally demonstrate involvement and growth as a contributor in a professional community. They gradually increase the breadth and scope of their involvement in professional activities within and outside of their workplace.

Standard V: Research

MEd ECE graduates are knowledgeable about the role of research in the field of early childhood education. As professionals who care about using research-based strategies and methods, they effectively reflect on their current practices and initiate their own research projects. They critically analyze and apply current educational research to their own settings.

SLOS for Standard 5

5.1 Students describe: 1) characteristics of early childhood research paradigms and designs, 2) strategies and techniques for inquiry in early childhood education, 3) ethical and social implications of various decisions, research strategies and reports by researchers; and, 4) strengths and weaknesses of quantitative and qualitative research approaches.

5.2 Students gather, critique, and interpret early childhood research in their areas of interest.

5.3 Students compose a written review of the literature that analyzes and synthesizes existing professional literature on a topic of study using credible, current research studies.

5.4 Students design, implement, and disseminate the findings from an original research study that is consistent with the protocols for inquiry and evidence in their area of specialization and practice.

1) Institutional Learning Objectives (ILOs) and Program Student Learning Outcomes (SLOs)

2) Your program's SLOs are published as follows. Please update as needed.

Department Website URL: https://coe.hawaii.edu/academics/curriculum-studies/med-ece

Student Handbook. URL, if available online: https://coe.hawaii.edu/academics/curriculum-studies/med-cs-ece

Information Sheet, Flyer, or Brochure URL, if available online:

UHM Catalog. Page Number:

Course Syllabi. URL, if available online: NA

Other: NCATE Advanced Programs Report available on COE Wiki

Other: Program Standards Handout, Plan B Portfolio Literature

3) Please review, add, replace, or delete the existing curriculum map.

Curriculum Map File(s) from 2015:

File (03/16/2020)

4) For your program, the percentage of courses that have course SLOs explicitly stated on the syllabus, a website, or other publicly available document is as follows. Please update as needed.

1-50%

51-80%

81-99%

100%

5) Did your program engage in any program learning assessment activities between June 1, 2014 and September 30, 2015?

Yes

No (skip to question 16)

6) What best describes the program-level learning assessment activities that took place for the period June 1, 2014 to September 30, 2015? (Check all that apply.)

Create/modify/discuss program learning assessment procedures (e.g., SLOs, curriculum map, mechanism to collect student work, rubric, survey)

Collect/evaluate student work/performance to determine SLO achievement

Collect/analyze student self-reports of SLO achievement via surveys, interviews, or focus groups

Use assessment results to make programmatic decisions (e.g., change course content or pedagogy, design new course, hiring)

Investigate curriculum coherence. This includes investigating how well courses address the SLOs, course sequencing and adequacy, the effect of pre-requisites on learning achievement.

Investigate other pressing issue related to student learning achievement for the program (explain in question 7)

Other:

7) Briefly explain the assessment activities that took place in the last 18 months.

A. Course Level Assessment

Students completed key assessments embedded within their respective classes in Summer 2014, Fall 2014 and Summer 2015. The assessments and corresponding SLOs are listed in the table below.

Semester	Assessment	SLOs	Course
Summer 2014	Analysis of an Ethical Dilemma	4.1, 4.2	EDCS 618
Fall 2015	Plan B – Portfolio Plan B – Oral Defense/Oral Presentations	Varies depending on exemplars selected by the student 4.3 , 4.4	EDCS 695
Summer 2015	Advanced Child Development - Literature Review Issues & Trends in the Field of ECE - Powerpoint presentation with detailed notes Professional Development Plan (initial)	1.1, 1.3 2.1, 2.2, 2.3 4.3, 4.4 (formative assessment)	FamR 491 EDCS 677B EDCS 677B

B. Program Level Assessment Activities

In preparation for beginning a new cohort, the following program level assessment activities took place:

1. The Faculty Steering Committee (FSC) conducted a comprehensive analysis of the overall program from the application process through program completion. Multiple data points were examined including: application requirements, student demographics, formative and summative student work samples across the program of study, feedback on internal annual student surveys and program completer surveys, feedback from faculty on key assessments, courses, and Plan B capstones.

2. The FSC identified strengths and weaknesses in the current program SLOs, course content and assessments, evaluation measures, and overall student supports in light of a changing student demographic.

3. The FSC revised current SLOS, course map, program requirements, and documents. We disseminated this information to students in multiple formats (e.g. handbook, Laulima site, handouts) and identified gaps in coherence between the program on paper and the program students experienced.

4. The FSC reviewed the proposed advanced degree SLOs from the MAO to determine alignment with current program SLOs and diiscussed modifications to address anticipated SLOs and interventions to support students in refining competencies over the duration of their program of study (e.g. conducting scholarly searches of the literature, developing an organizational system, synthesizing and communicating information in scholarly writing, CITI training, preparing information for dissemination in professional presentations).

5. The FSC analyzed current Plan B capstones and revisted the capstone's effectiveness and validity in light of the changing student demographics.

8) What types of evidence did the program use as part of the assessment activities checked in question 6? (Check all that apply.)

Direct evidence of student learning (student work products)

Artistic exhibition/performance

Assignment/exam/paper completed as part of regular coursework and used for program-level assessment

Capstone work product (e.g., written project or non-thesis paper)

Exam created by an external organization (e.g., professional association for licensure)

Exit exam created by the program

IRB approval of research

Oral performance (oral defense, oral presentation, conference presentation)

Portfolio of student work

Publication or grant proposal

Qualifying exam or comprehensive exam for program-level assessment in addition to individual student evaluation (graduate level only)

Supervisor or employer evaluation of student performance outside the classroom (internship, clinical, practicum)

Thesis or dissertation used for program-level assessment in addition to individual student evaluation

Other 1:

Other 2:

Indirect evidence of student learning

Alumni survey that contains self-reports of SLO achievement

Employer meetings/discussions/survey/interview of student SLO achievement

Interviews or focus groups that contain self-reports of SLO achievement

Student reflective writing assignment (essay, journal entry, self-assessment) on their SLO achievement.

Student surveys that contain self-reports of SLO achievement

Other 1: One-on-one follow-up conversations with cooperating faculty teaching coursework and adjunct/cooperating faculty who were second reading Plan B capstone and participating in orals.

Other 2: Informal feedback from employers who hire graduates of the program and community stakeholders who interact with students during field based assignments.

Program evidence related to learning and assessment
(more applicable when the program focused on the use of results or assessment procedure/tools in this reporting period instead of data collection)

Assessment-related such as assessment plan, SLOs, curriculum map, etc.

Program or course materials (syllabi, assignments, requirements, etc.)

Other 1:

Other 2:

9) State the number of students (or persons) who submitted evidence that was evaluated. If applicable, please include the sampling technique used.

Students Submitting Evidence

During the time period from June 1, 2014 through September 30, 2015, a total of 35 students submitted evidence to evaluate. All student samples were considered when assessing the program (no sampling technique was employed).

Cohort IV Students, (n=16) completed the last summer of their program in Summer 2014 and finished Plan B Capstone requirements in Fall 2014. All 16 students turned in the key assessment Analysis of an Ethical Dilemma (Summer 2014) and their Plan B Professional Portfolio (Fall 2014). They also completed a Plan B Oral Presentation (Fall 2014). In addition, all students completed the College of Education program completer survey and an internal MEd in ECE Program exit survey.

Cohort V Students, (n=18) completed the first summer of their program in Summer 2015 and submitted three key assessments: an Advanced Child Development - Literature Review, an Issues & Trends Powerpoint Presentation with Detailed Notes, and a Professional Development Plan (formative assessment). All students completed a first summer MEd ECE Program evaluation survey.

A Cohort II Student (n=1) who was finishing up after returning to status completed the last summer of her program in Summer 2014 and finished her requirements in Fall 2014. She submitted the key assessment Analysis of an Ethical Dilemma (Summer 2014), Plan B Professional Portfolio (Fall 2014) and Plan B Oral Presentation (Fall 2014).

Faculty Submitting Evidence (n=13)

Faculty evidence was collected in the form of informal interviews, anecdotal observations, de-briefs with each course instructor at the completion of each course. Cooperating and adjunct faculty who participated in evaluating Plan B capstones were also contacted for feedback on the Plan B process and student work samples. THe FSC also compared anectodal evidence and work samples from the Plan B advising process.

10) Who interpreted or analyzed the evidence that was collected? (Check all that apply.)

Course instructor(s)

Faculty committee

Ad hoc faculty group

Department chairperson

Persons or organization outside the university

Faculty advisor

Advisors (in student support services)

Students (graduate or undergraduate)

Dean/Director

Other:

11) How did they evaluate, analyze, or interpret the evidence? (Check all that apply.)

Used a rubric or scoring guide

Scored exams/tests/quizzes

Used professional judgment (no rubric or scoring guide used)

Compiled survey results

Used qualitative methods on interview, focus group, open-ended response data

External organization/person analyzed data (e.g., external organization administered and scored the nursing licensing exam)

Other: Used enrollment data, grades and other quantitative program records.

12) Summarize the results of the assessment activities checked in question 6. For example, report the percent of students who achieved each SLO.

Summer 2014

All students enrolled in EDCS 618 (n=17) submitted the Analysis of an Ethical Dilemma. The key assessment requires that students demonstrate an understanding of professional ethics and how the NAEYC Code of Ethical Conduct could be used to help resolve ethical dilemmas. Course instructors used a combination of a checklist/narrative comments to evaluate student drafts. This provided students with quantitative and qualitative feedback on their effectiveness in completing different elements of the assessment. Students revised their assessment and a final draft was scored using rubrics that differentiated three levels of performance 0 = does not demonstrate essential competencies, 1 = demonstrates essential competencies, 2 = demonstrates exceptional competencies.

17 students completed this assessment. Of these, all of the students either demonstrated essential competencies (n=7 or 44%) or demonstrated exceptional competencies (n=9 or 56%) for SLOs 4.1 and 4.2.

Fall 2014

In the final semester of the program, all Cohort IV students (n=16) and one Cohort II student (n=1) completed a capstone Plan B – Professional Portfolio, a narrative reflection and collection of exemplars documenting competency in addressing the five mandatory program standards and two optional standards of the studentʻs choice. All program standards are demonstrated in this summative assessment, however students are allowed to choose which SLOs within standards they will submit as evidence so data was only collected on SLOs 4.3 and 4.4 which all students are expected to demonstrate. Students prepared and carried out a Plan B Oral Presentation that shared their Plan B capstone.

The Plan B advisor and Plan B second readers provided comment and scored student drafts using a checklist/rubric throughout the Spring 2014 and Fall 2014. Students received quantitative and qualitative feedback on their narrative statements and choice of exemplars. The final Plan B Professional Portfolio was evaluated using the same rubric. The rubric differentiated three levels of performance 0 = does not demonstrate essential competencies, 1 = demonstrates essential competencies, 2 = demonstrates exceptional competencies. Plan B advisors and second readers evaluated the students Plan B - Oral Presentations using a similar common program rubric.

17 students completed this assessment. Of these, all of the students either demonstrated essential competencies (n=12 or 71%) or demonstrated exceptional competencies (n=5 or 29%).

Summer 2015

In the first summer of the program, all entering Cohort V students (n=18) completed an Advanced Child Development – Literature Review addressing program SLOs 1.1 and 1.3. Students prepared a formal literature review written in APA style focused on an important issue or trend in child development research. The topic was chosen in consultation with the course instructor. A reference list of relevant primary and secondary sources was required. Students submitted an early draft and participated in one-on-one conferences with the instructor before revising their final drafts. The rubric for the final submittal differentiated three levels of performance 0 = does not demonstrate essential competencies, 1 = demonstrates essential competencies, 2 = demonstrates exceptional competencies.

18 students in Cohort V completed this assessment. Of these, a majority of the students either demonstrated essential competencies (n=12 or 67%) or demonstrated exceptional competencies (n=4 or 22%). Two students (11%) did not demonstrate essential competencies. Faculty highlighted the need to provide additional support to these students in scholarly writing and will assist these students in making the transition to graduate level work.

All Cohort V students (n=18) also completed an Issues and Trends in the Field of ECE Powerpoint Presentation focusing on SLOs 2.1, 2.2, and 2.3. Students researched a major issue or trend in the field of early childhood education and prepared a PowerPoint with accompanying materials for a 90 minute presentation (at least 12-25 slides with speaker notes on each page) that could be shared with a professional group, parents, or a community group.

Students submitted a draft and received written feedback and a tentative rating on their draft using a rubric that differentiated three levels of performance 0 = does not demonstrate essential competencies, 1 = demonstrates essential competencies, 2 = demonstrates exceptional competencies. After revising, they submitted their presentations materials for a final grade.

18 students in Cohort V completed this assessment. Of these, 2 (11%) of the students did not demonstrate essential competencies, 10 (55%) demonstrated essential competencies, and 6 (33%) demonstrated exceptional competencies. Additional support will be provided to the 2 students who had difficulty synthesizing the professional literature and organizing the information to share in an oral format.

Finally, Cohort V students (n=18) submitted a new formative assessment, a Professional Development Plan focusing on SLOs 4.3 and 4.4. This new assessment was created to more intentionally begin the cohort experience by inviting students to reflect on their current competencies as they began the program. The students identified professional goals to guide their growth throughout the life of the program and began to reflect on kuleana, how they would use what they learn to serve their communities.

The course instructor used a combination of checklist/narrative comments to evaluate student drafts. This provided students with quantitative and qualitative feedback on their effectiveness in completing different elements of the draft. Students revised their assessment and the formative assessment was scored using rubrics that differentiated three levels of performance 0 = does not demonstrate essential competencies, 1 = demonstrates essential competencies, 2 = demonstrates exceptional competencies.

18 students completed this assessment. Of these, 2 (11%) of the students did not demonstrate essential competencies, 8 (44.5%) demonstrated essential competencies, and 8 (44.5%) demonstrated exceptional competencies. The plans will be revisited each summer and this will provide an opportunityt to strengthen the plans for the 2 students whose initial plans did not demonstrate essential competencies.

13) What best describes how the program used the results? (Check all that apply.)

Assessment procedure changes (SLOs, curriculum map, rubrics, evidence collected, sampling, communications with faculty, etc.)

Course changes (course content, pedagogy, courses offered, new course, pre-requisites, requirements)

Personnel or resource allocation changes

Program policy changes (e.g., admissions requirements, student probation policies, common course evaluation form)

Students' out-of-course experience changes (advising, co-curricular experiences, program website, program handbook, brown-bag lunches, workshops)

Celebration of student success!

Results indicated no action needed because students met expectations

Use is pending (typical reasons: insufficient number of students in population, evidence not evaluated or interpreted yet, faculty discussions continue)

Other: Change in Plan B Capstone, use of Plan B Seminars, program supports (e.g. financial assistance advising)

14) Please briefly describe how the program used the results.

During the 2014-2015 school year, the Faculty Steering Team conducted a comprehensive analysis of the overall program from application process through program completion. We used the results to inform the following program revisions:

1. The SLOS were revised to emphasize tangible actions that could be documented in key assessments.

2. Assignments were coordinated across courses to reinforce without duplicating student experiences. Key program assessments were remapped and the courses were rearranged to encourage growth and to faciliate a more sequential process whereby coursework and experiences would build upon one another over the life of the program of studies.

2. Program literature and on-line supports were revised with more explicit information about program SLOS, key assessment expectations and resources. A handbook was created and will be more intentionally utilized in seminars and one-on-one advising so that students have a clearer understanding of their overal progam of studies.

3. The program SLOs were revisited in light of the draft advanced degree SLOs. We identified competencies that needed strengthening and interventions that could support students in developing these competencies over the course of their program of study (e.g. conducting scholarly searches of the literature, developing an organizing system for collecting references, synthesizing and communicating information in scholarly writing, CITI training, preparing information to share with varied audiences).

5. The effectiveness of key assessments and current rubrics for accreditation were evaluated and revised. Program rubrics are in the process of being revised (as classes go theough a new cycle) and differentiates 3 idifferent levels of performance (does not demonstrate competencies, demonstrates essential competencies, and demonsrates exceptional competencies). Previously, rubrics just recorded two levels of performance (Met and Did Not Meet). It is felt that this will yield more useful data to inform program improvement and will also help faculty focus on essential competency relative to SLOs and to recognize exceptinoal work.

6. The most significant program level change was revisiting the Plan B - Professional Portfolio This has been the only capstone option available to students since the program's inception. The Faculty Steering Committee noted the trend towards an increased number of younger student with less experience in the field, and more students entering the program from undergraduate degrees in other fields. This compared with the program completers from the first two cohorts who were experienced teachers, administrators and leaders with positions of authority within their respective organizations. The Faculty Steering Committee decided to add other Plan B options (project and paper) in order to be able create flexibility and diversify capstones to address the needs of an increasingly heterogenous student population.

7. Based on COE program completer surveys, internal annual program surveys, and anecdotal feedback, the Faculty Steering Committee has also initiated other steps to enhance a sense of community, provide support, and reduce the barriers to student success in completing the program. (e.g. aggressively identifying sources of financial support for students and assisting students in seeking out financial assistance and scholarships).

15) Beyond the results, were there additional conclusions or discoveries? This can include insights about assessment procedures, teaching and learning, and great achievements regarding program assessment in this reporting period.

The Faculty steering Committee is concerned with the overall drop in enrollment across graduate programs. Although the MEd in ECE program recruited a sufficient number of students to sustain this program, there were considerably fewer applicants than in past years and most cited expensse as a major barrier.

Expanding the repetoire of current assessment tools to includle other responsive evaluation tools may allow us to gain a deeper understanding of the challenges that graduate students encounter. This can inform steps taken to enhance the capacity of the program to recruit, retain and support graduate students in the field. For instance, first year program surveys found that over 50% of the current students in the program are working professionals with an annual salary of less than $35,000. Knowing this, the program is actively looking for ways to assist students in identifying possible sources of financial assistance and successfully obtaining support.