Unit: Educational Administration
Program: Educational Admin (MEd)
Degree: Master's
Date: Fri Nov 20, 2009 - 12:32:15 pm

1) List your program's student learning outcomes (SLOs).

Program Standards/Outcomes
What candidates should know and do, and the ways that they should demonstrate professionalism
1. Educational leaders are knowledgeable about and understand organizational life in schools and the dynamics of school change processes by examining trends, traditions, theory and policies of institutions in order to improve educational practice which promotes the learning success of all students.
2. Educational leaders understand, can articulate, and act within the moral/ethical, political, collaborative, strategic and caring dimensions of administrative roles within diverse cultural contexts.
3. Educational leaders demonstrate a well developed analytic capacity that is informed by theory, research, and practice to solve organizational problems and generate policy.
4. Educational leaders can apply knowledge and skills to changing organization contexts impacted by social, political, economic, cultural, and technological forces in order to foster the growth and development of the organization and its members.

2) Where are your program's SLOs published?

Department Website URL:
Student Handbook. URL, if available online:
Information Sheet, Flyer, or Brochure URL, if available online: NA
UHM Catalog. Page Number:
Course Syllabi. URL, if available online: NA
Other: College of Education NCATE website
Other:

3) Upload your program's current curriculum map(s) as a PDF.

Curriculum Map File(s) from 2009:

4) What percentage of courses have the course SLOs explicitly stated on the course syllabus, department website, or other publicly available document? (Check one)

0%
1-50%
51-80%
81-99%
100%

5) State the SLO(s) that was Assessed, Targeted, or Studied

Program Standards/Outcomes Assessed between June 2008 and September 2009
What candidates should know and do, and the ways that they should demonstrate professionalism
1. Educational leaders are knowledgeable about and understand organizational life in schools and the dynamics of school change processes by examining trends, traditions, theory and policies of institutions in order to improve educational practice which promotes the learning success of all students.
2. Educational leaders understand, can articulate, and act within the moral/ethical, political, collaborative, strategic and caring dimensions of administrative roles within diverse cultural contexts.
3. Educational leaders demonstrate a well developed analytic capacity that is informed by theory, research, and practice to solve organizational problems and generate policy.
4. Educational leaders can apply knowledge and skills to changing organization contexts impacted by social, political, economic, cultural, and technological forces in order to foster the growth and development of the organization and its members.

6) State the Assessment Question(s) and/or Goal(s) of Assessment Activity

Assessment 1
Educational Leadership Platform

1.    Description of assessment and its use in the program

This assignment in EDEA 601 (Introduction to School Leadership) asks students to write their own educational leadership platforms, capturing their values and beliefs about education, schools, and school systems. As an assessment, it represents a preliminary evaluation of students’ understanding of school leadership as it is completed early in their masters program. It serves as baseline for students and can be used to consider how they develop a broader and more comprehensive understanding of the teaching and learning components in schools.

Assessment 2
Research and School Improvement Inquiry

1.    Description of assessment and its use in the program

As EDEA 602 (Research methods in educational administration) is designed for prospective administrators to consider educational problems using quantitative and qualitative approaches, this assessment determines if students are able to do so. This assignment has students identify an educational problem, conduct an appropriate literature review on that problem, and propose an empirical study to address that concern. Students should be knowledgeable of various research methods and able to apply an appropriate one to their line of inquiry. As this course is generally taken early in the program, it enables students to think about educational problems and directs their work on their final culminating projects.

Assessment 3
Case Study in Education Law and Leadership

1.    Description of assessment and its use in the program  

EDEA 630 (Education Law) focuses on providing prospective administrators with a comprehensive understanding of the Constitutional and statutory law, court decisions, and policies that govern the United States public school system.  The content of the course is contextualized within a framework of prudence and decision-making processes administrators should exercise with respect to the law. Since most school administrators are not attorneys, they need enough knowledge of the law to make sound decisions and run their schools efficiently, but they also need to know when they might need to seek counsel. This assignment consists of a comprehensive case involving a legal issue that a principal might encounter.  The assessment is based on their ability to address four aspects. First, they must identify the legal (and often accompanying ethical) issue that is central to the case.  Second, they must cite and discuss all relevant aspects of law (e.g., case law, legislation, policy). Third, they must present a plan for handling the situation that addresses the legal aspects and leads to the best possible resolution for all parties involved. Fourth, they must present an analysis of how, if possible, the situation could have been avoided and the action they would take to prevent a recurrence.  This course is generally taken early in the preparation program and is intended to provide students with knowledge they need to make informed decisions in an array of legal contexts, and to avoid putting themselves, their school, or others into legal jeopardy. It also helps prepare them to work within their schools to create and implement policies and procedures that are legal and ethical and are consistent with state and federal policy.
Assessment 4
Leadership/Organization Culminating Project

1.     Description of assessment and its use in the program 

This final assignment in EDEA 699 asks students to design, develop and execute an independent study related to their interest, work and study in educational leadership. For example, students may chose to explore a topic in greater depth through an extensive review of research. Or they may conduct an empirical study on a problem in their school organization. As a result of doing this project, students should demonstrate their knowledge and understanding of school leadership and organizations.

Leadership/Organization Culminating Project (Research)

1.    Description of assessment and its use in the program 

This final assignment in EDEA 699 asks students to design, develop and execute an independent study related to their interest, work and study in educational leadership. For example, students may chose to explore a topic in greater depth through an extensive review of research. Or they may conduct an empirical study on a problem in their school organization. As a result of doing this project, students should demonstrate their knowledge and understanding of school leadership and organizations.

7) State the Type(s) of Evidence Gathered

Assessment 1:  Students prepared a written leadership platform document.

Assessment 2: The students completed a written paper identifying an educational problem, conducting an appropriate literature review on the problem, and proposing an empirical (either quantitative or qualitative) study to address that concern.

Assessment 3:  Students write a response to a legal case.  This assignment consists of a comprehensive case involving a legal issue that a principal might encounter.  The assessment is based on their ability to address four aspects. First, they must identify the legal (and often accompanying ethical) issue that is central to the case.  Second, they must cite and discuss all relevant aspects of law (e.g., case law, legislation, policy). Third, they must present a plan for handling the situation that addresses the legal aspects and leads to the best possible resolution for all parties involved. Fourth, they must present an analysis of how, if possible, the situation could have been avoided and the action they would take to prevent a recurrence. 

Assessment 4: Students design, develop and execute an independent project related to their interest, work and study in educational leadership.

Assessment 5:  Students design, develop and execute an independent research project related to their interest, work and study in educational leadership

8) State How the Evidence was Interpreted, Evaluated, or Analyzed

Assessment 1:  

Students in the Fall, 2008 section of EDEA 601 were given the assignment related to developing their educational leadership platform/philosophy. Students were rated according to five criteria: Understanding of educational content (i.e., importance of emphasizing optimal teaching and learning processes in schools, creating a positive educational environment within the school); leadership (i.e., specific views about leadership (e.g., need for collaborative, shared decision making), development of an effective argument (e.g., use of examples that support major propositions); written effectiveness (e.g., attention to clarity of thought, writing mechanics); clear targeted audience (e.g., student recognizes who audience is and how to communicate effectively with them).

Students are rated on a three-point scale (U = unsatisfactory; A = acceptable; E = exceeds). Papers were rated by two independent raters. Across the five areas comprising the assessment, variability in scores due to raters ranged from 0% (content, writing effectiveness) to 14% (development of effective argument). This suggests minimal variation due to differences between raters

Assessment 2:  In terms of content criteria, students are assessed on their a) identifying and defining a problem as well as justifying its importance; b) specifying a conceptual or theoretical framework, c) providing appropriate methodology, d) stating what data and analysis will be employed, and e) stating what implications from the study. In addition, students must demonstrate writing competency, in terms of organization and development of ideas, style and language, and APA format. While less attention is given to this writing component compared with the content criteria, student papers are assessed accordingly.

Assessment 3:  Students are assessed on the assignment based on their a) identification of the central legal/ethical issue, b) ability to cite and articulate an understanding of the relevant law pertaining to the issue, c) formulation of an plan that encompasses both legal and interpersonal issues that will lead to a successful resolution and, d) analysis of how the situation might have been avoided, including implications for future action at the school level to ensure this. Each of these four areas is given equal weight.

Assessment 4:  Students are rated on a three-point scale (U = unsatisfactory; A = acceptable; E = exceeds). Papers were rated by one faculty member after a set of training papers was reviewed. 

Assessment 5:  Students are rated on a three-point scale (U = unsatisfactory; A = acceptable; E = exceeds). Papers were rated by one faculty member after a set of training papers was reviewed. 

9) State How Many Pieces of Evidence Were Collected

Assessment 1:  One piece of evidence

Assessment 2:  Two pieces of evidence

Assessment 3:  Three pieces of evidence

Assessment 4:  One piece of evidence

Assessment 5:  One piece of evidence

10) Summarize the Actual Results

Assessment 1:  Fall 2008 (n=14)
    _____________________________________________________________
                                                  Unacceptable     Acceptable     Exceeds               
     _____________________________________________________________ 
    Educational Content                 14%           86%           0%        

    Leadership                              14%          71%           14%

    Effective Argument                   14%          43%          43%

    Writing Effectiveness                14%           71%          14%

    Audience                                14%          86%           0%
    ______________________________________________________________
Assessment 2: Spring 2007 (n=16)
    _____________________________________________________________
                                               Unacceptable     Acceptable     Exceeds               
     _____________________________________________________________ 
    Problem identification            6%        93%        0%

    Framework/literature            19%        43%        38

    Research methods, analysis     6%        25%        69%

    Implications                       19%        50%        31%
    ______________________________________________________________
Assessment 3

Spring 2008 (n=12)
____________________________________________________________________
                    Unacceptable        Acceptable    Exceeds
____________________________________________________________________
Identification of legal issue             8%        75%        17%

Citing of relevant law                    8%        92%          0%

Resolution plan                            8%        75%        17%

Analysis and implications                8%        67%                 25%
____________________________________________________________________

Summer 2008 (n=18)
____________________________________________________________________
                    Unacceptable        Acceptable    Exceeds
____________________________________________________________________
Identification of legal issue              6%        83%        11%        

Citing of relevant law                    11%        83%        6%        

Resolution plan                            11%        78%        11%

Analysis and implications                 6%        72%        22%
____________________________________________________________________

Spring 2009 (n=12)
____________________________________________________________________
                    Unacceptable        Acceptable    Exceeds
____________________________________________________________________
Identification of legal issue             0%        75%        25%        

Citing of relevant law                    0%        67%        33%        

Resolution plan                            0%        67%        33%

Analysis and implications               0%        50%        50%
_____________________________________________________________________

Assessment 4:

_____________________________________________________________
                                                  Unacceptable     Acceptable     Exceeds               
     _____________________________________________________________ 
    Demonstration of knowledge about
and understanding of:

 #1  Organizations                       
                   2006 (N=6)                     0%             67%        33%   
           2007 (N=10)                            0%             60%        40%
                   2008 (N=12)                    0%             25%        75%         
#2 Administrative roles
                   2006  (N=6)                     0%                       33%                67%
                   2007  (N=10)                  20%                       30%                50%
                   2008  (N=12)                    0%                       75%                25%
    #4 Changing organizational
    Contexts
                  2006  (N=6)                      0%                       50%                50%
                  2007  (N=10)                     0%                       50%                50%
                  2008  (N=12)                     0%                       42%                58%
    ______________________________________________________________
Assessment 5:                                                      Unacceptable     Acceptable      Exceeds               
     ___________________________________________________________________ 

#3 Demonstration of skills:
analysis & application of theory
problem solving
generating policy

              2006    (N = 6)                       0%                      83%               17%
              2007    (N=10)                        0%                      60%               40% 
              2008    (N=12)                        0%                      33%               67%
_____________________________________________________________________

11) Briefly Describe the Distribution and Discussion of Results

The students received the results of specific assignments individually and discussed them with their respective professors.  The three faculty members in the K-12 strand of the Educational Administration discussed the results of the aggregate assessments to use in program evaluation and improvement.  These discussions took place in meetings with one another.  The information (with individual student names and other identifiers deleted)was also shared in a report with leaders in administrator preparation in the Hawaii Dept. of Education, with whom our department closely collaborates, to discuss the results and possible implications for program improvement.

12) Describe Conclusions and Discoveries

Assessment 1:  The data suggest that the majority of students (86+%) are able to articulate these first two program standards at an acceptable level, with some excelling.

Assessment 2:  Data can be used to determine what aspects require more attention. For example, students have been able to identify but not been able to conceptualize the problem in terms of its multifaceted issues. Or they might have difficulty articulating what theories inform their approach. In terms of research methods and data analysis, students might indicate what methods will be used but often do not provide sufficient substantiation for those approaches. These criteria have been helpful in targeting instruction and revise feedback to students in order to assist them in completing this assignment.

Assessment 3:  The data can be used to identify areas where students seem to be having difficulty and helping strengthen areas of difficulty.  It is our department’s goal that all students should be performing in the acceptable and exceeds ranges, and instruction and assessment strategies are designed toward that end.  When students fail to make the acceptable target, they are given explicit feedback about exactly what they missed. Another way the data can be used is to determine how the delivery method might influence student performance. For example, the Spring 2008 sample was an on-campus evening course that consisted of a random mixture of vice-principals in their second semester on the job, and teachers and other school personnel (a counselor, curriculum coordinator and librarian) with no administrative experience. The Summer 2008 sample was an online course that involved extensive online interaction, but no direct discussion among the students or the professor. The Spring 2009 sample consists of a cohort of first year vice principal interns who are given one day every other week off work to attend courses together, and who have many other opportunities to meet and discuss their work and studies. It appears from the samples presented here that the third option is working best (although due to small sample sizes, care must be taken in generalization). Still, it seems that the online delivery of the course still needs improving, and we are working to ensure that.

Assessment 4:  The assessment addresses three program standards directly. Students address how organizational concepts are important to consider in solving educational problems. This assessment generally suggests the students are on target in terms of being able to connect organizational theory to real educational problems, address their roles as emerging leaders in facilitating various types of school improvement projects, and in considering how changing organizational contexts impact the framing of improvement interventions.

Assessment 5:  The data suggest candidates are adequate prepared to examine research and use data to improve school practice. It appears over the last three-year period that the program is improving in having candidates exceed acceptable performance.

13) Use of Results/Program Modifications: State How the Program Used the Results --or-- Explain Planned Use of Results

Since the process of collecting data for department-wide assessment has been in place for one year, no major modifications have been made to the program.  However, the department has begun a systematic, ongoing process for collecting data to use in future program improvements and modifications.  Currently, students seem to be progressing well in all areas and there are no obvious areas of weakness.  Should this situation change, however, the department is prepared to take action to address the problem and make improvements.  The Educational Administration faculty collaborate with the Professional Development in Education and Research Institute branch of the Hawaii state Department of Education in the preparation of principals in the state, and Assessments 4 and 5 in the program are heavily field-based.  Should student outcomes in those areas change, the faculty are prepared to work with the students and DOE leaders to assess the problem and take action to make improvements.

14) Reflect on the Assessment Process

Overall, we believe that the assessments chosen to measure our progress in the SLO's is valid and reliable, however since Assessments 4 and 5 are capstone projects, we will have all three members of the department rate them in the future rather than just one person reduce the chance of bias.  Because Assessments 1-3 are based in individual courses and are currently evaluated by the person teaching the course, we are planning to meet to assess samples of papers in these courses to improve our inter-rater reliability.

15) Other Important Information

The assessments described in this report are currently only used in the K-12 portion of our department's program; they have not been developed and implemented in our Higher Education strand.  This is in the process of happening.  Also, the assessments identified here are tied to the Educational Leadership Constituent Council (ELCC) that uses professional standards to provide guidelines to administrator preparation and certification programs in the United States.  The ELCC standards and their relationship to our four program standards (ELO's)are:

ELCC Standards and Department SLO's

Standard 1: Candidates who complete the program are educational leaders
Who have the knowledge and ability to promote the success of all students by
facilitating the development, articulation, implementation, and stewardship
of a school or district vision of learning supported by the school community.                               
Standard 2: Candidates who complete the program are educational
Leaders who have the knowledge and ability to promote the success of all
students by promoting a positive school culture, providing an effective
instructional program, applying best practice to student learning and designing
comprehensive professional growth plans for staff.                                                                      Department SLO's #1-4

Standard 3: Candidates who complete the program are educational leaders who
have the knowledge and ability to promote the success of all students by
managing the organization, operations, and resources in a way that promotes
a safe, efficient, and effective learning environment.                                                            Department SLO's #1,3-4

Standard 4: Candidates who complete the program are educational leaders who
have the knowledge and ability to promote the success of all students by
collaborating with families and other community members, responding to diverse
community interests and needs, and mobilizing resources.                                                             Department SLO #2

Standard 5: Candidates who complete the program are educational leaders who
have the knowledge and ability to promote the success of all students by
acting with integrity, fairness, and in an ethical manner.                                                                 Department SLO #2

Standard 6: Candidates who complete the program are educational leaders who
have the knowledge and ability to promote the success of all students by
understanding, responding to, and influencing the larger political, social,
economic, legal and cultural context.                                                                                              Department SLO #2

Standard 7: Internship. The internship provides significant opportunities for candidates
to synthesize and apply the knowledge and practice and develop the skills identified
in Standards 1-6 through substantial and sustained work in real settings, planned and
guided cooperatively by the institution and school district for graduate credit.                              

Department SLO's #3-4

16) FOR DISTANCE PROGRAMS ONLY: Explain how your program/department has adapted its assessment of student learning in the on-campus program to assess student learning in the distance education program.

17) FOR DISTANCE PROGRAMS ONLY: Summarize the actual student learning assessment results that compare the achievement of students in the on-campus program to students in the distance education program.