Program: Educational Tech (MEd)
Degree: Master's
Date: Thu Oct 11, 2012 - 10:58:33 am
1) Below are your program's student learning outcomes (SLOs). Please update as needed.
The Department of Educational Technology (ETEC) delivers courses and programs at multiple levels including: undergraduate, masters-level graduate, and a doctoral emphasis. At the master’s level, our Masters in Educational Technology is designed for candidates in many learning environments including K-12 and higher education, government, business, industry, and health occupations, whether they are teachers, trainers, developers, administrators, or support personnel. The program places emphasis on applications of technology in educational settings rather than just technical skills. Individuals from diverse backgrounds immediately apply what they learn to their particular contexts. Upon graduation, these new professionals will have a clearer vision of how to prepare learners for the future.
The University of Hawaii and the College of Education support the use of national standards to guide programs; therefore, ETEC has chosen to incorporate Association for Educational Communications and Technology (AECT) standards into the objectives for our program. Recently, the Hawaii Department of Education has incorporated technology standards within content standards so no longer includes educational technology as a discrete category. Since ETEC students come from a broad variety of professions, AECT standards were determined as the best fit for our program. Students are provided the following information in the UH catalog, on the website, and in advising documents.
The department has set the following student learning outcomes for its graduate students based on national standards for accreditation from the Association for Educational Communications and Technology (AECT):
1. Design – Proficiency in instructional design, the systematic approach to designing educational/instructional systems, materials, and processes, including analyzing, designing, developing, implementing, and evaluating.
2. Development - Demonstration of major instructional models and their technological applications to develop instructional materials and experiences using print, multimedia, computer-based, and integrated technologies.
3. Utilization - Application of principles and theories of media utilization, diffusion, implementation, and policymaking, as well as, the attitudes, ethics, and, interpersonal and communication skills required for active involvement in appropriate professional organizations and community services.
4. Management - Ability to plan, organize, coordinate, and supervise instructional technology by applying principles of project, resource, delivery system, and information management.
5 Evaluation – Capability of planning and executing research using knowledge of the existing body of research in the field, and, ability to evaluate the adequacy of instruction and learning by applying principles of problem analysis, criterion-referenced measurement, formative and summative evaluation, and long-range planning.
2) Your program's SLOs are published as follows. Please update as needed.
Department Website URL: http://etec.hawaii.edu Student Handbook. URL, if available online: http://etec.hawaii.edu/programsmed.html Information Sheet, Flyer, or Brochure URL, if available online: NA UHM Catalog. Page Number: 209 Course Syllabi. URL, if available online: NA Other: Advising Documents
Other: 3) Select one option:
- File (03/16/2020)
4) For your program, the percentage of courses that have course SLOs explicitly stated on the syllabus, a website, or other publicly available document is as follows. Please update as needed.
0% 1-50% 51-80% 81-99%
100%5) Did your program engage in any program assessment activities between June 1, 2011 and September 30, 2012? (e.g., establishing/revising outcomes, aligning the curriculum to outcomes, collecting evidence, interpreting evidence, using results, revising the assessment plan, creating surveys or tests, etc.)
Yes No (skip to question 14)6) For the period June 1, 2011 to September 30, 2012: State the assessment question(s) and/or assessment goals. Include the SLOs that were targeted, if applicable.
We collected data through rubrics on each of the 5 standard in our field, AECT.
1. Design – Proficiency in instructional design, the systematic approach to designing educational/instructional systems, materials, and processes, including analyzing, designing, developing, implementing, and evaluating.
2. Development - Demonstration of major instructional models and their technological applications to develop instructional materials and experiences using print, multimedia, computer-based, and integrated technologies.
3. Utilization - Application of principles and theories of media utilization, diffusion, implementation, and policymaking, as well as, the attitudes, ethics, and, interpersonal and communication skills required for active involvement in appropriate professional organizations and community services.
4. Management - Ability to plan, organize, coordinate, and supervise instructional technology by applying principles of project, resource, delivery system, and information management.
5 Evaluation – Capability of planning and executing research using knowledge of the existing body of research in the field, and, ability to evaluate the adequacy of instruction and learning by applying principles of problem analysis, criterion-referenced measurement, formative and summative evaluation, and long-range planning.
7) State the type(s) of evidence gathered to answer the assessment question and/or meet the assessment goals that were given in Question #6.
The program employs an assessment system specific to its professional field. There are multiple sources of knowledge, skills and disposition data with multiple collection points (entry, middle, exit). The data are analyzed, shared and discussed among department faculty to make decisions about students and the program. An annual retreat allows faculty to reassess the vision, mission, goals and objectives of the program. The Educational Technology master’s program common assessment element with the unit is the conceptual framework. The program prepares professionals who are KNOWLEDGEABLE, EFFECTIVE, and CARING. The ETEC Masters uses six discrete assessments that are evaluated using detailed rubrics aligned to AECT standards.
(1) The Practicum Project/Paper is a culminating, year-long research or design project and paper. They may choose one of three types of projects: instructional design, action research, or other approved topic. ETEC 687 supports proposal writing. ETEC 690, during final semester, supports completion of the project and the culminating public presentation at an international online conference.
(2) The Electronic Portfolio is a collection of work in the program that clearly presents a student’s knowledge, skills, attitudes, and professionalism. The portfolio is initiated in ETEC 602. It is completed and formally presented during the ETEC 690. In it, students (a) provide a resume, (b) showcase their best work, and (d) provide reflections of their growth including how they achieved the AECT standards.
(3) The Front-end Analysis is conducted in the first semester of a core course. It includes: (1) the characteristics and interactions of a system, (2) a dissemination and diffusion plan with an analysis of a system with regard to change characteristics and an adoption plan, (3) a needs assessment plan and needs statement, (4) alternate solutions with justifications, (5) a professional mediated presentation.
(4) Practicum Project – The required practicum course 687 provides a field/clinical experience to put instructional design theories and research skills into practice in a guided setting. Students take this course during the first semester of their final year. They are responsible for choosing a topic, deciding on a schedule through a learning contract, and wrting and presenting a proposal for their practicum project. Approval of their practicum project plan is a requisite for enrolling in 690, the capstone course where students will implement their project and publish the results.
(5) The Instructional Design Project is conducted in the second semester of a core course. The student must achieve project milestones within the time allotted. It includes the following steps: (1) Identify instructional goals. (2) Conduct an audience analysis. (3) Conduct an instructional analysis. (4) Write performance objectives. (5) Develop criterion referenced test items. (6) Develop an instructional strategy. (7) Develop an instructional module. (8) Design and conduct a formative evaluation. (9) Recommend and make revisions. (10) Make a professional mediated presentation of the project.
(6) The Technology Project is conducted in the first semester of a core course. Its purpose is to prepare a product that promotes a particular instructional technology or application. The target audience is educators involved in teaching or training who need to stay current with technology trends. The product may be a website, video, animation, slide presentation, brochure or any other medium that allows for independent viewing. It includes the following steps: (1) Create a content outline. (2) Create a storyboard or flowchart. (3) Decide on media format (4) Identify software hardware needed. (5) Identify skill needs and locate or arrange for training. (6) Produce the promotional product. (7) Make a professional mediated presentation.
8) State how many persons submitted evidence that was evaluated. If applicable, please include the sampling technique used.
A total of 15 rubric assessments were submitted, one by each course instructor. Six different instructors were involved. The sample included all students enrolled in the 15 courses/sections involved.
9) Who interpreted or analyzed the evidence that was collected? (Check all that apply.)
Course instructor(s) Faculty committee Ad hoc faculty group Department chairperson Persons or organization outside the university Faculty advisor Advisors (in student support services) Students (graduate or undergraduate) Dean/Director Other: 10) How did they evaluate, analyze, or interpret the evidence? (Check all that apply.)
Used a rubric or scoring guide Scored exams/tests/quizzes Used professional judgment (no rubric or scoring guide used) Compiled survey results Used qualitative methods on interview, focus group, open-ended response data External organization/person analyzed data (e.g., external organization administered and scored the nursing licensing exam) Other: 11) For the assessment question(s) and/or assessment goal(s) stated in Question #6:
Summarize the actual results.
The assessment data shows that our master’s candidates have performed well in all three major areas:
content knowledge; professional and pedagogical knowledge, skills and dispositions; and student
learning. It has also been shown that the assessment supports the fulfillment of all five AECT standards
for master’s preparation: design, development, utilization, management and evaluation.
Content Knowledge
Content knowledge is measured through two assessments in our program: the Practicum Project and Paper
and the Portfolio. The Practicum Project and Paper assessment items are primarily aligned with the Design
and Management standards of AECT, while the Portfolio assessment items are primarily aligned with
the Development standards. However, all 5 AECT standards are addressed between both assessments.
While our candidates' content knowledge is consistently assessed in all of the courses in the program,
and especially in the 6 required courses, the master's project and paper and culminating electronic
portfolio are considered the best measures for overall content knowledge in Educational Technology.
Candidates have done remarkably well in meeting all of the standards for both assessments with a 100%
success rate. Much of the success may be due to the long process involved in completing both products,
where ample opportunity for feedback to the candidate is provided.
Professional and Pedagogical Knowledge, Skills and Dispositions
The assessments for professional and pedagogical knowledge, skills and dispositions are determined
from rubric scores from the Front-end Analysis conducted in ETEC 600 - Theory & Practice in Ed Tech
and ETEC 687- Instructional Development Practicum. Between the two courses, all of five AECT
Standards are covered: Design, Development, Management and Evaluation. The total scores for each
course fall well into the “acceptable” and “target” range indicating that students in general are meeting
standards in professional and pedagogical knowledge, skills and dispositions. We plan to look more
closely and be more systematic about ensuring that both the on-campus and online cohorts are wellaligned,
especially cases where two instructors teach the same course. Currently, candidates receive
feedback from the course instructors and proceed to improve accordingly. Between the initial semester
assessment, the Front-end Analysis and the mid-point Practicum assessment, students gain a measure of
sophistication in their knowledge, skills, and dispositions. It is clear that by the time they serve clients,
such as College of Education faculty, they are well-received. The additional feedback they receive from
such client groups adds authentic assessment of their capabilities.
Effects on Student/Client Learning in a Supportive Environment
The Instructional Development Project from ETEC 613 and the ETEC 602 Technology Promotion
Project are used to assess our master’s candidate’s effects on providing supportive learning
environments for student or client learning. Between the two project outcomes, four of five AECT
Standards are primarily covered: Design, Development, Utilization and Evaluation, although all five
Standards were addressed in the assessment rubrics. Candidates’ acceptable and target rating on both
assessments indicate that they are doing well in meeting the standards for student/client learning in a
supportive environment.
Our assessments the Instructional Design Project (Assessment 5) and Technology Project (Assessment 6) are based on
candidates' ability to apply a systems approach to designing and delivering and evaluating instruction or
training. Candidates receive both peer and instructor feedback on both projects and revisions are an
ongoing process. The ID Project emphasizes the small group testing of instructional modules for
formative evaluation. This experience helps to guide the improvement of student learning using actual
data.
While our assessment indicates that candidates are quite capable of meeting the standards aligned with
both projects, the department would like to improve its approach to collecting data. Currently, both
projects are completed by teams of 2-3 candidates. Under these conditions, it is sometimes difficult to
determine a true score for an individual. Assessment items and course activities will need to be reviewed
and revised to accommodate the assessment of individual performance.
12) State how the program used the results or plans to use the results. Please be specific.
Overall, faculty feel positive about the results of our 6 assessments for meeting our program goals. Students appear to be achieving the goals with no major problems. The faculty have taken steps to make changes when needed to improve student and program performances. While regular faculty meetings and annual faculty retreats allow for formal discussions about improvements, the department is small and frequent informal discussions about students and the program are conducted as needed. Ideas generated from these discussions are brought to the entire faculty for decisions.
13) Beyond the results, were there additional conclusions or discoveries?
This can include insights about assessment procedures, teaching and learning, program aspects and so on.
We look forward to growing as a program as we continue to learn from our assessments and continue to improve our assessment system.