Program: Educational Tech (MEd)
Degree: Master's
Date: Thu Nov 04, 2010 - 11:16:57 am
1) Below are the program student learning outcomes submitted last year. Please add/delete/modify as needed.
The Department of Educational Technology (ETEC) delivers courses and programs at multiple levels including: undergraduate, masters-level graduate, and a doctoral emphasis. At the master’s level, our Masters in Educational Technology is designed for candidates in many learning environments including K-12 and higher education, government, business, industry, and health occupations, whether they are teachers, trainers, developers, administrators, or support personnel. The program places emphasis on applications of technology in educational settings rather than just technical skills. Individuals from diverse backgrounds immediately apply what they learn to their particular contexts. Upon graduation, these new professionals will have a clearer vision of how to prepare learners for the future.
The University of Hawaii and the College of Education support the use of national standards to guide programs; therefore, ETEC has chosen to incorporate Association for Educational Communications and Technology (AECT) standards into the objectives for our program. Recently, the Hawaii Department of Education has incorporated technology standards within content standards so no longer includes educational technology as a discrete category. Since ETEC students come from a broad variety of professions, AECT standards were determined as the best fit for our program. Students are provided the following information in the UH catalog, on the website, and in advising documents.
The department has set the following student learning outcomes for its graduate students based on national standards for accreditation from the Association for Educational Communications and Technology (AECT):
1. Design – Proficiency in instructional design, the systematic approach to designing educational/instructional systems, materials, and processes, including analyzing, designing, developing, implementing, and evaluating.
2. Development - Demonstration of major instructional models and their technological applications to develop instructional materials and experiences using print, multimedia, computer-based, and integrated technologies.
3. Utilization - Application of principles and theories of media utilization, diffusion, implementation, and policymaking, as well as, the attitudes, ethics, and, interpersonal and communication skills required for active involvement in appropriate professional organizations and community services.
4. Management - Ability to plan, organize, coordinate, and supervise instructional technology by applying principles of project, resource, delivery system, and information management.
5 Evaluation – Capability of planning and executing research using knowledge of the existing body of research in the field, and, ability to evaluate the adequacy of instruction and learning by applying principles of problem analysis, criterion-referenced measurement, formative and summative evaluation, and long-range planning.
2) As of last year, your program's SLOs were published as follows. Please update as needed.
Department Website URL: http://etec.hawaii.edu Student Handbook. URL, if available online: http://etec.hawaii.edu/programsmed.html Information Sheet, Flyer, or Brochure URL, if available online: NA UHM Catalog. Page Number: 210 Course Syllabi. URL, if available online: NA Other: Advising Documents
Other: 3) Below is the link to your program's curriculum map (if submitted in 2009). If it has changed or if we do not have your program's curriculum map, please upload it as a PDF.
- File (03/16/2020)
4) The percentage of courses in 2009 that had course SLOs explicitly stated on the syllabus, a website, or other publicly available document is indicated below. Please update as needed.
0% 1-50% 51-80%
81-99% 100%5) State the assessment question(s) and/or goals of the assessment activity. Include the SLOs that were targeted, if applicable.
All SLOs are assessed in multiple forms throughout the program. There are five major SLOs that are based on our national standards for accreditation in our field (AECT):
1. Design – Proficiency in instructional design, the systematic approach to designing educational/instructional systems, materials, and processes, including analyzing, designing, developing, implementing, and evaluating.
2. Development - Demonstration of major instructional models and their technological applications to develop instructional materials and experiences using print, multimedia, computer-based, and integrated technologies.
3. Utilization - Application of principles and theories of media utilization, diffusion, implementation, and policymaking, as well as, the attitudes, ethics, and, interpersonal and communication skills required for active involvement in appropriate professional organizations and community services.
4. Management - Ability to plan, organize, coordinate, and supervise instructional technology by applying principles of project, resource, delivery system, and information management.
5. Evaluation – Capability of planning and executing research using knowledge of the existing body of research in the field, and, ability to evaluate the adequacy of instruction and learning by applying principles of problem analysis, criterion-referenced measurement, formative and summative evaluation, and long-range planning.
6) State the type(s) of evidence gathered.
The program's assessment questions focused on three major areas:
Content knowledge (Assessments 1 and 2)
Pedagogical and professional knowledge, skills and dispositions (Assessments 3 and 4)
Focus on student learning (Assessment 5)
7) Who interpreted or analyzed the evidence that was collected?
Course instructor(s) Faculty committee Ad hoc faculty group Department chairperson Persons or organization outside the university Faculty advisor Advisors (in student support services) Students (graduate or undergraduate) Dean/Director Other: 8) How did they evaluate, analyze, or interpret the evidence?
Used a rubric or scoring guide Scored exams/tests/quizzes Used professional judgment (no rubric or scoring guide used) Compiled survey results Used qualitative methods on interview, focus group, open-ended response data External organization/person analyzed data (e.g., external organization administered and scored the nursing licensing exam) Other: 9) State how many persons submitted evidence that was evaluated.
If applicable, please include the sampling technique used.
The program employs an assessment system specific to its professional field. There are multiple sources of knowledge, skills and disposition data with multiple collection points (entry, middle, exit). The data are analyzed, shared and discussed among department faculty to make decisions about candidates and the program. An annual retreat allows faculty to reassess the vision, mission, goals and objectives of the program. The Educational Technology master’s program common assessment element with the unit is the conceptual framework. The program prepares professionals who are KNOWLEDGEABLE, EFFECTIVE, and CARING. The ETEC Masters uses six discrete assessments that are evaluated using detailed rubrics aligned to AECT standards.
(1) The Masters Project/Paper is a culminating, year-long research or design project and paper. They may choose one of three types of projects: instructional design, action research, or other approved topic. ETEC 750 supports proposal writing. ETEC 690, during final semester, supports completion of the project and the culminating public presentation.
(2) The Electronic Portfolio is a collection of work in the program that clearly presents a student’s knowledge, skills, attitudes, and professionalism. The portfolio is initiated in ETEC 602 and evaluated again at the end of year one in ETEC 601. It is completed and formally presented during the ETEC 690. In it, candidates (a) provide a resume, (b) showcase their best work, and (d) provide reflections of their growth including how they achieved the AECT standards.
(3) The Front-end Analysis is conducted in the first semester of a core course. It includes: (a) the characteristics and interactions of a system, (b) a dissemination and diffusion plan with an analysis of a system with regard to change characteristics and an adoption plan, (c) a needs assessment plan and needs statement, (d) alternate solutions with justifications, and (e) a professional mediated presentation.
(4) Practicum—The required practicum course provides a field/clinical experience to put instructional design theories and skills into practice in a guided setting. Candidates take this course after they have completed the core and taken two electives. Candidates design, deliver and evaluate technology solution for a client group. Candidates are responsible for choosing a topic, deciding on a schedule, and executing their plan. A final summative report describing the process includes materials, goals, objectives, plans, evaluation summaries, and personal reflections including what worked and did not work.
(5) The Instructional Design Project is conducted in the second semester of a core course. The student must achieve project milestones within the time allotted. It includes the following steps: (a) Identify instructional goals. (b) Conduct an audience analysis. (c) Conduct an instructional analysis. (d) Write performance objectives. (e) Develop criterion referenced test items. (f) Develop an instructional strategy. (g) Develop an instructional module. (h) Design and conduct a formative evaluation. (i) Recommend and make revisions. (j) Make a professional mediated presentation of the project.
(6) The Technology Project is conducted in the first semester of a core course. Its purpose is to prepare a product that promotes a particular instructional technology or application. The target audience is educators involved in teaching or training who need to stay current with technology trends. The product may be a website, video, animation, slide presentation, brochure or any other medium that allows for independent viewing. It includes the following steps: (a) Create a content outline. (b) Create a storyboard or flowchart. (c) Decide on media format (d) Identify software hardware needed. (e) Identify skill needs and locate or arrange for training. (f) Produce the promotional product. (g) Make a professional mediated presentation.
10) Summarize the actual results.
The assessment data shows that our master’s candidates have performed well in all three major areas: content knowledge; professional and pedagogical knowledge, skills and dispositions; and student learning. It has also been shown that the assessment supports the fulfillment of all five AECT standards for master’s preparation: design, development, utilization, management and evaluation.
Content Knowledge Content knowledge is measured through two assessments in our program: the Master’s Project and Paper and the Portfolio. The Master's Project and Paper assessment items are primarily aligned with the Design and Management standards of AECT, while the Portfolio assessment items are primarily aligned with the Development standards. However, all 5 AECT standards are addressed between both assessments. While our candidates' content knowledge is consistently assessed in all of the courses in the program, and especially in the 7 required courses, the master's project and paper and culminating electronic portfolio are considered the best measures for overall content knowledge in Educational Technology. Candidates have done remarkably well in meeting all of the standards for both assessments with a 100% success rate. Much of the success may be due to the long process involved in completing both products, where ample opportunity for feedback to the candidate is provided.
Our first online cohort began in Fall 2006 and in the process of redesigning courses for online delivery, the faculty decided to revise the assessment instruments for the program. The Spring 2008 graduating class presented the first opportunity to assess candidates who had gone through the complete cycle. In analyzing the assessment data on candidate's content knowledge, it is apparent that both on-campus and online candidates perform equally well. This supports the department's contention that the online delivery of our graduate program is of the same quality as that offered to on campus students. The faculty are satisfied, in general, with the candidate’s content knowledge when exiting the program. However, it is apparent that more can be done to improve ratings on the standards aligned with content knowledge. While all candidates met the standards, only about 5-6% were in the “target” range. The department plans to implement measures on these standards earlier in the program. For example, a midpoint assessment for portfolios will be made during the second semester of the program, in ETEC 601, a required course. This has been part of the assessment plan but because of changes in our curriculum, assessment and staffing, a mid-point assessment was not implemented. Candidates enrolled in the Spring 2009 offering of ETEC 601 will receive an assessment on their developing portfolios. Also, while ample feedback is provided to candidates as they prepare and complete their master’s project and papers, specific feedback from the assessment instruments have been limited to completed work in the terminal course. The department plans to discuss how to implement an earlier assessment, perhaps in the proposal writing stage.
Professional and Pedagogical Knowledge, Skills and Dispositions The assessments for professional and pedagogical knowledge, skills and dispositions are determined from rubric scores from the Front-end Analysis conducted in ETEC 600 - Theory & Practice in Ed Tech and ETEC 687- Instructional Development Practicum. Between the two courses, all of five AECT Standards are covered: Design, Development, Management and Evaluation. The total scores for each course fall well into the “acceptable” and “target” range indicating that students in general are meeting standards in professional and pedagogical knowledge, skills and dispositions. We plan to look more closely and be more systematic about ensuring that both the on-campus and online cohorts are well aligned, especially cases where two instructors teach the same course. Currently, candidates receive feedback from the course instructors and proceed to improve accordingly. Between the initial semester assessment, the Front-end Analysis and the mid-point Practicum assessment, students gain a measure of sophistication in their knowledge, skills, and dispositions. It is clear that by the time they serve clients, such as College of Education faculty, they are well-received. The additional feedback they receive fromsuch client groups adds authentic assessment of their capabilities.
Effects on Student/Client Learning in a Supportive Environment The Instructional Development Project from ETEC 603 and the ETEC 602 Technology Promotion Project are used to assess our master’s candidate’s effects on providing supportive learning environments for student or client learning. Between the two project outcomes, four of five AECT Standards are primarily covered: Design, Development, Utilization and Evaluation, although all five Standards were addressed in the assessment rubrics. Candidates’ acceptable and target rating on both assessments indicate that they are doing well in meeting the standards for student/client learning in a supportive environment.
Our assessments the ID Project (Assessment 5) and Technology Project (Assessment 6) are based on candidates' ability to apply a systems approach to designing and delivering and evaluating instruction or training. Candidates receive both peer and instructor feedback on both projects and revisions are an ongoing process. The ID Project emphasizes the small group testing of instructional modules for formative evaluation. This experience helps to guide the improvement of student learning using actual data.
While our assessment indicates that candidates are quite capable of meeting the standards aligned with both projects, the department would like to improve its approach to collecting data. Currently, both projects are completed by teams of 2-3 candidates. Under these conditions, it is sometimes difficult to determine a true score for an individual. Assessment items and course activities will need to be reviewed and revised to accommodate the assessment of individual performance.
11) How did your program use the results? --or-- Explain planned use of results.
Please be specific.
Each year the ETEC faculty reviews our program to make continual improvement in the Masters program based on lessons learned, data collected, and feedback we have received over the year. This year we petitioned and received approval for the following changes.
1. Change: Delete ETEC 750B: Seminar in Educational Technology Issues: Instructional Development as a requirement. Reduce the program from 36 to 33 credits.
Justification: ETEC 750B is a course that was added a few years ago to help students with the practical portion of their master’s degree, in addition, they take a practicum course, as well as, another culminating course. Since many students finish in two years, this would eliminate the problem of students having a practicum project and a final project that they have to produce in the same semester. When reviewing our assessments during our recent national accreditation we discovered this redundancy.
Over the years our program has become more and more rigorous, outpacing other programs in this field. The culminating activities currently include a practicum, a project, a paper, an electronic portfolio, and an oral presentation. This change would merge the project and the practicum together into a single practical project, ETEC 687. This would help streamline the credit heavy program.
12) Beyond the results, were there additional conclusions or discoveries? This can include insights about assessment procedures, teaching and learning, program aspects and so on.
The current data provides a good baseline for assessing our candidates and program. Our intention is to design an efficient system for collecting and analyzing data and we have been looking closely at a
system being considered for our teacher education programs. Now that our program has stabilized, the
data we collect from now on will help us to do both short-term and longitudinal analyses to improve our
courses and program. In the preparation of this accreditation report, we realize that we need to
systematically involve more of our faculty in the each of assessments. Currently, assessments are limited to instructors who teach the courses that are tied to the assessments. While this is a convenient way to conduct our assessments, an additional reviewer or two would increase the validity of our ratings. We will consider this challenge at our next department retreat. We look forward to growing as a program as we continue to learn from our assessments and continue to improve our assessment system.
13) Other important information:
For nearly half a century, the Department of Educational Technology (ETEC) at the University of Hawaii at Manoa has provided a dynamic and high quality program for educators seeking innovative ways to integrate emerging technologies for teaching and learning. The department’s main target group is post-baccalaureate students committed to the improvement of instruction and of learning. For these graduate students, a set of strong objectives has been established, and appropriate activities devised to ensure high levels of competencies of the program graduates. Additionally, the departmental efforts go toward contributing and strengthening the teacher preparation programs of the College of Education, on both the undergraduate and post-baccalaureate levels.
The ETEC specialty has been designed to encourage intellectual excellence and participation in a scholarly community. Initial required seminar courses are completed within a cohort system where students entering at the same time take these seminars together to build support and professional community. Emerging technologies are actively deployed to support scholarly networking as well as learning and teaching. Students intern with faculty on research projects and college teaching. The ETEC department has made a serious commitment to retention and completion to degree for all entering ETEC students by creating a supportive and collegial learning environment. Our evaluation and research efforts on program design support the success of this model for student retention and achievement. Our graduates are in high demand, serving in educational leadership positions throughout the state, and in education, military, corporate and non-profit organizational training and development nationally and internationally.
The department of Education Technology prides itself on staying up-to-date with the latest technologies, while centering on the human side through the development of learning communities. Through a series of seven grants from the US Department of Education, the ETEC Department has brought in $10.1 million in federal monies and matching funds. LEI Aloha, the ETEC’s grant group uses these funds to improve technology integration and enhancement in the Department, College, and University System. The benefits have been immense -- supplying support for training, equipment, course development, graduate assistantships to name a few. These funds have provided opportunities for ETEC to stay on the leading edge of the field and give us an enhanced state, national and international reputation.