FAQ: Program-level Learning Assessment

FAQs

What is the goal of program-level learning assessment?

The goal is improved student learning and better programs. We rely on this definition of program assessment: “The systematic collection, review, and use of information about educational programs undertaken for the purpose of improving student learning and development.” (Palomba & Banta, 1999). The unit of interest is the program and the focus is on student learning.

It is not individual student, faculty, or course evaluation.

What’s the difference between program-level learning assessment and program review?

Program assessment is an ongoing process designed to monitor and improve student learning at the program (e.g., degree) level. Every other year, programs describe their assessment-related activities in an assessment report. The reports are available on our website.

Program review is a comprehensive review of the programs in a college. It takes place every 5-7 years. Program assessment and programs’ biennial reports are one part of program review. To learn more about program review, visit website of the Office of the Vice Provost for Academic Excellence and click on program review.

Who has to do program-level learning assessment?

All academic degree programs, co-curricular programs, and student support units.

What do terms such as “PLO” and “Used Results” mean?

PLO, SLO, ILO, CLO. These refer to learning outcomes: program student learning outcome (PLO); student learning outcome (SLO); institution student learning outcome/objective (ILO); course student learning outcome (CLO).

Curriculum Map: a graphic (often a matrix) or a description that shows the coverage of each PLO in the curriculum, which includes non-course requirements such as an internship, clinical practice, oral defense.

Collected Evidence, Direct Evidence, Indirect Evidence: Programs typically collect both direct and indirect evidence/data of student learning. Direct evidence refers to evidence/data based on actual behaviors and products. Direct evidence demonstrates the learning that has occurred. Examples: exams, course work, oral performances, research reports, dissertations. Indirect evidence refers to evidence/data based on self-reports and are typically collected using surveys.

Results: Results of assessment activities vary and include a broad scope of findings. Results can include student learning outcome achievements; the findings from a syllabi or curriculum map review; the findings from a survey of student and faculty’s perceived needs or experience, and so on.

Achievement Results: These are specific results that provide the student learning outcome achievement findings and are based on direct evidence (e.g., student work products and performances).

Used Results: A specific use of results (from direct and/or indirect assessment of student learning) such as changes to courses, curriculum, students’ out-of-course experiences (e.g., advising, workshops, lectures), policies, assessment procedures, and celebrations of student success.

Does assessment require approval from the Committee on Human Studies/Institutional Review Board (IRB)?

No. Program-level learning assessment is excluded from Institutional Review Board (IRB) review because it does not meet the definition of research. The Code of Federal Regulations found at 45 CFR 46.102(d) defines research in part as, “a systematic investigation, including research development, testing and evaluation, designed to develop or contribute to generalizable knowledge.” Program assessment per se does not meet this definition. In addition, investigations undertaken for program evaluation and internal use are excluded from IRB review. 

Important: If a person or program plans to disseminate or publish data beyond program improvement or accreditation purposes then we recommend consulting with IRB.

Programs that collect student work for the purpose of program assessment do not need student consent provided all personally identifiable information is removed and the results are used for internal university purposes.

Regardless of IRB, everyone involved must strictly adhere to the principles of ethical inquiry/research. 

We ask students to complete surveys–do these surveys count?

Self-reports via surveys, interviews, focus groups, etc., are valuable. They provide tremendous insight into student learning when coupled with direct measures of learning such as an exam or students’ projects. However, self-report data (often collected through surveys and interviews) are not sufficient evidence of learning because they are perceptions of learning and not direct evidence of learning. Direct evidence of student learning is required for program-level learning assessment. See Choose a Method for more information about direct and indirect measures.

Can the Assessment and Curriculum Support Center do our assessment for us?

It really is in the best interest of the program if it completes its own program-level student learning assessment. Program-level learning assessment, when driven by faculty members, is an opportunity for faculty to closely look at the alignment between courses and the program and the program’s effect on student learning. During the assessment process, faculty members speak with each other, students, and various program stakeholders about what they teach, their expectations, and how they teach. Faculty members are typically in the best position to figure out what to change and to implement that change, when needed, to improve student learning. These benefits would be lost if the Assessment and Curriculum Support Center took on program assessment for a program. However, a collaboration of the program and the Assessment and Curriculum Support Center is strongly recommended. The Assessment and Curriculum Support Center can provide technical expertise on learning assessment, evaluation, evidence-based interventions into the learning environment, and curriculum development support.

What can the Assessment and Curriculum Support Center do for programs?

The Assessment and Curriculum Support Center is here to support your program-level learning assessment endeavors–from establishing learning outcomes to using results for student learning improvement. Think of us as your assessment consultants and technical experts.

  • We offer an array of assessment workshops throughout the school year. If you are unable to attend, we have the PowerPoint and handouts available on our website. Link to Workshop/Events
  • Our web site also contains general assessment information, “how to” guides,  helpful resources and links, and  templates.
  • We are also available by appointment for individual program consultation; we can provide advice on how to assess your program, what to assess, and how to act on the assessment results.

Is the Assessment and Curriculum Support Center in charge of course evaluations?

No. The Office of Faculty Development and Academic Support (OFDAS) offers both mid-semester and end-of-course evaluations (CES). Although OFDAS and the Assessment Office work together, OFDAS focuses on course-level support while the Assessment and Curriculum Support Center focuses on program- and institution-level support.

What’s done with the learning assessment reports?

The Assessment and Curriculum Support Center uses the information from the reports in several ways. The Assessment and Curriculum Support Center:

  • locates examples of exemplary assessment practices that it can share with the faculty-at-large (a program’s permission is sought before the Assessment and Curriculum Support Center uses it as an example);
  • identifies issues and concerns that need attention and uses those to guide Assessment and Curriculum Support Center program development (e.g., when an analysis of reports reveals the need for more attention to curriculum mapping, the Assessment and Curriculum Support Center offers workshops on that topic);
  • summarizes the reports and conveys that summary to the Faculty Senate, Vice Provost for Academic Excellence, and the Western Association of Schools and Colleges (WASC). The summary serves as primary support in Mānoa’s case for re-accreditation and meets reporting requirements set by WASC.
  • conveys the reports to the Office of the Vice Provost for Academic Excellence for use as part of Program Review.

How can I get help completing our program’s report?