Unit: Office of the Vice Chancellor for Academic Affairs
Program: A1_Assessment_ProgExample
Date: Tue Sep 27, 2011 - 11:22:16 am

1) List your program's student outcomes.

Because the AO does not work with students or offer an academic degree, it has program outcomes.

  1. The AO has in place an infrastructure to sustain a culture of assessment.
  2. Faculty members are aware of opportunities to publish/present on the scholarship of teaching and learning in their field.
  3. Academic degree programs complete the assessment cycle, which includes faculty members using assessment results to improve student learning.
  4. Department leaders and administrators use student learning assessment to guide planning.
  5. The campus community (faculty members, administrators, staff, students) perceives program-level assessment as supporting student learning.

2) Where are your program's student outcomes published? Mark all that apply and include URLs when appropriate.

Program's Website. URL: http://manoa.hawaii.edu/assessment
Student Handbook. URL, if available online:
Information Sheet, Flyer, or Brochure. URL, if available online:
UHM Catalog. Page Number:
Other:
Other:

3) Provide the program's activity map or other graphic that illustrates how program activities/services align with program student outcomes. Please upload it as a PDF.

Activity Map File(s) from 2011:

4) For the period June 1, 2010 to September 30, 2011: State the assessment question(s) and/or assessment goals. Include the student outcomes that were targeted, if applicable.

Two assessments were conducted during this period.

Assessment A

Outcome #1: The Assessment Office has in place an infrastructure to sustain a culture of assessment.

Outcomes #3: Academic degree programs complete the assessment cycle, which includes

faculty members using assessment results to improve student learning.

Questions: a) When should the AO schedule workshops? b) What topics should be covered?

Assessment B

Outcome #3: Academic degree programs complete the assessment cycle, which includes faculty members using assessment results to improve student learning.

Question: How can the AO improve the annual assessment reporting system and streamline communications with assessment coordinators?

5) State the type(s) of evidence gathered to answer the assessment question and/or meet the assessment goals that were given in Question 4.

Assessment A: 4-question online survey

Assessment B: 7‐question online survey

6) State how many persons submitted evidence that was evaluated. If applicable, please include the sampling technique used.

Assessment A: 61. Assessment coordinators who submitted 2010 annual assessment reports and past workshop attendees were invited to take the online survey. A follow up invitation was sent a week later. Of the 194 invitees, 61 completed the survey, resulting in a 31% response rate.

Assessment B: 55. Assessment coordinators who submitted 2010 annual assessment reports were invited to take the online survey. Of the 102 assessment coordinators invited, 55 completed the survey, resulting in a 54% response rate.

7) Who interpreted or analyzed the evidence that was collected. Check all that apply.

Program faculty/staff
Faculty/staff committee
Ad hoc faculty/staff group
Director or department chairperson
Persons or organization outside the university
Students (graduate or undergraduate)
Dean
Other

8) How did he/she/they evaluate, analyze, or interpret the evidence? Check all that apply.

Compiled survey results
Used qualitative methods on interview, focus group, open-ended response data
Scored exams/tests/quizzes
Used a rubric or scoring guide
Used professional judgment (no rubric or scoring guide used)
External organization/person analyzed data (e.g., Social Science Research Institute)
Other:

9) For the assessment questions/goals stated in Question 4, summarize the actual results.

Assessment A

  • Preferred months for workshops are: February, March, and April
  • The top 7 days/times for workshops are:
    • Monday – Early Afternoon
    • Tuesday – Late Morning, Noon, Early Afternoon, Mid‐Afternoon
    • Wednesday – Early Afternoon
    • Thursday – Early Afternoon
  • Suggested Workshop topics include:
    • Specific Assessment Methods: focus groups/interviews, survey design, and rubric design/usage, etc.
    • How to increase faculty engagement in assessment
    • Assessing graduate programs

Assessment B

Over 50% of respondents:

  • Indicated they would like to receive the initial submission request around August 15th.
  • Indicated they would like to receive 2 reminder emails.
  • Perceived the report questions as clear.
  • Felt the report questions allowed for an accurate portrayal of assessment in their program(s).
  • Perceived the online interface was not difficult to use.
  • Believed they knew where to seek help if they had problems inputting their report(s).

Suggestions for improvements to the online system fell into these categories: formatting, table creation, uploads, navigating through the system

10) What was learned from the results?

Assessment A:  Days/times and topics for spring 2011 workshops

Assessment B:  Ways to improve the online reporting system and when assessment coordinators would like to receive communication regarding the submission of the annual assessment reports

11) State how the program used the results or plans to use the results. Please be specific.

Assessment A

  • Spring 2011 workshops were offered in March, April, and May, on Tuesdays/Thursdays, from 1:30 to 2:45 pm
  • New workshops were offered: Focus Group, Interview, or Survey: Which is right for Your Academic/Co-Curricular Program? Increasing Faculty Involvement in Program Assessment, Make the Most of Your Curriculum Map, Graduate Program Assessment: From Student- to Program-level Assessment

Assessment B

  • Improvements to the online system were made.  Users can now
    • Upload multiple curriculum maps and view them immediately
    • Jump to a specific questions
    • Spell check
    • Request feedback on their report
  • The AO sent the initial request for the annual assessment report on August 22.  Reminder emails are scheduled for September and October.

12) Reflect on the assessment process. Is there anything related to assessment procedures your program would do differently next time? What went well?

In the future, the AO will send 2-3 follow-up emails to increase the response rates.

13) Other important information