Are you teaching a course that emphasizes critical thinking?
Want to be involved with institutional learning assessment at Mānoa?
Interested in using authentic student work for assessment (and not a standardized test)?
The University of Hawai’i at Mānoa Assessment and Curriculum Support Center, General Education Office, and the Institutional Learning Objectives Implementation Committee have assisted the campus in examining student achievement with the aim to use findings for institutional and program improvement and, ultimately, for students’ benefit. We have coordinated with the Multi-State Collaborative (MSC) to Advance Quality Student Learning or VALUE Project in order to share ideas about teaching and learning across states and pool resources.
Project History
In 2018, we focus on critical thinking (below) and oral communication.
Recent projects include the following:
Critical Thinking: 2016 results
Quantitative Reasoning: 2017 results and Quantitative Reasoning: 2016 results
Written Communication: 2016 results
Visit the Institutional Reports page for earlier projects.
Purpose: Critical Thinking Project
This project supports and is in line with Mānoa’s ongoing institutional learning assessment projects. The purpose of this project is to (a) investigate student learning and (b) demonstrate that colleges and universities can evaluate learning without using standardized exams. It uses existing evidence (i.e., authentic student work in the form of written work, exams completed as part of the curriculum, etc.) for assessment. The results are reported in the aggregate (no individual faculty or student results) and are used for institutional- and program-level decision making.
This project is in collaboration with thirteen other states for the Multi-State Collaborative to Advance Quality Student Learning (MSC)/VALUE Project.
Significance of Mānoa Participating in the MSC Project
Participating in the MSC project is an important undertaking for Mānoa as we continue to develop long-term institutional-level assessment strategies. The MSC project allows us to collect evidence of student learning, evaluate that evidence, and use the findings for institutional improvement and future WASC accreditation reaffirmation.
The process is similar to what Mānoa faculty are already doing and promotes faculty members as the drivers of the assessment process. It also offers the opportunity to help align our assessment practices with other institutions and have our student work evaluated by nationally trained faculty scorers.
This multi-state approach and effort to understanding student learning and outcomes assessment allows Mānoa faculty to engage in conversations of student performance with faculty across the nation. Additionally, the external mechanism for scoring student work provides results that are useful in reporting our outcomes data to WASC.
Interested in Participating? Simpy submit:
- Coversheet
- Assignment
- Guidelines
Who’s Participating?
Mānoa faculty and instructors who teach courses that emphasize critical thinking skills are invited to participate.
What Do Faculty Participants Do?
Participating faculty and instructors . . .
- Submit a completed assignment from their randomly selected students (Assessment and Curriculum Support Center randomly selects students);
- Submit assignment guidelines;
- Submit a coversheet and simple matrix detailing the assignment context;
- Participate in a workshop or read materials related to critical thinking assignment design and pedagogy.
How Do Participants Submit Materials?
Faculty submit materials to the Assessment and Curriculum Support Centere: email airo@hawaii.edu or send via campus mail to Crawford 230.
We gladly scan and shred/return hard copies!
What Happens with the Submitted Materials?
Trained redactors remove all personal information (student name, faculty name, course, etc.) and a unique code is assigned to each piece. Redacting keeps all student work and faculty assignment guidelines anonymous. All information is stored in a secure format/locked area and electronic documents are password protected. The material is uploaded to a secure website. The faculty who review the student work (during summer) do not know the student names, faculty names, or campus.Nationally trained scorers, including Mānoa faculty members, evaluate the student work using the Critical Thinking VALUE rubric.
How Will the Results be Used?
Only Mānoa receives our campus’s results. Individual faculty participants are given only their students’ results. The aggregated results are distributed to all faculty members and instructors who participate and to various faculty groups: the General Education Committee and its Boards, the Mānoa Faculty Senate Committee on Educational Effectiveness, and the Institutional Learning Objectives Implementation Committee. The aggregated results are also evidence for Mānoa’s accreditation reaffirmation.
We remove all personal information.
Project Benefits
- It uses authentic student work (not standardized, multiple-choice or timed exams)..
- It employs the VALUE rubrics that Mānoa faculty have already used.
- It encourages Mānoa faculty to be the primary drivers of institutional learning assessment at Mānoa.
- Mānoa faculty can use the results to better understand students’ level of achievement.
- Only Mānoa receives its students’ results and these results are reported in the aggregate (that is, many campuses are collaborating but campuses receive only their results). The Assessment and Curriculum Support Center summarizes each projects’ results.
- Mānoa faculty have the opportunity to assist with the scoring and participate in discussions about student performance with faculty colleagues from across the U.S.
- Mānoa faculty are offered training in assignment design and use of rubrics to evaluate learning. To learn about about training opportunities and to request training, contact the Assessment and Curriculum Support Center (airo@hawaii.edu).
Other FAQs
What is “institutional-level assessment” of student learning?
Institutional-level assessment of student learning looks across many students in many majors to form an understanding of undergraduate students’ skills and knowledge. In 2016, Mānoa investigated written communication, critical thinking, and quantitative reasoning competencies.In 2017, Mānoa investigated quantitative reasoning and oral communication competencies. In 2018, Mānoa is completing an oral communication project and investigating critical thinking.
Nationally trained scorers evaluate work with Value Rubrics.
Why take part?
First, to gain an institutional perspective of how well Mānoa undergraduates as a whole are prepared. After faculty have aggregate results—across many students in many majors—institutional-level changes can be made to improve the undergraduate experience.
Second, Mānoa is required to evaluate undergraduate student achievement in five areas and use the overall results for improvement in order to remain accredited and for Mānoa students to be eligible to receive federal financial aid.The five areas are critical thinking, information literacy, oral communication, quantitative reasoning, and written communication.
Is this a personal research project?
No. This is an institutional learning assessment project that has been approved by President and Interim Chancellor David Lassner.
Does this project require IRB approval?
No. Learning outcomes assessment is not considered research that involves human subjects according to the Protection of Human Subjects regulation (section 46.102(d)). However, we do follow the regular procedures for privacy and securing of confidential information as is expected of all Mānoa employees. Students’ work is redacted to ensure confidentiality of students and faculty.
How are courses and students selected?
- We identify potential courses based on department chair/faculty recommendations of appropriate courses and courses’ general education designations. We primarily target 100-level courses and 400-level because we are looking for assignments completed by first-year students and seniors in order to gain a general understanding of entry-level and exit-level achievement levels.
- After identifying target courses and communicating with faculty instructors, we create a pool of eligible, enrolled students who meet the following requirements:
- Seeking a first undergraduate degree
- Either (a) completed 15 or fewer credits [100-level courses] or (b) completed 90 or more credits [300- and 400-level courses]
- After identifying the pool of eligible students, we use simple random sampling and proportional random sampling to obtain a representative sample across the curriculum and across other desired characteristics (e.g., gender, ethnicity, Pell eligibility status). That is, we take appropriate steps and measures to ensure that students are randomly sampled; we select no more than 10 students from a single faculty member; we select an appropriate proportion from each targeted course (to capture a variety of student work across the curriculum), and ensure that only one piece of work is submitted for a particular student.
- Only UH-M receives UH-M’s results.
Do we need to obtain permission and consent from students?
Prior consent from students is not needed. This is similar to other campus efforts and regular reporting of aggregate information (e.g., students’ SAT scores, GPAs, graduation and retention rates).However, students may opt out: see also the Catalog section, Collection of Student Work for Assessment Purposes.
Will this affect a student’s grade or graduation?
No. Grades are not affected. Graduation is not affected. The faculty who review the student work samples do so after the course has ended. Faculty scorers do not know the students’ names.
How can a scorer without content expertise on the work that is being scored effectively evaluate the work?
For this institutional-level project, the scoring is “not about the content of the student work, for which the faculty are indeed the best judges, but rather about the underlying abilities associated with the learning outcome. . . . The scoring is about whether the student can communicate about it in written form, oral form, etc., demonstrate critical thinking skills, ethical considerations, etc.” (Terrel Rhodes, Vice President, Office of Quality, Curriculum, and Assessment and Executive Director of VALUE, at AAC&U)
The faculty participating in the project indicate whether their assignment is aligned with the rubric.
See here for more FAQs from the Multi-State Collaborative (MSC).
Are there any risks?
No. Student names and faculty names and all personal identifiers are removed and personal information is kept strictly confidential to eliminate the risk of being individually identified by scorers. All information is stored in a secure format/locked area and electronic documents are password protected. The faculty who review the student writing do so in summer and they do not know the student names, faculty names, or campus.
Can I participate even if I did not receive an invitation?
Yes. If you teach an upper-division course that emphasizes critical thinking and enrolls seniors contact the Assessment and Curriculum Support Center (airo@hawaii.edu) to inquire about current project activities.
VALUE Rubrics:
Contact
Please direct all questions and send materials to the Assessment and Curriculum Support Center:
Email: airo@hawaii.edu
Send hard copy materials via campus mail to Crawford 230.
Mahalo on the behalf of the Assessment and Curriculum Support Center, General Education Office, and the Institutional Learning Objectives Implementation Committee.
| Monica Stitt-Bergh, Ph.D., Associate Specialist, Assessment and Curriculum Support Center |
| Christine Beaule, Director, General Education Office; Associate Professor, Spanish |
| Amy Schiffner, Chair, Institutional Learning Objectives Implementation Committee; Associate Professor, Theatre & Dance |