Formative Visual Assessment; Semiotics, Critique and Drawing Foundations

Architecture
Simon Bussiere and Lance Walters, Architecture, and Ara Laylo, Art

Targeted Institutional Learning Objectives:
(a) Think critically and creatively
(b) Communicate and report

In such a rapidly evolving contemporary visual environment, how can academic assessment methods evolve to allow 21st century design students to more effectively learn to both draw and understand images and symbols with greater confidence and fulfillment? What environmental, social, cultural or technical factors are the most critical in shaping how students learn these important skills? And how should those factors be adapted or adjusted from one project, semester or cohort to the next?

This assessment project builds on two new courses in the arts and architecture departments to develop and implement a process that encourages greater peer-to-peer exchange and interaction in the development of foundational drawing and composition skills. Two of the project instructors are currently teaching a new course (ARCH 490 Drawing in the Field) that explores a series of fundamental drawing exercises through site-visits in unique neighborhoods in Honolulu. From week to week, with increasingly complex and progressive media and distinct sites, students are challenged with different modes of experiential learning and academic assessment. Beginning with theoretical foundations in design drawing/sketching methods, then in the critical analysis of place and the synthesis and reinterpretation of visual features, students interrogate ways to draw what they observe, then learn from consequences of choices they’ve made in selecting study subjects and drawing methods while recording site-visits. Discussion around perceived “mistakes and successes” in the study and representation of those environments and in the skills needed to do so follow each visit. It is required that students make key temporal and media-based decisions and draw with intention both while on-site and in a classroom setting. Students are also required to take initiative in developing a unique style of their own in the course of the semester.

Taken together, this interdisciplinary project explores immersive and collective modes of review, critique and assessment while pushing students to be more individually accountable for the recording and documentation of their work. In focusing on flexible experimentation and subsequent feedback and formative assessment, students will develop skills in visual awareness and communication, understanding composition and editing, and in forming and verbalizing critical arguments.

• The two project faculty in architecture will conduct a review of the BEnvD program’s student learning outcomes in foundation drawing and discuss how relevant, meaningful, up to date, and how well those learning outcomes are currently written in policy across the design-foundations sequence.

• The two project faculty in architecture will also review the BEnvD program/curriculum map to identify gaps and possible redundancy in student learning outcomes, and will develop a memo/report to the Curriculum Committee, the Architecture Faculty Senate (AFS) and the Dean of the School of Architecture that identifies areas for improvement (where greater coherence and quality can be addressed.)

• The three project faculty will facilitate a broader faculty discussion and will provide feedback/recommendations on potential revisions to evaluation criteria for other courses that address the development of basic drawing and visualization skills. In a Curriculum Committee and/or AFS meeting, the project faculty will facilitate a session on ways to improve assignments and teaching/assessment approaches across the design-foundations sequence.

East Asian Languages and Literatures
Song Jiang, Li Jiang, Haidan Wang, Mingbao Yue, East Asian Languages and Literatures

View the project poster.

Targeted Institutional Learning Objectives:
(a) Think critically and creatively
(b) Conduct research

Project description. Elevating critical thinking of students has been an active practice in colleges and universities across the U.S., and one of the important institutional learning objectives (ILO) encompassed by the UHM campus wide. As a habit of mind, featuring the explorations of issues, ideas, artifacts and events comprehensively by learners prior to accepting or formulating opinions or conclusions (Association of American Colleges and Universities website), the critical thinking ILO may be translated differently into assignments and assessments in individual curriculum and program map. The majority of Chinese faculty has come to realize the significance of critical thinking to the program assessment, and the benefits of a unified rubric to the curricula articulations. A team of four Chinese faculty who have been teaching courses that identify critical thinking as one of the assessment components is committed to collaborate on creating a holistic rubric that would articulate with the critical thinking rubric developed by AAC&U in order to reinforce two of the Students Learning Outcomes regarding writing and research in Chinese BA curriculum map. The rubric will help collect data for both student performance evaluations in applicable courses and the annual program assessment mandated by the university.

Specifically, the team will collaborate on a series of discussions on (a) elaborating the critical thinking criteria (CTC) in Chinese BA requirements; (b) exploring approaches to piloting and implementing CTC in the Chinese undergraduate curricula through these followings:

(i) revisiting the Chinese BA curriculum map and identifying the courses and  curricula that address the CTC in the syllabi, focusing on writing intensive courses;
(ii) referring to the AACU rubric and creating a Chinese CTC along with clear iterations of descriptors from benchmarks to capstone, and applying this CTC to the pilot courses;
(iii) suggesting methods of designing meaningful assignments and collecting learning evidence, as well as providing faculty feedback on student assignments in piloted courses;
(iv) reporting the pilot results and facilitating a discussion among all Chinese professorial faculty;
(v)  seeking support from UHM Assessment Office tailored to the needs occurring during the process;
(vi) reaffirming the revised rubric to all disciplines of Chinese curriculum: language, linguistics, and literature survey and content courses, courses on woman studies, film, media and popular culture, etc. 

The ultimate goals of this project are to create a clearly defined, articulate CTC, implementable for Chinese section WI courses, with the aim to encourage Chinese major students to actively identify and challenge their assumptions during learning process in order to explore alternative ways of thinking and acting, and thereby develop critical thinking skills and competence.

Automation of Student Learning Assessment for the ABET Accreditation

Ocean and Resources Engineering
Justin Stopa, Kwok Fai Cheung, and Eva-Marie Nosal, Ocean and Resources Engineering

View the project poster.

Read the project report (PDF).

Targeted Advanced Degree Institutional Learning Objectives:
(a) Demonstrate comprehensive knowledge in one or more general subject areas related to, but not confined to, a specific area of interest
(b) Demonstrate understanding of research methodology and techniques specific to one’s field of study
(c) Apply research methodology and/or scholarly inquiry techniques specific to one’s field of study
(d) Critically analyze, synthesize, and utilize information and data related to one’s field of study
(e) Proficiently communicate and disseminate information in a manner relevant to the field and intended audience
(f) Conduct research or projects as a responsible and ethical professional, including consideration of and respect for other cultural perspectives
(g) Interact professionally with others

Project description. Background. We are reviewed every 6 years by the ABET. In each 6-year cycle we revise and update our learning objectives based on feedback from graduates and engineering/science professionals. We track student progress while each student is enrolled in the department and at least 5 years after graduation to monitor the effectiveness of our program. We monitor student learning outcomes each semester; this is done by both the students (indirect) and instructors (direct). The information collected is used to adapt our graduate program and write reports for our ABET assessments. Continuously adapting our graduate program is an essential component of education.

Project. We are currently in year 4.5 of the 6 -year cycle and will be reviewed by ABET in 1.5 years. This is a critical time in the process because we need to re-assess and update our student learning objectives (SLOs) since ABET recently changed its requirements. In Fall 2019, we changed from a paper survey to a digital version (Google Forms) to assess the ORE SLOs. We found the digital surveys improved automation of the information collected; reducing man-hours needed to record, analyze, and share (plot) the results. We would like to improve the data-collection surveys for both students and instructors.

In addition to the surveys, we would like to improve the automation of a student’s progress through the M.S. program through an on-line system. This will reduce paper and resources as well as HR support needed to monitor each student. Specifically, each student needs to meet the following criteria: 1) pre-program requirements (5 specific courses and 45 math/engineering courses), 2) entrance exam, and 3) UHM Forms 1-3 related to the M.S. thesis. Additionally, we would like to monitor other information such as projects contributed to or funding that might be supporting their research. After graduation, we monitor each student for at least 5 years. This is typically done through surveys using the same on-line system would streamline the entire progress. This tool would improve our ability to monitor the effectiveness of our program and reduce the man-hours needed to do so.

Goals:

1) Develop and refine student and faculty digital surveys to assess the SLOs.
2) Automate procedures to share the results of the student and faculty surveys with the ORE
faculty.
3) Develop and refine alumni and alumni superior [supervisor] surveys to assess the department’s
educational objectives.
4) Develop and refine the graduating student ORE survey to solicit feedback from students.
5) Improve the automation of monitoring individual student progress to ensure that our
department’s pre-program and program requirements adhere to the ABET standards.

Oral Communication for Language Teachers: Assessment Rubric Development

Second Language Studies
Dustin Crowther, Betsy Gilliland, and Kenton Harsch, Second Language Studies, and Cari Ryan, General Education Office

View the project poster.

Read the project report (PDF).

Targeted Institutional Learning Objective:
Communicate and report

Project description. The Department of Second Language Studies (SLS) BA major includes two electives, SLS 312 and SLS 313, that prepare students to teach key skills (reading, writing, speaking, listening) to learners of a second language. As pre-service teachers, their oral communication development must be framed within the field of second/foreign language teaching. As such, assessment of students’ oral communication requires not only consideration of their immediate ability to communicate clearly to their instructor and peers, but also a projection to how clearly they will communicate to second/foreign language learners (their future students). Importantly, both audiences have different needs and impose different communicative challenges for students. Our project’s goal is to develop the “Oral Communication for Language Teachers” rubric, which integrates University of Hawai’i Oral Communication Learning Objectives within a discipline-specific rubric for assessing students’ oral communication skills in the varied assignments of the two courses, including oral presentations and teaching demonstrations.

Rubric development will utilize an iterative process of discussion and workshopping, drawing upon the experiences of a range of contributors: SLS faculty, SLS graduate students, and General Education specialists. Our research team features three faculty/staff in SLS (Crowther, Gilliland, and Harsch), all with experience in both teacher training and second/foreign language instruction, as well as the educational specialist in the UHM General Education Office (Ryan), with significant experience working with the university’s Oral Communication Board. In the rubric development process, we also include graduate students.

The project will entail several group discussion and workshop sessions. Through discussion, we will draft a rubric that will be workshopped by faculty/staff and graduate students using videos of previous SLS 312/313 presentations for reference. In addition, we will include undergraduate students in these workshops, as they are the target population of assessment. Following an iterative process, the rubric will be revised through additional discussion, with an aim to have a working rubric ready to be implemented into SLS 313 for the Fall 2020 semester.