Degree Program Assessment Reports : View Reports

Unit: Oceanography
Program: Global Environmental Science (BS)
Degree: Bachelor's
Date: Mon Nov 09, 2020 - 4:07:06 pm

1) Program Student Learning Outcomes (SLOs) and Institutional Learning Objectives (ILOs)

1. Define and explain the basic principles and concepts of chemistry, physics, biology, calculus, geology, geophysics, meteorology, and oceanography.

(1a. General education, 1b. Specialized study in an academic field)

2. Apply their understanding of the fundamentals of science and mathematics to the description and quantification of the interactions of the atmosphere, hydrosphere, lithosphere, and biosphere, including humans.

(1a. General education, 1b. Specialized study in an academic field, 3c. Stewardship of the natural environment)

3. Employ the scientific approach to problem solving, and hypothesis formation and testing.

(1b. Specialized study in an academic field, 2a. Think critically and creatively, 2b. Conduct research)

4. Conduct scientific research, and analyze and evaluate results.

(1b. Specialized study in an academic field, 2a. Think critically and creatively, 2b. Conduct research)

5. Demonstrate information literacy by collection and evaluation of scientific literature.

(1a. General education, 1b. Specialized study in an academic field, 2a. Think critically and creatively, 2b. Conduct research)

6. Express themselves clearly and concisely in written form.

(1a. General education, 1b. Specialized study in an academic field, 2c. Communicate and report)

7. Demonstrate skilled delivery of well organized informal and formal oral presentations.

(1a. General education, 1b. Specialized study in an academic field, 2c. Communicate and report)

2) Your program's SLOs are published as follows. Please update as needed.

Department Website URL: https://www.soest.hawaii.edu/oceanography/GES/curriculum.html#OUTCOME

Student Handbook. URL, if available online:

Information Sheet, Flyer, or Brochure URL, if available online:

UHM Catalog. Page Number:

Course Syllabi. URL, if available online:

Other:

3) Please review, add, replace, or delete the existing curriculum map.

Curriculum Map File(s) from 2020:

File (10/08/2020)

4) For your program, the percentage of courses that have course SLOs explicitly stated on the syllabus, a website, or other publicly available document is as follows. Please update as needed.

1-50%

51-80%

81-99%

100%

5) Does the program have learning achievement results for its program SLOs? (Example of achievement results: "80% of students met expectations on SLO 1.")(check one):

Yes, on some(1-50%) of the program SLOs

Yes, on most(51-99%) of the program SLOs

Yes, on all(100%) of the program SLOs

6) Did your program engage in any program learning assessment activities between November 1, 2018 and October 31, 2020?

Yes

No (skip to question 17)

7) What best describes the program-level learning assessment activities that took place for the period November 1, 2018 and October 31, 2020? (Check all that apply.)

Create/modify/discuss program learning assessment procedures (e.g., SLOs, curriculum map, mechanism to collect student work, rubric, survey)

Collect/evaluate student work/performance to determine SLO achievement

Collect/analyze student self-reports of SLO achievement via surveys, interviews, or focus groups

Use assessment results to make programmatic decisions (e.g., change course content or pedagogy, design new course, hiring)

Investigate other pressing issue related to student learning achievement for the program (explain in question 8)

Other:

8) Briefly explain the assessment activities that took place since November 2018.

Data collection: We use a rubric to evaluate graduating students' required written thesis and oral presentation at the end of every semester. We also conducted a focus group interview with 6 graduating students and alumni in July 2019.

We reflected on the assessmenet results and the focus group's feedback and are piloting an undergraduate research project website for all GES majors in fall 2020 to enhance students' undergraduate research journey from the day they start in the major to the day they graduate. The website is broken down into stages (1-5 for now) that address different milestone points in the individual student's progression on their research experience trajectory.

9) What types of evidence did the program use as part of the assessment activities checked in question 7? (Check all that apply.)

Artistic exhibition/performance

Assignment/exam/paper completed as part of regular coursework and used for program-level assessment

Capstone work product (e.g., written project or non-thesis paper)

Exam created by an external organization (e.g., professional association for licensure)

Exit exam created by the program

IRB approval of research

Oral performance (oral defense, oral presentation, conference presentation)

Portfolio of student work

Publication or grant proposal

Qualifying exam or comprehensive exam for program-level assessment in addition to individual student evaluation (graduate level only)

Supervisor or employer evaluation of student performance outside the classroom (internship, clinical, practicum)

Thesis or dissertation used for program-level assessment in addition to individual student evaluation

Alumni survey that contains self-reports of SLO achievement

Employer meetings/discussions/survey/interview of student SLO achievement

Interviews or focus groups that contain self-reports of SLO achievement

Student reflective writing assignment (essay, journal entry, self-assessment) on their SLO achievement.

Student surveys that contain self-reports of SLO achievement

Assessment-related such as assessment plan, SLOs, curriculum map, etc.

Program or course materials (syllabi, assignments, requirements, etc.)

Other 1:

Other 2:

10) State the number of students (or persons) who submitted evidence that was evaluated. If applicable, please include the sampling technique used.

From fall 2018 to spring 2020, the program evaluated 14 written theses and public oral presentations via faculty survey forms.

Fall 2018: 3
Spring and Summer 2019: 8
Fall 2019: 1
spring 2020: 2

Also, six students participated in a focus group discussion in July 2019.

11) Who interpreted or analyzed the evidence that was collected? (Check all that apply.)

Course instructor(s)

Faculty committee

Ad hoc faculty group

Department chairperson

Persons or organization outside the university

Faculty advisor

Advisors (in student support services)

Students (graduate or undergraduate)

Dean/Director

Other: independent faculty evaluators

12) How did they evaluate, analyze, or interpret the evidence? (Check all that apply.)

Used a rubric or scoring guide

Scored exams/tests/quizzes

Used professional judgment (no rubric or scoring guide used)

Compiled survey results

Used qualitative methods on interview, focus group, open-ended response data

External organization/person analyzed data (e.g., external organization administered and scored the nursing licensing exam)

Other:

13) Summarize the results from the evaluation, analysis, interpretation of evidence (checked in question 12). For example, report the percentage of students who achieved each SLO.

The sources of evidence of Program Learning Outcomes (PLO) Achievement are from Thesis and Oral Presentation Evaluation and Student Self-Assessment. We collected results every semester and summarized the results for the current reporting period (2018-2020), as well as from (2014-2018)

	Before 2018	Before 2018	2019	2018-2020	2018-2020	Final Achievement
% of students who scored proficient level or higher	Written Thesis Evaluation	Oral Presentation	Student Self-Assessment	Oral Presentation (2018-2020)	Written Thesis Evaluation	(Evidence source)
N	28	44	6	14	8
PLO1 Basic principles/concepts			100%			100% (self-assessment)
PLO2 Apply the fundamentals of science			100%			100% (self-assessment)
PLO3 Employ the scientific approach (Form Hypothesis)	93%		100%		86%	86% (2018-2020 Thesis Scores)
PLO4 Conduct Research, Analyze, & Evaluate Results	93%	100%	100%	100%	95%	98% (2018-2020 Oral and Thesis Scores)
PLO5 Information Literacy	86%		100%		100%	100% (2018-2020 Thesis Scores)
PLO6 Written Communication	93%		83%		96%	96% (2018-2020 Thesis Scores)
PLO7 Oral Communication		100%	67%	100%		100% (2018-2020 Oral Scores)

I. Oral Presentation Evaluation Results from 2018-2020 (n=14)

- 100% of evaluated students met or exceeded the expectations of the oral presentation for Q1 through Q8.

Percentage of scores that are at least 2 points - Presentation
Q1.	Q2.	Q3.	Q4.	Q5.	Q6.	Q7.	Q8.
100%	100%	100%	100%	100%	100%	100%	100%

II. Written Thesis Evaluation Results from 2018-2020 (n=8)

- 100% of evaluated students met or exceeded the expectations of the written thesis for Q2, Q4, Q5, Q6, and Q7.

- 86% of evaluated students met or exceeded the expectations of the written thesis for Q1, Q3, and Q8.

Percentage of scores that are at least 2 points - Thesis
Q1.	Q2.	Q3.	Q4.	Q5.	Q6.	Q7.	Q8.
86%	100%	86%	100%	100%	100%	100%	86%

The Alignment Between Rubric Dimensions and GES Program Student Learning Outcomes are as follows

PLO 3

Scientific Approach (Hypothesis)

PLO 4

Conduct Research, Analyze, evaluate results

PLO 5

Info Lit

PLO 6

Writing

PLO 7

Oral presentations

T1: Hypothesis

T2: Methods

T3: Results

T4: Tables/Graphs

O4: Methods

O5: Results Presentation

T5: Citation

T1 - T8

O1 - O8

Note. T = Thesis evaluation dimension; O = Oral presentation dimension

Results from the six person July 2019 focus group. Students were asked to self evaluate their achievement on the 7 program SLOs. The results are:

Program Student Learning outcomes	% of the students reported proficient or exemplary (n =6)
PLO 1. Basic principles/concpets	100%
PLO 2. Apply the fundamentals of science	100%
PLO3. Employ the scientific approach	100%
PLO4. Conduct Research	100%
PLO5. Information literacy	100%
PLO6. Written communication	83%
PLO7. Oral Communication	67%

14) What best describes how the program used the results? (Check all that apply.)

Assessment procedure changes (SLOs, curriculum map, rubrics, evidence collected, sampling, communications with faculty, etc.)

Course changes (course content, pedagogy, courses offered, new course, pre-requisites, requirements)

Personnel or resource allocation changes

Program policy changes (e.g., admissions requirements, student probation policies, common course evaluation form)

Students' out-of-course experience changes (advising, co-curricular experiences, program website, program handbook, brown-bag lunches, workshops)

Celebration of student success!

Results indicated no action needed because students met expectations

Use is pending (typical reasons: insufficient number of students in population, evidence not evaluated or interpreted yet, faculty discussions continue)

Other:

15) Please briefly describe how the program used its findings/results.

GES students had high achievement on the program's student learning outcomes. However, the focus group data reveals that the undergraduate research is a challenging journey for students even with the robust curricular and co-curricular support that the program offers. As a result, the following actions were taken:

1. Better use of assessment tools

To help faculty resarch mentors better understand the program's expectations, the thesis and oral presentation rubrics were shared with all mentors for them to use in helping guide the students' thesis and oral presentation preparations.

2. Creation of a Content Management System for the GES Research Experience

There are many different elements of the GES research experience that students must navigate and complete to be successful. These elements must also be assessed and evaluated to determine if PLOs and SLOs are being met. Some of these elements are curricular and others are co-curricular. What we need is a way to manage all these curricular and co-curricular research-related elements and their assessment in one place. With this in mind, using UHM's Laulima content management system, we've created an interface that allows the program and faculty to deliver all the necessary elements of the GES research experience to the students at a one-stop shop. We launched this Laulima developmental site in Fall 2020 with the input of faculty, current GES students, and alumni. This site is a work in progress and is being refined, sometimes in real time, by both student and faculty input and feedback.

The following curriculuar and co-curricular experiences are already being offered and assessed:

1) Three research-experience courses - OCN 100, OCN 399a, and OCN 399b

2) Practice oral presentation preparation

3) Draft thesis review experience

4) Onboarding survey of new GES majors

5) Post-graduate survey of GES graduates using the Undergraduate Research Student Self-Assessment

16) Beyond the results, were there additional conclusions or discoveries? This can include insights about assessment procedures, teaching and learning, and great achievements regarding program assessment in this reporting period.

We submitted a manuscript to the Journal of Geoscience Education (with Dr. Yao Hill) based on the program evaluation discussed in this report.

1) Program Student Learning Outcomes (SLOs) and Institutional Learning Objectives (ILOs)

1. Define and explain the basic principles and concepts of chemistry, physics, biology, calculus, geology, geophysics, meteorology, and oceanography.

2. Apply their understanding of the fundamentals of science and mathematics to the description and quantification of the interactions of the atmosphere, hydrosphere, lithosphere, and biosphere, including humans.

3. Employ the scientific approach to problem solving, and hypothesis formation and testing.

4. Conduct scientific research, and analyze and evaluate results.

5. Demonstrate information literacy by collection and evaluation of scientific literature.

6. Express themselves clearly and concisely in written form.

7. Demonstrate skilled delivery of well organized informal and formal oral presentations.

2) Your program's SLOs are published as follows. Please update as needed.

3) Please review, add, replace, or delete the existing curriculum map.

4) For your program, the percentage of courses that have course SLOs explicitly stated on the syllabus, a website, or other publicly available document is as follows. Please update as needed.

5) Does the program have learning achievement results for its program SLOs? (Example of achievement results: "80% of students met expectations on SLO 1.")(check one):

6) Did your program engage in any program learning assessment activities between November 1, 2018 and October 31, 2020?

7) What best describes the program-level learning assessment activities that took place for the period November 1, 2018 and October 31, 2020? (Check all that apply.)

8) Briefly explain the assessment activities that took place since November 2018.

9) What types of evidence did the program use as part of the assessment activities checked in question 7? (Check all that apply.)

10) State the number of students (or persons) who submitted evidence that was evaluated. If applicable, please include the sampling technique used.

11) Who interpreted or analyzed the evidence that was collected? (Check all that apply.)

12) How did they evaluate, analyze, or interpret the evidence? (Check all that apply.)

13) Summarize the results from the evaluation, analysis, interpretation of evidence (checked in question 12). For example, report the percentage of students who achieved each SLO.

14) What best describes how the program used the results? (Check all that apply.)

15) Please briefly describe how the program used its findings/results.

16) Beyond the results, were there additional conclusions or discoveries? This can include insights about assessment procedures, teaching and learning, and great achievements regarding program assessment in this reporting period.

17) If the program did not engage in assessment activities, please justify.