Unit: Psychology
Program: Psychology (BS)
Degree: Bachelor's
Date: Tue Oct 06, 2015 - 10:08:39 am

1) Institutional Learning Objectives (ILOs) and Program Student Learning Outcomes (SLOs)

1. 1) Psychological Knowledge: Synthesis and Application: Students will be able to describe key concepts and develop a working knowledge of psychologys content domains, e.g., cognition and learning, developmental, biological, and sociocultural, etc.

(1a. General education, 1b. Specialized study in an academic field, 2a. Think critically and creatively, 2b. Conduct research, 2c. Communicate and report)

2. 2) Scientific inquiry and critical thinking: Students will be able to use scientific reasoning to interpret psychological phenomena; demonstrate psychology information literacy; engage in innovative and integrative thinking and problem solving; interpret, design, and conduct basic psychological research; incorporate sociocultural factors in scientific inquiry.

(1a. General education, 1b. Specialized study in an academic field, 2a. Think critically and creatively, 2b. Conduct research, 2c. Communicate and report)

3. 3) Ethical and social responsibility in a diverse world: Students will be able to apply ethical standards to evaluate psychological science and practice; build and enhance interpersonal relationships; adopt values that build community at local, national, and global levels.

(1a. General education, 1b. Specialized study in an academic field, 2a. Think critically and creatively, 3b. Respect for people and cultures, in particular Hawaiian culture, 3c. Stewardship of the natural environment, 3d. Civic participation)

4. 4) Communication: Students will be able to demonstrate effective writing for different purposes; exhibit effective presentation skills for different purposes; interact effectively with others.

(1a. General education, 1b. Specialized study in an academic field, 2a. Think critically and creatively, 2b. Conduct research, 2c. Communicate and report, 3d. Civic participation)

5. 5) Professional development: Students will be able to apply psychological content and skills to career goals; exhibit self-efficacy and self-regulation; refine project- management skills; enhance teamwork capacity; develop meaningful professional direction for life after graduation.

(1a. General education, 1b. Specialized study in an academic field, 2a. Think critically and creatively, 3a. Continuous learning and personal growth, 3c. Stewardship of the natural environment, 3d. Civic participation)

2) Your program's SLOs are published as follows. Please update as needed.

Department Website URL: www.psychology.hawaii.edu
Student Handbook. URL, if available online:
Information Sheet, Flyer, or Brochure URL, if available online:
UHM Catalog. Page Number:
Course Syllabi. URL, if available online: Specific faculty websites may contain course syllabi.
Other:
Other:

3) Please review, add, replace, or delete the existing curriculum map.

Curriculum Map File(s) from 2015:

4) For your program, the percentage of courses that have course SLOs explicitly stated on the syllabus, a website, or other publicly available document is as follows. Please update as needed.

0%
1-50%
51-80%
81-99%
100%

5) Did your program engage in any program learning assessment activities between June 1, 2014 and September 30, 2015?

Yes
No (skip to question 16)

6) What best describes the program-level learning assessment activities that took place for the period June 1, 2014 to September 30, 2015? (Check all that apply.)

Create/modify/discuss program learning assessment procedures (e.g., SLOs, curriculum map, mechanism to collect student work, rubric, survey)
Collect/evaluate student work/performance to determine SLO achievement
Collect/analyze student self-reports of SLO achievement via surveys, interviews, or focus groups
Use assessment results to make programmatic decisions (e.g., change course content or pedagogy, design new course, hiring)
Investigate curriculum coherence. This includes investigating how well courses address the SLOs, course sequencing and adequacy, the effect of pre-requisites on learning achievement.
Investigate other pressing issue related to student learning achievement for the program (explain in question 7)
Other:

7) Briefly explain the assessment activities that took place in the last 18 months.

The ongoing assessment of the Psychology Department’s BA and BS programs has consisted of entrance and exit surveys of Psychology majors, first when they declare as a major and then again when they file for graduation. In our continuing effort to improve our assessment practices, the survey was revised last year. For instance, we have included the same number of questions and have taken care to ensure that the wording between surveys is as similar as possible. While the improvement in the survey was needed and is important, it has created some difficulty in being able to accurately compare entrance and exit surveys, as students who graduated and exited the program were given a different exit survey from the original entrance survey that they would have completed approximately four years ago. While this issue will continue for the next two to three years, we feel that using the new survey is an improvement as it better reflects the current SLOs. An additional issue with the previous survey was that it neglected to ask any demographic questions pertaining to whether the student was in the BS or the BA program. Thus, providing data specific to BS students was previously impossible. This has been addressed in the new version of the survy (Fall 2015) which will result in specifc data relating to the BS program. We look forward to being able to report this in future assessments. 

Having said that, the validity of survey data is questionable in general, and therefore the Department is moving toward using discipline based criteria based on student writing in senior level writing intensive psychology courses. A rubric was created last year and approved by the undergraduate committee. We will be implementing this assessment in the Fall of 2015 in the BS program, and then eventually in the BA in Psychology. 

 

8) What types of evidence did the program use as part of the assessment activities checked in question 6? (Check all that apply.)

Direct evidence of student learning (student work products)


Artistic exhibition/performance
Assignment/exam/paper completed as part of regular coursework and used for program-level assessment
Capstone work product (e.g., written project or non-thesis paper)
Exam created by an external organization (e.g., professional association for licensure)
Exit exam created by the program
IRB approval of research
Oral performance (oral defense, oral presentation, conference presentation)
Portfolio of student work
Publication or grant proposal
Qualifying exam or comprehensive exam for program-level assessment in addition to individual student evaluation (graduate level only)
Supervisor or employer evaluation of student performance outside the classroom (internship, clinical, practicum)
Thesis or dissertation used for program-level assessment in addition to individual student evaluation
Other 1: The new assessment will eventually use final written projects in senior level writing intensive psychology courses.
Other 2:

Indirect evidence of student learning


Alumni survey that contains self-reports of SLO achievement
Employer meetings/discussions/survey/interview of student SLO achievement
Interviews or focus groups that contain self-reports of SLO achievement
Student reflective writing assignment (essay, journal entry, self-assessment) on their SLO achievement.
Student surveys that contain self-reports of SLO achievement
Other 1:
Other 2:

Program evidence related to learning and assessment
(more applicable when the program focused on the use of results or assessment procedure/tools in this reporting period instead of data collection)


Assessment-related such as assessment plan, SLOs, curriculum map, etc.
Program or course materials (syllabi, assignments, requirements, etc.)
Other 1:
Other 2:

9) State the number of students (or persons) who submitted evidence that was evaluated. If applicable, please include the sampling technique used.

An entry survey is taken immediately after a student declares psychology as the major. The exit survey is administered to all graduating Psy majors when the Goldenrod is completed. The goal of these surveys is to assess the extent to which the Psychology undergraduate student perceives that SLO outcomes were enhanced by completing a number of our Psy courses. In the last academic year 97 students completed the entrance survey and 134 completed the exit survey. But note, given that the previous version of the survey did not distinguish between BS and BA students, it is impossible to provide specific data for only BS students at this point. This has been addressed in the new survey and a more thorough assessment of whether students in the BS program are meeting SLOs will be possible. 

10) Who interpreted or analyzed the evidence that was collected? (Check all that apply.)

Course instructor(s)
Faculty committee
Ad hoc faculty group
Department chairperson
Persons or organization outside the university
Faculty advisor
Advisors (in student support services)
Students (graduate or undergraduate)
Dean/Director
Other: The undergraduate committee has helped in the development of the new assessment rubric, and will help in assessing written work in the future.

11) How did they evaluate, analyze, or interpret the evidence? (Check all that apply.)

Used a rubric or scoring guide
Scored exams/tests/quizzes
Used professional judgment (no rubric or scoring guide used)
Compiled survey results
Used qualitative methods on interview, focus group, open-ended response data
External organization/person analyzed data (e.g., external organization administered and scored the nursing licensing exam)
Other:

12) Summarize the results of the assessment activities checked in question 6. For example, report the percent of students who achieved each SLO.

Date: Fall 2014 – Spring 2015

 

UHM Undergraduate Psychology Entrance Survey (97 Respondents)

 

1. I currently feel knowledgeable about psychological concepts, theoretical perspectives, research findings, or historical trends.                                                              

M = 3.70 Sd=.96

 

1 2 3 4 5
Strongly disagree                          Agree                                              Strongly agree 

 

 

2. I have a background in basic research methods, including research design, data analysis, and interpretation, and able to use critical and creative thinking in solving problems.

M = 3.79 Sd=1.15

 

 

1 2 3 4 5
Strongly disagree                          Agree                                              Strongly agree 

 

 

3. I understand how psychological concepts can be used in everyday life and organization as well as the ethical complexities involved in applying psychology to social situations

M =4.01 Sd=1.17

 

 

1 2 3 4 5
Strongly disagree                          Agree                                              Strongly agree 

 

 

4. I currently feel comfortable with my communication skills including writing, interpersonal and oral skills, showing quantitative literacy and collaborating with others.

M=3.79 Sd=1.00

 

 

1 2 3 4 5
Strongly disagree                          Agree                                              Strongly agree 

 

5. I feel capable of understanding mental processes, applying effective strategies for self-management, including self-regulation, and integrity. I also feel capable of using psychological skills, values and information for my future career.

M=3.79 Sd=1.20

 

 

Date Fall 2014 – Spring 2015

 

UHM Undergraduate Psychology Exit Survey (upon graduation) (134 respondents)

 

1. As a result of majoring in Psychology, I feel more knowledgeable in psychological concepts, theoretical perspectives, research findings, or historical trends.

M=4.48 Sd=.90

 

 

1 2 3 4 5
Strongly disagree                          Agree                                              Strongly agree 

 

 

2. I have a better understanding of basic research methods, including research design, data analysis, and interpretation and able to use critical and creative thinking in solving problems

M=3.82 Sd= 1.35

 

 

1 2 3 4 5
Strongly disagree                          Agree                                              Strongly agree 

 

 

3. I now understand how psychological concepts can be used in everyday life and organization as well as the ethical complexities involved in applying psychology to social situations   

M=4.47 Sd=.86

 

 

1 2 3 4 5
Strongly disagree                          Agree                                              Strongly agree 

 

 

4. I developed effective communication skills including writing, interpersonal and oral skills, showing quantitative literacy and collaborating with others

M=3.74 Sd=1.31

 

 

1 2 3 4 5
Strongly disagree                          Agree                                              Strongly agree 

 

5. I benefited from studying psychology with regards to developing an understanding of mental processes, applying effective strategies for self-management, including self-regulation, and demonstrations of integrity. I also feel capable of applying my skills, values and information I received as a Psychology major to my future career.

M=3.99, Sd=1.41

 

 

 

 

Summary Table

 

SLO1

SLO2

SLO3

SLO4

SLO5

Entrance (n=97)

3.7

3.79

4.01

3.79

3.79

Exit (n=134)

4.48

3.82

4.47

3.74

3.99

 

Note, in the past we have statistically compared each item, however, this is not possible due to the fact that many students took different surveys. As mentioned in question 6, this is due to the implementation of an improved survey over the course of the year, resulting in students having taken a different entrance survey from the exit survey. Further complicating the analysis, the scales were reversed for some items in the new survey (e.g., strongly disagree was placed on right of the page rather than the left of the page). This was done in order to ensure that survey takers were paying close attention to the questions. This strategy unfortunately resulted in a very difficult to interpret data set, as it appears that many students failed to notice the scale reversal (despite the instructions) in the survey. Statistical improvement has traditionally (see 2014 report for example) been seen for all items with the exception of item 4 (communication). Unfortunately with the given data set it is difficult to determine how accurate the responses are, and impossible to reliably analyze. Having said that, it should be noted that no changes were made to item 1 (in the question or scale order), and we see the typical improvement from entrance to exit survey. Lastly, it should again be noted that these data reflect both BS and BA student responses. The survey that was used unfortunately was not capable of distinguishing between BS and BA students. This has been addressed and future assessments will be able to assess the surveys specific to program.  

13) What best describes how the program used the results? (Check all that apply.)

Assessment procedure changes (SLOs, curriculum map, rubrics, evidence collected, sampling, communications with faculty, etc.)
Course changes (course content, pedagogy, courses offered, new course, pre-requisites, requirements)
Personnel or resource allocation changes
Program policy changes (e.g., admissions requirements, student probation policies, common course evaluation form)
Students' out-of-course experience changes (advising, co-curricular experiences, program website, program handbook, brown-bag lunches, workshops)
Celebration of student success!
Results indicated no action needed because students met expectations
Use is pending (typical reasons: insufficient number of students in population, evidence not evaluated or interpreted yet, faculty discussions continue)
Other: New course proposals are discussed at the Undergraduate Committee and recommendations consistent with the SLOs are forwarded to the Faculty for approval. The SLOs are also distributed to faculty and lecturers and discussed at faculty meetings. Previously, data collected has led to our modification of our methods and statistics courses. Prospective students are now required to successfully complete Psychological Methods (PSY 212) and Statistics (PSY 225/SOCS 225) prior to declaring their major in psychology. In addition, instructors who fail to meet our expectations of SLOs are removed from our teaching pool. Many upper level courses are now monitored on the basis of feedback from instructors and eCAFE in how well students met the SLOs of the course.

14) Please briefly describe how the program used the results.

The analysis of the survey questions provides insights into the strengths and weakness of our psychology courses. For example, in the last seven years when we required successful completion of our Methods and Statistics courses before a student can declare their major in psychology, we began to observe significant differences in survey questions 1 and 2, which evaluate student knowledge of psychological concepts and methods. While it is difficult to make this claim due to the difficulties we have in comparing old entrance and new exit surveys, we have no reason to believe that this is not the case. This would equally apply to critical thinking issues in Psychology (question 3), and how psychological concepts and ethical issues can be addressed in everyday life and organization (question 5). Traditionally, improvements have not been observed for question 4, which focuses on communication issues. All PSY 4X9 seminar courses are currently only taught by faculty, many of which have an approved GenEd Focus, i.e., Writing, Ethics, Oral, etc. We hope that by doing so, undergraduates will be exposed to communication and/or ethics courses that improve their skills in writing and oral communication.

15) Beyond the results, were there additional conclusions or discoveries? This can include insights about assessment procedures, teaching and learning, and great achievements regarding program assessment in this reporting period.

Over the past year the Undergraduate Committee began a new assessment measure, focusing on evaluating student writing obtained in capstone course.The development of this new assessment procedure has been challenging, but a fruitful exercise for faculty members to collaborate on. It has required much faculty engagement, interaction, and collaboration. A rubric was designed and approved by the undergraduate committee. This rubric will assess student writing samples from senior level writing intensive psychology courses and determine whether students are reaching the Department’s SLOs and UHM ILOs. While the implementation is still ongoing, first at the BS level, we are excited about the possibility of better assessing student outcomes.
 
As mentioned earlier, we have also realized that at the moment we have no means of differentiating between BA and BS students in our current assessment protocols. Therefore we have added a new question to our survey in order to have students indicate whether they are a BA or BS student. We are hopeful to be able to assess students more accurately based on which program they are in this year. 
 
One final point worth mentioning is that the VCAA has recently approved a hybrid online BA in Psychology program. We are currently discussing the best way to assess students who complete the degree in this program. 
 

16) If the program did not engage in assessment activities, please explain.