Unit: Communicology
Program: Communicology (BA)
Degree: Bachelor's
Date: Tue Oct 20, 2009 - 8:34:06 am

1) List your program's student learning outcomes (SLOs).

  • To demonstrate understanding of key concepts and theories in communication
    1. To demonstrate understanding of key concepts and theories of message processing
    2. To demonstrate understanding of key concepts and theories of social influence
    3. To demonstrate understanding of key concepts and theories of relational communication
    4. To demonstrate understanding of the relationship between culture and communication

  • To demonstrate the ability to apply the understanding of key concepts and theories in communication in a variety of contexts, including:
    1. To demonstrate the ability to apply the understanding of key concepts and theories of message processing
    2. To demonstrate the ability to apply the understanding of key concepts and theories of social influence
    3. To demonstrate the ability to apply the understanding of key concepts and theories of relational communication
    4. To demonstrate the ability to apply the understanding of the relationship between culture and communication

  • To demonstrate understanding of basic communication research concepts and processes, including:
    1. The ethics involved in conducting human subjects research
    2. Basic library and Internet research
    3. Basic research design, data collection, and analysis
    4. Structure of basic research reports

  • To demonstrate the ability to analyze demographic and situational factors and the ability to apply that information to achieve communicative goals

    • 2) Where are your program's SLOs published?

    Department Website URL:
    Student Handbook. URL, if available online:
    Information Sheet, Flyer, or Brochure URL, if available online:
    UHM Catalog. Page Number:
    Course Syllabi. URL, if available online: http://www.hawaii.edu/speech/c_description.php
    Other:
    Other:

    3) Upload your program's current curriculum map(s) as a PDF.

    Curriculum Map File(s) from 2009:

    4) What percentage of courses have the course SLOs explicitly stated on the course syllabus, department website, or other publicly available document? (Check one)

    0%
    1-50%
    51-80%
    81-99%
    100%

    5) State the SLO(s) that was Assessed, Targeted, or Studied

    All of them.

    6) State the Assessment Question(s) and/or Goal(s) of Assessment Activity

    Overview:

    A comparison groups method was used to compare graduating Speech majors to Speech-naive students using an assessment instrument measure understanding of communicological material reflective the the required course in the Speech major.

    Process:

    Initial data were collected in 2002 from SP151 (Personal and Public Speech) students. These data serve as the comparison group data. Sp151 is typically taken by non-majors as a requirement for other majors. Each semester, comparable data are collected from graduating seniors and added to the accumulated data of prior semesters.

    Instrument:

    A pool of multiple choice items was culled from midterm and final exams of the 6 courses required for Speech majors. An instrument was developed by selecting items at random from each of the content areas, creating a balanced (across required courses) set of 48 items.

    Analyses:

    Various statistical analyses are used each semester allowing detailed comparisons of assessment scores between majors and the Speech-naive counterparts within and across required courses.

    Conclusion:

    Accumulated evidence across 8 years shows that graduating Speech majors are significantly more knowledgeable regarding rquired course content relative to Speech-naive students. The retention effects vary across required courses suggesting that some content areas are learned and/or retained better than others. Pedagogical strategies are being implemented to improve learning/retention in those areas with the smallest effect sizes.

    7) State the Type(s) of Evidence Gathered

    8) State How the Evidence was Interpreted, Evaluated, or Analyzed

    9) State How Many Pieces of Evidence Were Collected

    10) Summarize the Actual Results

    Overall Assessment scores:

    Speech-naive students: 21.87

    Graduating Speech majors: 30.04

    t(177) = 10.52

    r = .62

    Graduating Speech majors exhibit significantly greater knowledge and understanding of comprehensive communicological course content relative to Speech-naive counterparts.

    SP251 Public Speaking:

    Speech-naive students: 5.36

    Graduating Speech majors: 5.79

    t(177) = 2.35

    r = .17

    Graduating Speech majors exhibit significantly greater knowledge and understanding of Public Speaking course content relative to Speech-naive counterparts. The effect size is small, however SP251 is often taken before declaring as a major and the content is minimally reinforced throughout the major.

    SP301 Speech Communication Theory:

    Speech-naive students: 4.08

    Graduating Speech majors: 4.91

    t(186) = 4.10

    r = .31

    Graduating Speech majors exhibit significantly greater knowledge and understanding of a range of communicological theories relative to Speech-naive counterparts. The effect size is smaller relative to some other required courses, suggesting a need for greater redundancy of this course content across other courses.

    SP302 Research Methods:

    Speech-naive students: 3.90

    Graduating Speech majors: 4.98

    t(186) = 4.92

    r = .35

    Graduating Speech majors exhibit significantly greater knowledge and understanding of research methods, procedures, design, and data analysis relative to Speech-naive counterparts. The effect size is smaller relative to some other required courses, suggesting a need for greater redundancy of this course content across other courses. Speech faculty have agreed to give greater attention to research methods whenever discussing the results of research in their other classes.

    SP364 Persuasion:

    Speech-naive students: 2.95

    Graduating Speech majors: 4.62

    t(186) = 6.93

    r = .48

    Graduating Speech majors exhibit significantly greater knowledge and understanding of social influence, compliance-gaining, attitude change, and persuasion relative to Speech-naive counterparts. The effect size is large among required courses, indicating this course content is probably reinforced in other required and elective courses.

    SP Interpersonal Relations:

    Speech-naive students: 2.46

    Graduating Speech majors: 4.96

    t(187) = 12.09

    r = .70

    Graduating Speech majors exhibit significantly greater knowledge and understanding of interpersonal communication and relationships relative to Speech-naive counterparts. The effect size is quite large, indicating that this course content is probably reinforced in other required and elective courses.

    SP370 Verbal Communication:

    Speech-naive students: 1.34

    Graduating Speech majors: 1.88

    t(184) = 3.38

    r = .26

    Graduating Speech majors exhibit significantly greater knowledge and understanding of verbal communication and the social/psychological aspects of language processing relative to Speech-naive counterparts. The effect size is smaller relative to most other required courses but the content of SP370 is least likely to be redundantly covered across other required and elective courses.

    SP470 Nonverbal Communication:

    Speech-naive students: 1.72

    Graduating Speech majors: 2.87

    t(185) = 6.79

    r = .53

    Graduating Speech majors exhibit significantly greater knowledge and understanding of nonverbal communication and its role in message processing relative to Speech-naive counterparts. The effect size is robust among required courses which is probably an indication of how much nonverbal communication plays a role in theory and research in other content areas across required and elective courses.

    11) Briefly Describe the Distribution and Discussion of Results

    Results are presented to faculty annually in a regular faculty meeting and discussed as part of an annual assessment retreat wherein implications of results for pedagogical practices are discussed, as well as a review of assessment procedures.

    12) Describe Conclusions and Discoveries

    Summary:

    Across all of our required courses, assessment evidence clearly indicates that graduating Speech majors have a better understanding of communicological processes, theory, and research relative to Speech-naive students. Effect sizes vary, providing an indication of what materials are being learned and retained well relative to materials that may require more attention to increase levels of understanding and retention.

    Limitations:

    Comparison groups are useful but not optimal. In order to obtain data from Speech-naive students, data are collected from students in a lower division class. Students in the control group -- as an aggregate -- are likely to be younger and less experienced in university level work than graduating seniors. Many may transfer without completing a degree at UHM. Consequently our assessment at this point is not adequately protected against participant attrition or maturation effects.

    Instrument:

    Multiple faculty members teach the required courses. To date, the department has made only minimal efforts to oversee course content for consistency. There is no guarantee at this point that the item pool from which the assessment instrument is created reflects material coverd in all required classes. Some error is introduced to the extent that graduating seniors respond to items that reflect materials they are not familiar with.

    Future Directions:

    The Speech faculty  are in the process of standardizing course content (as well as assessment pool items) across all required courses. In addition, the process of collecting data from a Speech-naive control group is being phased out. In its place we are discussing the possibility of using a Time 1 - Time 2 data collecting process that will employ an initial assessment at the point at which a student signs up as a major, to be followed by a second assessment at the time a major graduates.

    13) Use of Results/Program Modifications: State How the Program Used the Results --or-- Explain Planned Use of Results

    14) Reflect on the Assessment Process

    We are fast reaching the point at which we cannot go further with our assessment practices without some support from administration. There are real time costs associated with these efforts and the department has absorbed these costs as best as possible while resources have been continuously cut. We have moved data collection from paper-pencil processes to web-based methods (which required considerable time to design, implement and maintain). Attitudinal and perceptual data collected from Seniors across multiple semesters in a paper-pencil format have still not been entered into the computer for analysis.

    We have taken inexpensive assessment methods as far as we can take them. The university will have to decide if it wishes to take student learning seriously enough to invest resources in it.

    15) Other Important Information

    16) FOR DISTANCE PROGRAMS ONLY: Explain how your program/department has adapted its assessment of student learning in the on-campus program to assess student learning in the distance education program.

    17) FOR DISTANCE PROGRAMS ONLY: Summarize the actual student learning assessment results that compare the achievement of students in the on-campus program to students in the distance education program.