Degree Program Assessment Reports

Industrial Advisory Board (IAB) Reports:   During the IAB meetings, the board members complete a survey of how our students achieve each of our Outcomes with ratings 1 (poor), 2 (marginal), 3 (satisfactory), and 4 (exemplary). 

Course Surveys:  At the end of every course, the students are surveyed about how the course helps achieve each of the Outcomes with ratings 0 (not at all), 1 (insignificant), 2 (moderate), and 3 (significant).  Every year, the data is used to estimate for each Outcome the number of Contributing Courses from the Student Experience.  There is a similar estimate done by the faculty from the instructor’s perspective referred to as Contributing Courses from the Faculty Expectations.  The two are compared to determine the level of student achievement.

EE 496 Senior Capstone Design Reviews:  The faculty annually reviews a random sample of EE 496 Senior Capstone Design projects through surveys.  For each project, there are two faculty reviewers:  the faculty project advisor and an “outside reviewer”. 

Measuring Outcomes Achievement in Courses:   The performance criteria and scoring rubrics for each SLO is done by expanding each SLO into measurable criteria.  These criteria are then measured in different EE undergraduate courses.  Instructors make direct assessments about how students are achieving these criteria by using the scoring rubrics. 

EBI Survey: EBI conducted an exit survey of our recent graduates in Fall 2008 where 19 students participated.  

  We have an assessment committee to look at data and interpret the results.  Data was gathered

and analyzed statistically.  To assess accuracy of results we look at how outcomes are rated using

different assessment methods.  

The committee looking at this was

our ABET (Engineering accreditation) committee.  The faculty members involved are Profs. Gurdal

Arslan, Luca Macchiarulo, Galen Sasaki, Anders Host Madsen, David Garmire, Yingbin Liang, and

Anthony Kuh (chair of EE dept.).

Table 1 summarizes the student level of achievement of Outcomes using the different assessment methods, where  “3” = exemplary, “2” = satisfactory, “1” = marginal, and “0” = unsatisfactory.   The overall level of student achievement is shown at the bottom row of the table.   The two assessment methods that are direct assessments are shaded in the table.

Table 1.  Level of student achievement of Outcomes.

The Table shows that Outcome 8 has an overall level of achievement of 1 (“marginal”).  Though there is some emphasis in the curriculum as shown in Figure 3, there is no strong emphasis in any EE course.  Note that this Outcome has some emphasis in some of the non-EE courses which is not considered in the evaluation in Table 1.   Note that we do not yet Measure this Outcome Achievement in an EE Course.   However, there is a proposal for this, which is a research and writing assignment.  The proposal is described in the section “Proposal to Measure Outcome 8” of this folder.

Appendix A.  IAB Reports

During Industrial Advisory Board (IAB) meetings, the board members complete a survey of how our students achieve our Outcomes giving ratings from 1 (“not at all”) to 4 (“a great deal”).   Table A.1  has the average ratings from the 2005 and 2007 IABs as well as the overall average of the ratings.  This overall average is translated into a level of achievement as follows:  

Table A.1.  2005 and 2007 IAB average ratings) of level of achievement of Outcomes. 

Appendix B.  Course Surveys

Our department surveys the students of all undergraduate courses every semester.  At the end of every course, students rate how the course contributes to each outcome:   3 = “significant”, 2 = “moderate”, 1 = “insignificant, and 0 = “not at all”.  For each course, the average scores are translated to a fraction

F = max{average rating - 1, 0}/2,

which corresponds to

F is interpreted as the fraction of the course that contributes significantly to the Outcome.  Thus, F = 1 implies that the entire course contributes, while F = 0.5 implies that half of the course contributes.

Given these estimates of F, we can estimate how many EE required 3-credit courses in Figures 1 and 2 contribute significantly to an Outcome.  For example, if there are 3 courses that have F = 1, 2 courses that have F = 0.5, and the rest of the courses have F = 0 then the number of courses is 3 x 1 + 2 x 0.5 = 4.  We refer to the estimate as the number of contributing courses as experienced by students.  Note that we have estimates per Track, which covers all the courses required by all EEs (in Figure 1) and courses required per Track (in Figure 2).  For example, a Computer Track student must take all courses in Figure 1 and all Computer Track Group I courses in Figure 2.

Note that 1 credit courses count as 1/3 of a 3-credit course, and 4 credit courses count as 4/3 of a 3-credit course.  Also note that student project courses, EE 296, 396, and 496, were not counted since the student experience can vary quite a bit depending on the project.  

Figure B.1 has the number of contributing courses as experienced by students in academic year 2005-06.  We should note that we did have some occasional missing data.  Since this missing data does not add to the contributing courses, our estimates are conservative. 

We also did a similar estimate of contributing courses but from the perspective of faculty instructors.  Faculty members, and in particular course coordinators, were given surveys to rate how each course contributes to Outcomes.  The data was used to estimate the number of contributing courses from faculty expectations which is shown in Figure B.2.

We compared the two estimates of contributing courses by subtracting faculty expectations from student experiences.  Figures B.3, B.4, and B.5 has these differences for 2005-06, 2006-07, and 2007-08.  Student experiences generally meet or exceed faculty expectations for all Outcomes.  (The only cases when student experience was below faculty expectations were for Outcome 3; but as Figure B.3 shows, faculty expectations are quite high for the Outcome and difficult to achieve.)  Thus, we translate the results of the figures to a student level of achievement of 2 (“satisfactory”) for all Outcomes.

Figure B.1.  Contributing Track Required for 2005-06 Academic Year.

Appendix C.  EE 496 EE Senior Capstone Design Reviews

Table C.1 shows the results over two years. In the first year (2005-06), only three project reports were reviewed, and six project reports were reviewed.  Per year, scores were average over the number of respondents.  Note that some faculty did not provide a rating to every Outcome, e.g., if the faculty member did not feel qualified to provide a rating.

There is a column with an overall average score over the two years.  The overall scores were then translated into student level of achievement as follows:  

Note that the average scores can range from a minimum of 1 to a maximum of 4 because the individual scores are: 1 (“None”), 2 (“Marginal”), 3 (“Good”) to 4 (“Very Good”).  Thus, this is a conservative translation from the average scores to level of student achievement.

Table  C.1 not shown here as table is too big, but SLOs all have ratings of satisfactory or marginal.

Appendix D.  EBI Surveys

EBI conducted an exit survey of our recent graduates in Fall 2008 where 19 students participated.  The survey had questions relating to the eleven Outcomes, where some Outcomes had multiple related questions.  For each question, numerical scores given by our graduates were averaged and compared to 68 other institutions.  The comparison led to a ranking from 1 (best) to 68 (worst).  We translated these rankings into a level of achievement where

Table D.1 has the resulting student level of achievement per Outcome.

Table D.1.  Level of achievement from EBI exit survey.

The assessment  process is undergoing changes as we get new tools on how to assess SLOs.

We are in the process of measuring  SLOs using outcome achievements  in undergraduate

courses.  This is a three year process that was started in 2008 and will finish in 2011.