Unit: Urban & Regional Planning
Program: Urban & Regional Plan (MURP)
Degree: Master's
Date: Mon Nov 16, 2020 - 2:56:26 pm

1) Program Student Learning Outcomes (SLOs) and Institutional Learning Objectives (ILOs)

1. Critically and creatively develop planning inquiries or processes to foster solutions-oriented decision-making;

(1. Demonstrate comprehensive knowledge in one or more general subject areas related to, but not confined to, a specific area of interest., 4. Critically analyze, synthesize, and utilize information and data related to one’s field of study.)

2. Effectively collaborate as a planning team to work with a client and/or stakeholders to assess and address a relevant planning problem to create a plan or professional report;

(7. Interact professionally with others.)

3. Effectively present oral and written work (as a plan, professional report, or research paper) in a coherent, persuasive and professional manner; and

(5. Proficiently communicate and disseminate information in a manner relevant to the field and intended audience., 7. Interact professionally with others.)

4. Reflect upon the ethical implications of the choices planners make as professionals.

(6. Conduct research or projects as a responsible and ethical professional, including consideration of and respect for other cultural perspectives.)

5. Explain, critique and apply prominent planning theories/concepts to analyze a planning issue(s);

(1. Demonstrate comprehensive knowledge in one or more general subject areas related to, but not confined to, a specific area of interest., 4. Critically analyze, synthesize, and utilize information and data related to one’s field of study.)

6. Demonstrate an understanding of urbanization processes and rationales for planned interventions;

(1. Demonstrate comprehensive knowledge in one or more general subject areas related to, but not confined to, a specific area of interest., 4. Critically analyze, synthesize, and utilize information and data related to one’s field of study.)

7. Apply planning methods to organize, analyze, interpret and present information;

(2. Demonstrate understanding of research methodology and techniques specific to one’s field of study., 3. Apply research methodology and/or scholarly inquiry techniques specific to one’s field of study.)

2) Your program's SLOs are published as follows. Please update as needed.

Department Website URL: https://manoa.hawaii.edu/durp/admissions/masters-in-urban-and-regional-planning/#1581108983776-fc212498-288d
Student Handbook. URL, if available online: https://manoa.hawaii.edu/durp/wp-content/uploads/2020/01/Program-Guide-2019-2020.pdf
Information Sheet, Flyer, or Brochure URL, if available online:
UHM Catalog. Page Number: https://manoa.hawaii.edu/catalog/schools-colleges/arts-sciences/socs/urp/
Course Syllabi. URL, if available online:

3) Please review, add, replace, or delete the existing curriculum map.

Curriculum Map File(s) from 2020:

4) For your program, the percentage of courses that have course SLOs explicitly stated on the syllabus, a website, or other publicly available document is as follows. Please update as needed.


5) Does the program have learning achievement results for its program SLOs? (Example of achievement results: "80% of students met expectations on SLO 1.")(check one):

Yes, on some(1-50%) of the program SLOs
Yes, on most(51-99%) of the program SLOs
Yes, on all(100%) of the program SLOs

6) Did your program engage in any program learning assessment activities between November 1, 2018 and October 31, 2020?

No (skip to question 17)

7) What best describes the program-level learning assessment activities that took place for the period November 1, 2018 and October 31, 2020? (Check all that apply.)

Create/modify/discuss program learning assessment procedures (e.g., SLOs, curriculum map, mechanism to collect student work, rubric, survey)
Collect/evaluate student work/performance to determine SLO achievement
Collect/analyze student self-reports of SLO achievement via surveys, interviews, or focus groups
Use assessment results to make programmatic decisions (e.g., change course content or pedagogy, design new course, hiring)
Investigate other pressing issue related to student learning achievement for the program (explain in question 8)

8) Briefly explain the assessment activities that took place since November 2018.

The Assessment Coordinator, with assistance from faculty, developed a multi-year program assessment plan that recommends evaluating one to two SLOs each year. Implemented incrementally, this plan allows for comprehensive assessment by DURP that is systematic and manageable. 

In preparing for re-accreditation, we undertook the following assessment activities:

1. Revised the MURP curriculum map and aligned it with an assessment plan. 

2. Conducted three surveys -- current students, recent graduates (last 2-5 years), and employers -- to gather data about the MURP program and its alignment with industry needs. In April 2019, the Department surveyed 121 students who graduated from the MURP degree program between August 2012 and December 2018, inclusive of those two points in time. E-mail addresses were available for 114 graduates. Of those graduates, 50 responded with valid responses to the survey. The 50 respondents could be considered a representative sample because they included MURP graduates from each year of the review period. Of the 50 respondents, 22 reported that they had graduated between 2 and 5 years ago. The survey provided data on: #1 Graduate Satisfaction; #2 Graduate Service to Community and Profession; and #3 Graduate Employment. The DURP ‘Ohana (alumni association) Accreditation Committee administered an employer survey in June 2019. The survey included 48 organizations (public, private, non-profit) and received a 33% response rate. Its main purpose was to assess the skills and attributes of MURP graduates in the last seven years to examine how the master's degree is meeting the needs of the firms/organizations that employ them. 

3. Developed assessment tools (e.g. rubric, client evaluation form) for core courses in the MURP program. 

9) What types of evidence did the program use as part of the assessment activities checked in question 7? (Check all that apply.)

Artistic exhibition/performance
Assignment/exam/paper completed as part of regular coursework and used for program-level assessment
Capstone work product (e.g., written project or non-thesis paper)
Exam created by an external organization (e.g., professional association for licensure)
Exit exam created by the program
IRB approval of research
Oral performance (oral defense, oral presentation, conference presentation)
Portfolio of student work
Publication or grant proposal
Qualifying exam or comprehensive exam for program-level assessment in addition to individual student evaluation (graduate level only)
Supervisor or employer evaluation of student performance outside the classroom (internship, clinical, practicum)
Thesis or dissertation used for program-level assessment in addition to individual student evaluation
Alumni survey that contains self-reports of SLO achievement
Employer meetings/discussions/survey/interview of student SLO achievement
Interviews or focus groups that contain self-reports of SLO achievement
Student reflective writing assignment (essay, journal entry, self-assessment) on their SLO achievement.
Student surveys that contain self-reports of SLO achievement
Assessment-related such as assessment plan, SLOs, curriculum map, etc.
Program or course materials (syllabi, assignments, requirements, etc.)
Other 1:
Other 2:

10) State the number of students (or persons) who submitted evidence that was evaluated. If applicable, please include the sampling technique used.

SLO 1: Explain major planning paradigms and their applications

Faculty members, who are on student capstone committees, scored two criteria from the capstone assessment rubric for SLO 1: "The student demonstrates working knowledge of the stated planning sub-field," and "the capstone document presents a clear analysis and cohesive argument/logic." Average scores for these two criteria were calculated for all students who submitted capstones between Fall 2015 and Fall 2018. The maximum score for each criterion was 5. Students received an average of 3.3 for “demonstrating working knowledge of the stated planning sub-field,” and an average of 3.0 for “presenting a clear analysis and cohesive argument/logic.” Thus, for all students who submitted capstones during this review period, combining the two criteria for assessing SLO 1 yielded an average score of 3.2 (out of the maximum of 5). 


SLO 2: Articulate processes leading to urbanization and rationales for planned interventions

Using several questions on a mid-term exam, 47 students enrolled in PLAN 600: Public Policy and Planning Theory were evaluated in Fall 2017 (23) and Fall 2018 (24) to assess the attainment of SLO 2. Four of these exam questions related to the urbanization process (numbers 1a, 1b, 1c, and 3) and five related to planning history (2, 4, 5, 6 and 7). The maximum possible score in each set of questions was 4.5. In Fall 2017, we found that students correctly answered 64% of the questions in the set pertaining to understanding urbanization processes, for an average total score of 2.7; likewise, with 80% correct answers, the average total score on questions to test the understanding of planning history was 3.8. Combining both scores yielded an average score of 3.3 for SLO 2. In Fall 2018, 62% correct answers on understanding urbanization processes produced an average total score of 2.8, and, for understanding planning history, the respective figures were 78% and 3.7. Combining the two total scores indicated that the Fall 2018 average score for SLO 2 was 3.3. 


SLO 3: Apply planning methods to organize, analyze, interpret and present information

To assess the achievement of SLO 3, in Fall 2017 and Fall 2018, the instructor analyzed responses to Question 25 on the mid-term exam in PLAN 601: Planning Methods: “Big Island residents were polled regarding their attitudes toward tourism expansion on their island. The data were broken down by location of residence. The purpose is to determine if there is an association between location of residence and attitude towards tourism expansion at the significance level α = 0.05.” Of the 11 students in Fall 2018, the performance of 73% demonstrated achievement of SLO 3 because, of a maximum possible score of 12 points, the average score was 9.4. This was a marked improvement over Fall 2017 when SLO 3’s fulfillment was met by only 39% of the 18 students, and the average score was 8.2 points. The difference in the results could be due to changes in the course design that introduced small group discussion to assist students in understanding key concepts in this course.


A rubric in PLAN 678: Site Planning also contributed to assessing SLO 3. A team of external jurors evaluated student teams for the following:

Concept: articulation of concept for developing the site and its link to the existing context.

Site plan: translation of the concept into a program of land uses.

Quality: transition from site analysis and synthesis of site conditions, market assumptions, and stakeholder involvement to the conceptual site plan; graphic and oral communication.


External jurors, four in Fall 2016 and five in Fall 2017, gave 11 teams (34 students in all) the following average scores: 4.5 and 4.6 (out of a maximum of 5) for concept; 14 and 15.5 (out of 20) for site plan; 7.4 and 7.8 (out of 10) for graphic communication; and 4.6 and 4.6 (out of 5) for oral communication. The scores were supported by qualitative evaluation of the strengths and weaknesses of students’ final assignments. Overall, the results revealed that students did well in articulating a concept and presenting it verbally. Design development and graphic communication, however, had scope for improvement. Although we typically provide an intensive graphic communication workshop featuring Adobe Photoshop, InDesign, and Illustrator in the course curriculum, efforts are ongoing to develop and offer a 1-credit course on graphic communication for planners during the summer session. 


SLO 4: Critically and creatively develop planning inquiries or processes to foster solutions-oriented decision-making 

A client assessment form for PLAN 751: Planning Practicum was piloted in Fall 2018 and administered again in Fall 2019. The results indicate that clients were very satisfied with student performance, their professionalism, and the quality of project deliverables. They agreed with students' recommendations and found them useful.  


SLO 5: Effectively collaborate as a planning team to work with a client and/or stakeholders to assess and address a relevant planning problem to create a plan or professional report

In Fall 2018, PLAN 751: Planning Practicum introduced a questionnaire that allowed students to self-assess their ability to address a planning problem. Each student team was asked to first discuss and then describe two aspects of the project they deemed successful, and two that they felt they could have approached or executed better. They were also asked to describe what they would do differently if they were to do the project again. The results highlighted aspects of the project (process and outcomes) students found challenging such as preparing appropriate outreach materials and enhancing stakeholder participation.   


SLO 6: Effectively present oral and written work (as a plan, professional report, or research paper) in a coherent, persuasive and professional manner

Two criteria from the capstone assessment rubric inform SLO 6: “Overall, the Capstone document is of professional quality” and “Overall, the presentation is of professional quality.” For all 239 students who submitted their capstones between Fall 2015 and Fall 2018, calculating the average score for each criterion (maximum 5 points) and then computing the average for both criteria produced the score for SLO 6. On average, students scored 3.2 for presenting a professionally written document and 3.3 for making a professional oral presentation, which gave an average score of 3.2 for SLO 6. For the academic year 2017-2018, specifically, the aggregate score for SLO 6 was 4.3; 4.2 for presenting a professionally written document and 4.4 for making a professional oral presentation.


SLO 7: Reflect upon the ethical implications of the choices planners make as professionals

Question 21b of the mid-term exam in PLAN 601 helped in testing the achievement of SLO 7. The assessment evaluated the grades of all 11 students enrolled in the course during Fall 2017 and 18 students enrolled in Fall 2018. A maximum of 4 points could be earned for a correct answer to Question 21b, which asked: “Who is the researcher similar to in hypothesis testing: the defense attorney or the prosecuting attorney? Why? [Please limit your answers to 150 words].” With an average score of 3.6 out of a maximum 4, the sensitivity sought by SLO 7 was demonstrated by 73% of the students in Fall 2017; in Fall 2018, the average score was 3.1 and 67% of the students attained SLO 7. Fall 2018 also saw assessment measures for SLO 7 pilot tested in PLAN 751: Planning Practicum. Upon completing the project, students submitted one to two-page reflection statements, which the instructor qualitatively analyzed using a rubric that categorized them as suggesting: non-reflection, understanding, reflection, and critical reflection. Of the 14 students evaluated, 57% demonstrated reflection and critical reflection. Their reflection ranged from description and recording of experiences (low degree) to questions about the fundamental nature of planning (high degree). The results led to introducing more readings and inviting professional planners to speak on professional ethics to emphasize its importance in practice.




11) Who interpreted or analyzed the evidence that was collected? (Check all that apply.)

Course instructor(s)
Faculty committee
Ad hoc faculty group
Department chairperson
Persons or organization outside the university
Faculty advisor
Advisors (in student support services)
Students (graduate or undergraduate)

12) How did they evaluate, analyze, or interpret the evidence? (Check all that apply.)

Used a rubric or scoring guide
Scored exams/tests/quizzes
Used professional judgment (no rubric or scoring guide used)
Compiled survey results
Used qualitative methods on interview, focus group, open-ended response data
External organization/person analyzed data (e.g., external organization administered and scored the nursing licensing exam)

13) Summarize the results from the evaluation, analysis, interpretation of evidence (checked in question 12). For example, report the percentage of students who achieved each SLO.

Assessment of SLOs

The evaluation results suggest a steady, even strong, improvement in attaining SLOs, with our most recent assessments revealing that the majority of our students are meeting SLOs—from about 60% for any to well over 80% for some. For instance, over 70% of the master’s students have demonstrated fulfilling SLO 3 (apply planning methods for organizing, analyzing, interpreting, and presenting information) and SLO 7 (ethical implications of planning choices). This underscores that our graduates are gaining well-rounded proficiency in planning, and developing much need sensitivity to be effective and progressive professionals. Much improvement is also visible in realizing SLO 6 (effectively present oral and written work in a coherent, persuasive and professional manner). Regarding SLO 2, too, the performance of students has been encouraging, yet we see scope for continued improvement. We can see that, relative to the impressive proportion of students (over 80%) demonstrating a strong understanding of planning history and design rationales, the demonstrated grasp (60-70%) of the urbanization process and planning’s legal foundations can be enhanced. The expansive breadth of PLAN 600 likely leaves limited space to explore these topics deeper. But since these topics are also covered elsewhere, we are exploring how they could be further emphasized and evaluated, systematically, in other core and elective courses. 


Employer Survey Results

Overall, of the 16 respondents, 62.5% reported being “satisfied” and 37.2% reported being “very satisfied” with the DURP graduates they have employed. Their assessment of the 12 skills/attributes of DURP graduates on a scale that ranged from Outstanding, Very Good, Good, Could Improve, Need to Improve indicated that graduates were assessed as Very Good in teamwork (50%), knowledge of general planning principles (43.7%), ability to incorporate feedback and suggestions (43.7%), reliability (43.7%) and Good in effort to improve (50%), flexibility (46.7%) and communication skills (43.7%). There were no responses for Need to Improve.  


Alumni Survey Results

Of 22 survey respondents who graduated two to five years ago, 20 or 91% reported that they were satisfied to some degree with how the program prepared them for employment. Similarly, 20 or 91% said that they had made contributions to meeting the needs of their community and the profession. Fourteen or 70% of these 20 graduates provided details about their contribution. Of all 50 graduates who responded to the survey in April 2019, 45 or 88% said that they pursued paid employment in professional planning or a planning-related position after graduating from the MURP degree program. Of those 45, 39 or 87% said that they gained such employment within one year of graduation.  




14) What best describes how the program used the results? (Check all that apply.)

Assessment procedure changes (SLOs, curriculum map, rubrics, evidence collected, sampling, communications with faculty, etc.)
Course changes (course content, pedagogy, courses offered, new course, pre-requisites, requirements)
Personnel or resource allocation changes
Program policy changes (e.g., admissions requirements, student probation policies, common course evaluation form)
Students' out-of-course experience changes (advising, co-curricular experiences, program website, program handbook, brown-bag lunches, workshops)
Celebration of student success!
Results indicated no action needed because students met expectations
Use is pending (typical reasons: insufficient number of students in population, evidence not evaluated or interpreted yet, faculty discussions continue)

15) Please briefly describe how the program used its findings/results.

To enable students to meet SLOs, we advise them to take courses sequentially (e.g., core courses before elective courses), to the extent possible. Based on assessment results, we are revising PLAN 604: Qualitative Methods in Planning to strengthen students’ understanding and application of research methodologies. We have also proposed an additional advanced quantitative methods course (expected to be approved for Fall 2021). Furthermore, the curriculum sub-committee has finalized the assessment form for the client of the practicum project to assess SLOs 4-6. Discussion about the assessment process and its results will continue in upcoming faculty and curriculum committee meetings. 

16) Beyond the results, were there additional conclusions or discoveries? This can include insights about assessment procedures, teaching and learning, and great achievements regarding program assessment in this reporting period.

We used the findings to prepare our re-accreditation report. The Planning Accreditation Board site visit team stated "The Student Learning Outcomes Assessment was particularly rigorous with attention being given to course revisions necessary to improve outcomes. The quality of this work clearly requires a significant commitment of resources from the Department. As being carried out, it could serve as a model for other programs" (April, 2020). 

17) If the program did not engage in assessment activities, please justify.