Educational Assessment

Our goal is to facilitate a cycle of continuous improvement in teaching and learning. The strategy for achieving this goal is multi-tiered.

    Primary Components

    • Align learning outcomes at the assignment, course, program, and institutional levels in order to conduct analyses of curricular strengths and weaknesses (often called Gap Analysis) using innovative methodology, including the Curriculum Mapping Initiative tool
    • Demonstrate student competency of learning outcomes by linking outcomes to evidence of student learning
    • Engage and support faculty in evidence-based assessment and scholarship of teaching and learning
    • Assess activities aimed at improving student success

    Measure Results

    SMU uses a variety of qualitative and quantitative data as indicators of engagement in a vibrant culture of assessment. They include:

    Faculty Engaged in Assessment and Continuous Improvement of Teaching and Learning

    • 116 faculty studies on improving teaching with technology
    • Learning outcomes aligned at the assignment, course, program, and Institutional levels and displayed as music and insightful display
    • Faculty collecting exemplars of excellence demonstrating learning outcomes and linking them to assignments

    Policy Change

    Scholarship of teaching and learning included in faculty research symposium and considered in rank and promotion.

    At Samuel Merritt University we developed a unique tool for assessing teaching and learning called the Curriculum Mapping Initiative (CMI). It displays interactive maps that highlight curricular integrity, strength and gaps. These colorful maps provide faculty with clear views of the overall curriculum and where to focus next steps.CMI's key features include:

    • Curricular maps that serve as institutional and program portfolios.
      At a glance faculty and accreditors can determine how well our learning outcomes are aligned at the course, program and institutional levels. This data visualization shows “the big picture” of curricular alignment. While looking at a Program Learning Outcome map faculty click on a Course Learning Outcome and up pops a text box with exemplars of student work along with sample assignments, rubrics, and resources. These displays are powerful representations of the whole being more than the sum of its component parts.
       
    • Learning outcomes are exported into every course in our Learning Management System (Canvas) making it easy for faculty to link learning outcomes to assignments and rubrics. 
       
    • Time and stress in preparation for specialty accreditation is decreased. Groups of faculty participate in mapping specialty accreditation standards to learning outcomes along with exemplars of evidence of student learning (democratization). Faculty members are recognized for adding evidence of learning to the repository (rather than having a few individuals responsible for collecting it all).

    Assessment of student learning ensures that students are achieving educational goals. Facutly designed our curriculum around expected learning outcomes and competencies that students will be able to demonstrate upon graduation. Faculty assess student learning to make sure that students demonstrate those competencies as they progress through their courses.

    The key questions we ask in assessment are:

    • Are students learning and able to apply the skills, knowledge, and attitudes necessary to be competent practitioners upon graduation?
    • How can we improve teaching and learning to achieve the stated learning outcomes?

    At Samuel Merritt University we developed a unique methodology for assessing educational effectiveness. Integral to our approach are unique curricular maps that show alignment between course, program, and institutional learning outcomes. Samuel Merritt University faculty defined three levels of learning outcomes:

    • Institutional Learning Outcomes (ILO): What every graduate of Samuel Merritt University will demonstrate
    • Degree Program Learning Outcomes (PLO): What every graduate of a particular degree Program will demonstrate
    • Course Learning Outcomes (CLO): What every graduate of a course will demonstrate

    Our assessment strategy begins at the assignment level. Faculty in each academic degree program align their assignments with CLOs. Then they assess if students are demonstrating competency of the CLO. They gather samples of student work demonstrating competency. Faculty in each academic program also align their CLO with PLOs and their PLOs with ILOs.

    We designed and developed the Curriculum Mapping Initiative (CMI) to facilitate meaningful analysis of our curricular alignment. CMI software displays curricular maps in ways that facilitate analysis of learning outcomes at the course, program and institutional levels and link outcomes to exemplar assignments, authentic evidence of student learning, rubrics and learning resources.

    Assessing the Assignment 

    At the course level faculty align assignments to CLOs. As students complete assignments faculty evaluate each student’s competency in completing the assignment in the form of feedback or a grade. Then faculty assess whether, overall, the students demonstrated full or partial competency of the CLO.  

    Assessing the CLOs

    Faculty member assesses student learning of each CLO by selecting samples of excellence in student learning demonstrating competency (full or partial) of the CLO. These exemplars are added to the program portfolio in CMI as authentic evidence of student learning. For example, a student reflection on a clinical experience, a presentation, or record of demonstration of skill may be uploaded to demonstrate competency (partial or full) of a particular CLO.

    Since our CLOs are fairly broad, it may require several assignments to demonstrate competency. So, one CLO may be aligned with more than one assignment. Conversely, some completed assignments demonstrate competency of multiple CLOs.

    Assessment of Program Learning Outcomes

    Each CLO is aligned with one or more PLO.  Faculty assess the educational effectiveness in a variety of ways.

    Gap Analysis:

    Faculty analyze curricular strengths and weaknesses by determining which PLOs have sufficient coverage.  For example, it becomes easy to identify which PLOs have many CLOs aligned and which PLOs have none or few CLOs aligned.

    Quality:

    Faculty also look at the quality of the evidence of student learning to gauge how well students are achieving our stated outcomes. For example, if faculty review the evidence for a CLO and decide that the evidence does not meet the stated criteria, it is an indication that students are not demonstrating the stated learning outcomes and either pedagogical or curricular changes are needed.

    Closing the Loop:

    Faculty meet annually to make a plan to improve either pedagogical or curricular gaps or redundancies.

    Assessment of Institutional Learning Outcomes

    Faculty align PLOs with ILOs in each academic program. Members of the Academic Council and administrators conduct a gap analysis and look at samples of evidence of student learning to assess how well the PLOs and CLOs are meeting the stated ILOs.

    The Curriculum Mapping Initiative (CMI) tool facilitates the management and analysis of assessment of student learning by displaying vivid maps that show alignment of learning outcomes with links to evidence of student learning, as well as rubrics, and assignments.

    The bar chart shows how many PLOs are aligned with each of the ILOs.

    This chart shows that most of the PLOs align with Clinically Competent with Critical Thinking and Communication skills not far behind. For a health sciences institution this distribution makes sense. There are not many of PLOs aligned with Information Technology. This is an area that may require further analysis and a plan for improvement.

    The CMI software also can display a unique visualization and sonifacation of the alignment of ILOs and PLOs.

    Figure 2: Distribution of alignment of PLOs to ILOs. This map shows the distribution of PLOs that align with ILOs in a degree program.

    Each ILO is represented in a color-coded row. Each PLO is represented by a column. Where the PLO aligns with the CLO the square is filled in with the color of the ILO. For example, in Figure 2, PLO 10 aligns with the ILO of Information Technology in this sample degree program.

    Sonification

    Each of the color-coded ILO rows is assigned a musical note. When the user clicks on “Play Sonifacation” the alignment of PLOs with ILOs plays as a musical score.

    The columns are played starting at the top left column. The score can be played note by note or as a score. 

    After hearing their curricular alignment played as a musical score faculty members tend to find new insight and analysis on how to improve their curriculum.

    Click to watch a screen capture and hear the sonifacation of the alignment of Program Learning Outcomes and Institutional Learning Outcomes.

    Mosaic Maps

    The Mosaic Map view allows us to dig deeper and to conduct both qualitative and quantitative analysis of curriculum as well as educational effectiveness.

    Mosaic maps show the alignment of  ILOs, PLOs, CLOs as well as artifacts showing evidence of student learning, and teaching tools such as sample assignments, rubrics, and resources.

    SMU uses a variety of qualitative and quantitative data as indicators of engagement in a vibrant cutlure of assessment. They include:

    • Faculty Engaged in Assessment and Continuous Improvement of Teaching and Learning
    • Learning Outcomes Aligned at the Assignment, Course, Program and Institutional Levels
    • Policy Change

    Faculty Engaged in Assessment and Continuous Improvement of Teaching and Learning

    Policy Change

    Faculty changed the definition of Scholarship to include the Scholarship of Teaching and Learning at SMU. Scholarship of Teaching and Learning is now considered part of the criteria for promotion.

    We work with faculty to support the scholarship of teaching and learning. Dozens of reports have been completed for the Improving Teaching with Technology Grants Program  instituted in 2013. The Improving Teaching with Technology posters proved an effective incentive for faculty to experiment with new pedagogy, gather evidence, and analyze results. Click here to review a list of reports and posters from the Improving Teaching with Technology Grants Program submitted to date.

    The Improving Teaching with Technology grants encourage faculty to engage in evidence-based practice around teaching and learning and facilitates the sharing of information about effective and ineffective practices. The grant process is simple. Here are the instructions:

    Recognition of SMU’s Assessment Improvement Process

    Samuel Merritt University serves as model for other organizations seeking new pathways to assessment that meet the growing demand by students, faculty, and accreditors for a more flexible and effective approach to assessing student learning.

    Accreditation Commendations

    The following is an excerpt from the WASC Educational Effectiveness Team re-accreditation letter:

     

     

    SMU has moved at mach speed to ramp up systems, now those systems must be sustained into the future.”

    “The University is to be commended for its accomplishments in developing the Curriculum Mapping Initiative…This CMI interactive tool is the culmination of considerable foundational work across the University to document the intricate alignment of numerous and diverse course-level outcomes and program-level student learning outcomes with the institution-wide Core Learning Competencies.”

    “The CMI has permitted the SMU faculty to engage in meaningful conversations and programmatic improvements related to the cohesion, structure and comprehensiveness of the curriculum… It has also promoted faculty agreement and understanding of program learning outcomes within programs, and consideration of inter-disciplinary curricular development across programs.”

    Awards from Professional Organizations

    SMU School of Nursing Honored with QSEN Award
    “During the [QSEN] team’s visit to the School of Nursing in April, they were extremely pleased with how the School of Nursing has integrated the QSEN standards into the clinical evaluations and course assignments.” Arlene Sargent, Associate Dean of Nursing stated.

    Faculty award Best Poster at QSEN Forum

     

    SMU faculty were awarded best poster in the category of Academic/Research at the Quality and Safety Education for Nurses (QSEN) National Forum, May 27-29 in Baltimore, Maryland. The poster, entitled  “Aligning QSEN Competencies with Student Learning Outcomes and Student Work? There's an App for That!” 

    Article: SMU Faculty Awarded Best Poster Award

     

     

     

     

     

     

    Award-winning poster by 
Nancy Haugen PhD., Associate Professor, Chair, Accelerated Bachelor of Science in Nursing; Margaret Rivero Early PhD. Associate Professor, Bachelor of Science in Nursing; and Valerie Landau, Director of Assessment. 

    Valerie Landau
    Director of Assessment

    510.907.2431
    vlandau@samuelmerritt.edu

    Valerie Landau collaborates with faculty and staff to develop innovative methods and tools that facilitate a cultural shift in which improving teaching and learning is an integral part of SMU's core practice. Our accreditation agency WASC describes SMU’s transformation in building a culture of assessment as having “moved at mach speed to ramp up systems.

    Landau graduated from the Harvard Graduate School of Education with a specialization in technology, innovation, and education. She is the author of two books, "Developing an Effective Online Course" and "The Engelbart Hypothesis: Dialogs with Douglas Engelbart." She is also organizing a multi-institutional repository of effective faculty centered practices to spark innovation in assessment.