How should we measure success in schools?

When I arrived at my current school as principal, one of the first ideas we implemented was a schoolwide writing assessment. At that time, writing was taught once a week as part of a life skill block. My thinking was, if we assess it more formally, then student writing will improve because teachers will see it as a priority and start to teach it more frequently.

And at first, the approach seemed to work. Shortly after announcing the schoolwide writing assessment, one teacher came up to me and asked, “So writing is a focus for the building now?” I was noncommittal in my response but I did not exactly correct her thinking.

The heightened sense of writing as a priority eventually gave way to reality. Specifically, while writing instruction was happening more frequently in classrooms due in part to dedicated professional development, the actual assessment results did not reveal a lot of helpful information. Our leadership spent a whole day in the fall and in the spring calibrating the rubric with samples, assessing writing pieces with partners, and then adding students’ scores to the spreadsheet. For all of this work, we couldn’t make heads or tails if students were collectively growing as writers.

This experience has led me to ask some hard questions. Why is it so hard to measure student success in complex tasks such as writing? Related, why are we as teachers often the only ones who should be making a determination of success? If students should be a part of the assessment process, can the task itself be used for more than just measurement, such as motivating students to learn? 

Defining Success

The dictionary defines success as “the accomplishment of an aim or purpose”. So knowing what the aim is along with the purpose for our work seems important for students and teachers. Mindless assessment practices should be called into question, but not without a clear understanding of success in the broader sense of the word.

In their article “What Do We Measure and Why? Questions About the Uses of Measurement“, Margaret Wheatley and Myron Kellner-Rogers address the general culture of organizations and our infatuation with numbers. They note a disconnect between what is being measured and what the goals are of the group, along with how people move the organization toward these goals. 

“The desire to be good managers has compelled many people to become earnest students of management. But are measures and numbers the right pursuit? Do the right measures make for better managers? Do they make for a stellar organization?”

For our challenge, it felt (to me at least) that we were scoring our students’ writing in order to have a valid and reliable way of measuring their abilities. The result was a collective effort that was misaligned with greater goals.

  • There was misalignment between who was scoring the writing and who wrote it. For me, there was little context as I read and responded to each piece. For example, were they excited about the topic? If not, why not? Could the kids also present their work for an authentic audience? I wanted to pull each student aside to ask him or her more questions about their process that led them to what they produced.
  • There was misalignment between what was being taught and what was being assessed. Just because we had a common rubric did not mean the success criteria were clear for everyone in the school. Part of this issue is with rubrics themselves; they are lengthy and often too generic to use as a teaching tool. 
  • There was misalignment between our school’s mission and vision and how we were trying to realize these big goals.We were using simple instruments to give us information about complex work. Yes, kids had to respond to a task that had the potential to encourage students to write. But writing is more than words on paper. There is research involved, peer feedback and revision, and time to simply think, activities not a part of a writing assessment. 

Wheatley and Kellner-Rogers have also observed cognitive dissonance between the necessary actions people should take to realize an organization’s vision and how leaders choose to evaluate success due to a group’s actions.

“We believe that these behaviors are never produced by measurement. They are performance capabilities that emerge as people feel connected to their work and to each other. They are capacities that emerge as colleagues develop a shared sense of what they hope to create together, and as they operate in an environment where everyone feels welcome to contribute to that shared hope. Each of these qualities and behaviors-commitment, focus, teamwork, learning, quality–is a choice that people make. Depending on how connected they feel to the organization or team, they choose to pay attention, to take responsibility, to innovate, to learn and share their learnings. People can’t be punished or paid into these behaviors. Either they are contributed or withheld by individuals as they choose whether and how they will work with us.”

Described in this way, assessment is framed as an essential part of the learning journey toward success. The qualities of a community of learners should be embedded in the mission and vision of a school. Pulling these qualities out so they are understandable by all requires a description of success along with more authentic approaches for developing assessments that allow students to make their learning and thinking visible to all.

Engaging in this work is not necessarily more difficult. As Wheatley and Kellner-Rogers note about organizations that make assessment work for them, “their process was creative, experimental, and the measures they developed were often non-traditional.” It may instead require a shift in how we view success for all students, especially from their points of view. Maybe the word “measure” is itself too basic a label for what we are trying to really do: help students see themselves as genuinely successful and know how they arrived at this point. 

Rethinking Rubrics

In our Google+ Community on digital portfolios for students, we have been discussing the pros and cons of rubrics. Yes, they spell out what is expected regarding a summative assessment for a unit of study. Differentiating between levels of understanding can help teachers more efficiently assess assigned performance tasks of student learning. For teachers who are now evaluated within the Danielson Framework for Instruction, the focus is on a rubric.

Source: Wikimedia Commons
Source: Wikimedia Commons

So what’s the problem? Clear expectations and easy-to-apply assessment tools can make the learning lives of students and teachers more manageable.

This may be exactly why there is a problem. Assessment is not an easy practice to apply. Expectations for what excellence looks like for student learning can become more confusing when we parse out an understanding of mastery in the name of efficiency. Plus, there is the debate about defining poor performance. How much attention should a “1” really be given? Why is a “1” (see: failure) even an option offered to our students?

In this post, I propose three alternatives to rubrics when designing units of study. I am not anti-rubric; rather, we should consider the possibilities when designing instruction for deep student understanding and strong skill development.

Possibility #1: Analyzing Exemplary Pieces of Student Work

This approach works really well with skill-focused learning, such as writing. Showing students what is expected to achieve excellence with examples from past learners can have a better impact.

Below is an example: A mastery wall of student writing, compiled by grade level teams as a mid-year informative writing check.

Screen Shot 2015-07-17 at 10.43.28 PM

How might sharing and analyzing exemplary student work be an improvement over rubrics?

Possibility #2: Standards of Excellence

I don’t know if you have noticed, but the Common Core State Standards, especially the literacy anchor standards, read like a rubric. Each phrase within the standard addresses a specific understanding or skill. Standards of excellence are paragraph-length descriptions of what a student should know and be able to do after a progression of learning activities. What is described is what is expected. Anything less is scored below a “4”, largely at the discretion of the student + teacher discussing the work.

Here is an example I created for a unit of study on narrative writing:

As an author, craft an original story, real or imagined, that has a beginning, middle, and end. This story shall have an attention-grabbing lead, rising action that keeps the reader going, and a satisfying conclusion. It shall be free of confusing language and grammatical errors. In addition, your story shall be both entertaining and informative.

How might crafting a standard of excellence be an improvement over rubrics?

Possibility #3: Novice vs. Expert Understanding

If the concept of a rubric is hard to depart from, consider this alternative. It comes from the Understanding by Design framework by Grant Wiggins and Jay McTighe. The teacher determine what is a basic understanding derived from a unit of study, and contrasts that with a deep understanding which warrants a high level of recognition. Here is an example from a 4th grade unit on state history and geography:

Screen Shot 2015-07-17 at 11.11.20 PM

Both effort and skill development are recognized within this assessment.

How might differentiating between novice and expert understanding be an improvement over rubrics?

What are your thoughts on this topic of rubrics and alternative assessments? Please share your thinking in the comments.