Just want to connect a couple of dots between a very thoughtful, challenging essay by Dan Willingham and Andrew Rotherham that was re-released by Educational Leadership just recently, and another snip from Steve Hargadon’s interview with Linda Darling Hammond from last week. I think they frame the really huge problem we’re facing with the current assessment regime that should have us all rolling up our sleeves and setting to work despite the fact that none of our elected leaders seem to have a clue as to what’s best for our kids when it comes to this stuff.
The Ed Leadership essay suggests that while these â€œ21st Century Skillsâ€ are really any century skills, the path to â€œsuccessâ€ (depending on how you define that) is more dependent on having those skills today than ever before. And the greater problem right now is that getting really deep exposure to those skills is at best a hit or miss (mostly miss) proposition for kids in our country today. As the authors say, it’s â€œakin to a game of bingo.â€ But there are big hairy problems here regarding curriculum, professional development and assessment. And here’s one part that really resonated:
Another curricular challenge is that we don’t yet know how to teach self-direction, collaboration, creativity, and innovation the way we know how to teach long division. The plan of 21st century skills proponents seems to be to give students more experiences that will presumably develop these skillsâ€”for example, having them work in groups. But experience is not the same thing as practice. Experience means only that you use a skill; practice means that you try to improve by noticing what you are doing wrong and formulating strategies to do better. Practice also requires feedback, usually from someone more skilled than you are.
It’s that last part that really hits home, and again speaks to the pressing need for rethinking the professional development we provide to our current teachers, and the preservice preparation we give new teachers. I know this is a recurring theme here, but if we really value these skills (and I think we should), how can we teach self-direction, collaboration, creativity et al if we’re not practicing those things ourselves? It would be like being asked to teach Physics with only a textbook understanding of it. It’s what Sheryl and I and our many community leaders are constantly trying to nudge teachers toward, being learners in all of those contexts. And for most, it’s hard work to get out of the traditional roles and expectations which don’t include much beyond management of the classroom, the curriculum and the outcomes.
But at the end of the day, it’s the assessments that drive this. And this is the most depressing piece, I think. Here are Rotherham and Willingham again, longish, I know, but important:
There is little point in investing heavily in curriculum and human capital without also investing in assessments to evaluate what is or is not being accomplished in the classroom. Fortunately, as Elena Silva (2008) noted in a recent report for Education Sector, the potential exists today to produce assessments that measure thinking skills and are also reliable and comparable between students and schoolsâ€”elements integral to efforts to ensure accountability and equity. But efforts to assess these skills are still in their infancy; education faces enormous challenges in developing the ability to deliver these assessments at scale.
The first challenge is the cost. Although higher-level skills like critical thinking and analysis can be assessed with well-designed multiple-choice tests, a truly rich assessment system would go beyond multiple-choice testing and include measures that encourage greater creativity, show how students arrived at answers, and even allow for collaboration. Such measures, however, cost more money than policymakers have traditionally been willing to commit to assessment. And, at a time when complaining about testing is a national pastime and cynicism about assessment, albeit often uninformed, is on the rise, getting policymakers to commit substantially more resources to it is a difficult political challenge.
Producing enough high-quality assessments to meet the needs of a system as large and diverse as U.S. public schools would stretch the capacity of the assessment industry, and incentives do not exist today for many new entrants to become major players in that field. We would need a coordinated public, private, and philanthropic strategyâ€”including an intensive research and development effortâ€”to foster genuine change.
Substantial delivery challenges also remain. Delivering these assessments in a few settings, as is the case today, is hardly the same as delivering them at scale across a stateâ€”especially the larger states. Because most of these assessments will be technology-based, most schools’ information technology systems will require a substantial upgrade. [Emphasis mine.]
They paint a daunting picture. But what really irks me is that once again, we’re trailing the field when it comes not just implementing more effective assessments but even conceptualizing them. Listen to this short snip from the Linda Darling Hammond interview (full recording here), most of which I’ve excerpted below:
In other countries they’ve got assessments that are fewer, that are higher quality, that include predominantly open ended essays and items research projects and scientific investigations, and so when they think about what it means to going to school, and what you should be learning, they take seriously the question of what is the intellectual activity that we want to have going on here. They don’t just attach a bunch of rewards and sanctions to low quality test measures as we’ve done here and say let’s let that be the tail that wags the dog without thinking about what we want kids to be learning, doing and able to do with their knowledge when we get out. [Emphasis mine.]
What a concept.
Maybe it’s just that the people in charge don’t have the creativity, innovation and problem solving skills to figure this out. (They are products of the system, after all.) But here’s the deal: so what? School’s starting, and it’s 2010 which means we’re in â€œdoing bothâ€ mode. We’re making sure the kids pass the test, but we also have to make sure that our own assessments are doing more to evaluate our students ability to do all those other things we want them to be able to do that aren’t currently being assessed.
So, I’m wondering, how are you doing that?
dave cormier says
If educators don’t see themselves as learners… we’re toast either way. More experience (in some ways) learners working with newer learners who are (usually) less experienced.
Bill Fitzgerald says
I wrote about this a facet of this a little while back in a post titled (wait for it) Assessment.
The whole issue of how to measure teacher effectiveness and student learning is the iceberg beneath the recent flare-up over the Value Added Assessment debacle sparked by the LA Times. Oddly, much of the emphasis was on the use/accuracy of Valued Added Assessment as a means of analysis; the whole notion that standardized tests are a flimsy foundation for analysis to start with got lost among the other debates.
And this is part of the issue: educators are reacting to, rather than driving, policy decisions and curricular decisions.
Ernest Koe says
I’d like to know which “other countries” are doing a better job at student assessment.
Scott McLeod says
I’d like to see examples of some of the assessments of higher-order thinking that other countries are using. So far all I can find is the College Workforce Readiness Assessment (www.cae.org/content/pro_collegework.htm). Would love to see others!
Matt Townsley says
I would also like to know…are ALL students in these countries required to take these “higher-order thinking” assessments? A current reality in the United States is teaching and assessing every student. If this focus was narrowed, it would require fewer resources to score/assess/provide feedback among other benefits.
Mary Ann Reilly says
I think we do have fine models of complexity in public school classrooms such as Brent Davis’s and Elaine Simmt’s work regarding professional learning and mathematics or Luis Moll’s Funds of Knowledge research (see citations below). One issue though is that work of this sort is not featured in the journals many administrators rely on such as ASCD’s Educational Leadership or NSDC’s JSD. These two journals offer brief texts and in the former case often offer “solutions” alongside a product to sell.
Occasioning complex learning (self direction, collaboration, creativity) is decidedly different from teaching the “basics”. I discuss this in a recent blog post (http://maryannreilly.blogspot.com/2010/08/slow-fuse-of-possibility-why-race-to.html). I also have a few articles recently published that explore complexity being occasioned inside public school classrooms (see list below). It is being done, but may not be reported in more main stream press as it cannot be packaged and sold.
Davis, B. & Simmt, E. (2003). Understanding learning systems: Mathematics teaching and complexity science. Journal for Research in Mathematics Education, 34(2), 137-167.
Moll, L. (2005). Reflections and possibilities. In N. GonzÃ¡lez, L. Moll, & C. Amanti (Eds.). Funds of knowledge: theorizing practices in households, communities, and classrooms (pp. 2275-287). Mahwah, NJ: Lawrence Erlbaum Associates.
Moll, L., Amanti, C., Neff D., & GonzÃ¡lez, N. (2005). Funds of knowledge for teaching: Using a qualitative approach to connect homes and classroom. In N. GonzÃ¡lez, L. Moll, & C. Amanti (Eds.). Funds of knowledge: theorizing practices in households, communities, and classrooms (pp. 71-87). Mahwah, NJ: Lawrence Erlbaum Associates.
Reilly, M.A. (2009). Restoring points of potentiality: Sideshadowing in elementary classrooms. The Reading Teacher, 63 (4), pp. 298-306.
Reilly, M.A. (2009). Saying what you see in the dark: Engaging students through art. LEARNing Landscape, 3(1),69-89.
Reilly, M.A. (2009). Dressing the Corpse: Professional learning and the play of singularities. Journal of Curriculum & Pedagogy, 6(1), 79-99.
Reilly, M.A. (2009). Opening spaces of possibility: Teacher as bricoleur. Journal of Adolescent & Adult Literacy 52 (5), 376-384.
Laura Deisley says
The Willingham/Rotherham piece is excellent. It was first published in the September (?) 2009 issue of Educational Leadership which focused on 21st century learning. The entire issue was so accessible for teachers, we ordered magazine copies for everyone in our Dobbs 21st Century Learning Cohort–a cohort of public and private school teachers that was working together in a PLC model for a year. I encourage everyone to read the article, and go back to that issue if they have not read it.
You ask, “what are we doing”? For one, I think it is incumbent on those of us who are really passionate about this topic to initiate and persist the conversation. Internally, I have pushed our academic deans and school heads to make this a priority, and I’m asking teachers to share with me their planning documents and rubrics whenever possible. I really like the idea of having a common rubric (nod to Lehmann and the folks at SLA), and I’m pondering how to evolve that notion within our schools.
McLeod tweeted this weekend about alternative assessment models that measure this important stuff, and mentioned the College Work Readiness Assessment(CWRA). Tony Wagner has been involved with those folks, and we’ve sent two groups now up to St. Andrews (Delaware) to explore that assessment. Although the jury is out, I think we will consider offering it to our high school students-if nothing else to get a better idea of whether what we are teaching is indeed helping develop these core competencies.
As you and I have discussed before, as a large independent school community we are not “hung” by a battery of meaningless tests day in and day out; however, we are strangled by these high stakes “end of the line” tests that are the gatekeeper for our students in a college prep environment. Wish it were different, but it’s the current reality. I do sense, more than I have, that the tide is starting to turn a little. I’m spending a lot more time with parents in order to bring them into the conversation. The more people we have talking about it, the more likely shifts can be made.
Thanks for the post. Assessment is vital.
J. R. Radney says
I think there is a bit of a problem with us thinking we have the process of teaching (even long division) down pat, and regarding some more recently-seen relevant skills (collaboration, self-direction, and so forth) I believe the observations of Martin Heidegger are quite relevant (the quote is lengthy, but worth the read):
J. R. Radney says
I’m sorry for the inaccuracy above. The Heidegger quote is not from Being and Time, but from an essay, What Is Called Thinking?, published in 1954 and translated into English in 1968.
Terry Elliott says
Mastery emerges from practice not ‘exposure’–that is my takeaway from this article and from my own experience. Getting out of the ed echo chamber for a moment I think we might take a cue from weightlifters and triathletes. There are two important elements to success in these disciplines: social support and real practice. In other words you need to be surrounded by a cadre of like-minded practitioners for all the obvious reasons AND you need to challenge yourself to a public performance on a regular basis. For these folks there is always and end as well as a new beginning.
Assessment is the problem in education as long as it crowds out feedback. The main difference between the two is that one promotes fear while the other defuses it. The anticipation of a powerlifting event or a sprint triathon is not the same as the anticipation of a grade in a class or at the end of the semester. The former aids us on the road to mastery while the latter is to be gotten over and forgotten.
Will, we are indeed doing both but I think that means we are doing neither very well. And worse the system cannot abide a parallel track. Education is broken, pulling along a trailer with flat tire behind it, sparking and weaving until it flips or grinds down to the axle. Well…that may be too goofy an image but my point is that as long as we are doing both we are doing neither very well. And that is built into any system that doesn’t play well with others.
Of course, the world is full of a parallel tracks, informal trains of learning. I think that the key to American success is “to organize a school just well enough to get teachers and students together and see that it gets no better organized.” This quote is from Myles Horton who is describing the experience of the Dutch Folk School movement in 1931! (Horton, 1997, p. 53)
Horton, M. (1997). The Long Haul: An Autobiography. Teachers College Press.
This makes me think of an article I read a while back and continue to reference as I think about problems in education. It was published in Psychology Today in 2008 and is about how the children of modern day hunter-gatherers educate themselves. Link to the article:
What I find interesting is that the “play” of the children is structured, loosely at first and more cohesively as they age, around the tasks that they observe adults doing. Because they are not by and large separated from their parents or other adults in the community, they learn by observation and, as they age, from direct instruction in the skills they will need to be socially adept providers for their eventual families. And, when we boil it down even in 21st century modern America, aren’t we really striving to have socially adept, skillful providers of goods and services that are capable of raising their own offspring so as to perpetuate the cycle?
The “short, nasty, brutal life” hypothesis of stone-age man and modern hunter-gatherers has been debunked. They have legitimate cultures that, while very different in expression from our own, provide the same fundamental service to society as our own cultural products do.
I digress; apologies. To bring this back to what it means for education, I wonder if the potential for more parents to telecommute or bring their children with them to work, at least some of the time, would not begin to build the framework needed for teaching “21st Century Skills?”
And I agree that these “21st Century skills” are not really 21C-specific – they are the life skills we all need regardless of the temporal frame of our existence. But if children see their parents doing work in the context of what our modern workplace demands, as opposed to just what home chores demand, they may begin to take education more seriously and engage themselves, rather than forcing teachers and administrators to scramble to design innovative curricula and delivery to keep kids focused, attentive, and productive.
This would require taking project based learning to a whole new level. After the “parrot-stage” of grammar school where facts and language arts/math rules are drilled into memory, children would need to start seeing how those facts and skills are used in real-life settings. The need for creativity in a child’s educational products really can’t express itself until certain facts and figures are there to apply to a given situation. Assessing how much they’ve absorbed is one thing that is easy to do. Seeing how they apply that knowledge is where assessment needs to go – especially for older children.
Mrs. Levine says
If we are among the lower performing countries in terms of student achievement, what in the ‘world’ is so tough about following the models of higher achieving systems? And even in some studies in the U.S., high schools that start 1 hour LATER are showing a 30% increase in test scores. Why is this still considered ‘experimental’? And why can’t the kids kick the football an hour later, or start their ‘farm work'( job) an hour later? Studies also show that during the summer, children who are not able to participate in meaningful summer activities ( meaningful generally translates from ‘expensive) such as science camps, travel opportunities, etc. end up losing a lot of what they learned during the year. Why is it so difficult to change the school calendar? Why do we just plug along doing the same thing when it is so obvious that the same isn’t working? Sign me, ‘Frustrated Parent”.
Brenda Marfin says
While I completely agree with all that was articulated by the authors, I feel crucial a component is missing. Recently, I have been reading Breakthrough by Michael Fullan, Peter Hill and Carmel Crevola. Both groups discuss the same concerns and suggested solutions for revamping our current status quo.
We need more meaningful assessments, understandings of content and skill, and a delineation of practice and skill.
If breakthroughs, utilizing instruction that works, assessing for readiness and understanding, learning to learn, focused instruction unique to content learning, and other more high quality improvements need to be made, then why do things still look so much the same? The old adage, wherever I go, there I am, rings true as we stay stuck in the status quo. How much have we really changed since the establishment of the first compulsory school in the United States? Sure, we have become more cognizant of the science behind learning. We are aware of motivation and the importance of learner readiness. The breadth of knowledge and understanding is staggering. My argument is that we have not been able to build our systems within schools and districts to genuinely improve. The way in which we schedule classes, structure the day, all those logistics, actually confine us to a system designed to meet the demands of early America, not the 21st century. Articles such as this only add to frustration because we are reminded of the design constraints of a school.
We have to begin to explore ways to allow teachers more time to collaborate and grow professionally. Schedules need to be redesigned to meet the needs of the learner and the teacher, not just transportation, UIL or athletics. Is this radical? Is there time for such an extreme remedy?
Personally, I need to find answers to a few questions. How do other countries find the time for teachers to collaborate? Is there any wasted time during the school day? Our country is accustomed to the 8-3 schedule of schools that giving students a day off during the week so teachers can collaborate is often fraught with concerns from parents about finding a sitter.
Anyway, great article and good thought.
Carl Anderson says
How does one define “higher-order thinking?” If you define it too narrowly you may be able to assess it but then anyone engaged in critical thinking on the matter may produce responses that indeed are beyond the scope of the assessment’s definition and thus be considered wrong. Additionally, where in the traditional scope of the curriculum spectrum is the capacity for critical thinking nourished? In as far as I can tell it is in the areas of learning that are not well assessed through objective means, in subjects that are only truly measured via subjective assessments. Sure you can measure a painting, a song, a poem, or dance strictly by the objective elements used to produce it but to do so kills the essence of it. Typically students engaged in the fine arts, so long as they are engaged at a level that includes analysis, interpretation, and judgment, are developing this capacity for critical thought. However, since reducing these sorts of activities to their elements to measure them objectively kills what it is we value in them they are more and more being undervalued in this environment of “accountability.” Every year I see more and more of these programs cut, so much so that I was forced to make a career change three years ago to become a technology integration specialist instead of clamor after what remains of mostly part-time art teacher positions along with all the other displaced by this system of “accountability.” Who will be held accountable when our subjective values are lost, when what is valued in our schools is reduced to what can be objectified?