“Multiple choice tests have been the dominant type of test in this country since World War I, but that is about to change.” — Roger Benjamin
If not the SAT, What?
By C. M. Rubin with Harry Rubin and Michael Freeborn
What if educators were able to assess the 21st century skills that thought leaders in The Global Search for Education series talk about week after week? What if this 21st century replacement for the SAT was able to measure high school students’ critical thinking, analytical reasoning, problem solving, and written communication; in other words, all the essential skills that both educators and employers have said students need to succeed in college and the workplace?
Until we are able to assess critical thinking, it probably won’t get taught. PISA’s international problem-solving standardized test does assess this, but according to Harvard University’s Dr. Tony Wagner, when it comes to the forward thinking model for American schools and colleges to watch, “The College and Work Readiness Assessment (CWRA/CLA) is really in a class by itself.” Wagner sees CWRA as an essential part of what he calls “Accountability 2.0.” But he adds it “should be accompanied by audits of students’ digital portfolios which show evidence of progressive mastery of the skills that matter most, such as critical and creative thinking, communication, and collaboration.”
Naturally, I was curious to learn more about this promising candidate for the gold standard in the future world of assessment. I recently got the opportunity to chat with the President of the New York based CAE (Council for Aid to Education), home of the Collegiate Learning Assessment (CLA), Dr. Roger Benjamin. Prior to his role at CAE, Roger was a senior research scientist at RAND from 1990 to 2005 (director of RAND Education from 1994-1999).
“It is becoming clear that schools, colleges and employers are eager to move beyond reliance only on multiple choice tests.” — Roger Benjamin
Do you think that standardized tests should be used to evaluate the abilities of high school students for admission to college? Do you think that GPA alone would be sufficient?
I do think there is a role for standardized tests because of the grade inflation in high school GPA’s, widely disparate metrics used to evaluate students, and the advantages students from affluent high school districts have compared to students from disadvantaged schools. There are students everywhere in the United States that, when given the chance, demonstrate extraordinary potential. Standardized tests are therefore an important additional tool for admissions officers. However, standardized test protocols must become better aligned with the education reform movement underway, including the common core standards movement which calls for open ended tests instead of the heavy reliance on multiple choice tests, which are not worth teaching to.
In your own words, would you describe for me what you believe the current SAT measures versus your own performance assessments?
The SAT measures the aptitude high school seniors have for doing well in college through reasoning and verbal abilities tests. The CWRA measures high school students’ critical thinking, analytical and quantitative reasoning skills, problem solving, writing mechanics and writing persuasiveness skills that educators and employers believe high school graduates need to have to succeed in college and work.
“There are important soft skills such as creativity and collaboration, but the challenge is how to measure them at the same level of scientific reliability as the skills that we are currently measuring reliably.”— Roger Benjamin
Why is the College and Work Readiness Assessment a better option to the SAT?
I would replace the word “better” with the word “different.” The CWRA is more congruent with the requirements of today’s “Knowledge Economy” in which it is more important to be able to access, structure and use information than to only accumulate facts. Multiple-choice questions require the ability to recognize a painting. In comparison, performance tasks require the student to paint. Definitions of learning have shifted to the ability to apply what one knows to new situations. Performance assessments capture this change. We are not sure whether the CWRA+ will compete directly with the SAT or be thought of as additional important information for college admissions officers. However, it is becoming clear that schools, colleges and employers are eager to move beyond reliance only on multiple-choice tests.
Students learn best when they are being taught to learn as opposed to being taught to pass a test. Can we overcome the problem of teaching to a test?
A strong point of the performance tasks (case studies and realistic problems) of the CWRA+ and CLA+ is that it is very difficult to teach to these tests. There are not simple “right” or “wrong” answers. The focus is on how well the student reasons, not on whether they get the “facts” right. Indeed, all the facts needed to answer the tasks are provided to the students taking the test.
The SAT is a multiple-choice test with the exception of the writing test. Grading of the multiple-choice test is easy. The answers are either right or wrong. The CWRA assessment is more about judgment. How do you assure grading consistency across the exams taken in different locations and at different time periods?
Performance assessments such as CWRA have been around a long time. Teachers have not liked multiple-choice tests but we could never figure out how to take to scale the more complicated problem solving assessments that constitute the CWRA. However, we can now train human scorers to score open-ended essays based on scientifically designed rubrics as reliably as multiple-choice tests. Moreover, computer assisted scoring, built on our human scoring protocols, score these responses at levels that are as reliable as human scoring, which further reduces error and cost.
“Multiple-choice questions require the ability to recognize a painting. In comparison, performance tasks require the student to paint.” — Roger Benjamin
What would be your best argument for keeping the SAT?
It is relatively cheap and easy to administer and score. It has a significant number of reliability and validity studies that corroborate its efficacy in predicting a student’s GPA in the first year of college.
You speak a lot about measuring critical thinking skills. How much do these new CWRA assessments require students to have honed their creative skills?
That’s a good question. We know that students in the arts and sciences at the college level do better than students who are in vocational or applied subjects. We think that is probably because they do more analytic based writing and are involved in more open Q and A sessions. However, there is a lot of work to be done on how creativity plays into doing well on assessments like this. So that is an interesting question.
Do you believe that strong creative skills are important to the process of identifying problems and finding solutions?
There are important soft skills such as creativity and collaboration, but the challenge is how to measure them at the same level of scientific reliability as the skills that we are currently measuring reliably. Hopefully we will succeed in measuring these additional skills that are also very important.
“The CWRA measures high school students’ critical thinking, analytical and quantitative reasoning skills, problem solving, writing mechanics and writing persuasiveness skills that educators and employers believe high school graduates need to have to succeed in college and work.”— Roger Benjamin
How many schools nationally are using the CWRA performance assessment and how many college admissions offices are asking for it? How many will accept it instead of the SAT?
This year, 120 high schools are using it. We are now talking to college admissions officers and leaders of colleges who are aligned with high schools that feed into them, and are also CWRA+ users, about accepting the CWRA+ results in addition to SAT or ACT results. There is much controversy in education at present about standardized testing, with many believing all standardized testing should be eliminated during primary and secondary school, with just one test given at the end of secondary school.
What are the logistical issues you face in getting broader distribution for the CWRA assessment?
We now have a new version of CWRA called CWRA+ to be used in the college admissions space. We have started to market this new assessment through partnerships with colleges who will accept the CWRA+ results as part of admissions’ records. The Obama administration’s Race to the Top program launched the biggest testing educational initiative ever funded by the federal government and we won the right to develop a number of assessments. These newer assessments are becoming mainstream almost overnight. It’s because of the “Knowledge Economy” - content is important but when you can google for facts you’ve got to be able to think about what it is you’re going to be googling for. Multiple-choice tests have been the dominant type of test in this country since World War I, but that is about to change.
Dr. Roger Benjamin and C. M. Rubin
Photos courtesy of Tech Valley High School, Rensselaer NY; New Tech Network and Traverse Bay Area Independent School District, Michigan/Mancelona Public Schools/Joanie Moore.
In The Global Search for Education, join me and globally renowned thought leaders including Sir Michael Barber (UK), Dr. Michael Block (U.S.), Dr. Leon Botstein (U.S.), Professor Clay Christensen (U.S.), Dr. Linda Darling-Hammond (U.S.), Dr. Madhav Chavan (India), Professor Michael Fullan (Canada), Professor Howard Gardner (U.S.), Professor Andy Hargreaves (U.S.), Professor Yvonne Hellman (The Netherlands), Professor Kristin Helstad (Norway), Jean Hendrickson (U.S.), Professor Rose Hipkins (New Zealand), Professor Cornelia Hoogland (Canada), Honourable Jeff Johnson (Canada), Mme. Chantal Kaufmann (Belgium), Dr. Eija Kauppinen (Finland), State Secretary Tapio Kosunen (Finland), Professor Dominique Lafontaine (Belgium), Professor Hugh Lauder (UK), Professor Ben Levin (Canada), Lord Ken Macdonald (UK), Professor Barry McGaw (Australia), Shiv Nadar (India), Professor R. Natarajan (India), Dr. Pak Tee Ng (Singapore), Dr. Denise Pope (US), Sridhar Rajagopalan (India), Dr. Diane Ravitch (U.S.), Richard Wilson Riley (U.S.), Sir Ken Robinson (UK), Professor Pasi Sahlberg (Finland), Andreas Schleicher (PISA, OECD), Dr. Anthony Seldon (UK), Dr. David Shaffer (U.S.), Dr. Kirsten Sivesind (Norway), Chancellor Stephen Spahn (U.S.), Yves Theze (Lycee Francais U.S.), Professor Charles Ungerleider (Canada), Professor Tony Wagner (U.S.), Sir David Watson (UK), Professor Dylan Wiliam (UK), Dr. Mark Wormald (UK), Professor Theo Wubbels (The Netherlands), Professor Michael Young (UK), and Professor Minxuan Zhang (China) as they explore the big picture education questions that all nations face today.
C. M. Rubin is the author of two widely read online series for which she received a 2011 Upton Sinclair award, “The Global Search for Education” and “How Will We Read?” She is also the author of three bestselling books, including The Real Alice in Wonderland.
Follow C. M. Rubin on Twitter: www.twitter.com/@cmrubinworld
Finnish teachers talk with Harvard professor Tony Wagner in The Finland Phenomenon
More Focus on Finland
By C. M. Rubin with Harry Rubin and Michael Freeborn
“The Finns had a crisis,” life-long educator, best-selling author, and Harvard professor Tony Wagner explains as we discuss his new film, The Finland Phenomenon, made with acclaimed documentary filmmaker, Bob Compton. “Their economy was failing. Their education system was poor. They knew that to grow their economy, they had to transform their educational system.” Starting with the principle that cooperation is a key pillar of success, the Finns revised their educational framework.
“I saw teachers in Finland that were better than 90% of the teachers I see in America,” says Wagner. There were many things that led to Finland topping the international education league tables (ten years and counting). A key driver: a tremendous investment in teaching made it the most sought-after profession in Finland.
Compulsory schooling now begins at seven. School is a place where students discover who they are and what they can contribute. National testing and school inspections are banished (teachers are trusted to assess their students). Classroom size has been reduced (limited to 20 students). Students are permitted to transfer to an academic or vocational school at the age of 16, and no university fees are charged for Finnish or European Union students.
This educational reformation has made them world leaders. Not surprisingly, global policy makers are paying more attention. Pasi Sahlberg, Director General of CIMO in Helsinki, Finland (the Centre for International Mobility and Cooperation) now advises policy-makers in over 40 countries on matters relating to education and its reform. Four months before the release of his highly anticipated new book, “Finnish Lessons: What can the world learn about educational change in Finland,” Pasi Sahlberg spoke with me about the characteristics of successful educational systems, and about what is missing from many systems around the world.
What kind of education system will permit a country to have the people skills needed to compete globally?
The education system must be equitable, accessible, and flexible. Global competitiveness requires that all people develop competencies for life and work, not just some people. This means that a successful education system should help young people to discover their talents and build their lives based on them. Reading, mathematical, and scientific literacy will remain important, but their role as ‘core subjects’ in competitive education systems will be challenged by creativity, networking skills, and imagination.
An equitable education system makes sure that all students will perform well. It will provide early support to those who need more help in their learning than others. It will also emphasize caring and well-being in school (through healthy nutrition, medical, dental and psychological health), rights of students in school, and shared responsibilities in education and upbringing of children with parents.
Accessibility means that the education system provides good schooling for all, regardless of where people live or what they do. The education system that can offer unified and comprehensive basic education, rather than diversified provision of schooling (through private or non-public schools), will have better opportunities to respond to the changing needs of the competitive and complex world.
Flexibility is about providing adequate individual personalization in school, and freedom for schools to craft their curricula based on their capacities and local needs.
I know that Finland has banished national testing. How do you see the problems with standardized testing?
The main problem with standardized testing today is the quality of these tests. As learning in the globalized world is becoming increasingly complex and diverse, to test what pupils have learned through standardized tests is becoming more complicated. The increasing amount of what students learn cognitively today, let alone what they will learn tomorrow, is due to out-of-school influences, not the teacher or school. Standardized tests by definition are designed based on curriculum and textbooks, not the real world. Therefore, most standardized tests promote narrowing pedagogies, focus on core subjects and knowledge, and prevent teachers from teaching their curricula flexibly. Another problem with standardized tests is that as soon as you have invested in them, you want to also use them for all sorts of purposes for which they were not meant to be used, like determining the quality of schools and comparing them to each other, or measuring the effectiveness of teachers.
What elements are missing from the preponderance of the current systems?
Education systems in general pay too little attention to helping everybody find their own talent in school. It is evident everywhere that most people, after they have completed compulsory education at the age of 16 or 17, think that they are not good at anything. There is a small minority of those young people who say that they know what their talents are and that this is because of what they did in school. Another missing emphasis in current education systems all around the world is focus on helping young people to develop social skills and competencies that they need in their lives (that are dominated by communication through gadgets). This could also be called a lack of focus on developing social intelligences in school.
What can be done to better address the emotional well being and intellectual potential of the individual, which appear to be suffering under current systems?
Emotional well-being can be addressed by reducing the academic dominance in schools and by increasing the social and creative aspects in what students do. It is a common misconception that competitive economies in a globalized world would require that children and students be prepared for them by environments that are based on more competition. It is the opposite. To prepare young people for the competitive world requires more cooperation in classrooms and between schools. All national programs, like Race to the Top, will jeopardize school, teacher, and student efforts to cooperate as they reward winners in the race and punish losers in public tests.
From a larger perspective, does your country’s definition of educational excellence take into account the quality of life of individuals and of a society?
Educational excellence in Finland is a broad concept that spans far beyond academic achievement measured in standardized tests. Indeed, quality of life, overall well-being, and happiness are important criteria when teachers and schools decide whether their individuals or organizations have performed well or not. Artistic and cultural achievements are seen in most of our schools as the main indications of being an educated individual.
World Wisdom from Finland
Global competitiveness requires that all students develop competencies for life and work, not just some students. Therefore, a country’s educational system must be equitable, accessible, and flexible. Cooperation, not competition, is a principal pillar of educational system success. Also essential is a tremendous investment in teaching quality. But beware of standardized testing, as it will undermine the achievement of these objectives.
In The Global Search for Education, join C.M. Rubin and globally renowned thought leaders including Sir Michael Barber (UK), Dr. Leon Botstein (US), Dr. Linda Darling-Hammond (US), Dr. Madhav Chavan (India), Professor Michael Fullan (Canada), Professor Howard Gardner (US), Professor Yvonne Hellman (The Netherlands), Professor Kristin Helstad (Norway), Professor Rose Hipkins (New Zealand), Professor Cornelia Hoogland (Canada), Mme. Chantal Kaufmann (Belgium), Professor Dominique Lafontaine (Belgium), Professor Hugh Lauder (UK), Professor Ben Levin (Canada), Professor Barry McGaw (Australia), Sridhar Rajagopalan (India), Sir Ken Robinson (UK), Professor Pasi Sahlberg (Finland), Andreas Schleicher (PISA, OECD), Dr. David Shaffer (US), Chancellor Stephen Spahn (US), Yves Theze (Lycee Francais US), Professor Charles Ungerleider (Canada), Professor Tony Wagner (US), Professor Dylan Wiliam (UK), Professor Theo Wubbels (The Netherlands), Professor Michael Young (UK), and Professor Minxuan Zhang (China) as they explore the big picture education questions that all nations face today.
Finnish class in session
Professor Pasi Sahlberg and C. M. Rubin
C.M. Rubin has more than two decades of professional experience in development, marketing, and art direction for a diverse range of media businesses. She is also the author of three bestselling books, including The Real Alice In Wonderland.
Follow C. M. Rubin on Twitter: www.twitter.com/@cmrubinworld