Cross-posted from Education Week
I vividly recall the first time I visited at the main offices of Cambridge International Examinations at Cambridge University in England. It was somehow a bit daunting to know that Isaac Newton had taught and written on mathematics only steps away. The university is over 800 years old. Cambridge Assessment itself goes back more than 150 years. The largest organization of its kind in the world, its examinations are today used in more than 160 countries worldwide.
But it was not the pedigree of Cambridge International Examinations that impressed me most. It was their attitude toward their work. In the United States, test construction is largely in the hands of testing experts, whose expertise lies in the science of measuring mental capacities and processes. But, at Cambridge, and in much of the rest of the world, test construction is mostly directed by people whose primary expertise is curriculum. Actually, to get persnickety, one should say “test” when we mean to assess what a student knows and can do irrespective of the particular curriculum to which the student is exposed and “examination” when our aim is to find out the degree to which the student has mastered a particular curriculum. In the United States, we mostly build tests. In much of the rest of the world, they mainly do examinations.
This is not a minor difference. Because decisions on curriculum in the United States have been highly localized, often down to the classroom level within a school, we long ago decided that it would be unfair to have standardized examinations based on a particular curriculum. Some students would be highly advantaged if they had happened to have a curriculum that lined up with the assessment, and others would be similarly disadvantaged, we thought.
Many American testing experts care more about whether a given test item produces a well-shaped curve of student responses than whether it accurately captures some aspect of the subject being assessed. That would never happen in most other countries, where the test of the adequacy of an examination question is the degree to which it captures the essence of the course. That is why curriculum experts, not testing experts, are put in charge of examination construction.
This gets us to the form, not just the purpose, of American tests versus others’ examinations. Cambridge International Examinations does not build tests to be scored by computers. They build examinations to be scored by human beings. They and their customers worldwide know that that is more expensive, but they think the difference in cost is worth it. In this case, as in so many others, form follows function. They want to know how students deal with the kind of complex problems and complex thinking that lie at the heart of their courses. They want to know how good students are at coming up with good answers that the test-constructors never thought of. They want to know how good a student is at making a competent and even compelling oil painting. Or constructing a robot. Or writing a 20-page history research paper. These are not things that multiple-choice, computer-scored tests can measure, but they are important to people who have built curricula intended to enable students to do these things. Allow me to observe that the environment now being created by the epic changes taking place in the global economy will demand exactly the kinds of skills that the Cambridge examinations are designed to measure.
The shortcomings of American-style tests are understood by leading American curriculum and testing experts and some have been trying to do something about them, but many states that improved their assessment systems in recent years have been forced by the fiscal crisis to revert to the multiple-choice, computer-scored testing regimes they were trying to get away from.
Interestingly, the United States is home to a program that is, to my knowledge, the world leader in encouraging and assessing the kind of non-fiction writing that is now in greatest demand in the world: the Concord Review, run by Will Fitzhugh. The Concord Review is a refereed journal of history writing done by high school students all over the world. The quality of the thinking and writing in the articles that appear in the Concord Review is nothing short of remarkable.
As high stakes standardized testing in the United States took center stage in recent years, serious writing in high school came under siege. Very few high school students are asked to read an entire non-fiction book while in school — just one. Even fewer are asked to write serious research papers. Why? Because they are not assessed for these skills on the high stakes tests, and, that being the case, the students are not assigned this kind of reading or writing. In the name of focusing on the development of a limited set of basic skills, we have set up our system so that it has depreciated the more advanced skills on which the future of our nation depends. It is as if we cannot talk and chew gum at the same time.
The Concord Review may be the last bastion of high standards and good sense in this arena. It is telling that Fitzhugh, who operates the Concord Review on a shoestring, cannot attract foundation support, which would be modest, to do what must be done to make sure that the work goes on when he can no longer continue. Fitzhugh’s work comes closer than any other that I know of in the United States to mirroring the values that infuse the work at Cambridge International Examinations. While Fitzhugh has had to give up the paper version of his extraordinary journal, the nation is spending enormous sums to build tests that are very good at measuring only a small part of the skills and knowledge on which our future depends. This does not auger well.
