Cross-posted from Education Week
Last week, I made the case for international benchmarking. Today, I will describe how both the goals and the methods of international benchmarking differ from those of conventional education research, while at the same time incorporating those methods wherever appropriate.
The aim of a lot of education research is to identify the causes of poor student performance and then to identify effective solutions to those problems, policies and practices that will, if properly implemented, improve education outcomes for children.
The dominant mental model of the second part of this process, in the mind of the researcher and practitioner, is the idea of replication. Researchers aim to show that a particular policy or practice is more effective than any other to address a particular problem. The obligation of the practitioner is to replicate it. The warranty of the researcher is no good if the original policy or practice is not faithfully replicated. To make that model work, the researcher has to not only show the policy or practice works, but must also describe it in detail and tell the consumer of the research what the context was in which the policy or practice was used, so, to the extent necessary, the practitioner can replicate that, too.
You are, of course, thoroughly irritated with me now for wasting your time by saying the obvious. But it isn’t obvious. Or, more exactly, it is not obvious that this is the most useful way to do education research.
American researchers have concentrated mostly on researching particular policies and practices—everything from time on task to charter schools to particular reading methods. Interestingly, the findings are almost always disappointing. That is, the effects are smaller than the designers had hoped for. The reason is that the effects of the intervention are always heavily mitigated by the myriad features of the larger environment in the school and beyond that, often work to defeat the purpose of the intervention. You can think of these as system effects.
I will make a case in a later post that the relatively poor performance of the United States in international comparisons of student performance is largely due to our lack of an effective system of education. It is how the parts and pieces fit together and reinforce one another—or don’t—that are the problem in the United States.
Now here is the thing about a state or national system of education: No one state or nation is interested in replicating any other state or nation’s education system. The values, history and politics of any state or nation will be different from any others and that is what will determine their education goals and strategies. Education research, in that case, should not be aiming to define a model to be faithfully replicated but rather to provide resources that decision-makers can use to create their own strategies.
This challenge the United States faces now in education is very similar to the problem faced by American manufacturing companies when the Japanese came, seemingly from nowhere, to challenge the Americans in the late 1970s with higher quality products sold and much lower prices than their American competitors. The Americans that survived put teams of engineers together to figure out how the Japanese did it. They visited many Japanese firms and found that different firms were better at different things. Some great firms did things no one wanted to copy because they were workarounds to solve problems unique to them. The American firms also thought that, good as the Japanese firms were, they could add their own, American, ideas that would be an improvement on anything the Japanese could think of.
What I have just described are the goals of benchmarking: Not to identify one superior company or intervention or system, to be faithfully copied, but to put together a composite picture based on research into many companies, interventions, even countries of the reasons for their superior performance. Then having done so, to select the principles and practices, which, when combined with your own ‘secret sauce’ will enable you to outperform your competitors. Benchmarkers are not interested in replication, because they don’t want to match their competitors’ performance; they want to surpass it.
When this kind of benchmarking is done well, it is not done with fly-by junkets for a few days to the country of interest. It involves intense, sustained research using a wide range of methodologies, some from the familiar canon of American researchers, and some that look a lot more like journalism and the work of historians and sociologists. It is no less demanding than conventional education research, and it often incorporates much conventional education research, but its purpose is different. It is not a search for the one best approach—say the way Finland does it or Singapore—that everyone is expected to emulate, but to seek to understand, at a deep level, the common principles that the most successful countries have embraced on the road to their success and to point to examples of policies and practices that are consistent with those principles that another state or country might use to match or even exceed their accomplishments.
