The latest results from the triennial Programme for International Student Assessment (PISA) surveys are out. Four Chinese provinces topped the league tables again. And again, it is so obvious to the critics that China must be cheating that they argue the results should be dismissed out of hand. Beyond the results themselves, their scorn is being poured on the Organization for Economic Co-Operation and Development (OECD) education directorate for having let the Chinese get away with it.
It is not so obvious to me that the Chinese have cheated.
Years ago, when it was announced that the OECD had agreed to let the Chinese province of Shanghai participate, I asked key officials in Hong Kong and Singapore who were very familiar with the Shanghai system how they expected Shanghai to do in the PISA rankings. Everyone I spoke to said they would not be surprised if Shanghai placed among the top performers. It might, they said, even turn out to lead the top performers.
I was astounded. Mao Zedong, former chairman of the People’s Republic of China, had virtually destroyed the Chinese education system during the Cultural Revolution. It was a very poor country then, with a very high illiteracy rate and very little money. He had disbanded the schools and sent teachers into the countryside to be ‘reeducated.’ But the Hong Kong and Singapore officials were dead serious. And they turned out to be right on the money.
It did not take long for the American critics to dismiss Shanghai’s achievement. Two charges were made, the first being that Beijing had cherry-picked one province in order to make the whole country look good. The second was that Shanghai finessed the standard sampling procedure by sending underperforming students out of Shanghai prior to the administration of PISA so that these underperforming students would not be counted in the report of the results.
The first of these charges was ridiculous. Shanghai had participated in PISA not at the behest of the government in Beijing, but at the request of Minxuan Zhang, the deputy chairman of the Shanghai Municipal Education Commission. Zhang wanted Shanghai teachers to move away from a curriculum based largely on rote memorization of facts and procedures. He knew that they would not want Shanghai to be embarrassed by a poor showing on the PISA assessment and would do what they could to provide the kind of instruction that would enable them to do well.
Yes, in a sense, he wanted them to teach to the test, in this case a test that was based on thinking and real understanding of the subject being studied. He also wanted them to focus less on the curriculum and more on having students who had the skills and understanding needed to apply what they were learning to real world problems. Zhang knew that these are just the skills needed to do well on PISA. He was much less interested in having China look good on an exam than in using PISA as an incentive for Shanghai teachers to teach in a very different way than they were teaching and to embrace a different set of goals than the traditional goals of Chinese education. It worked.
Of course, Shanghai is not all of China. Neither China nor the OECD ever suggested that it was or that the rest of China would perform as well as Shanghai. In fact, within China, Shanghai had long been regarded as the nation’s best performer in education. Chinese educators were used to coming to Shanghai to learn from its system and the government in Beijing had looked to Shanghai to pioneer innovations in education that the central government would then try to export to the rest of the country. No one ever tried to hide these facts. Nor did the OECD try to hide the fact that the score was for Shanghai, not all of China.
But one can hardly say that, because it was only Shanghai, it should not have been reported. Shanghai is both a city and a province of China and has a population of about 24 million people. It is by itself bigger than most of the countries that are included in the PISA rankings, larger than all but two American states and equal in size to the total population of our 11 smallest states combined. From an economic standpoint, the United States is not competing with all of China, much of which is a rural hinterland. It is competing with China’s most advanced provinces, not least Shanghai. The fact that Shanghai is doing a much better job of educating its students than is the United States ought to be big news, because Chinese workers still make much less than their counterparts in the U.S. In a globalized world, that means that global firms can get much higher quality labor in Shanghai than in the United States for a much lower price. That is the important fact. The fact that Shanghai is not all of China is simply irrelevant.
The charge that Shanghai was cheating by excluding from the sample students who they knew would perform poorly needs some explanation. Under Chinese law, the social services, including education, to which a citizen is entitled can only be claimed in the province in which that citizen is registered, unless a province to which such a citizen moves volunteers to extend those services to the individual and his family. As modern China was being built, millions of Chinese moved from dirt poor farms to the big cities to make more money than they could possibly make on their tiny farms. They are referred to as migrants. The vast majority moved from their home province to a wealthier province, one in which their children did not have the right to go to the public schools.
Roughly 40 percent of the residents of Shanghai are migrants. Since they were not entitled to send their children to the public schools in Shanghai, they typically organized their own schools for their children. But they had very little money and these schools were, to put it mildly, substandard. Over the last 20 years or so, the Shanghai Municipal Education Commission began to take these migrant children into the public schools. They knew that, whatever the law was, these families were not going home. Their children would grow up to be a substantial portion of the future Shanghai workforce and they needed a good education if Shanghai was going to succeed.
But these officials could not bring all of these students into the system all at once, because the citizens of Shanghai did not want to share their wealth with these migrants. It was a case of Shanghai for the Shanghainese. In this sense, the political situation was very similar to the situation many undocumented immigrant students find themselves in in the United States. Or the situation of many ‘guest workers’ in Europe. At first, these migrant students were placed in segregated schools that were crowded and poorly staffed. Later, they were put in ever increasing numbers in schools with the children of families registered in Shanghai. When this began, the migrant children were often segregated within the school. Then that began to change. The education they were offered was getting better as the system evolved.
In 2009, when Shanghai students first took the PISA assessments, the integration of migrant children into the system had begun but was only partially complete. Most migrant children in what we would call elementary and middle schools had been admitted to the regular Shanghai system. In fact, many were in brand new schools that had been built where most of the migrant families lived. The children from migrant families could still not enroll in Shanghai academic high schools, but they could enroll in Shanghai vocational schools. PISA samples 15-year-old students. It turned out that some 15-year-old students were in middle school and some were in high school. Those in middle school were in the sample. But those who had been in middle school but had gone on to academic high schools were back in the province in which their parents were registered. It was true, then, that many 15-year-olds were not eligible for the PISA sample because they had been sent back to the province in which their parents were registered, but many were eligible, either because they were in vocational school or because they were in middle school.
So, was it true that the Chinese had deliberately excluded the children of migrants from the PISA sample because they wanted to look good to the world? If that was the case, they would have first excluded not the migrant children who went back to their home province, but the migrant students who were in Shanghai in vocational schools. The students who went back to their home province were going to the academic high schools. Their achievement on PISA would without a doubt have been higher than the students who were in the Shanghai vocational high schools, whose academic standards are lower than the standards in the academic high schools. If the Chinese had wanted to look good, they would have sent away the vocational high school students and kept the academic high school students.
It is, I think, more reasonable to conclude that the migrant high school students in the academic high schools went back to their home province because the domestic politics of China made it impossible for the Shanghai school reformers to admit the migrant high school kids in Shanghai into the public academic high schools. It had nothing to do with PISA.
The question then, is what difference it made to exclude from the PISA sample the migrant students who went back to their home provinces.
In the first sample of Shanghai, the coverage rate of 15-year-olds was 60 percent. Forty percent—those left out—were a big proportion. But the coverage rate for the four Chinese provinces in the most recent sample was 81.2 percent. The exclusion rate (enrolled students not tested) was 3.8 percent in the U.S. and 3.2 percent in the four Chinese provinces for which results were released. There are two points to be made here. First, obviously, the exclusion rates for the U.S. and the Chinese provinces are now comparable, so, whatever exclusion problem there is in those provinces is more than matched by exclusion problems in the U.S. Second, there is a big difference between 60 percent and 81.2 percent. The Chinese have clearly made a lot of progress since the first sample was taken. That should not surprise us. The central government in Beijing made a big point when it issued its last big forward plan of the need to address the problems caused by its provincial registration system. They are a long way from fixing the problem, but they have made a lot of progress. Would that we here in the United States had made as much progress on school segregation by race over the last few decades as the Chinese have made on the segregation of their migrants in their big cities.
Why have I bothered you with this long, detailed story about China’s PISA scores? Does it really matter that much?
Yes, I’m afraid it does. The most visible attack on the latest Chinese PISA scores comes from Mark Schneider in an article in The 74 Million titled, ”The Strange Case of ‘China’ and its Top PISA Rankings—How Cherry-Picking Regions to Take Part Skews its High Scores.” In the article, as you can tell from the title, Schneider criticizes the OECD for representing the four Chinese provinces now assessed as representing all of China, asserts that the Chinese government originally picked Shanghai to represent the country in the PISA sweepstakes, decries the exclusion of migrant children from the survey, and contends that we have only to look no further than Massachusetts for world-class performance that can be validated with reliable data.
Schneider is no bit player. He is the director of the Institute of Education Sciences at the U.S. Department of Education and a former vice-president and Institute Fellow at the American Institutes of Research.
Much of what Schneider says in his article is just a rehash of the what critics said when Shanghai’s scores were first released. I’ve already made it clear that neither the Chinese nor the OECD ever suggested that Shanghai’s performance was representative of China’s performance as a whole; that Beijing never picked Shanghai to be the first province to appear in the PISA rankings and that Shanghai’s superior performance was not the result of excluding the migrant students.
So what about the other charge—that we need look no further than Massachusetts for inspiration and can forget about the Chinese? Andreas Schleicher, director of education and skills at the OECD and leader of the PISA team responded to Schneider in a recent OECD publication. In the article, Schleicher pointed out that Massachusetts, the U.S. state with the highest scores on NAEP, in the PISA 2012 assessment of mathematics scored 100 points below Shanghai. The gap, he said, is “as wide as that between Massachusetts and Mexico.”
Schneider’s tone in his article is dismissive. He describes U.S. researchers who try to uncover the factors that account for the success of PISA top performers as edutourists. If Americans worked half as hard at trying to understand how the countries at the top of the PISA league tables get there and stay there as they do at dismissing Chinese achievements, we might stand a chance of catching up to them. The effort we put into dismissing those achievements is counterproductive and disgraceful.
The American taxpayer has put billions of dollars into American education research funded by the National Institute of Education since the early 1970s. But the nation’s high school NAEP scores have hardly changed in half a century. There is no evidence that American education research has made any difference at all in the achievement of our high school graduates in all that time. Maybe a little humility is in order. Maybe it is time to admit that we have something to learn from other countries.
