In my course on Comparative Politics, I like to use a 20/20 episode by John Stossel entitled Stupid in America. Stossel presents a superficially appealing argument (criticizing the "monopolistic" character of the US educational system) based, in part, on a number of different comparisons, both cross-national and domestic. The logic of his comparisons, however, is fundamentally flawed and highly biased. This is reflected in his selective use of statistics, especially international statistics.
In particular, Stossel draws from the PISA exam, which test students across a range of countries in math, science and reading. Of the many top-scoring countries Stossel could focus on, he chooses Belgium. Not surprisingly, Belgium has the type of educational system Stossel recommends--a voucher-based system that encourages competition among schools for students. He tells us that it is only through competition that an educational system can succeed.
I was reminded of this argument by a recent story in NPR, which discusses he results of the latest PISA exam (from 2009). Again, the US falls short. Stossel, I'm sure, feels the results further confirm his argument.
Interestingly, though, among the top ten performers, eight are in the Asia-Pacific region -- led by China, Singapore, South Korea and Japan. Now, if Stossel is correct, these Asian educational systems would all be voucher-based, but this is not the case. Most, if not all, are based on the same principle as exists in the US--that is, a system in which students are assigned to particular schools based on where they live. There is competition, it must be said, but it is a competition among students. Even more, the country that historically performs the best overall is Finland. Yet, Finland, too does not have a voucher-based system. The fact that Stossel failed to examine these top-performing countries fatally undermines his argument, comparatively speaking.
Methodologically, you cannot "pick-and-choose" your cases. A good comparativist will examine as many relevant cases as possible, regardless of whether they seem to support or disprove the argument. Indeed, a good comparativist will fully embrace those cases that, on the surface, seem contradictory. For, in showing that her theory can account for the "hard case," a good comparativist will have build an even stronger argument.
