Is John Stossel Stupid? How a Lack of Good Comparison Makes for Shaky Analysis

May 5, 2006 • In his ABC report, Stupid in America: How a Lack of Choice Cheats Our Kids Out of a Good Education, John Stossel makes a compelling and purposely provocative case. His argument is encapsulated in the subtitle of his report: "How a lack of choice cheat our kids out of a good education." According to Stossel, the basic problem facing American schools is that they are insulated from competition. As he puts it, "American schools don't teach as well as schools in other countries because they are government monopolies, and monopolies don't have much incentive to compete." His reference to "other countries" is an important part of his argument. Indeed, much of his argument hinges on a type of comparative analysis, in which he ostensibly reveals the fundamental flaws of the American educational system through comparison with other cases. For example, to "prove" his basic claim, he focuses on Belgium, where the government forces schools to compete for students by funding students as opposed to schools. In other words, education money "follows" children to whatever school they decide to attend. It is, as Stossel is careful to point out, a kind of voucher system. Stossel peppers his report with a number of other international comparisons as well. He says, for instance, "The longer kids stay in American schools, the worse they do in international competition. They do worse than kids from poorer countries that spend much less money on education, ranking behind not only Belgium but also Poland, the Czech Republic and South Korea." Significantly, he implies, but does not explicitly state, that Poland, the Czech Republic and South Korea must also have schools systems premised on the same competitive principles that exist in Belgium. Indeed, based on Stossel's basic comparative argument, we should expect every country in the world that scores better than the U.S. to have a "competition-based" school system. We don't know, however, because Stossel doesn't tell us. His use of "facts" (and comparative cases) in other words, is extremely selective. Is he hiding something? Or is he simply too stupid to know that that sort of selective analysis is inherently flawed?

What happens when we examine some of these other countries or "cases" in a bit more depth? One country to consider is Finland, which, according to a 2004 study done by PISA (Programme for International Student Assessment), has the "best" school system in the world. So, does Finland have a "competition-based" system? Well, at least not based on Stossel's criteria. In fact, Finland has a "unified" school system, which sees children staying at the same school between the ages of seven and 16, rather than having primary and secondary schools. In other words, not only do schools not compete for students, but students are "stuck" in the same school 10 consecutive years. Interestingly, Finland also "spends more per elementary, middle- and high- school student than any other nation on Earth, and comes in second on spending for higher education. School lunches, health care, most class materials and university tuition are all free." Voilà! With just one additional case we have fatally undermined Stossel's contention that by their very nature any monopolistic, government-run school system is bad. We have also, with this single case, raised questions about the importance of money in education, which is another of Stossel's peeves.

Indeed, according to Stossel's report, money is largely irrelevant in determining educational quality. Not only does he belittle those who think money is important, but he suggests that there is plenty of emprical evidence and careful analysis to show this. As he explains it, "while many people say, 'We need to spend more money on our schools,' there actually isn't a link between spending and student achievement." Here, he's making a strong claim, asserting that there is no (causal) relationship between "spending" and "student achievement." His wording suggests that there are plenty of studies to back this up, but, not suprisingly, he doesn't cite any. However, he approvingly quotes Jay Greene, author of Education Myths, who asserts, "If money were the solution, the problem would already be solved....We've doubled per pupil spending, adjusting for inflation, over the last 30 years, and yet schools aren't better." True, but in keeping with Stossel's generally superficial (stupid?) analysis, there is no further discussion of why this is the case. Perhaps the problem is not money per se, but how the money is spent. Indeed, early on in his report, Stossel notes how one very successful charter school in Oakland, California uses its money to pay its teachers "more than what public school teachers earn." Hmm. Maybe if increased student spending went entirely to teachers, the problems of low student achievement would be solved in the United States? At least that is the implication of this "fact."

To be clear, however, I am not suggesting that higher salaries are a magic bullet; this almost certainly isn't the case. (How do I know? Because I checked teacher salaries, both nationally and internationally: Germany has the highest paid teachers in the world, but does not have the highest "student achievement.") Still, it is disingenuous, at best, to assert that just because state funding has increased that money is irrelevant to a quality education. A better, less "stupid" analysis would have examined in detail and depth the manner in which money is being spent by school districts today. Moreover, a comparative analysis of school spending in the US and other countries would also likely have shed light on the issue. By the way, teacher pay in Finland is relatively low, but this may ultimately hurt the education system since so many young teachers are leaving after only a few years.

It is important to emphasize that I am not making the case that "competition-based" schools systems are bad. That would be stupid. Rather, I am saying that the issue is far more complex than Stossel is willing to admit. Certainly, the simplest comparative analysis will tell us that ostensibly monopolized school systems are not destined to "cheat" children of a high quality education. Finland, in fact, is far from alone. So, what is the answer? I do not purport to have one, if only because I know I haven't done the research necessary to develop an adequate understanding of the issue. What I do know, however, is that anyone serious about the issue must avoid polemics and instead engage in serious analysis. And, one way to do "serious analysis" is to be much more careful about doing comparisons.

On this point, consider what Stossel could have done if he had looked at Belgium in more depth. In so doing, we will easily discover that there are a lot of differences between Belgium and the United States. One obvious difference is simply that Belgium has a population of only 10.3 million people, compared to almost 300 million in the United States. It is also an overwhelmingly Catholic country with two basic ethnic groups, the Flemish (58 percent) and the French-speaking Walloon (31 percent). Further analysis would show, moreover, that the educational performance of the Flemish and Walloon communities in Belgium are significantly different: while the mean scores on the math scale for the Flemish community were higher than those in the best-performing OECD countries, Finland and South Korea, the means scores for the French-speaking community were only "average" (relative to other OECD countries). It is also worth noting that, in Belgium, children can start mainstream nursery education at the age of 30-months. Very few countries in the world provide publicly financed education at such an early age (almost all children in the Flemish parts of Belgium receive nursery education). Perhaps early schooling is the key difference? In fact, identifying differences such as these are crucial.The reason is clear: each is a potential explanatory variable. That is, each could possibly explain why the school system in Belgium produces better-educated, "smarter" students. We cannot know for sure because a comparison of only two cases (i.e., the US and Belgium) suffers from the small-N problem ("smart" people know what this is). To determine which factors--or combination of factors--are important or essential to an explanation requires more cases and more systematic comparative analysis. This is hard, which might explain why Stossel didn't do this. Maybe he's not just stupid, but lazy.

Of course, I'm being facetious. Stossel isn't stupid and he isn't lazy (okay, maybe he's intellectually lazy). Clearly, though, he has an ax to grind. Unfortunately, as with many TV personalities (it's hard to call him a reporter), he's willing to make his case even if it means ignoring or, worse still, hiding important facts. Perhaps this is why he talked about Belgium rather than Finland, for the Finnish case didn't fit his view of the world. Apparently, he sees only the world he wants to see, and will use his considerable resources to ensure that the rest of the world sees the same thing. Fortunately, as a political scientist, I can easily identify poorly constructed, highly biased "analysis." Indeed, it is quite ironic that, while bemoaning the poor educational system in the United States, Stossel intentionally propagates ignorance through dumbed-down reporting. He doesn't really want a well-educated, critical and analytically adept American public; instead, he wants a public that is easily manipulated and conned by facile and glitzy analysis with a clumsily concealed political agenda. That's too bad.

Post Script. I think it is worth noting that I spent about 2~3 hours on this piece, including research. This means that all the information I cite is easily available to anyone with a computer. We have to assume that Stossel has a team of motivated researchers to help him on his stories (we should also assume that Stossel himself knows how to do basic research), which raises the question: why didn't Stossel find the same information? It's not hard to imagine why.