For most high school students who want to attend an elite college, the SAT is more than a test. It is one of life’s landmarks. Waiting for the scores—one for verbal, one for math, and now one for writing, with a possible 800 on each—is painfully suspenseful. The exact scores are commonly remembered forever after.
So it has been for half a century. But events of recent years have challenged the SAT’s position. In 2001, Richard Atkinson, president of the University of California, proposed dropping the SAT as a requirement for admission. More and more prestigious small colleges, such as Middlebury and Bennington, are making the SAT optional. The charge that the SAT is slanted in favor of privileged children—“a wealth test,” as Harvard law professor Lani Guinier calls it—has been ubiquitous. I have watched the attacks on the SAT with dismay. Back in 1961, the test helped get me into Harvard from a small Iowa town by giving me a way to show that I could compete with applicants from Exeter and Andover. Ever since, I have seen the SAT as the friend of the little guy, just as James Bryant Conant, president of Harvard, said it would be when he urged the SAT upon the nation in the 1940s.
I considered the SAT to be the friend of the little guy, just as James Bryant Conant, president of Harvard, said it would when he urged the SAT upon the nation in the 1940s.
Conant’s cause was as unambiguously liberal in the 1940s as income redistribution is today. Then, America’s elite colleges drew most of their students from a small set of elite secondary schools, concentrated in the northeastern United States, to which America’s wealthy sent their children. The mission of the SAT was to identify intellectual talent regardless of race, color, creed, money, or geography, and give that talent a chance to blossom. Students from small towns and from poor neighborhoods in big cities were supposed to benefit—as I thought I did, and as many readers of the american think they did.
But data trump gratitude. The evidence has become overwhelming that the SAT no longer serves a democratizing purpose. Worse, events have conspired to make the SAT a negative force in American life. And so I find myself arguing that the SAT should be ended. Not just deemphasized, but no longer administered. Nothing important would be lost by so doing. Much would be gained.
To clarify my terms: Here, “SAT” will always refer to the verbal and mathematics tests that you have in mind when you recall your own SAT scores. They, along with the writing test added in 2005, are now officially known as “reasoning tests” or SAT I (labels I will ignore). The College Board also administers one-hour achievement tests in English literature, United States history, world history, biology, chemistry, physics, two levels of math, Chinese, French, German, Hebrew, Italian, Japanese, Korean, Latin, and Spanish. These are now called “subject tests” or SAT II (more labels I will ignore).
I do not discuss the College Board’s advanced placement (AP) tests that can enable students to get college credit, because they cannot serve as a substitute for either the SAT or the achievement tests. Not all schools offer AP courses, and the AP’s five-point scoring system conveys limited information.
Start with the proposition that nothing important would be lost by dropping the SAT. The surprising empirical reality is that the SAT is redundant if students are required to take achievement tests.
In theory, the SAT and the achievement tests measure different things. In the College Board’s own words from its website, “The SAT measures students’ verbal reasoning, critical reading, and skills,” while the achievement tests “show colleges their mastery of specific subjects.” In practice, SAT and achievement test scores are so highly correlated that SAT scores tell the admissions office little that it does not learn from the achievement test scores alone.
The coaching industry touts the clever test-taking strategies it teaches, but the bulk of the contribution comes from garden-variety preparation that is easily open to any student at no cost.
The pivotal analysis was published in 2001 by the University of California (UC), which requires all applicants to take both the SAT and achievement tests (three of them at the time the data were gathered: reading, mathematics, and a third of the student’s choosing). Using a database of 77,893 students who applied to UC from 1996 to 1999, Saul Geiser and Roger Studley analyzed the relationship among high school grades, SAT scores, achievement test scores, and freshman grades in college. Here is what they found:
Achievement tests did slightly better than the SAT in predicting freshman grades. High school grade point average, SAT scores, and achievement test scores were entered into a statistical equation to predict the grade point that applicants achieved during their freshman year in college. The researchers found that achievement tests and high school grade point each had about the same independent role—that is, each factor was, by itself, an equally accurate predictor of how a student will do as a college freshman.
But the SAT’s independent role in predicting freshman grade point turned out to be so small that knowing the SAT score added next to nothing to an admissions officer’s ability to forecast how an applicant will do in college—the reason to give the test in the first place. In technical terms, adding the SAT to the other two elements added just one-tenth of a percentage point to the percentage of variance in freshman grades explained by high school grade point and the achievement tests.
But what about the students we’re most concerned about—those with high ability who have attended poor schools? The California Department of Education rates the state’s high schools based on the results from its standardized testing program for grades K–12. For schools in the bottom quintile of the ratings—hard as I found it to believe—the achievement tests did slightly better than the SAT in predicting how the test-takers would perform as college freshmen.
What about students from families with low incomes? Children of parents with poor education? Here’s another stunner: after controlling for parental income and education, the independent role of the SAT in predicting freshman grade point disappeared altogether. The effectiveness of high school grade point and of achievement tests to predict freshman grade point was undiminished.
Combine these edges, the critics say, and it comes down to this: if you're rich, you can buy your kids a high SAT score.
All freshman grades are not created equal, so the UC study took the obvious differences into account. It broke down its results by college campus (an A at Berkeley might not mean the same thing as an A at Santa Cruz) and by freshman major (an A in a humanities course might not mean the same thing as an A in a physical science course). The results were unaffected. Again, the SAT was unnecessary; it added nothing to the forecasts provided by high school grades and achievement tests.
Thorough as the Geiser and Studley presentation was, almost any social science conclusion can be challenged through different data or a different set of analyses. The College Board, which makes many millions of dollars every year from the SAT, had every incentive and ample resources to refute the UC results. But it could not.
In 2002, the College Board published its analysis, “The Utility of the SAT I and SAT II for Admissions Decisions in California and the Nation.” The College Board’s study disentangled some statistical issues that the UC study had not and used a different metric to express predictive validity, but its bottom line was effectively identical. Once high school grade point and achievement test scores are known, the incremental value of knowing the SAT score is trivially small.
Still reluctant to give up on the SAT, I wondered whether the College Board had been unwilling to make the best defense. Perhaps the SAT had made an important independent contribution to predicting college performance in earlier years, but by the time research was conducted in the last half of the 1990s, the test had already been ruined by political correctness. To see where this hypothesis comes from, a little history is required.
Originally, the point of the SAT—whose initials, after all, stood for Scholastic Aptitude Test—was to measure aptitude, defined by the dictionary as “inherent ability,” rather than to measure academic achievement. But in the aftermath of the 1960s, the concept of aptitude became troublesome. The temper of the times meant that long-observed ethnic and class differences in mental test scores had to be interpreted as the fault of the tests that produced them. Like all other mental tests, the SAT persistently showed such differences; therefore, the SAT had to be a bad test, culturally biased in favor of upper-middle-class white kids.
Did the pre-1994 SAT measure something importantly different from what the post-1994 SAT had measured? Don’t bother asking the College Board. The data for answering that question would require the College Board to reveal just how well the original and revised SATs measure the general mental factor g, the stuff of intelligence/aptitude, and the College Board does not want to acknowledge that the SAT measures g at all or, for that matter, that g even exists.
Seen from an outsider’s perspective, the changes in 1993–1994 do not look particularly important. Twenty-five antonym items in the SAT Verbal were replaced with reading-comprehension items, on grounds that the antonym items could be compromised by students who memorized vocabulary lists. The math test saw some changes in the answer format. But samples of the new items appear to be plausible measures of g and not obviously inferior to the items they replaced.
Despite the College Board’s rhetoric about revamping the SAT to reflect curriculum, the changes in the test in 1993–1994 probably did not have much effect on the SAT’s power to measure g—in the jargon, its g-loading. (I would not make the same statement about today’s SAT, which has eliminated the highly g-loaded analogy items and added a writing component that carries with it a multitude of scoring problems.)
The College Board was run by people who were eager to demonstrate their own progressive credentials. They ran from the concept of aptitude as the Florentines fled the plague.
If I am wrong, and the pre-1994 SAT measured g much better than the SAT used for the UC study, then I hope some disaffected College Board psychometrician leaks that news immediately. I will thereupon join a crusade to restore the old SAT. But given the available information, I think it is probable that even analyses conducted prior to the revisions in the test would not have shown a major independent role for the SAT after taking high school transcript and achievement test scores into account. To put it another way, those of us who thought that the SAT was our salvation were probably wrong. Even coming from mediocre high schools, our scores on achievement tests would have conveyed about the same picture to college admissions committees as our scores on the SAT conveyed.
I know how counterintuitive this sounds (I am presenting a conclusion I resisted as long as I could). But the truth about any achievement test, from an AP exam down to a weekly pop quiz, is that the smartest kids tend to get the highest scores. All mental tests are g-loaded to some degree. What was not realized until the UC study was just how high that correlation was for the SAT and the achievement tests.
Before, studies of the relationship had been based on self-selected samples of students who chose to take achievement tests along with the SAT, and there was good reason to think those students were unrepresentative. But by requiring all applicants to take both the SAT and achievement tests, the University of California got rid of this problem—and the correlations were still very high.
After the College Board did all of its statistical corrections in its 2002 study and applied them to test-takers from California, it found, for example, that the correlation between the SAT Verbal and the Literature Achievement test was a very high 0.83 (a correlation of 1.0 represents a perfect direct relationship). The correlation between the SAT Math and the Math IC achievement test was 0.86. So I conclude that bright students who do not go to first-rate high schools will do fine without the SAT. Consider these scenarios:
Start with motivated, high-ability students who go to truly bad schools, meaning the worst schools in the inner cities. The bright students’ achievement test scores are likely to be depressed by the schools’ dreadfulness, but even scores that are just fair will get the attention of an admissions office if the transcript shows As and the recommendations are enthusiastic. The nation’s top colleges desperately want to increase their enrollment of inner-city blacks and Hispanics, and are willing to make large allowances for bad schooling to do so.
Next, turn to the much larger number of high-ability students who are in schools that are not awful, but mediocre—the typical urban or small-town public school. The curriculum includes all the standard college-prep courses with standard textbooks. A few of the teachers are terrific, but most are no more than ordinary.
The SAT test isn't the problem. The children of the well educated and affluent get most of the top scores because they constitute most of the smartest kids. They are smart not because their parents are well educated, but because their parents are smart.
The high-ability students in such schools who are playing the game, studying hard, have no problem at all if the SAT is eliminated. They have nearly straight As on their transcripts, which most college admissions offices treat as the most important single source of information. Their letters of recommendation are afire with zeal on their behalf. These students also do well on the achievement tests. A hard-working, high-ability physics student is likely to absorb enough physics from the textbook to do well on the physics achievement test despite a so-so teacher. In addition, high-ability kids who play the game have usually been reading voraciously—and in the process picked up a great deal of knowledge about history, literature, and culture on their own. This information has been gathered inefficiently, but high-ability students absorb knowledge like a sponge, no matter what schools they attend.
Now consider high-ability students in mediocre schools who do not play the classroom game. They are bored with their classes and sometimes get Bs and the occasional C, but they have active minds and are looking for ways to occupy themselves. They spend all their time on the debate team or writing for the high school newspaper, or in the drama department. By the end of high school, they have a long list of accomplishments studding their applications. One way or the other, by the end of high school, students in this category are very likely to have done things that will catch the attention of an admissions officer. And again, their achievement test scores are high. These students are at least as intellectually curious as those who play the game. Their Bs do not mean they didn’t absorb the substance of the coursework, and they too have typically encountered and retained large amounts of information outside school.
That leaves the worst case: high-ability students who are alienated by school and perhaps by life. They don’t study, don’t go out for the debate team, don’t read on their own, don’t even watch the Discovery Channel. It is possible for them nonetheless to achieve a high score on an individually administered IQ test, despite being hostile and uninterested. Arthur Jensen relates the time he was testing a sullen subject in a juvenile detention facility and came to the vocabulary item “apocryphal.” The boy answered, “How the hell should I know? I think the whole Bible is [bunk].” In an individually administered IQ test, the examiner could score his answer as correct, but that same alienated boy is unlikely to get a high score on the SAT because no one, no matter how smart, gets a high score on the SAT without concentrating and trying hard over the course of three stressful hours. So keeping the SAT will not help most students in this category. They won’t try hard, and their SAT scores will be mediocre despite their ability.
That leaves an extremely odd set of high-ability students who will be harmed by dropping the SAT—so alienated that they do nothing to express their ability in school, so completely walled off from independent learning that they do poorly on the achievement tests, and yet able to buckle down on the SAT and get a good score. I am not sure that getting a good score under such circumstances is even possible on the SAT Math—too many of the questions presuppose hard work in algebra class—but perhaps it could be done on the SAT Verbal.
In any case, we are now talking about a very few students, and even for them it is not clear whether dropping the SAT introduces an injustice. Should such a student be given a slot that could have been filled by a less-talented student who is eager to give a competitive college his best effort? Being forced to go to an unselective college instead could well be the better outcome for all concerned.
There is good reason to think that a world in which achievement tests have replaced the SAT is not going to be a world in which motivated high-ability students from bad or mediocre schools have less opportunity to get into the college where they belong. It may be a marginally worse world for a small number of unmotivated high-ability students who want to attend selective colleges, but that outcome is not necessarily undesirable.
But why get rid of the SAT? If it works just about as well as the achievement tests in predicting college success, what’s the harm in keeping it?
The short answer is that the image of the SAT has done a 180-degree turn. No longer seen as a compensating resource for the unprivileged, it has become a corrosive symbol of privilege. “Back when kids just got a good night’s sleep and took the SAT, it was a leveler that helped you find the diamond in the rough,” Lawrence University’s dean of admissions told The New York Times recently. “Now that most of the great scores are affluent kids with lots of preparation, it just increases the gap between the haves and the have-nots.”
As those who get the high SAT scores are increasingly from socially and economically privileged families, a sense of entitlement among the privileged is becoming unmistakable. It would be better is no one had those numbers in their head.
If you’re rich, the critics say, you can raise your children in an environment where they will naturally acquire the information the SAT tests. If you’re rich, you can enroll your children in Kaplan, or Princeton Review, or even get private tutors to coach your kids in the tricks of test-taking, and thereby increase their SAT scores by a couple of hundred points. If you’re rich, you can shop around for a diagnostician who will classify your child as learning-disabled and therefore eligible to take the SAT without time limits. Combine these edges, and it comes down to this: if you’re rich, you can buy your kids a high SAT score.
Almost every parent with whom I discuss the SAT believes these charges. In fact, the claims range from simply false, in the case of cultural bias, to not-nearly-as-true-as-you-think, in the case of the others. Take coaching as an example, since it seems to be so universally accepted by parents and has been studied so extensively.
From 1981 to 1990, three separate analyses of all the prior studies were published in peer-reviewed journals. They found a coaching effect of 9 to 25 points on the SAT Verbal and of 15 to 25 points on the SAT Math. In 2004, Derek Briggs, using the National Education Longitudinal Study of 1988, found effects of 3 to 20 points for the SAT Verbal and 10 to 28 points for the SAT Math. Donald Powers and Donald Rock, using a nationally representative sample of students who took the SAT after its revisions in the mid-1990s, found an average coaching effect of 6 to 12 points on the SAT Verbal and 13 to 18 points on the SAT Math. Many studies tell nearly identical stories. On average, coaching raises scores by no more than a few dozen points, enough to sway college admissions in exceedingly few cases.
But the coaching business is booming, with affluent parents being the best customers. If the payoff is really so small, why has the market judged coaching to be so successful?
Most obviously, parents who pay for expensive coaching courses ignore the role of self-selection: the students who seem to profit from a coaching course tend to be those who, if the course had not been available, would have worked hard on their own to prepare for the test.
Then parents confuse the effects of coaching with the effect of the basic preparation that students can do on their own. No student should walk into the SAT cold. It makes sense for students to practice some sample items, easily available from school guidance offices and online, and to review their algebra textbook if it has been a few years since they have taken algebra. But once a few hours have been spent on these routine steps, most of the juice has been squeezed out of preparation for the SAT. Combine self-selection artifacts with the role of basic preparation, and you have the reason that independent studies using control groups show such small average gains from formal coaching.
A hard-working, high-ability physics student is likely to absorb enough physics from the textbook to do well on the physics achievement test--despite the shortcomings of the teacher.
It makes no difference, however, that the charges about coaching are wrong, just as it makes no difference that the whole idea that rich parents can buy their children high SAT scores is wrong. One part of the indictment is true, and that one part overrides everything else: the children of the affluent and well educated really do get most of the top scores. For example, who gets the coveted scores of 700 and higher, putting them in the top half-dozen percentiles of SAT test-takers? Extrapolating from the 2006 data on means and standard deviations reported by the College Board, about half of the 700+ scores went to students from families making more than $100,000 per year. But the truly consequential statistics are these: Approximately 90 percent of the students with 700+ scores had at least one parent with a college degree. Over half had a parent with a graduate degree.
In that glaring relationship of high test scores to advanced parental education, which in turn means high parental IQ, lies the reason that the College Board, politically correct even unto self-destruction, cannot bring itself to declare the truth: the test isn’t the problem. The children of the well educated and affluent get most of the top scores because they constitute most of the smartest kids. They are smart because their parents are smart. The parents have passed their smartness along through parenting practices that are largely independent of education and affluence, and through genes that are completely independent of them.
The cognitive stratification of American society—for that’s what we’re talking about—was not a problem 100 years ago. Many affluent people were smart in 1907, but there were not enough jobs in which high intellectual ability brought high incomes or status to affect more than a fraction of really smart people, and most of the really smart people were prevented from getting those jobs anyway by economic and social circumstances (consider that in 1907 roughly half the adults with high intelligence were housewives).
From 1907 to 2007, the correlation between intellectual ability and socioeconomic status (SES) increased dramatically. The socioeconomic elite and the cognitive elite are increasingly one. If you want the details about how this process worked and how it is transforming America’s class structure, I refer you to The Bell Curve (1994), the book I wrote with the late Richard Herrnstein. For now, here’s the point: Imagine that, miraculously, every child in the country were to receive education of equal quality. Imagine that a completely fair and accurate measure of intellectual ability were to be developed. In that utopia, a fair admissions process based on intellectual ability would fill the incoming classes of the elite colleges predominantly with children of upper-middle-class parents.
A world in which achievement tests have replaced the SAT is not going to be a world in which motivated high-ability students from bad or mediocre schools have less opportunity to get into the college where they belong.
In other words, such a perfect system would produce an outcome very much like the one we see now. Harvard offers an easy way to summarize the revolution that accelerated after World War II. As late as 1952, the mean SAT Verbal score of the incoming freshman class was just 583. By 1960, the mean had jumped to 678. In eight years, Harvard transformed itself from a college with a moderately talented student body to a place where the average freshman was intellectually in the top fraction of 1 percent of the national population. But this change did not mean that Harvard became more socioeconomically diverse. On the contrary, it became more homogeneous. In the old days, Harvard had admitted a substantial number of Boston students from modest backgrounds who commuted to classes, and also a substantial number of rich students with average intelligence. In the new era, when Harvard’s students were much more rigorously screened for intellectual ability, the numbers of students from the very top and bottom of the socioeconomic ladder were reduced, and the proportion coming from upper-middle-class backgrounds increased.
The other high-ranking schools have similar stories to tell. In a sample of 11 of the most prestigious colleges studied by William Bowen and his colleagues between the mid-1970s and the mid-1990s, the proportion of students in the top SES quartile rose from about a third to a half of all students, while the share in the bottom quartile remained constant at one-tenth. And these were schools such as Princeton and Yale that get first chance to admit the scarce and sought-after candidates of high ability from poor backgrounds.
When, in 2003, Anthony Carnevale and Stephen Rose expanded the definition of top-tier colleges to include 146 schools, fully 74 percent of the students came from families in the top SES quartile, while only 3 percent came from the bottom quartile. Ethnic diversity has increased during the last half century, but not socioeconomic diversity.
Because upper-middle-class families produce most of the smartest kids, there is no way to reform the system (short of disregarding intellectual ability altogether) to prevent their children from coming out on top. We can only make sure that high-ability students from disadvantaged backgrounds realize that the nation’s best colleges yearn for their applications and that their chance of breaking out of their disadvantaged situations has never been better—in short, that the system is not rigged. Now, the widespread belief is that the system is rigged, and the SAT is a major reason for that belief. The most immediate effect of getting rid of the SAT is to remove an extremely large and bright red herring. But there are more good effects.
Getting rid of the SAT will destroy the coaching industry as we know it. Coaching for the SAT is seen as the teaching of tricks and strategies—a species of cheating—not as supplementary education. The retooled coaching industry will focus on the achievement tests, but insofar as the offerings consist of cram courses for tests in topics such as U.S. history or chemistry, its taint will be reduced.
A low-income student shut out of opportunity for an SAT coaching school has the sense of being shut out of mysteries. Being shut out of a cram course is less daunting. Students know that they can study for a history or chemistry exam on their own. A coaching industry that teaches content along with test-taking techniques will have the additional advantage of being much better pedagogically—at least the students who take the coaching courses will be spending some of their time learning history or chemistry.
The substitution of achievement tests for the SAT will put a spotlight on the quality of the local high school’s curriculum. If achievement test scores are getting all of the parents’ attention in the college admissions process, the courses that prepare for those achievement tests will get more of their attention as well, and the pressure for those courses to improve will increase.
The final benefit of getting rid of the SAT is the hardest to describe but is probably the most important. By getting rid of the SAT, we would be getting rid of a totem for members of the cognitive elite.
People forget achievement test scores. They do not forget cognitive test scores. The only cognitive test score that millions of people know about themselves is the SAT score. If the score is high, it is seen as proof that one is smart. If the score is not high, it is evidence of intellectual mediocrity or worse. Furthermore, it is evidence that cannot be explained away as a bad grade can be explained away. All who enter an SAT testing hall feel judged by their scores.
A few high-profile colleges could have a domino effect. Suppose, for example, that this fall Harvard and Stanford were jointly to announce that SAT scores will no longer be accepted.
Worse yet, there are few other kinds of scores to counterbalance the SAT. Of the many talents and virtues that people possess, we have good measures for quantifying few besides athletic and intellectual ability. Falling short in athletic ability can be painful, especially for boys, but the domain of sports is confined. Intellectual ability has no such limits, and the implications of the SAT score spill far too widely. The 17-year-old who is at the 40th percentile on the SAT has no other score that lets him say to himself, “Yes, but I’m at the 99th percentile in working with my hands,” or “Yes, but I’m at the 99th percentile for courage in the face of adversity.
Conversely, it seems to make no difference that high intellectual ability is a gift for which its recipients should be humbly grateful. Far too many students see a high score on the SAT as an expression of their own merit, not an achievement underwritten by the dumb luck of birth.
Hence the final reason for getting rid of the SAT: knowing those scores is too dispiriting for those who do poorly and too inspiriting for those who do well. In an age when intellectual talent is increasingly concentrated among young people who are also privileged economically and socially, the last thing we need are numbers that give these very, very lucky kids a sense of entitlement.
How are we to get rid of the SAT when it is such an established American institution and will be ferociously defended by the College Board and a large test-preparation industry?
Actually, it could happen quite easily. Admissions officers at elite schools are already familiar with the statistical story I have presented. They know that dropping the SAT would not hinder their selection decisions. Many of them continue to accept the SAT out of inertia—as long as the student has taken the test anyway, it costs nothing to add the scores to the student’s folder.
It could happen, and it should happen. There is poignance in calling for an end to a test conceived for such a noble purpose. But the SAT score, intended as a signal flare for those on the bottom, has become a badge flaunted by those on top. We pay a steep educational and cultural price for a test that no one really needs.
Charles Murray is the W. H. Brady Scholar at the American Enterprise Institute.
The SAT’s bias toward the privileged was first alleged in the 1960s and 1970s on grounds that SAT questions used vocabulary and situations that a poor black student from the inner city would never encounter. The critics asserted, and much of the public still believes, that the SAT is mainly a test of upper-middle-class socialization.
The purpose of the SAT is to predict college performance. If the SAT is biased against members of a group, then applicants from that group will do better than their scores predict if they are given the opportunity to show their real ability in a college classroom. The test underpredicts their college performance.
No comments:
Post a Comment