In a editorial in the New York (NY, NY, US) Times questioning the design and use of the Scholastic Aptitude Test (SAT), Mark Franek challenges the granting of extra time on the test for students with Learning Disabilities. Although Mr. Franek, who is dean of students for a prestigious private school, also addresses other matters (e.g., use of multiple-guess format), it’s the accommodation that caught my attention. Mr. Franek professes an agreement with using accommodations, but his argument is apparent.
As high school juniors file into classrooms for their SAT’s on Saturday, there will probably be some chatter about how more than 4,000 of last fall’s tests were scored too low. What they probably won’t be aware of is how many of their fellow students may end up with higher scores because they are allowed more time to take the test. Last year, more than 40,000 of the two million SAT takers were granted special accommodations, mainly because of learning disabilities. This represents a doubling in the past decade and a half.
In a perfect world, accommodations on the SAT would level the playing field for all test-takers with learning disabilities. Is that the case? The College Board, the overseer of the SAT, declines to give figures on the family incomes of students who get extra time.
I have two major concerns about this argument. First, how many of those students who receive accommodations because of Learning Disabilities actually were found eligible simply because they or their parents were seeking an edge on the SATs? Later in the editorial, Mr. Franek paints an image of wealthy families using private resources to secure eligibility and then completing high school courses using accommodation. The large and often-lamented growth in the number of students identified as having Learning Disabilities since the 1970s means that many of the students who qualify for accommodations on the SAT this year were probably part of that growth during the 1990s. At least part of the doubling to which Mr. Franek refers must be students who were identified as having Learning Disabilities during their elementary years. We need data to determine how many of the students were determined eligible for special education services during their high school years.
Second, Mr. Franek’s argument overlooks the evidence about the effects of accommodations on students’ test scores. Brian Jablonski, Beth Edgemon, and I recently reviewed the literature on accommondations on high-stakes tests, so I happen to have much of the research at hand. Extended time is one of the two most common and most widely studied accommodations (the other is having tests read aloud). Extended time on tests does improve the scores of students with disabilities, but the improvement for students without disabilities is not so clear. In fact, in studies of multi-day testing—an accommodation that Mr. Franek rejects on other grounds—the scores of students without disabilities are no higher with extended time, but those of their peers with disabilities are higher. The absence of a differential effect undermines Mr. Franek’s argument.
Mr. Franek expressly suggests eliminating the timing of the SATs. This is probably a good suggestion, but not for the reasons he suggests.
Link to Mr. Franek’s editorial (free registration required).
Sphere: Related Content
What about the other part of the argument, about the reliability of “SAT+accomodations” as predictor of later college achievement?
AND I have anecdotal evidence that the College Board is getting less accomodating…that is, really scrutinizing the requests and denying more than they used to.
“Extended time on tests does improve the scores of students with disabilities, but the improvement for students without disabilities is not so clear.”
The first part of this is true, but the second part is tricky. With tests that have speededness as a factor, I can only think of two studies that compared students without disabilities to students with LD. Munger and Loyd (1991) did a study with the ITBS and found a very small effect for extended time and no interaction between extended time and disability status. Mandrinach, Bridgeman, Cahalan and Trapani (2005) did a study with the SAT and found that extra time benefited medium and high ability students, but not low ability students (in fact extra time was slightly detrimental to this group’s scores). Mandrinach, et al. did not find an interaction effect between extra time and disability status. With an N of 2, the research seems to say that extended time is not differentially beneficial to students with LD on tests with a speededness factor.
“In fact, in studies of multi-day testing&—an accommodation that Mr. Franek rejects on other grounds—the scores of students without disabilities are no higher with extended time, but those of their peers with disabilities are higher. The absence of a differential effect undermines Mr. Franek’s argument.”
I don’t think we can compare multi-day testing with SAT testing. The task of responding under time limits (referred to as “speedednees” in the testing community) like with the SAT is a much different task then taking multiple days to compose writing samples (e.g. Crawford, Helwig, & Tindal, 2005).
“Mr. Franek expressly suggests eliminating the timing of the SATs. This is probably a good suggestion, but not for the reasons he suggests. The extended time on the SAT helps in the analysis by colleges and universities when examining applicants.”
I think we need to know whether accommodations, like extra time, allow the score to act more like those of students without disabilities (who did not have extra time) in a regression equation that includes high school GPA when predicting first year college performance. SAT scores are useful for their predictive validity. Requiring other validity, such as construct validity, is less important in this case.
Brian, thanks for the extension and additional data. By the way, for those who might wonder, I’m not the Lloyd in Munger and Lloyd (1991).
JohnL
Liz, your question is a good one. As Brian’s response notes, if we use multiple independent variables (SAT + HS GPA + ?), we can get a better prediction of first-year college grades; for academic reasons, I’m interested in the relationships among those variables, but I have little applied interest in the dependent variable (first-year grades) itself. I don’t see that variable as particularly satisfactory and would expect only to be able to explain a little of the variation in it.
I find the SAT intriguing because it’s a carefully constructed (psychometrically) assessment that measures variation in something, whatever that something is. I’d like to understand what that something is, just as I’d like to understand what that score we call “IQ” represents. But, whatever these things are, they are abstractions that are at least a couple of times removed from the outcome of greater interest to me. Class grades also at least once removed from the outcome of greater interest to me, too.
I’m interested in what the student knows and is able to do given instruction in an area. General aptitude (SAT or IQ) probably predicts that outcome, but I suspect that we could arrange instruction powerful enough to ensure that most (not all, to be sure) students can meet some minimum standards indicating competence in the area under study. No, of course I don’t mean that we can teach severely handicapped babies to solve complex probability problems; I just mean that something about the instruction in a course on probability would predict whether individuals would meet an end-of-course assessment better than IQ or SAT.
That’s a long non-answer, no?