Last week I invited readers to participate in a little experiment. I’d had what I thought might be a big idea: a possible correlation between rate of reading speed and facility with learning foreign languages. My younger son and I are slow in both categories. My older son and my wife are quick in both categories. I had taken a few informal polls—friends, students—and found a correlation rate of 100 percent. Now I asked readers of LWON. The sampling wouldn’t be scientific, I acknowledged, but I still wanted to know whether, in their experience, the correlation held true.
I was wrong in more ways than one.
“Let me be the first to spoil your 100%,” wrote LWON’s own Ann. She is, she said, a “fast reader” but a “mediocre language-learner.” A reader named Veronique was the second to spoil my 100 percent (well, you can’t be the second to spoil 100 percent, because once it’s spoiled, it’s spoiled; you can’t even be “the first,” because that wording implies that there could be a second; but Ann, by her own admission, is no math whiz): “My husband is the world’s slowest reader, but something of a language prodigy. I’m a fast reader and medium-ish language learner.” Becky, however, did report a correlation: “fast reader…quick to pick up languages.” Giedra reported the same correlation, but added that neither of her two children show any evidence of either correlation.
And so on, throughout that day and for days to come. Some readers reported a close correlation, some reported a maybe-correlation, some reported no correlation. A conclusion was inescapable: My hypothesis had been wrong.
But then I realized that so, too, had been my characterization of the experiment as “unscientific.” It was scientific, just not in a way I anticipated.
In retrospect I could see why I had used the word “unscientific.” The reading/language correlation rate in my previous survey had been 100 percent; I had unthinkingly assumed that the 100-percent correlation rate would hold in my LWON survey, too. But my previous survey had been unscientific: The sample was too small, at least on the scale of scientific studies. Therefore the LWON survey, with its similarly small sample size, would also be unscientific. Even if my unthinking assumption were true, and the polling wound up showing the 100-percent correlation I had expected, the result would be only, at best, a provocation for further study. In itself, though, it wouldn’t be statistically significant. It wouldn’t be scientific.
But the polling was in fact statistically significant. It was scientific. Ann’s phrasing might have been infelicitous, but her point was correct: She spoiled my 100 percent, because all it takes is one counter-example to prove a proposition wrong.
I’m not saying that her counter-example and the other counter-examples in the Comments section of my post are proof that no correlation exists. Maybe it does. Maybe for some neurological reason some people do experience a correlation between rate of reading speed and facility with learning languages. But the rate of correlation is not 100 percent.
You might say, “Oh, but Richard, you didn’t say you were testing a 100-percent rate of correlation between reading and language; you just said you were testing a correlation. And the sample is too small to say whether a population-wide correlation exists or doesn’t exist. So you were right on both counts: Some correlation might exist, and the polling wasn’t scientific.”
To which I would say, “You’re too kind.”
I know what I was thinking, even if I didn’t express it felicitously. (Hi, Ann!) In my mind I was testing a 100-percent rate of correlation, and because the hypothesis made a prediction that evidence could falsify—and did falsify—it was scientific.
I closed my previous post with the hope that the survey “might get a conversation going.” I hadn’t anticipated that so much of the conversation would be in my head, and would involve the meaning of “unscientific.” But then, that’s a kind of scientific result, too: the answer to a question, just not the one you thought you were asking.
* * *
Illustration: Freepik.
Utterly off-topic but, “a provocation for further study.” jumped off the page at me. At first “provocation” seemed like too strong a word – but actually it is perfectly appropriate to what happens when seriously (pathologically ?!) curious people come across an intriguing problem.
And as another point on your scatterplot; I’m a fast reader who is terrible at languages – the link probably being that I do not recall most of what I have read.
INFELICITOUS phrasing? My dear sir.
In true science fashion, a number of your commenters proposed an addition to your hypothesis, that maybe language ability and fast reading are themselves correlated with the ability to hear sounds mentally and to sing on pitch. So my personal data point: mediocre language, fast reading, complete inability to sing on pitch, good accent, can play good bits of symphonies in my head, and here’s one more variable — I remember sentences and phrases as though they were music and for a long time.
I wouldn’t go on about this, but I’ve always thought that verbal abilities were fundamentally auditory so I’m delighted to see I have company. I also anticipate being delighted at your third post on this subject: dealing with a multivariate hypothesis with indefinable variables.
Richard,
Your piece set me thinking about the use of “scientific” and “unscientific.” I had never thought of it before, yet I have used those words all the time.
So what is a scientific study? I fear you are way off track.
Data are data. You came up with a hypothesis: fast readers learn language more rapidly than average. You collected data. You plotted x versus y and there was no correlation. You published your results (in LWON).
Some things are missing though, that make the study wholly unscientific. First, you did not hold a press conference prior to your study to announce to the world that you were going to do the definitive study of Panek Connection (it needs a name or you will not get in the NYT). You neglected a catchy acronym for your team. May I suggest ReaDing and Fast Talkers of America Regression Study Experiment or RedFaTArse? It is so easy to make catchy acronyms now that we are allowed to capitaLizE rAndOm leTtERs.
Of course, you are using humans in your experiment, so you need to fill out the various forms required by the federal government for human experimentation. These must be signed and witnessed. This is required if for your federal grant. You DO have a grant, right? The Richard Panek Institute needs the indirects, of which you will get back about 10% after the Dean, the Vice President for Research, and the Provost have taken their share.
You have your plot in front of you. There are outliers. What do you do? You can use a three sigma clipping, but that does not help. You could just plot the outliers with a smaller symbol, but you did that last time. Ah! To be really cool, you must use Bayesean statistics. You need a prior. A Richard Prior, one that excludes journalists from Baltimore, for instance. That will get rid of obstreperous data points.
no, let’s not go there.
You have the preprint written. Now the dilemma. Do you submit it to the preprint server before it has been refereed or not? You, being older than 25 years, feel that one should not put it in the server until after the referee process. But you have heard of some hotshot from Large Midwestern University who is also doing this study, and you don’t want to get scooped. So you send the preprint to the journal AND the preprint server.
But wait! What if that person from LMU also submits? You need to hold a press conference. Now! Fortunately, there is a conference of the American Fast Talking Bilingual Readers Society being held in Albany, and you arrange the PR. You suggest a few other experts, from Ivy League colleges and Stanford, to appear with you, to help field questions and look serious. Maybe you can get Steven Pinker. He seems everywhere at the same time. And has cool hair. If you are lucky, it gets in the NYT (that is why you asked Steven Pinker). But chances are it gets picked up by Slate or even worse, Wired.
The referee’s report on RedFaTArse came back. Most of the comments are related to the fact that you did not refer to some of the referee’s papers. Easy to fix. You send in the final version, and it finally gets published.
Done!
well, not quite. You start to get emails from strange people in Oklahoma who knew your result already because it was in John 13:2. And then there are the retired engineers in Sepulveda California who write emails IN ALL CAPS SHOWING THAT THE PANEK CONNECTION IS EXPLAINED BY THE CORIOLIS EFFECT IN MAGNETIC VORTICES AND WOULD YOU PLEASE READ THIS 68 PAGE PDF (which is copyrighted and cannot be quoted unless you have permission from the author)?
Now, that is what I call a scientific study.
cheers nick
I’m fascinated as usual, and that was before I had to go look up “infelicitous.” =) Fast reader, decent with languages, but that was back in high school and college (four semesters of Spanish to get my B.A. in Journalism and escape Calculus.) Like Ann, I, too, correlate words with music, and I have proof that it works, at least for me. My kids laugh all the time at the useless jingles I can remember (mostly from radio) and never need encouragement to belt out in traffic, often with the windows down. I laugh because each of them is afflicted by nature (or is it nurture?)
I find it amazing that you are a slow reader. And I kinda want to see that two-page essay on “The Odyssey” and “King Lear.”
Awesome stuff! The head “conversation”, IMO, is what makes you “scientifically-minded” in the first place (A clearly unscientific assertion). The ability to explain yourself to an audience of other thinkers in a way that creates a similar “out-of-head” conversation with them, sparking ideas for further inquiry and critiques, is what makes you a “scientist” (ditto on this assertion).