About a year ago I was reading the excellent book Thinking Fast and Slow by Daniel Kahneman. In one chapter the author mentions an experiment in which participants were asked whether the letter “k” was more likely to appear in the first or third position within a word. The point was to show that humans tend to overemphasize information that is readily available to them. So instead of making a genuinely valid statistical prediction they would simply predict more “k”s at the start because it’s easier to think of words that start with “k” than words that end with “k”. The book asserted this was not true and there were in fact about three times as many words with “k” as the third letter.

First Analysis

I tried this on my father and sure enough he believed there were more words with “k” at the start. I showed him the book but he was still skeptical. So I decided to write a brief program to prove the point. It would simply scan through the Unix spell checking dictionary (I was using Ubuntu at the time) and compare the number of words with “k” in the third position to the number with “k” in the first position. But to my surprise the results showed that “k” was actually more common in the first position than the third. I don’t have my original results from that day. But I recently heard the claim that “k” is most common in the first position again on a youtube video (warning, silliness). So I decided to write a new script to find out. You can see the dictionary I used at [https://github.com/dwyl/english-words] and my code at [https://github.com/WilliamRitson/letter-positon-frequency]

Here were my initial results

I did it for the full alphabet but I omitted everything after K to make the table fit better. Regardless, you can see that K is more common in the first position than the third, but less common than in the fourth. This looked like it might be an off by one error to me. But I checked my code several times and could not find one. Feel free to look at my python code on github to make sure I did not make such an error.

Round Two

Having failed to replicate the expected pattern I decided to look up the original paper that the book referred to. I found it here eventually. The actual quote from the paper is “In fact, a typical text contains twice as many words in which K is in the third position than words that start with K.” (note, it’s possible that by “third positon” they mean the third position in a zero indexed system. I think this is unlikely as it would probably confuse the participants of the psychological study as much as it confused me). But as you can see my methodology was wrong. The claim was not that there are more words in the dictionary with “k” as the third letter than the first. But rather that these words show up more often in a typical text.

So I decided to do a new analysis. This time I would take a large corpus of English text and tokenize it into words. Then I would count how many of each word existed. Finally, I would analyze what position each letter occurred in weighting it by the number of times the word appeared in the text.

Here were my results on the Open American National Corpus

As you can see not much changed. K is still more common in the first positon than the third (although still less common than the fourth).

Conclusion

So the result of several hours of programing and research was that I still could not replicate the original claim. I don’t by any means claim this as a debunking or rebuttal. Its entirely possible and likely that either misunderstood the claim, my code contains an error, or the data I used is not statistically valid. Regardless, it was an interesting result and I thought I would share it.