On the illusion of explanatory depth, forty-two college application essays, and the one time in my life somebody actually checked.

Here’s a question I’d like you to answer, and then I’d like you to answer it again in about ninety seconds.

On a scale of one to seven, how well do you understand how a zipper works?

Most people land on a five or a six, and they aren’t being arrogant about it. You have operated thousands of zippers. You have, at some point, tried to unstick one with your teeth in a hurry (which has never once worked for anybody, and which all of us keep doing anyway), and you know perfectly well that there are teeth, and that the teeth interlock, and that the little slider is the thing doing the interlocking. Five feels modest. Six feels defensible.

Now write down the mechanism.

Not “the teeth interlock”, because that’s the outcome and not the mechanism, in the same way that “the car moves” is not an explanation of an internal combustion engine. What shape is the channel inside the slider? What is it doing to one individual tooth as that tooth passes through it? And why does the exact same slider, moving over the exact same teeth in the opposite direction, take them apart instead of putting them together? Nothing about it changes. It has no moving parts. It is one solid piece of shaped metal that performs two opposite operations depending on which way your hand goes, and I would put ten dollars on the proposition that you have never once wondered about it.1

Your number just went down.

Not because you learned anything, either; you have precisely as much information about zippers as you did ninety seconds ago. It went down because you went looking for the explanation and found out what was actually in there.

This has a name, and a genuinely lovely paper behind it. In 2002, two researchers at Yale, Leonid Rozenblit and Frank Keil, sat people down with a list of thoroughly ordinary objects (a bicycle lock, a sewing machine, a zipper) and had them rate how well they understood each one on that same one-to-seven scale.2 Then they asked their participants to imagine they had just met somebody who didn’t understand how the thing worked, and to write out as complete an explanation as they could manage. Then they asked for the rating again.

The ratings fell. Across objects, across groups, and across a great many replications since. They called it the illusion of explanatory depth, and the part that got me isn’t that people are wrong about zippers, because who genuinely cares about zippers. It’s that the wrongness is comfortable. It doesn’t feel like anything from in there.

The feeling of understanding something is calibrated on how familiar it is, and not on whether you actually have a working causal model of it sitting anywhere in your head. Familiarity generates a warm and comfortable sense of comprehension which is, from the inside, completely indistinguishable from the real article (I have now read this paper properly, taken notes on it, and written several thousand words about it, and you will find me down in the footnotes having rated myself a six anyway). And the illusion is stable, because the only operation that ever actually touches the model is trying to produce the explanation out loud, and nobody has ever asked you to do that about a zipper.

There’s a second thing folded into this, and it’s the part I can’t stop turning over. The repair only fires if you go after the mechanism. Ask somebody instead to list their reasons for holding a position, and nothing happens at all. No drop, no correction, nothing.3 Reasons are fluent and infinitely available; they run on values and principles and something a cousin said at a wedding once, and not one of them requires you to know a single thing about how anything actually works.

Mechanism is the only part that can come back empty.

Here is what comes back instead, when it isn’t mechanism. In 1977, Richard Nisbett and Timothy Wilson set up a table in a shop and asked passers-by to look at four pairs of nylon stockings laid out in a row, and to say which pair was the best quality, and why.4

The four pairs were identical.

People overwhelmingly picked the one on the right; the rightmost pair beat the leftmost by very nearly four to one. Not a single person spontaneously brought up position as a factor. And when the experimenters asked them directly, straight out, whether the position of the stockings in the row might have had something to do with it, virtually all of them said no, and I’m going to quote the paper here because I don’t think I can do better than it did. They denied it “usually with a worried glance at the interviewer suggesting that they felt either that they had misunderstood the question or were dealing with a madman.”

Those people were not lying, which is the whole finding. They were asked why they had chosen, and the explanation-making machinery in their heads did what it always does, which is produce something fluent and plausible about sheen and elasticity and weave. It has no special access to the actual cause. It just runs. And the output feels, from the inside, exactly like a memory of having reasoned.

So there are two things in here and they run in opposite directions. Ask about a mechanism out in the world and you can come back empty, and the empty is the whole finding; you go quietly back down to a three. Ask a person why they did something and nothing ever comes back empty, because the machinery will always, always produce one more sentence.

I want to tell you about the one time in my life I am certain the correction actually fired, because it took a stranger to do it and I have never managed it alone since.

I was fifteen. I spent a lot of that year in a physics help channel on Discord, answering questions for people I had never met, which is the sort of thing that feels enormously important when you are fifteen and have just discovered that you are good at something. And for months I had been explaining redshift. I was good at explaining redshift. I had the equation, I had the derivation, I could produce the whole thing from memory, and if you had sat me down and asked me to rate my understanding of it on a scale of one to seven I would have said a six without blinking and I would have been a little insulted by the question.

Then there was a student. I never knew their real name, I have never met them, and (barring something genuinely strange happening) I am never going to.

I typed the explanation out a dozen different ways. Each one more carefully worded than the last, each one with the analogies sharpened and the ordering fixed and the one confusing clause from the previous attempt surgically removed. And every single time I hit send I got back the same five words:

I still don’t get it.

I want to be honest about what that did to me at the time, which is that it made me irritated. Not with myself. Twelve explanations is a lot of explanations, and somewhere around the eighth I had privately concluded that the problem was on the other end.

And then, eventually, something new:

I told you. The star’s receding, sure. But that changes the wavelength? I can’t see it happening.

I noticed the word see. I drew a diagram on a page of my notebook, took a photo of it, uploaded it into the channel, and got back an immediate “OH! That makes SO much sense!”

So here’s what had actually been going on. I had been teaching redshift for months as an equation, fully believing that I understood it, and I had no picture of it anywhere in my head at all. None. The equation had been doing all the work and I had mistaken having the equation for having the thing, which is the exact error this entire essay is about, and I was committing it fluently and confidently in public to an audience of strangers.

I did not find that out by introspecting harder. I did not find it out by rereading my own notes, which contained the same equation and would have gone on agreeing with me forever. I found it out because there was a person standing on the other side of the explanation with the standing to reject it, and who had no particular reason to be polite to me, and who went ahead and did it twelve times in a row.

That’s a failing test case. It is, as far as I can tell, the only one I have ever been handed.

I have a folder on my desktop with forty-two attempts to explain myself in it.

They’re college application essays, written over a few frantic months when I was sixteen and seventeen, for eleven different schools. Some of them are 650 words and some are 100. The prompts vary the way these prompts always vary (describe a challenge, describe your community, describe what brings you joy, describe a time you disagreed with somebody and what you took away from it), but underneath every single one of them is the same question, which is: explain yourself. Why are you like this. What made you this way. Show your working.

And I did. Fluently, at length, and repeatedly.

Take the mosquito.

I was raised by a Hindu father and a Jain mother, and somewhere in my early teens I worked out a version of that which I could actually live inside. In the essays, it goes like this:

When a mosquito settles on me, I shake it off, unharmed.

That sentence, or something within a few words of it, turns up in six separate files. By the later drafts it has picked up choreography; the mosquito has relocated to my thigh and I have started tapping my heel against the floor to dislodge it, which is a better sentence than the original, and I want to be honest that it is a better sentence because I sat there and revised it into one, and not because I had learned anything new about mosquitoes.

The paragraph itself is stable. What moves around is what it is supposed to mean. For Stanford, that upbringing produced somebody who would go on to facilitate interfaith discourse. For CMU it produced a “creative disruptor” who refuses to be forced into a single mold, which was CMU’s language on their own website before it was ever mine. For Cornell it taught me to honor coexistence over conflict. For Harvard, to navigate complexity. Same childhood, same mosquito, four different lessons, every one of them selected because it matched what that particular school had already said about the sort of person it was hoping to admit.

Here’s the detail that I think about more than any of that, though. My religious practice, as described in those files, scales with the word limit.

The 350-word version of that paragraph has me fasting. I forsake grains and vegetables on Janmashtami, I go hungry for thirty-six hours straight on Samvatsari, I recite the navkar mantra every night and I go to the temple with my grandfather on Sundays. The 250-word version drops the fasting. The 150-word version doesn’t have room for the mosquito at all and so the whole thing compresses down to a claim about respecting differing perspectives, which is the sort of sentence that could have been written by anybody about anything.

Every one of those files is true. I did fast. But a reader at Cornell met a considerably more observant person than a reader at Harvard did, and the variable doing that work was not my religion; it was how many words I had been given.

The nightlight does the same trick with a different knob. Somebody slipped on a puddle in a dark bathroom in the middle of the night, and I sat on the floor afterwards feeling completely useless, and then dug out a robotics kit that an uncle had given me and which I had never opened because the manual looked frightening, and built a motion-activated nightlight by morning. It flickered. It mostly didn’t work. It’s a good story, and it is the opening paragraph of five separate essays.

The only thing that moves between those five is what I call the person on the floor, because English makes you pick and my family doesn’t. The boy I grew up alongside is my bhai, and that distinction doesn’t really exist for us, or in most Gujarati families I know. “Cousin” is correct on the family tree and lands in an American reader’s ear as somebody you see twice a year at weddings, which is not the relationship. “Brother” is wrong on the family tree and right on every single other axis that actually matters. So it came out one way in some of the files and the other way in the rest, and I was not lying in either of them; I was choosing which true thing to lose.

I also want to be honest that I did not do any of this entirely by myself. There was a college counselor, and there were months of back-and-forths, and there are files in that folder with names like (2nd edit) with more edits.5 Nobody ever suggested I make anything up. The entire craft, and it genuinely is a craft, is arrangement: which detail leads; which sentence carries the paragraph; what this particular school has already announced on its own website that it cares about; and which of the true things about you happens to sit nearest to that. I got quite good at it, and I am still a little proud of how good at it I got.

Which is the part I didn’t understand at seventeen. None of it trips any alarm. There’s no lie in there for anybody to find.

A Finder window showing the essay folder: eleven school subfolders, forty-two documents.

Forty-two files across eleven schools, and every one of them written to be read by somebody who had never met me.

And the student is in there.

Four times. The same eight lines of chat log, pasted into four different applications, and here is what that conversation taught me, according to me:

  • To Stanford, it taught me that teaching isn’t about designing the most efficient explanation, but about understanding how different individuals receive learning.
  • To MIT, it taught me that the challenge was never the student’s confusion; it was my own inflexibility.
  • To the UCs, it was evidence of my creativity.
  • To Penn, written up as a thank-you note addressed to the student directly, it taught me that communication isn’t about showing what you know, but about meeting somebody where they are.

Four morals, one conversation, eight lines of chat.

I would like you to notice what I did there, because it took me two years to notice it myself. The single occasion in my life on which somebody reached in from outside and corrected a thing about me that I had no access to, the one genuine failing test case I have ever been handed, got harvested. It went into the machine and came out four different ways, arranged each time to demonstrate what a thoughtful and adaptable person I am, which is precisely the sort of claim that no admissions officer on earth was in a position to check.

I took the only evidence I had that I can be wrong, and I used it as evidence that I’m not.

I have spent a while trying to argue myself out of this, because it is a suspiciously tidy story and I have read enough of my own writing to be wary of those.

The obvious objection is that I have been counting repetitions as the tell this whole time, and the student is in four files. So it is entirely possible that I have this backwards; that the most-repeated story is most-repeated because it’s load-bearing and actually true, that the nightlight really is why I build things, and that all of this is an elaborate way of avoiding a straight answer I have had the entire time.

The second objection is better, and it’s that the arranging was done by a professional. There was a counselor. Maybe I was only ever reporting, and somebody else did the tuning, in which case the file in my head is fine and it’s only the files on disk that are bent.

I can’t fully rule out either one. But there is exactly one claim in that entire folder that anybody other than me ever independently checked, and it isn’t any of the ones you’d pick.

In the note I wrote to my future Stanford roommate, I explained that I am a very physically animated person, that my ears twitch when I’m thinking hard, and that my nostrils shiver when I’m upset. Some weeks later I told Harvard the same thing and threw in, for good measure, that my shoulders wiggle when my music gets particularly “vibey.”

All of that is true. But notice how I know it’s true, because it isn’t how I know anything else in that folder. You can’t see your own face while you’re upset. That one arrived from outside, from people who happened to be in the room looking at me and who eventually got round to mentioning it.6 I didn’t find it by introspecting. I couldn’t have.

So: forty-two documents on why I build things, and what my upbringing taught me, and what kind of person I intend to become, every single one of them sourced entirely from the one witness who can’t be cross-examined. And the only sentence in there that anybody ever stood next to me and confirmed is the one about my nose.

That’s the direction the honest information travels. It comes in from outside, or it doesn’t come.

Which is the whole problem, I thought, for about two years.

But go back to the slider. The same single piece of metal runs both ways.

So does this.

Run the explanation machinery in one direction and it is reading you, badly, producing confident accounts of a mechanism it cannot actually see. Run it the other way and it is writing you. There’s a line of work going back to the seventies on what happens when you describe somebody to an audience whose views you already know: you tune the description toward the audience, which everybody does and nobody thinks twice about, and then your own memory of the person quietly drifts toward the version you told out loud rather than the one you actually stood there and watched.7

I wrote forty-two tuned descriptions of one person to eleven known audiences. Then I revised them, and revised them again, and read the results back to myself more times than I can count, over a period of months, at seventeen, which is not an age at which the file is exactly write-protected.

Those essays were never only a description of a person. Forty-two of them went out, and what came back has had two years to settle in and start answering to its own name. I didn’t report a mechanism in that folder. I specified one, eleven times, to eleven different sets of requirements, all of it true, none of it checked, and then I went away and built from it.

I would like to know whether I actually honor coexistence over conflict, or whether Cornell wanted 350 words and that is what fit.

I could go and learn how a zipper actually works this afternoon. That question has a real answer, sitting in a patent from 1917, and if I got it wrong the zipper would tell me immediately, on the grounds that it would not close.

Sheet one of Gideon Sundback's 1917 separable fastener patent. Fig. 1 shows the Y-shaped slider with two rows of teeth entering the upper arms and a single joined chain leaving below it.

Gideon Sundback’s separable fastener, US Patent 1,219,881, filed in August 1914 and granted on the twentieth of March 1917. Fig. 1 is the Y-shaped channel. Fig. 2 has the body cut away so you can see the wedge sitting in the middle of it. Fig. 4 is a single tooth, nub on one face and hollow on the other. It has been sitting in a public archive for a hundred and nine years, and I had to write four thousand words about self-knowledge before it occurred to me to go and look.

The student told me immediately too. Twelve times, in fact, with a patience I did not deserve and did not, at the time, particularly appreciate.

The folder has been sitting on my desktop for two years and has never once told me anything. Not because there’s anything wrong with it. Because there is nobody standing on the other side of it. Every one of those essays got read, and several of them got read very carefully, and I am typing this from Berkeley, so I know how that went. But an admissions decision is a reward signal and not a truth signal; it tells you the story worked on somebody who had never met you, which is a fact about the story. Nobody at Cornell was in any position to write back and say that isn’t actually why you build things, because they had never met me and they could not possibly have known.

So I know exactly what the test looks like. It’s five words, from somebody with no particular reason to be polite, who has been standing there the whole time watching me explain myself, and who finally goes ahead and says it.

Nobody has ever typed those five words about me.

And I have been sitting with this long enough now to have arrived at the genuinely unpleasant part, which is that I know what would happen if somebody did. I wouldn’t come back empty. Not for a second. I would come back fluently, immediately, and in my own voice, with a reason.

  1. Since I’ve now made a whole thing of it, I owe you the answer. A zipper slider is a Y-shaped channel with a wedge sitting in the middle of it. The two rows of teeth come in through the upper arms of the Y and leave as one joined chain through the bottom. Every tooth has a little nub on its top face and a matching hollow underneath, and here’s the part almost nobody has, which is that the teeth cannot simply be pressed together; they only engage if they arrive at each other at an angle, so that the nub of one can seat itself into the hollow of the one opposite. Controlling that angle is the wedge’s entire job. Drag the slider the other way and the same wedge, now meeting the joined chain from underneath instead, levers the teeth back apart at the angle at which they come loose. No moving parts, two opposite operations, one piece of shaped metal. I have been using zippers for nineteen years and I rated myself a six. ↩

  2. Rozenblit, L., & Keil, F. (2002). “The misunderstood limits of folk science: an illusion of explanatory depth.” Cognitive Science, 26(5), pp. 521-562. Worth reading properly, partly because they take the obvious objection seriously (maybe people just get flustered when a stranger asks them to write an essay about a sewing machine) and then go after it from several different directions across twelve experiments. ↩

  3. The reasons-versus-mechanism asymmetry is cleanest in Fernbach, P., Rogers, T., Fox, C., & Sloman, S. (2013), “Political Extremism Is Supported by an Illusion of Understanding,” Psychological Science, 24(6), pp. 939-946, which pointed the whole thing at policy positions and found that explaining the mechanism moderated people while listing reasons did nothing. In the interest of not committing the exact sin this post is about: that political result has had a rough time of it lately, and a 2021 paper reports three preregistered failures to closely replicate it. The zipper-level effect is on much firmer ground than the policy-level one. I find it quietly hilarious that the study about explanations collapsing under scrutiny went on to partly collapse under scrutiny, and I would rather tell you that myself than have you find out later. ↩

  4. Nisbett, R., & Wilson, T. (1977). “Telling More Than We Can Know: Verbal Reports on Mental Processes.” Psychological Review, 84(3), pp. 231-259. One of the most-cited papers in the whole field, and the stockings take up about a page of it. ↩

  5. There is a real file in that folder called Stanford Short Questions (2nd edit) with more edits, which sat next to Stanford Short Questions (reviewed) and Stanford Short Questions (final), and I would like it on the record that seventeen-year-old me had absolutely no system and was not going to be getting one. ↩

  6. I have still never actually seen it happen, and at this point I don’t expect to. I am taking the whole thing on the word of people who were in the room, which I notice is the exact epistemic position I was cheerfully asking eleven admissions offices to adopt about everything else in that folder, just with considerably less at stake. ↩

  7. The classic is Higgins, E. T., & Rholes, W. (1978), “Saying is believing: Effects of message modification on memory and liking for the person described,” Journal of Experimental Social Psychology, 14(4), pp. 363-378. Participants described a person to an audience they knew liked or disliked that person, adjusted the description accordingly, and then recalled the original person as more like their own adjusted version. It’s usually filed under “audience tuning.” I’ll note, since this essay would be insufferable otherwise, that I have not gone through this one’s replication record with anything like the care I gave the Fernbach result, and you should hold it a little more loosely than I am currently holding it. ↩