What Would Python Display?

On owning five mechanical keyboards and barely typing code on any of them, and the most annoying exercise in CS 61A turning out to be the whole job.

I acquired five mechanical keyboards in a single year (and I’m choosing that verb carefully, because two of them were given to me, which means I can only be held fully responsible for three), which is a sentence I would like you to read with exactly as much judgment as it deserves.

I could try to justify it (and believe me, I have tried, out loud, to my parents, to my roommate, to my best friend Krrish every single time he calls me out for it on our every-other-day FaceTime calls, and once to a very patient customer service representative about a return window), but the honest version is just that I love typing. I love it in a way that I think is slightly embarrassing to admit in public. I love the small, round thock a good switch makes at the bottom of its travel, and the way a long line of code sounds like a short burst of rain on a window when it’s going well, and the faintly ridiculous ritual of pulling every keycap off a board on a Sunday afternoon to lubricate the switches underneath with a tiny brush, one at a time, all eighty-something of them, while a podcast plays in the background that I will later realize I didn’t hear a single word of.¹ For most of my life, writing code was the single most satisfying physical thing I did with my hands. I’d sit down at one of those boards at eleven at night with a problem in my head and look up at two in the morning with a working program on the screen and sore wrists and absolutely no regrets, and the typing wasn’t incidental to that feeling. It was that feeling, or at least a big part of it.

Four of the five, photographed on three different surfaces, one of which is the floor. The one in the bottom left is in the middle of a keycap swap, which is why its switches are showing and half its keys are still in the plastic trays next to it.

Which makes it a little awkward that, these days, I barely type code at all.

Most of what goes through my keyboards now is English. Long paragraphs of it, to agents running in terminal windows, describing what I want built and (more often, and at considerably greater length) what I don’t. Review comments. Specifications. Sentences like no, not like that, read the test first, typed with a little more force than strictly necessary. The code itself mostly arrives from somewhere else, in big confident blocks, and most of my job now is reading it and deciding whether it gets to stay.

And the strangest part, the part that makes people squint at me a little when I explain it, is what I actually spend my time building.

---

For a slice of every week, I’m a campus lead for SpaceXAI, which mostly means standing at the front of rooms full of Berkeley students and getting them to use Grok and Cursor for whatever it is they’re building, which is to say, getting them to type less.² For a much larger slice of every week (and most of my nights, and a frankly worrying number of my weekends) I build Provenance, which is a piece of software that lives inside a student’s code editor and records, edit by edit, how their code actually came to exist. If you open one of its replays, you watch a student’s file assemble itself in front of you like a time-lapse of a building going up, colour-coded: anything they typed stays plain, anything they pasted glows orange, and anything that arrived from outside the editor entirely (an agent running in another terminal, say) shows up in pink.

A replay of one of my own test sessions from the twentieth of May, two days after the first commit, on a made-up homework file for a calculator. Line 5 is highlighted because it came from outside the editor. That’s event #37 in the list on the right, the red one, and it’s where I’ve stopped the scrubber.

So for half the week I’m encouraging people to type less, and for the other half I’m building software that checks whether they typed, and I’m doing both of those things on keyboards I own specifically because I like typing so much.

Me, Krithin, and Nil, left to right, at a SpaceXAI event, with Nil holding down the far side of the logo by himself.

People find this funny, and I get it. The question I hear, in a range of tones running from genuinely curious all the way to faintly accusatory, is which side I’m actually on. It does look a little like a man selling umbrellas out of one window and lawsuits against the rain out of the other.

But I don’t think it’s a contradiction. In fact, I’d like to go a fair bit further than that, which is to say that I think AI is the best thing to happen to software in my lifetime (nineteen years, granted, which is a smaller sample than it sounds), and that it’s going to turn out to be good for how we teach it, too. And I’d like to skip the reasons you’ve already heard, because you’ve already heard them. It writes the boilerplate; it makes you faster; you will never again have to search for how to center a div. All true, and none of them very interesting. The reasons I actually believe are weirder, and almost every one of them comes out of the last five months of building an anti-cheating tool with the help of the very thing it is supposedly against.

---

Provenance has a recorder for VS Code, which is what the overwhelming majority of students in Berkeley’s lower-division courses use. It has a recorder for JetBrains, for the people who like their editor to have opinions. And it has a recorder for Neovim.

The Neovim recorder is, if I’m honest, my favourite thing in the entire project, and it is a slightly absurd object. It’s written in pure Lua with no native dependencies whatsoever, which means that the ed25519 signatures and the XChaCha20-Poly1305 encryption it needs had to come along in pure Lua too, vendored in, so that nobody installing it would ever have to compile anything.³ Because a Neovim plugin isn’t a signed artifact the way a VS Code extension is, it proves it hasn’t been tampered with by hashing its own source code, like a person checking their own pulse. And its README contains a genuinely stern paragraph warning you not to lazy-load it, because for an integrity tool the worst possible outcome is that your work goes quietly unrecorded and nobody finds out until the exact moment it matters.

The paragraph in question, which uses bold type twice and the phrase “the worst outcome,” and which I stand by completely. It’s there because LazyVim, which a lot of Neovim users run, lazy-loads plugins by default, and would quietly stop the recorder from ever starting.

I don’t know exactly how many students it’s for. My guess is that you could fit every single one of them into one discussion section with a few chairs to spare. And I know, with something close to total certainty, that in any previous era of software those students would have received the same answer that small groups of people have always received, which is please just use VS Code.

Nobody says it to be mean. Software costs an enormous amount to build and almost nothing to hand out one more copy of, so it gets written for big audiences, and everybody below some invisible threshold rounds down to zero.

I’ve lived on the wrong side of that threshold more than once. Back in Ahmedabad, when I was studying for my Cambridge exams, the only real way to prepare was past papers, hundreds of them, scattered across websites that each wanted you to click through one PDF at a time, and so I wrote a clunky little Python program with an even clunkier little window on top of it that would go and fetch every paper for a subject and staple them all together into one enormous file.⁴ It was built for an audience of me, plus maybe four friends who were tired of watching me complain. It is, to this day, the most-starred repository I have ever made, beating out everything I have built since, which mostly tells you how many other people were also sick of downloading past papers one PDF at a time.

The clunky little window in question, as screenshotted for its own README. Four text boxes, six checkboxes, two buttons, and a title bar that says “by Geagle,” because I had a username and I was going to use it.

And then, later, there was TechNetra, which ate the better part of two years of my high school life. It’s a print-to-audio reader for visually impaired students, built with the Blind People’s Association in Ahmedabad, and the reason that particular piece of work fell to a teenager with a Raspberry Pi and a soldering iron and no idea what he was doing is that the market for a cheap reading device for blind students in Gujarat is, in the eyes of every company that might ever plausibly have built one, approximately nothing. It worked, in the end. It even got a patent. But the only reason it existed at all is that I didn’t need it to make anybody any money.

An early prototype of TechNetra. The page goes in the wooden frame, the webcam on the green arm reads it, and the grey box with the two big buttons does everything else.

Now hold both of those next to Provenance. Since the eighteenth of May, it has picked up nearly fourteen hundred commits, all but a handful of them under my name,⁵ spread across three editor recorders in three different languages (TypeScript, Kotlin, and Lua, the last of which I had written approximately zero lines of before this summer), a server, a web analyzer, and a little Python gateway that pulls submissions out of Gradescope on a schedule. This semester it’s running across four courses and more than 2,800 students. And I want to be precise about how that happened, because it’s easy to tell this story in a way that makes me sound like either a genius or a fraud, and I don’t think I’m either. I designed it. I wrote the requirements, and argued with myself about them for weeks, and read every change before it went in. And agents did a truly enormous amount of the typing.

$ git log --format='%an' | awk '{ print ($0 == "Aaryan Mehta") ? $0 : "everyone else" }' \
    | sort | uniq -c | sort -rn
   1392 Aaryan Mehta
      3 everyone else

The entire history of the project, according to git, as of the copy on my laptop.

Without them, this is a year of work for a team of five people. And I can promise you that a team of five people on a university budget does not spend a month on the Neovim port.

So that’s the first thing I believe, and it’s the one I care about most. The usual pitch is that AI makes software faster, and it does, but I think the far more important thing is that it makes software smaller. It makes it viable for audiences that used to round down to zero: the handful of students who live in a terminal editor, a course staff of six who need a script that wakes up every hour, the competitor management system for a Garba competition with a couple hundred dancers in it, a reading device for a language with a few million readers instead of a few hundred million. Most of the useful software that could exist has never been written, and it’s usually not for lack of ideas; nobody could afford to write it for so few people.

I think this is the change almost nobody talks about, because it doesn’t show up in anybody’s quarterly numbers. It shows up as a handful of 61A students who get to keep using Neovim.⁶

---

The fear I hear most from people who write software for a living runs in the opposite direction. It’s that all of this is going to drown us in slop: that once code becomes cheap, nobody will bother making it good anymore, and in five years every codebase on earth will be a forty-thousand-line landfill that a model produced and nobody alive understands.

I think it’s about half right.

Here’s something I’ve noticed over five months of working this way. An agent reads your repository the way a nervous new hire reads it on their first morning, which is to say as a style guide. It does whatever the code around it is already doing. If your codebase is clean and consistent, it writes clean, consistent code; if your codebase happens to have the same helper function copy-pasted into three different files (which mine did, for a while, and which turned out to be three separate quadratic-time bugs stacked on top of each other wearing a trench coat⁷), then an agent asked to add a fourth code path will happily write you a fourth copy, with perfect formatting. Good code makes the agent better and bad code makes it worse, and both of those happen a lot faster than they used to.

That alone would be an argument that code quality matters more than it used to, not less. But the bigger shift, the one that took me months to notice, is in where the quality actually lives.

There’s a file at the root of Provenance called CLAUDE.md, which is a standing set of instructions for every agent that works on the codebase, and two lines in it do more work than any other two lines in the whole repository. The first: “Inventing architecture is the single biggest failure mode on this project.” The second: “Tests encode requirements; loosening them is a product decision, not a coding decision.” The JetBrains recorder has to produce output that is byte-for-byte identical to what the VS Code one produces, and the way it proves that is by running against a set of reference outputs that, in the words of its own README, “are never edited to make it pass.” And when I rewrote the code that replays a student’s edits to rebuild their file, I didn’t trust the new version, not even a little, until it had been run against the old one on a thousand randomly generated streams of edits with the results compared byte for byte, every single time.

The top of the “Working agreement” in Provenance’s CLAUDE.md, with everything except the two lines I’m talking about faded out. It reads like a set of rules for a new hire because that’s basically what it is, except this new hire forgets everything at the end of every conversation and has to read it all again.

What all of those have in common, which took me an embarrassingly long time to notice, is that none of them are the code that actually does the work. They all exist to tell that code when it’s wrong.

When an agent can write you any implementation you ask for in about forty seconds, the implementation stops being the place where correctness lives. It moves into whatever you’ve written down that the agent isn’t allowed to touch: the tests, the types, the specification, the reference outputs, the numbers coming off your profiler. “Good code” used to mean code a human could read, and it still means that, but now it also has to mean code whose intentions are written down somewhere in a form that a machine is obliged to obey.

I learned this the expensive way, on a weekend back in June that I’ve written about before. I was hunting a performance bug in Provenance’s ingest pipeline, a single student’s submission was taking two whole minutes to process, and for hours I tried to get Claude to find the problem for me, and for hours it kept pointing, in that calm and reasonable and completely unbothered voice, at the step that saved files to storage. And every single time it did, I’d glance over at my profiler, which was sitting in the other half of my screen saying, in plain numbers, that storage was not the problem and had never been the problem. I would love to tell you that I found the real bug through some elegant flash of insight. I found it by reading every function on the ingest path, one at a time, by hand, for hours, hunched over my desk like an animal.

$ INGEST_PROFILE=1 npm run profile:large -- 50000

─── ingest profile · one 50k-event bundle ───
phase                    total     share
parse_bundle             0.21s      0.2%
materialize_events       2.50s      1.9%
run_validation          15.60s     11.8%
compute_stats           47.70s     36.0%
run_heuristics          66.30s     50.0%
──────────────────────────────────────────
handler_total          132.50s

What the profiler was saying the entire time. The two minutes live almost entirely in the bottom three rows, and nothing that touches storage comes anywhere near them. I pointed this out to the agent more than once, and it kept suggesting storage anyway.

And then the very next day I handed that same agent my old CS 61A homework folder and asked it to build me a realistic model of how students actually type, one keystroke at a time, with real auto-indentation and a real mix of characters and newlines in roughly the right proportions, and it did a better and more honest job of that in an afternoon than I would have managed in a week.

That was the same tool on the same weekend, and I think the difference between those two days is most of what I’ve learned this year. It’s very, very good at producing things, and it is not a reliable judge of whether the thing it produced is right; the only reason its confidence on Saturday didn’t cost me more than a few hours is that I had a profiler that disagreed with it. So more and more of my job is building things that can disagree with it: tests, measurements, invariants, anything that can look at a confident answer and check it.⁸ That’s what code quality turns into, and honestly, I find it a more interesting job than the one it’s replacing, you know?

---

Which brings me to education, and to the fear everybody has about it, which goes something like: if an agent writes all the code, students will never learn to write code, and we’ll graduate an entire generation of computer scientists who can’t do the one thing in their job title.

There’s something real in that, and I’ll come back to it. But first I want to point out that Berkeley’s introductory course has been teaching the most important skill of the AI era for years, and that most of the people taking it (and I include myself here, in the fall of last year, at roughly one in the morning) experience it as a toll booth.

If you’ve taken CS 61A, you know about unlocking tests.⁹ Before the autograder will let you run the tests for a question, it shows you each test case with the answer blanked out, and makes you type in what you think the output will be. Only once you’ve gotten every one of them right does it let you actually run your code:

>>> def make_adder(n):
...     return lambda k: n + k
>>> add_three = make_adder(3)
>>> add_three(4)
? 7
-- OK! --

Then there are the “What Would Python Display?” questions in the labs, which are exactly what they sound like, and the environment diagrams, where you trace by hand, frame by frame and arrow by arrow, everything the interpreter is doing as it runs a program somebody else wrote. None of these involve writing a single line of code. Every one of them is, underneath, the same exercise: here is code you didn’t produce; before you run it, tell me what it’s going to do.

The make_adder example from above, drawn as an environment diagram by Python Tutor (61A makes you draw these by hand, on exams, with a pencil). The lambda’s own frame only has k in it. The n it needs is one arrow away, inside a function call that has already finished and returned. Being able to see that before you run anything is exactly what those exercises are drilling.

Which is pretty much my entire job now.

When an agent hands me a two-hundred-line diff at midnight, the question I actually have to answer is literally what would this display: in what order, and what happens when the list is empty, and what happens when two of these run at the same time, and what happens on the one input that I know, from experience, nobody ever tests. The skill I lean on hardest as a developer who works with agents all day is reading code I didn’t write, quickly and suspiciously, and predicting how it’ll behave before I trust it. 61A was drilling that into people years before anyone had a reason to care, and it was doing it in the part of the course that people are most tempted to brute-force (you can keep guessing until something sticks, and a sufficiently tired student at one in the morning will absolutely, and I say this from experience, just try typing Error to see what happens).

We spent decades teaching students to write code, and only in the margins, almost by accident, taught some of them to read it. I would love to see intro courses do it on purpose: more unlocking, more tracing, more “here’s a diff, find the bug before the tests do,” fewer blank files. Not zero blank files, because you cannot read fluently in a language you’ve never once written (that’s the part of the fear I agree with), but fewer.

There’s a second piece of evidence for all this, and I find it a great deal funnier than the first.

Provenance’s first real pilot was on a 61A project this past summer called Cats, which (and I swear I’m not making this up) is a project where students build a typing test. They write the code that measures how fast and how accurately somebody types, while my software sits underneath recording how fast and how accurately they type it, which I promise I did not plan. And ever since, in my informal conversations with students who’ve used it, I’ve made a habit of asking the same question: if you wanted to get around this thing, how would you do it?¹⁰ I’ve heard a few creative answers. But the one that keeps coming back, from people who have never met each other, is the same one.

Everything a student actually sees of Provenance while they work: one line in the status bar saying it’s recording, which can’t be dismissed, on purpose. The square function is from my own test workspace, because apparently square is the only function I can think of when I need an example.

This semester’s Cats page, cat GIF and all. The note at the top covers most of what this post is arguing: you can use one AI tutor (Preceptor, which the course runs itself), the Provenance bundle records how you used it, and to get credit you have to explain your solution out loud to a member of staff, which is basically “What Would Python Display?” for an entire project.

Get the answer from an AI. Then retype it by hand, so the recording sees typing instead of a paste.

They’re right, by the way. We say so publicly. Retyped output looks exactly like typing, because it is typing, and there’s no honest way for a tool like mine to tell the difference.

But I’d like you to sit with what that bypass actually is for a second, because it happens to be one of the oldest ways of learning to program that exists. In the 1980s, computer magazines printed entire programs as long dense pages of BASIC, and a whole generation of kids learned to code by sitting at the family computer and typing them in, line by painstaking line, from paper. One magazine even printed a little helper program that would check each line as you entered it and show you a short code to compare against the one printed next to that line in the listing, so you’d know the moment you’d made a typo; which is to say, a hash check on student keystrokes, forty years before I got around to writing one.¹¹ And decades later, Learn Python the Hard Way built its entire method around the same single instruction: type every exercise in yourself, and never, ever copy and paste.

The Automatic Proofreader, from page 121 of the October 1985 COMPUTE!’s Gazette, courtesy of the Internet Archive. Read the instructions closely: you type the Proofreader itself in by hand, and then the very first thing it does is check itself, adding up its own DATA statements and refusing to start if you got one wrong. My Neovim recorder checks itself in more or less the same way, by hashing its own source, and I had no idea anybody had done it first until I found this page.

So the cheapest available way to fool my software involves reading every single character of the answer, in order, and putting each one in with your own fingers. Which, if I’m being honest, is more attention than a lot of people pay to code they wrote themselves.

I’m not saying retyping is learning, exactly, or that it’s allowed. It isn’t, if your course says it isn’t. And I know better than most that doing something with your hands is not the same as understanding it: I could solve a Rubik’s cube in under thirty seconds when I was eleven, and I could not have told you why a single one of the algorithms I’d memorized actually worked. But I think it shows what we were really worried about all along, which was never whether a machine produced the characters, but whether the student had actually engaged with them.

---

There’s an idea that has been sitting in my notes for Provenance for a while now, and it’s the closest thing I have to a thesis for this entire post.

Two students submit identical working code. One of them took forty minutes: read the spec, wrote a test, watched it fail, fixed the exact line the error message pointed at. The other took four hours of more or less random changes, one after another, until the tests finally went green. They get the same grade. Everything a teacher could possibly want to know about those two students is invisible in what they turned in, and completely present in how they got there.

And that was always true. It was true when the answer came from an older cousin, or from a friend’s repository from last semester, or from a website I’m not going to name. The gap between the thing you hand in and the thing you actually understand is as old as the autograder; I spent my entire first year at Berkeley letting the words “All Public Tests Passed” decide how my evenings went, and not once in that year did they tell me whether I’d understood anything.

AI didn’t create that gap. It just made it impossible to ignore, at scale, in every course, all at the same time. And honestly, I think that’s a good thing. An assessment that only works as long as nobody can get hold of the answer was broken the whole time; we just had enough friction in the way not to notice. Now we actually have to measure what we meant to measure.

Which, finally, is my real answer to the question of which side I’m on, because Provenance isn’t anti-AI either. There’s no classifier in it that looks at a student’s code and guesses whether a model wrote it, and that’s entirely deliberate. (Those classifiers are unreliable, and “the detector said so” is not a sentence I would ever want to see as the evidence in somebody’s integrity hearing.) If a student has Copilot or Cursor running in their editor, it shows up on their submission as information, and it does not move them one inch up the review queue.¹² Our student FAQ puts it better than I can: “Recorded isn’t flagged, and flagged isn’t in trouble.” Whether a given use of AI is allowed is the course’s call to make. The log just says what happened.

From the student FAQ, which is public, and which is the only part of the entire project that most students will ever actually read, so it’s the part I was most careful with.

And yes, Provenance was built with agents, the way almost everything I make is built now, which turned out to be useful in a way I genuinely did not expect. I spent five months staring at what an agent’s edits look like inside an editor, from the inside, because I was producing them all day long. For a while, an edit applied by Claude Code showed up in Provenance’s own recording as ordinary typing, which is an extremely funny bug to find in an academic-integrity tool, and a considerably funnier one to find in an academic-integrity tool that was written in large part with a combination of Cursor and, you guessed it, Claude Code. (It’s fixed now. I found it in exactly the way you would expect.)

I should be honest that right now, for most students, Provenance is something they put up with because their course requires it. The students I’ve talked to have been kinder about it than I’d feared (the thing I hear most often is that they forget it’s even there), but hardly any of them had ever considered that the same record could clear them if their work was ever questioned, which is, to me, the most important thing it does. If you did the work, the record is the best evidence you have that you did.

And there’s a version of it I want far more than the one that exists today. That four-hour student, the one shuffling code around until it turns green, is going to fall off a cliff in about three weeks. They’re going to show up at office hours, and a TA is going to ask them what they’ve tried so far, and the answer is going to be everything. The same history that exists for integrity could answer that question precisely, in the moment, for a student who is already asking for help. Or, as I put it in my notes at some point well past two in the morning: nobody forges a log to get better help at office hours.

---

So, which side am I on?

I don’t think I’m on two sides at all. Both halves of my week come out of the same belief, which is that using AI is completely fine and lying about how you used it isn’t. And the skill that’s going to matter most, for students and for everyone who used to be one, is the skill 61A has been quietly testing all along: given code you didn’t write, can you say what it’s going to do before you run it?

I still use all five keyboards, for the record. (Well, four. One of them has a stabilizer rattle I’ve been meaning to fix since March.) But what I type on them has changed. Less code, more of everything around the code: what it should do, what it must never do, what I expect it to display before I let it run. They sound exactly the same as they always did.

If I have to pick a side, I’ll pick the little ? in the 61A terminal, the one that won’t let you run anything until you’ve typed in what you think is going to happen. It has never asked who wrote the code.

It just wants to know if you can read it.

1. For the uninitiated: a mechanical keyboard has a separate physical switch under every key, as opposed to the thin rubber membrane in most laptop keyboards, and there is an entire hobby (and an entire economy, and an entire vocabulary, and several very heated subreddits) devoted to which switches feel best, how much force they take, what they sound like, and how to make them sound better. “Thock” is a real technical term in this community. I’m sorry to be the one to tell you. ↩

2. In practice this means running events and workshops, connecting student orgs and hackathons with SpaceXAI’s resources, and telling a great many people about Grok. I’d like it on the record that nobody there asked me to write this, and nobody there has read it, which you could probably have worked out from the number of times I mention Claude. ↩

3. To be extremely clear: vendored, not hand-rolled. You should never write your own cryptography, and you should especially never write your own cryptography in Lua, and the fact that I needed cryptography in Lua at all is a pretty good illustration of the kind of problem that only exists once you’ve decided to support a few dozen people that nobody else was going to support. ↩

4. It was for the CAIE (Cambridge International) exams, whose past papers were the best way to study and also, at the time, incredibly tedious to actually collect. It has eleven stars on GitHub. Provenance, which runs for thousands of students across four courses, has eight. ↩

5. Of the ones that aren’t mine, two are authored by “Cursor Agent,” which is what happens when you let a cloud agent push code on its own, and one is the summer instructor for CS61A fixing a typo in my README. I’ll note that the commit log credits me for every single line in the other fourteen hundred-odd commits, regardless of whose hands (or whose tokens) actually typed them. A finished piece of code simply doesn’t carry any record of where it came from. That’s more or less the reason Provenance exists in the first place, and it’s a little funny that its own history has the same problem. ↩

6. In the interest of honesty: this particular semester, in 61A at least, that handful is closer to zero. The course is also running Preceptor, an AI tutor built to work inside VS Code, and since it only lives there for now, everybody in the course is in VS Code too, Neovim people included. I don’t blame the Preceptor team for that at all. They’re building something genuinely hard, and when you’re building something hard, you build it for the editor most students already use and worry about everybody else later, which is a completely reasonable call that I’d probably have made in their position. (Provenance only got its other two recorders because I was stubborn and had agents.) But it does mean that the Neovim recorder, the thing I’m proudest of in the whole project, is currently sitting on the bench in its own home course. I really hope that changes, and that sometime soon the people who’ve spent years tuning their editor configs get to bring them back. ↩

7. The short version: a helper that converted a cursor position into a character offset was re-splitting the entire file on every single call, inside a loop that ran once for every edit, in three separate files that all had to stay in perfect lockstep. Fixing it made one stage of ingest roughly four hundred times faster, which I am still a little smug about. ↩

8. I wrote a whole essay last month about how rarely anybody hands you a genuine failing test case about yourself. Software is the only place I know of where you can write your own, on demand, as many as you like, and an agent is the first collaborator I’ve ever had that reads every single one of them without getting even slightly defensive about it. ↩

9. For non-Berkeley readers: 61A’s assignments are checked by a little command-line program called ok, and you unlock a question’s tests with a flag before you’re allowed to run them. The example above is my own, but the format is faithful, right down to the faintly smug -- OK! -- at the end. I have typed a great many answers into that prompt over the years, and I would estimate that a respectable fraction of them were wrong on the first try. ↩

10. I want to stress that these are conversations, not research: in a hallway, after a demo, or with a friend who happens to be in one of the courses. Nobody should go and cite any of this as a finding. I ask partly out of curiosity and partly because I would much, much rather hear about a hole from a student than find out about it later from a log. ↩

11. The magazine pictured is COMPUTE!’s Gazette, the Commodore-only sibling of COMPUTE!. Every line of every listing ended in a little comment like :rem 123, which you were explicitly told not to type; it was there purely so you could compare it with the number the Proofreader flashed up on your screen after you pressed RETURN. Which means the kids of 1985 were doing tamper-evidence on their own keystrokes, against a paper reference, for fun, on a Commodore 64. I’ve spent five months building a much more complicated version of the same idea. ↩

12. Specifically, a list of AI extensions (Copilot, Codeium, Continue, Cursor, Cody, Amazon Q, and a few others) gets checked against what’s running in the editor, and a match is logged at “info” severity, which the code itself describes as “informational, not by itself an integrity flag.” It’s there so that a course which does allow those tools can see that a student was using them, and so that a course which doesn’t can have an honest conversation instead of a suspicious one. ↩