Does code quality still matter? by Mark Seemann
I honestly don't know, but I guess we'll find out.
You can't open a social-media feed or listen to a podcast without being bombarded with stories of how LLMs write code faster than programmers. Not only for greenfield development, but also in already-established code bases.
As long as you use LLMs as a tool to generate code that humans are ultimately responsible for, the quality of the code still matters. Humans need to review that code, work with it, fix its bugs, etc. But what happens if humans are no longer in the loop?
I recently came across this tweet:
"What if we let people YOLO the code base without LLMs (coding by hand)?
"No? Bad idea?
"Then why is it okay with LLMs? Why does agentic code get a pass, but human code doesn't?"
I believe that Tim Ottinger has a slightly different valid point to make, but it comes close enough to something I've been mulling over for a few years: If, in the future, LLMs write all the code, does code quality matter?
Why code quality matters #
I suppose that 'to YOLO a code base' means writing code with no eye to code quality, a practice that, frankly, has been predominant for decades. Still, development organizations that 'let people YOLO the code base' eventually find that they have a problem. They have accumulated so much technical debt that they can no longer respond timely to business demands.
To be clear, code quality isn't the same as software quality. Code quality is an intrinsic property of a code base: How the code is structured, how readable it is, how easy it is to change.
Until now, all of this has ultimately been reducible to questions about human cognition. That's the point of my book Code That Fits in Your Head. Humans wrote the code. Humans had to edit it.
What if this is no longer the case?
Programming languages for LLMs #
Consider, as a thought experiment, a future in which humans no longer write or read code. Does code quality still matter?
It seems evident that if humans no longer work with the code, then all the constraints related to human cognition become irrelevant. The resulting code might have long methods, high cyclomatic complexity, obscure variable names, tight coupling.
For all we know, some of those notions may not even be meaningful. The concept of a long method assumes that the software is encoded in a way that even has methods.
Spoiler alert: Machine code doesn't.
Realistically, I don't expect LLMs to directly generate machine code, if for no other reason than that machine code isn't portable.
On the other hand, there's no reason to believe that, in this hypothesized future, LLMs would stick to the languages that exist today. With the exception of machine code, all programming languages exist to make programming easier for humans. If no humans look at code, it's conceivable that new languages will emerge; languages optimized for LLMs to consume, generate, and manipulate.
For the sake of argument, let's assume that only one such language emerges. Let's call it LLaMe.
Technical debt for LLMs #
Future LLMs write code in LLaMe. Do they accrue technical debt?
Honestly, I don't know. Perhaps this will never become an issue. If so, the rest of this article is irrelevant. We just let the LLMs YOLO LLaMe code from here to eternity, and it never becomes a problem.
Still, I find it conceivable that technical debt could still accumulate. Even with machine code, you have choices in how to structure the code. Do you reuse a register, or use two different registers for an operation?
For that reason, we must assume that LLaMe allows alternative code structures. Some of those structures may be harder to change than others. Some may require more resources (tokens?) for LLMs to 'understand'. Well-structured LLaMe code will enable LLMs to add features and fix bugs quickly and inexpensively. Badly-structured LLaMe code will take more time, and be more expensive, to improve.
Can LLMs prevent technical debt? #
What does bad LLaMe code look like? We don't even know what LLaMe looks like, and we also don't know what patterns and idioms will prove beneficial, and which to avoid.
We humans have accumulated experience over generations. We know a thing or two about what works and what doesn't. Avoid long methods. Use descriptive names. Beware of coupling. All that experience, however, is deeply entangled with constraints of human cognition.
LLMs will not be able to learn from our experience. All our books, all our conference talks, all our podcasts, all our blog posts are about human problems. They are unlikely to apply to avoiding technical debt in LLaMe.
LLMs will have to learn from their own experiences.
Acceleration #
Do LLMs currently learn from their own experiences? I don't know, but I suppose they could. They could write incidence reports and think pieces for other LLMs.
They can do that much faster than humans, and I suppose it's possible to train a next generation of coding LLMs on the experiences learned by a previous generation. This may happen much quicker than humans absorb similar knowledge.
So perhaps LLaMe technical debt will be an issue for a few years, but then that problem will be solved too. LLMs will not YOLO their code, but follow appropriate LLaMe coding practices.
Conclusion #
Code quality matters to human programmers because of cognitive constraints. LLMs are unlikely to constrained by the same limitations as ours, but they may have their own. Such machine constraints could give rise to technical debt that slow LLMs down when developing and improving software.
Perhaps this is a problem that never materializes. Or perhaps it's a problem that will appear for a while, and then be solved. Or perhaps it never will.
If this happens, we may have a problem on hand that we can't solve. The limitations are so beyond human cognition that we have no chance to troubleshoot.
If nothing else, this thought experiment suggests to me that letting LLMs produce software without human oversight is going to be more complicated than most of the AI optimists let on.
