Does code quality still matter? by Mark Seemann
I honestly don't know, but I guess we'll find out.
You can't open a social-media feed or listen to a podcast without being bombarded with stories of how LLMs write code faster than programmers. Not only for greenfield development, but also in already-established code bases.
As long as you use LLMs as a tool to generate code that humans are ultimately responsible for, the quality of the code still matters. Humans need to review that code, work with it, fix its bugs, etc. But what happens if humans are no longer in the loop?
I recently came across this tweet:
"What if we let people YOLO the code base without LLMs (coding by hand)?
"No? Bad idea?
"Then why is it okay with LLMs? Why does agentic code get a pass, but human code doesn't?"
I believe that Tim Ottinger has a slightly different valid point to make, but it comes close enough to something I've been mulling over for a few years: If, in the future, LLMs write all the code, does code quality matter?
Why code quality matters #
I suppose that 'to YOLO a code base' means writing code with no eye to code quality, a practice that, frankly, has been predominant for decades. Still, development organizations that 'let people YOLO the code base' eventually find that they have a problem. They have accumulated so much technical debt that they can no longer respond timely to business demands.
To be clear, code quality isn't the same as software quality. Code quality is an intrinsic property of a code base: How the code is structured, how readable it is, how easy it is to change.
Until now, all of this has ultimately been reducible to questions about human cognition. That's the point of my book Code That Fits in Your Head. Humans wrote the code. Humans had to edit it.
What if this is no longer the case?
Programming languages for LLMs #
Consider, as a thought experiment, a future in which humans no longer write or read code. Does code quality still matter?
It seems evident that if humans no longer work with the code, then all the constraints related to human cognition become irrelevant. The resulting code might have long methods, high cyclomatic complexity, obscure variable names, tight coupling.
For all we know, some of those notions may not even be meaningful. The concept of a long method assumes that the software is encoded in a way that even has methods.
Spoiler alert: Machine code doesn't.
Realistically, I don't expect LLMs to directly generate machine code, if for no other reason than that machine code isn't portable.
On the other hand, there's no reason to believe that, in this hypothesized future, LLMs would stick to the languages that exist today. With the exception of machine code, all programming languages exist to make programming easier for humans. If no humans look at code, it's conceivable that new languages will emerge; languages optimized for LLMs to consume, generate, and manipulate.
For the sake of argument, let's assume that only one such language emerges. Let's call it LLaMe.
Technical debt for LLMs #
Future LLMs write code in LLaMe. Do they accrue technical debt?
Honestly, I don't know. Perhaps this will never become an issue. If so, the rest of this article is irrelevant. We just let the LLMs YOLO LLaMe code from here to eternity, and it never becomes a problem.
Still, I find it conceivable that technical debt could still accumulate. Even with machine code, you have choices in how to structure the code. Do you reuse a register, or use two different registers for an operation?
For that reason, we must assume that LLaMe allows alternative code structures. Some of those structures may be harder to change than others. Some may require more resources (tokens?) for LLMs to 'understand'. Well-structured LLaMe code will enable LLMs to add features and fix bugs quickly and inexpensively. Badly-structured LLaMe code will take more time, and be more expensive, to improve.
Can LLMs prevent technical debt? #
What does bad LLaMe code look like? We don't even know what LLaMe looks like, and we also don't know what patterns and idioms will prove beneficial, and which to avoid.
We humans have accumulated experience over generations. We know a thing or two about what works and what doesn't. Avoid long methods. Use descriptive names. Beware of coupling. All that experience, however, is deeply entangled with constraints of human cognition.
LLMs will not be able to learn from our experience. All our books, all our conference talks, all our podcasts, all our blog posts are about human problems. They are unlikely to apply to avoiding technical debt in LLaMe.
LLMs will have to learn from their own experiences.
Acceleration #
Do LLMs currently learn from their own experiences? I don't know, but I suppose they could. They could write incidence reports and think pieces for other LLMs.
They can do that much faster than humans, and I suppose it's possible to train a next generation of coding LLMs on the experiences learned by a previous generation. This may happen much quicker than humans absorb similar knowledge.
So perhaps LLaMe technical debt will be an issue for a few years, but then that problem will be solved too. LLMs will not YOLO their code, but follow appropriate LLaMe coding practices.
Conclusion #
Code quality matters to human programmers because of cognitive constraints. LLMs are unlikely to constrained by the same limitations as ours, but they may have their own. Such machine constraints could give rise to technical debt that slow LLMs down when developing and improving software.
Perhaps this is a problem that never materializes. Or perhaps it's a problem that will appear for a while, and then be solved. Or perhaps it never will.
If this happens, we may have a problem on hand that we can't solve. The limitations are so beyond human cognition that we have no chance to troubleshoot.
If nothing else, this thought experiment suggests to me that letting LLMs produce software without human oversight is going to be more complicated than most of the AI optimists let on.
Comments
LLMs are good at following patterns. Having a project template with one example (I'm talking web here) is usually enough to turn "A.I." into faster auto-complete method.
We can still handle the structure/architecture part, while letting the machines do the labour of (mostly) boilerplate code. Majority of code is boring anyway CRUD which I no longer have to deal with, except writing tests.
This leaves us with more time to deal with actual interesting problems, and I think that's a good thing.
It's an interesting thought, but seems unlikely to me. The way LLMs (currently?) get trained is by throwing a huge amount of data at them. That means to train LLMs in LLaMe would require a large amount of good, working LLaMe code. So the question becomes: where is that code supposed to come from? Having people write it by hand seems unlikely as the language was designed to be "written" and "read" by LLMs. Transpiling from another language might work, but then it's unclear if it produces the best possible LLaMe code.
They also don't seem to be able to learn by themselves. The big LLM providers are already struggling with the sloppification of their own models, because their output is flooding the internet and therefor "polluting" the data they used for training so far. There is even a company selling a list of books that were not created with LLMs as a source of "good" training data. It seems to me that LLMs need a constant stream of external, non-LLM generated data to change without becoming worse and worse. I guess you could imagine an additional system that produces large amounts of training data, then different versions of the LLM get trained on subsets of that data. Afterwards the output of these versions is compared against each other to see which subset of data actually improved it. Considering how incredibly expensive model training is, I can't see that happening, either (and hope it won't happen).
Marlemie, thank you for writing. You may be right that LLaMe code is unlikely. Let's pursue that thought.
In that scenario, LLMs will be 'best' at producing more code in C, C++, Java, C#, JavaScript, and Python. Will code quality matter if LLMs mostly produce, say, Python code?
What humans consider good coding practices for Python may not be of any use to an LLM. It may even turn out that some of our software engineering principles keep LLMs from operating at their full potential: If we let them loose, they might produce software of a better quality faster, backed by a code base that LLMs can modify faster than if it had to follow human practices.
On the other hand, what we consider bad code may not be a problem for LLMs. Take, as an example, code duplication. The reason that duplication is bad is because the human brain can't search through an entire code base to enumerate all cases of duplication. A computer, on the other hand, may be able to do so effectively. So even if LLM-generated Python code has lots of duplication, it may not be an issue if LLMs reliably can find all duplicates and modify them uniformly.
I'm aware of all sorts of reasons why this might be easier said than done. I mostly use that particular example because it's easy to explain.
As to whether LLMs are able to work effectively with new languages, I wouldn't entirely write off that idea. As I wrote in Programming languages for AI there are already people experimenting with LLM-first languages.
In general, 'reasoning' capabilities of LLMs seem to be improving, so if presented with a new programming language defined by a formal grammar, I would expect LLMs to be able to generate (small) working programs.
It's possible to define Turing-complete programming languages using mathematical notation (e.g. context-free grammars). The simpler the specification, however, the more awkward the language is to use: Hard to write and hard to read.
Mainstream programming languages come with various features that make them useful to humans. Such features tend to make it difficult to formally specify the language in question.
If LLMs don't need 'ergonomic' language features, it would be possible to define a programming language as a formal system that LLMs can follow based on their 'reasoning' capabilities. I believe this already works 'in the small'; it's possible that this could work for larger software systems in a few years, even without training data.
To be clear, though, this is mostly idle speculation.