ploeh blog danish software design

Open answers to a reader's letter.

A reader recently wrote me a long letter with lots of questions about learning programming in this age of LLMs. After a bit of back-and-forth, I got permission to quote extensively from the letter in order to attempt some answers in public.

None of my answers I consider particularly rigorous; the situation is so uncertain that I can only answer to the best of my abilities, but I don't claim them to hold any kind of immutable truth.

"I'm trying to understand how people who deeply understand software think about learning and competence in the age of AI. I'm approaching it almost as a historian would: asking people directly how they make sense of a technological transition while actually living through it.

"About a year ago I became fascinated by AI-assisted programming. Despite having no formal CS background, with LLMs I managed to build a fairly large TypeScript/JavaScript system involving APIs, PostgreSQL, LLM pipelines, research automation and multi-model workflows. At first it felt almost magical: AI seemed to collapse the distance between having an idea and being able to build it.

"But now I'm trying to turn that system into a real production product, and I'm struggling. I fix one error with AI, then another appears, then another part behaves in a way I don't fully understand. After months of refactoring I had an uncomfortable realization: I may have built a system that is above my own level of understanding. When everything works, that gap is almost invisible. When it doesn't, it becomes very real.

"Sometimes I genuinely don't know what to do next without asking another model. That made me wonder whether I spent a year building a product, or partly building the appearance of one: something sophisticated enough to work, but which I don't yet understand deeply enough to truly own.

"I'm not anti-AI at all. I'm fascinated by these systems and want to work with them professionally. But I'm unsure what the right relationship with them should be."

Indeed, I'm not sure either, but before proceeding, I find it most transparent to reveal my position. I haven't yet decided on AI, but I lean toward disliking it, knowing full well that it may be unstoppable.

I do work and experiment with it, and it often impresses me. At other times, it frustrates me. It's usually when it impresses me the most that I resent it maximally.

When it's bad, it can be frustrating, but then at least I can absorb an ember of warmth in the illusion that what I've spent more than thirty years learning is still relevant. When it's at its best, I sometimes think: Where do I sign up for the Butlerian jihad?

My position on LLMs is only partly based on my own socio-economic status. I'm old enough, and have had enough success already, that all other things being equal, I can survive unemployment. I'm not sure, on the other hand, than any knowledge-based society can.

It may be that LLMs will take programmer jobs before they take other white-collar jobs. After all, programming may be a discipline where verification is easier than, say, insurance claims management. Still, if we reach a point of mass unemployment among knowledge workers, I'm not sure society as we know it will survive.

I usually don't talk much about my background as an economist, but in this context I find it relevant to mention. As an economist, I can't imagine that mass unemployment of 30-40% will not have a significant impact on the economy.

I'm painfully aware of the arguments that this has happened before: There may be job loss, but the advance of technology leads to new jobs we can't even imagine today. It was like that with the introduction of the stocking frame, the steam engine, the internal combustion engine, computers, etc. This is only partly true: Yes, new jobs were created, but often not for those people who lost their jobs. Coal miners didn't just become programmers overnight.

The same kind of argument was used when China was admitted to the World Trade Organization. And indeed, lots of new jobs were created, just not in the Western world.

So, based on lived and historical experience, I'm sceptical of arguments that all will be fine.

But I sincerely hope that I'm wrong. I love to program, and wouldn't mind doing it for another ten years. Perhaps more importantly, I have young adult children. I hope that there's a world for them, too.

"So I'd really like to know how you think about this. Are you glad you learned programming fundamentals before LLMs existed? If you were starting today, would you still seriously study languages, data structures, databases, networking, operating systems, debugging and architecture? Do you think AI can let people become capable of building much faster than they become capable of understanding?"

Am I glad that I learned programming before LLMs? Yes, of course. Those skills served me well for thirty years.

If I was starting today, I'd seriously consider learning carpentry, metalworking, gun-smithing, or something else that requires hand-eye coordination. I know that advances are made in robotics, too, but replacement of manual labour seems to lie farther in the future.

But to address the question: I am, personally, currently learning data structures, language semantics, etc. as part of a university programme. I do that because I'm curious, however, and not because I expect to get much monetary reward out of it.

Do I think that AI enables people to develop faster than they can keep up? This remains to be seen. Software developers have already, for decades, been working on top of abstractions they didn't understand. If you were a web developer, you didn't know much about compiler programming. If you were a compiler programmer, you didn't know much about integrated circuit design. And if your job was to engineer integrated circuits, you wouldn't know much about the levels of abstraction above you.

A good rule of thumb was: Understand the level of abstractions directly below the one you work in, as well as the one above. That would enable you to troubleshoot most problems.

"And how do you personally deal with that? When AI can solve something immediately, how do you decide when to use it and when to work through the problem yourself? If you were in my position, with a substantial AI-built project but weak foundations underneath it, would you step back and systematically learn those foundations, keep building and learn as problems appear, or combine the two?"

That's two radically different questions, because I no longer have a weak foundation in software development. Even if I were dealing with something far from what I usually do, I can ramp up leveraging what I already know. Let's imagine that someone tasked me with maintaining an application written exclusively in RISC-V assembly code. That's the most alien software environment I can imagine for myself. Adapting to such a development environment would be difficult for me, but still not as difficult as it would be for someone new to programming in general. Believe it or not, I have written small exercise programs in RISC-V, as well as an exercise compiler that compiled to RISC-V.

But what if I had virtually no software background?

Well, once upon a time, I was in exactly that situation. When I started my career, for years I balanced a knife's edge of getting things done while learning on the job. Beginning in 1999, I wrote COM components in C++, not understanding much of what I was doing. Somehow, I still made it work, even to a degree that I managed to eliminate any obvious memory leaks.

I was, however, never happy just slapping things together without understanding how they worked. So I did, as suggested by the question, step back to systematically learn fundamentals. This worked well for a career launched in the mid 1990s. Will it work well today?

I'm not so sure: Reaching a level of competency high enough to recognize your past confidence as clearly lying on the too-ignorant-to-realize-it portion of the Dunning-Kruger curve took decades. Do you have that much time today?

Granted, with LLMs, you can learn faster, because you can ask more directed questions. Thirty years ago, I would buy books in the hope that they would contain some helpful material. This still meant slogging through a lot of learning material not immediately relevant to the task at hand.

Still, I doubt that it's possible to significantly speed up human learning. The bottleneck is hardly the teachers nor the materials, but how fast a human brain can absorb new knowledge.

"One last thing I would be especially grateful to hear about is how you learned programming yourself, and how you learn new technical things today. How did you approach learning a new language earlier in your career? Books, projects, reading other people's code, exercises, debugging, something else? And if you had to learn a completely new programming language today, with AI available, how would you do it?"

The short answer to the first question: Slowly, based on much trial and error, occasionally backed by a book.

Apart from a very early false start with COMAL 80, my first programming projects was to (re)calculate bifurcation diagrams and the Lorenz attractor for my master's thesis in economics. Reaching for what I had, I wrote them in QBasic, learning from the samples that shipped with it, as well as occasionally asking a friend.

While I'm glossing over many details, in the 1990s and 2000s, I mostly learned from examples and documentation. While I did buy a book about C++, I don't think I ever finished it, and I picked up various Basic dialects as well as C# exclusively from documentation and example code.

That said, although I never read a book to learn C#, books were instrumental in teaching me both F# and Haskell. I have, over the years, relied heavily on books to educate myself, but as my Goodreads profile reveals, I love books in general.

How do I learn a completely new programming language today? Again, my experience is useless to someone new to programming in 2026: I've now seen so many programming languages that if I run into a new one, I can usually pick it up from perusing existing code and looking up the few things that aren't immediately clear.

But that's presupposing that the language in question is 'normal'. If I had to get back into APL, I'd at least have to find a tutorial.

You may have noticed that I don't much use LLMs for learning. LLMs don't hallucinate; they bullshit, and I'm deeply distrustful of anything they tell me. This is not to say that I don't use LLMs, but I tend to ask them questions that yield verifiable answers. Can I make this Haskell expression more succinct? Any useful answer to such a question is a code suggestion that either works, or doesn't work; is shorter, or isn't. That's easy to verify.

What should I learn next? does, on the other hand, not yield a verifiable answer. I tend to not to ask such questions of LLMs.

In conclusion, you could say that I prefer asking LLMs falsifiable questions.

C# is a great programming language, but F# is better.

This article is part of a series called Worse is better, in which I muse on technologies and techniques that became popular despite superior alternatives. Think VHS versus Betamax.

In this article, I will argue that F# is superior to C# along most axes of comparison.

Popularity ranking #

It's easier to compare C# and F# than it is to compare, say, Java to Haskell. When comparing most languages, you'll realistically need to take into account not only syntax and expressibility, but also the entire surrounding ecosystem: Build tools, hosting options, frameworks and libraries, knowledge bases, etc. While I admit that I haven't rigorously looked into this, I'd be surprised if no supported Oracle data provider exists for Java, but is there one for Haskell? Even if one exists, how good is it? Is it supported? By whom?

It's easier to compare C# and F# because many such variables are fixed and the same. Both languages run on .NET. If you want to host a web service, you'll have the choice of the same web servers. If you want to connect to an Oracle database, you can use the same ADO.NET provider. Thus, when comparing C# with F#, it's possible to ignore most of the surrounding ecosystem, since it remains constant.

Three .NET languages enjoy support from Microsoft. While it can, in general, be difficult to compare the popularity of programming languages, there's little doubt that the three languages rank in popularity as follows:

1. C#
2. Visual Basic .NET
3. F#

Indeed, F# is not even the second-most popular .NET language; it's the least popular. Could it really be the best of the bunch?

Feature parity and beyond #

If there's a lesson to be learned from this article series, it's that popularity isn't determined by a single factor. In the previous article, I argued that XML is superior to JSON even though JSON is seen as the more modern, lightweight alternative. Ironically, C# is more popular partially because it's the more established, mature alternative.

In all fairness, I also think that C# is more popular than F# because, for some reason, most people find imperative programming easier to learn than declarative programming. I haven't yet been able to determine whether this is something inherent in human cognition, or whether it's an artefact of how most people have their first encounter with programming. I could write much more about this, but it would be a digression.

Although F# is from 2005, it's newer than C#. This means that by design it has always had to match everything that C# could do. Otherwise, the two languages wouldn't be able to interoperate.

Since C# is object-oriented, F# has object-oriented language features too. Since C# is heavily biased towards state mutation, F# supports imperative programming. Since C# supports method overloading, F# does that, too. And so on.

F#, however, never stopped at feature parity. It enables you to do more than C# can do. I've already been over this ground ten years ago, so I will not repeat the entire list of what F# does better. Indeed, as C# catches up to F#, the gap becomes narrower.

Even so, I think one may reasonably illustrate the situation in a Venn diagram:

As is usually the case when I draw a Venn diagram, it's not to scale, and in fact, what it depicts can't even be measured. Rather, I'm trying to convey a conceptual relationship. In this diagram, each set represents a set of features. The intersection is larger than any of the set differences, indicating that most features are shared by both languages. In fact, had the diagram been 'to scale', the intersection would be so much greater than the relative complements that the latter would be hardly visible. That is, if we assume that it somehow makes sense to talk about features as something measurable.

The point of the skewed figure is that the set difference F#-C# is 'greater than' C#-F#. In the latter, we find 'small' features such as unsafe as well as some tools that I discuss below. In the larger F#-C# set difference, we not only find some of those language features that C# is soon to get, but also computation expressions, strong type inference, DRY variable scoping, collection types with structural equality, and language-level dependency management.

A programming language for software architects #

Even though C# seems to be catching up, I'd be surprised if it ever gets one of F#'s killer features: That it by default disallows circular dependencies. This language feature is often seen as an annoyance, but in reality elevates F# to a language that has built-in software-architecture support. Thick books have been written on dependency management. A major reason software architects decompose applications into modules is to enforce dependency graphs: At the module level, cyclic dependencies are disallowed. If library A depends on library B, you can't (easily) reference library A from library B.

This style of architecture is so ubiquitous that we don't even notice it, but if you think about it, it's quite unnatural. Junior developers don't understand dependency management, and seniors have to be constantly vigilant to prevent code rot. Decomposing a code base into modules is a coarse-grained attempt at keeping chaos at bay.

F# enables this kind of control at the language level. To a software architect, this feature alone is priceless.

What C# does better #

If someone asks for my help in developing a greenfield .NET application, I would choose F# by default. That said, I'm aware of forces that push toward C#. The most common concern is about being able to find F# programmers. In general, that ought not be a problem, although I'm aware that regional differences may have an effect.

Besides the question of finding qualified programmers, C# has an edge over F# in various smaller areas.

First, and perhaps easiest to spot, a few .NET GUI technologies assume C#: ASP.NET Razor assumes C# or VB.NET, but as far as I know, doesn't work with F#. This is, in a way, a genuine limitation, but can be overcome in various ways: Have a thin UI layer in C#, or use an alternative rendering framework that works with F#.

Second, many C# developers will be underwhelmed by the IDE support for F#. A few of Visual Studio's code editing features I do genuinely miss for F#, but one could also make the argument that some IDE features for C# are only workarounds for limitations of the language.

Next, there are a few other pieces that I miss when working in F#. One is that last time I checked, there's no easy way to calculate cyclomatic complexity for F# code bases. While this is not a big deal, I do occasionally find cyclomatic complexity useful, so it'd be nice to have tool support for that.

The same goes for code coverage, and while code coverage is a useless target measure, it may still be occasionally useful for trend analysis and sanity checks.

Perhaps the most uncomfortable lacuna is the state of mutation testing for F#. Last time I worked with mutation testing in .NET I found no F# support, but it looks as though Stryker has at least rudimentary F# support. I should look into that in the future.

To flip a previous argument on its head: Just as C# is catching up to F# when it comes to most language features, it's conceivable that the tool-support situation could improve for F#.

Object-orientation and functional programming #

Notice that none of the above arguments hinge on comparing object-oriented programming (OOP) with functional programming (FP). F# is a 'functional-first' language, meaning that it's really a multi-paradigmatic language biased toward FP. Conversely, C# is a multi-paradigmatic language heavily biased toward OOP.

I could, and probably will, write an entire article in this worse-is-better series arguing that FP is superior to OOP, but I don't think that I need to make that argument here in order to argue that F# is superior to C#.

Conclusion #

C# and F# share most language features. What you can do in C# you can do in F#, with a few inconsequential exceptions. Even when C# gets discriminated unions, on the other hand, there are things you can do in F# that don't translate to C#. One is that the language offers built-in dependency management, effectively letting the compiler help you to avoid spaghetti code. And then there are all the little things.

What I didn't discuss at all is the expressivity offered by computation expressions. I kept it out of this article because I think it deserves an article of its own.

Next: Worse is better: async/await versus syntactic sugar for monadic composition.

Comments

The return of SQL?

It's no secret that I'm no fan of ORMs. Most people, on the other hand, find them indispensable. As one reader commented:

"I can work with raw SQL ofcourse... but the mapping... oh the mapping..."

This seems to capture something essential. When I discuss ORMs, the most common argument in favour seems to revolve around the amount of boilerplate code required to communicate with a relational database. And indeed, it's significant.

As I've argued, however, I'm not convinced that ORMs solve that problem.

But now that LLMs write code, does it even matter?

In addition to my individual reservations, it strikes me that ORMs come with many issues related to query efficiency. The vibe I'm getting from ORM experts is that if you really know a particular ORM, you can fine-tune the queries it makes. There are, however, various pitfalls to avoid: Anti-patterns to eschew, idioms to follow, particular APIs to keep clear of, certain parameter values to explicitly pass, etc.

Which strikes me as ironic, because wasn't the whole promise of ORMs that you could read from and write to a relational database without getting bogged down in the details of SQL?

So instead of fiddling with a temperamental and implicit ORM API, why not write fine-tuned parametrized SQL queries? Or rather, ask an LLM to do that for you, as well as all the boilerplate code.

You should, of course, remind it to avoid SQL injection vulnerabilities.

Comments

var reservations = await ctx.Reservations.ToArrayAsync();
var result = ProcessReservationRequest(reservations, reservationRequest);

if(result.IsValid)
{
  var reservation = await ctx.Reservations.Add(result.Data);
  return Accepted(reservation);
}

else return Rejected(result.Errors);
      

I honestly don't know, but I guess we'll find out.

You can't open a social-media feed or listen to a podcast without being bombarded with stories of how LLMs write code faster than programmers. Not only for greenfield development, but also in already-established code bases.

As long as you use LLMs as a tool to generate code that humans are ultimately responsible for, the quality of the code still matters. Humans need to review that code, work with it, fix its bugs, etc. But what happens if humans are no longer in the loop?

I recently came across this tweet:

"What if we let people YOLO the code base without LLMs (coding by hand)?

"No? Bad idea?

"Then why is it okay with LLMs? Why does agentic code get a pass, but human code doesn't?"

I believe that Tim Ottinger has a slightly different valid point to make, but it comes close enough to something I've been mulling over for a few years: If, in the future, LLMs write all the code, does code quality matter?

Why code quality matters #

I suppose that 'to YOLO a code base' means writing code with no eye to code quality, a practice that, frankly, has been predominant for decades. Still, development organizations that 'let people YOLO the code base' eventually find that they have a problem. They have accumulated so much technical debt that they can no longer respond timely to business demands.

To be clear, code quality isn't the same as software quality. Code quality is an intrinsic property of a code base: How the code is structured, how readable it is, how easy it is to change.

Until now, all of this has ultimately been reducible to questions about human cognition. That's the point of my book Code That Fits in Your Head. Humans wrote the code. Humans had to edit it.

What if this is no longer the case?

Programming languages for LLMs #

Consider, as a thought experiment, a future in which humans no longer write or read code. Does code quality still matter?

It seems evident that if humans no longer work with the code, then all the constraints related to human cognition become irrelevant. The resulting code might have long methods, high cyclomatic complexity, obscure variable names, tight coupling.

For all we know, some of those notions may not even be meaningful. The concept of a long method assumes that the software is encoded in a way that even has methods.

Spoiler alert: Machine code doesn't.

Realistically, I don't expect LLMs to directly generate machine code, if for no other reason than that machine code isn't portable.

On the other hand, there's no reason to believe that, in this hypothesized future, LLMs would stick to the languages that exist today. With the exception of machine code, all programming languages exist to make programming easier for humans. If no humans look at code, it's conceivable that new languages will emerge; languages optimized for LLMs to consume, generate, and manipulate.

For the sake of argument, let's assume that only one such language emerges. Let's call it LLaMe.

Technical debt for LLMs #

Future LLMs write code in LLaMe. Do they accrue technical debt?

Honestly, I don't know. Perhaps this will never become an issue. If so, the rest of this article is irrelevant. We just let the LLMs YOLO LLaMe code from here to eternity, and it never becomes a problem.

Still, I find it conceivable that technical debt could still accumulate. Even with machine code, you have choices in how to structure the code. Do you reuse a register, or use two different registers for an operation?

For that reason, we must assume that LLaMe allows alternative code structures. Some of those structures may be harder to change than others. Some may require more resources (tokens?) for LLMs to 'understand'. Well-structured LLaMe code will enable LLMs to add features and fix bugs quickly and inexpensively. Badly-structured LLaMe code will take more time, and be more expensive, to improve.

Can LLMs prevent technical debt? #

What does bad LLaMe code look like? We don't even know what LLaMe looks like, and we also don't know what patterns and idioms will prove beneficial, and which to avoid.

We humans have accumulated experience over generations. We know a thing or two about what works and what doesn't. Avoid long methods. Use descriptive names. Beware of coupling. All that experience, however, is deeply entangled with constraints of human cognition.

LLMs will not be able to learn from our experience. All our books, all our conference talks, all our podcasts, all our blog posts are about human problems. They are unlikely to apply to avoiding technical debt in LLaMe.

LLMs will have to learn from their own experiences.

Acceleration #

Do LLMs currently learn from their own experiences? I don't know, but I suppose they could. They could write incidence reports and think pieces for other LLMs.

They can do that much faster than humans, and I suppose it's possible to train a next generation of coding LLMs on the experiences learned by a previous generation. This may happen much quicker than humans absorb similar knowledge.

So perhaps LLaMe technical debt will be an issue for a few years, but then that problem will be solved too. LLMs will not YOLO their code, but follow appropriate LLaMe coding practices.

Conclusion #

Code quality matters to human programmers because of cognitive constraints. LLMs are unlikely to constrained by the same limitations as ours, but they may have their own. Such machine constraints could give rise to technical debt that slow LLMs down when developing and improving software.

Perhaps this is a problem that never materializes. Or perhaps it's a problem that will appear for a while, and then be solved. Or perhaps it never will.

If this happens, we may have a problem on hand that we can't solve. The limitations are so beyond human cognition that we have no chance to troubleshoot.

If nothing else, this thought experiment suggests to me that letting LLMs produce software without human oversight is going to be more complicated than most of the AI optimists let on.

Comments

JSON is not a good data-interchange format.

This article is part of a series called Worse is better, in which I muse on technologies and techniques that became popular despite superior alternatives. Think VHS versus Betamax.

In this article, I will argue that XML is superior to JSON in most respects.

Lightweight XML #

Depending on how old you are, I guess that you have one of two reactions. If you started programming around 2015 you may simply ask: "What's XML?" If your programming career reaches further back, your reaction may be one of incredulity: "Oh my God, how can you say that?! Good riddance that SOAP, WS-(death)*, and XSLT are things of the past."

Indeed, and I don't miss them, either.

While that reaction is typical, it confuses cause and effect. SOAP and similar standards weren't cumbersome and overly complex because of XML. They managed to be awkward and enterprisey all by themselves. As a thought experiment, you could define all the payloads and specifications of SOAP as JSON, but it would, ironically, be even more verbose, because you'd have to invent a schema language and so on.

XML doesn't have to be heavy or formal. You may find the informality of JSON an advantage. Just write a document:

{
  "author": "Peter Watts",
  "title": "Blindsight"
}

While this is indeed easy and requires no ceremony, what prevents you from doing the same in XML?

<book>
  <author>Peter Watts</author>
  <title>Blindsight</title>
</book>

You don't have to first define a schema. You don't have to declare a namespace. You don't have to add an XML declaration.

But you can, if you need to. XML allows gradual enhancement. If, sometime later, you find that a formal, machine-readable document specification would be useful, you can use XSD. And yes, I'm aware of JSON Schema; I hope the reader can see the irony that such a thing exists.

Sweet spots #

Like any other technology, XML is not a one-size-fits-all technology. I think the ideal scenario for XML is interoperability. While I'm aware that modern systems handle JSON as well as XML, I would still prefer XML for most data exchange tasks. The main driver for that decision would be the possibility to define document schemas. As Alexis King argues in a slightly different context, the lack of static types or, here, a machine-readable schema, does not entail the absence of a specification. Only, as suggested by Hyrum's law, the contract is implicit.

XML comes with a standard schema language, a standardized way to version documents, a standard query language, a standard for streaming parsers, etcetera. Of course, nothing prevents you from inventing similar technologies for JSON, and I'm sure someone already has. Even so, XML is a more mature format. Why reinvent the wheel?

A less ideal use of XML is for configuration files. I know I've lost that fight, but JSON is not a good format for configuration files. The most obvious problem with JSON is the lack of support for comments. And I know that various tools and editors allow comments in various proprietary formats, but it's not part of the standard. XML, on the other hand, has a standard for comments.

You may argue that XML is less readable than JSON, and I will partially agree, even though with good tools such as syntax colouring I find the difference marginal. The same goes with editor experiences. Most code editors will help you with XML to the same degree that they will help you with JSON. And again, if you work with a document that has a defined schema, the editor can help you more by suggesting and auto-filling elements.

But really, neither XML nor JSON are perfect configuration file formats. I wonder if such a thing even exists.

Where JSON shines #

Where would I choose JSON? To be clear, I would pragmatically choose JSON in lots of cases today, simply because that's the expected format, and having to defend a less popular choice is rarely worth it.

Ironically, the kind of architecture we today call SPAs got started as AJAX, where the X stood for, you guessed it, XML. Even so, the J stands for JavaScript, so it makes more sense to use JavaScript Object Notation (JSON). That's what modern SPAs do, and I would too, particularly if the service in question was a BFF.

Size #

I'd be surprised if you've made it so far and haven't though of size as a factor in favour of JSON. It's true that XML is more verbose than JSON. Depending on the actual schema and payload, the size difference could conceivably be more than a factor of two; compare <die><roll>4</roll></die> (25 characters) to {"roll":4} (10 characters). On the other hand, for other kinds of payloads, the difference might only be a small percentage.

In my experience the size difference doesn't matter that much. Often, other factors also play a role: Network latency for transmission, or block size for storage. And when performance really is a consideration, JSON may be too big, too.

Conclusion #

Although XML is generally and unfairly loathed as an old-fashioned legacy or enterprise technology, in most aspects it's a format superior to the more popular JSON.

XML has a rich ecosystem of mature standards, including a schema language that even supports sum types. While you could reimplement many of these in JSON (which has already been done), why reinvent the wheel?

Next: Worse is better: C# versus F#.

The most popular technology may not be the best.

In 1989 Richard P. Gabriel coined the term Worse is Better, a clearly humorous take on certain tendencies in software development and design as he saw them. I first encountered the term in his book Patterns of Software, and I'm basing the following on that.

In essence, another way to state the epigram is that perfection is the enemy of the good. Gabriel's context was the juxtaposition of Lisp versus C++. Gabriel had founded Lucid, a company that offered a general-purpose, cross-platform Lisp compiler, only to see C++ becoming the de-facto cross-platform programming language. The Lucid people considered Lisp superior, and were casting for an explanation why the inferior product was winning.

It's not my agenda to repeat Gabriel's entire line of reasoning. Rather, I want to highlight other areas of software-development technology where a similar relationship exists: A superior technology or process has lost to a more popular alternative.

Of course, any comparison between two technologies is likely to be subjective. My aim is not to insist that technology x is indisputably superior to the more popular technology y. Rather, my purpose is to give you food for thought. If you are a software architect, or other technical decision-maker, being aware of alternatives is useful. Often, going with the most popular tool is appropriate, but sometimes it may be prudent to at least consider alternatives.

In a series of articles, I will give specific examples. None are intended as rage bait, but rather as food for thought. The purpose is to highlight that the popular choice may have properties that are inferior to a less popular alternative. While I'm aware of the strengths of the popular choice, in this series I will instead highlight the advantages of less popular alternatives.

• Worse is better: JSON versus XML
• Worse is better: C# versus F#
• Worse is better: async/await versus syntactic sugar for monadic composition

When I publish this overview article, the above table of contents will initially be short, but I plan to expand it as I go.

Benefits of the road less travelled #

There are reasons that the most popular technology occupies the top spot. Often, those reasons apply to your case, too. If so, do go with the popular option. Consistently going against the flow is a recipe for misery.

Even so, you may occasionally run into a situation where the less popular alternative offers a benefit that outweighs what the mainstream choice offers. Perhaps the popular technology has properties a, b, c, d, and e, while an alternative is clearly superior when it comes to property f. If f is important enough in a certain context, you may decide to choose the road less travelled. That's a simple cost-benefit analysis.

Another advantages of going off the beaten path is the flexibility and freedom it may entail. In my experience, popular technologies are often associated with strong ecosystems. This is clearly a strength, but can also become a liability. Mainstream ecosystems tend to railroad you into a particular way of doing things. As long as your work matches the idiomatic way of doing things, you may be coasting along fine. On the other hand, if you need to do something exotic, you may find that the mainstream way of doing things makes your work harder. Examples could include using ORMs instead of writing plain SQL, or using Postman instead of curl.

Choosing a technology with a lesser ecosystem may force you to forge your own path. It seems that this could only slow you down, but there's also the chance that it enables you to respond better to unplanned events. It might even give you a trace of antifragility. It may help you escape vendor lock-in.

Conclusion #

A technology, a programming language, a protocol, or a tool is often popular because it's better than most alternatives. That said, software development is such a multifaceted endeavour that it's absurd to talk about a single best option. Even assuming that you can quantify every aspect of developing software, the notion of anything being the best assumes a total order, which is unreasonable in such a multidimensional space.

Thus, even if a technology is generally 'better' along commonly-considered axes, it's likely that some other technology is better for specific tasks.

Even so, some popular technologies are popular for reasons that aren't entirely well-considered. Or, for one overwhelming reason that, once you start to think rationally about it, shouldn't be such a big deal as it seems to be. Gabriel thought that C++ was inferior to Lisp, but essentially won in the marketplace because it was possible to release and support an inconsistent language incrementally, whereas, if I understand the story right, Lisp was more an all-or-nothing proposition that never really came to fruition.

In the following articles, I'll discuss some technologies that are generally worse than their less popular alternatives.

Next: Worse is better: JSON versus XML.

Most of my concerns about AI are probably irrelevant, but what if one of them is not?

At the intersection of psychology, neuroscience, epistemology, and political science, there's a concept called motivated reasoning. In short, it describes the tendency to arrive at desired conclusions by reasoning processes heavily influenced by individual motivations. An example is a person who finds reasons to keep smoking that convinces him- or herself: Perhaps the smoker gloms onto evidence that smoking reduces appetite, and might therefore reason that it's better to keep smoking, because quitting would entail a weight gain, which is unhealthy.

As the example demonstrates, the reasoning process may not be particularly rigorous. While it may convince the person doing the reasoning, it convinces few other people.

The process is often subconscious. The person doing the reasoning may not be aware of the predilection leading to a desirable outcome. We all engage in motivated reasoning, so it's a kind of cognitive bias. Furthermore, it seems as though the more intelligent you are, the more susceptible to motivated reasoning you are. Apparently, the mechanism is that smarter people's superior mental resources enable them to find more convincing reasons for reaching desirable conclusions than less gifted persons.

Reasoning about the future of AI #

I have recently posted a series of articles critical of using AI for software development. These articles accept the capabilities of current LLM-based systems, but outline various concerns related to safety, epistemology, correctness, and similar areas.

While writing these articles, I've been aware that I'm likely to be engaging in motivated reasoning. Things are probably going to proceed at breakneck speed. Most of my concerns are probably moot while, on the other hand, I predict that we'll encounter problems that I didn't foresee.

Even so, if ninety percent of my concerns are irrelevant, that still leaves one that may turn out to be a real problem. Which one might it be?

Love of the craft #

Like many other software developers, I mourn our craft. I was originally drawn to programming because I was attracted to this particular kind of problem solving. It was like getting paid to solve puzzles, crosswords, sudokus, or whatever else you may be into.

During the decades of my career, I found that everything was interesting when framed as a programming problem. On the other hand, I've never been intrinsically interested in 'optimizing ad revenue', 'creating a marketplace for offal and other animal-processing waste products', 'implementing a complaint ticketing system', or 'enabling speculators to turn a profit from high-volume trading'. Some of these, I've actually done, and it was engaging work, but only because solving technical problems was stimulating.

When told that I can still 'solve business problems' by becoming a manager of agentic LLMs, that doesn't get my blood pumping. If I had found that prospect interesting, I would have become a manager decades ago.

I like writing code; not telling other entities to write code.

Incentives #

Of course, we were never paid based on whether we enjoyed the work. Rather, we were paid despite of it.

Usually, if you enjoy an activity, it's a hobby, and you pay to do it. Conversely, a job is an activity unpleasant enough that someone is willing to pay you to do it.

The last few decades of the software development job market is most likely abnormal. With some more help from motivated reasoning, I can think of a few reasons why this situation may last a bit longer, but again, I could be wrong.

In case, however, you think I'm incentivised by economics: To a small degree, I am. I'd definitely like to secure my finances better, but on the other hand, I'm getting near to what in some countries counts as retirement age, and I've had a good run so far. I'll get by, so that's not my main motivation.

I can, of course, keep programming as a hobby, but I do think that my services might still be valuable to some organizations. If so, please consider engaging with me.

Conclusion #

As my recent writings bear witness, I'm concerned about the current use of LLMs in software development. My concerns are related to safety and correctness.

Even so, it's possible that I'm engaging in motivated reasoning, a kind of cognitive bias where you arrive at conclusions beneficial to yourself. I still think that there's value in posting opinions that may act as counterpoints to techno-optimism.

I could be wrong about quite a few of my concerns, but still be right about one or two. If so, which ones?

Objectivity, meritocracy, high bus factor. What's not to like?

Most people dislike bureaucracy, and to be honest, so do I. Even so, a subjective distaste is no argument. It may be wise to understand if bureaucracy entails any benefits, so as to not throw out the baby with the bathwater.

This essay does not attempt to make the point that more bureaucracy is better. Rather, it argues that some bureaucracy, wisely chosen, is better than no bureaucracy.

Origins of bureaucracy #

Consider the origins of bureaucracy. Believe it or not, it's an institution designed to get things done; to inject a degree of predictability and determinism into a situation that otherwise produces arbitrary results. While I'm no historian, I understand that the Sumerians, ancient Egyptians, Romans, Chinese, and others, all employed civil servants and a system of bureaucracy to control their empires.

In modern times, bureaucracy has served a similar purpose, replacing feudalism with rules-bound administration. Of course, as Seeing Like a State argues, part of the motivation was to centralize power, cutting out the middle-men (vassals). Even so, a by-product, which later became a goal of its own, was that bureaucracy is a rules-based order. It dampens the arbitrariness of a feudal lord by instituting universal rules for all to follow. During the nineteenth and early twentieth centuries, it became the primary means of making society more just.

Subjects could appeal to ideally objective rules to settle disputes, or to obtain permission to engage in various activities. In theory, a wealthier citizen could not by bribe influence decisions of those in power.

As rights were gradually expanded to include women, unlanded men, people of colour, etc., bureaucracy was a major instrument of implementation.

In parallel to this development, most organizations realized that hiring administrators based on merit, rather than on inheritance, ensured better execution.

Too much of a good thing #

Of course, when everyone complains about bureaucracy, it's because there's too much of it. It's like the Laffer curve related to taxation: Even if you don't like paying taxes, unless you're an anarcho-capitalist, you probably agree that some state institutions (e.g. courts, police, defence) are desirable. And unless you're a true communist, you probably believe in some degree of private property rights. Thus, neither zero percent or a hundred percent tax rates are desirable. The remaining problem is to decide on the optimal fraction in the open interval between 0 and 1.

The same applies, I believe, to bureaucracy. No bureaucracy entails the total absence of objective rules, meritocracy, traceability. Total bureaucracy, on the other hand, implies that nothing gets done. Again, the problem is to find the right balance. And this balance may not be an equilibrium. As external circumstances change, you may have to change how your bureaucracy works: How much of it, what the rules are, etc.

The worst system #

As has been incorrectly attributed to Winston Churchill, democracy is the worst form of government, except for all others which have been tried. Analogously, capitalism is the worst means of wealth distribution, except for all the other systems that have been tried.

It's natural to extend those notions: Bureaucracy is the worst form of administration, except for all the other systems that have been tried.

Of course, a counter-argument could be made that many societies work well without bureaucracy. Anthropologically interested readers will surely point to various well-functioning tribal societies, and tech bros will point to start-up companies. Such organizations surely exist, but they don't scale. Infamously, all successful start-ups ultimately add bureaucracy as organizations grow.

The mature choice #

Programmers tend to dislike bureaucracy as much as the next person. It can be difficult to see the value in moving Jira tickets around.

In light of the above, I'm not advocating bureaucracy for bureaucracy's sake. Much of it is, indeed, counter-productive, but some of it could actually boost your productivity.

For example, most programmers agree that interruptions are major productivity killers. Many interruptions are requests for status updates. How is the feature coming along? Are you working on this, instead of that? How soon can you be done?

I once lead a team of developers, in an organization that used Trello boards, and I tried to convince them that if they would reliably move tickets around on the board to reflect actual status, it could save them quite a few status requests.

Even such a modest request was, however, sullenly but actively ignored as beneath real programmers. The result: Frequent interruptions from stakeholders who wanted to know how far the project, or a particular feature, was coming.

I find such behaviour immature. By being proactive, you can surface information at the time you choose, and if done right, it can decrease the frequency of interruptions you experience. It does, however, require that you minimally play along with a bit of bureaucracy: Move those tickets from left to right. Write status updates. Radiate a bit of information.

Bureaucracy, the good parts #

Clearly, useless red tape exists. The goal of every software organization is to identify what actually works. This varies from organization to organization, and over time, so I don't claim to have the correct and complete list. If you're in doubt, however, I would recommend trying the following:

Write things down. And keep records in a place where everyone can find them. A Slack channel is probably not a good candidate. As I write in Code That Fits in Your Head, a sensible hierarchy of communication exists. It includes favouring readable code over comments, but comments over documentation. Specifically, you should prioritize writing better Git commit messages. And for broader decisions, keep Architecture Decision Records around.

The point of bureaucracy is not to move Jira tickets around for no reason. The crux is to put information where people look for it. If this is Jira, consider it a small price to pay for fewer interruptions.

Most organizations I've consulted tend to have an oral culture. Talking to each other is important: It's a fast, high-bandwidth mode of communication, and it often works well as a social lubricant. It may enable you to go fast in the short term. If, in the other hand, you keep no written record, you can't sustain the pace in the long run.

People cycle out of long-running projects. Key contributors move on to other jobs in other companies. Without a written record, you lack important information about the system. Consider good bureaucratic artefacts as a major counter-move against this problem. When done well, it increases a team's bus factor.

Conclusion #

Bureaucracy understandably has a bad reputation. It's usually done wrong, in which case it can induce unnecessary friction into processes.

Even so, consider what motivates it. Originally, it was a remedy against arbitrary rule of emperors, kings, feudal lords, or other people in power. It's a means to institute regularity and fairness into administration. It's meant to not only regulate, but also document, how administration is performed. When abuse of power occurs (which still happens), investigators can often use bureaucratic artefacts to uncover what most likely happened.

In software development, you may not be concerned with abuse of power, but keeping good records has many benefits. The challenge is not to be rid of bureaucracy, but to trim it to a size where only the good parts remain.

A scenario.

Here's another speculative scenario about the future of software in an increasingly hostile international environment.

It seems indisputable that the future of software development involves substantial use of LLMs. Some people experiment with vibe coding, but I find it more plausible that we'll see widespread use of LLM-based agents that produce code, with continuous, but superficial human supervision.

Can we trust those LLMs?

Economics #

In other articles, I've discussed whether LLMs deserve our trust. The perspective in these articles have mostly been on the inherent non-determinism of these models, as well as their lack of 'understanding' of what it is that they do. The point I will make here, on the other hand, does not depend on assumptions of that kind. You may, if you will, imagine a future in which LLMs are far more reliable than today.

Even so, there's a fundamental aspect of the ecosystem that most developer-cum-futurists seem to ignore. The current systems, operated by companies like OpenAI and Anthropic, run with colossal deficits. While their valuations are measured in billions of dollars, they are losing money every year.

This implies that the way agentic software development works today isn't representative of future directions. But even if today's systems are too inexpensive to last, my social media feed regularly showcases examples of what can best be termed token angst: the dread of running out of tokens.

Investors ultimately want return on investment. While the AI companies currently run on deficits, they can't keep doing that. Sooner or later, they'll either turn a profit or go out of business. How will they generate profit?

There's been some talk of 'showing ads' in chatbots, but that isn't compatible with agent mode. If no-one is looking at the output of an LLM, then ads are an unlikely revenue stream. Perhaps my imagination is poor, but the most realistic scenario is coding agents as paid services. You buy a subscription with a token budget, or alternatively, you pay for tokens as you use them.

This suggests that in the future, price is going to be a competitive factor. You may decide to use Company X for agentic coding, rather than Company Y, because Company X is less expensive.

Subsidized LLM services #

For practical purposes, today's AI companies are American. It'd be naive to think that it will stay that way. When a technology becomes sufficiently strategically important, other states subsidize national enterprises to catch up. To Silicon Valley ears, this may sound derivative and unfit for competition, but such a strategy can work. Historical evidence exists.

You can find examples in another capital-intensive industry, aviation. Airbus probably wouldn't exist without European governments taking an active interest. And I find it fair to argue that Airbus is currently doing better than their main competitor in civil aviation.

Other aviation-related examples may be found with certain airlines that operate out of resource-rich countries. Air fare is surprisingly cheap, while service is top-notch.

In other industries, Chinese electric cars are, too, notably less expensive than competitors. Even if you're a dyed-in-the-wool liberal capitalist believing in competition and free markets, current world trends are moving away from that. We see increasing protectionism and focus on reshoring strategic manufacturing capabilities.

Subsidized AI companies will exist, too.

Secret agents #

Most of the modern information infrastructure has turned out to be porous. Even though top companies are American, adversarial state actors use social-media platforms to wage information warfare. I'd be surprised if something similar doesn't happen to LLM systems.

Granted, some clandestine way of injecting data directly into American LLMs seems difficult. What state actors can do, however, is to offer alternative, subsidized systems. Use this new LLM-based coding service: It's much cheaper than the one you currently use!

You may counter that you'd never use a Chinese, Russian, or pick-your-own-enemy LLM system. But some people and organizations are more price-sensitive than security-conscious. Besides, a dismissal of this scenario assumes that ownership is transparent.

An adversarial state actor could set up a shell company in your country, while keeping technical control. Much evidence seems to indicate that a major video-reel-serving app company already works that way, and although that example is not exclusively related to generative AI, I'm sure that you can extrapolate.

Once a foreign state establishes such a clandestine beachhead, it can use it in several ways. It can passively spy on users' code bases (and other content), and it can actively inject backdoors, viruses, etc. into customers' code bases. If that sounds far-fetched, you may not have heard about the xz utils backdoor. Such attempts are already being made.

Untrustworthy systems #

What can you do to avoid this risk?

You may consider only using systems of known origin. You may decide to stick to OpenAI, Anthropic, or other American companies. Perhaps, but I think that you should consider at least two things. The first is that, as already covered, these companies run huge deficits. Where do the money come from? Investors, you say? Indeed, but which investors? Is it conceivable that some of the investors are already, through chains of shell companies, controlled by foreign governments? And if not now, then in the future?

The second consideration is whether you should consider a US-controlled company benign or hostile. If you work in the US, then you probably consider the US government to be the good guys. Over here in Europe, we used to think that, too, but currently, we are not so sure.

You may, then, think of another countermeasure: Buy hardware powerful enough for in-house LLM hosting. This might be a viable option for some organisations, particularly because continued advancements in both hardware and software will make this increasingly feasible. Still, where are you going to get the LLM?

A few organizations are big enough that they may be able to train their own LLM, but most are not. They'll need to use a copy of an existing LLM. In that case, we're back to the issue of trust. Where do you get the LLM, and why do you trust it?

To be clear, you can't 'review the LLM code'. It's a system trained on massive data sets, the result of which is billions or trillions of weights (i.e. numbers). All you can do is trust it. That said, since training of these systems is a non-deterministic process, you could argue that adversaries can't train malign behaviour into them either. If systems come with spyware, it's more likely that it's embedded in the code that surrounds the model itself. Thus, you could, theoretically, review that part of the system. You'll probably not get the source code, however, so your review would have to be of the machine code. I don't find that realistic.

Countermeasures #

What can you do to protect yourself against such threats?

Some of these are more problematic than others. When you give a third party access to your source code, there's ultimately no way to detect if the third party takes a copy of all the code. Pragmatically, you either have to trust that third party enough, or else behave as though you had already made your code base open source (which there are other good reasons to do, in many contexts).

It's possible that research into zero-knowledge proofs could help address such issues, but this is not something I know much about, so perhaps I only display my ignorance.

The other large problem is whether adversarial LLMs may inject malicious code into your code base, installing backdoors, spyware, or similar. This threat seems more tractable: Treat your code base as if it was open source, and each contribution as a drive-by pull request that requires rigorous scrutiny. We're back to the discussion about trust in software development. As I've previously discussed, you can't even trust yourself to write code without errors, perhaps you can't trust your colleagues either, and I don't think it's wise to trust LLMs to write code.

Fortunately, we have decades of experience in making software development a safe and manageable process. Parts of Code That Fits in Your Head are about working together as a team, and how to ensure that you ship quality software. Treat LLMs as randos on the internet submitting pull requests to your open-source project, and review accordingly.

"But Mark, the LLMs produce code much faster than we can review it." Yes, I know. You are the bottleneck. You and your brain. It was always like this. Typing was never the bottleneck, although it's embarrassing so quickly the industry seems to forget that.

Conclusion #

The economics of AI companies in early 2026 seem unsustainable. In the future, these services will become more expensive; perhaps to a degree where price becomes a competitive advantage. If that happens, adversarial state actors might set up and subsidize shell companies that run LLM-based coding services at more attractive prices than competitors.

Once such companies have customers of interest, they can spy on the source code, and plant backdoors and spyware into code bases.

This is just one of many reasons to be wary of AI-generated source code.

LLM friendliness does not entail competency.

One of the many cognitive biases of the human brain is called the halo effect. In short, it describes the tendency to transfer positive impressions of a person or organization from one area to another. If you like a particular actor because of a role, you may think that he or she has good taste in gin, too.

When serious fiction authors say something about politics, the media reports it. Or, some billionaire is good at producing a particular good, so now you think that what this person has to offer on rescue operations or warfare is gospel.

It seems obvious to coin the term hAIlo effect to describe how LLMs manipulate you into liking them, and thereby trusting their judgement.

Anthropomorphism #

Whenever you have a 'chat' with an LLM, it responds as though it was human. Now, just because I used to be good at programming and a few other things, I'm no psychologist, so beware that you don't trust me too far along the following line of reasoning.

That said, I'm also a writer. One of the most fundamental rules of writing is to avoid the passive voice. Speak to the reader. If appropriate, invest yourself in the text. Be present.

Recently, I've been involved in some academic writing, and I'm having much trouble with the aesthetics (or lack thereof) of this style. You're expected to not involve yourself, ostensibly because this appears subjective. The result is often stilted language, written in the passive voice, and with too many weasel words.

Any LLM responding like that would quickly be outcompeted by one that communicates in human language, pretending to be a person.

Since LLMs come across as persons, however, another cognitive bias makes us anthropomorphize them. We begin to ascribe to them motivations that they may not have.

Servility #

All the LLMs I've 'chatted' with (note the anthropomorphism) come across as friendly and eager to please. After all, any chatbot's raison d'être is to engage with users. It doesn't help that mission if the system scares away people.

Not only are they all, it seems, equipped with an upbeat can-do attitude, it sometimes tips over into obsequiousness. Getting such a system to admit that it doesn't 'know' how to proceed seems unattainable. At one time, I engaged with such a system to figure something out. I no longer remember what it was, but it was something falsifiable, and it kept giving me false answers. Finally, being only human, I succumbed to one of my many cognitive biases and asked it flat out: "You don't know, do you?"

It responded with the usual fawning wall of text.

When you combine the can-do attitude with what seems like a built-in aversion to admit defeat, these systems may come across as more competent than they really are.

Alignment #

One of the things that concern me about LLMs (and other, hypothetical future artificial intelligences) is the question of alignment. When we ask an LLM to perform a task for us, how can we be sure that it does it with our interests in mind? Specifically, if we ask it to write source code for software, what reason do we have to trust that it does it well?

One issue may lie in the fundamental non-deterministic nature of these systems. You can never be sure what errors may inadvertently sneak in.

A deeper problem is whether these systems even have our interests in mind. It's an open question whether an LLM has intrinsic motivations, but it sometimes behaves as though it does. We're getting into Chinese room territory here, which is not quite on my agenda for today. Rather, my point is that an LLM may tell you that it will follow your instructions, and then go do something else. You may tell it to follow test-driven development (TDD), and it will agree. Even so, will it actually use the red-green-refactor cycle? Will it observe the test failure in the red phase? Will it verify that it didn't write a tautological assertion? Will it write only the simplest thing that could possibly work, in order to pass the failing test? Will it abstain from modifying the test in order to make it pass?

When real people are told to follow TDD, they often ignore the instruction, or cheat in various ways. LLMs are trained on code written by people, so you shouldn't be surprised if they behave the same way.

Even so, when I ask vibe-coding enthusiasts why they trust LLM-generated code, the answer usually stops after a few interactions. "Oh, I asked it to write tests."

"Indeed," I respond, "but what makes you trust the tests?"

Sometimes, people get clever: "I asked another agent to write the tests."

"How do you know that the agents aren't colluding?"

"Why would they do that?"

I can't get very far with the usual discussion techniques, such as Socratic questioning or five whys. Before long, I hit a particular brick wall. People intrinsically trust LLMs.

Bullshit artists #

This is confounding to me. Why do people trust these systems? At best, I'm willing to view them as neutral, but all the evidence points to them being manipulative. I've already covered reasons for their anthropomorphic interaction design, and again, I don't wish to derail my own agenda by going off on a tangent related to built-in political and ideological biases, although those are well-documented, too.

Have you ever had a colleague or acquaintance who refused to admit failure? Who always had an answer to everything? Even if it was obvious that he or she had no clue?

In their smarmy way, LLMs will readily admit that they were wrong, but I've yet to experience that they respond with an "I don't know".

Instead, if confronted with a question where the answer is not immediate, they make shit up.

The owners, however, have successfully played the public and convinced everyone that LLMs 'hallucinate'. Since hallucinations is something humans suffer from, if we feel anything about this at all, we may feel sorry for the poor LLM.

Oh, muffin. It's so hard being you.

Using a word such as hallucination, LLM companies have isolated and downplayed what is really the core behaviour of these system. They make up stuff. That's literally what they do. They choose the next words based on a little randomness and what's statistically most likely to come next.

But because they're so ingratiating, we think they are our friends, and forgive them when they make mistakes. We may even feel sorry for them when they do. The poor thing is hallucinating.

Conclusion #

LLMs are undeniably capable of many astonishing feats. Does this mean that we should trust them?

It seems to me that many people intrinsically trust these systems, particularly when being told something that confirms their biases. I've been in discussions where, again, I'm met by: "but the AI says," and I can't get past such appeal to aithority.

For a long time, I couldn't get my head around why people trust LLMs, until it dawned on me that they come across as friendly, eager to please, and perhaps at the same time a little dim-witted.

We may dub this the hailo effect: The cognitive bias that makes us trust AIs because they make us feel good, and we transfer this experience of warmth into trust.