In the commercial context, profit is obviously the selections pressure. However, even here you have to make a product that people are willing to put up with.

We’ve already gone through something similar back in the early 2000s with the whole outsourcing craze. Companies thought that they could massively cut costs by just outsourcing all the development to places like India. And then came the painful realization that having contractors who don’t give a shit about your product pump out slop doesn’t actually work all that well in practice. The exact same thing applies to LLMs.

And of course, software development world extends far beyond commercial companies. People working on projects outside commercial constraints can use them as they see fit, and they don’t have to compromise quality to meet profit margins.