Over the years I’ve oscillated between reading quite a bit and reading not much at all. Lately I’ve been trying to read a lot more and the book I’m currently reading is called “System Error: Where Big Tech Went Wrong and How We Can Reboot”. Well while escorting my wife to a doctor’s appointment this morning I read a chapter that I found particularly striking as it spends most of it’s time dissecting this famous Donald Knuth quote:

“Premature optimization is the root of all evil (or at least most of it) in programming”

This is a statement that is widely regarded as being an unassailable truth in our industry. Anybody who has been slinging code for any amount of time has likely heard this truism repeated to them time and time again either while having their code reviewed or while reviewing the code of others. But here is the twist: Does it actually mean what we think it means?

We collectively interpret this quote as a caution against spending too much time up front trying to write the most efficient code possible. This makes a lot of sense once you get enough experience with software development as you soon learn nobody involved in this process has any goddamn clue what it is they are doing. This of course is why it takes at least ten thousand iterations before we produce something that actually comes close to either (a) solving the customers problem or (b) spending enough of their time and / or money that they just find some way to live with it.

So great, blog post over, right? Not quite. The chapter goes on to dissect this a bit more and manages to turn it on it’s head:

For Knuth, the time and place to be efficient is when you have figured out what is worth making efficient by analyzing the effects of efficiency at a higher level.

But that’s not really the point the chapter is trying to make. Rather as it continues on, it follows up with this thought:

It’s worth emphasizing that whoever makes the choice of what to optimize is effectively deciding what problems are worth solving.

The point being that as technologists we wield incredible power because what we choose to spend our time on, or rather what the rich assholes above us decide that we should spend our time on, directly influences the fates and fortunes of so many around us whether we choose to acknowledge that or not.

By mere coincidence I started re-watching the classic sci-fi series “Firefly” over the last weekend and last night I watched the episode, “The Train Job”. The basic gist of the episode is the crew of Serenity takes a job to rob a train of some its payload but as with all plans, things go a bit awry and two of crew are forced to stay on the train until it reaches its destination while allowing their cohort to escape with the goods. While being questioned by the sheriff at the destination they find out that the stolen goods in question was rare but highly valued medication for the people in the town who were clearly suffering without it. Once they realize what happened, they decide to return the medicine to the town knowing that it will put them on the shit list of the crime lord who hired them to do the job.

To me these things directly tie together. In tech we have fallen into the terrible habit of adopting narrow views of the so-called “problems” we are trying to solve and working like mad dogs to solve them. The only problem is at some point the people crafting the actual software decided to let other members of the team / organization figure out all of the bigger picture stuff. We are so obsessed with the mere technical details that we decided all of that other stuff wasn’t really worth our time.

Those of you who know me (which isn’t many of you as this blog is largely anonymous) know that I’m generally a big fan of Agile and I despise SCRUM. One of the reasons for this is that SCRUM is basically what happens when you unleash a horde of slavering greedy MBAs on a perfect acceptable set of principles like what was outlined in the Agile Manifesto. For those of you not in the know, I’ll briefly repeat those principles below:

  1. Individuals and interactions over processes and tools
  2. Working software over comprehensive documentation
  3. Customer collaboration over contract negotiation
  4. Responding to change over following a plan

That’s it. So simple and yet so true. The primary problem with SCRUM is that it basically ruins all of those principles. In the case of this particular blog post, the primary failing of concern is its bastardization of the third principle.

The people writing the code generally don’t directly collaborate with the customers. Ever. We talk to a Product Owner who if we are lucky talks directly to the customers. Where I work now that Product person talks to the Support staff (who are slowly being replaced by LLMs sad to say) who might talk to the customer if we are lucky. The collaboration cycle has been effectively broken. I imagine it’s the same at a lot of other places.

But it’s worse than that. We aren’t just not listening to our customers by subjecting them to an increasingly insane game of Telephone but we are also actively going out of our way to ignore their real problems. Why are we doing this?

  1. Solving an individual customers problems isn’t relevant at scale. If you have a handful of customers, solving their individual problems is going to be your top priority. If you have a thousand, a hundred thousand or even a million customers, their individual problems cease to matter. The specific pain just becomes easily discarded noise in an illusion propped up by selectively chosen metrics that exist to convince us of an alternate aggregate reality.

  2. The bigger our systems get and the more customers they serve, the easier it becomes to ignore the corner and edge cases that exist within it. If 95% of customers are doing well enough not to complain then perhaps the other 5% are the source of the problem rather than the canaries warning you of a problem.

  3. None of us are being paid to solve customer problems. At least not at the medium or big tech level. We are being paid to deliver results and results have to be measurable. The way those results are measured isn’t in terms of customer satisfaction or touchy-feely metrics, but rather by whatever system your bosses have setup.

On some level technologists like myself have always been dangerously disconnected from the world around them, even decades ago. But when we were primarily producing word processors and financial calculation systems, it didn’t really matter. Fast forward a few decades and we now live in a world where people aren’t able to calmly discuss political points of disagreement around their family dinner table anymore because all of us are spending are days immersed in curated echo chambers in which we are only being exposed to whatever takes serve to prop up our existing opinions regardless of how sane or insane they happen to be.

The reason for that is engagement. Engagement is the terribly shitacular foundation that so much of our failure as an industry has been built upon. At some point our real goal became to maximize our revenue and the best way to do that is to keep the customer inside your world as long as possible. If they have less reason to leave, they have more reason to spend money, which in turn makes us more money.

I despise this business model. I love solving problems. In fact when my therapist reads my post, she’ll probably comment on the fact that I seem compulsively driven to solve problems people share with me. It wasn’t always like this with me, but I believe working in this industry for 25 years made me this way. Thankfully that’s not a bad thing because at least to me it shows me that my heart is still in the right place.

So fuck the MBAs and their bullshit renditions of actual problems. They can’t be trusted and everything they do revolves around maximizing the company’s revenue. I don’t care about the company’s revenue. I don’t care about the company’s profit margin. Don’t like it? Go fuck yourself. I got into this industry to solve people’s problems. I’m getting out of it because not only do we not care to solve their problems, but we don’t even care enough to take the time to properly define them.

Instead we are just solving our problems and lining our own pockets while forcing the customers, which is a cute slur that basically refers to the group of people we are currently actively exploiting, to come along for the ride.

Consequences be damned.