Pensieri di un lunatico minore

Hard problems and the Temple of Complexity

Some might interpret my ranting against enterprisey astronauts as denying the existence of truly hard problems in the world. This is the farthest from the truth. There are seriously hard problems. A hard problem to me is one whose solution might fall into one of the following categories:

• P or NP-complete
• Analysis of huge (terabytes) of data in real-time
• Stream prediction
• Provable systems
• Trusted systems (see provable)

Collecting customer data, sales information and offering a few pretty pie charts is not a “hard problem.” It has a lot of pieces, and it might be complex if you let it become that, but it’s not a hard problem. There are no hidden Markov models predicting customer behavior. There is no link analysis to determine latent connection. There’s just some tables, some presentation and a tiny “wafer thin” controller layer.

Hard problems require leaps of intellectual capacity that 99% of the population is incapable of—including myself. Hard problems challenge us as human beings. We devalue the truly difficult by pretending, for the benefit of our own egos, that what we do is hard. So why do we do this?

I call it the Temple of Complexity: a religion so ingrained in the IT industry that it has slipped from a conscious behavior to a subconscious assumption. We are programmed by the industry—and society to a lesser extent—to proclaim our littlest accomplishments as comparable to the great works of the world. I witness this every day when people sigh and tell me how much work something is going to take. It’s not a lot of work, but our nature says we want to believe what we do is “real work.”

There is a saying¹ that I use repeatedly with my coworkers: “With infinite flexibility comes infinite complexity.” The complexity of a “solution” becomes asymptotic as you approach total flexibility. Learning where to stop on that curve before it becomes unmanageable is the art that requires self-observation, a keen eye and a perspective that questions the conventional wisdom.

I am continually amazed at the obsession with the complex that I witness in my day-to-day experiences. People will drag an entire project into the weeds to examine, in excruciating detail, a feature that perhaps 1% of our customer base might use, some time, if they discover it and don’t get fed up with the rest of the product first. The question is, to me, can they accomplish their job—their real business goal—without that feature? If so, then it isn’t important and should be removed from the system until we reach a point where it enables some new capability hither-to unavailable in the system.

How many features are in Word? Windows? MacOS X even? Do you even know? Do the programmers even know? The argument that “we need to give people options” is the tacit admission that “we have no idea what people need.” Rather than step back and examine the situation—or better yet get something out there so you can get real feedback—people just pile on the feature set, pile on the complexity, and watch their timelines slip into the future.

Let me, for a moment, look at a specific category of software that someone brought up as “enterprise problems,” and discuss the complete and total systemic failure of it as a solution to the underlying business problem: Customer Relationship Management—CRM.

I have worked with 3 major packages (all of which equally inadequate and so unworthy of mention), which represent I would suspect 80% of the “enterprise CRM” market. I have worked with some of them in multiple deployments at Fortune 500 companies. In each case, without exception, they have been: over budget (sometimes by a factor of 10), geologically behind schedule, and when measured against actual benefits delivered to the company, woefully underperforming. Tends of millions are spent, and little value is delivered. Thousands of features are delivered, a tiny handful are actually used.

Why is this? Why do companies spend millions of dollars on CRM “solutions” that solve nothing except how a consulting firm is going to hit their profit goals? Their drivers are inverted from their natural order. For example, the sales cycle component is focused around the reporting needs of management, and the sales individual is saddled with a burdensome process that doesn’t make them more money, and money is what drives sales people. So you get people who fake the numbers, put them in 2 hours before their manager is going to use them, and use a swag at best to manage thing. Why should they do anything different? They get zero value from the system. That’s enterprisey at its most pure form.

Complexity isn’t power. Clarity and simplicity are power. Power is having the right feature for the right person, not 100 features. Knowing what that one feature is requires a discipline that is anathema to what we are taught. It requires saying “no” to yourself, your coworkers and to others. It requires the removal of the facade of complexity and difficulty that we hide behind, both to make ourselves feel better and to absolve ourselves of the risk of failure. Only when we remove that pretension can we begin to understand the real problem.

¹ I stole this from Spiderman, so shoot me.