People in Silicon Valley Choose the Wrong Problems

People in Silicon Valley choose the wrong problems to work on. There’s this pressure to find a problem immediately and work on a solution to it.

There are three types of problems you can solve, and you should only optimize for two of the types. I categorize these groups by their respective solutions: low improvement with an end in sight, high improvement with an end in sight, and high improvement with no apparent end in sight. [1]

Low improvement with end in sight

People should not solve problems that involve making better calendar apps — low improvement problems where the end is in sight. Minor improvement with end in sight solutions make marginal improvements on current solutions. The solutions that fall into this group are things like better email clients, better messaging systems, and new calendar interfaces. People mistake working on these problems because they think these problems are sexy and derisked. The thing that makes these ideas seem hot is that you interact with these current solutions daily. And the thing that makes them seem derisked is that similar problems have been solved before. If you want to maximize impact, the real risk is to choose to work on these problems because you’ll be devoting a lot of time to something that will potentially only have marginal impact in the world. [2] The expected value of working on a high improvement problem is much higher. Minor improvement with end in sight problems are attractive because short-term successes, such as acquisitions by larger companies, are overhyped by the press.

Don’t make the mistake of confusing something that is a seemingly minor improvement solution with something is an actual low improvement solution. In fact, the more toyish it seems, the more different it likely is from current solutions. And if such a toy succeeds, it means that it was exceedingly better than a low improvement with end in sight solution. Slack is a great example of the “seemingly minor improvement” phenomenon. When Slack shipped, it looked like a pretty IRC client on the surface, but Slack showed that it aimed to replace email altogether by centralizing many work utilities. And because Slack seeks to replace email, Slack has to be a much larger better than email to have people switch (i.e. 10x better). And if you look at Slack’s growth, you’ll see that people believe it’s that much better. [3]

High improvement with end in sight

Companies like Palantir, Salesforce, and Zenefits solve high improvement with end in sight problems. The kinds of problems that are in the large improvement with end in sight category are problems like: “system x is broken in the government. We need to take the data in the current version of system x, move it so that it is more accessible, and build a tool that can allow more people to make use of such data as well as improve organizational insight”. Solutions to high improvement with end in sight problems are still hard to execute. A lot of major improvement with end in sight solutions are technically robust. Just because there’s an end in sight doesn’t mean it’s not a hard problem; it just means there’s a defined end goal.
There are so many problems in this category that people ignore. A common thing I hear from people is that “all the good ideas have been taken, all the ‘Facebooks’ in the world have been done.” If you just worked in an industry for a few months, you’d realize that there are plenty of problems that haven’t been addressed. And, if built, these solutions would boost productivity and efficiency of the industry enormously. There’s an abundance of these types of challenges waiting to be solved.

High improvement with no obvious end in sight

High improvement with no obvious end in sight problems are interesting because they are slightly harder to describe. I like to divide them into two groups. The first group is less obvious human needs and more obvious human needs.
Less obvious human needs capitalize on psychological tendencies that we need itched, or even tendencies we didn’t even know we needed itching. One of the great examples of a less obvious human need is Facebook. The desire to learn anything about a person by looking them up is powerful.Being able to see photos of a particular person, to know their relationship status, to know what they do on a day to day basis — these are behaviors(some of which we didn’t know we had) that we were able to itch because of Facebook. Twitter is another example of a less obvious human need — it itches our desire to know the opinions of people and to learn what is going on in the world. [4]
More distinct human needs are things like a cure for heart disease and a fix for the energy problem. These problems are tremendous improvement with no visible end in sight; if we knew what the cure for cancer was, people would be working on it already/it would be finished. I think that a lot of people plan to work on these problems when they are young, but as they get older, they decide to work on problems that are minor improvement to have seemingly stable lives.
I think that too many people focus on the minor improvement problems rather than major improvement problems. If more people worked on major improvement problems, I believe there’d be a first order consequence and a second order consequence. Initially, most of these problems would get solved, which would make the world better and more efficient. At second order, people would receive returns proportional to the impact they have, which would incentivize people economically to work on high improvement problems.

How to choose problems that you’ll actually want to work on

I think there are things you can work on that are solely interesting to you. These things aren’t actually problems, but when you think within the scope of these interests, you can “create” problems that justify the use of working on/with the interesting thing. Machine learning is an example of a ‘fake problem’/solution masquerading as a problem; a lot of the people I talk to get wrapped up in thinking about the solution first without thinking about the problem. They pose “What would be a cool thing to work on? I want to work on/with machine learning”, and they follow with problems that they come up with which they believe machine learning will solve. [5] Personally, I believe that people should work on things that bother them. [6]

The amount of fulfillment you’ll have from fixing something that bothers you is roughly two-fold compared to working on something that’s solely interesting; I believe this for the for the following reasons. Firstly, when you know what the actual problem is, you don’t have to approximate or continually recruit people to talk to in early days for customer development to discover what their problem is — this means you’re much more likely to correctly solve a problem, as opposed to working and building something but not solving the problem well . Secondly, it’s more satisfying to work on a problem you have yourself because you get to use your own solution for your problem, so you know when you’re making progress. Lastly, your own problems are likely interesting to you, in the sense that humans are interested in themselves, and therefore interested in their own problems.

Thanks to Raphael KatsAneesh Pappu, and Chris Barber for listening to/reading drafts of this.

If you enjoyed this, let me know by recommending it. If you disagree/have thoughts, feel free to email me or leave a response.

[1] An “end in sight” means that the end solution can be defined. Something that has “no end in sight” would be something like the cure to cancer.

[2] See for more on this type of risk evaluation.

[3] See for more on seemingly toyish ideas.

[4] Some people would argue that these products would have been made regardless if Mark or Jack worked on them; therefore, they aren’t innovative. There may be merit to that, but more importantly, these products were undoubtedly incredibly impactful.

[5] This is not to be confused with solving a problem within the realm of machine learning research or making machine-learning technology more accessible.

[6] If you don’t know what the problems are in your field (e.g. research), then you should optimize for working on things that are interesting. I believe, though, you can reduce your motivation to a problem statement i.e. there’s is not enough known about x field of research, and the solution would be to tinker and figure out more.