The Perspective of Failure

Success is a charm, struggle still has some dignity to it but failure seems like a dark ink blot on your crisp white shirt hard to wash off forever. There is something about failure that is deeply embarrassing to most. Growing up as part of the Indian education system, I starkly remember the horrors of those who had failed a grade. Failing a grade would often deem you unfit to be promoted to the next grade and more than the burden of studying the same things again, the social rejection these students faced was daunting. For half the class you are a misfit to the batch dynamics, and for the other half being you is the worst nightmare. Most chose not to go through this ordeal and rather change school where they get a fresh start with dignity. I deeply respect those who had the courage to go through such humiliation at a tender age and find their way to happiness through it.

Failure by structure is deeply dreaded and fundamentally embarassing in our society. Partly because failing at something that others are not failing at is a sign of incompetence and weakness relative to your peers. Partly because, no matter how idealistic a society might be, failure by design does not bear the kind of fruits that are borne by success. However, failure is inevitable in the road to success and the only way to avoid failure is to not try. Failure is a byproduct of experimentation and the only way to never fail is not to experiment at all.

Most organisations in our time systemically despise failure. Many organizations are mature enough to tolerate them and move ahead but very rarely does an organization accept and hail failure as a symptom of experimentation and innovation. There is no one to blame, it is just hard. If your junior comes up to tell you that a project he tried for six long months ended up in a failure, it is not going to make you feel elated no matter how much you choose to rationalize it. Failure is uncertain, unrewarding, and ungratifying by nature. One might be able to ignore failures and encourage persistence but applauding failure is very hard.

The aversion to failure laced with organizational bureaucracy is the exact combination that is lethal to a culture of experimentation. Because everyone along the ladder is afraid to fail, most experiments are closed if they do not show positive results early in the test or if there is not anyone senior enough to vouch for those ideas. Experimentation Giants like Facebook, Amazon, and Microsoft have hence developed an organizational framework that allows anyone to test directly on the product without bureaucratic interference.

Battling the Perspective of Failure

It is a sad reality but how much you can battle the problem of failure is strongly proportional to your seniority in the organization. To fundamentally address the fear of failure as a manager, I have segregated three fundamental concepts about failure:

1. Difference between mistakes and failures: The quality of failure can often be judged by where they lie on the scale of mistakes and failures. Pure failures are those that were difficult to see coming for everyone. These are failures that are to be celebrated if caught because they are a low-cost solution to the bigger problem. Pure mistakes are those that could have easily been expected but ignored due to a lack of priority. Because they were to be expected they do not generate many learnings. But putting things on this scale is hard and highly relative. Something that looks obvious to someone on top might not be that obvious to someone working for the first time on it.
2. Finding the root cause of failure: All failures happen due to something in the system that did not work as you expected it to work. If failures are cheap we are tempted to let them go. If failures are embarrassing, we are tempted to hide them from the world. But letting go of a failure without understanding "why?" is like throwing away a gold coin you have already paid for. A subjective and hierarchical diagnosis of all failures is important because they are goldmines of insight.
3. Letting the failures go: Once a failure is juiced out as much as it can be, it should be accepted, celebrated, and then let passed on. Failures that bother us too much are also the ones that cripple us from trying again. Past failures in the right spirit should be forgotten once you have internalized their learnings.

The three steps above are easier said than done because you face intense backlash from the societal perception of failures. Moreover, a failure that was a failure for you might be a mere mistake for someone who has a larger stake. Lastly, the fruits of failure are often hard to see because they are usually the avoidance of bigger failures.

Knowledge Discovery is Asymmetrical

To understand the right aphorism toward failure let me tell you about an interesting concept by Nasim Nicholas Taleb he tells in the book, The Black Swan. Taleb explains that knowledge discovery is asymmetrical. On a fundamental level, you can never completely prove any statement to be true because there is always a possibility that in the future a contradiction comes up. For instance, take the statement "All swans are white", which someone would believe is a commonly known fact. But Taleb says that you can never prove this statement to be true always because there is always a teeny-tiny possibility that tomorrow a 'black swan' pops out of the ocean.

However, the opposite is true you can always disprove a statement if you find a single instance that contradicts the statement. So, knowledge discovery is like that. Everything that you ever prove is based on some assumptions and there is always a small probability that your assumptions are wrong. Hence, in science, we start with a hypothesis, prove it for a class of objects, and then develop a heuristic that will stay with us until it gets disproved. No hypothesis will stay forever because none can ever be proved 100%.

Essentially then, everything that we know and believe can never be proved to be entirely true but can be disproved much easily. Most people strive to be objectively correct but being right is much harder than being wrong. Those who understand this, choose to become comfortable with being wrong and updating their beliefs as soon as they find new ideas and contradictions. These people look out to being wrong and celebrate when they find a convincing explanation to why they were wrong. Experimentation today demands a similar kind of personal and cultural framework.

Conclusion

People whose work involves a lot of uncertainty in outcomes have effectively learned that the best approach toward failure is to "Fail Fast and Fail Cheap". Failure cannot be objectively avoided but being skeptical of all your learnings helps you start with designing low-cost experiments to test your beliefs. As soon as an experiment fails, you quickly move over to make a new hypothesis about what is really happening.

Many successful startups now claim that aggressively moving over product and feature ideas, and hastily invalidating what is not working has been the core of their success. However, such a fierce iteration of experimental learning requires an organization to have the intellectual humility to experiment with everything that they believe. Those who are not able to develop this humility, often end up either rejecting experimentation or making experimentation just a tool for validating what they have already deemed to be true.