The Culture of Experimentation
Reference: This note details the fourth topic in the 7 broader topics of discussion in experimentation. Further, the points listed in this blog will be later converted into their own blog posts.
Through most of my formative years, I fostered the notion that as a child you make mistakes, but slowly as you grow up you learn from these mistakes and polish yourself in most practical aspects of life. Society expects successful adults to rarely make mistakes and have the experience and skill to lead through life with finesse and confidence. However, as I grew up, I realized that most adults make more mistakes than they imagine. The desire to fit into the molds of successful reliable adults leads most people to cover up for their mistakes, blame someone else for the damage, and often fail to try new things in fear that they might do something wrong.
The truth is that we all are extremely susceptible to making mistakes not because we have a learning disability but because the world is much more chaotic and random than we imagine. What led to a favorable outcome in the past might not lead to a favorable outcome in the future as well. And to learn the dynamics of our environment, we need to try out new things and see the result. According to me, an individual who makes very few mistakes probably does not try out new things. And to improve at things you probably need to be ready to look stupid in front of the world.
Experimentation might seem like a popular and easy-to-implement concept but establishing a culture of experimentation is harder than you can imagine. To embrace experimentation in your life and in your work, one needs to make some fundamental changes into how they view things. Making those changes in your own perspective might still be easy, but bringing these changes into an entire organisation is very hard. The culture of experimentation is probably the biggest barrier in the journey to create an experiment-driven organisation.
In this note, I discuss some core fundamentals of the culture of experimentation that one should understand and adopt. In later blogs, I will give a detailed overview of each of these topics.
Cultural Elements of Experimentation
I have collected these ideas from various places in my study of experimentation. I am sure there might be many more cultural nuances to experimentation than can be covered in this blog post. I request the interested reader to share the same with me if you feel, I am missing out on a crucial element.
- Avoid hubris: Hubris is the quality of having an exaggerated confidence in your beliefs, so much that you are ready to risk your life on it. In 1849, Cholera had spread through Europe and the common belief was that Cholera spreads through the air. New studies had started to show that probably the cause of Cholera was not the air but bad water. Under the circumstances, a tenant complained to the landlord that many people in the building have gotten Cholera and the water in the tank regularly stinks, so there might be a correlation. The landlord came down to the apartment, filled a glass of water from the tap, and chugged it down to show that he was certain there was nothing wrong with the water. He died three days later. The first step in your journey to experimentation is to realize that your beliefs might be wrong.
- The HiPPO Culture: Hippo is an acronym for Highest Paid Person's Opinion (HiPPO). Most organizations operate with the logic of HiPPO first and hence find it hard to implement a culture of experimentation. If the credibility of data is placed below the credibility of the senior management, a culture of experimentation would never be able to thrive and most experimentation would only be done to confirm the opinions of the senior management. It is crucial that the HiPPO culture is successfully weeded out from an organization before it can adopt the culture of experimentation. It needs to be understood that successful ideas can come from all individuals irrespective of their rank in the organization.
- Embracing Failures: Failures are inevitable in the journey to experimentation and failures by nature impart more learnings than successes. If you think an idea will work and it worked, you probably were right but you did not learn anything new. If you think an idea will work but it failed, you have to debug your causal model and figure out why you were wrong. Most organizational cultures do not reward for the number of failures a person made. On the contrary, most failures are condemned in large organizations and erode the reputation of an individual. Organizations need to understand the differences between genuine failures and avoidable mistakes. The former needs to be rewarded and the latter needs to be mitigated.
- A learning mindset: Experimentation is useless if an organization wants to stick with the old ways of doing things. The curiosity to learn and develop better ways to solve things is a must if one wants to harness the power of experimentation. A learning mindset hounds the underlying truths of reality and rejoices at the fact that we are less wrong today than we were yesterday. A learning mindset does not stop at simply reading and absorbing new ideas. A learning mindset creates the necessary motivation for you to experiment with these ideas in real life and see what works for you or not. Sadly, inculcating a learning mindset into an entire organization is much harder than inculcating it in individuals.
- Hyperrealism: Some people are inclined toward an optimistic account of reality whereas others are inclined toward the pessimistic one. However, the perspective of optimism and pessimism comes in when you mix judgment with observation. Great experimenters are patrons of hyperrealism. Hyperrealists meditatively seek the truth without mixing their own personal judgments with observation. For hyperrealists, data has the highest credibility and they do not mold the truth in their interest or towards their initial beliefs. A true culture of experimentation demands a complimentary culture of hyperrealism that primarily focuses on the desire to know the objective truth.
- Quantifying the truth: Truth always remains subjective as long as it is qualitative and not quantitative. If I ask you how were sales last month and you tell me that they were better than last year, it is not an objective truth. On some days it might be better than last year, on some days it might be worse. The search for truth requires that you simplify systems and create reliable metrics to study these systems. Hence, a culture of quantifying the most intangible metrics in your organization and in your life will help you get started on tracking finer nuances of truth. These quantified metrics then help you get started on studying the impact of simple changes in action.
- Semmelweis Reflex: The quest for truth often lands you in situations where data contradicts strongly entrenched beliefs of the scientific community. Sometimes data shows the objective truth, but does not provide a causal explanation to things and hence is very hard to believe. In the mid-nineteenth century, childbed fever was a severe issue among women who were giving birth and over a million women had died of childbed fever in Europe in the nineteenth century. Ignaz Philipp Semmelweis hypothesized that the fever is caused by students and doctors not washing hands and experimented with his hypothesis. Deaths reduced by almost a factor of 10 but the medical community did not accept Semmelweis's findings (and rather rediculed them) in the lack of a causal model. His findings were accepted not before 1879, when Louis Pasteur discovered the Streptococcus bacteria to explain the causal model behind Semmelweis's findings. The backlash that most scientists face for surprise findings is called Semmelweis's reflex. Building a culture that mitigates this reflex is a need in experimentation.
- Evolution of Experimentation Culture: The culture and framework for experimentation in most organisations is not established in a small frame of time. Usually there are four phases of a continuous experimentation culture - Crawl, Walk, Run and Fly. All the four phases see uniquely different challenges ranging from setting up a data pipeline to building robust Overall Evaluation Criterias. The linked paper explains various dimensions of this evolution along the four different phases. The defined classification is widely accepted the experimentation community to segregate the stages of experimentation development in all companies.
The way ahead
I plan to write detailed blog-posts on each of the above topics in later blogposts. The topic of culture of experimentation is a deep topic and delves into many different threads. If I find more topics of interest, I will add those to the list above.
I believe all of these ideas are equally applicable to individuals as to organisations. If you are interested in building a culture of experimentation for yourself, you should first start by applying these ideas in your life.