Modern testing for modern stacks

We have gotten into the habit of thinking deeper about one topic on a weekly basis. We pick topics based on anything interesting we read - so the topics can range from 'how to express the value of testing' to 'Dieter Rams' design principles' to 'effective remote work habits'. Employees are guided to spend no more than one hour researching the topic online. The emphasis is on coming up with their own ideas and interpretations. We then meet as a group to exchange ideas. I love this habit and consider it one of the more unique benefits you will enjoy at Qxf2.

Disclaimer: These are highly compressed notes of our ~30-minute long discussion. So a lot is lost in translation. Further, the ideas below represent what each team member thought about the topic as of that day. Their views may have evolved and changed.

Topic: Survivorship bias

16-Aug-2016

We have a tendency to over-simplify what it takes to succeed. This leads to a lot of cookie-cutter practices being adopted and touted without critical evaluation. One bias affecting this tendency to have an overly simplistic model of success is called 'survivorship bias'. We tend to draw conclusions from a list of successes but for the analysis to be rigorous, you need to include the failures too.

Our thoughts

Arun

I feel this topic is relevant as Qxf2 matures. As we grow larger, we will: have job roles other than testing, get more people who have worked in large environments,find it really hard to share information across the company,work with more clients of varying process-maturity levels,find that our tech solutions have limited shelf life once we reach a certain size. In these cases, a natural instinct is create rules and process around them. But it is important to have perspective when creating rules and not fall for survivorship bias. We won't be able to get past this bias. But being mentally aware of this bias is useful.
E.g.1: When advising clients, bite your tongue before going-"we did this at this other company"
E.g.2: Where we have been smart in the past: we did not 'play house',e.g. no dedicated sales just because larger companies have them
E.g.3: Another time to think about this: when you start comparing yourself to others and question your path
I think the reverse kind of thinking applies to testers too. What you think matters may not actually matter! e.g.learning from what went good and from not just from production issues

Avinash

We tend to concentrate on the people or things that "survived" overlooking those that did not because of their lack of visibility. All the talks we hear are focused on successful people and what they did good to achieve it.We tend to draw conclusion just based on the success stories. I like the analogy of using the old equipments about music and felt its true. I had some experience about some bad software and concluded it was bad as development team was bad. But later found it was not an issue with develepment team but management issue. While testing we may overlook some features because of our past experience and we need to be careful.

Annapoorani

Misconception: we should focus on the successful if we wish to become successful.
The Truth: When failure becomes invisible, the difference between failure and success may also become invisible.When we think to start some business,we always see the success one ,never the failure side.This leads to survivor bias. How can we handle this: getting feedback from employee those who left the organization,instead of getting advice from successful people ,get it from unsuccessful people,it doesn't mean that copy failure, we can still learn a lot by understanding the widest range of customer experiences as possible,find the difference between success and failure people.Get the stories of unsuccess people and analyze it where they tripped off,imaginary in alternate ways.Ask so many questions by ourselves like "what if","how it's" and avoid the survivor bias.

Rohan

Survivor bias refers to our tendency to focus on the living instead of the dead or on winners instead of losers or on success instead of failures. It is easy to do. After any process that leaves behind survivors, the non-survivors are often removed from our views. If non-survivors become invisible then naturally, we will pay more attention to success. Many of us have a misconception about success, they think, if we want to be success; we need to focus on successfully stories. Many times I heard success stories of Dhirubhai Ambani, PM Narendra Modi from elders. I feel success stories are good for motivation. But every coin has two sides. There were many people who worked at petrol pump, small hotels, but they are invisible now. If we focus on one side of coin, other will disappears. Many be because of these, APJ Abdul Kamal suggest that, 'don't read success stories, you will get only message, read failure stories, you will get some ideas to become success..!!'

Shiva

It is not possible for any organization to emulate the success story of another to be successful. Although it may work it does not guarantee success. Everyone think value investing would make them rich like Warren Buffet. There are even organizations that did not change after being successful initially. It is vital that you change your outlook on things when other things change around you. It is obvious that software has changed since it ran only on computers.. Testing too cannot be done based on how successfully some software was tested in the past. Software is used in pretty much everything these days . We may see integration testing playing a big role in testing softwares in the future.

paper cut