Thursday, July 2, 2015

My thoughts on The Dunning-Kruger Effect and its influence on perception of Skill and Time

I was skimming through the internet when I happened to come over this question and answer on Quora
Read Piaw Na's answer to I am confident that I am going to build a search engine that will compete with Google at least in the smallest scale possible first, but for now I don't know any programming. What should I do? on Quora

Intrigued, I started reading about the Dunning-Kruger effect. It basically states that people who don't know a complex skill overestimate their abilities and underestimate that skill, while people who do know complex skills underestimate their abilities and knowledge of those skills.

I realized that most of the times I was affected by this, in both ways of underestimating and overestimating my skill level, especially when it comes to computer science and programming.

I realized that everyone at some point in their lives will experience this effect. It especially occurs when you try teaching something you are good at to someone else who is a novice. The most obvious things to you about that skill are so difficult to the other person, that you feel he's not trying hard enough. In reality, it might take time for him to get accustomed to that difficulty level.

Another realization came when I was reading about failed startups and reasons for failing. Probably the number one reason that startups fail is because the initial founders overestimate their skills, and/or underestimate the difficulty of the problem. Now if they are capable people, they will learn the way to overcome their difficulties. But in most of the cases, it just takes too much time and the startup cannot be sustained, or the skills required are too difficult to learn at that point in time.

So my final "Aha!" moment of realization came in that time is one of the crucial factors that affects how you perceive a problem, and how well you understand your skill level. Time also influences your learning ability, and your perception of the difficulty of the problem greatly influences how efficiently you learn. So its this "A influences B influences C influences A ..." cyclic relation where time influences your skill which influences your perception which influences your estimation of time which goes on and on.

This is the sole reason why you cannot really learn something without spending enough time on it. This was exactly what I learned from Prof. Yegnanarayana of IIIT-H in his Artificial Neural Networks class. The neural network requires multiple iterations over an aspect of learning before it is committed to long term memory. Multiple iterations can only be possible with enough time given to that aspect.

So the next time you see "Crash Course! Learn blah blah blah in 30 days", remember that it most definitely will take more than just 30 days time to get good proficiency. And do remember the Dunning-Kruger effect the next time someone says "That's so easy, anyone can do it!"

Cheers ;)