There is a personality test that was invented by Myers and Briggs that helps groups of people work together better because it shows each member how the other members are different from each other. When you see this, you can make adjustments and understand what feelings motivate each person.
As a part of this Myers-Briggs test you typically answer a series of questions and then your answers show aspects of your personality. You can take the test for free here.
Now this essay is not about personalities or psychology. It is about how to ask questions that reveal things and how to structure judging platforms that are more challenging but provide more solid results. One of the questions is the title of this essay – “Do you prefer mercy or justice?” Of course, you prefer both in your life. We all do. But if forced to make a choice, you have to think that through more carefully. Right?
Now, let’s take the hypothetical situation where there is a judging panel of 5 people who will score some contest – a business plan contest will do fine for our example here. Each judge will score the business teams on five categories and are allowed to award any category up to 10 points where 10 is the best score, and 0 is worst. So a top plan could score 50 points, and a dismal plan could score 0 points. Let’s also suppose that we have ten teams but only 3 of them can “win” the competition and be awarded prizes.
If you ever look at the results of such judging, typically you will find that even a range of teams and plans all tend to get scores between about 35 and 50 points. And so differences between teams is often quite small numerically even though they are not close in capability at all!
A judge in the competition can have an undo influence on the judging panel and its process if that person simply uses the entire scale of scores for the same set of teams.
Let’s suppose there are ten teams and 4 of the 5 judges gives scores that are have a small spread in scores. But one judge gives 7 of the teams 0 points and 3 of them 50 points. If this happens, it is pretty clear that those 3three teams have a very strong likelihood that they will be the 3 winners!!
An alternative way to do this that preserves fairness is to insist that each judge may only award a total of 50 points throughout the entire contest. That is that there are ten teams, and each team can get an average of 50 points. So it is now not possible for every team to even get 35 points let alone the typical average score of perhaps 42 points. So such a scoring system forces choices akin to the mercy or justice question of Myers-Briggs.
Such a judging system is easy to do using simple electronic computers/tablets or even on a smartphone. But we often don’t want to be made to judge and accept that there are only going to be a few winners.
I will visit this topic again in the next essay but from an entirely different use case.