Suppose you take a test and pass it along with nine other people. Initially you are only told that you have passed and hence you are all too happy about it. But later, you discover this webpage where a list of passed candidates is given. The list is numbered and your name is mentioned at the first place. However, nowhere is it mentioned that the list is sorted according to the scores obtained in the test. So may be, it is just arranged in the order in which the papers were checked. Or may be, the order is just random. The question is, should this new information make you happier than before?
Suppose you had never seen this list and were just told that you had passed. In that case, it would only be fair to assign equal probabilities to you being at any of the positions from one to ten. Your expected position, therefore, would be .
You should be happier than before if the new information raises your expected position to something above However, since you have no idea about the order in which the list is sorted, all permutations of the present order are equally likely and since there are equal number of permutations with you occupying position for any from to , you are once again, equally likely to occupy any of the ten available positions. So this new information has absolutely zero information content and your expected position is still exactly equal to .
Now let’s say, that a friend of yours comes and tells you that the list is either sorted in the increasing order of scores obtained in the test or in the decreasing order and he seems quite confident about it, should you be happier than before now? The answer, once again, is ‘no’. This is because your expected position is still In fact, this holds even if you were at position instead of 1 in the list given on the webpage. Your expected value would be
I find this analysis interesting because the above situation occurs quite often and in most cases, one feels tempted to draw conclusions about his actual position in the test from the position in the list provided. Also, increasing order of merit and decreasing order of merit are two most likely orders that come to mind.
One thought on “On test reports and their interpretations”
An enlightening analysis indeed! Thank you for opening my eyes. Keep posting. 🙂
-Sandy (The Village Idiot.)