One of the few rule changes for FSA over the “Rules 2017” is the revised BPP scoring (Rule 5.2 in V1.0). The Formula looks close enough to the original one so that the ramifications may not be immediately apparent.
Let’s start with “Why” – why do we think this change is necessary ?
This is the Rules 2017 scoring formula:
The finalists receive roughly the same score from 71-75, immediately followed by the rest from 70 downwards.
To consider this system as “fair” a few assumptions have to be made:
- The points space is, especially around the top, uniformly populated (i.e. the competitors are close)
- The finalists are almost at the same quality level
If you throw enough teams in the pot, these will likely apply and that is usually the case for big competitions like FSG, FSUK or the FSAE MI. At FSA however, the pool is much smaller with a maximum of 30/20 (IC/EV) participants per class. That could lead to the assumptions falling apart in a number of ways:
- There is a considerable quality difference between the finalists (The judging is somewhat comparative, but not across “lanes”, so it only becomes apparent after the finals)
- The scores are not uniformly distributed but sparse. This can manifest in a number of ways, e.g. if there were a few great presentations (three, all in the finals), and the rest of them is average or below.
This leads to questionable effects when applying the standard rules:
- Finalists of different quality get pulled together within 3-5 points
- If there is a significant quality gap between the finalists and the non-finalists, the gap closes to almost nil (1-3 points)
So what can we do about it?
This led to new premises for a modified BPP scoring:
- should still be simple to calculate
- should maintain well-earned quality/scorning gaps between participants in the final score
- should allow for stronger differentiation between finalists
So this is what we came up with:
What this changes is best described by a two examples:
Note 1: In both scenarios, the “raw” scores have been normalized to 75 for better comparability.
Scenario 1: Disconnected Finalists
In this case, the finalists are significantly better than the rest. The FSA scoring makes sure that they maintain their relative lead as the fourth place is scored relative to the third place using the original points distribution. Using the original formula everybody would have received an undue points boost towards the winners.
Scenario 2: Spread finalists
In this case the scores are relatively uniformly distributed around the top, but there is a significant spread between the finalists (75-64). Here the third (and second) place would receive an undue boost (9 points for the third). The judges here decided that the third is really not on par with the winner and awarded only 69 points – which is possible since the fourth placing team only has 66 points.
