Predictions are not necessarily bad if they don't turn out to be true

This was in The Guardian:

 Oh dear!  This again.  I have no drum to bang for pollsters and they may well have messed things up completely, but this paragraph is fairly hopeless. 

If we have a prediction, with a probability attached, it is natural to look at the biggest probability and to make use of this.  If there is a “15% chance that Donald Trump will win” then this is deemed to be unlikely.  But unlikely isn’t impossible.  By saying that the statement was ‘wrong’ we would also be saying that if a total of 6 comes up when you roll two dice then you must have cheated!

This crops up over and over.  An example is the (in-)famous prediction by the Met Office in 2009 of a ‘barbecue summer’.  Here is how the Daily Mail website refers to it:

The April statement says sun is “odds on” and that it is “likely” to be warmer.  That it didn’t happen doesn’t make the prediction necessarily “wrong”, just that a less likely scenario prevailed this time around.  (I am not sure it makes sense to make such a high profile prediction where you think it is only 2/3 likely to happen, though.)  

To be clear, I am not saying that either of these predictions are definitely “right”, but just that a scenario that is deemed to be unlikely can occur without the probabilities being wrong.

FFT

As teachers we also see this with FFT grades.  These grades used to be presented as percentages.  They would show that, for a pupil with a similar background (eg KS2 results, etc) a certain proportion of the pupils went on to get an A*, A, B, C, D, etc grade in each GCSE subject.  

Nowadays only the median version is given, and this is shown as _the_ FFT grade for the pupil.  In the new system these are given to the nearest third of a grade. 

If a  pupil has B+ as their FFT grade what this really means is that about half of children with a similar background to that pupil will get a high grade B or above, and half will get a high grade B or below.  It might well be the case that a small fraction actually get a high grade B.  If that pupil doesn’t get a high grade B that doesn’t mean that over- or under-achieved, or that their FFT grade was ‘wrong’, just that it didn’t happen. 

[The initial article also forms part of a Quibans.  Find it here.]

Sources:

https://www.theguardian.com/commentisfree/2016/nov/09/polls-wrong-donald-trump-election

http://www.dailymail.co.uk/news/article-1202982/Met-Office-left-red-faced-Britains-forecast-barbecue-summer-turns-washout.html