Forecast temperature verification

I’ve been trying to compare forecast temperatures from the Met Office and MeteoGroup to see which NWP model was the more accurate. Here are their forecasts for yesterday (fig 1).

Figure 1 – Courtesy of the BBC and the Met Office

As you can see they use a more or less common set of major cities across the UK which helps. On Twitter the Met Office seem to prefer labelling the 1400 temperature rather than the 1600 temperature as the BBC do, so a direct comparison is difficult. It might be a little easier if both of them displayed a much more useful maximum temperature [06-18] for the day, but that would never do would it? I noticed that in their video forecast for today, the Met Office had switched to labelling temperatures at 1500, although I can’t seem to figure out where that 11°C label is for in this mornings (fig 2).

Figure 2 – Courtesy of the Met Office

As you know daytime temperatures are notoriously difficult to forecast with variable cloud at anytime of the year, but especially so in spring with breezes of cold seas, and as you can see there were was a large range in forecast temperatures from the two models for yesterday. Here are the actual air temperatures for 1500 BST for comparison purposes (fig 3).

Figure 3 – Courtesy of Me

Despite the timing differences I mentioned above, I make it that as far as yesterday goes, the Met Office forecast temperature was closer to the actual temperature (at 1500 BST) than was the MeteoGroup forecast at 60% of the 14 city sites. Of course I need to put in quite a bit of work over a longer period to see which model is the more accurate.

MeteoGroup v Met Office

I’ll fully admit before I start that a inter-comparison between the ECMWF based forecast presentation from MeteoGroup and that of the Met Office’s with their own model data on a day of reasonably benign weather like today is a little bit over the top.

Figure 1 – Courtesy of the BBC

The ECMWF model does have a finger of showers running down the east coast but Simon King doesn’t (or forgets) to mention them in this particular broadcast at 12 am, even though he’s under no obvious time pressure (fig 1). The MeteoGroup graphics showed little or no low cloud across the southeast of England, which prompted him to say “a bit of cloud feeding into central areas, but either side of that we’ll continue with the sunshine into the afternoon” (fig 2). So nothings changed with the presenters, they believe the NWP in the graphics rather than check the latest observations, weather radar and visible satellite image. It’s as much the presenters fault as it is MeteoGroup and they now all work for the BBC and not them.

Figure 2
Figure 3 – Courtesy of the Met Office

In the forecast video on the Met Office web site, Aidan McGivern does mention the showers which seem to be more obvious on the Met Office model, but then puts his foot right in it by adding that classic cliché ‘high pressure in charge’ (fig 3).

Figure 4

This visible image reveals how the BBC graphics have underdone the amount of low cloud across the southeast, the Met Office seem to have done better in its distribution (fig 4).

Figure 5

As for the light showers that are aligned along the weak cold front, both models have underplayed them, the Met Office seem to have made slightly more of them (fig 5).

In the forecast for this lunchtime, I think the Met office and their model did enough to just edged it over the BBC. Next time I do an inter-comparison between the two, I’ll have to choose a day with a bit more weather.

Forecast of -14°C from the Met Office a little on the low side

Figure 1 – Courtesy of Twitter and the Met Office

Yesterday afternoon the Met Office were promising the coldest night since February 2012 (fig 1) when they said on Twitter:-

“Temps in rural parts of Scotland could fall as low as -14 or -15 °C tonight, which would make it the UK’s coldest night since 11 February 2012, almost 6 years ago! High pressure will bring clear & calm conditions and is dragging cold arctic air from the north”

Unfortunately, the low cloud of yesterday evening didn’t finally clear till towards midnight, neither was it ever  perfectly calm, and with only a patchy or thin cover of snow, the air temperature never got down as low as the -14°C that was promised.

In the end after trawling through all climate reports from sites which they like to keep to themselves, they did manage to find a -9.1°C at Dalwhinnie, just down the A9 from Aviemore (fig 2). The model does have problems, the -9°C instead of -15°C is just a symptom, it mishandled the overnight forecast of low cloud, surface wind, and probably the whole pressure field.

Figure 2 – Courtesy of Twitter and the Met Office

For the rest of us poor mortals who have to rely on the kindness of a man in Barcelona and his OGIMET website, here are the coldest spots from the SYNOP reports (fig 3). As you can see the other forecast temperatures outwith the Highlands of Scotland were much more accurate, except for southern areas and across East Anglia, where the strength of the wind prevented an air frost.

Figure 3
How did the BBC do?

The BBC who presumably use the same model data as the Met Office, almost got it almost right with a -10°C minimum (fig 4). I don’t know what they were thinking with a -4°C for the Western Isles and a -3°C for the Northern Isles though.

Figure 4

At one time the Met Office would have been just content in announcing the coldest night for six years after it had happened. But now because of the pressures of social media and to maintain their corporate image, they feel they must announce it before it does. This faux pas is certainly not on the same scale as the barbecue summer forecast was, and probably will go unnoticed by the vast majority of people anyway.

Not too hot on last night’s mins

Figure 1 – Forecast rural minima courtesy of Twitter & Met Office [27/28 Dec 2017]
Figure 2 – Actual [18-06] minima from SYNOPs courtesy of OGIMET [27/28 Dec 2017]

I wonder what the verification score was for last night’s forecast minima temperatures across the UK by the Met Office? The forecast was for a moderate or severe frost (fig 1) which never quite made it. The forecast of a minimum of -10°C at Aviemore for example, ended up being -0.7°C (compare figs 1 & 2). A lot of places may have had a bit of a snow cover, but it was much cloudier in the north than forecast, and the wind never quite died down either to make it a more ideal radiation night (fig 3). Despite having no snow cover, Exeter airport were number two in the coldest spots with -3.7°C when the forecast minimum was -1°C. I wonder just how many times they have to get it wrong before they fine tune their forecast for Exeter Airport and get it right?

Figure 3

I would have liked to have added a graphic of what the BBC weather forecast for last night looked like, but the BBC has not made any news bulletin it broadcast yesterday available on iPlayer.

Right kind of snow – wrong kind of place

Figure 1 – Courtesy of Twitter and the Met Office (8 Dec 1303 UTC)

These image on the Met Office Twitter tell the story of the snowfall event on Sunday across England and Wales. Yes, the Met Office got it right, there was snow, and yes the estimated accumulations were quite accurate, but as you can see from these two images the heaviest of the snow was forecast to fall a little bit further north than it actually did. The NWP models ran low Xanthos perhaps 0.5° further north, maybe the later runs of the model nailed the track, but by then the alerts and nice graphics, had been tweeted and posted to Facebook, and it was too late now to make any final course corrections.

Figure 2 – Courtesy of Twitter and the Met Office (8 Dec 1422 UTC)

Even the amber alert they issued was for the wrong area. It was interesting to see that the BBC had even dispatched reporters further north to Llangollen and Nottingham to catch some shots based on this amber area (fig 3), when the reporters should have been in south Wales or High Wycombe to capture the worst of it!

Figure 3 – Courtesy of the Met Office

Here are the snowfall depths reported by the SYNOP stations at 18 UTC yesterday (fig 4). I’ll leave you to make your own minds up about how well they did and how far out they were. I suppose nothing really changes, every Winter we have to show off our total inability to cope with snow to Europe and the rest of the world. I just wonder how we would cope now if we saw a repeat of the snowy winter of 1946-47 or 1978-79?

Figure 4