Saturday, 24 April 2010

Put not thy faith in Computer models

Back in the late eighties, an old chum of mine who sailed racing yachts excitedly showed me a piece of computer software running on his eldest son's Atari (Remember when we thought those were the Mutts Nuts?). Said software did a modest job of calculating water flows around a Yacht keel and other such relatively simple shapes. Of course it completely failed to show things like trailing vortices and other such drag related phenomena.

Modern flow modelling software I am assured, is a little more sophisticated than way back then, but when all the calculations have been done, you still need to build and test physical half or full scale replicas to see how much difference various finishes, coatings and anti-fouling treatments will have on the finished product. There is software that can perform that task, but these things never quite turn out the way the models demonstrate under all conditions. This much is accepted.

Don't get me wrong. Computer models have their uses and can stop you making stupid mistakes. Flight simulator software is nowadays terrific for training pilots on how to deal with various emergencies. The utter brilliance of it always brings an admiring smile to my face. Well, I'm easily pleased. Said flight models have been hammered out by Software Engineers with feedback from professional pilots, rather like Racing car designers interface with the drivers. Driver flags up a concern and the Engineers get to work on dealing with the issue, be that a bit of drift on a fast left or brake fade. You need the physical to improve the theoretical. This is basic Engineering practice to make things better, stronger and faster. It works, don't knock it.

When it comes to weather however, as the recent volcano plume fiasco has demonstrated, the current computer weather models often don't reflect reality. When the satellites and sampling aircraft went to check the volcano plumes reality against the models, they found that the models were, not wrong, but, how should I put this, unrepresentative. The Met office models calculated that the dust cloud would be enough to damage Jet engines. When the flights finally went up to check the model results, they found that yes, there was volcanic dust up there, but not what the models said.

There was also the issue of the Engine makers being asked what the tolerance was for particulate matter passing through a jet turbine. Apparently the answer came back that Jet engines should under no circumstances be exposed to volcanic ash. Considering I've been on flights that have landed in hailstorms and torrential downpours without much incident, this gave me pause for thought. Jets land and take off through polluted air / extreme weather every hour of the day all around the world. The only time volcanic ash has damaged jet engines is when such an aircraft has been flown through a sufficient concentration of airborne particulates that would choke an unprotected human. Like the Speedbird 9 incident in 1982 or the Finnish F-18's and NATO F-16's that reportedly suffered engine damage from the Eyjafjallajökull plume.

People reported clear skies while the flight ban was in place. There was also some reported ash fallout in the UK, but having examined a few press photographs of the fallout on cars, I saw less 'volcanic dust' on the cars portrayed than the recent Birch pollen we've been getting over here that lands on my old 4x4. Needless to say, said Birch pollen has not stopped the local aircraft buzzing around. All right, that's bit unfair, not comparing like with like, but that's what happened with the 'no-fly' ban. It wasn't until the dust was sampled that the 'oops' factor struck, and the picture the models were feeding found to be at odds with reality.

My point is simply this. Models succeed when they are based in reality, not theory. Theoretical models are usually heavily modified or even discarded if found to be misrepresentative. Not so with weather. The current models are shown not to predict as well as advertised, but there seems to be this boneheaded persistence in insisting that they should, come what may.

"Make it work." A vociferous lobby is saying. "We want this to be true." I will not speculate on their reasons, whatever they might be, here. What remains is that the current Met Office models are demonstrably inadequate, and have been for some time. You can want something to be true all you like, but that will not make it so. Your model may tell you the world is flat, but that won't stop people circumnavigating the Earth, or change the curve of the horizon. Nor can a computer model conjure snow on a cloud free day. The world and the weather simply don't work that way.

No comments:

Related Posts with Thumbnails