Thinking outside the ‘black box’ of catastrophe models takes looking at all data with a critical eye

Actuarial science has long been central to the insurance process. The ability to aggregate information, analyze it, draw assumptions from masses of statistical data and then use that data to effectively set rates and predict losses is central to the business.  But, in recent years, actuarial science has been getting a major assist from another form of statistical analysis more akin to the physical sciences.  Specifically, catastrophe modeling.

In the wake of more frequent and costly natural catastrophes, insurers are turning to a variety of cutting-edge, data-mining tools aimed at helping to achieve a much better handle on predicting the frequency and severity of catastrophe losses.

As a senior vice president at a leader in the new breed of catastrophe modeling firms, Karen Clark & Company, Glen Daraskevich was quick to point out the benefits of such arcane statistical models. But he offered words of caution about letting the models do all of our thinking for us. Such models can be wonderful predictive tools, but as the legendary New York Yankees Manager Yogi Berra once said:  “Prediction is very hard — especially when it’s about the future.”

“Consider the models used to gauge risk transfer in the economic arena prior to the financial crisis,” Daraskevich said. “Over reliance on models that simply were not complete enough to anticipate all of the potential permutations helped lead to the mess that occurred.”

Windstorms edge out fires as main threats

Some of the same concerns surround the field of natural catastrophe modeling, which is still largely in its infancy. But the stakes are high. While actuarial and engineering science has helped the industry make terrific inroads against traditional causes of loss such as structure fires, trend lines related to natural catastrophes are headed in the other direction.

“From the 1970s to today, we have done an absolutely amazing job of managing and reducing fire risk in the U.S.,” Daraskevich said. “We have literally cut fire risk in this country by a third over the last three decades. It is a testament to what can be accomplished with improved construction and greater emphasis on prevention.

“However, over this same time period we saw a steady increase in catastrophe losses associated with natural disasters, principally windstorm,” he added. “As fire losses kept falling, the frequency and severity of catastrophe losses kept rising, until a line was crossed in 1992.”

That line, of course, was Hurricane Andrew. The category 5 hurricane became the worst single insured property loss up to that time, presenting the insurance industry and government with the most expensive cleanup in history, and providing the impetus for the developing science of catastrophe modeling, of which Daraskevich ’s firm, Karen Clark & Company, is a leading practitioner.

What made Hurricane Andrew such a watershed was abundantly clear when Daraskevich showed an archival aerial photo of Miami in 1920 contrasted with a shot taken from the same angle and altitude in 2007. To say that the pictures were barely recognizable as the same city would be an understatement. The Miami of 2007 is a bustling, modern, multi-billion-dollar metropolis as populated and developed as any city on earth, situated in the middle of one of the most active windstorm regions on the planet.

“Hurricane Andrew made it clear that standard actuarial techniques have become ineffective when it comes to predicting the impact of natural catastrophes on such high-value locations,” Daraskevich said. “There are few datapoints because such events are rare, but when they do happen they are hugely expensive.”

GDark1: Daraskevich stresses, “(CAT models) are great tools but you have to understand their strengths and weaknesses and use them appropriately.”

A science is born

Clearly, the insurance industry needed a solution, and they found one in the statistical analysis and catastrophe modeling pioneered by Clark and others. Analyzing post-Andrew data, Karen Clark actually predicted the final Andrew losses within 10 percent, immediately capturing the attention of the insurance industry. Suddenly, the “black box” tool of catastrophe modeling became one of our most important supplementary tools in the prediction of natural catastrophe risk. Over the years, millions of dollars have been spent on CAT models, with the resulting numbers and data often treated like gospel, prompting Daraskevich ’s caution.

“CAT models are fantastic tools,” he said, “but we began to see some of the results going directly into rate filings. That raises some concern. Again, these are great tools but you have to understand their strengths and weaknesses and use them appropriately.”

Daraskevich noted that, like other sciences, CAT modeling relies both on hard data and educated assumptions, the same way that scientists arrive at conclusions. The trouble is, the more scientists you squeeze into the same room, the more and different assumptions you are likely to get.

“Has anyone here ever attended a scientific conference and seen them disagree with each other?” Daraskevich quipped. “It’s fun, believe me. But it underscores the fact that even the smartest, most educated people can see the data very differently. And that makes it risky to base some of your most important business decisions solely on those assumptions.”

Andrew’s ghost gains strength

Interestingly, Daraskevich noted that Hurricane Andrew continued to gain strength more than 10 years after the storm dissipated as scientists continued to slice, dice and argue about the data.  Ultimately, Andrew was officially upgraded from a category 4 to a more serious category 5.

Hurricanes aren’t the only natural catastrophes coming under modeling scrutiny. Earthquakes are less frequent, equally worrisome and even more problematic to model and harder to predict. The United State Geological Survey (USGS) is the definitive authority on earthquake science, and even they change the rules every few years, which can throw the models up in the air. Indeed, like reassessment of the strength of Hurricane Andrew, the USGS has even revisited the strength of perhaps the most celebrated earthquake in U.S. history – the great New Madrid Quake of 1811 – studying newspaper accounts of the day and digging trenches to measure evidence of related soil liquefaction.

“As a result of this research, the USGS actually learned that there are technically no faults around New Madrid, Missouri where the epicenter occurred,” Daraskevich said. “Instead, they believe that there are something like five ‘hypothetical faults’ in the area, faults we know must be there but haven’t actually found yet.”

Daraskevich closed his presentation by referring to perhaps one of the most controversial aspects of catastrophe modeling today – whether or not climate change is actually producing more numerous and violent hurricanes, an issue of urgent concern to the insurance industry. In fact, the early statistical evidence is to the contrary. Rather than an increase in the number and severity of hurricanes, as predicted due to climate change by some theoreticians, it may be that the increased number of hurricanes may be due to improved detection and tracking techniques. Indeed, the research shows an actual downward trend in the number of hurricanes actually making landfalls.

More hurricanes or better observation?

“The models said that hurricanes were going up, but the data says they are not,” Daraskevich said. “We are seeing more hurricanes because observation is better, which if it proves out is a very good thing for the insurance industry.”

Bottom line, catastrophe models can be immensely useful to the insurance industry, but over reliance on them can lead to the wrong assumptions.

What models can do:

• Provide a framework for tying together the three main components of risk:  hazard, engineering, exposure
• Provide many scenarios of what could happen and estimates of loss from different types of events
• Deliver rough estimates of probabilities of losses of different sizes

What models can’t do:

• Produce accurate point estimates such as 1 in 100 and 1 in 250 loss amounts
• Produce credible estimates of losses for specific locations
• Predict “near term” catastrophe losses

Click here to view Daraskevich’s presentation.

Contact Info:

Glen Daraskevich
Sr. Vice President
Karen Clark & Company
6174.423.2800, Ext. 204
gdaraskevich@kcc.us.com
www.kcc.us.com