Summary of the New Statistical-Dynamical Intensity Forecast Models for the Indian Ocean and Southern Hemisphere and Resulting Performance
Final Report prepared for the Navy’s Joint Typhoon Warning Center
Andrea Schumacher (CSU/CIRA), Mark DeMaria (NOAA/NESDIS/StAR), and John Knaff (NOAA/NESDIS/StAR)
October 21, 2013
The statistical-dynamical Statistical Hurricane Intensity Prediction Scheme (SHIPS) and the Logistic Growth Equation Model (LGEM) are operational statistical-dynamical tropical cyclone intensity models currently available for the Atlantic and East and Central Pacific basins. These models provide five day intensity forecasts using predictors from global model forecast fields, ocean analyses and infrared satellite data. The rapid intensification index (RII) estimates the probability of an increase in maximum winds of 30 kt or greater in 24 hours and complements SHIPS and LGEM. Versions of SHIPS were LGEM are were developed for the western North Pacific basin (WP) as part of the Hurricane Forecast Improvement Project (HFIP) for use by the Joint Typhoon Warning Center (JTWC).
The primary objective of this project is to develop operational versions of SHIPS and LGEM, for the Indian Ocean (IO) and Southern Hemisphere (SH), and the RII for the IO, SH and WP to cover the full area of responsibility of JTWC.
In this report, we summarize the development of these new statistical-dynamical forecast models, present the test results, and make recommendations for operational implementation and future improvements.
2. Development of SHIPS, LGEM, and the RII for the IO, SH and WP
This work has built on the HFIP and Joint Hurricane Testbed (JHT) efforts to provide SHIPS/LGEM/RII for the western North Pacific and continue to improve these models for the Atlantic and eastern/central North Pacific. The result of the current project is that the SHIPS/LGEM/RII models are now available for global tropical cyclones. These include five separate versions for (1) the Atlantic (AL), (2) the eastern and central North Pacific (EP/CP), (3) the western North Pacific (WP), (4) the North Indian Ocean (IO) and (5) the Southern Hemisphere (SH). The IO and SH versions of the model were developed under this project, and the WP version was updated.
a. Developmental data Table 1 shows the databases needed for the statistical model development. The JTWC best track data was used for the WP, SH and IO model development. The global model fields were obtained from the operational analyses of the NCEP Global Forecasting System (GFS). The SST fields were from the weekly Reynolds analyses and the OHC fields back to 2005 were from the NCODA analyses. The OHC data before 2005 were obtained from the NOAA/AOML analyses provided by Gustavo Goni. The IR satellite imagery is from MTSAT and Meteosat. The developmental sample includes 2000-2012 for the WP, 1999-2012 for the SH and 1998-2012 for the IO. These periods were determined based on the availability of the OHC analyses and IR satellite data. Figure1 shows the developmental data sample sizes for all five basins. The WP and SH basin sample sizes are comparable to those for the AL and EP/CP, but the IO sample is much smaller. It is possible that there will be some over-fitting of that model at the longer ranges. An effort will be made to increase the size of the IO sample in the future by adding the previous year of data during annual updates, similar to what is done for the AL and EP/CP versions.
The land databases were expanded for the IO and SH versions. The climatological SST, OHC and IR databases are needed so that the models can still run when the real time data is missing. These climatological databases were also expanded to include the IO and SH.
Table 1. Data needed for the statistical model development
Figure 1. The SHIPS model developmental data sample sizes for each basin.
b. Statistical development The coefficients for the SHIPS, LGEM and RII were fit to the data from the five basins. Table 2 shows the 24 predictors for the linear regression based SHIPS model. Predictors 1-4 and 6-9 are only available at t=0. The others are averaged along the storm track for each forecast interval. For the model development, the “perfect prog” approach is used where the track, current intensity and previous 12 h intensity change are from the best track and the atmosphere fields are from analyses, rather than model forecasts. LGEM uses a more sophisticated prediction equation that bounds the solution for the maximum wind to between zero and the maximum potential intensity (MPI) estimated from the SST. LGEM uses all the variables in Table 2 except 3, 4, 10, 14, and 17.
Most of the input variables in Table 2 are calculated in the same way for each basin. The exceptions are the MPI variables (9 and 10) and the shear direction variable (18). The MPI is determined empirically from the SST by fitting an exponential or linear curve to the upper bound of the observed maximum wind estimates versus the SST. New curves were developed for the IO and SH versions. Figure 2 shows the empirical MPI curves for all five basins and the data used to determine these relationships. The optimal shear direction is a function of latitude and reverses direction between low and high latitudes. The latitudes of the reversal were adjusted for each basin to maximize the fit of the SHIPS model.
Figure 3 shows the normalized regression coefficients for the 60 h SHIPS forecast for all five basins. The coefficients for the new basins (IO and SH) are generally similar to those from the other basins, providing some confidence in the statistical fit for the new models. Note that for the SH, the coefficients for the variables that depend on tangential wind or vorticity have opposite signs than those for the rest of the basins.
Figure 4 shows the variance explained by the SHIPS regression and Fig. 5 shows the mean absolute error of the intensity change predicted by the model for the dependent sample. The dependent sample for each forecast time only includes cases that were over water during the entire forecast interval. In real time, the SHIPS forecast is made without land effects and then adjusted using a climatological inland decay rate for the portion of the forecast track that crosses land (Decay-SHIPS). The variances explained and mean absolute error for the IO and SH are similar to those for the other basins, except for the IO at the longer forecast periods. The variance explained at the longer forecast periods for the IO is likely inflated due to the small sample size, resulting in the very low intensity errors. As will be seen in the next section, the very low intensity errors in Fig. 5 do not occur when the models were tested with independent data.
The fitting of the LGEM model is similar to that of the SHIPS model, except that the growth rate is fitting using linear regression rather than the intensity changes. Because LGEM has some constraints built in to the prediction equation, a smaller number of predictors are needed. Similar to SHIPS, the LGEM coefficients for the IO and SH were similar to those for the other basins (not shown).
The Rapid Intensification Index (RII) uses a discriminant analysis to estimate the probability of an increase in the maximum wind of 30 kt or more in the next 24 h. The discriminant function is a linear combination of a subset of the SHIPS predictors that optimally separates the RI from non-RI cases, where the probability of RI increases as the discriminant function increases. Nine predictors were selected as being significant for inclusion in the RII for the IO, SH and WP. Although the operational version of the RII for the AL and EP/CP use a different set of predictors and slightly different formulation, the version developed for the WP, IO and SH was also run for all basins for comparison. Figure 6 shows the discriminant function weights for the all basins. The weights for the IO and SH are similar to those in other basins.
A commonly used metric for probabilistic forecasts is the Brier Skill Score (BSS). The BSS is the percent reduction of the Brier Score, relative to a simple no-skill baseline forecast, where the Brier is the square of the difference between the forecasted and observed probability summed over all forecast cases. The observed probability is 100% if RI occurred and 0% if it did not. For the BSS, the baseline forecast is climatological probability of RI for each basin. Figure 7 shows the BSS for the dependent samples, which shows that the RI is skillful in all baselines.
Figure 2. The empirical MPI curves versus SST for the five basins (blue curves). The orange points show the observed maximum wind versus SST.
Figure 3. The normalized SHIPS regression coefficients for the 60 h SHIPS forecast for the five ocean basins.
Figure 4. The variance of the observed intensity changes explained by the SHIPS model regression for each basin.
Figure 5. The mean absolute error of the SHIPS intensity forecast for the dependent sample.
Figure 6. The normalized discriminant coefficients for the five basins.
Figure 7. The Brier Skill Score for the Rapid Intensification Index for the dependent sample of each basin.
3. Test results
All of the results shown above are for the dependent sample with perfect prog input. As a further test of the new models, all available WP, IO and SH cases from the 2011 and 2012 seasons were run with input that would be available in real time. The track was obtained from the interpolated official JTWC forecast and all of the atmospheric variables were from the GFS model forecasts. Although these cases are not completely independent because 2011-2012 were included in the dependent sample, previous experience with AL and EP/CP basins has shown that when the perfect prog input is replaced with real time input, the results provide a very good estimate of the model accuracy for a fully independent sample. For the runs with real time input, the decay version of SHIPS and LGEM are run.
The 2011-2012 sample did not include any RI cases for the IO. For this reason, the two IO cyclones from the 2013 season were also included in the test sample. The second of these (Cyclone Phailin) was an RI case with maximum winds that reached 140 kt.
Figure 8 shows the sample sizes for the test cases. The WP and SH samples are reasonably large, but the IO sample somewhat marginal for the evaluation.
Figure 9 shows the mean absolute intensity errors and biases for the new models for the WP, IO and SH. The JTWC final best tracks for 2011-2012 and the working best track for the 2013 IO cases were used as ground truth for the evaluation of the forecast errors. Also shown for comparison are the errors from the simple statistical model ST5D, which is sometimes used as a baseline to evaluate forecast skill. This figure shows that both the new models have skill in all three basins since the errors are smaller than those from of the ST5D model. LGEM has a more negative bias than D-SHIPS in all three basins, and the biases of LGEM and D-SHIPS are often of opposite sign. This suggests that a consensus forecast that includes both models may have lower errors then either by itself due to the cancellation of the biases. The errors for the IO do not show any unusual behavior at the longer forecast times, but do not have the very low errors of the dependent sample. This suggests that the possible over-fitting for the IO due to the relatively small dependent sample is not causing any major problems in the real time runs.
The BSS was calculated for the test cases and the results were 14.3%, 22.4% and 10.9% for the WP, IO and SH. These are a little smaller than those shown in Fig.3 for the dependent sample, but the results shown that the RII for the new bases has some skill relative to a climatological forecast.
To give a better idea of the behavior of the RII, Fig. 10 shows the forecasted RI probability for every IO case in the testing sample with a 24 h verification point, and where the initial position was over the water. The observed probabilities (100% or 0% depending on whether or not RI occurred) are also shown. There was only one TC (IO0213) that had periods of RI. Fig. 10 shows that the RI probabilities were less than 25% for all of 2011 and 2012, but were as high as 84% for IO0213. The probabilities for IO0213 were too low at the beginning of the RI period, but became much larger for as the RI period continued.
Figure8. The sizes of the new statistical model test samples.
Figure9. Mean absolute intensity error (solid lines) and bias (dashed lines) for the new Decay-SHIPS and LGEM models for the WP, IO and SH. The errors from the simpler ST5D model are shown for comparison.
Figure 10. The observed and RII probability for all of the IO cases from 2011 to 2013. The x-axis label is the ATCF ID (e.g., IO01), the 2-digit year and date/time of each case.
4. Implementation and Future Plans
The global versions of SHIPS, LGEM and RII and related data and coefficient files have been made available to NRL via an ftp site. Project scientists are coordinating with B. Sampson to install the new code for JTWC.
Several upgrades are planned for SHIPS and LGEM. These include replacing the empirical MPI formulas with a theoretically based formula, extending LGEM to 7 days, and modifying the statistical fitting for LGEM so that the maximum wind forecast is minimized. An experimental version of the RII that includes lightning data as a discriminator is also under development.