U. S. Department of Transportation
Download 2.66 Mb.
|
Methods chosenGiven the discussion above, it is clear that each model has some pros and cons associated with it. As noted earlier, it is important for not only the chosen model to be desirable, but also the comparison models. Recall that the decision criteria for a model to be desirable included tractability, precision, and how well it reflects reality. The specific nature of the runway incursion data does not suggest any particular model choice. Though the data does have some sense of ordering to the categories, multinomial models provide some advantages in terms of analysis, especially as the ordering present in the data may be the result of multiple processes. Due to the nature of the data (i.e., severity ratings from A to D), it was initially desired to focus on the analysis on the ordered family of models. However, as discussed below, the assumptions of the ordered model were not satisfied in many cases.89 This led to the use of multinomial models to relax the ordering constraint. Logit models were chosen over probit models due to computational simplicity, similarity in results when a subset of models were compared head-to-head, and evidence that that the assumption of IIA is not violated with these data.
The models presented below do not contain all of the variables presented in the previous chapter. The results of the tests presented in that section helped inform the modeling process. Though the ordering models’ assumptions may not be satisfied, the results are presented for comparison and completeness. Also note that these results are restricted to only OE incidents, limiting the sample size and the variables that could be included. Finally, the dependent variable is the severity of the incident, with category A being considered most severe and given a rank of four. This model contains variables relating to the aircraft involved at the time of the incident. The results of the ordered model are presented in Table 183. Category D incursions were excluded due to the inclusion of the variable measuring the number of aircraft involved. As Category D incursions involve only one aircraft by definition, including category D incursions obscures the true impact of the number of aircraft on severity. Table – Ordered Logit Model for Aircraft Variables
The sign of the coefficients can be interpreted just as they would for simple logits: positive values increase the likelihood of the incursion being rated as a category A while negative values decrease that likelihood.91 The opposite impact is had for category C incursions – positive indicates less chance of being a category C while negative increases the probability of being category C. The impact on category B is ambiguous and requires further calculations to determine. The signs of variables are consistent with many of the conclusions drawn in Section 3.3. Takeoff is still more dangerous than taxiing. The impact of landing is not statistically different from zero (i.e., the model cannot distinguish if there is a change in probability due to landing or not). Commercial carrier status reduces the likelihood of a category A incident. The daily operations at an airport appear not to impact the likelihood of a category A incident, once these other variables are controlled for. Finally, additional aircraft increase the likelihood of a category A incident. Interestingly, this model satisfies the constraints on the ordered logit model.92 This is likely due to the exclusion of the category D incursions. To be consistent with the other categories, the results of binary logit – similar to those presented in Section 3.3 – and of a multinomial logit are presented below. They are broadly consistent with the ordered model, though the multinomial model provides a more nuanced look at the relationship among these variables. The consistency is not surprising given that the assumptions of the ordered logit are satisfied. Table – Binary Logit of Aircraft Variables
The binary logit results are almost identical to the ordered results – though they are presented in odds ratio form. Recall that category D incursions are excluded, making the alternative category C only. The stability of the relationships indicates that collapsing categories A and B had little impact on the estimate of the effect of these variables. The multinomial results presented below offer a slightly more in-depth look at these effects. Table – Multinomial Logit of Aircraft Variables
With the ability to distinguish between category A and B, some additional insights arise. It is important to note that the total change in probably across categories must equal zero as the total probability across categories is constrained (i.e., you must be in one of these categories, so a reduction in the probability of one category must be countered by an increase in the probability of another). For example, commercial carrier status reduces the probability of a category B incursion by approximately .03 (from approximately p = .047 to approximately p = .017)93. The likelihood of a category A incursions is reduced by approximately .045. Therefore, commercial carrier status increases the likelihood of a category C incursion is increased by approximately .075. Another lesson to take from this is that, although the variable had a similar estimated coefficient between categories, the impact in terms of probability can be different. This is a function of the formulation of the multinomial logit model. Thus, all coefficients must be interpreted in terms of changes in probability within their category, rather than directly compared across categories. The results of the categorical variables (in this case commercial carrier status and aircraft phase of flight) are presented in Table 186. The figures following that table provide the impact of the continuous variables on each category. In both the table and figures, the variables not changing are held at their mean. Table – Change in Probability of Severity Categories for Categorical Variables
Figure – Impact on Probability of Severity Categories of Number of Aircraft 
Figure – Impact on Probability of Severity Categories of Daily Operations, Aircraft The impact of the number of daily operations is fairly slight (not surprising given that the coefficients are not statistically significant). Number of aircraft, on the other hand, appears to increase the probability of category A fairly dramatically as the number of aircraft involved increases. As noted above, the disparity between categories A and B are of interest. The model does not appear to describe the underlying process of category B incursions very well. The variables that appear significant in the ordered model appear to maintain significance only for category A (and only moderately for the number of aircraft). Thus, it appears that the impact of number of aircraft and aircraft phase of flight are localized to category A incursions rather than category B. Finally, it is important to check that the assumptions underlying the multinomial logit model are met. As noted earlier, the major assumption for a multinomial logit model is that of IIA. Testing for violation of IIA revolves around estimating models excluding one alternative at a time and comparing coefficients. While the test statistics and associated p-values are presented, research suggests that these tests are not particularly useful for testing for violations of the IIA assumption.95,96 While the test for violation of IIA is not particularly powerful, it represents the best available test. Additionally, information from the test can be combined with prior knowledge of the categorization (i.e., ranking) system for a better understanding of the IIA issue. The following table presents the results of a test for IIA in this model.97 Insignificant test statistics suggest that the IIA assumption is valid in this case. For this model, the test statistics are insignificant regardless of which outcome is removed. This model provides some interesting insights. First, it appears that amount of daily traffic at an airport does not have an impact on incident severity in the presence of these other variables. This is in contrast to models presented in subsequent section and is likely due to the exclusion of category D incursions. Second, phase of flight (specifically takeoff) appears to impact category A incursions, rather than both categories A and B. This is possibly a definitional effect, rather than a true relationship with severity. Similarly, number of aircraft involved appears to only increase the likelihood of category A incursions rather than both severe categories (although the coefficient is barely significant at a wider 10% criterion; also recall the earlier caution about multiple comparisons). Commercial carrier status appears to reduce the likelihood of both severe categories. This may be related to pilot experience, but it is surprising that that effect would show up for OE incidents as well. This further supports the idea that commercial carriers and GA pilots must be considered separately, even from a controller’s perspective. Table – Results of IIA Test for Aircraft Variables
|