declare days_in_month: integer

2.4 Project Costs - Questions B1 to B4

Table 4 shows the costs of each project broken down into staff, hardware, software and other costs. The hardware and software costs vary considerably. Some companies do not cost in the basic network servers because they are included in the company IT budget and not associated with a particular project. Some companies use hardware rigs and software packages supplied free by the client.

Note the low percentage costs of the hardware and software, somewhere at 20% of total cost on some of the smaller projects and down to as low as 5% on the larger. There are hardware (rigs) and software (test code) that are specific to projects but a large proportion of these costs could be amortised over future projects and are therefore, in reality, even lower.

If the percentage costs of software tooling is so low why then the concern by the industry of these costs? The reason may lie in the way defence projects are funded. The Ministry of Defence, for good reasons, tends to fund man-hours but not tools, therefore, as a general rule, the cost of tools is set against the company profits not the total costs.

Returning to the cost of Ada LoC, the data in Table 4 can be plotted, Total Cost of Project vs Lines of Code. Leaving aside the large project which will be off the graph, the remaining projects are represented graphically in Table 6. The red line is the "least squares fit" and shows, for what it's worth, that on average, 1000 lines of Ada code cost £48,000 throughout the entire life of the project.

2.5 Lines of Code per Day (LoC) Question B5

These statistics speak for themselves. Firstly, over the life of a project, they show how few Ada instructions are actually engineered per day. Secondly, they show what a huge discrepancy exists between the best and the worst performing projects, five times, in fact. The Ministry of Defence independently arrived at the same spread and quoted that LoC per engineer day was 2 to 9.

Development Performance related to Project Size

On the evidence, this large spread is clearly unrelated to project size, which is a little surprising. One would expect large amounts of code to incur a great deal of overhead cost due to integration problems. One is tempted to infer that maybe Ada keeps integration costs down, although only three companies specifically mentioned the benefit of Ada in this respect.

Development Performance related to SIL

Nor is there any evidence, from this sample at least, that critical systems (SIL levels > 0) are more labour intensive than non-critical systems (SIL levels = 0). In fact, by removing the first and last entries in Table 7, where the project performance can be explained in other ways, it appears that the higher the SIL level the more code per man-day is achieved. Why this should be so is not easy to explain from the data. It could be conjectured that critical systems can only be developed by the highest quality software engineers who are indeed the best performing coders.

Using all the entries in Table 7 gives the following average performance by SIL:

SIL 0 4.76 LoC per day

Development Performance related to Experience

It is difficult to glean from these statistics why exactly one company can only write 2 LoC per day and another 9 LoC per day. Looking at the lowest performing projects there are two things that may be significant. In one case, the company had great difficulty employing graduate programmers and had to resort to training numerate non-programmers in Ada. In the other case, the company was obliged to share the application development with their French counterparts who were using a different development regime (e.g. C + different testing tools).

A closer examination of the importance of experienced Ada programmers reveals that four companies had evidence that their code performance improved with the experience of their programmers. In particular, one company reported an incremental improvement in performance starting with 4 LoC per day on the first project, rising to 6.6 LoC on the next and 9 on the next. Another reported that initially they were only achieving 2 LoC but this rose to 9 LoC in three years. One company went so far as to say they could eventually achieve 20 LoC per day.

Development Performance relating to Management

Three of the lowest four performing projects were developments split over two sites. No judgement is being made on the wisdom of such decisions; there are good managerial and political reasons for sharing sites. However, the increased burden on management, which is an expensive resource, inevitably brings down overall development performance. On the evidence of one company, sharing developments operating to different development regimes would appear only to exacerbate the problem.

2.6 The Costliest Lifecycle Stages – Questions B6 and B7

The projects that specified their most expensive stages fell into basically two equal camps. In one were those that experienced volatile requirements and therefore lots of design changes. There were four in this category. The other camp was those that rated elements subsequently to the design phase, the coding/testing/integration group. Again there were four projects in this category.

Having identified the most expensive stages the projects were asked to specify where costs were unacceptably high. Two companies recognised that requirements inevitably changed but they were concerned that there was insufficient control and management of these changes. Three companies concentrated more on the testing stages, but their problems were very different. One needed an additional hardware rig, one lacked experienced safety critical test engineers and one found the safety critical SPARK analyses just unacceptable expensive. Finally one company found the integration phase too expensive due to the shortage of one hardware rig. These results are given in Table 8.

2.7 Choice of Language - Question 8

Finally the companies were asked to speculate whether, with the benefit of hindsight, they would have chosen a different language to develop the application.

With possibly one dissenting opinion, the unanimous answer was that they would not have changed the language. One project was not complete but this company was critical of C so it is unlikely that they would change in the future. The dissenting opinion speculated that using C might have reduced costs but insignificantly. Two companies were reviewing their language choice for future projects.

3. Conclusion

In answer to the key question posed in the introduction of this study, that is does Ada make a significant contribution to the efficiency of the software development process, there is no evidence that it does.

The cost of designing, coding, testing, integrating and delivering 1000 lines of Ada code varies from £36,000 to £150,000. Developing safe real time embedded systems is one expensive exercise (!) and Ada is regarded as the only candidate language.

On the other hand, there is no evidence either that the only viable alternative language, which is C, would do a better job of improving the efficiency of development. This opinion is only based on rhetorical evidence since this limited study did not encounter equivalent C based metrics. No company thought there would be a significant reduction in development costs using C instead of Ada.