6 Statistical Methods and Measurement making process which in turn affects the process of interest, the measurements made on it, and the analyses done on those measurements. It is often the case that too little thought is given to the multilevel nature of this situation measurements are made because it is possible to do so, statistical analyses
are done in a formulaic way, and decisions are made with little data or analysis. In the area of software metrics, Basili et al. (1994) created the “Goal/Question/Metric” framework, which emphasizes that every metric collected must be defined so as to
answer some specific question, and every question posed must be relevant to some decision- making goal. This ensures that the entire process depicted in Fig. 1 remains aligned with the overall goal studying a process in order to make various decisions about it (whether research conclusions or process improvements).
The reason for dwelling on such a banal topic is precisely because it is so often taken for granted problems with any of these processes or the relations between them become easily lost in the assumption that the overall scheme of things is functioning correctly. Yet if the statistical process is not functioning properly (e.g., incorrect analyses are being performed) decisions will be made on the basis of incorrect analysis and bad outcomes maybe misattributed to the decision-making process rather than its statistical inputs. Similarly, it is typically assumed that the measurement process is functioning correctly and that the data it provides are accurate and valid enough to make a statistical analysis worth doing. As Fig. 1 shows, there is no point to a statistical analysis if the data going into it come from a measurement process which is malfunctioning. This involves not only the nature of the measurements involved (discussed in Sect. 3), but also the quality of data obtained.
Share with your friends: