Guide to Advanced Empirical
The Context of Measurement
Download 1.5 Mb. View original pdf
|
3299771.3299772, BF01324126
| 2.2. The Context of Measurement While the context of measurement is typically taken for granted and not examined, it nevertheless has a serious impact on the nature and quality of the measurements. First, the meaning of measurements will vary depending on whether they derive from observation or experiment. If the former, questions of potential bias arise due to various sampling difficulties discussed below. Experiments, on the other hand, while potentially giving precise measurements under controlled conditions, may suffer from alack of generalizability if they are not carefully designed and interpreted. Second, it is often the case that the available measurements are not immediately connected with the phenomena of interest the measures maybe what are termed leading or lagging indicators. The former are highly desirable for forecasting, but the latter are more common both cases are problematic in steering an organization, because the cause and effect are so separated in time. For example, number of customer-reported software defects might seem to be a good metric for evaluating the performance of a software development organization, but it is usually the case that today’s customer complaint stems from a defect introduced months or years ago, perhaps by a different set of developers. Similarly, customer satisfaction is typically measured and goaled on an annual or quarterly basis, but it lags a 158 J. Rosenberg company’s products and services typically by several years. Leading/lagging measures are thus difficult to use in managing day-to-day operations. Third, while measurements are presumably fora purpose, they can often take on a life of their own, produced because someone once decreed they should be produced, but with no-one paying much attention to them because the rationale has been lost, or is no longer meaningful. Worse, the measurement process can have side-effects, where the numbers are massaged or the work process altered in order to produce the right results. Finally, good measurements are actionable; they can be used to do something. Measurements made for measurement’s sake are worse than useless they divert resources from the real problems. A single global measure of customer satisfaction or product quality may alert management to a problem, but it gives no indication of what to do. Overtime, an organization or researcher will sharpen the questions asked and the corresponding metrics used this process forms the most important context for measurement and analysis. Download 1.5 Mb. Share with your friends:
|