*Software Engineering* **27**(11), 1999–1013. Novo, A. (2002), Analysis of multivariate normal datasets with missing values. Ported to R by Alvaro A. Novo. Original by J.L. Schafer. R Development Core Team (2005), *R: A language and environment for statistical computing*, R Foundation for Statistical Computing, Vienna, Austria. ISBN 3-900051-07-0. http://www. R-project.org Roth, PL, Missing data A conceptual review for applied psychologist, *Personel **Psychology* **47**, Rubin, DB, *Multiple Imputation for Nonresponse in Surveys*, John Willey & Sons. Schafer, J. L. (1997), *Analysis of Incomplete Data*, Monograph on Statistics ans Applied Probability, Chapman & Hall. Schafer, J. L. & Olsen, MK, Multiple imputation for multivariate missing data problems, *Multivariate Behavioural Research* **33**(4), 545–571. Schafer, J. S. (1999), Software for multiple imputation. http://www.stat.psu.edu/˜jls/misoftwa. html Strike, K, Emam, K. E. & Madhavji, N. (2001), Software cost estimation with incomplete data, *IEEE Transactions on Software Engineering* **27**(10), Swanson, E. B. (1976), The dimensions of maintenance, *in* ‘Proc. 2nd Conf. on Software Engineering, San Francisco, pp. 492–497. Twala, B, Cartwright, M. & Shepperd, M. (2006), Ensemble of missing data techniques to improve software prediction accuracy, *in* ‘ICSE’06’, ACM, Shanghai, China, pp. 909–912. Weisberg, S. (1985), *Applied Linear Regression, 2nd Edition*, John Wiley & Sons, USA.
**Index**abduction 318 abductive inference 317, 318 ACM code of ethics 230, 233, 237 ACM SigDoc action research 258, 301–303, 307, actor analytical phase of research 99, anomalies in data anonymity 238–239 ANOVA autonomy 232, available case 193 Basili’s framework of experimentation in software engineering 373–375, Bayesian analysis 90, behavioural sciences benchmark 65, beneficence between subjects design bias 77, 297, 299, 302, 306, 346, 347, 351, 352, 354, binomial data 90, 166, 169, 171, 172, blinding 207, blocks 295 Bonferroni procedure Brainstorming 11, 12–13, 97, Calibration 121, case study 50–51, 296–298, 313, 327, case study confirmatory 296, exploratory 296, 328 casewise deletion categorical data 166, 169, 171, causality 289, censoring central tendency 165, classical measurement theory 159 CMM 80, codes preformed 49 postformed coding phase of simulation 143, coding axial 49, selective 50, coercion 235–236, 247, cold fusion Common Rule the 241, compensation complete case, method compound metrics conceptual development conceptual modeling conclusion validity confidence in empirical results 337–338, confidence interval confidentiality 102, 216, 232, 238–240, 246, 248, 253, confidentiality of data 232, consent form 234, 250, 252, 253–254, constant comparison method construct validity 80, 161, 222, 223, 305 constructivism 291, 304 constructivists 290, 291, 293, 298, 302, constructs 223, 313, content analysis 59–60, content validity 80, context information 21, 24, 202, controlled experiment 291, 294–296, 306, convenience sampling correlation 166, cover letter 245 383
384 Index criterion validity 80, critical theorists 291–292, 298, critical theory 291–292, 302 Cronbach’s alpha 79, 161, 221 cross-case analysis cultural conflict data analysis 46, anomalies in binomial 90, 166, 169, 171, 172, categorical cleaning collection 37, 219, 279, 297, 303, confidentiality of 232, missing 89, 180–181, see also Chapter 7 multinomial nominal objective ordinal 90, 166–167, 169, quality 178–182, 189, subjective 58, validation 88–89, 178, visualization of deception 234–235, deductive methods 318 (also figure, pg. deletion techniques 186, dependent variable 187, 209, 217, descriptive statistics 97, 156, 161, 163, 164, design documents 118, direct observation 37, disclosure full partial discrimination validity dispersion 165, dynamic analysis effect size 221, 352, 354, 355, EMPEROR approach empirical research project 257, Empirical Software Engineering Journal empirical support 318, 319, 332 empirically-based theory 312, empiricists epistemology equivalence testing 168 ERB ethics review boards ethnographic research ethnography 21, 299, 300–301, evaluating theories evaluative phase of research 99, expedited review 244, Experience Factory 356, experience portal experimental unit 84, 181, 207, explanatory power 316, 319, 320, 332, explanatory theory EXTEND external validity 222, 223, 295, face validity 80, fairness 346 falsifiability field memo field notes 39, 45, 48, 49 FIML focus groups 12–13, 77, see also Chapter forecasting full information maximum likelihood function points 132, 133, fusion, cold 370 Goal/Question/Metrics (GQM) 206, 212, 215, 340, goodness of a theory Grounded theory 48, 293, 317, group mean substitution Group Support Systems 102–105 GSS 102–105 Hackystat Hawthorne effect 308 IEEE-CS/ACM code of ethics 230, 233, imputation hot deck 192, 194 k-nearest neighbor mean multiple 192, independent variable 217, 219, induction 293, 297, inductive analysis 372 industrially-sponsored project inferential statistics 156, 163, inflows 127, informational phase of research 99, informed consent 232–236, input parameter values Institutional Review Board 241 instrumentalism 316, 318
Index integrating evidence 352, intellectual property 262, 266, 276 inter-coder reliability (see also inter-rater reliability) internal validity 208, 223, 306 interpretivism 35–36, 291 inter-rater reliability 29, 79, 167, interval scale interview 13–15, 43–46, 82, 237, 268, 298–299, guide 45, scribe 45 semi-structured 14, 44, structured 13–14, 44, 105, unstructured 44, 45 Kaplan-Meier Estimate 190 Kendall’s tau 29, 90, 167 Kruskal-Wallis test laboratory package level variables 127, 128, 136, 138, life cycle 120 listwise deletion local theory 291, 292, 301, logistic regression 90, longitudinal study 14, 22, 68, 173, 190, 276, mapping studies MAR mathematical theory 160 MATLAB 126 MCAR mean absolute deviation mean absolute percentage error mean percentage error mean squared error mean substitution measurement 74, precision process scale 80, measurement theory 159, measures of association measures of dispersion median medical research, evidence in mental model 11, 16, 19, 1 18, member checking 53–54 meta-analysis 221, 225, qualitative 293, 309, see also Chapter quantitative 36, 60, 97, metric definition metric evaluation MI minimal risk minimize harm Mining Software Repositories, International Workshop on 25, missing at random missing completely at random missing data 89, 180–181, see also Chapter mixed methods 60, model calibration 121, model validation 120, moderator 94, modes 164 MSR 25, 26 multinomial data multiple imputation multiple regression 162, 192, multivariate regression 170 N-version programming negative case analysis 52 NMAR nominal data nominal scale 160 non-parametric model 163 non-parametric test 29–30, 163, 167, nonrandom sampling not missing at random null hypothesis 167, 168, 222, objective data objectivity 54, 48, 345, 346 object-oriented system, maintainability of observation 11, 12, 20–21, 37–43, 243, 244, 246, 250, ontology open coding 49, open questions 71 open-ended question 14, 44, 67, openness 345, ordinal data 90, 166–167, 169, ordinal scale 73–74, 76, 90, 160, organizational characteristics outflows 127 outlier 52–53, 164, output parameter values overrun of software projects 67, 350
pairwise deletion parametric models participant observation 21–22, 37–43, 234, 250, peer review of studies 275, 342, 349, 358, 366, 367 perspective-based reading 212, 215, 217, philosophical stance 36, physical theory pilot test 70, 78, 83, positivism 35–36, 291, pragmatism 292, 313, 316, pragmatists 292, prediction 163, 169–173, 175–176, 292, 314, 315, 319, probability sampling 86, problem statement 119–120, 204, 206, process control process improvement 120, 187, 258, 292, product quality 46, 122, 158, 226, 238, program comprehension 19, 315, project cost estimation project description 244, project management 66, 122, 123, propositional phase of research 99, proposition 48, 50–52, 54, 296, 305, 313, 316, 19, 320, 322, 323, publication bias 360 purposive sampling 94, 297 QSIM 125 QUAF qualitative analysis of causal feedback quasi experiment 202, questionnaire 11, 15–16, 65, 70, 74, 242, 245, 298–299, 329, 373, 376, see also Chapter questionnaire specification questions base-rate causality exploratory knowledge 289 open-ended 14, 44, 67, relationship quota sampling random sample randomization 85–86, 207, 215, rate variables 127, 128, 132, 136, rater agreement exercises ratio scale rationalists recruitment of subjects 239, 246, reductionist 291, reference behaviour 119, regression equation regression methodology 170, regression substitution reliability 59, 77–79, 80–81, 160, 161, repeated measures analysis replication 53, 214, 216, 240, 302, 326, see also Chapter replication close 375, criteria for differentiated exact external 365, information required to allow internal 365, partial reporting guidelines for reporting guidelines abstracts acknowledgements analysis analysis procedure appendices authorship background conclusion deviations from the plan discussion experiment design experiment planning goals hypotheses introduction keywords materials participants procedure references results task threats to validity title representative sample 68, 82, 295, requirements 11, 98, 104, 120, 123, 130, 144, 186, 317, 322, requirements prioritization 98, research design 294, 309, method question 100, 287–290, 309 386 Index
Index Research Ethics Board 233, response rate 15, 75, 87, 268, risk management 66, 98, 118, robustness sample size 87, 88, 94, 163, 168, 175, 185, 186, representative 68, 82, 295, bias 178, 181, 182, method 83, unit chain cluster based convenience 86 purposive 94, quota random snowball stratified 85, systematic scientific review scientific value 240, 244, scope 120, 319, 320, scope conditions 316, 323, searching for empirical evidence segmentation of participants sensitivity analysis 191, shadowing 11, 20–21, 38, 232, 237, synchronized 20, 38, similar response pattern imputation simple metrics 158–159 Simpson-traction replication simulation model deterministic dynamic 123 event-driven static stochastic 122–123 simulation DE 124, discrete event 124, hybrid modeling tools qualitative quantitative SD 127–128, system dynamic 127–128, verification 121–122, skewed distribution 159, skill 267, software engineering theory 314, software industry software metrics 157, 160, 170, 171, software process 13, 21, 79, 117, 120, 148, software process improvement 13, software reliability software system 314, 321, specificity static analysis statistical analysis 29, 156–157, statistical power 168, 215, 353, statistical significance 222, 269, stratified random sample 85, structured abstract subject information sheet subject selection of vulnerable 233, 243, subjective data 58, substitution techniques survey design documentation instrument instrument evaluation reliability survey research 298–299, see also Chapter survival curve synchronized shadowing 20, 38, systematic literature review 340, target population 82, technology transfer 99, 100, 262–263 testability 319, theory 313, theory building 292, theory development 318, theory generation 48 think-aloud 11, 19–20, threats to validity 222–223, 269, time series 173, 174, time sheets 11, 17, transcription 19, 29, treatment triangulate 30, 52, triangulation 30, 52, 132, 304, concurrent truncation 159 t-test type I error 168, type II error 168, 223 UML unit of analysis 297
validity 52, 80, 161, 222–223, 305–307, construct 80, 161, 222, 223, content 80, conclusion criterion 80, discrimination external 222, 223, 295, face 80, internal 208, 223, 306 VENSIM 126, verification 121–122, visualization of data 54–57 voluntariness 232–233, 235, 236, 248, 252, vulnerable subjects 233, 243, weighting of evidence sources 354, 358 Wilcoxon/Mann-Whitney test within subjects design work diaries 11, 17–18 388 Index
**Share with your friends:** |