Local bias and its impacts on the performance of parametric estimation models

Ye Yang, Lang Xie, Zhimin He, Qi Li, Vu Nguyen, Barry Boehm, Ricardo Valerdi

Research output: Chapter in Book/Report/Conference proceedingConference contribution

6 Scopus citations

Abstract

Background: Continuously calibrated and validated parametric models are necessary for realistic software estimates. However, in practice, variations in model adoption and usage patterns introduce a great deal of local bias in the resultant historical data. Such local bias should be carefully examined and addressed before the historical data can be used for calibrating new versions of parametric models. Aims: In this study, we aim at investigating the degree of such local bias in a cross-company historical dataset, and assessing its impacts on parametric estimation model's performance. Method: Our study consists of three parts: 1) defining a method for measuring and analyzing the local bias associated with individual organization data subset in the overall dataset; 2) assessing the impacts of local bias on the estimation performance of COCOMO II 2000 model; 3) performing a correlation analysis to verify that local bias can be harmful to the performance of a parametric estimation model. Results: Our results show that the local bias negatively impacts the performance of parametric model. Our measure of local bias has a positive correlation with the performance by statistical importance. Conclusion: Local calibration by using the whole multi-company data would get worse performance. The influence of multi-company data could be defined by local bias and be measured by our method.

Original languageEnglish (US)
Title of host publicationPROMISE 2011 - 7th International Conference on Predictive Models in Software Engineering, Co-located with ESEM 2011
DOIs
StatePublished - Oct 19 2011
Externally publishedYes
Event7th International Conference on Predictive Models in Software Engineering, PROMISE 2011, Co-located with ESEM 2011 - Banff, AB, Canada
Duration: Sep 20 2011Sep 21 2011

Publication series

NameACM International Conference Proceeding Series

Other

Other7th International Conference on Predictive Models in Software Engineering, PROMISE 2011, Co-located with ESEM 2011
CountryCanada
CityBanff, AB
Period9/20/119/21/11

Keywords

  • Accuracy indicator
  • Effort estimation
  • Local bias
  • Parametric model

ASJC Scopus subject areas

  • Software
  • Human-Computer Interaction
  • Computer Vision and Pattern Recognition
  • Computer Networks and Communications

Fingerprint Dive into the research topics of 'Local bias and its impacts on the performance of parametric estimation models'. Together they form a unique fingerprint.

  • Cite this

    Yang, Y., Xie, L., He, Z., Li, Q., Nguyen, V., Boehm, B., & Valerdi, R. (2011). Local bias and its impacts on the performance of parametric estimation models. In PROMISE 2011 - 7th International Conference on Predictive Models in Software Engineering, Co-located with ESEM 2011 [2020404] (ACM International Conference Proceeding Series). https://doi.org/10.1145/2020390.2020404