Local bias and its impacts on the performance of parametric estimation models

Ye Yang, Lang Xie, Zhimin He, Qi Li, Vu Nguyen, Barry Boehm, Ricardo Valerdi

Research output: Chapter in Book/Report/Conference proceedingConference contribution

5 Citations (Scopus)

Abstract

Background: Continuously calibrated and validated parametric models are necessary for realistic software estimates. However, in practice, variations in model adoption and usage patterns introduce a great deal of local bias in the resultant historical data. Such local bias should be carefully examined and addressed before the historical data can be used for calibrating new versions of parametric models. Aims: In this study, we aim at investigating the degree of such local bias in a cross-company historical dataset, and assessing its impacts on parametric estimation model's performance. Method: Our study consists of three parts: 1) defining a method for measuring and analyzing the local bias associated with individual organization data subset in the overall dataset; 2) assessing the impacts of local bias on the estimation performance of COCOMO II 2000 model; 3) performing a correlation analysis to verify that local bias can be harmful to the performance of a parametric estimation model. Results: Our results show that the local bias negatively impacts the performance of parametric model. Our measure of local bias has a positive correlation with the performance by statistical importance. Conclusion: Local calibration by using the whole multi-company data would get worse performance. The influence of multi-company data could be defined by local bias and be measured by our method.

Original languageEnglish (US)
Title of host publicationACM International Conference Proceeding Series
DOIs
StatePublished - 2011
Externally publishedYes
Event7th International Conference on Predictive Models in Software Engineering, PROMISE 2011, Co-located with ESEM 2011 - Banff, AB, Canada
Duration: Sep 20 2011Sep 21 2011

Other

Other7th International Conference on Predictive Models in Software Engineering, PROMISE 2011, Co-located with ESEM 2011
CountryCanada
CityBanff, AB
Period9/20/119/21/11

Fingerprint

Industry
Calibration

Keywords

  • Accuracy indicator
  • Effort estimation
  • Local bias
  • Parametric model

ASJC Scopus subject areas

  • Human-Computer Interaction
  • Computer Networks and Communications
  • Computer Vision and Pattern Recognition
  • Software

Cite this

Yang, Y., Xie, L., He, Z., Li, Q., Nguyen, V., Boehm, B., & Valerdi, R. (2011). Local bias and its impacts on the performance of parametric estimation models. In ACM International Conference Proceeding Series [2020404] https://doi.org/10.1145/2020390.2020404

Local bias and its impacts on the performance of parametric estimation models. / Yang, Ye; Xie, Lang; He, Zhimin; Li, Qi; Nguyen, Vu; Boehm, Barry; Valerdi, Ricardo.

ACM International Conference Proceeding Series. 2011. 2020404.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Yang, Y, Xie, L, He, Z, Li, Q, Nguyen, V, Boehm, B & Valerdi, R 2011, Local bias and its impacts on the performance of parametric estimation models. in ACM International Conference Proceeding Series., 2020404, 7th International Conference on Predictive Models in Software Engineering, PROMISE 2011, Co-located with ESEM 2011, Banff, AB, Canada, 9/20/11. https://doi.org/10.1145/2020390.2020404
Yang Y, Xie L, He Z, Li Q, Nguyen V, Boehm B et al. Local bias and its impacts on the performance of parametric estimation models. In ACM International Conference Proceeding Series. 2011. 2020404 https://doi.org/10.1145/2020390.2020404
Yang, Ye ; Xie, Lang ; He, Zhimin ; Li, Qi ; Nguyen, Vu ; Boehm, Barry ; Valerdi, Ricardo. / Local bias and its impacts on the performance of parametric estimation models. ACM International Conference Proceeding Series. 2011.
@inproceedings{fa8771ed15924c439514328ef466de56,
title = "Local bias and its impacts on the performance of parametric estimation models",
abstract = "Background: Continuously calibrated and validated parametric models are necessary for realistic software estimates. However, in practice, variations in model adoption and usage patterns introduce a great deal of local bias in the resultant historical data. Such local bias should be carefully examined and addressed before the historical data can be used for calibrating new versions of parametric models. Aims: In this study, we aim at investigating the degree of such local bias in a cross-company historical dataset, and assessing its impacts on parametric estimation model's performance. Method: Our study consists of three parts: 1) defining a method for measuring and analyzing the local bias associated with individual organization data subset in the overall dataset; 2) assessing the impacts of local bias on the estimation performance of COCOMO II 2000 model; 3) performing a correlation analysis to verify that local bias can be harmful to the performance of a parametric estimation model. Results: Our results show that the local bias negatively impacts the performance of parametric model. Our measure of local bias has a positive correlation with the performance by statistical importance. Conclusion: Local calibration by using the whole multi-company data would get worse performance. The influence of multi-company data could be defined by local bias and be measured by our method.",
keywords = "Accuracy indicator, Effort estimation, Local bias, Parametric model",
author = "Ye Yang and Lang Xie and Zhimin He and Qi Li and Vu Nguyen and Barry Boehm and Ricardo Valerdi",
year = "2011",
doi = "10.1145/2020390.2020404",
language = "English (US)",
isbn = "9781450307093",
booktitle = "ACM International Conference Proceeding Series",

}

TY - GEN

T1 - Local bias and its impacts on the performance of parametric estimation models

AU - Yang, Ye

AU - Xie, Lang

AU - He, Zhimin

AU - Li, Qi

AU - Nguyen, Vu

AU - Boehm, Barry

AU - Valerdi, Ricardo

PY - 2011

Y1 - 2011

N2 - Background: Continuously calibrated and validated parametric models are necessary for realistic software estimates. However, in practice, variations in model adoption and usage patterns introduce a great deal of local bias in the resultant historical data. Such local bias should be carefully examined and addressed before the historical data can be used for calibrating new versions of parametric models. Aims: In this study, we aim at investigating the degree of such local bias in a cross-company historical dataset, and assessing its impacts on parametric estimation model's performance. Method: Our study consists of three parts: 1) defining a method for measuring and analyzing the local bias associated with individual organization data subset in the overall dataset; 2) assessing the impacts of local bias on the estimation performance of COCOMO II 2000 model; 3) performing a correlation analysis to verify that local bias can be harmful to the performance of a parametric estimation model. Results: Our results show that the local bias negatively impacts the performance of parametric model. Our measure of local bias has a positive correlation with the performance by statistical importance. Conclusion: Local calibration by using the whole multi-company data would get worse performance. The influence of multi-company data could be defined by local bias and be measured by our method.

AB - Background: Continuously calibrated and validated parametric models are necessary for realistic software estimates. However, in practice, variations in model adoption and usage patterns introduce a great deal of local bias in the resultant historical data. Such local bias should be carefully examined and addressed before the historical data can be used for calibrating new versions of parametric models. Aims: In this study, we aim at investigating the degree of such local bias in a cross-company historical dataset, and assessing its impacts on parametric estimation model's performance. Method: Our study consists of three parts: 1) defining a method for measuring and analyzing the local bias associated with individual organization data subset in the overall dataset; 2) assessing the impacts of local bias on the estimation performance of COCOMO II 2000 model; 3) performing a correlation analysis to verify that local bias can be harmful to the performance of a parametric estimation model. Results: Our results show that the local bias negatively impacts the performance of parametric model. Our measure of local bias has a positive correlation with the performance by statistical importance. Conclusion: Local calibration by using the whole multi-company data would get worse performance. The influence of multi-company data could be defined by local bias and be measured by our method.

KW - Accuracy indicator

KW - Effort estimation

KW - Local bias

KW - Parametric model

UR - http://www.scopus.com/inward/record.url?scp=80054069256&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=80054069256&partnerID=8YFLogxK

U2 - 10.1145/2020390.2020404

DO - 10.1145/2020390.2020404

M3 - Conference contribution

AN - SCOPUS:80054069256

SN - 9781450307093

BT - ACM International Conference Proceeding Series

ER -