### Abstract

The genetic correlation between a character in two environments is of considerable interest in the context of plant and animal breeding for the prediction of evolutionary trajectories and for the evaluation of the amount of genetic variance maintained at equilibrium in subdivided populations. The two-way analysis of variance with genotype and environment as crossed factors is the usual basis for estimating this genetic correlation. In plasticity experiments, the genetic variance can differ widely between environments, for instance when the variance component associated with the genotype-environment interaction is not constant over environments. When this is the case, the assumption of homoscedasticity is violated, and the ANOVA method tends to underestimate the absolute value of the genetic correlation. To solve this problem, a variance-stabilizing transformation previously applied in a multivariate ANOVA context was developed. This development resulted in a new procedure (method 3), in which the genetic correlation is estimated from the transformed data (i.e. after among-environment heteroscedasticity is removed, while the within-environment means are maintained). In a simulation study and an analysis of Chlamydomonas reinhardtii growth rate data, we compared method 3 with two existing methods in which the genetic correlation is estimated from the raw data. Method 1 uses one 'global' variance component associated with the genotype-environment interaction, and method 2 uses two variance components associated with the genotype and obtained from one-way ANOVAS conducted separately in the two environments. Under increasing among-environment heteroscedasticity, method 1 produces increasingly biased genetic correlation estimates, whereas method 3 almost consistently provides accurate estimates; the performance of method 2 is intermediate, with more estimates out of range or indeterminate. This is the first demonstration that a variance-stabilizing transformation of the data removes the bias in the estimation of genetic correlation caused by among-environment heteroscedasticity, while allowing valid statistical testing in an ANOVA-based approach.

Original language | English (US) |
---|---|

Pages (from-to) | 403-413 |

Number of pages | 11 |

Journal | Heredity |

Volume | 80 |

Issue number | 4 |

State | Published - 1998 |

Externally published | Yes |

### Fingerprint

### Keywords

- Among-environment heteroscedasticity
- Genetic correlation
- Phenotypic plasticity
- Variance-stabilizing transformation

### ASJC Scopus subject areas

- Agricultural and Biological Sciences(all)
- Genetics
- Genetics(clinical)

### Cite this

*Heredity*,

*80*(4), 403-413.

**Among-environment heteroscedasticity and the estimation and testing of genetic correlation.** / Dutilleul, Pierre; Carriere, Yves.

Research output: Contribution to journal › Article

*Heredity*, vol. 80, no. 4, pp. 403-413.

}

TY - JOUR

T1 - Among-environment heteroscedasticity and the estimation and testing of genetic correlation

AU - Dutilleul, Pierre

AU - Carriere, Yves

PY - 1998

Y1 - 1998

N2 - The genetic correlation between a character in two environments is of considerable interest in the context of plant and animal breeding for the prediction of evolutionary trajectories and for the evaluation of the amount of genetic variance maintained at equilibrium in subdivided populations. The two-way analysis of variance with genotype and environment as crossed factors is the usual basis for estimating this genetic correlation. In plasticity experiments, the genetic variance can differ widely between environments, for instance when the variance component associated with the genotype-environment interaction is not constant over environments. When this is the case, the assumption of homoscedasticity is violated, and the ANOVA method tends to underestimate the absolute value of the genetic correlation. To solve this problem, a variance-stabilizing transformation previously applied in a multivariate ANOVA context was developed. This development resulted in a new procedure (method 3), in which the genetic correlation is estimated from the transformed data (i.e. after among-environment heteroscedasticity is removed, while the within-environment means are maintained). In a simulation study and an analysis of Chlamydomonas reinhardtii growth rate data, we compared method 3 with two existing methods in which the genetic correlation is estimated from the raw data. Method 1 uses one 'global' variance component associated with the genotype-environment interaction, and method 2 uses two variance components associated with the genotype and obtained from one-way ANOVAS conducted separately in the two environments. Under increasing among-environment heteroscedasticity, method 1 produces increasingly biased genetic correlation estimates, whereas method 3 almost consistently provides accurate estimates; the performance of method 2 is intermediate, with more estimates out of range or indeterminate. This is the first demonstration that a variance-stabilizing transformation of the data removes the bias in the estimation of genetic correlation caused by among-environment heteroscedasticity, while allowing valid statistical testing in an ANOVA-based approach.

AB - The genetic correlation between a character in two environments is of considerable interest in the context of plant and animal breeding for the prediction of evolutionary trajectories and for the evaluation of the amount of genetic variance maintained at equilibrium in subdivided populations. The two-way analysis of variance with genotype and environment as crossed factors is the usual basis for estimating this genetic correlation. In plasticity experiments, the genetic variance can differ widely between environments, for instance when the variance component associated with the genotype-environment interaction is not constant over environments. When this is the case, the assumption of homoscedasticity is violated, and the ANOVA method tends to underestimate the absolute value of the genetic correlation. To solve this problem, a variance-stabilizing transformation previously applied in a multivariate ANOVA context was developed. This development resulted in a new procedure (method 3), in which the genetic correlation is estimated from the transformed data (i.e. after among-environment heteroscedasticity is removed, while the within-environment means are maintained). In a simulation study and an analysis of Chlamydomonas reinhardtii growth rate data, we compared method 3 with two existing methods in which the genetic correlation is estimated from the raw data. Method 1 uses one 'global' variance component associated with the genotype-environment interaction, and method 2 uses two variance components associated with the genotype and obtained from one-way ANOVAS conducted separately in the two environments. Under increasing among-environment heteroscedasticity, method 1 produces increasingly biased genetic correlation estimates, whereas method 3 almost consistently provides accurate estimates; the performance of method 2 is intermediate, with more estimates out of range or indeterminate. This is the first demonstration that a variance-stabilizing transformation of the data removes the bias in the estimation of genetic correlation caused by among-environment heteroscedasticity, while allowing valid statistical testing in an ANOVA-based approach.

KW - Among-environment heteroscedasticity

KW - Genetic correlation

KW - Phenotypic plasticity

KW - Variance-stabilizing transformation

UR - http://www.scopus.com/inward/record.url?scp=2642648638&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=2642648638&partnerID=8YFLogxK

M3 - Article

VL - 80

SP - 403

EP - 413

JO - Heredity

JF - Heredity

SN - 0018-067X

IS - 4

ER -