2023/10/19 by David Kogan, Kogan, David, Dimitrios Diamantidis +5
Biochemistry, Genetics and Molecular Biology · Medicine · #Applications (stat.AP) #Evolution and Genetic Dynamics #FOS: Biological sciences #FOS: Computer and information sciences #FOS: Mathematics #Genetic Mapping and Diversity in Plants and Animals #Mathematical and Theoretical Epidemiology and Ecology Models #Populations and Evolution (q-bio.PE) #Probability (math.PR)
paper · pdf · doi:10.48550/arxiv.2310.12435
openalex publication_date 2023/10/19 · openalex created_date 2023/10/21 · openalex updated_date 2026/07/28
The correlation among the gene genealogies at different loci is crucial in biology, yet challenging to understand because such correlation depends on many factors including genetic linkage, recombination, natural selection and population structure. Based on a diploid Wright-Fisher model with a single mating type and partial selfing for a constant large population with size N, we quantify the combined effect of genetic drift and two competing factors, recombination and selfing, on the correlation of coalescence times at two linked loci for samples of size two. Recombination decouples the genealogies at different loci and decreases the correlation while selfing increases the correlation. We obtain explicit asymptotic formulas for the correlation for four scaling scenarios that depend on whether the selfing probability and the recombination probability are of order O(1/N) or O(1) as N tends to infinity. Our analytical results confirm that the asymptotic lower bound in [King, Wakeley, Carmi (Theor. Popul. Biol. 2018)] is sharp when the loci are unlinked and when there is no selfing, and provide a number of new formulas for other scaling scenarios that have not been considered before. We present asymptotic results for the variance of Tajima's estimator of the population mutation rate for infinitely many loci as N tends to infinity. When the selfing probability is of order O(1) and is equal to a positive constant s for all N and if the samples at both loci are in the same individual, then the variance of the Tajima's estimator tends to s/2 (hence remains positive) even when the recombination rate, the number of loci and the population size all tend to infinity.