↓ Skip to main content

Improving machine learning reproducibility in genetic association studies with proportional instance cross validation (PICV)

Overview of attention for article published in BioData Mining, April 2018
Altmetric Badge

Mentioned by

twitter
1 X user

Citations

dimensions_citation
6 Dimensions

Readers on

mendeley
31 Mendeley
You are seeing a free-to-access but limited selection of the activity Altmetric has collected about this research output. Click here to find out more.
Title
Improving machine learning reproducibility in genetic association studies with proportional instance cross validation (PICV)
Published in
BioData Mining, April 2018
DOI 10.1186/s13040-018-0167-7
Pubmed ID
Authors

Elizabeth R. Piette, Jason H. Moore

Abstract

Machine learning methods and conventions are increasingly employed for the analysis of large, complex biomedical data sets, including genome-wide association studies (GWAS). Reproducibility of machine learning analyses of GWAS can be hampered by biological and statistical factors, particularly so for the investigation of non-additive genetic interactions. Application of traditional cross validation to a GWAS data set may result in poor consistency between the training and testing data set splits due to an imbalance of the interaction genotypes relative to the data as a whole. We propose a new cross validation method, proportional instance cross validation (PICV), that preserves the original distribution of an independent variable when splitting the data set into training and testing partitions. We apply PICV to simulated GWAS data with epistatic interactions of varying minor allele frequencies and prevalences and compare performance to that of a traditional cross validation procedure in which individuals are randomly allocated to training and testing partitions. Sensitivity and positive predictive value are significantly improved across all tested scenarios for PICV compared to traditional cross validation. We also apply PICV to GWAS data from a study of primary open-angle glaucoma to investigate a previously-reported interaction, which fails to significantly replicate; PICV however improves the consistency of testing and training results. Application of traditional machine learning procedures to biomedical data may require modifications to better suit intrinsic characteristics of the data, such as the potential for highly imbalanced genotype distributions in the case of epistasis detection. The reproducibility of genetic interaction findings can be improved by considering this variable imbalance in cross validation implementation, such as with PICV. This approach may be extended to problems in other domains in which imbalanced variable distributions are a concern.

X Demographics

X Demographics

The data shown below were collected from the profile of 1 X user who shared this research output. Click here to find out more about how the information was compiled.
Mendeley readers

Mendeley readers

The data shown below were compiled from readership statistics for 31 Mendeley readers of this research output. Click here to see the associated Mendeley record.

Geographical breakdown

Country Count As %
Unknown 31 100%

Demographic breakdown

Readers by professional status Count As %
Student > Ph. D. Student 5 16%
Student > Bachelor 4 13%
Student > Doctoral Student 4 13%
Student > Postgraduate 4 13%
Student > Master 4 13%
Other 6 19%
Unknown 4 13%
Readers by discipline Count As %
Computer Science 10 32%
Biochemistry, Genetics and Molecular Biology 5 16%
Agricultural and Biological Sciences 4 13%
Medicine and Dentistry 3 10%
Business, Management and Accounting 1 3%
Other 4 13%
Unknown 4 13%
Attention Score in Context

Attention Score in Context

This research output has an Altmetric Attention Score of 1. This is our high-level measure of the quality and quantity of online attention that it has received. This Attention Score, as well as the ranking and number of research outputs shown below, was calculated when the research output was last mentioned on 24 April 2018.
All research outputs
#20,483,282
of 23,045,021 outputs
Outputs from BioData Mining
#290
of 310 outputs
Outputs of similar age
#288,447
of 327,386 outputs
Outputs of similar age from BioData Mining
#5
of 5 outputs
Altmetric has tracked 23,045,021 research outputs across all sources so far. This one is in the 1st percentile – i.e., 1% of other outputs scored the same or lower than it.
So far Altmetric has tracked 310 research outputs from this source. They typically receive more attention than average, with a mean Attention Score of 7.7. This one is in the 1st percentile – i.e., 1% of its peers scored the same or lower than it.
Older research outputs will score higher simply because they've had more time to accumulate mentions. To account for age we can compare this Altmetric Attention Score to the 327,386 tracked outputs that were published within six weeks on either side of this one in any source. This one is in the 1st percentile – i.e., 1% of its contemporaries scored the same or lower than it.
We're also able to compare this research output to 5 others from the same source and published within six weeks on either side of this one.