↓ Skip to main content

Assessing the net benefit of machine learning models in the presence of resource constraints

Overview of attention for article published in Journal of the American Medical Informatics Association, February 2023
Altmetric Badge

About this Attention Score

  • In the top 5% of all research outputs scored by Altmetric
  • Among the highest-scoring outputs from this source (#22 of 3,305)
  • High Attention Score compared to outputs of the same age (99th percentile)
  • High Attention Score compared to outputs of the same age and source (97th percentile)

Mentioned by

news
26 news outlets
twitter
50 X users

Citations

dimensions_citation
4 Dimensions

Readers on

mendeley
14 Mendeley
You are seeing a free-to-access but limited selection of the activity Altmetric has collected about this research output. Click here to find out more.
Title
Assessing the net benefit of machine learning models in the presence of resource constraints
Published in
Journal of the American Medical Informatics Association, February 2023
DOI 10.1093/jamia/ocad006
Pubmed ID
Authors

Karandeep Singh, Nigam H Shah, Andrew J Vickers

Abstract

The objective of this study is to provide a method to calculate model performance measures in the presence of resource constraints, with a focus on net benefit (NB). To quantify a model's clinical utility, the Equator Network's TRIPOD guidelines recommend the calculation of the NB, which reflects whether the benefits conferred by intervening on true positives outweigh the harms conferred by intervening on false positives. We refer to the NB achievable in the presence of resource constraints as the realized net benefit (RNB), and provide formulae for calculating the RNB. Using 4 case studies, we demonstrate the degree to which an absolute constraint (eg, only 3 available intensive care unit [ICU] beds) diminishes the RNB of a hypothetical ICU admission model. We show how the introduction of a relative constraint (eg, surgical beds that can be converted to ICU beds for very high-risk patients) allows us to recoup some of the RNB but with a higher penalty for false positives. RNB can be calculated in silico before the model's output is used to guide care. Accounting for the constraint changes the optimal strategy for ICU bed allocation. This study provides a method to account for resource constraints when planning model-based interventions, either to avoid implementations where constraints are expected to play a larger role or to design more creative solutions (eg, converted ICU beds) to overcome absolute constraints when possible.

X Demographics

X Demographics

The data shown below were collected from the profiles of 50 X users who shared this research output. Click here to find out more about how the information was compiled.
Mendeley readers

Mendeley readers

The data shown below were compiled from readership statistics for 14 Mendeley readers of this research output. Click here to see the associated Mendeley record.

Geographical breakdown

Country Count As %
Unknown 14 100%

Demographic breakdown

Readers by professional status Count As %
Student > Ph. D. Student 3 21%
Professor 2 14%
Unspecified 1 7%
Professor > Associate Professor 1 7%
Lecturer 1 7%
Other 1 7%
Unknown 5 36%
Readers by discipline Count As %
Medicine and Dentistry 3 21%
Mathematics 2 14%
Unspecified 1 7%
Pharmacology, Toxicology and Pharmaceutical Science 1 7%
Nursing and Health Professions 1 7%
Other 1 7%
Unknown 5 36%
Attention Score in Context

Attention Score in Context

This research output has an Altmetric Attention Score of 240. This is our high-level measure of the quality and quantity of online attention that it has received. This Attention Score, as well as the ranking and number of research outputs shown below, was calculated when the research output was last mentioned on 02 January 2024.
All research outputs
#156,871
of 25,394,081 outputs
Outputs from Journal of the American Medical Informatics Association
#22
of 3,305 outputs
Outputs of similar age
#4,199
of 425,786 outputs
Outputs of similar age from Journal of the American Medical Informatics Association
#2
of 46 outputs
Altmetric has tracked 25,394,081 research outputs across all sources so far. Compared to these this one has done particularly well and is in the 99th percentile: it's in the top 5% of all research outputs ever tracked by Altmetric.
So far Altmetric has tracked 3,305 research outputs from this source. They typically receive a lot more attention than average, with a mean Attention Score of 14.8. This one has done particularly well, scoring higher than 99% of its peers.
Older research outputs will score higher simply because they've had more time to accumulate mentions. To account for age we can compare this Altmetric Attention Score to the 425,786 tracked outputs that were published within six weeks on either side of this one in any source. This one has done particularly well, scoring higher than 99% of its contemporaries.
We're also able to compare this research output to 46 others from the same source and published within six weeks on either side of this one. This one has done particularly well, scoring higher than 97% of its contemporaries.