US 11,816,539 B1
Selection system for machine learning module for determining target metrics for evaluation of health care procedures and providers
Marc Granson, Bethlehem, PA (US); Jennifer Shields, Coopersburg, PA (US); and Thomas A. Woolman, Amissville, VA (US)
Assigned to SurgeonCheck LLC, Bethlehem, PA (US)
Filed by SurgeonCheck LLC, Bethlehem, PA (US)
Filed on Mar. 24, 2017, as Appl. No. 15/469,149.
Claims priority of provisional application 62/350,073, filed on Jun. 14, 2016.
Int. Cl. G06N 20/00 (2019.01); G06F 16/22 (2019.01)
CPC G06N 20/00 (2019.01) [G06F 16/22 (2019.01)] 16 Claims
OG exemplary drawing
 
1. A method of optimizing target metric classifications for a medical procedure using a competitive scoring model to select a machine learning model among a plurality of machine learning models, the method comprising:
populating a database with input data related to health care provider quality metrics, input data related to health care facility quality metrics, or input data related to both health care provider quality metrics and health care facility quality metrics;
selecting a medical procedure;
providing a plurality of target metrics for the selected medical procedure, each of the plurality of target metrics being one of the health care provider quality metrics or one of the health care facility quality metrics;
determining an ideal metric value for each of the plurality of target metrics, each ideal metric value being associated with a successful medical outcome of the selected medical procedure;
generating a new metric value for each of the plurality of target metrics for the selected medical procedure using each one of a plurality of machine learning models executed on a processor based on a model training set of data in the database, wherein each one of the plurality of machine learning models is executed in parallel for each of the plurality of target metrics to generate the new metric value to compete against each other to compensate for one or more deficiencies in a particular one of the plurality of machine learning models, and wherein at least one of the plurality of machine learning models include an input layer, one or more hidden layers and an output layer with each of the one or more hidden layers having a plurality of nodes that are connected to the input layer, the output layer, or at least one other hidden layer of the one or more hidden layers;
determining an aggregate score for each distinct combination of (i) one machine learning model of the plurality of machine learning models and (ii) one target metric of the plurality of target metrics, the determination of each aggregate score including (i) performing a plurality of validation tests for the distinct combination of the one machine learning model and the one target metric, each of the plurality of validation tests being a distinct technique for measuring an accuracy of the one machine learning model in the generation of the new metric value of the one target metric, and (ii) adding together a distinct value produced by each of the plurality of validation tests, each aggregate score describing an overall accuracy of each of the plurality of machine learning models in matching the generated new metric values to the ideal metric values;
selecting a first one of the plurality of machine learning models for a first one of the plurality of target metrics, based on the determined aggregate score for each of the plurality of machine learning models and the first one of the plurality of target metrics, the selected first one of the plurality of machine learning models having a lowest aggregate score for the first one of the plurality of target metrics across all of the plurality of machine learning models;
selecting a second one of the plurality of machine learning models for a second one of the plurality of target metrics, based on the determined aggregate score for each of the plurality of machine learning models and the second one of the plurality of target metrics, the selected second one of the plurality of machine learning models having a lowest aggregate score for the second one of the plurality of target metrics across all of the plurality of machine learning models, wherein a number of the plurality of machine learning models are selected based on a number of the plurality of target metrics;
executing the first one of the plurality of machine learning models based on the database when the medical procedure is selected to determine a current value of the first one of the plurality of target metrics; and
executing the second one of the plurality of machine learning models based on the database when the medical procedure is selected to determine a current value of the second one of the plurality of target metrics.