US 9,811,074 B1
Optimization of robot control programs in physics-based simulated environment
Fabian Aichele, Leinfelden-Echterdingen (DE); and Albert Groz, Stuttgart (DE)
Assigned to TruPhysics GmbH, Stuttgart (DE)
Filed by TruPhysics GmbH, Stuttgart (DE)
Filed on Jun. 5, 2017, as Appl. No. 15/614,489.
Application 15/614,489 is a continuation in part of application No. 15/188,932, filed on Jun. 21, 2016, granted, now 9,671,777.
Int. Cl. G06F 19/00 (2011.01); G05B 19/4069 (2006.01); G06N 7/04 (2006.01); B25J 9/16 (2006.01)
CPC G05B 19/4069 (2013.01) [B25J 9/1605 (2013.01); B25J 9/1656 (2013.01); B25J 9/1666 (2013.01); G06N 7/04 (2013.01)] 18 Claims
OG exemplary drawing
 
1. A system for optimizing a robot control program in a physics-based simulated environment using machine learning, the system including:
a physically plausible virtual runtime environment to simulate a real environment for a simulated robot;
a test planning and testing component operable to:
define a robotic task;
generate a plurality of virtual test cases for the robotic task;
determine a control strategy for executing the plurality of virtual test cases and initial parameters for the control strategy; and
create the physics-based simulated environment; and
a robot controller operable to:
based on the control strategy, execute the plurality of virtual test cases in parallel in the physics-based simulated environment;
measure a success of the execution of each of the plurality of virtual test cases;
store training and validation data to a historical database, the training and validation data including at least measurements of success of the execution of the plurality of virtual test cases;
train a machine learning algorithm using the training and validation data; and
continuously execute the plurality of virtual test cases while using the machine learning algorithm to adjust parameters of the control strategy until one or more optimal test cases are determined; wherein the robot controller is further operable to: execute the one or more optimal test cases on a physical robot; measure a success of the execution for each of the one or more optimal test cases on the physical robot in view of the robotic task; and based on the measurements of success, select at least one optimal test case from the one or more optimal test cases.
 
18. A system for optimizing a robot control program in a physics-based simulated environment using machine learning, the system including:
a physically plausible virtual runtime environment to simulate a real environment for a simulated robot;
a test planning and testing component operable to:
define a robotic task;
generate a plurality of virtual test cases for the robotic task, wherein generating the plurality of virtual test cases includes:
determining the physically plausible virtual runtime environment for a simulated robot; and
creating a plurality of plausible tests for the control strategy, the plurality of plausible tests being designed to execute the robotic task;
determine a control strategy for executing the plurality of virtual test cases and initial parameters for the control strategy; and
create the physics-based simulated environment, wherein the creating the physics-based simulated environment includes:
generating sensory inputs for the physically plausible virtual runtime environment;
using a machine learning algorithm to map the sensory inputs to commands for the simulated robot to carry out in order to perform the robotic task; and
determining physical forces exerted by the simulated robot and reaction forces of an object on which the physical forces are inserted are simulated; and
a robot controller operable to:
based on the control strategy, execute the plurality of virtual test cases in parallel in the physics-based simulated environment;
measure a success of the execution of each of the plurality of virtual test cases;
store training and validation data to a historical database, the training and validation data including at least measurements of success of the execution of the plurality of virtual test cases;
train a machine learning algorithm using the training and validation data;
continuously execute the plurality of virtual test cases while using the machine learning algorithm to adjust parameters of the control strategy until one or more optimal test cases are determined;
execute the one or more optimal test cases on a physical robot;
measure a success of the execution for each of the one or more optimal test cases on the physical robot in view of the robotic task; and
based on the measurements of success, select at least one optimal test case from the one or more optimal test cases.