US 11,832,582 B2
Vision system for leg detection
Mark A. Foresman, Houston, TX (US); Bradley J. Prevost, Pearland, TX (US); Marijn Van Aart, Marknesse (NL); Peter Willem van der Sluis, Ijsselmuiden (NL); and Alireza Janani, Sugarland, TX (US)
Assigned to Technologies Holdings Corp., Houston, TX (US)
Appl. No. 16/092,333
Filed by Technologies Holdings Corp., Houston, TX (US)
PCT Filed Aug. 17, 2017, PCT No. PCT/US2017/047365
§ 371(c)(1), (2) Date Oct. 9, 2018,
PCT Pub. No. WO2018/035336, PCT Pub. Date Feb. 22, 2018.
Application 16/092,333 is a continuation of application No. 15/455,382, filed on Mar. 10, 2017, granted, now 10,477,827.
Application 15/455,382 is a continuation of application No. 15/448,821, filed on Mar. 3, 2017, granted, now 10,499,609.
Application 15/448,821 is a continuation of application No. 15/448,761, filed on Mar. 3, 2017, granted, now 10,499,607.
Application 15/448,761 is a continuation of application No. 15/448,914, filed on Mar. 3, 2017, granted, now 10,349,615.
Application 15/448,914 is a continuation of application No. 15/448,854, filed on Mar. 3, 2017, granted, now 10,349,613.
Application 15/448,854 is a continuation of application No. 15/448,879, filed on Mar. 3, 2017, granted, now 10,349,614.
Application 15/448,879 is a continuation of application No. 15/239,526, filed on Aug. 17, 2016, granted, now 9,974,278.
Application 15/239,526 is a continuation of application No. 15/239,597, filed on Aug. 17, 2016, granted, now 9,980,457.
Application 15/239,597 is a continuation of application No. 15/239,559, filed on Aug. 17, 2016, granted, now 9,936,670.
Application 15/239,597 is a continuation of application No. 16/092,333.
Application 16/092,333 is a continuation of application No. 15/239,477, filed on Aug. 17, 2016, granted, now 9,807,972, issued on Nov. 7, 2017.
Application PCT/US2017/047365 is a continuation of application No. 15/239,425, filed on Aug. 17, 2016, granted, now 9,984,470.
Application PCT/US2017/047365 is a continuation of application No. 16/092,333.
Application 16/092,333 is a continuation of application No. 15/239,300, filed on Aug. 17, 2016, granted, now 9,807,971, issued on Nov. 7, 2017.
Prior Publication US 2020/0344972 A1, Nov. 5, 2020
Int. Cl. A01J 5/00 (2006.01); A01J 5/007 (2006.01); A01J 5/017 (2006.01); G06T 7/73 (2017.01); G06V 10/25 (2022.01); G06V 10/44 (2022.01); G06V 10/50 (2022.01); G06V 10/75 (2022.01); G06V 20/10 (2022.01); G06V 20/64 (2022.01); H04N 13/204 (2018.01); H04N 13/271 (2018.01)
CPC A01J 5/007 (2013.01) [A01J 5/017 (2013.01); A01J 5/0175 (2013.01); G06T 7/75 (2017.01); G06V 10/25 (2022.01); G06V 10/44 (2022.01); G06V 10/50 (2022.01); G06V 10/751 (2022.01); G06V 20/10 (2022.01); G06V 20/647 (2022.01); H04N 13/204 (2018.05); H04N 13/271 (2018.05); G06T 2207/10028 (2013.01); G06T 2207/30204 (2013.01); G06V 2201/06 (2022.01)] 20 Claims
OG exemplary drawing
 
1. A leg detection system comprising:
a robotic arm comprising a gripping portion for holding a teat cup for attaching to a teat of a dairy livestock;
an imaging system coupled to the robotic arm and configured to capture a first three-dimensional (3D) image of a stall with a rearview of the dairy livestock in the stall, the imaging system comprising a 3D camera or a laser, wherein each pixel of the first 3D image is associated with a depth value;
one or more memory devices configured to store a reference 3D image of the stall without any dairy livestock; and
a processor communicatively coupled to the imaging system and the one or more memory devices, the processor configured to:
access the first 3D image and the reference 3D image;
subtract the first 3D image from the reference 3D image to produce a second 3D image of the rearview of the dairy livestock;
perform morphological image processing on the second 3D image to produce a third 3D image;
perform image thresholding on the third 3D image to produce a fourth 3D image;
cluster data from the fourth 3D image;
identify, using the clustered data from the fourth 3D image, one or more legs of the dairy livestock; and
provide instructions for movements of the robotic arm to avoid the identified one or more legs while attaching the teat cup to the teat of the dairy livestock.