Conference Paper
Robotic harvesting involves navigation and environmental perception as first operations before harvesting
of the bolls can commence. Navigation is the distance required for a harvester’s arm to reach the cotton boll while
perception is the position of the boll relative to surrounding environment. These two operations give a 3D position of the
cotton boll for picking and can only be achieved by detection and tracking of the cotton bolls in real-time. It means
detection, tracking and counting of cotton bolls using a moving camera allows the robotic machine to harvest easily.
GPU-accelerated deep neural networks were used to train the convolution networks for detection of cotton bolls. It was
achieved by using pretrained tiny yolo weights and DarkFlow, a framework which translates YOLOv2 darknet neural
networks to TensorFlow. A method to connect tracklets using vectors that are predicted using Lucas-Kanade algorithm
and optimized using robust L-estimators and homography transformation is proposed. The system was tested in defoliated
cotton plants during the spring of 2018. Using three video treatments, the counting performance accuracy was around
93% with standard deviation 6%. The system average processing speed was 21 fps in desktop computer and 3.9 fps in
embedded system. Detection of the system achieved an accuracy and sensitivity of 93% while precision was 99.9% and
F1 score was 1. The Tukey’s test showed that the system accuracy and sensitivity was the same when the plants were
rearranged. This performance is crucial for real-time robot decisions that also measure yield while harvesting.