Object Classification using Ultra Low Resolution Time-of-Flight Sensor and Tiny Convolutional Neural Network

Ultra-Low-Resolution (ULR) Time-of-Flight (ToF) sensors are utilized in combination with higher-resolution image sensors in several industrial applications, such as object recognition and accurate auto-focus. However, the heterogeneous combination of these sensors and the complex post-processing systems, makes it challenging to meet the requirements of compactness, lightness, and aesthetics in edge devices used in industrial and consumer applications, such as smart visors and glasses. This work investigates the feasibility of using ULR ToF as the only sensing element in object recognition systems, combined with machine learning techniques. The outcome of this investigation resulted in a very compact system based on the STMicroelectronics VL53L8CX 8 × 8 pixel ToF sensor, coupled with a custom Convolutional Neural Network (CNN) capable of classifying four objects even under partial occlusion conditions. The designed solution overcomes the limitations mentioned above, achieving an accuracy of 92.23% when activations and weights are quantized to 8 bits. The CNN has been profiled and deployed on STM32 microcontrollers, occupying 2.65 kB of memory, performing one inference in 1 ms, and consuming 32 μ J of energy per inference. These results are a significant improvement over the state-of-the-art in the literature in terms of inference time, memory occupation, and energy consumption per inference.