Counting people by RGB or depth overhead cameras

In this paper we present a vision based method for counting the number of persons which cross a virtual line. The method analyzes the video stream acquired by a camera mounted in a zenithal position with respect to the counting line, allowing to determine the number of persons that cross the virtual line and providing the crossing direction for each person. The proposed approach has been specifically designed to achieve high accuracy and computational efficiency, so as to allow its adoption in real scenarios. An extensive evaluation of the method has been carried out taking into account the main factors that may impact on the counting performance and, in particular, the acquisition technology (traditional RGB camera and depth sensor), the installation scenario (indoor and outdoor), the density of the people flow (isolated people and groups of persons), the acquisition frame rate, and the image resolution. We have also analyzed the combination of the outputs obtained from the RGB and depth sensors as a way to improve the counting performance. The experimental results confirm the effectiveness of the proposed method, especially when combining RGB and depth information, and the tests over three different CPU architectures demonstrate the possibility of deploying the method both on high-end servers for processing in parallel a large number of video streams and on low power CPUs as those embedded on commercial smart cameras.