Exploiting a multi-device knowledge meshing to agent-based activity tracking

Nowadays, systems of systems, composed of multiple cooperative smart devices, reached popularity in many areas, including surveillance, digital forensics, agriculture and more. The analysis of great amounts of data coming from different sources could be very time consuming for humans, so they require automatic tools that help them to monitor vast and complex environments to detect anomalous situations. To this purpose, this paper introduces an agent-based model to allow IoT systems to monitor outdoor environments and detect suspicious or critical situations. The agent-modeling allows the accomplishment and coordination of various tasks, including object detection and data collection achieved through tracking, environmental context detection by using frame classification and semantic segmentation, contextual knowledge generation and activity detection through ontology reasoning. Finally, the agents report to humans about what happened by estimating the scene criticality through a fuzzy controller. A case study shows the potential of the whole framework and experiences evaluate the skills of the framework for activity detection.