Multi-task road surface condition recognition through a self-conditioned multi-label approach

Autonomous vehicle applications are gaining growing attention for their increasing reliability and capability to reduce the ecological impact of transportation. However, most prototypes fail to perceive the conditions of the road surface, which are crucial to adapt the driving controllers and avoid risks, e.g., due to hydroplaning. We propose a method to identify the major causes of grip reduction, namely friction, unevenness, and type of material, using a sequential architecture based on an LSTM module. As the multiple combination of these conditions alters the appearance of the ground surface, our model is trained to identify the most evident conditions first, then to use the predicted information to infer the less evident conditions. After the sequential prediction, a final classification stage allows for surpassing the limits of the sequential approach and recovering errors that occurred in early iterations, thus reducing their impact on subsequent ones. Experimental results show that the method outperforms previous ones on a challenging 27-class and 3-task dataset, by about 2.5 and 5.0 percentage points in terms of overall and balanced accuracy, respectively, while marginally increasing the inference time. Moreover, the conducted thorough error analysis demonstrates that our approach can implicitly learn the application constraints, like the absence of the material label if the surface is covered by snow, and provide useful information to interpret the sequence of predictions produced at each iteration of the sequential architecture.