A BERT-based Model for Semantic Consistency Checking of Automation Rules

Trigger-Action Platforms (TAPs) allow users to automate behaviors involving IoT devices either by programming rules from scratch or by accessing a catalog of user-defined rules. Users can search the catalog based on their interests and needs, browsing through rules that are expressed according to textual descriptions supplied by the rule's creator. However, TAPs do not perform any control over these User-defined descriptions (UDDs), which means that there is no way to ensure their suitability. This lack of control might lead to the inclusion of erroneous information, making it challenging for users to retrieve the relevant rules they need during searches. To address this issue, this paper proposes the use of a BERT-based classification model to check the semantic consistency of a rule's UDD with respect to its trigger-action components. We evaluate the proposed solution with the popular TAP, namely If-This-Then-That (IFI'll), by training the model on a dataset consisting of 9643 labeled samples. Each sample is composed a fa pattern derived from the rule components, the corresponding rule's UDD, and a label expressing whether they are semantically related. The code of the software is publicly available on GitHub.