End-User Customization of Trigger-Action Rules Through Fine-Tuned LLMs

While Trigger-Action Platforms (TAPs) provide an effective solution for end-users to automate interactions between smart devices and online services through customizable rules, their flexibility is often limited by restricted access to source code, confining users to predefined templates and basic configurations, or the need for programming expertise when modifications are allowed. To address this challenge, we propose a methodology using fine-tuned autoregressive Large Language Models (LLMs) that translates natural language descriptions into executable rule code. This approach removes the need for manual coding, reducing the technical barrier and enabling users to express their automation intents more intuitively. To evaluate our approach, we used a dataset of trigger-action rules from the If-This-Then-That (IFTTT) platform along with their natural language descriptions. These descriptions underwent refinement through an LLM-based preprocessing step and were then used to evaluate the performance of eight open-source LLMs in generating syntactically correct and functionally equivalent rule implementations. Among these models, Codestral achieved the best results, obtaining a ROUGE-L score of 63% and a METEOR score of 54%.