SecLLM: Enhancing Security Smell Detection in IaC With Large Language Models

The emergence and expansion of Infrastructure-as-Code (IaC) paradigms have transformed cloud provisioning methodologies, while simultaneously introducing complex security smells. Traditional detectors are unable to handle the semantic challenges (i.e., understanding the meaning and context of code beyond its syntax) introduced by contemporary IaC ecosystems due to their reliance on heuristic rules and syntactic patterns. This work introduces SecLLM, an innovative framework that leverages the capabilities of Large Language Models (LLMs) to reconceptualize the detection of security smells through contextual semantic analysis. Our approach avoids conventional intermediate representations, exploiting natural language understanding to examine Ansible, Puppet, and Chef configurations through adaptive prompt engineering methodologies. The framework architecture supports multiple IaC infrastructures and LLMs while simultaneously enabling optimization of the underlying LLM responses for each of the nine security smell analyzed. Furthermore, SecLLM provides detailed feedback regarding the analyzed scripts that can help practitioners define their infrastructure’s risk profile and resolve the identified smells. The empirical validation demonstrates that SecLLM outperforms GLITCH, a state-of-the-art security smell identification tool, achieving precision improvements of 12-21 percentage points and F1-score gains of 17-32 percentage points in terms of detection accuracy while simultaneously maintaining low operational costs ($0.003-$0.015 per script). Additionally, we address the non-deterministic behavior inherent to LLMs through three mitigation strategies (confidence-based filtering, ensemble voting, and response caching) that achieve perfect reproducibility (Fleiss’ k = 1.00) and preserve detection accuracy.