Opening the black box: how EUD interfaces foster user agency and quality in generative phishing simulations

Human error remains a primary vector for cyber attacks, making ethical phishing simulations a critical defense strategy. Designing such simulations requires balancing ease of use with meaningful control over generative AI. We introduce Damocles, a platform for ethical phishing campaigns, and report a within-subjects study (N = 30) comparing two interaction modalities built on the same scaffolded prompt-configuration workflow. In the Black-Box condition, users configured scenarios through high-level controls while the LLM prompt remained hidden. In the Glass-Box condition, the same choices generated a prompt that was visible, updated in real time, and editable before generation. Results show that the Glass-Box approach achieved comparable email quality and safety to the Black-Box condition, while maintaining comparable usability ((Formula presented.)) and workload (NASA-TLX). Behavioural analysis revealed a non-significant trend toward lower AI Retention Rate in the Glass-Box condition (p =.085, r =.52), interpreted cautiously as a preliminary signal of possible differences in textual reliance on AI-generated drafts. These findings position scaffolded prompt visibility and editability as EUD-inspired mechanisms for supporting end-user control without degrading output quality or increasing perceived workload.