MUDIPA: A Human-in-the-Loop AI-Assisted Platform for Discourse Graph Annotation in Multi-party Dialogues

Annotating discourse structure in multi-party dialogues is a cognitively demanding task that requires specialized expertise, yet existing annotation tools are either fully manual or provide only limited assistance. We present \tool, a web-based, human-in-the-loop platform for discourse graph annotation in multi-party dialogues, built around two pluggable AI backends: a suggestion engine that produces confidence-ranked arc proposals, and an LLM-based reasoning engine that generates on-demand natural-language rationales for candidate attachments and relation labels. In a counterbalanced, within-subjects study with eight annotators, we compare AI-assisted annotation with \tool against manual annotation. The AI-assisted condition achieves higher inter-annotator agreement on relation labels, higher accuracy against the gold standard, and lower perceived workload, while requiring only a modest increase in annotation time. These findings indicate that the principal value of LLMs in discourse annotation lies in supporting human decision-making, improving both annotation quality and inter-annotator agreement relative to unassisted annotation.