Can ChatGPT provide intelligent diagnoses? A comparative study between predictive models and ChatGPT to define a new medical diagnostic bot

Intelligent diagnosis processes rely on Artificial Intelligence (AI) techniques to provide possible diagnoses by analyzing patient data and medical information. To make accurate and quick diagnoses, it is possible to use AI tools to efficiently analyze huge amounts of data and find patterns that a clinician might miss. In recent years, new large language models (LLMs), such as ChatGPT and Google BARD, have shown remarkable capabilities in several domains, including intelligent diagnostics. This research aims to compare the performances of ChatGPT and traditional machine learning models for making diagnoses of low- and medium- risk diseases only based on their symptoms. On the basis of our study, we defined four research questions: RQ1) What are the benefits and limitations of using ChatGPT in intelligent diagnosis? RQ2) How do traditional machine learning approaches compare to ChatGPT for intelligent diagnosis? RQ3) How does ChatGPT compare with other LLMs and domain-specific natural language processing models in the intelligent diagnosis tasks?, and RQ4) What are the implications of the predictive models and ChatGPT for healthcare, and how can they be used to support people?. To answer these RQs, we first evaluate the performances of different engines of ChatGPT, also introducing a new prompt engineering methodology specifically tailored for achieving accurate diagnostic outcomes. Moreover, we compare these results with those achieved by different predictive models trained for intelligent diagnosis tasks, i.e., Google BARD, and two domain-specific NLP models. Finally, we propose a new interactive bot available for users that relies on the best-performing models evaluated in the previous steps. The experiments have been conducted using two medical datasets for disease prediction consisting of more than 100 symptoms associated with several diagnoses.