Identifying fake reviews for refund purposes: Evaluating the effectiveness of a transfer-learning model against emerging Large Language Models

Background: Recently, dishonest sellers are using social platforms to advertise products that can be purchased for free through a refund mechanism, which is based on the writing of five-star fake reviews. The aim is to increase product visibility by influencing their ranking compared to similar products. This mechanism is leading to a significant distortion of e-commerce platforms, eroding trust among customers and sellers. Objective: In this paper, we address the problem of identifying fake reviews, aiming to provide an approach for mitigating fraudulent practices that compromise the integrity and transparency of e-commerce platforms. Methods: We propose a supervised model tailored for identifying fake reviews for refund purposes and compare its performance with some of the most recent generative models. Since, to the best of our knowledge, no datasets exist in the literature suitable for fake review identification in the process of Purchasing, Requesting reviews, and Refunding a product, we first proposed a new dataset of fake and genuine reviews from Amazon, collected with the help of a domain expert. Then, we defined five other new datasets containing reviews automatically generated by language models. To interact with these models, we designed new prompt approaches specifically tailored to our goal, which exploit the iterative refinement behind these models for improving classification results. Results: Experimental results demonstrated the effectiveness of the supervised model in detecting both types of fake reviews, outperforming state-of-the-art models with improvements ranging from 0.23 to 0.70 in terms of accuracy, precision, and recall.