Automatic Text Summarization is the result of more than 50 years of research. Several methods for creating a summary from a single document or a group of related documents have been proposed over time, all of which have shown very efficient results. Artificial intelligence has enabled advancement in generating summaries that include other words compared to the original text. Instead, the issue is identifying how a summary may be regarded as ideal compared to a reference summary, which is still a topic of research that is open to new answers. How can the outcomes of the numerous new algorithms that appear year after year be assessed? This research aims to see if the ROUGE metric, widely used in the literature to evaluate the results of Text Summarization algorithms, helps deal with these new issues, mainly when the original reference dataset is limited to a small field of interest. Furthermore, an in-depth experiment is conducted by comparing the results of the ROUGE metric with other metrics. In conclusion, determining an appropriate metric to evaluate the summaries produced by a machine is still a long way off.

Different Metrics Results in Text Summarization Approaches

Barbella, M
;
Risi, M;Tortora, G;Auriemma Citarella, A
2022-01-01

Abstract

Automatic Text Summarization is the result of more than 50 years of research. Several methods for creating a summary from a single document or a group of related documents have been proposed over time, all of which have shown very efficient results. Artificial intelligence has enabled advancement in generating summaries that include other words compared to the original text. Instead, the issue is identifying how a summary may be regarded as ideal compared to a reference summary, which is still a topic of research that is open to new answers. How can the outcomes of the numerous new algorithms that appear year after year be assessed? This research aims to see if the ROUGE metric, widely used in the literature to evaluate the results of Text Summarization algorithms, helps deal with these new issues, mainly when the original reference dataset is limited to a small field of interest. Furthermore, an in-depth experiment is conducted by comparing the results of the ROUGE metric with other metrics. In conclusion, determining an appropriate metric to evaluate the summaries produced by a machine is still a long way off.
2022
978-989-758-583-8
File in questo prodotto:
Non ci sono file associati a questo prodotto.

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11386/4853701
 Attenzione

Attenzione! I dati visualizzati non sono stati sottoposti a validazione da parte dell'ateneo

Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus ND
  • ???jsp.display-item.citation.isi??? 0
social impact