Evaluation of Fingerprint Selection Algorithms for Local Text Reuse Detection
2020
Gints Jēkabsons

Detection of local text reuse is central to a variety of applications, including plagiarism detection, origin detection, and information flow analysis. This paper evaluates and compares effectiveness of fingerprint selection algorithms for the source retrieval stage of local text reuse detection. In total, six algorithms are compared – Every p-th, 0 mod p, Winnowing, Hailstorm, Frequency-biased Winnowing (FBW), as well as the proposed modified version of FBW (MFBW). Most of the previously published studies in local text reuse detection are based on datasets having either artificially generated, long-sized, or unobfuscated text reuse. In this study, to evaluate performance of the algorithms, a new dataset has been built containing real text reuse cases from Bachelor and Master Theses (written in English in the field of computer science) where about half of the cases involve less than 1 % of document text while about two-thirds of the cases involve paraphrasing. In the performed experiments, the overall best detection quality is reached by Winnowing, 0 mod p, and MFBW. The proposed MFBW algorithm is a considerable improvement over FBW and becomes one of the best performing algorithms. The software developed for this study is freely available at the author’s website http://www.cs.rtu.lv/jekabsons/.


Keywords
Document fingerprinting; fingerprint selection; local text reuse detection; plagiarism detection
DOI
10.2478/acss-2020-0002
Hyperlink
https://content.sciendo.com/view/journals/acss/25/1/article-p11.xml

Jēkabsons, G. Evaluation of Fingerprint Selection Algorithms for Local Text Reuse Detection. Applied Computer Systems, 2020, Vol. 25, No. 1, pp. 11-18. ISSN 2255-8683. e-ISSN 2255-8691. Available from: doi:10.2478/acss-2020-0002

Publication language
English (en)
The Scientific Library of the Riga Technical University.
E-mail: uzzinas@rtu.lv; Phone: +371 28399196