Using keyword spotting systems as tools for the transcription of historical handwritten documents: Models and procedures for performance evaluation

The paper proposes a performance model for estimating the improvement of the time needed to transcribe small collections of handwritten documents by using a keyword spotting system (KWS) with respect to the time for manually achieving the transcription. The proposed model assumes that no other information than those obtained from the samples and the KWS system performance on the training set are available, and, depending on them, establishes analytically the condition the performance measures must satisfy to make it profitable to use the system, and, in the affirmative case, estimates the gain and the accuracy of such estimation. The model is complemented by a step-by-step procedure for building the training set, running the KWS on it, estimating the performance parameters on the data set, and eventually estimating the overall improvement and the accuracy of this estimate.