IIIT Hyderabad Publications |
|||||||||
|
A Modified Annotation Scheme for Semantic Textual SimilarityAuthors: darshan.agarwal ,vandan.mujadia ,Dipti Misra Sharma,Radhika Mamidi Conference: 18th International Conference on Computational Linguistics and Intelligent Text Processing (CICLing-2017 2017) Date: 2017-04-17 Report no: IIIT/TR/2017/26 AbstractThis paper presents an annotation schema for annotating semantic textual similarity. Given two sentences, the goal of the annotation is to give the similarity score between two sentences on a scale of 0 to 5. Annotators faced several difficulties in assigning similarity scores by following the [1] annotation scheme. To overcome those difficulties, we propose a new set of annotation guidelines which takes into account two major aspects of a sentence: events and entities. The semantic similarity score between a pair of sentences is assigned by finding the similarity of events and relations like hypernymy, co-hyponymy, meronymy, etc. between the entities in the sentences individually. Using our scheme we annotated the degree of semantic relatedness on 750 pairs of mononlingual Hindi sentences which were collected from newspapers, essays. We observed a significant improvement in interannotator agreement from 0.55 to 0.81 Fleiss’ kappa measure. Full paper: pdf Centre for Language Technologies Research Centre |
||||||||
Copyright © 2009 - IIIT Hyderabad. All Rights Reserved. |