Automatic Sentence Simplification for Hindi: Methods and Application

Abstract

Cognitive and psychological studies on human reading indicate that the effort required to read and understand text increases with sentence complexity (Klein and Kurkowski, 1974). Modern natural language processing (NLP) applications face similar challenges. Processing complex sentences with high accuracy remains difficult in computational linguistics, necessitating automatic sentence simplification techniques (Chandrasekar et al., 1996). Sentence complexity can be classified into ’lexical complexity’ and ’syntactic complexity’. Lexical complexity can be managed by utilizing resources like lexicons, dictionaries, and thesauruses to replace infrequent words with common ones. To address syntactic complexity, analyzing sentence structure and applying simplification operations is essential. There are many applications of sentence simplification in NLP. Machine translation systems struggle with long, complex sentences, especially when dealing with divergent language pairs. For parsing, (McDonald and Nivre, 2007) showed that syntactic parsing of long sentences and identifying long-distance dependencies remain challenging. Simplifying sentences can aid parsing and machine translation by breaking down sentences into smaller parts. In automatic summarization, simplification can improve accuracy by extracting smaller units of information. In this work, we studied and analyzed existing sentence simplification methods for Hindi. We developed new approaches, both rule-based and statistical, to design a better system that overcomes the limitations of existing systems while improving quality and readability. As an application, we examined the effects of sentence simplification on Hindi to English machine translation systems.