IIIT Hyderabad Publications |
|||||||||
|
Starting Small Learning Strategies for Speech RecognitionAuthors: Hari Vydana,Brij Mohan lal,Anil Kumar Vuppala,Manish Shrivastava Conference: 13th International IEEE India conference INDOCON 016 Location IISc Bengaluru, India Date: 2016-12-16 Report no: IIIT/TR/2016/48 AbstractDesigning various learning strategies has been gain- ing a lot of scientific interest during the recent progress of deep learning methodologies. Curriculum learning is a learning strategy aimed at training the neural network model by presenting the samples in a specific meaningful order rather than randomly sampling the training examples from the data distribution. In this work, we have explored starting small paradigm of curriculum learning technique for speech recognition. The starting small paradigm of curriculum learning is performed by a two step learning strategy. Training dataset is re-organized as a set of easily classifiable examples followed by the actual training dataset and the model is trained on the re-organized dataset. We hypoth- esize that by following the starting small learning paradigm the learning gets initialized in a better way and progresses to attain a better convergence. We propose to rank the toughness of the training example based on the posterior probabilities obtained using a pre-trained model. Apart from re-arranging the training corpus starting small paradigm of curriculum learning is applied at model level. We consider the broad manner class classification objective function as the smoother version of the phone class classification objective function. The model initially trained for broadclass classification is later adapted for phone classification. In this work, we have used TIMIT and a subset of Wall Street Journal (WSJ) corpus to validate the experiments, both the learning strategies have shown consistently better performances across the two datasets compared to the baseline system trained by randomly sampling the dataset Full paper: pdf Centre for Language Technologies Research Centre |
||||||||
Copyright © 2009 - IIIT Hyderabad. All Rights Reserved. |