TitleFeature-Frequency-Adaptive On-line Training for Fast and Accurate Natural Language Processing
AuthorsSun, Xu
Li, Wenjie
Wang, Houfeng
Lu, Qin
AffiliationPeking Univ, Key Lab Computat Linguist, Minist Educ, Beijing 100871, Peoples R China.
Peking Univ, Sch EECS, Beijing 100871, Peoples R China.
Hong Kong Polytech Univ, Dept Comp, Kowloon 999077, Hong Kong, Peoples R China.
Issue Date2014
Publishercomputational linguistics
CitationCOMPUTATIONAL LINGUISTICS.2014,40,(3),563-586.
AbstractTraining speed and accuracy are two major concerns of large-scale natural language processing systems. Typically, we need to make a tradeoff between speed and accuracy. It is trivial to improve the training speed via sacrificing accuracy or to improve the accuracy via sacrificing speed. Nevertheless, it is nontrivial to improve the training speed and the accuracy at the same time, which is the target of this work. To reach this target, we present a new training method, feature-frequency-adaptive on-line training, for fast and accurate training of natural language processing systems. It is based on the core idea that higher frequency features should have a learning rate that decays faster. Theoretical analysis shows that the proposed method is convergent with a fast convergence rate. Experiments are conducted based on well-known benchmark tasks, including named entity recognition, word segmentation, phrase chunking, and sentiment analysis. These tasks consist of three structured classification tasks and one non-structured classification task, with binary features and real-valued features, respectively. Experimental results demonstrate that the proposed method is faster and at the same time more accurate than existing methods, achieving state-of-the-art scores on the tasks with different characteristics.
Appears in Collections:计算语言学教育部重点实验室

Files in This Work
There are no files associated with this item.

Web of Science®

Checked on Last Week


Checked on Current Time


Checked on Current Time

Google Scholar™

License: See PKU IR operational policies.