TitleStructure regularization for structured prediction
AuthorsSun, Xu
AffiliationMOE Key Laboratory of Computational Linguistics, Peking University, China
School of Electronics Engineering and Computer Science, Peking University, China
Issue Date2014
Publisher28th Annual Conference on Neural Information Processing Systems 2014, NIPS 2014
Citation28th Annual Conference on Neural Information Processing Systems 2014, NIPS 2014.Montreal, QC, Canada,2014/1/1,3(2402-2410).
AbstractWhile there are many studies on weight regularization, the study on structure regularization is rare. Many existing systems on structured prediction focus on increasing the level of structural dependencies within the model. However, this trend could have been misdirected, because our study suggests that complex structures are actually harmful to generalization ability in structured prediction. To control structure-based overfitting, we propose a structure regularization framework via structure decomposition, which decomposes training samples into mini-samples with simpler structures, deriving a model with better generalization power. We show both theoretically and empirically that structure regularization can effectively control overfitting risk and lead to better accuracy. As a by-product, the proposed method can also substantially accelerate the training speed. The method and the theoretical results can apply to general graphical models with arbitrary structures. Experiments on well-known tasks demonstrate that our method can easily beat the benchmark systems on those highly-competitive tasks, achieving state-of-the-art accuracies yet with substantially faster training speed.
Appears in Collections:计算语言学教育部重点实验室

Files in This Work
There are no files associated with this item.

Web of Science®

Checked on Last Week


Checked on Current Time

License: See PKU IR operational policies.