NDLI: An empirical investigation of word class-based features for natural language understanding

Content Provider	ACM Digital Library
Author	Deoras, Anoop Jeong, Minwoo Celikyilmaz, Asli Sarikaya, Ruhi
Copyright Year	2016
Abstract	There are many studies that show using class-based features improves the performance of natural language processing (NLP) tasks such as syntactic part-of-speech tagging, dependency parsing, sentiment analysis, and slot filling in natural language understanding (NLU), but not much has been reported on the underlying reasons for the performance improvements. In this paper, we investigate the effects of the word class-based features for the exponential family of models specifically focusing on NLU tasks, and demonstrate that the performance improvements could be attributed to the regularization effect of the class-based features on the underlying model. Our hypothesis is based on empirical observation that $\textit{shrinking}$ the sum of parameter magnitudes in an exponential model tends to improve performance. We show on several semantic tagging tasks that there is a positive correlation between the model size reduction by the addition of the class-based features and the model performance on a held-out dataset. We also demonstrate that class-based features extracted from different data sources using alternate word clustering methods can individually contribute to the performance gain. Since the proposed features are generated in an unsupervised manner without significant computational overhead, the improvements in performance largely come for free and we show that such features provide gains for a wide range of tasks from semantic classification and slot tagging in NLU to named entity recognition (NER).
Starting Page	994
Ending Page	1005
Page Count	12
File Format	PDF
ISSN	23299290
e-ISSN	23299304
Volume Number	24
Issue Number	6
Journal	IEEE/ACM Transactions on Audio, Speech and Language Processing (TASLP)
Language	English
Publisher	Association for Computing Machinery (ACM)
Publisher Date	2016-06-01
Access Restriction	One Nation One Subscription (ONOS)
Subject Keyword	Class-based features Conditional random fields Exponential models Natural language understanding Regularization Shrinkage features
Content Type	Text
Resource Type	Article
Subject	Instrumentation Computational Mathematics Signal Processing Electrical and Electronic Engineering Acoustics and Ultrasonics Speech and Hearing Media Technology

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Supervised single-microphone multi-talker speech separation with conditional random fields

Combination of language models for word prediction: an exponential approach

Graph-based lexicon regularization for PCFG with latent annotations

Supervised Single-Microphone Multi-Talker Speech Separation with Conditional Random Fields

Syntactic and semantic features for code-switching factored language models

Exponential language modeling using morphological features and multi-task learning

Application of Deep Belief Networks for natural language understanding

Application of Deep Belief Networks for Natural Language Understanding

Triangular-Chain Conditional Random Fields

An empirical investigation of word class-based features for natural language understanding

Similar Documents

Supervised single-microphone multi-talker speech separation with conditional random fields

Combination of language models for word prediction: an exponential approach

Graph-based lexicon regularization for PCFG with latent annotations

Supervised Single-Microphone Multi-Talker Speech Separation with Conditional Random Fields

Syntactic and semantic features for code-switching factored language models

Exponential language modeling using morphological features and multi-task learning

Application of Deep Belief Networks for natural language understanding

Application of Deep Belief Networks for Natural Language Understanding

Triangular-Chain Conditional Random Fields

An empirical investigation of word class-based features for natural language understanding