NDLI: Deep sentence embedding using long short-term memory networks: analysis and application to information retrieval

Content Provider	ACM Digital Library
Author	Deng, Li He, Xiaodong Palangi, Hamid Shen, Yelong Gao, Jianfeng Ward, Rabab Chen, Jianshu Song, Xinying
Copyright Year	2016
Abstract	This paper develops a model that addresses sentence embedding, a hot topic in current natural language processing research, using recurrent neural networks (RNN) with Long Short-Term Memory (LSTM) cells. The proposed LSTM-RNN model sequentially takes each word in a sentence, extracts its information, and embeds it into a semantic vector. Due to its ability to capture long term memory, the LSTM-RNN accumulates increasingly richer information as it goes through the sentence, and when it reaches the last word, the hidden layer of the network provides a semantic representation of the whole sentence. In this paper, the LSTM-RNN is trained in a weakly supervised manner on user click-through data logged by a commercial web search engine. Visualization and analysis are performed to understand how the embedding process works. The model is found to automatically attenuate the unimportant words and detect the salient keywords in the sentence. Furthermore, these detected keywords are found to automatically activate different cells of the LSTM-RNN, where words belonging to a similar topic activate the same cell. As a semantic representation of the sentence, the embedding vector can be used in many different applications. These automatic keyword detection and topic allocation abilities enabled by the LSTM-RNN allow the network to perform document retrieval, a difficult language processing task, where the similarity between the query and documents can be measured by the distance between their corresponding sentence embedding vectors computed by the LSTM-RNN. On a web search task, the LSTM-RNN embedding is shown to significantly outperform several existing state of the art methods. We emphasize that the proposed model generates sentence embedding vectors that are specially useful for web document retrieval tasks. A comparison with a well known general sentence embedding method, the Paragraph Vector, is performed. The results show that the proposed method in this paper significantly outperforms Paragraph Vector method for web document retrieval task.
Starting Page	694
Ending Page	707
Page Count	14
File Format	PDF
ISSN	23299290
e-ISSN	23299304
Volume Number	24
Issue Number	4
Journal	IEEE/ACM Transactions on Audio, Speech and Language Processing (TASLP)
Language	English
Publisher	Association for Computing Machinery (ACM)
Publisher Date	2016-04-01
Access Restriction	One Nation One Subscription (ONOS)
Subject Keyword	Deep learning Long short-term memory Sentence embedding
Content Type	Text
Resource Type	Article
Subject	Instrumentation Computational Mathematics Signal Processing Electrical and Electronic Engineering Acoustics and Ultrasonics Speech and Hearing Media Technology

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Memory-enhanced neural networks and NMF for robust ASR

On training targets for supervised speech separation

Memory-Enhanced Neural Networks and NMF for Robust ASR

Application of Deep Belief Networks for natural language understanding

Learning spectral mapping for speech dereverberation and denoising

IEEE Signal Processing Society Information

IEEE Signal Processing Society Information

IEEE Signal Processing Society Information

IEEE Signal Processing Society Information

Deep sentence embedding using long short-term memory networks: analysis and application to information retrieval

Similar Documents

Memory-enhanced neural networks and NMF for robust ASR

On training targets for supervised speech separation

Memory-Enhanced Neural Networks and NMF for Robust ASR

Application of Deep Belief Networks for natural language understanding

Learning spectral mapping for speech dereverberation and denoising

IEEE Signal Processing Society Information

IEEE Signal Processing Society Information

IEEE Signal Processing Society Information

IEEE Signal Processing Society Information

Deep sentence embedding using long short-term memory networks: analysis and application to information retrieval