NDLI: A pragmatic model for new Chinese word extraction

Content Provider	IEEE Xplore Digital Library
Author	Haijun Zhang Heyan Huang Chaoyong Zhu Shumin Shi
Copyright Year	2010
Description	Author affiliation: School of Computer Science and Technology, Xinjiang Normal University, Urumqi 830054, China (Haijun Zhang) \|\| School of Computer Science and Technology, Beijing Institute of Technology, 100081, China (Heyan Huang; Shumin Shi) \|\| Research Center of Computer and Language Information Engineering, Chinese Academy of Sciences, Beijing 100097, China (Chaoyong Zhu)
Abstract	This paper proposed a pragmatic model for repeat-based Chinese New Word Extraction (NWE). It contains two innovations. The first is a formal description for the process of NWE, which gives instructions on feature selection in theory. On the basis of this, the Conditional Random Fields model (CRF) is selected as statistical framework to solve the formal description. The second is an improved algorithm for left (right) entropy to improve the efficiency of NWE. By comparing with baseline algorithm, the improved algorithm can enhance the computational speed of entropy remarkably. On the whole, experiments show that the model this paper proposed is very effective, and the F score is 49.72% in open test and 69.83% in word extraction respectively, which is an evident improvement over previous similar works.
Starting Page	1
Ending Page	8
File Size	687741
Page Count	8
File Format	PDF
ISBN	9781424468966
e-ISBN	9781424468997
DOI	10.1109/NLPKE.2010.5587846
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	2010-08-21
Publisher Place	China
Access Restriction	Subscribed
Rights Holder	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subject Keyword	repeat New words extraction Educational institutions computational efficiency formal description left (right) entropy
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Chinese Unknown Words Extraction Based on Word-Level Characteristics

Chinese New Words Extraction Based on Machine Learning Approach

Which performs better for new word detection, character based or Chinese Word Segmentation based?

Two-Character Chinese Word Extraction Based on Hybrid of Internal and Contextual Measures

A New Machine Learning Method for Chinese Overlapping Disambiguity--Conditional Random Fields

A hybrid model for word alignment with bilingual corpus

A Method for New Word Extraction on Chinese Large-scale Query Logs

A maximum entropy based reordering model for Mongolian-Chinese SMT with morphological information

Automatic Extraction Method of Tibetan New Valid Words

A pragmatic model for new Chinese word extraction

Similar Documents

Chinese Unknown Words Extraction Based on Word-Level Characteristics

Chinese New Words Extraction Based on Machine Learning Approach

Which performs better for new word detection, character based or Chinese Word Segmentation based?

Two-Character Chinese Word Extraction Based on Hybrid of Internal and Contextual Measures

A New Machine Learning Method for Chinese Overlapping Disambiguity--Conditional Random Fields

A hybrid model for word alignment with bilingual corpus

A Method for New Word Extraction on Chinese Large-scale Query Logs

A maximum entropy based reordering model for Mongolian-Chinese SMT with morphological information

Automatic Extraction Method of Tibetan New Valid Words

A pragmatic model for new Chinese word extraction