NDLI: Natural Language Compression Optimized for Large Set of Files

Content Provider	IEEE Xplore Digital Library
Author	Prochazka, P. Holub, J.
Copyright Year	2013
Description	Author affiliation: Dept. of Theor. Comput. Sci., Czech Tech. Univ. in Prague, Prague, Czech Republic (Prochazka, P.; Holub, J.)
Abstract	Summary form only given. The web search engines store the web pages in the raw text form to build so called snippets (short text surrounding the searched pattern) or to perform so called positional ranking functions. We address the problem of the compression of a large collection of text files distributed in cluster of computers, where the single files need to be randomly accessed in very short time. The compression algorithm Set-of-Files Semi-Adaptive Two Byte Dense Code (SF-STBDC) is based on the word-based approach and the idea of combination of two statistical models: the global model (common for all the files of the set) and the local model. The latter is built as the set of changes which transform the global model to the proper model of the single compressed file. Except very good compression ratio the compression method allows fast searching on the compressed text, which is an attractive property especially for search engines property especially for search engines. Exactly the same problem (compression of a set of files using byte codes) was first stated in. Our algorithm SF-STBDC overcomes the algorithm based on (s,c) - Dense Code in compression ratio and at the same time it keeps a very good searching and decompression speed. The key idea to achieve this result is a usage of Semi-Adaptive Two Byte Dense Code which provides more effective coding of small portions ofof the text and still allows exact setting of the number of stoppers and continuers.
Sponsorship	Univ. Arizona
Starting Page	514
Ending Page	514
File Size	495246
Page Count	1
File Format	PDF
ISBN	9781467360371
ISSN	10680314
e-ISBN	9780769549651
DOI	10.1109/DCC.2013.93
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	2013-03-20
Publisher Place	USA
Access Restriction	Subscribed
Rights Holder	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subject Keyword	Data compression Educational institutions Web search Engines Computational modeling Natural languages Computer science
Content Type	Text
Resource Type	Article
Subject	Computer Networks and Communications

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Query Disambiguation for Cross-Language Information Retrieval Using Web Directories

A Web-based Question Answering System for Effective e-Learning

Modeling delta encoding of compressed files

An Effective Feature-Weighting Model for Question Classification

A system for Chinese question answering

Universal Text Preprocessing and Postprocessing for PPM Using Alphabet Adjustment

Using Cross-Language Information Retrieval Methods for Bilingual Search of the Web

Compressed Delta Encoding for LZSS Encoded Files

An Evaluation of How Search Engines Respond to Greek Language Queries

Natural Language Compression Optimized for Large Set of Files

Similar Documents

Query Disambiguation for Cross-Language Information Retrieval Using Web Directories

A Web-based Question Answering System for Effective e-Learning

Modeling delta encoding of compressed files

An Effective Feature-Weighting Model for Question Classification

A system for Chinese question answering

Universal Text Preprocessing and Postprocessing for PPM Using Alphabet Adjustment

Using Cross-Language Information Retrieval Methods for Bilingual Search of the Web

Compressed Delta Encoding for LZSS Encoded Files

An Evaluation of How Search Engines Respond to Greek Language Queries

Natural Language Compression Optimized for Large Set of Files