NDLI: An empirical study of required dimensionality for large-scale latent semantic indexing applications

Please wait, while we are loading the content...

Dynamic faceted search for discovery-driven analysis

How does clickthrough data reflect retrieval quality?

Error-driven generalist+experts (edge): a multi-stage ensemble framework for text categorization

Web-scale named entity recognition

Content-based filtering for efficient online materialized view maintenance

Can all tags be used for search?

Exploiting temporal contexts in text classification

Local approximation of pagerank and reverse pagerank

An algorithm to determine peer-reviewers

Anomaly-free incremental output in stream processing

Generalized inverse document frequency

Improved query difficulty prediction for the web

Mining term association patterns from search logs for effective query reformulation

Rewriting of visibly pushdown languages for xml data integration

Retrievability: an evaluation measure for higher order information access tasks

Clustered subset selection and its applications on it service metrics

Exploiting pipeline interruptions for efficient memory allocation

Can phrase indexing help to process non-phrase queries?

Simultaneous multilingual search for translingual information retrieval

Fast mining of complex time-stamped events

Wildcards for lightweight information integration in virtual desktops

Efficient techniques for document sanitization

Joke retrieval: recognizing the same joke told differently

Tapping on the potential of q&a community by recommending answer providers

REDUS: finding reducible subspaces in high dimensional data

The social (open) workspace

To swing or not to swing: learning when (not) to advertise

Key blog distillation: ranking aggregates

A consensus based approach to constrained clustering of software requirements

Multi-aspect expertise matching for review assignment

Structural relevance: a common basis for the evaluation of structured document retrieval

Identification of class specific discourse patterns

Valid scope computation for location-dependent spatial query in mobile broadcast environments

Answering questions with authority

Intra-document structural frequency features for semi-supervised domain adaptation

Privacy-preserving data publishing for horizontally partitioned databases

Transaction reordering with application to synchronized scans

An approximate string matching approach for handling incorrectly typed urls

An integration strategy for mining product features and opinions

Efficient processing of probabilistic spatio-temporal range queries over moving objects

Natural language retrieval of grocery products

Group-based learning: a boosting approach

Energy-efficient skyline query processing and maintenance in sensor networks

Suppressing outliers in pairwise preference ranking

Boosting social annotations using propagation

Humane data mining

Minimum-effort driven dynamic faceted search in structured databases

Efficient and effective link analysis with precomputed salsa maps

A sparse gaussian processes classification framework for fast tag suggestions

Semi-automated logging of contact center telephone calls

A step towards incremental maintenance of the composed schema mapping

Comparing citation contexts for information retrieval

Kernel methods, syntax and semantics for relational text categorization

Link privacy in social networks

Spam characterization and detection in peer-to-peer file-sharing systems

SNIF TOOL: sniffing for patterns in continuous streams

TinyLex: static n-gram index pruning with perfect recall

Understanding the relationship between searchers' queries and information goals

Non-local evidence for expert finding

Some rewrite optimizations of DB2 XQuery navigation

Statistical power in retrieval experimentation

The query-flow graph: model and applications

A new method for indexing genomes using on-disk suffix trees

Matching task profiles and user needs in personalized web search

Translation enhancement: a new relevance feedback method for cross-language information retrieval

Predicting individual disease risk based on medical history

Finding informative commonalities in concept collections

Vanity fair: privacy in querylog bundles

Ranked feature fusion models for ad hoc retrieval

SoRec: social recommendation using probabilistic matrix factorization

Mining influential attributes that capture class and group contrast behaviour

Search advertising using web relevance feedback

Blog site search using resource selection

Data weaving: scaling up the state-of-the-art in data clustering

Dr. Searcher and Mr. Browser: a unified hyperlink-click graph

A generative retrieval model for structured documents

Scalable community discovery on textual data with relations

Adaptive distributed indexing for structured peer-to-peer networks

Cache-aware load balancing for question answering

Academic conference homepage understanding using constrained hierarchical conditional random fields

CE2: towards a large scale hybrid search engine with integrated ranking support

Yizkor books: a voice for the silent past

Speed up semantic search in p2p networks

Overlapping community structure detection in networks

Data degradation: making private data less sensitive over time

Improve the effectiveness of the opinion retrieval and opinion polarity classification

Collaborative partitioning with maximum user satisfaction

Table summarization with the help of domain lattices

Re-considering neighborhood-based collaborative filtering parameters in the context of new data

Effective pattern taxonomy mining in text documents

Markov logic: a unifying language for knowledge and information management

A language for manipulating clustered web documents results

Achieving both high precision and high recall in near-duplicate detection

Transfer learning from multiple source domains via consensus regularization

MedSearch: a specialized search engine for medical information retrieval

Modeling and exploiting query interactions in database systems

Social tags: meaning and suggestions

BNS feature scaling: an improved representation over tf-idf for svm text classification

On effective presentation of graph patterns: a structural representative approach

Predicting web spam with HTTP session information

Real-time new event detection for video streams

Revisiting the relationship between document length and relevance

Active relevance feedback for difficult queries

Discovering leaders from community actions

Pruning nested XQuery queries

Comparing metrics across TREC and NTCIR: the robustness to system bias

A framework for estimating complex probability density structures in data streams

Supporting sub-document updates and queries in an inverted index

Beyond the session timeout: automatic hierarchical segmentation of search topics in query logs

High-dimensional descriptor indexing for large multimedia databases

Identification of gene function using prediction by partial matching (PPM) language models

Association thesaurus construction methods based on link co-occurrence analysis for wikipedia

Dual encryption for query integrity assurance

AdaSum: an adaptive model for summarization

Probabilistic polyadic factorization and its application to personalized recommendation

Real-time data pre-processing technique for efficient feature extraction in large scale datasets

A two-stage text mining model for information filtering

An effective statistical approach to blog post opinion retrieval

EDSC: efficient density-based subspace clustering

Modeling multi-step relevance propagation for expert finding

A densitometric approach to web page segmentation

Information shared by many objects

PROQID: partial restarts of queries in distributed databases

A system for finding biological entities that satisfy certain conditions from texts

Identifying table boundaries in digital documents via sparse line detection

Scaling up duplicate detection in graph data

A note on search based forecasting of ad volume in contextual advertising

Coreference resolution using expressive logic models

A light weighted damage tracking quarantine and recovery scheme for mission-critical database systems

A latent variable model for query expansion using the hidden markov model

Efficient frequent pattern mining over data streams

Protecting location privacy against location-dependent attack in mobile services

Efficient estimation of the size of text deep web data source

Incorporating topical support documents into a small training set in text categorization

Unsolved problems in search: (and how we approach them)

Integrating web query results: holistic schema matching

Are click-through data adequate for learning web search rankings?

Classifying networked entities with modularity kernels

An empirical study of required dimensionality for large-scale latent semantic indexing applications

A novel optimization approach to efficiently process aggregate similarity queries in metric access methods

Mining social networks using heat diffusion processes for marketing candidates selection

Learning a two-stage SVM/CRF sequence classifier

Characterizing and predicting community members from evolutionary and heterogeneous networks

Inferring semantic query relations from collective user behavior

Linear time membership in a class of regular expressions with interleaving and counting

Relating dependent indexes using dempster-shafer theory

Query suggestion using hitting time

Learning to link with wikipedia

A heuristic approach for checking containment of generalized tree-pattern queries

How evaluator domain expertise affects search result relevance judgments

Proactive learning: cost-sensitive active learning with multiple imperfect oracles

Modeling LSH for performance tuning

Learning latent semantic relations from clickthrough data for query suggestion

On low dimensional random projections and similarity search

Fast correlation analysis on time series datasets

Peer production of structured knowledge -: an empirical study of ratings and incentive mechanisms

Records retention in relational database systems

Modeling hidden topics on document manifold

A random walk on the red carpet: rating movies with user reviews and pagerank

Structure feature selection for graph classification

Automatic online news topic ranking using media focus and user attention based on aging theory

An effective algorithm for mining 3-clusters in vertically partitioned data

Trada: tree based ranking function adaptation

Using structured text for large-scale attribute extraction

Extremely fast text feature extraction for classification and indexing

ROAD: an efficient framework for location dependentspatial queries on road networks

An extension of PLSA for document clustering

A method to predict social annotations

Query optimization in xml-based information integration

A survey of pre-retrieval query performance predictors

GHOST: an effective graph-based framework for name distinction

Polyhedral transformation for indexed rank order correlation queries

A georeferencing multistage method for locating geographic context in web search

Exploiting context to detect sensitive information in call center conversations

View and index selection for query-performance improvement: quality-centered algorithms and heuristics

Online spam-blog detection through blog search

Large maximal cliques enumeration in sparse graphs

Estimating the number of answers with guarantees for structured queries in p2p databases

Modeling document features for expert finding

Deriving non-redundant approximate association rules from hierarchical datasets

Workload-based optimization of integration processes

Incorporating place name extents into geo-ir ranking

Multi-scale characterization of social network dynamics in the blogosphere

SQL extension for exploring multiple tables

Nested region algebra extended with variables for tag-annotated text search

Summarization of social activity over time: people, actions and concepts in dynamic networks

Evaluating partial tree-pattern queries on XML streams

Mining named entity transliteration equivalents from comparable corpora

Pattern-based semantic class discovery with multi-membership support

The effect of contextualization at different granularity levels in content-oriented xml retrieval

Semi-supervised text categorization by active search

PBFilter: indexing flash-resident data through partitioned summaries

Searching the wikipedia with contextual information

Using tag semantic network for keyphrase extraction in blogs

Characterization of TPC-H queries for a column-oriented database on a dual-core amd athlon processor

Estimating retrieval effectiveness using rank distributions

Detecting significant distinguishing sets among bi-clusters

Using the current browsing context to improve search relevance

Clustering multi-way data via adaptive subspace iteration

Winnowing-based text clustering

Handling implicit geographic evidence for geographic ir

Semi-supervised ranking aggregation

Semi-supervised metric learning by maximizing constraint margin

Using a graph-based ontological user profile for personalizing search

A coarse-grain grid-based subspace clustering method for online multi-dimensional data streams

Using sequence classification for filtering web pages

Estimating real-valued characteristics of criminals from their recorded crimes

Ranking in folksonomy systems: can context help?

On quantifying changes in temporally evolving dataset

Measuring user preference changes in digital libraries

A matrix-based approach for semi-supervised document co-clustering

Passage relevance models for genomics search

Representative entry selection for profiling blogs

Evaluating topic models for information retrieval

Fast spatial co-location mining without cliqueness checking

Utilization of navigational queries for result presentation and caching in search engines

Categorizing blogger's interests based on short snippets of blog posts

Cross-document cross-lingual coreference retrieval

Efficient web matrix processing based on dual reordering

A novel statistical chinese language model and its application in pinyin-to-character conversion

Decomposition of terminology graphs for domain knowledge acquisition

SHOPSMART: product recommendations through technical specifications and user reviews

Siphon++: a hidden-webcrawler for keyword-based interfaces

Coreex: content extraction from online news articles

Integrating clustering and multi-document summarization to improve document understanding

In the development of a spanish metamap

Trust, authority and popularity in social information retrieval

Investigating external corpus and clickthrough statistics for query expansion in the legal domain

A novel email abstraction scheme for spam detection

Answering general time sensitive queries

Scalable complex pattern search in sequential data

A spam resistant family of concavo-convex ranks for link analysis

Corpus microsurgery: criteria optimization for medical cross-language ir

Tag-based filtering for personalized bookmark recommendations

Search-based query suggestion

Combining concept hierarchies and statistical topic models

Metadata extraction and indexing for map search in web documents

Closing the loop in webpage understanding

Entity-based query reformulation using wikipedia

An empirical study of required dimensionality for large-scale latent semantic indexing applications

Content Provider	ACM Digital Library
Author	Bradford, Roger B.
Abstract	The technique of latent semantic indexing is used in a wide variety of commercial applications. In these applications, the processing time and RAM required for SVD computation, and the processing time and RAM required during LSI retrieval operations are all roughly linear in the number of dimensions, k, chosen for the LSI representation space. In large-scale commercial LSI applications, reducing k values could be of significant value in reducing server costs. This paper explores the effects of varying dimensionality. The approach taken here focuses on term comparisons. Pairs of terms are considered which have strong real-world associations. The proximities of members of these pairs in the LSI space are compared at multiple values of k. The testing is carried out for collections of from one to five million documents. For the five million document collection, a value of k ≈ 400 provides the best performance. The results suggest that there is something of an 'island of stability' in the k = 300 to 500 range. The results also indicate that there is relatively little room to employ k values outside of this range without incurring significant distortions in at least some term-term correlations.
Starting Page	153
Ending Page	162
Page Count	10
File Format	PDF
ISBN	9781595939913
DOI	10.1145/1458082.1458105
Language	English
Publisher	Association for Computing Machinery (ACM)
Publisher Date	2008-10-26
Publisher Place	New York
Access Restriction	Subscribed
Subject Keyword	Dimensionality Lsi Latent semantic indexing
Content Type	Text
Resource Type	Article

Central Library (ISO-9001:2015 Certified)
Indian Institute of Technology Kharagpur
Kharagpur, West Bengal, India | PIN - 721302

See location in the Map
03222 282435
Mail: support@ndl.gov.in

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in