Loading...
Please wait, while we are loading the content...
Similar Documents
Autonomous Data Management and Federation to Support High-throughput Query Evaluations over Voluminous Datasets
| Content Provider | Semantic Scholar |
|---|---|
| Author | Malensek, Matthew Pallickara, Sangmi Lee Pallickara, Shrideep |
| Copyright Year | 2015 |
| Abstract | In recent years, both the breadth and depth of information being generated and stored has continued to grow rapidly, causing an information explosion. Observational devices and remote sensing equipment are no exception to this rule, giving researchers new avenues for detecting and predicting phenomena at a global scale. To cope with these storage loads, hybrid clouds offer an elastic solution that also satisfies processing and budgetary needs. This paper describes our algorithms and system design for dealing with voluminous datasets in a hybrid cloud setting. Our distributed storage framework autonomously tunes in-memory data structures and query parameters to ensure efficient retrievals and minimize resource consumption. To circumvent processing hotspots, we predict changes in incoming traffic and federate our query resolution structures to the public cloud for processing. We also demonstrate the efficacy of our framework on a real-world, petabyte dataset consisting of over 20 billion files. |
| File Format | PDF HTM / HTML |
| Alternate Webpage(s) | http://galileo.cs.colostate.edu/papers/FederatedQueries-ieee-CC.pdf |
| Language | English |
| Access Restriction | Open |
| Content Type | Text |
| Resource Type | Article |
