NDLI: Conquering big data with spark and BDAS

Please wait, while we are loading the content...

Rumor source detection with multiple observations: fundamental limits and algorithms

An online auction framework for dynamic resource provisioning in cloud computing

Index policies for a multi-class queue with convex holding cost and abandonments

Conquering big data with spark and BDAS

Hershel: single-packet os fingerprinting

The behavior of epidemics under bounded susceptibility

Tunable survivable spanning trees

GemDroid: a framework to evaluate mobile platforms

Pricing link by time

CApRI: CAche-conscious data reordering for irregular codes

On the energy and performance of commodity hardware transactional memory

Scheduling using interactive oracles: connection between iterative optimization and low-complexity scheduling

Collecting, organizing, and sharing pins in pinterest: interest-driven or social-driven?

Energy procurement strategies in the presence of intermittent sources

Concave switching in single and multihop networks

Noise can help: accurate and efficient per-flow latency measurement without packet probing and time stamping

Studying social networks at scale: macroscopic anatomy of the twitter social graph

Balanced resource allocations across multiple dynamic MapReduce clusters

Understanding the impact of network dynamics on mobile video user engagement

Serving content with unknown demand: the high-dimensional regime

Neighbor-cell assisted error correction for MLC NAND flash memories

Impact of correlated mobility and cluster scalability on connectivity of wireless networks

Unified localization framework using trajectory signatures

Jointly clustering rows and columns of binary matrices: algorithms and trade-offs

Randomized auction design for electricity markets between grids and microgrids

The power of online learning in stochastic network optimization

A measurement study of google play

Stochastic bandits with side observations on networks

On the relevance of adversarial queueing theory in practice

Error estimating codes for insertion and deletion channels

Network-design sensitivity analysis

ANATOMY: an analytical model of memory system performance

Unleashing the potentials of dynamism for page allocation strategies in SSDs

TS-CLOCK: temporal and spatial locality aware buffer replacement algorithm for NAND flash storages

Filter & follow: how social media foster content curation

Pricing data center demand response

Is sharing with retransmissions causing instabilities?

IntroPerf: transparent context-sensitive multi-layer performance inference using system stack traces

On maximizing diffusion speed in social networks: impact of random seeding and clustering

Data dissemination performance in large-scale sensor networks

The multi-shop ski rental problem

The efficacy of error mitigation techniques for DRAM retention failures: a comparative experimental study

Traffic congestion: models, costs and optimal transport

Multi-source multipath HTTP (mHTTP): a proposal

Privacy tradeoffs in predictive analytics

Non-work-conserving effects in MapReduce: diffusion limit and criticality

Non-intrusive, out-of-band and out-of-the-box systems monitoring in the cloud

Movers and shakers: kinetic energy harvesting for the internet of things

GDM: device memory management for gpgpu computing

Randomized routing schemes for large processor sharing systems with multiple service rates

Performance analysis of MAC for power-line communications

Automated analysis of multithreaded programs for performance modeling

Impact of DHCP churn on network characterization

A comparison of core power gating strategies implemented in modern hardware

Online algorithms for joint application-VM-physical-machine auto-scaling in a cloud

Topic modeling from network spread

Distributed optimal load shedding for disaster recovery in smart electric power grids: a second-order approach

BATS: budget-constrained autoscaling for cloud performance optimization

TARDIS: stably shifting traffic in space and time

What's your choice?: learning the mixed multi-nomial

Exact analysis of TTL cache networks: the case of caching policies driven by stopping times

Revisiting caching in content delivery networks

Measuring throughput of data center network topologies

FLOWR: a self-learning system for classifying mobileapplication traffic

Efficient task replication for fast response times in parallel computation

Accurate and robust time reconstruction for deployed sensor networks

Effect of proactive serving on user delay reduction in service systems

Optimal energy source selection and capacity planning for green datacenters

Conquering big data with spark and BDAS

Content Provider	ACM Digital Library
Author	Stoica, Ion
Abstract	Today, big and small organizations alike collect huge amounts of data, and they do so with one goal in mind: extract "value" through sophisticated exploratory analysis, and use it as the basis to make decisions as varied as personalized treatment and ad targeting. Unfortunately, existing data analytics tools are slow in answering queries, as they typically require to sift through huge amounts of data stored on disk, and are even less suitable for complex computations, such as machine learning algorithms. These limitations leave the potential of extracting value of big data unfulfilled. To address this challenge, we are developing Berkeley Data Analytics Stack (BDAS), an open source data analytics stack that provides interactive response times for complex computations on massive data. To achieve this goal, BDAS supports efficient, large-scale in-memory data processing, and allows users and applications to trade between query accuracy, time, and cost. In this talk, I'll present the architecture, challenges, results, and our experience with developing BDAS, with a focus on Apache Spark, an in-memory cluster computing engine that provides support for a variety of workloads, including batch, streaming, and iterative computations. In a relatively short time, Spark has become the most active big data project in the open source community, and is already being used by over one hundred of companies and research institutions.
Starting Page	193
Ending Page	193
Page Count	1
File Format	PDF
ISBN	9781450327893
DOI	10.1145/2591971.2611389
Language	English
Publisher	Association for Computing Machinery (ACM)
Publisher Date	2014-06-16
Publisher Place	New York
Access Restriction	Subscribed
Subject Keyword	Cluster computing Big data Distributed algorithms
Content Type	Text
Resource Type	Article

Central Library (ISO-9001:2015 Certified)
Indian Institute of Technology Kharagpur
Kharagpur, West Bengal, India | PIN - 721302

See location in the Map
03222 282435
Mail: support@ndl.gov.in

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in