NDLI: Making sense of big data with the Berkeley data analytics stack

Please wait, while we are loading the content...

Making sense of big data with the Berkeley data analytics stack

Education and career paths for data scientists

On the combination of relative clustering validity criteria

Providing multi-scale consistency for multi-scale geospatial data

Fast computation of approximate biased histograms on sliding windows over data streams

Search and result presentation in scientific workflow repositories

GPS: a graph processing system

Accelerating gene context analysis using bitmaps

Learning to explore scientific workflow repositories

A fast handshake join implementation on FPGA with adaptive merging network

Computational challenges in next-generation genomics

Parameter-free and domain-independent similarity search with diversity

Reasoning about RFID-tracked moving objects in symbolic indoor spaces

Multi-scale dissemination of time series data

Tuning large scale deduplication with reduced effort

RMiCS: a robust approach for mining coherent subgraphs in edge-labeled multi-layer graphs

The open connectome project data cluster: scalable analysis and vision for high-throughput neuroscience

Best of both worlds: relational databases and statistics

Adaptive exploration for large-scale protein analysis in the molecular dynamics database

A multidimensional data model with subcategories for flexibly capturing summarizability

Towards a universal tracking database

Multi-query scheduling for time-critical data stream applications

HmSearch: an efficient hamming distance query processing algorithm

SMIX: self-managing indexes for dynamic workloads

Real-time collaborative analysis with (almost) pure SQL: a case study in biogeochemical oceanography

Data management systems on GPUs: promises and challenges

Parallel online aggregation in action

Nearest group queries

GIPSY: joining spatial datasets with contrasting density

Learning uncertainty models from weather forecast performance databases using quantile regression

pcApriori: scalable apriori for multiprocessor systems

Inverted indices for particle tracking in petascale cosmological simulations

Optimizing fastquery performance on lustre file system

Nesting the earth mover's distance for effective cluster tracing

Astronomical data processing in EXTASCID

Publishing trajectories with differential privacy guarantees

Shortlisting top-K assignments

Graywulf: a platform for federated scientific databases and services

Semantic query reformulation: the NIF experience

Data vaults: a database welcome to scientific file repositories

Autonomous clustering for wireless sensor networks

Forecasting in hierarchical environments

Towards efficient discovery of coverage patterns in transactional databases

Bulk sorted access for efficient top-k retrieval

DoS: an efficient scheme for the diversification of multiple search results

Research lattices: towards a scientific hypothesis data model

Sharing confidential data for algorithm development by multiple imputation

Mining multidimensional contextual outliers from categorical relational data

Making sense of big data with the Berkeley data analytics stack

Content Provider	ACM Digital Library
Author	Franklin, Michael J.
Abstract	The Berkeley AMPLab was founded on the idea that the challenges of emerging Big Data applications require a new approach to analytics systems. Launching in early 2011, the project set out to rethink the traditional analytics stack, breaking down technical and intellectual barriers that had arisen during decades of evolutionary development. The vision of the lab is to seamlessly integrate the three main resources available for making sense of data at scale: Algorithms (such as machine learning and statistical techniques), Machines (in the form of scalable clusters and elastic cloud computing), and People (both individually as analysts and en masse, as with crowd-sourced human computation). To pursue this goal, we assembled a research team with diverse interests across computer science, forged relationships with domain experts on campus and elsewhere, and obtained the support of leading industry partners and major government sponsors. The lab is realizing its ideas through the development of a freely-available Open Source software stack called BDAS: the Berkeley Data Analytics Stack. In the nearly three years the lab has been in operation, we've released major components of BDAS. Several of these components have gained significant traction in industry and elsewhere: the Mesos cluster resource manager, the Spark in-memory computation framework, and the Shark query processing system. In this talk I'll describe the current state of BDAS with an emphasis on the key components that have been released to date. I'll then discuss ongoing efforts on machine learning scalability and ease of use, including the MLbase system, as our focus moves higher up the stack. Finally I will present our longer-term views of how all the pieces will fit together to form a system that can adaptively bring the right resources to bear on a given data-driven question to meet time, cost and quality requirements throughout the analytics lifecycle.
Starting Page	1
Ending Page	1
Page Count	1
ISBN	9781450319218
DOI	10.1145/2484838.2484884
Language	English
Publisher	Association for Computing Machinery (ACM)
Publisher Date	2013-07-29
Publisher Place	New York
Access Restriction	Subscribed
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Making Sense of Big Data with the Berkeley Data Analytics Stack

The Berkeley Data Analytics Stack: Present and future

VAST Challenge 2012: Visual analytics for big data

Building use Cases With Activity Reference Framework for Big Data Analytics

Analytics on public transport delays with spatial big data

Experiences Running and Optimizing the Berkeley Data Analytics Stack on Cray Platforms

A workflow-enabled big data analytics software stack for escience

Scaling Spark on HPC Systems

Big Data Analytics for Marketing Intelligence

Making sense of big data with the Berkeley data analytics stack

Similar Documents

Making Sense of Big Data with the Berkeley Data Analytics Stack

The Berkeley Data Analytics Stack: Present and future

VAST Challenge 2012: Visual analytics for big data

Building use Cases With Activity Reference Framework for Big Data Analytics

Analytics on public transport delays with spatial big data

Experiences Running and Optimizing the Berkeley Data Analytics Stack on Cray Platforms

A workflow-enabled big data analytics software stack for escience

Scaling Spark on HPC Systems

Big Data Analytics for Marketing Intelligence

Making sense of big data with the Berkeley data analytics stack