THE BERKELEY DATA ANALYSIS SYSTEM (BDAS): AN OPEN SOURCE PLATFORM FOR BIG DATA ANALYTICS

reportActive / Technical Report | Accession Number: AD1039023 | Open PDF

Abstract:

The goal of this proposal was to deliver a modular open-source software stack that can support a new generation of large-scale analytic tools that provide answers over arbitrarily large datasets. This work was carried out by Berkeleys AMPLab, a research lab consisting of eleven faculty members and over 40 students. In addition to this grant, AMPLab which ended in December 2016 was supported by industry affiliates and an NSF Expeditions grant. This grant was instrumental in improving our software stack, Berkeley Data Analytic System BDAS, so that it can serve as a platform for the broader community. In particular, this grant enabled us to implement significant portions of the code-bases, integrate BDAS with commonly used tools, and make BDAS much easier to manage. In addition, it allowed us to extend the functionality of BDAS in several key area, including streaming, and query processing. Thanks to xData, BDAS has enjoyed a big success both in academia and industry. Today, Apache Spark is used by thousands of companies in production and counts over 400K meetup members worldwide, while Apache Mesos and Alluxio formerly known as Tachyon are used by hundreds of companies around the world.

Author(s):
Stoica, Ion ; Franklin, Michael ; Jordan, Michael ; Fox, Armando ; Joseph, Anthony ; Mahoney, Michael ; Katz, Randy ; Patterson, David ; Shenker, Scott
Author Organization(s):
University of California, Berkeley Berkeley United States
Descriptive Note:
Technical Report,01 Sep 2012,31 Mar 2017
Pagination:
0029

Security Markings

DOCUMENT & CONTEXTUAL SUMMARY

Distribution:
Approved For Public Release
Distribution Statement:
Approved For Public Release;

RECORD

Collection: TR
Identifying Numbers
Report Number(s):
AFRL/RI-AFRL-RI-RS-TR-2017-173
Task Number(s):
A0
Project Number(s):
XDAT
Monitor Series:
AFRL-RI-RS-TR-2017-173
 Subject Terms
Joint Capability Areas:
JCA_5_Command and Control; JCA_5.2_Understand; JCA_5.2.3_Share Knowledge and Situational Awareness; JCA_5.3_Planning; JCA_5.5.2_Task; JCA_5.5_Direct; JCA_1.2.1_Training; JCA_4_Logistics; JCA_1.2.3_Educating
Modernization Areas:
Autonomy; Microelectronics
Communities of Interest:
Engineered Resilient Systems
Descriptor(s):
computer programs, Data Management, DATA ANALYSIS, Machine Learning, Cloud Computing
Field(s)/Group(s):
Computer Programming and Software
Keyword(s):
Cluster Computing, Crowdsourcing, Big Data, BDAS(Berkeley Data Analytic System)
Report Date:
2017 Sep 01
Creation Date:
2017 Sep 28