Accession Number:

AD1000187

Title:

Real-Time Information Extraction from Big Data

Descriptive Note:

Technical Report

Corporate Author:

Institute for Defense Analyses Alexandria

Report Date:

2015-10-01

Pagination or Media Count:

20.0

Abstract:

We are drowning under the 3 Vs volume, velocity and variety of big data. Real-time information extraction from big data,at global as well as local levels, is critical for making rapid decisions in many important DoD scenarios. However, such real-time information from big data is enormously complex and extremely challenging. We argue that data movement is ofcrucial importance in big data analytics, and show that recently developed photonic communication technologies has the potential to extract real-time information, and improve performanceWatt by a hundredfold with corresponding cost savings at the local, rack-size level, provided the entire system, consisting of communications hardware and network, computer hardware, algorithms, software and architecture, is holistically optimized. We provide examples of some applications of interest to DoD for which such holistic system optimization could be a game changer.

Subject Categories:

  • Computer Programming and Software
  • Information Science

Distribution Statement:

APPROVED FOR PUBLIC RELEASE