Real-Time Information Extraction from Big Data
Institute for Defense Analyses Alexandria
Pagination or Media Count:
We are drowning under the 3 Vs volume, velocity and variety of big data. Real-time information extraction from big data,at global as well as local levels, is critical for making rapid decisions in many important DoD scenarios. However, such real-time information from big data is enormously complex and extremely challenging. We argue that data movement is ofcrucial importance in big data analytics, and show that recently developed photonic communication technologies has the potential to extract real-time information, and improve performanceWatt by a hundredfold with corresponding cost savings at the local, rack-size level, provided the entire system, consisting of communications hardware and network, computer hardware, algorithms, software and architecture, is holistically optimized. We provide examples of some applications of interest to DoD for which such holistic system optimization could be a game changer.
- Computer Programming and Software
- Information Science