Accession Number:

ADA596393

Title:

Large-Scale Exploratory Analysis, Cleaning, and Modeling for Event Detection in Real-World Power Systems Data

Descriptive Note:

Conference paper

Corporate Author:

PACIFIC NORTHWEST NATIONAL LAB RICHLAND WA

Report Date:

2013-11-01

Pagination or Media Count:

10.0

Abstract:

In this paper, we present an approach to large-scale data analysis, Divide and Recombine DR, and describe a hardware and software implementation that supports this approach. We then illustrate the use of DR on large-scale power systems sensor data to perform initial exploration discover multiple data integrity issues, build and validate algorithms to filter bad data, and construct statistical event detection algorithms. This paper also reports on experiences using a non-traditional Hadoop distributed computing setup on top of a HPC computing cluster.

Subject Categories:

  • Computer Systems

Distribution Statement:

APPROVED FOR PUBLIC RELEASE