Conference proceeding
Data Reduction, Compression, and Recovery for Online Performance Monitoring
2019 IEEE 12TH INTERNATIONAL CONFERENCE ON CLOUD COMPUTING (IEEE CLOUD 2019), v 2019-
01 Jan 2019
Abstract
The volume of data needed for effective monitoring of datacenters poses significant challenges in its collection, transmission, analysis, and storage. Considering a setting wherein data collected locally at a server is sent to a monitoring station for analysis, this paper develops computationally efficient methods for systematic reduction of this data during the transfer and its subsequent recovery at the monitoring station. Specifically, we develop a low-cost method of obtaining a sparse representation of the data collected at each individual server while preserving a specified fidelity with respect to the original signal. The sparsified representation obtained from the data-collection step is amenable to further compression prior to transmission to the monitoring station. Upon receipt of the compressed-data stream at the monitoring station, a method of sparse-signal recovery is utilized to reconstruct the original full-length signal for further analysis. The techniques are validated using workload traces collected from one of Google's production clusters. Experiments show that the achieved data reduction, which is a function of the specified fidelity, is significant: to reconstruct the signal with a fidelity between 90%-95%, the sample size that must be be transferred to the monitoring station is under 10% of the original. We also verify that the recovered signal tracks the target minimum fidelity requirements specified by the operator with high precision.
Metrics
Details
- Title
- Data Reduction, Compression, and Recovery for Online Performance Monitoring
- Creators
- Salvador DeCelles - Drexel UniversityMatthew C. Stamm - Drexel UniversityNagarajan Kandasamy - Drexel UniversityIEEE
- Publication Details
- 2019 IEEE 12TH INTERNATIONAL CONFERENCE ON CLOUD COMPUTING (IEEE CLOUD 2019), v 2019-
- Series
- IEEE International Conference on Cloud Computing
- Publisher
- IEEE
- Number of pages
- 8
- Resource Type
- Conference proceeding
- Language
- English
- Academic Unit
- Electrical and Computer Engineering
- Web of Science ID
- WOS:000556208000039
- Scopus ID
- 2-s2.0-85072333206
- Other Identifier
- 991019168982604721
InCites Highlights
Data related to this publication, from InCites Benchmarking & Analytics tool:
- Web of Science research areas
- Computer Science, Hardware & Architecture
- Computer Science, Information Systems
- Computer Science, Theory & Methods