Logo image
Back
Optimal caching of large multi-dimensional datasets
Dissertation   Open access

Optimal caching of large multi-dimensional datasets

Dinesh Obalappa
Doctor of Philosophy (Ph.D.), Drexel University
May 2004
DOI:
https://doi.org/10.17918/etd-307
pdf
Obalappa_Dinesh_2004646.41 kBDownloadView
PDF Open Access Open Access (License Unspecified)

Abstract

Electrical and computer engineering Cache memory Multidimensional databases
We propose a novel organization for multi-dimensional data based on the conceptof macro-voxels. This organization improves computer performance by enhancingspatial and temporal locality. Caching of macro-voxels not only reduces therequired storage space but also leads to an efficient organization of the dataset resulting in faster data access. We have developed a macro-voxel caching theory that predicts the optimal macro-voxel sizes required for minimum cache size and access time. The model also identifies a region of trade-off between time and storage, which can be exploited in making an efficient choice of macro-voxel size for this scheme. Based on the macro-voxel caching model, we have implemented a macro-voxel I/O layer in C, intended to be used as an interface between applications and datasets. It is capable of both scattered access, typical in online applications, and row/column access, typical in batched applications. We integrated this I/O layer in the ALIGN program (online application) which aligns images based on 3D distance maps; this improved access time by a factor of 3 when accessing local disks and a factor of 20 for remote disks. We also applied the macro-voxel caching scheme on SPEC.s Seismic (batched application) benchmark datasets which improved the read process by a factor of 8.

Metrics

62 File views/ downloads
34 Record Views

Details

Logo image