140 likes | 147 Views
Develops cache-oblivious algorithms for optimal query processing without cache parameter knowledge, achieving high performance in complex memory hierarchies. Focus on CPU caches, Cache-Conscious (CC) techniques, automatic tuning, and buffer optimizations. Challenges, opportunities, and empirical results discussed.
E N D
Cache-Oblivious Query Processing Bingsheng He, Qiong Luo {saven, luo}@cse.ust.hk Department of Computer Science & Engineering Hong Kong University of Science & Technology
Cache-Oblivious Algorithms[Frigo et al., FOCS 1999] • Assuming no knowledge about cache parameter values, e.g., cache size • Optimal cache complexity • For an ideal cache model • Two-level hierarchy: cache on top of memory • Automatic, optimal cache replacement • Fully associative • For more realistic cache models as well
Motivation • Relational database systems have too many knobs to tune for performance. • Tuning may be difficult, ineffective, and sometimes infeasible. • The memory hierarchy becomes increasingly complex.
Our focus: CPU caches Memory Hierarchy Main memory Disk L2 L1 Registers CPU
Cache-Conscious (CC) Techniques • Aware of cache parameters of a target level in a specific memory hierarchy • Cache block size, e.g., B+-trees • Cache capacity, e.g., blocked NLJ • Achieve a high performance with correct parameter values
Tuning the parameter is difficult • The best parameter value varies with the platform. • It may be none of the cache parameters of the platform. • It may vary with different data and algorithmic characteristics.
Our Goal To automatically and consistently achieve a good performance on various memory hierarchies at all times
Challenges • How to optimize query processing cache-obliviously? • Divide-and-conquer methodology • Amortization methodology • How to achieve a comparable overall performance with fine-tuned cache-conscious algorithms? • Work complexity • Recursion overhead
Fit into the cache Reuse Divide-and-conquer
R Partitioner Buffer Partition Amortization • Reduce the average cost for a set of operations • A buffer hierarchy • Buffer sizes are recursively defined.
Limitations • Employ sophisticated data structures and mechanisms. • Require some automatic and machine-independent optimization to improve their efficiency.
Opportunities • Storage models • Transactions • New architectural features • CMP/SMT • GPUs • Transactional memory
Conclusion • First cache-oblivious query processor • Complexity results on our CO alg. • Empirical results of our CO alg. on three hardware platforms in comparison with their CC counterparts http://www.cse.ust.hk/cactus/