2014/06/19 by Jeremy Kepner, William Arcand, David Bestor +10 · 3 citations
Computer Science · Physics and Astronomy · #Benchmark (surveying) #Cloud Computing and Resource Management #Computer cluster #Consistency (knowledge bases) #Data modeling #Database server #Distributed and Parallel Computing Systems #Distributed database #Graph Theory and Algorithms #Real-time database #Software #Supercomputer #astro-ph.IM #cs.CE #cs.DB #cs.DC #cs.MS
paper · pdf · doi:10.1109/hpec.2014.7040945
6 pages; to appear in IEEE High Performance Extreme Computing (HPEC) 2014
arxiv created 2014/06/19 · openalex publication_date 2014/09/01 · arxiv updated 2015/05/26 · openalex created_date 2016/06/24 · openalex updated_date 2026/08/05
The Apache Accumulo database is an open source relaxed consistency database that is widely used for government applications. Accumulo is designed to deliver high performance on unstructured data such as graphs of network data. This paper tests the performance of Accumulo using data from the Graph500 benchmark. The Dynamic Distributed Dimensional Data Model (D4M) software is used to implement the benchmark on a 216-node cluster running the MIT SuperCloud software stack. A peak performance of over 100,000,000 database inserts per second was achieved which is 100× larger than the highest previously published value for any other database. The performance scales linearly with the number of ingest clients, number of database servers, and data size. The performance was achieved by adapting several supercomputing techniques to this application: distributed arrays, domain decomposition, adaptive load balancing, and single-program-multiple-data programming.