DarkHorseDB

Analytics at Scale using Modest Hardware

We are excited to announce our powerful new database engine, DarkHorseDB 1.0. We're working through a number of benchmarks while this engine is in the Beta stage and will keep the public updated as we go. The primary objective in building DarkHorseDB was to make the most out of existing hardware without an absolute requirement on expensive cloud computing, allowing for significant insights into large scale (100M or 1B+ records etc) data even locally on a laptop if necessary, while also allowing cloud computing options.

CPU+GPU+FOG

We're combining the power DarkHorseDB with Desktop level CPUs and GPUs, along with Fog Mining. The results so far are pretty impressive, allowing for more efficient and cost effective Big Data analytics in part by re-using already existing hardware within an organization's infrastructure.

Benchmark results

Please see the benchmarked results sections below to see why we are excited!! For details about the TPC benchmarks, please see tpc.org for more details.

DarkHorseDB Performance over the TPC Benchmarks

Side-by-side Comparisons of our Benchmark to Other Options

Over the TPC benchmark, the bottom two rows of the above table shows the power of DarkHorseDB with a modest hardware configuration versus other significantly less cost effective options. In other words given what you pay for in terms of hardware, DarkHorseDB is more giving (bigger bang for your buck). We're working to produce a 100K scale result so that the value of DarkHorseDB is even more clear, should it be claimed that scaling doesn't hold between the left 2 columns and the one on the right.

DarkHorseDB vs DuckDB over the TPC-H Benchmark

This is a conservative estimate of performance for the DarkHorseDB versus the (also blazing fast) DuckDB engine for SF=30, with results in seconds. Another 2.6x speedup is achieved if the tables were held in RAM instead of loaded from disk at the start of each query. Another 31x speedup is achieved using the GPU. Any way of viewing the results, DarkHorseDB is fast!

DarkHorseDB Query Runtimes Example

Comparative results in tabular form between DarkHorseDB and DuckDB over the TPC-H (SF=30) benchmark

The table above shows comparative results in tabular form between DarkHorseDb and DuckDB over the TPC-H (SF=30) benchmark. A 2.6x and 31x speedup further increases the baseline 2.31 speedup of DarkHorseDB over DuckDB by fully utilizing RAM and GPU.