How is data virtualization configured for petabyte-scale data warehouses, and how fast is the caching engine?

Timbr’s data virtualization engine is an optimized version of Spark that scales up and out with an auto-scaler, and is used by customers across petabytes of data. The caching engine offers four tiers from normal to fastest:

1. Local Database – caches where the data lives, avoiding high-load compute time queries.
2. Data Lake – works across big data environments.
3. SSD – provides data locality and 10x faster performance than a data lake.
4. In-Memory – the fastest option, supporting near real-time performance with automated projections for optimized cache.

All tiers include a Job Scheduler that allows customers to materialize and cache in three clicks as CRON jobs, setting recurring refresh based on time intervals or triggers. All options also include partitioning and, when supported by the target source, indexes.