What is a multi-tenant query engine?

A service that runs SQL for many independent tenants over shared, pooled compute rather than a dedicated cluster per customer. It separates a control plane (the gateway, admission control, scheduler, and planner — long-lived and shared) from a data plane (ephemeral executor pools that actually scan object storage and run the query). Snowflake calls this the "multi-cluster, shared-data architecture": storage and compute are "handled by two loosely coupled, independently scalable services."

How does a serverless query engine stop one tenant from slowing everyone else down?

Two mechanisms. Fairness: per-tenant admission queues with weighted fair sharing, so each tenant gets a guaranteed slice of slots and bursts only borrow idle capacity. Isolation: every query runs under a per-query memory/CPU cap (via cgroups or containers), and a runaway is killed rather than allowed to starve its neighbours. Interactive and ETL work run in separate executor pools so a slow batch job never sits in front of a dashboard. Snowflake keeps worker nodes unshared across warehouses "resulting in strong performance isolation for queries."

How can a query survive a spot instance being reclaimed mid-execution?

By making the unit of recovery a fragment, not the whole query. A query is broken into many small tasks whose intermediate outputs are spooled durably. When a spot host gets its ~2-minute interruption notice, the scheduler drains it, and only the fragments that were in flight on that host recompute from their inputs (lineage) on surviving executors — completed work is preserved, so the progress bar never resets. Trino calls this fault-tolerant execution: "intermediate exchange data is spooled and can be re-used by another worker in the event of a worker outage or other fault during query execution."

What happens if the coordinator or driver node dies mid-query?

That is the one failure fragment-level recovery cannot mask. The coordinator holds the query plan and the assignment state that tells survivors what to recompute; lose it and there is nothing to reroute fragments to. So the coordinator (and other control-plane services) run on reliable on-demand instances, not spot, and are replicated — a worker dying is a blip that recomputes; a coordinator dying fails the query and it is retried from scratch. Only stateless, reclaimable executors belong on spot.

How does a query engine autoscale to zero?

Because compute is decoupled from storage, the executor fleet holds no durable state — so it can shrink to nothing when idle and grow on demand. The scaling signal is queue depth (pending admitted queries) plus utilisation; a warm pool of a few pre-booted executors absorbs the first burst while cold machines boot, and hysteresis (scale up fast, down slow) avoids thrashing. As Snowflake puts it, virtual warehouses "are pure compute resources… created, destroyed, or resized at any point, on demand," and you "pay only for the queries you run" (Athena).

Design a Multi-Tenant Query Engine on Object Storage

Thousands of tenants, ten thousand concurrent queries, sub-second dashboards next to multi-hour ETL — all on a fleet of reclaimable spot machines. Split the control plane from the data plane, schedule fairly, and make a query survive its own executors being yanked mid-flight.

System design · Systems. The source ↗

A free, interactive, animated visual explainer of Design a Multi-Tenant Query Engine on Object Storage — built to be understood, not skimmed.

Questions

What is a multi-tenant query engine?: A service that runs SQL for many independent tenants over shared, pooled compute rather than a dedicated cluster per customer. It separates a control plane (the gateway, admission control, scheduler, and planner — long-lived and shared) from a data plane (ephemeral executor pools that actually scan object storage and run the query). Snowflake calls this the "multi-cluster, shared-data architecture": storage and compute are "handled by two loosely coupled, independently scalable services."
How does a serverless query engine stop one tenant from slowing everyone else down?: Two mechanisms. Fairness: per-tenant admission queues with weighted fair sharing, so each tenant gets a guaranteed slice of slots and bursts only borrow idle capacity. Isolation: every query runs under a per-query memory/CPU cap (via cgroups or containers), and a runaway is killed rather than allowed to starve its neighbours. Interactive and ETL work run in separate executor pools so a slow batch job never sits in front of a dashboard. Snowflake keeps worker nodes unshared across warehouses "resulting in strong performance isolation for queries."
How can a query survive a spot instance being reclaimed mid-execution?: By making the unit of recovery a fragment, not the whole query. A query is broken into many small tasks whose intermediate outputs are spooled durably. When a spot host gets its ~2-minute interruption notice, the scheduler drains it, and only the fragments that were in flight on that host recompute from their inputs (lineage) on surviving executors — completed work is preserved, so the progress bar never resets. Trino calls this fault-tolerant execution: "intermediate exchange data is spooled and can be re-used by another worker in the event of a worker outage or other fault during query execution."
What happens if the coordinator or driver node dies mid-query?: That is the one failure fragment-level recovery cannot mask. The coordinator holds the query plan and the assignment state that tells survivors what to recompute; lose it and there is nothing to reroute fragments to. So the coordinator (and other control-plane services) run on reliable on-demand instances, not spot, and are replicated — a worker dying is a blip that recomputes; a coordinator dying fails the query and it is retried from scratch. Only stateless, reclaimable executors belong on spot.
How does a query engine autoscale to zero?: Because compute is decoupled from storage, the executor fleet holds no durable state — so it can shrink to nothing when idle and grow on demand. The scaling signal is queue depth (pending admitted queries) plus utilisation; a warm pool of a few pre-booted executors absorbs the first burst while cold machines boot, and hysteresis (scale up fast, down slow) avoids thrashing. As Snowflake puts it, virtual warehouses "are pure compute resources… created, destroyed, or resized at any point, on demand," and you "pay only for the queries you run" (Athena).

Design a Multi-Tenant Query Engine on Object Storage

Questions

Related explainers