Approximate Query Processing

QueryAnalyticsPerformance

Stable On by default · production-ready

What it is

Some questions are intrinsically expensive: “how many distinct users visited last month?” forces a classical engine to scan every row, deduplicate, and count. On petabyte-scale tables that can mean waiting minutes for a single number.

AetheriusDB keeps probabilistic summaries — HyperLogLog for cardinality, T-Digest for quantiles, count-min sketches for top-k — alongside each chunk of data. When you call an approximate function, the engine merges those tiny per-chunk summaries instead of rereading raw rows. The result comes back in milliseconds with a quantified error bound (typically under 2%) that you can reason about.

Why it matters

For dashboards, exploration, and real-time monitoring, the question is usually not “what is the exact answer?” but “is the answer roughly here, and is it changing?” Approximate queries are built for that second question — and when you do need the exact value, the exact path is right there alongside.

• Fast distinct counts — merging fixed-size sketches scales with chunk count, not row count.
• Quantiles without sorting — T-Digest gives p50/p95/p99 directly.
• Honest error bars — every approximate result carries an explicit margin you can render in dashboards.

flowchart LR
  classDef cube fill:#1f2640,stroke:#7c5cff,color:#e8eaf0
  classDef agg  fill:#1f2640,stroke:#00d4ff,color:#e8eaf0
  classDef out  fill:#1f2640,stroke:#4ade80,color:#4ade80

  C1[Chunk · HLL · T-Digest]:::cube
  C2[Chunk · HLL · T-Digest]:::cube
  C3[Chunk · HLL · T-Digest]:::cube
  CN[. . .]:::cube

  C1 --> M[Sketch merge]:::agg
  C2 --> M
  C3 --> M
  CN --> M

  M --> R[Approx answer<br/>+ error bound]:::out

How you use it

Use the APPROX_* family of functions where the answer can tolerate small bounded error, or use SQL’s TABLESAMPLE for stratified sampling on more complex aggregations.

-- Distinct user count over billions of events, in milliseconds.
SELECT APPROX_COUNT_DISTINCT(user_id) AS dau
FROM   events
WHERE  event_date = CURRENT_DATE;

-- p95 latency, no sort, no full scan.
SELECT APPROX_PERCENTILE(response_ms, 0.95) AS p95
FROM   request_log;

-- Stratified sample — exact aggregation over ~1% of the table.
SELECT AVG(revenue)
FROM   orders
  TABLESAMPLE BERNOULLI(1);

Approximate functions can return the estimate alongside a relative_error column when you request it, so dashboards can show “12.4M ± 1.2%” rather than implying false precision.

Key concepts

Term	What it means
HyperLogLog (HLL)	A fixed-size sketch (~1 KB) that estimates distinct counts with ~1.5% error, regardless of input size.
T-Digest	A compact structure that approximates quantiles with high accuracy at the tails (p95, p99).
Stratified sampling	Sampling balanced chunks of the table so the sample preserves the full table’s structure.
Confidence interval	The explicit bound reported next to each approximate value, derived from the sketch’s mathematics.

Notes & caveats

Tip

For dashboards, prefer the APPROX_* functions even when you have an exact equivalent. The payload is dramatically smaller and the latency is bounded, so user-facing widgets stay responsive even as the underlying table grows.

Heads up

Approximate functions are not appropriate for billing or audit pipelines, where a 1% error bound is unacceptable. Use the exact COUNT(DISTINCT ...) form there — the engine runs it gladly, just not in microseconds.

Advanced Indexing (PTI) →

Branch-level mirrors that make these summaries queryable at index speed.

Horizon Materialization →

Materialised views that pair well with sketches for predictive caching.

Adaptive Query Execution →

When exactness is required but estimates are off, the planner adapts.

SQL Grammar & Types →

The function and aggregate surface.