Continuous Ingestion
What it is
Section titled “What it is”Continuous ingestion is the path your write traffic takes when you stream rows into AetheriusDB over time — telemetry, click streams, IoT, CDC mirrors. Rows are accepted into a fast in-memory buffer and acknowledged in microseconds; a background worker materializes them into clean columnar storage out of band. The result: writers are acknowledged immediately and never block the analytical queries running alongside them.
Newly-arrived rows are visible to readers as soon as they are accepted — there is no “wait until materialization” gap from a client’s perspective.
Why use it
Section titled “Why use it”Continuous workloads overwhelm traditional engines because every insert contends for the same index node. The continuous path removes that contention by accepting writes into the buffer first and doing the heavier organizing work in the background.
- Line-rate ingest — practically anything the network socket can deliver, the engine can absorb.
- No write-side blocking — concurrent analytical queries are unaffected.
- Graceful overload — if writes outrun materialization, the engine applies backpressure rather than dropping data.
sequenceDiagram participant Client participant Ingest as Ingest buffer participant Reader Client->>Ingest: INSERT batch Ingest-->>Client: ACK (microseconds) Note over Ingest: organizes in background Reader->>Ingest: SELECT Ingest-->>Reader: rows visible immediately
How to use it
Section titled “How to use it”Declare a table as continuously ingested, then write to it from any wire protocol AetheriusDB speaks.
-- Declare a table that uses the continuous ingest path.CREATE TABLE telemetry ( device_id BIGINT, sampled_at TIMESTAMP, metric TEXT, value DOUBLE)WITH (ingest = 'continuous');
-- Write traffic streams in; acknowledgement is immediate.INSERT INTO telemetry VALUES (42, now(), 'cpu', 0.74);
-- Observe how much is buffered versus materialized.SELECT table_name, ring_rows, spilled_rows, materialized_rows FROM aether.ingest_stats WHERE table_name = 'telemetry';