Memory Compaction

StorageMaintenanceBackground

Stable On by default · production-ready

What it is

When you update or delete rows, AetheriusDB writes new versions rather than overwriting in place. Over time the old versions become unreachable — they occupy memory with nothing referencing them. The compactor is the background process that reclaims that space and keeps the live data packed together so scans stay fast.

It runs automatically. You do not schedule it or trigger it for routine operation.

Why it matters

Most databases make compaction loud: long stop-the-world pauses or noisy bursts of disk activity. AetheriusDB’s compactor runs continuously in the background, repacking only the regions that need it, and never pauses your queries to do so.

• No GC pauses — compaction is incremental and reactive, not a scheduled stall.
• Steady query speed — keeping live data contiguous keeps scans fast as tables churn.
• No query interference — in-flight scans keep their consistent snapshot; new scans see the compacted layout.

How you use it

The compactor runs on its own. You can inspect how much space is reclaimable, force a pass before a benchmark or audit, and tune how aggressively it works on specific tables.

-- See how much memory is reclaimable across the database.
SELECT table_name,
       live_bytes,
       orphaned_bytes,
       round(orphaned_bytes * 100.0 / (live_bytes + orphaned_bytes), 1) AS waste_pct
  FROM aetherius_system.memory_fragmentation
 ORDER BY orphaned_bytes DESC;

-- Force a compaction pass on a heavily-updated table.
COMPACT TABLE sessions;

-- Adjust how aggressively the background compactor handles a table.
ALTER TABLE hot_updates
  SET compaction_target_pct = 75;   -- repack when live share drops below 75%

-- Inspect recent compactor activity.
SHOW COMPACTION STATUS;

The fragmentation view is your primary signal — sustained waste above roughly 30% is worth tuning. Forcing a compaction on a single table is cheap; forcing it across everything at once is not, so save explicit compaction for the tables that need it.

Illustrative syntax

The memory_fragmentation columns and the COMPACT TABLE / compaction_target_pct spelling shown here illustrate the feature shape; confirm the exact grammar against your server version.

Reclaiming disk and shrinking the resident set

Two more knobs govern space beyond live-row compaction:

• DROP TABLE reclaims disk immediately. Dropping a table unlinks its dedicated container (so the bytes are freed at once and can’t resurface on restart); tables that share a container are compacted out, keeping only the survivors. No manual VACUUM step is required.
• Resident LZ4 compression (opt-in). Start the daemon with AETHERIUS_COMPRESS_RESIDENT=1 to keep sealed cubes LZ4-compressed in RAM, decompressing them on scan. This trades a little read CPU for a smaller memory footprint; results are identical. The default keeps cubes uncompressed for zero-copy reads. For write-side memory pressure during large loads, see AETHERIUS_INGEST_SPILL in Configuration.

Notes & caveats

Tip

For tables with bursty update workloads (sessions, queues, counters), lowering compaction_target_pct keeps memory tight at the cost of slightly more background work. For batch-loaded tables that rarely change, the defaults are usually fine.

Long-open transactions block reclamation

A long-running open transaction holds references to old row versions and prevents the compactor from reclaiming them. If you see reclaimable space that will not clear, check for a forgotten open transaction before tuning compaction settings.

Multi-Tier Residency →

Compaction works with eviction to keep the hot tier neither too full nor too fragmented.

Caching →

The cache whose churn the compactor cleans up after.

Data Persistence Engine →

How durable storage is laid out and reclaimed on disk.

Operate →

Running and maintaining a production deployment.