Skip to Main Content
EventStoreDB Ideas Portal

Ideas for improvements and new features in EventStoreDB, client libraries, and database extensions.

Workspace EventStoreDB
Categories Operations
Created by Alexey Zimarev
Created on Jan 4, 2024

Scheduled Index Merge

ESDB has an option to disabled automatic index merge, so users can do it manually. Usually, this option is enabled because automatic index merge can start at any time, and it might also get running on all the nodes at the same time. Since it's an IO-intensive operation, it can substantially degrade performance of the whole cluster at any time.

The suggestion is to support orchestrated, scheduled index merge, where cluster nodes would ensure that only one node is doing the merge at a time, and it is done during a pre-defined time window.

ESDB should also be aware of the index merge potential memory footprint, so it would not run out of memory and cause the node to restart. One way would be to observe GC events and run the merge when GC has finished the collection cycle and freed up managed memory.

  • Attach files