EventStoreDB Ideas Portal

Ideas for improvements and new features in EventStoreDB, client libraries, and database extensions.

Add a new idea Subscribe
Workspace EventStoreDB
Categories Operations
Created by Alexey Zimarev
Created on Jan 4, 2024

RELATED IDEAS

Delete streams without a trace

Customers who actively clean up their database encounter a situation when the scavenge.db file grows uncontrollably because it keeps the information about stuff that has been deleted.

Eventually, it might lead to a situation when the scavenge.db file will be larger than the database itself, and it will be very confusing.

If ESDB allows deleting streams without leaving any trace, it won't need to track those streams in scavenge.db, and it will save space. It might also help with compliance if someone has things like username in the stream name, and we keep it, although inaccessible (not entirely true as $streams will show it).

  • Attach files
  • Admin
    Alexey Zimarev
    Reply
    |     Jun 26, 2024

    I believe that we need to elaborate on use cases. For example, one thing I heard about requires streams to be deleted completely so the new stream with the same name can be created with NoStream state right away. When using UnsafeIgnoreHardDeletes it would be necessary to wait until the stream and the meta gets scavenged. Unless I don't fully understand how it works.

  • Admin
    Tim Coleman
    Reply
    |     May 8, 2024

    We do currently have UnsafeIgnoreHardDeletes which does this, but as the name implies it is unsafe - the user has to take care to completely remove a stream across the cluster before re-creating it (or alternatively, do not recreate it). It would be good if we could make it a better user experience.


    On the downside, if streams are recreated, then of course a two reads of event 5 in stream A could return different events at different times, breaking immutability.