Skip to content

Improve background cleanup of persistence actors to be more resource efficient #2274

@thjaeckle

Description

@thjaeckle

Under certain scenarios it is not ideal that Ditto loads Things/Policies Actors into memory regularly just for the "Background cleanup" (of event journal and snapshots).
E.g. when using Ditto for "Things" which are:

  • rarely updated (e.g. only once a month)
  • rarely accessed (e.g. only by few users via a frontend)
  • but are e.g. big in payload size
  • or are very many (millions)

Some ideas:

  • Don't load PersistenceActor in memory if no cleanup is required
    • E.g. by filtering those (things/policies) which do not have recent changes out directly in the "cleanup stream"
    • Ideally not even sending any message to the PA (which would recover it from the DB)
  • If a PA was loaded into memory just for cleanup:
    • Shut it down a lot earlier than "normally" (2 days is the default for things in Helm config) to free up memory
    • Make it configurable (if enabled and how long those shall be kept "live")

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions