PData support to reuse memory proposal

[bogdandrutu]

Component(s)

pdata

Describe the issue you're reporting

The goal of this issue is to get to a point where whenever we create a new PData, to be able to reuse as much as possible previous heap allocated objects (especially for internal structs like Spans/Metrics/etc.).

There is a well known technique in golang on how to do this using sync.Pool and this proposal will leverage that for allocating/deallocating the objects, so nothing "revolutionizing" here.

Object Allocation - Every time an object is allocated will get a new object from its own memory pool. Here are all the cases when this happens:

• UnmarshalProto and UnmarshalJSON.
• Append* operations for slices.
• CopyTo operations for any message that we expose (simple message or slice).
• New* operations.
• Put* operations on pcommon.Map
• Set* operations on pcommon.Value

All these cases can be easily implemented during because of the PData design that hides the internal allocated memory objects from the user.

Object Deallocation - Every time an object is deallocated will put it back to its own memory pool. When a object is deallocated all its sub-objects are also deallocated. Here are all the cases when this happens:

• CopyTo operations for slices when the destination slice is larger than the source and extra allocated objects must be cleared.
• RemoveIf operations for slices (including pcommon.Map) when objects are removed.
• Set* operations on pcommon.Value when previous value was not empty.
• When a PData object is added to the pipeline and "fully" processed, the whole object (including all sub-objects) can be deallocated.

All these cases (except the last one) can be easily implemented during because of the PData design that hides the internal allocated memory objects from the user.

How to determine when an object is "fully" processed?

The first thing we need to do is to implement a ref-counting mechanism that can be used to keep count of all references for a specific object. Every new object when allocated starts with a ref-count of 1 and will expose functions to increase/decrease reference counts, when hitting 0 the object is deallocated.

IMPORTANT we will keep references only for the top level objects plog.Logs, pmetric.Metrics, pprofile.Profiles, ptrace.Traces. The reason behind this is that any "sub-object" if ever created it is created to eventually be "moved" to a top level object, and when memory moved to one of the top level objects they will deallocate these memory when they go out of scope.

The proposal relies on the fact that all allocated PData objects will "eventually" hit one of the processing pipeline, and that is when the pipeline reference counting begins. This is how the algorithm will work:

1. Will ingest a "pipeline refcount consumer" after every component that calls into the pipeline (receiver/processor/connector). When a new object (first time seen) is added to the processing pipeline (calls next from the component that created it receiver/processor/connector) will mark the object state (we have a state already in the top level objects that only supports mutable/readonly but will extend) as seen by the pipeline and will only unref (not ref) when the processing finish (calls to next return) because the first reference was created when the object was created. If the object memory was moved to another object there is no problem because the new top-level owner will own that memory and deallocate when done.
2. Change the fanoutconsumer to correctly deallocate (call unref) the copied ojects when that happens for mutable pipelines. For the not copied objects nothing is needed.
3. Change every component that switches from sync processing model to async (uses any kind of queue/async ops) to ref and unref the top level data. There are some important components that need to be changed here: exporter sending queue, batch processor, group by trace id, etc. There should not be more than 10 components that need to do this.
4. Ensure that no component keeps a reference to any data without referencing the top object (especially sub-objects like attributes, etc.). This was always forbidden, because we clearly say in pdata that the objects are reference to the underlying data and a component does not have ownership of the data after it returns from "Consume" calls. But there may still be mistakes and we need to make sure we capture this in tests.

Alternatives:

• An alternative implementation can be done by asking every component that creates a new PData object to call "Release/Delete" after done using it. This will be a very large change and most likely hard to implement across all components.

Execution Plan for the reuse memory feature

1. For all places where objects are allocated, we will call a NewOrig* function instead of allocating manually the object. The NewOrig* will initially not support memory pools.
2. For all simple places where objects are deallocated (except pipeline processing done), we will call a DeleteOrig* function instead of doing nothing and except GC to take care of this. The DeleteOrig* will initially do nothing.
3. Implement a "ResetOrig" operation so previously allocated objects can be re-used.
4. Implement the pipeline refcount logic proposed in the How to determine when an object is "fully" processed?.
5. Implement the reuse memory feature and protect it by a build directive (alternative can consider a featuregate, but that adds unnecessary overhead to each allocation/deallocation).6. Design a test suite in mdatagen that can ensure every component follows the requirements for the reuse memory feature.
6. Enable by default the reuse memory feature.

Tip

_{React with 👍 to help prioritize this issue. Please use comments to provide useful context, avoiding +1 or me too, to help us triage it. Learn more here.}

[github-actions]

Pinging code owners:

• pdata: @bogdandrutu @dmitryax

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[jmacd]

I wonder, would it be possible to make this behavior a featureflag choice so to be careful in the early stages, while those 10 components or so that do asynchronous work are upgraded?

[bogdandrutu]

I wonder, would it be possible to make this behavior a featureflag choice so to be careful in the early stages, while those 10 components or so that do asynchronous work are upgraded?

I think, if I am not wrong, we can enable by default to always allocate via the sync.Pool and only protect the refcounting by the feature flag, this way we don't encounter an extra atomic var check for the feature gate when allocating.