[testbed][filelog] - large file tests for filelog receiver.

[VihasMakwana]

Component(s)

receiver/filelog, testbed

Describe the issue you're reporting

• Add automated test cases for filelog receiver to test very large files.
• Use the existing testbed for this.

---

Large files would typically range in a couple of GBs

[github-actions]

Pinging code owners:

• receiver/filelog: @djaglowski
• testbed: @open-telemetry/collector-approvers

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[VihasMakwana]

@djaglowski we could typically use some sort of correctness tests for this. What do you think?

---

we already have a perf test data generator for logs.
We can reuse that or create a new one which would validate the log data as well.

[djaglowski]

@VihasMakwana, I don't have a strong opinion on this other than that the test should generate the large file each time, rather than store it in the repo, and assuming that this can be done without too much trouble on systems as small as CI runners.

[VihasMakwana]

@djaglowski I see. That’s what I’m thinking.

We can make use of t.TempDir(..) style file creation to ensure that we remove our log files immediately after test succeeds/fails.

We recently had an issue reported for filelog in which the user consistently lost +1k logs out of ~600k. The file size is not even that big (it would range in MBs for this particular issue).

In my opinion, it makes sense to have some sort of correctness tests for large files to ensure that we don't drop any log out of the blue.

[VihasMakwana]

I would appreciate if other folks from @open-telemetry/collector-contrib-approvers can share some thoughts on this issue.

[VihasMakwana]

Can someone from @open-telemetry/collector-contrib-triagers assign this to me?

[djaglowski]

Closed by #34417