Error unmarshalling json file with jsonlogencoding extension - slice unexpected end of JSON input

[dentonk]

Component(s)

extension/encoding/jsonlogencoding

What happened?

Description

When using the jsonlogencoding extension in combination with the awss3 receiver I am getting a json: slice unexpected end of JSON input error. The json file that I am uploading to s3 is a single sample otlp log message that's 977.0 B in size. This file is is handled correctly by both the built-in otlp_json encoding option, as well as the otlpencoding, but when I switch to the jsonlogencoding extension and upload the exact same log file, I get the above error. I'm using an otlp json formatted file as a test simply to demonstrate the JSON formatting is correct and the file is able to be unmarshalled by other methods. The ultimate goal is to use the jsonlogencoding extension with CloudTrail logs.

Most of my testing has been with the awss3 receiver, but I also quickly tested the googlepubsub receiver because it also supports using encoding extensions and I saw similar results. When I used the built-in raw_text encoding option the otlp json formatted message was output to the filelog exporter, but when I switched to the jsonlogencoding extension I did not get an error, but also nothing was output to the file.

Steps to Reproduce

My Testing was specifically with the new SQS functionality in the S3 receiver, but I assume you'll see the same results however you configure your S3 source.

• configure your awss3 receiver following standard steps and using the built-in otlp_json encoding
• upload an example otlp json-formatted json file into the bucket
• monitor both the collector logs and the file output of the filelog exporter.
• update the collector config to use the jsonlogencoding extension and repeat the test
• update the collector config to use the otlpencoding extension and repeat the test

Expected Result

With all of the encoding options I would expect to get similar results, i.e. no error and a log message output by the filelog exporter.
The actual log output would be different, since the jsonlogencoding would take the entire otlp json-formatted log and place it in the body of the new log, but the content is not important, just that a log is emitted.

Actual Result

When using the built-in otlp_json encoding and the otlpencoding extension everything works as expected. There are no errors in the collector logs and a log is emitted by the filelog exporter.

With the jsonlogencoding extension, we get an error in the collector log and no log is emitted by the filelog exporter.

Collector version

v0.127.0

Environment information

Environment

OS: Debian GNU/Linux 11 (bullseye)

OpenTelemetry Collector configuration

receivers:
    awss3:
        encodings:
            - extension: json_log_encoding
              suffix: .json
        sqs:
            queue_url: https://sqs.us-east-1.amazonaws.com/xxxxxxxxx/test-sqs
            region: us-east-1
        s3downloader:
            region: us-east-1
            s3_bucket: aws-cloudtrail-logs-xxxxxxxx-xxx
            s3_prefix: AWSLogs

extensions:
    json_log_encoding:
        mode: body

exporters:
    file:
      path: /tmp/s3-json.log

service:
    extensions:
        - json_log_encoding
    pipelines:
        logs:
            receivers:
                - awss3
            processors: []
            exporters:
                - file

    telemetry:
      logs:
          level: debug

Log output

with jsonlogencoding extension:

Jun 08 10:16:20 debian otelcol-contrib[171518]: 2025-06-08T10:16:20.074-0700        debug        awss3receiver@v0.127.0/s3sqsreader.go:135        Received messages from SQS        {"resource": {}, "otelcol.component.id": "awss3", "otelcol.component.kind": "receiver", "otelcol.signal": "logs", "messageCount": 0}
Jun 08 10:16:26 debian otelcol-contrib[171518]: 2025-06-08T10:16:26.008-0700        info        awss3receiver@v0.127.0/s3sqsreader.go:184        Processing new S3 object        {"resource": {}, "otelcol.component.id": "awss3", "otelcol.component.kind": "receiver", "otelcol.signal": "logs", "bucket": "aws-cloudtrail-logs-xxxxxxxx-xxxx", "key": "AWSLogs/xxxxxxxx/CloudTrail/us-east-1/2025/06/08/otlp_json_2.json"}
Jun 08 10:16:26 debian otelcol-contrib[171518]: 2025-06-08T10:16:26.161-0700        debug        awss3receiver@v0.127.0/receiver.go:255        Processing log file        {"resource": {}, "otelcol.component.id": "awss3", "otelcol.component.kind": "receiver", "otelcol.signal": "logs", "key": "AWSLogs/xxxxxxxx/CloudTrail/us-east-1/2025/06/08/otlp_json_2.json", "format": ".json"}
Jun 08 10:16:26 debian otelcol-contrib[171518]: 2025-06-08T10:16:26.161-0700        error        awss3receiver@v0.127.0/s3sqsreader.go:200        Failed to process S3 object content        {"resource": {}, "otelcol.component.id": "awss3", "otelcol.component.kind": "receiver", "otelcol.signal": "logs", "key": "AWSLogs/xxxxxxxx/CloudTrail/us-east-1/2025/06/08/otlp_json_2.json", "error": "json: slice unexpected end of JSON input"}
Jun 08 10:16:26 debian otelcol-contrib[171518]: github.com/open-telemetry/opentelemetry-collector-contrib/receiver/awss3receiver.(*s3SQSNotificationReader).readAll
Jun 08 10:16:26 debian otelcol-contrib[171518]:         github.com/open-telemetry/opentelemetry-collector-contrib/receiver/awss3receiver@v0.127.0/s3sqsreader.go:200
Jun 08 10:16:26 debian otelcol-contrib[171518]: github.com/open-telemetry/opentelemetry-collector-contrib/receiver/awss3receiver.(*awss3Receiver).Start.func1
Jun 08 10:16:26 debian otelcol-contrib[171518]:         github.com/open-telemetry/opentelemetry-collector-contrib/receiver/awss3receiver@v0.127.0/receiver.go:105

with otlpencoding extension:
Jun 08 10:47:00 debian otelcol-contrib[173241]: 2025-06-08T10:47:00.779-0700        debug        awss3receiver@v0.127.0/s3sqsreader.go:135        Received messages from SQS        {"resource": {}, "otelcol.component.id": "awss3", "otelcol.component.kind": "receiver", "otelcol.signal": "logs", "messageCount": 0}
Jun 08 10:47:06 debian otelcol-contrib[173241]: 2025-06-08T10:47:06.392-0700        info        awss3receiver@v0.127.0/s3sqsreader.go:184        Processing new S3 object        {"resource": {}, "otelcol.component.id": "awss3", "otelcol.component.kind": "receiver", "otelcol.signal": "logs", "bucket": "aws-cloudtrail-logs-xxxxxxxx-xxxx", "key": "AWSLogs/xxxxxxxx/CloudTrail/us-east-1/2025/06/08/otlp_json_3.json"}
Jun 08 10:47:06 debian otelcol-contrib[173241]: 2025-06-08T10:47:06.562-0700        debug        awss3receiver@v0.127.0/receiver.go:255        Processing log file        {"resource": {}, "otelcol.component.id": "awss3", "otelcol.component.kind": "receiver", "otelcol.signal": "logs", "key": "AWSLogs/xxxxxxxx/CloudTrail/us-east-1/2025/06/08/otlp_json_3.json", "format": ".json"}

Additional context

otlp_json_2.json

[github-actions]

Pinging code owners:

• extension/encoding/jsonlogencoding: @VihasMakwana @atoulme

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[axw]

/label -needs-triage waiting-for-code-owners

Thanks for the detailed report!

[VihasMakwana]

@dentonk This is caused due to a breaking change we introduced in #36156.

To overcome this, you can try following file. I just converted a single OTLP object to a array with one OTLP object.

otlp_json_2 (1).json

[dentonk]

@VihasMakwana Thanks for the quick update. The challenge is, as mentioned above, the otlp_json example was simply to test the ingestion since I had another encoding option to compare with. My actual use case is to ingest CloudTrail logs and have no way to update their format. They are formatted like {"Records":[{"eventVersion":"1.11", .....

If I may offer a suggestion, if the jsonlogencoding extension is always expecting an array, it would be helpful to update the README to make that more clear.

Unless there are plans for changing the behavior of the jsonlogencoding extension to accommodate non-array inputs, I imagine I'll either need to wait for the textencoding extension to move into alpha then unmarshal in the processors, or wait for this to be merged: #40246

[VihasMakwana]

@dentonk

If I may offer a suggestion, if the jsonlogencoding extension is always expecting an array, it would be helpful to update the README to make that more clear.

Sure. I'll update that.

You also have a valid use case of allowing the extension to only ingest one log. We can restore the original behaviour and keep current behaviour as it is (we can add a config option to determine the behaviour).

I'll work on rolling out a PR.