Add support for S3 access logs

[constanca-m]

Component(s)

extension/encoding/awslogsencoding

Is your feature request related to a problem? Please describe.

S3 access logs have this format. We can add support for them so we can get an understanding of the usage of our S3 buckets.

Describe the solution you'd like

awslogs_encoding:
   format: s3_access_log

Describe alternatives you've considered

N/A

Additional context

N/A

[github-actions]

Pinging code owners:

• extension/encoding/awslogsencoding: @axw @constanca-m

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[axw]

/label -needs-triage

[constanca-m]

Before opening any PR, I think it is best to be in sync about the name of the attributes.

The S3 access logs have this fields.

I thought of this mapping:

Editing table based on follow up

AWS field	OpenTelemetry Field	Available in semantic conventions?
Bucket owner	cloud.account.id	Yes
Bucket	aws.s3.bucket	Yes
Time	Log timestamp
Remote IP	source.address	Yes
Requester	user.id	Yes
Request ID	aws.request_id	No
Operation	rpc.method	No
Key	aws.s3.key	Yes
Request URI	Split by space, in 3 fields: 1. http.request.method 2. url.path 3. url.scheme	Yes
HTTP status	http.response.status_code	Yes
Error code	error.type	Yes
Bytes sent	http.response.body.size	Yes
Object size	aws.s3.object.size	No
Total time	duration	No
Turn around time	aws.s3.turn_around	No
Referer	http.request.header.referer	No, format from semantic conventions.
User-Agent	user_agent.original	Yes
Version ID	aws.s3.version_id	No
Host ID	aws.extended_request_id	No
Signature version	aws.signature.version	No
Cipher suite	tls.cipher	Yes
Authentication Type	aws.s3.auth_type	No
Host header	http.request.header.host	No, format from semantic conventions.
TLS version	tls.protocol.version	Yes
Access point ARN	aws.s3.access_point.arn	No
aclRequired	aws.s3.acl_required	No

[axw]

Looks good overall, but I think there's a couple we should change to align with https://github.com/open-telemetry/semantic-conventions/blob/main/docs/object-stores/s3.md

e.g. I think Host ID should aws.extended_request_id (not very intuitive from the name!), and Operation should be rpc.method

Not too sure about aws.s3.owner, but I guess it's OK to start with. Did you check for any precedent in AWS or S3-related instrumentation? Would it make sense for this to be cloud.account.id?

[constanca-m]

Thanks @axw, I have updated the tabled based on your comment.

Changed:

• What you suggested:
  • Operation is rpc.method now
  • Host ID changed to aws.extended_request_id (indeed, not so obvious, but since the documentation mentions "x-amz-id-2" value, it seems to make sense)
  • And the aws.s3.owner to cloud.account.id: also a bit in doubt on this one, but this way everything stays in sync among the logs of this extension and we can keep the resource attribute for the account.
• And request ID: from aws.s3.request_id to aws.request_id.

[constanca-m]

Actually, maybe I will be using aws.s3.owner after all @axw Because there is also cloud.resource.id which would then fit aws.s3.bucket as well, but changing one and not the other seems odd. So then I will keep both as aws.s3.*, what do you think? I'm very in doubt on this one

[axw]

@constanca-m that sounds fine, maybe we'll come back to it.