AWS xray exporter doesn't respect the sts_region attribute

[jpujari]

Component(s)

exporter/awsxray

What happened?

Description

We have otel collector running in multiple regions. We set the sts_region attribute on the sigv4auth extension to be the region the collector is running on. We also use the xray exporter for traces. xray exporter doesn't seem to respect this attribute and is making cross region call to the sts endpoint for credentials

Steps to Reproduce

Setup otel collector in us-east-2 region, set the sts_region attribute to us-east-2
Setup xray exporter config
Monitor the logs

Expected Result

The STS calls from xray exporter should go to the us-east-2 sts endpoint

Actual Result

The STS calls from xray exporter are sent to us-east-1 sts endpoint

Collector version

v0.43.0

Environment information

Environment

OS: AL2
Compiler(if manually compiled): (e.g., "go 14.2")

OpenTelemetry Collector configuration

extensions:
  health_check:
    endpoint: *********:13133
    path: /health

  sigv4auth:
    assume_role:
      arn: ${OBSERVABILITY_ASSUME_ROLE_ARN}
      sts_region: ${REGION}
    region: ${OBSERVABILITY_REGION}

receivers:
  awsecscontainermetrics:
    collection_interval: 15s

  otlp:
    protocols:
      http:
        endpoint: *******:4318

  prometheus:
    config:
      scrape_configs:
        - job_name: 'otel-collector'
          scrape_interval: 15s
          static_configs:
            - targets:
              - *********:8888
              - ${OBSERVABILITY_PROMETHEUS_RECEIVER_TARGET}
        - job_name: 'otel-collector-fluentbit'
          metrics_path: /api/v1/metrics/prometheus
          scrape_interval: 15s
          static_configs:
            - targets:
              - ${OBSERVABILITY_FLUENTBIT_RECEIVER_TARGET}

processors:
  memory_limiter:
    check_interval: 1s
    limit_mib: 150

  resourcedetection:
    detectors:
      - ecs
      - ec2
    timeout: 20s
    override: false
    ec2:
      tags:
        - ^aws:autoscaling:groupName$

  filter:
    metrics:
      include:
        match_type: regexp
        metric_names:
          - ^container\.(cpu|memory|network|storage)\..*
          - ^fluentbit_build_info$
          - ^fluentbit_input_bytes_total$
          - ^fluentbit_input_records_total$
          - ^fluentbit_output_dropped_records_total$
          - ^fluentbit_output_errors_total$
          - ^fluentbit_output_retried_records_total$
          - ^fluentbit_output_retries_total$
          - ^fluentbit_output_retries_failed_total$
          - ^haproxy_.*
          - ^otelcol_exporter_enqueue_failed_metric_points$
          - ^otelcol_exporter_enqueue_failed_spans$
          - ^otelcol_exporter_queue_capacity$
          - ^otelcol_exporter_queue_size$
          - ^otelcol_exporter_send_failed_metric_points$
          - ^otelcol_exporter_send_failed_spans$
          - ^otelcol_exporter_sent_metric_points$
          - ^otelcol_exporter_sent_spans$
          - ^otelcol_processor_batch_batch_size_trigger_send$
          - ^otelcol_processor_batch_timeout_trigger_send$
          - ^otelcol_processor_dropped_metric_points$
          - ^otelcol_processor_dropped_spans$
          - ^otelcol_processor_refused_metric_points$
          - ^otelcol_processor_refused_spans$
          - ^otelcol_receiver_accepted_metric_points$
          - ^otelcol_receiver_accepted_spans$
          - ^otelcol_receiver_refused_metric_points$
          - ^otelcol_receiver_refused_spans$
          - ^process_start_time_seconds$
          - ^statsd_exporter_build_info$
          - ^statsd_exporter_events_total$
          - ^statsd_exporter_lines_total$
          - ^statsd_exporter_metrics_total$
          - ^statsd_exporter_samples_total$
          - ^statsd_exporter_tag_errors_total$
          - ^statsd_exporter_tags_total$
          - ^statsd_exporter_udp_packet_drops_total$

  resource/global:
    attributes:
      - key: availability_zone
        action: upsert
        from_attribute: cloud.availability_zone

      - key: account.id
        value: ${ACCOUNT_ID}
        action: upsert
      - key: environment
        value: ${ENVIRONMENT}
        action: upsert
      - key: region
        value: ${REGION}
        action: upsert
      - key: tier
        value: ${TIER}
        action: upsert

  resource/metrics:
    attributes:
      - key: cloud.availability_zone
        action: delete
      - key: cloud.account.id
        action: delete
      - key: cloud.region
        action: delete

      - key: aws.ecs.service.name
        action: delete
      - key: aws.ecs.launchtype
        action: delete
      - key: aws.ecs.docker.name
        action: delete
      - key: aws.ecs.task.id
        action: delete
      - key: aws.ecs.task.known_status
        action: delete
      - key: aws.ecs.task.launch_type
        action: delete
      - key: aws.ecs.task.pull_started_at
        action: delete
      - key: aws.ecs.task.pull_stopped_at
        action: delete
      - key: aws.ecs.container.created_at
        action: delete
      - key: aws.ecs.container.started_at
        action: delete
      - key: aws.ecs.container.know_status
        action: delete
      - key: aws.ecs.task.version
        action: delete
      - key: aws.log.group.names
        action: delete
      - key: aws.log.group.arns
        action: delete
      - key: aws.log.stream.names
        action: delete
      - key: aws.log.stream.arns
        action: delete
      - key: container.id
        action: delete
      - key: container.duration
        action: delete

      - key: cloud.provider
        action: delete
      - key: cloud.platform
        action: delete
      - key: aws.autoscaling.group.name
        action: upsert
        from_attribute: ec2.tag.aws:autoscaling:groupName
      - key: ec2.tag.aws:autoscaling:groupName
        action: delete
      - key: host.name
        action: delete

      - key: http.scheme
        action: delete
      - key: net.host.name
        action: delete
      - key: net.host.port
        action: delete
      - key: server.address
        action: delete
      - key: server.port
        action: delete
      - key: url.scheme
        action: delete

      - key: service.instance.id
        action: delete
      - key: service_instance_id
        action: delete
      - key: service.name
        action: delete
      - key: service_name
        action: delete
      - key: service.version
        action: delete
      - key: service_version
        action: delete

  attributes/metrics:
    actions:
      - key: service.instance.id
        action: delete
      - key: service_instance_id
        action: delete
      - key: service.name
        action: delete
      - key: service_name
        action: delete
      - key: service.version
        action: delete
      - key: service_version
        action: delete

  attributes/traces:
    actions:
      - key: net.peer.ip
        action: upsert
        from_attribute: net.peer.name

  batch:
    timeout: 1m

exporters:
  awsxray:
    index_all_attributes: true
    role_arn: ${OBSERVABILITY_ASSUME_ROLE_ARN}
    region: ${OBSERVABILITY_REGION}

  prometheusremotewrite:
    endpoint: ${OBSERVABILITY_PROMETHEUS_ENDPOINT}/api/v1/remote_write
    auth:
      authenticator: sigv4auth
    add_metric_suffixes: false
    resource_to_telemetry_conversion:
      enabled: true
    target_info:
      enabled: false
    send_metadata: true
    max_batch_size_bytes: 1000000

service:
  extensions:
    - health_check
    - sigv4auth
  pipelines:
    metrics:
      receivers:
        - awsecscontainermetrics
        - prometheus
      processors:
        - memory_limiter
        - resourcedetection
        - filter
        - attributes/metrics
        - resource/global
        - resource/metrics
        - batch
      exporters:
        - prometheusremotewrite

    traces:
      receivers:
        - otlp
      processors:
        - attributes/traces
        - resource/global
        - batch
      exporters:
        - awsxray

Log output

info	awsutil@v0.117.0/conn.go:256	STS Endpoint 	{"kind": "exporter", "data_type": "traces", "name": "awsxray", "endpoint": "https://sts.us-east-1.amazonaws.com"}

Additional context

No response

[github-actions]

Pinging code owners:

• exporter/awsxray: @wangzlei @srprash

See Adding Labels via Comments if you do not have permissions to add labels yourself.