ThreadPoolTaskExecutor is not fully utilised

[otbe]

Hi there,

its me again and I have another question. Let's say we have two queues, two message listener containers and a shared ThreadPoolTaskExecutor. Something similar to this

@Bean("myTpe")
public ThreadPoolTaskExecutor executor() {
  final var taskExecutor = new ThreadPoolTaskExecutor();
  taskExecutor.setQueueCapacity(0);
  taskExecutor.setMaxPoolSize(120);
  taskExecutor.setCorePoolSize(120);
  // ...
  return taskExecutor;
}

@Bean(name = "containerOne")
public MessageListenerContainer<Object> containerOne(SqsAsyncClient sqsAsyncClient,  MessageListener<Object> listener, ThreadPoolTaskExecutor taskExecutor) {
  return SqsMessageListenerContainer.builder()
      .id("containerOne")
      .sqsAsyncClient(sqsAsyncClient)
      .messageListener(listener)
      .configure(
          c ->
              c.maxConcurrentMessages(60)
                  .componentsTaskExecutor(taskExecutor))
      .queueNames("queue1")
      .build();
}

@Bean(name = "containerTwo")
public MessageListenerContainer<Object> containerTwo(SqsAsyncClient sqsAsyncClient,  MessageListener<Object> listener, ThreadPoolTaskExecutor taskExecutor) {
  return SqsMessageListenerContainer.builder()
      .id("containerTwo")
      .sqsAsyncClient(sqsAsyncClient)
      .messageListener(listener)
      .configure(
          c ->
              c.maxConcurrentMessages(60)
                  .componentsTaskExecutor(taskExecutor))
      .queueNames("queue2")
      .build();
}

I would've expected to see at max 120 threads being used to process 60 messages from each queue. But this is not what is happening. Most of the time the overall active tasks in the ThreadPoolTaskExecutor is 60 (sometimes 61, 62 but not higher). Both queues receive a large amount of messages (~2k) basically at the same the time and my assumption is now that we process 60 messages (in 60 threads) from both queues.
I already played around with a lot of settings like the back pressure mode (setting it to always high throughput) but nothing really helps.

If I make it 80/40 in terms of max concurrent messages, 80 threads are occupied. If I make it 40/40, 40 threads are occupied. If I add more instances of my application then all of them have the same amount of threads being used (so there are really messages left in the queue to be processed). I feel like I hit some hidden limitation like there's only one max concurrent messages number applied for all containers.

Where is my conceptional misunderstanding of how things work?

[tomazfernandes]

Hey @otbe, welcome back.

I feel like I hit some hidden limitation

It's hard to say without a sample project reproducing the scenario, but most likely you have a bottleneck somewhere in your flow, the app is processing the messages faster than more messages are being delivered, and thus never gets over 60 / 80.

Some usual suspects:

• SqsAsyncClient

SqsAsyncClient
   .builder()
   .httpClientBuilder(NettyNioAsyncHttpClient.builder()
       .maxConcurrency(1000))
       .build()

• pollTimeout
• Localstack throughput might be a bottleneck if you're running against it
• Network latency might be a bottleneck if you're running locally against AWS
• If you're sending messages while consuming

Let me know if that helps.

[RyanVerWey]

TL;DR: maxConcurrentMessages = per-container, per-queue in-flight permits (not threads). Your overall parallelism is likely capped by the shared SqsAsyncClient's Netty maxConcurrency unless you increase it or give each container its own client.
Spring Documentation

What maxConcurrentMessages really does

In Spring Cloud AWS SQS v3, maxConcurrentMessages is the cap on in-flight messages per container per queue. It is not a thread count and doesn't guarantee you'll see the same number of active threads in a shared ThreadPoolTaskExecutor. The reference explains it as "the maximum number of messages being processed by the container at a given time," enforced per queue, and notes why observed in-flight counts can exceed the value when acknowledgement batching is used.
Spring Documentation

The executor you pass via .componentsTaskExecutor(...) is used by the container's components (e.g., the MessageSink and any blocking bits). The API explicitly says this executor is shared by those components and "should have enough threads to support the full load, including if it's shared between containers."
Spring Documentation

The real bottleneck you're observing

When both containers share one SqsAsyncClient, they also share the underlying Netty-based async HTTP client. That client has a global concurrency limit (maxConcurrency) that bounds the total concurrent requests across all users of that client instance. If you don't raise it (or isolate clients), it becomes your aggregate cap—which is why patterns like 40/40 → ~40, 80/40 → ~80, 60/60 → ~60 show up: the larger container consumes most of the available HTTP concurrency, and the other can't push the total any higher. The AWS SDK docs show how to configure and share a NettyNioAsyncHttpClient and how to set maxConcurrency.
AWS Documentation

Handy doc entry points

• Spring Cloud AWS SQS reference for maxConcurrentMessages, polling, batching, and throughput behavior.
  Spring Documentation

• ContainerOptionsBuilder.componentsTaskExecutor(...) API notes on the shared executor and sizing guidance.
  Spring Documentation

• AWS SDK for Java v2: configuring the Netty async HTTP client and maxConcurrency (including sharing across clients).
  AWS Documentation

[otbe]

Yeah that makes sense. We do use the same SQS client for all of them.