rdkafka doesn't honor the same broker ID when moved to new host

[liljenback]

If migrating an existing broker to a new host and re-using its broker ID (from the host that is decommissioned) rdkafka still remember the old host name broker ID combination even when it is no longer available (brought offline). The client does not start using the new host. This results in messages being lost for partitions that are reassigned to the new host. The Kafka documentation seems to hint that the clients should be able to start using the new host with the same ID without having to be restarted.

[edenhill]

By messages lost you mean they are succesfully produced to the old broker, right?

[liljenback]

Right, the messages will be produced to the old broker but in this case it's offline.

[edenhill]

Should be fixed now.

[edenhill]

If possible please try verifying on your end