Add command and REST endpoint to list stream tracking info

[acogoluegnes]

No description provided.

[acogoluegnes]

Acceptance Steps

Get the branch and start the broker:

cd /tmp
git clone [email protected]:rabbitmq/rabbitmq-server.git
cd rabbitmq-server
git checkout stream-list-tracking-info
make run-broker PLUGINS="rabbitmq_stream_management"

Open another tab in /tmp/rabbitmq-server and get the stream performance tool:

wget https://github.com/rabbitmq/rabbitmq-java-tools-binaries-dev/releases/download/v-stream-perf-test-latest/stream-perf-test-latest.jar

Once the broker is started, get the tracking info for a stream that does not exist:

sbin/rabbitmq-streams list_stream_tracking stream

Expected output:

Listing tracking information for stream stream ...
Error:
The stream does not exist.

Run the performance tool to create the stream:

java -jar stream-perf-test-latest.jar --rate 1000 --time 5

Get the tracking info:

sbin/rabbitmq-streams list_stream_tracking stream

Expected output (no tracking info yet):

Listing tracking information for stream stream ...

Run a consumer that does some offset tracking:

java -jar stream-perf-test-latest.jar --rate 1000 --time 5 --consumer-names consumer-1 --store-every 100

Get the tracking info:

sbin/rabbitmq-streams list_stream_tracking stream

Expected output (tracking_value does not matter):

Listing tracking information for stream stream ...
┌────────┬────────────┬────────────────┐
│ type   │ name       │ tracking_value │
├────────┼────────────┼────────────────┤
│ offset │ consumer-1 │ 14940          │
└────────┴────────────┴────────────────┘

Run a consumer with another name:

java -jar stream-perf-test-latest.jar --rate 1000 --time 5 --consumer-names consumer-2 --store-every 100

Get the tracking info:

sbin/rabbitmq-streams list_stream_tracking stream

Expected output, there should be 2 lines:

Listing tracking information for stream stream ...
┌────────┬────────────┬────────────────┐
│ type   │ name       │ tracking_value │
├────────┼────────────┼────────────────┤
│ offset │ consumer-2 │ 20085          │
├────────┼────────────┼────────────────┤
│ offset │ consumer-1 │ 14940          │
└────────┴────────────┴────────────────┘

Run a producer with deduplication:

java -jar stream-perf-test-latest.jar --rate 1000 --time 5 --consumer-names consumer-2 --store-every 100 --producer-names publisher-1

Get the tracking info:

sbin/rabbitmq-streams list_stream_tracking stream

Expected output, there should be 3 lines, one is for the producer (writer):

Listing tracking information for stream stream ...
┌────────┬─────────────┬────────────────┐
│ type   │ name        │ tracking_value │
├────────┼─────────────┼────────────────┤
│ writer │ publisher-1 │ 4994           │
├────────┼─────────────┼────────────────┤
│ offset │ consumer-2  │ 25033          │
├────────┼─────────────┼────────────────┤
│ offset │ consumer-1  │ 14940          │
└────────┴─────────────┴────────────────┘

Run a producer with another name:

java -jar stream-perf-test-latest.jar --rate 1000 --time 5 --consumer-names consumer-2 --store-every 100 --producer-names publisher-2

Get the tracking info:

sbin/rabbitmq-streams list_stream_tracking stream

Expected output, there should be 4 lines, one more for the producer:

Listing tracking information for stream stream ...
┌────────┬─────────────┬────────────────┐
│ type   │ name        │ tracking_value │
├────────┼─────────────┼────────────────┤
│ writer │ publisher-2 │ 5008           │
├────────┼─────────────┼────────────────┤
│ writer │ publisher-1 │ 4994           │
├────────┼─────────────┼────────────────┤
│ offset │ consumer-2  │ 30183          │
├────────┼─────────────┼────────────────┤
│ offset │ consumer-1  │ 14940          │
└────────┴─────────────┴────────────────┘

Get only the writer lines:

sbin/rabbitmq-streams list_stream_tracking stream --writer

Expected output:

Listing tracking information for stream stream ...
┌────────┬─────────────┬────────────────┐
│ type   │ name        │ tracking_value │
├────────┼─────────────┼────────────────┤
│ writer │ publisher-2 │ 5008           │
├────────┼─────────────┼────────────────┤
│ writer │ publisher-1 │ 4994           │
└────────┴─────────────┴────────────────┘

Get only the consumer lines:

sbin/rabbitmq-streams list_stream_tracking stream --offset

Expected output:

Listing tracking information for stream stream ...
┌────────┬────────────┬────────────────┐
│ type   │ name       │ tracking_value │
├────────┼────────────┼────────────────┤
│ offset │ consumer-2 │ 30183          │
├────────┼────────────┼────────────────┤
│ offset │ consumer-1 │ 14940          │
└────────┴────────────┴────────────────┘

Get both (that's the default):

sbin/rabbitmq-streams list_stream_tracking stream --all

Expected output:

Listing tracking information for stream stream ...
┌────────┬─────────────┬────────────────┐
│ type   │ name        │ tracking_value │
├────────┼─────────────┼────────────────┤
│ writer │ publisher-2 │ 5008           │
├────────┼─────────────┼────────────────┤
│ writer │ publisher-1 │ 4994           │
├────────┼─────────────┼────────────────┤
│ offset │ consumer-2  │ 30183          │
├────────┼─────────────┼────────────────┤
│ offset │ consumer-1  │ 14940          │
└────────┴─────────────┴────────────────┘

Wrong type:

sbin/rabbitmq-streams list_stream_tracking stream --foo

Expected output:

Error (argument validation): Invalid options for this command:
--foo :
Arguments given:
        list_stream_tracking stream --foo

Usage

rabbitmq-streams [--node <node>] [--longnames] [--quiet] list_stream_tracking <stream> [--all | --offset | --writer] [--vhost <vhost>]

REST endpoint:

curl -s -u guest:guest http://localhost:15672/api/stream/%2F/stream/tracking | jq .

Expected output (the actual offset/sequence values do not matter, only the names do):

{
  "writers": {
    "publisher-1": 4997,
    "publisher-2": 5017
  },
  "offsets": {
    "consumer-1": 7613,
    "consumer-2": 22775
  }
}

Only the producers:

curl -s -u guest:guest http://localhost:15672/api/stream/%2F/stream/tracking?type=writer | jq .

Expected output:

{
  "writers": {
    "publisher-1": 4997,
    "publisher-2": 5017
  }
}

Only the consumers:

curl -s -u guest:guest http://localhost:15672/api/stream/%2F/stream/tracking?type=offset | jq .

Expected output:

{
  "offsets": {
    "consumer-1": 7613,
    "consumer-2": 22775
  }
}

[kjnilsson]

LGTM!