You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
feat: refine Produce stage descriptions to clarify handling of multiple topics and partitions, including validation, response structure, and offset management
Copy file name to clipboardExpand all lines: stage_descriptions/produce-06_mf2.md
+1-2Lines changed: 1 addition & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -4,8 +4,6 @@ In this stage, you'll add support for producing to multiple partitions of the sa
4
4
5
5
When a Kafka broker receives a Produce request targeting multiple partitions of the same topic, it needs to validate that the topic and all partitions exist, write records to each partition's log file independently, and return a response containing results for all partitions. Each partition maintains its own offset sequence independently, so partition 0 and partition 1 can both have records starting at offset 0.
6
6
7
-
The request will contain multiple RecordBatches, one for each partition. Each RecordBatch will contain a single record.
8
-
9
7
We've created an interactive protocol inspector for the request & response structures for `Produce`:
Copy file name to clipboardExpand all lines: stage_descriptions/produce-07_ar4.md
+12-11Lines changed: 12 additions & 11 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -2,7 +2,12 @@ In this stage, you'll add support for producing to multiple topics in a single r
2
2
3
3
## Cross-Topic Production
4
4
5
-
Your Kafka implementation should handle complex requests with multiple topics, manage independent offset tracking per topic and partition, and handle complex response structures. Each topic maintains completely independent offset sequences.
5
+
When a Kafka broker receives a Produce request targeting multiple topics with their respective partitions, it needs to validate that all topics and partitions exist, write records to each topic-partition's log file independently, and return a nested response structure containing results for all topics and their partitions. Each topic-partition combination maintains its own independent offset sequence, so topic "foo" partition 0 and topic "bar" partition 0 can both have records starting at offset 0.
6
+
7
+
We've created an interactive protocol inspector for the request & response structures for `Produce`:
@@ -13,6 +18,7 @@ The tester will execute your program like this:
13
18
```
14
19
15
20
It'll then connect to your server on port 9092 and send a `Produce` (v11) request targeting multiple topics with their respective partitions.
21
+
The request will contain data for multiple topics, and a single partition for each topic.
16
22
17
23
The tester will validate that:
18
24
@@ -25,19 +31,14 @@ The tester will validate that:
25
31
- The correct `name` field matching the topic name in the request.
26
32
- Each partition in the request has a corresponding partition response.
27
33
- Each partition response contains:
28
-
- The correct `partition` field matching the request.
34
+
- The correct `index` field matching the partition in the request.
29
35
- An error code of `0` (NO_ERROR).
30
-
- A valid `offset` field with the assigned offset for that topic-partition.
31
-
-A valid `timestamp` field.
32
-
-A correct `log_start_offset` field.
36
+
- A valid `base_offset` field with the assigned offset for that topic-partition.
37
+
-The `log_append_time_ms` field contains `-1` (signifying that the timestamp is latest).
38
+
-The `log_start_offset` field is `0`.
33
39
- Records are persisted to the correct topic-partition log files.
34
40
- Offset assignment is independent per topic-partition combination.
35
41
36
42
## Notes
37
43
38
-
- Each topic-partition combination maintains its own independent offset sequence.
39
-
- Multiple topics can be written to simultaneously in a single request.
40
-
- The response structure is nested: topics contain partition responses.
41
-
- Topic-level and partition-level errors should be handled independently.
42
-
- This is the most complex produce scenario, combining multi-topic and multi-partition handling.
43
-
- The official docs for the `Produce` request can be found [here](https://kafka.apache.org/protocol.html#The_Messages_Produce).
44
+
- The official docs for the `Produce` request can be found [here](https://kafka.apache.org/protocol.html#The_Messages_Produce). Make sure to scroll down to the "Produce Response (Version: 11)" section.
0 commit comments