Skip to content

Support incremental validation of fragmented messages sent by client #671

@jfallows

Description

@jfallows

Describe the desired outcome from the user's perspective
Large messages are often fragmented into smaller partial messages, but should still be validated according to the same rules as if they are observed as a single unified value.

Acceptance criteria

  • Fragmented HTTP request messages can be validated or rejected; string, integer, json
  • Fragmented MQTT publish messages can be validated or rejected; string, integer, json
  • Fragmented Kafka produce messages can be validated or rejected; string, integer, json
  • Fragmented messages should not require buffering into a complete message before validation

Additional context
Conversion from json to avro (for example) is much more challenging to implement incrementally without buffering the complete message, so we should separate the concepts of validator and converter, supporting converters only at the kafka cache where we already have a view of the complete message by design.

Tasks

  • Separate the concepts of Validator and Converter in Java APIs
  • avro converter should only be configurable in kafka cache by schema (not other protocols)
  • protobuf converter should only be configurable in kafka cache by schema (not other protocols)
  • all validators should only be configurable on HTTP and MQTT by schema
  • both validators and converters should be configurable on Kafka by schema
  • implement string fragmented validator
  • implement integer fragmented validator
  • implement json fragmented validator

Metadata

Metadata

Assignees

Labels

storyFeature description from user's perspective

Type

No type

Projects

Status

Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions