feat(kafka-client): Allow for kafka client configuration to be passed through via CLI

[eddyv]

• Integrate properties-file for Kafka configuration management
• Added support for loading Kafka client configurations from a properties file via the CLI.
• Enhanced CLI options to include a new --kafka-config-file argument for specifying the properties file path.
• Refactored error handling in file reading functions to throw errors instead of exiting the process.
• Updated the DefaultClientManager to merge configurations from the properties file with environment variables, improving flexibility in Kafka client setup.
• Introduced a new utility function to parse properties files, enhancing configuration management.

This pull request introduces enhancements to the CLI and Kafka client configuration, improving flexibility and error handling. It adds support for loading Kafka configurations from a properties file, refactors Kafka configuration management, and replaces abrupt process exits with exceptions for better error propagation.

CLI Enhancements:

• Added a new --kafka-config-file option to the CLI, allowing users to specify a properties file for Kafka client configuration.
• Implemented the parsePropertiesFile function to read and parse properties files.

Error Handling Improvements:

• Replaced process.exit calls with thrown exceptions in functions like readFileLines and loadEnvironmentVariables, ensuring errors propagate properly.
• Updated error logging to include stringified error objects for better readability.

Kafka Configuration Refactor:

• Transitioned from KafkaJS.CommonConstructorConfig to GlobalConfig for Kafka client configuration, aligning with a more flexible and standardized approach.
• Refactored the Kafka consumer creation logic to dynamically merge user-provided configurations with defaults, enabling more granular control.
• Simplified Kafka producer initialization by removing hardcoded configurations like compression.type and linger.ms.

Environment and Kafka Config Merging:

• Merged Kafka configurations from CLI options and environment variables, allowing users to override defaults via the properties file.

These changes enhance the flexibility and maintainability of the application while improving the user experience for configuring Kafka clients.

[Copilot]

Pull Request Overview

This PR adds support for loading Kafka client configurations via a CLI-provided properties file, updates error handling to use thrown exceptions, and refactors the Kafka client manager to use a unified GlobalConfig.

• Switched from CommonConstructorConfig to GlobalConfig in index.ts and client manager, merging environment and CLI settings.
• Introduced --kafka-config-file CLI option and parsePropertiesFile utility for properties-file parsing.
• Refactored file-reading utilities and CLI parsing helpers to throw errors instead of calling process.exit.

Reviewed Changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

File	Description
src/index.ts	Use GlobalConfig and merge CLI-provided Kafka properties
src/confluent/client-manager.ts	Refactor DefaultClientManager to accept GlobalConfig and build consumer config dynamically
src/cli.ts	Add --kafka-config-file option, implement parsePropertiesFile, and replace exits with exceptions
package.json	Add properties-file dependency

Comments suppressed due to low confidence (2)

src/cli.ts:294

• [nitpick] New utility 'parsePropertiesFile' lacks unit tests; add tests to cover scenarios like missing file, parse errors, and successful parsing.

export function parsePropertiesFile(filePath: string): KeyValuePairObject {

src/confluent/client-manager.ts:229

• Passing a GlobalConfig object as the consumer configuration may be incompatible with KafkaJS.Consumer's expected configuration; consider using the ConsumerConfig type or mapping keys to the correct consumer config properties (e.g., groupId, fromBeginning).

return this.kafkaClient.get().consumer(consumerConfig);

[eddyv]

closes #21

[Copilot]

Pull Request Overview

This PR enhances the Kafka client configuration by integrating a new properties file-based configuration via the CLI, refactoring error handling, and migrating from KafkaJS.CommonConstructorConfig to GlobalConfig.

• Introduces a new CLI option (--kafka-config-file) that enables users to supply Kafka client configuration via a properties file.
• Replaces process exits with exceptions in file reading functions to improve error propagation.
• Refactors client and consumer creation by merging environment variable–based configurations with CLI-provided overrides.

Reviewed Changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
src/index.ts	Updates import and configuration merging for Kafka client setup.
src/confluent/client-manager.ts	Adapts consumer and producer initialization to use GlobalConfig and removes hardcoded producer options.
src/cli.ts	Adds support for Kafka config file CLI option, updates error handling logic.
package.json	Adds dependency for the "properties-file" package.
README.md	Updates documentation with the new CLI option description.

[Copilot]

Consider clarifying the precedence of configuration merging between environment variables and CLI-provided Kafka configuration; ensure that CLI settings are intended to override environment variables as designed.