Skip to content

Conversation

shashankmehra
Copy link
Contributor

Summary

This PR introduces a major architectural refactoring to improve modularity, simplify core components, and clarify responsibilities. The core idea is to make brokers more self-contained by moving job consumption logic (polling/push) and serialization into the broker implementations.

Key Changes:

  • Simplified Broker Interface:

    • The Broker interface is streamlined. Methods like Dequeue, CreateQueue, and Capabilities have been removed from the core.Broker interface.
    • A new Start(ctx context.Context, jobChan chan<- job.Job) method is introduced. Each broker is now responsible for fetching jobs and sending them to the provided channel.
    • A Queues() method was added to provide queue information.
  • Removed Poller from Core:

    • The Poller interface and StandardPoller have been removed from the core package.
    • Pull-based brokers (like Redis) now manage their own polling loop internally. A new pollers package contains a helper StandardPoller for this.
    • Push-based brokers (like RabbitMQ) implement their consumer logic directly in the Start method.
  • Decoupled Serializer from Engine:

    • The Serializer is no longer a core component passed to the Engine.
    • Instead, each Broker is now responsible for its own serialization and receives a Serializer on initialization. This better encapsulates the data format with the transport layer.
  • Configuration Moved to Brokers:

    • Configuration for Queues and PollInterval has been removed from core.Engine options.
    • This configuration is now part of the Options for each respective broker (redis.Options, rabbitmq.Options), making them more self-contained and easier to configure.
  • Simplified Core and Engine:

    • The core.Engine is simplified. It no longer needs to create or manage a Poller. It just calls broker.Start().
    • Worker and WorkerPool are also simplified, as they no longer need queues or a serializer during initialization.

…fig to brokers

Remove separate Poller interface by adding Start() method to Broker interface.
Move queue configuration from Engine options to broker-specific options for better separation of concerns.
…nstructor

- Move Serializer interface from core to individual broker packages
- Remove serializer parameter from NewEngine and NewWorkerPool constructors
- Move QueueOptions type to rabbitmq broker package
- Remove Redis CreateQueue method (no-op)
@shashankmehra shashankmehra requested a review from shrooot June 30, 2025 17:12
@shashankmehra shashankmehra changed the base branch from fix-logging-interface to master June 30, 2025 17:13
@shashankmehra shashankmehra merged commit 84afd03 into master Jul 14, 2025
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants