Skip to content

es_out: support Upstream Servers with configuration overriding #7608

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 7 commits into
base: master
Choose a base branch
from
Open

es_out: support Upstream Servers with configuration overriding #7608

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
02c9431
pipeline: outputs: es: basic support of Upstream, only parameters kno…
mabrarov Jun 17, 2023
756a9ef
engine: support of tests which require generation of context passed t…
mabrarov Jun 23, 2023
ccc3c58
pipeline: outputs: es: fixed tests after modification of formatter ca…
mabrarov Jun 23, 2023
7e16c01
pipeline: outputs: es: support overriding the most plugin parameters …
mabrarov Jun 17, 2023
a1afba4
pipeline: outputs: es: tests for upstream node configuration properties
mabrarov Jul 6, 2023
2791bd8
engine: support of tests which require generation of context passed t…
mabrarov Dec 5, 2024
2ceb6d2
pipeline: outputs: es: code deduplication
mabrarov Aug 20, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 9 additions & 1 deletion include/fluent-bit/flb_lib.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@

#include <fluent-bit/flb_macros.h>
#include <fluent-bit/flb_config.h>
#include <fluent-bit/flb_input.h>
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Avoid pulling internal headers into the public API; forward-declare instead.

Including flb_input.h in a public header expands API surface and coupling. Forward-declare struct flb_input_instance and remove the include.

-#include <fluent-bit/flb_input.h>
+/* forward declaration to avoid exposing internal header here */
+struct flb_input_instance;
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
#include <fluent-bit/flb_input.h>
/* forward declaration to avoid exposing internal header here */
struct flb_input_instance;
🤖 Prompt for AI Agents 
In include/fluent-bit/flb_lib.h around line 25, remove the internal include
'#include <fluent-bit/flb_input.h>' and instead forward-declare 'struct
flb_input_instance;' at the top of the header; update any use sites in this
header to reference only pointers to that struct (e.g., struct
flb_input_instance *), ensuring no in-header dereferences or inline uses that
require the full definition, and keep the full include in source files or
private headers where the struct layout is needed.


/* Lib engine status */
#define FLB_LIB_ERROR -1
Expand Down Expand Up @@ -69,7 +70,14 @@ FLB_EXPORT int flb_output_set_test(flb_ctx_t *ctx, int ffd, char *test_name,
void (*out_callback) (void *, int, int,
void *, size_t, void *),
void *out_callback_data,
void *test_ctx);
void *flush_ctx);
FLB_EXPORT int flb_output_set_test_flush_ctx_callback(flb_ctx_t *ctx, int ffd,
char *test_name,
void *(*flush_ctx_callback)(
struct flb_config *,
struct flb_input_instance *,
void *, void *),
void *flush_ctx);
FLB_EXPORT int flb_output_set_callback(flb_ctx_t *ctx, int ffd, char *name,
void (*cb)(char *, void *, void *));
FLB_EXPORT int flb_output_set_http_test(flb_ctx_t *ctx, int ffd, char *test_name,
Expand Down
17 changes: 16 additions & 1 deletion include/fluent-bit/flb_output.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -151,9 +151,24 @@ struct flb_test_out_formatter {
*/
void *rt_data;

/* optional context for flush callback */
/* optional context for "flush context callback" */
void *flush_ctx;

/*
* Callback
* =========
* Optional "flush context callback": it references the function that extracts
* optional flush context for "formatter callback".
*/
void *(*flush_ctx_callback) (/* Fluent Bit context */
struct flb_config *,
/* plugin that ingested the records */
struct flb_input_instance *,
/* plugin instance context */
void *plugin_context,
/* context for "flush context callback" */
void *flush_ctx);

/*
* Callback
* =========
Expand Down
1 change: 1 addition & 0 deletions plugins/out_es/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
set(src
es_bulk.c
es_conf_parse.c
es_conf.c
es.c
murmur3.c)
Expand Down
Loading
Loading