Skip to content

Commit 51ad7fd

Browse files
reegnzreegnz
authored
feat: Expose lua filter type_array_key parameter (#1323)
* feat: Expose lua filter type_array_key parameter The lua filter was missing a parameter `type_array_key`. I've copied the parameter description into the type comment from the fluent-bit docs. Ran `make generate manifests docs-update verify build`. Signed-off-by: Zoltán Reegn <[email protected]> * fix var name Signed-off-by: Zoltán Reegn <[email protected]> --------- Signed-off-by: Zoltán Reegn <[email protected]>
1 parent 850cb1b commit 51ad7fd

File tree

10 files changed

+72
-2
lines changed

10 files changed

+72
-2
lines changed

‎apis/fluentbit/v1alpha2/plugins/filter/lua_types.go

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,9 @@ type Lua struct {
2828
// Note that starting from Fluent Bit v1.6 integer data types are preserved
2929
// and not converted to double as in previous versions.
3030
TypeIntKey []string `json:"typeIntKey,omitempty"`
31+
// If these keys are matched, the fields are handled as array. If more than
32+
// one key, delimit by space. It is useful the array can be empty.
33+
TypeArrayKey []string `json:"typeArrayKey,omitempty"`
3134
// If enabled, Lua script will be executed in protected mode.
3235
// It prevents to crash when invalid Lua script is executed. Default is true.
3336
ProtectedMode *bool `json:"protectedMode,omitempty"`
@@ -51,7 +54,7 @@ func (l *Lua) Params(_ plugins.SecretLoader) (*params.KVs, error) {
5154

5255
if l.Code != "" {
5356
var singleLineLua string = ""
54-
var lineTrim = ""
57+
lineTrim := ""
5558
for _, line := range strings.Split(strings.TrimSuffix(l.Code, "\n"), "\n") {
5659
lineTrim = strings.TrimSpace(line)
5760
if lineTrim != "" {
@@ -76,12 +79,17 @@ func (l *Lua) Params(_ plugins.SecretLoader) (*params.KVs, error) {
7679
kvs.Insert("type_int_key", strings.Join(l.TypeIntKey, " "))
7780
}
7881

82+
if l.TypeArrayKey != nil && len(l.TypeArrayKey) > 0 {
83+
kvs.Insert("type_array_key", strings.Join(l.TypeArrayKey, " "))
84+
}
85+
7986
if l.ProtectedMode != nil {
8087
kvs.Insert("protected_mode", strconv.FormatBool(*l.ProtectedMode))
8188
}
8289

8390
if l.TimeAsTable {
8491
kvs.Insert("time_as_table", "true")
8592
}
93+
8694
return kvs, nil
8795
}

‎apis/fluentbit/v1alpha2/plugins/filter/zz_generated.deepcopy.go

Lines changed: 5 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎charts/fluent-operator/charts/fluent-bit-crds/crds/fluentbit.fluent.io_clusterfilters.yaml

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -464,6 +464,13 @@ spec:
464464
If you desire timestamp precision enabling this option will pass the timestamp as
465465
a Lua table with keys sec for seconds since epoch and nsec for nanoseconds.
466466
type: boolean
467+
typeArrayKey:
468+
description: |-
469+
If these keys are matched, the fields are handled as array. If more than
470+
one key, delimit by space. It is useful the array can be empty.
471+
items:
472+
type: string
473+
type: array
467474
typeIntKey:
468475
description: |-
469476
If these keys are matched, the fields are converted to integer.

‎charts/fluent-operator/charts/fluent-bit-crds/crds/fluentbit.fluent.io_filters.yaml

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -464,6 +464,13 @@ spec:
464464
If you desire timestamp precision enabling this option will pass the timestamp as
465465
a Lua table with keys sec for seconds since epoch and nsec for nanoseconds.
466466
type: boolean
467+
typeArrayKey:
468+
description: |-
469+
If these keys are matched, the fields are handled as array. If more than
470+
one key, delimit by space. It is useful the array can be empty.
471+
items:
472+
type: string
473+
type: array
467474
typeIntKey:
468475
description: |-
469476
If these keys are matched, the fields are converted to integer.

‎config/crd/bases/fluentbit.fluent.io_clusterfilters.yaml

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -464,6 +464,13 @@ spec:
464464
If you desire timestamp precision enabling this option will pass the timestamp as
465465
a Lua table with keys sec for seconds since epoch and nsec for nanoseconds.
466466
type: boolean
467+
typeArrayKey:
468+
description: |-
469+
If these keys are matched, the fields are handled as array. If more than
470+
one key, delimit by space. It is useful the array can be empty.
471+
items:
472+
type: string
473+
type: array
467474
typeIntKey:
468475
description: |-
469476
If these keys are matched, the fields are converted to integer.

‎config/crd/bases/fluentbit.fluent.io_filters.yaml

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -464,6 +464,13 @@ spec:
464464
If you desire timestamp precision enabling this option will pass the timestamp as
465465
a Lua table with keys sec for seconds since epoch and nsec for nanoseconds.
466466
type: boolean
467+
typeArrayKey:
468+
description: |-
469+
If these keys are matched, the fields are handled as array. If more than
470+
one key, delimit by space. It is useful the array can be empty.
471+
items:
472+
type: string
473+
type: array
467474
typeIntKey:
468475
description: |-
469476
If these keys are matched, the fields are converted to integer.

‎docs/plugins/fluentbit/filter/lua.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,5 +9,6 @@ The Lua Filter allows you to modify the incoming records using custom Lua Script
99
| call | Lua function name that will be triggered to do filtering. It's assumed that the function is declared inside the Script defined above. | string |
1010
| code | Inline LUA code instead of loading from a path via script. | string |
1111
| typeIntKey | If these keys are matched, the fields are converted to integer. If more than one key, delimit by space. Note that starting from Fluent Bit v1.6 integer data types are preserved and not converted to double as in previous versions. | []string |
12+
| typeArrayKey | If these keys are matched, the fields are handled as array. If more than one key, delimit by space. It is useful the array can be empty. | []string |
1213
| protectedMode | If enabled, Lua script will be executed in protected mode. It prevents to crash when invalid Lua script is executed. Default is true. | *bool |
1314
| timeAsTable | By default when the Lua script is invoked, the record timestamp is passed as a Floating number which might lead to loss precision when the data is converted back. If you desire timestamp precision enabling this option will pass the timestamp as a Lua table with keys sec for seconds since epoch and nsec for nanoseconds. | bool |

‎docs/plugins/fluentbit/output/open_telemetry.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@ The OpenTelemetry plugin allows you to take logs, metrics, and traces from Fluen
1515
| tracesUri | Specify an optional HTTP URI for the target web server listening for traces, e.g: /v1/traces | string |
1616
| header | Add a HTTP header key/value pair. Multiple headers can be set. | map[string]string |
1717
| logResponsePayload | Log the response payload within the Fluent Bit log. | *bool |
18-
| logsBodyKeyAttributes | Adds unmatched keys as attributes when true. | *bool |
1918
| addLabel | This allows you to add custom labels to all metrics exposed through the OpenTelemetry exporter. You may have multiple of these fields. | map[string]string |
19+
| logsBodyKeyAttributes | If true, remaining unmatched keys are added as attributes. | *bool |
2020
| tls | | *[plugins.TLS](../tls.md) |
2121
| networking | Include fluentbit networking options for this output-plugin | *plugins.Networking |

‎manifests/setup/fluent-operator-crd.yaml

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -463,6 +463,13 @@ spec:
463463
If you desire timestamp precision enabling this option will pass the timestamp as
464464
a Lua table with keys sec for seconds since epoch and nsec for nanoseconds.
465465
type: boolean
466+
typeArrayKey:
467+
description: |-
468+
If these keys are matched, the fields are handled as array. If more than
469+
one key, delimit by space. It is useful the array can be empty.
470+
items:
471+
type: string
472+
type: array
466473
typeIntKey:
467474
description: |-
468475
If these keys are matched, the fields are converted to integer.
@@ -14959,6 +14966,13 @@ spec:
1495914966
If you desire timestamp precision enabling this option will pass the timestamp as
1496014967
a Lua table with keys sec for seconds since epoch and nsec for nanoseconds.
1496114968
type: boolean
14969+
typeArrayKey:
14970+
description: |-
14971+
If these keys are matched, the fields are handled as array. If more than
14972+
one key, delimit by space. It is useful the array can be empty.
14973+
items:
14974+
type: string
14975+
type: array
1496214976
typeIntKey:
1496314977
description: |-
1496414978
If these keys are matched, the fields are converted to integer.

‎manifests/setup/setup.yaml

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -463,6 +463,13 @@ spec:
463463
If you desire timestamp precision enabling this option will pass the timestamp as
464464
a Lua table with keys sec for seconds since epoch and nsec for nanoseconds.
465465
type: boolean
466+
typeArrayKey:
467+
description: |-
468+
If these keys are matched, the fields are handled as array. If more than
469+
one key, delimit by space. It is useful the array can be empty.
470+
items:
471+
type: string
472+
type: array
466473
typeIntKey:
467474
description: |-
468475
If these keys are matched, the fields are converted to integer.
@@ -14959,6 +14966,13 @@ spec:
1495914966
If you desire timestamp precision enabling this option will pass the timestamp as
1496014967
a Lua table with keys sec for seconds since epoch and nsec for nanoseconds.
1496114968
type: boolean
14969+
typeArrayKey:
14970+
description: |-
14971+
If these keys are matched, the fields are handled as array. If more than
14972+
one key, delimit by space. It is useful the array can be empty.
14973+
items:
14974+
type: string
14975+
type: array
1496214976
typeIntKey:
1496314977
description: |-
1496414978
If these keys are matched, the fields are converted to integer.

0 commit comments

Comments
 (0)