Skip to content

Conversation

niedbalski
Copy link
Collaborator

Description

This PR fixes an issue where the Record Accessor API couldn't handle direct array access patterns like $array[0].

The fix changes the condition that checks if a lookup succeeded by replacing !found with matched == 0, which properly tracks array element access.

Added test cases to verify array access works:

  • Direct array access ($array[0])
  • Nested array access ($matrix[1][2])
  • Mixed array and map access ($records[1]['name'])
  • Handling of edge cases (out-of-bounds indices, nonexistent keys, type mismatches)

Fixes #9958

Testing

  • Example configuration file for the change
pipeline:
  inputs:
    - name: dummy
      dummy: '{"array": ["a", "b", "c"]}'
  outputs:
    - name: stdout
      match: "*"
      processors:
        logs:
          - name: calyptia
            actions:
              - type: put
                opts:
                  key: "worked"
                  value: "true"
                condition:
                  operator: AND
                  rules:
                    - field: "$array[1]"
                      operator: eq
                      value: "b"
  • Debug log output from testing the change
[2025/02/25 14:55:32] [ info] [processor:calyptia:logs.0] added record accessor rule "$array[1]" eq "b"
[2025/02/25 14:55:32] [ info] [processor:calyptia:logs.0] rule condition result: true
[2025/02/25 14:55:32] [ info] [output:stdout:stdout.0] {"array":["a","b","c"],"worked":"true"}
  • Attached Valgrind output that shows no leaks or memory corruption was found
    [N/A]

  • Run local packaging test showing all targets build
    [N/A]

Documentation

  • Documentation required for this feature
    [N/A]

Backporting

  • Backport to latest stable release.
    [N/A]

Copy link
Contributor

@pwhelan pwhelan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Must applaud the rigorous testing. Good work!

@edsiper edsiper added this to the Fluent Bit v4.0.0 milestone Feb 26, 2025
}

/* Handle map objects */
if (cur.type != MSGPACK_OBJECT_MAP) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in this case, what will happen if the accessed value is another array ?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question. When the accessed value is another array, the function will process it in the next iteration through if (entry->type == FLB_RA_KEY_ARRAY), which runs before the map type check. This is validated in cb_nested_array_access() test, which successfully accesses $matrix[1][2] from {"matrix": [[1, 2, 3], [4, 5, 6], [7, 8, 9]]} to get the value 6.

Record Accessor API couldn't handle direct array access
patterns like $array[0]. This fix changes the condition that
checks if a lookup succeeded by replacing !found with matched == 0,
which properly tracks array element access.

Added test cases to verify array access works:
- Direct array access ($array[0])
- Nested array access ($matrix[1][2])
- Mixed array and map access ($records[1]['name'])
- Access with out-of-bounds indices
- Nonexistent key access
- Type mismatch handling
- Nested path failures

Fixes #9958

Signed-off-by: Jorge Niedbalski <[email protected]>
@niedbalski
Copy link
Collaborator Author

niedbalski commented Feb 27, 2025

@edsiper @cosmo0920 I added a few extra test cases to cover cfl conversion as well. PTAL

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Record Accessor does not support direct array access notation
3 participants