[C++] Support dictionaries directly in case_when kernel

[asfimport]

case_when (and other similar kernels) currently dictionary-decode inputs, then operate on the decoded values. This is both inefficient and unexpected. We should instead operate directly on dictionary indices.

Of course, this introduces more edge cases. If the dictionaries of inputs do not match, we have the following choices:

1. Raise an error.

2. Unify the dictionaries.

3. Use one of the dictionaries, and raise an error if an index of another dictionary cannot be mapped to an index of the chosen dictionary.

4. Use one of the dictionaries, and emit null if an index of another dictionary cannot be mapped to an index of the chosen dictionary. (This is what base dplyr if_else does with factors.)

   All of these options are reasonable, so we should introduce an options struct. We can implement ARROW-10: Fix mismatch of javadoc names and method parameters #3 and ARROW-15: Fix a naming typo for memory.AllocationManager.AllocationOutcome #4 at first (to cover R); ARROW-9: Replace straggler references to Drill #2 isn't strictly necessary, as the user can unify the dictionaries manually first, but it may be more efficient to do it this way. Similarly, ARROW-5: Update drill-fmpp-maven-plugin to 1.5.0 #1 isn't strictly necessary.

#3 and #4 are justifiable (beyond just "it's what R does") since users may filter down disjoint dictionaries into a set of common values and then expect to combine the remaining values with a kernel like case_when.

As described on GitHub.

Reporter: David Li / @lidavidm
Assignee: David Li / @lidavidm

Related issues:

• [C++] Support variable-width types in case_when function (relates to)
• [C++] Improve performance on dictionaries for 'case_when' kernel (is related to)
• [C++] Reconcile type promotion rules between if_else, case_when, coalesce, select (is related to)
• [C++] Optimize dictionary support in kernels/Support nulls in DictionaryUnifier (is related to)

PRs and other links:

• GitHub Pull Request #11022

_{Note: This issue was originally created as ARROW-13573. Please see the migration documentation for further details.}

[asfimport]

David Li / @lidavidm:
Also, we can use the 'fast' approach for dictionaries (being able to write into slices, using preallocated outputs as implemented for numeric inputs, as opposed to the builder-based approach in ARROW-13222 for variable-width types) though we'll want to support nested dictionaries too (lists of dictionaries and such).

[asfimport]

Antoine Pitrou / @pitrou:
Issue resolved by pull request 11022
#11022