[C++] Create "engine" module for the query engine

[asfimport]

Circular dependencies are popping up in the query engine as the compute module is very low level. For example, it would be nice if the default registry included the scan node and dataset write node. We will want to be adding spillover support at some point and that will rely on parquet/dataset operations.

We should create a dedicated engine module which includes the query plans, the nodes, etc. This module would not contain the kernels or other low level compute primitives. This way we could have something like...

engine -> datasets (for scanning) -> parquet -> compute (for calculating statistics)

The base ExecPlan itself could either go in compute or engine depending on which has the least amount of friction.

Reporter: Weston Pace / @westonpace
Assignee: Jeroen van Straten / @jvanstraten

Related issues:

• [C++] Simplify ExecPlan's C++ interface (is depended upon by)

PRs and other links:

• GitHub Pull Request #11707
• GitHub Pull Request #12279

_{Note: This issue was originally created as ARROW-15238. Please see the migration documentation for further details.}

[asfimport]

Weston Pace / @westonpace:
One artifact of this is that the arrow::dataset::internal::Intialize function should go away and the adding of scan/write tasks to the exec node registry should happen statically when the default exec node registry is created (this default exec node registry creation will need to happen in the engine module).

[asfimport]

Weston Pace / @westonpace:
Issue resolved by pull request 12279
#12279