Skip to content

Conversation

@alamb
Copy link
Contributor

@alamb alamb commented Nov 3, 2023

Which issue does this PR close?

N/A

Rationale for this change

In general I think it would be good for us to communicate more broadly what we are working on, for at least two reasons:

  1. So people who may be interested / impacted have a chance to weigh in on designs
  2. To help find people to help with projects that may need additional assistance

I would love additional ideas about how to do this -- maybe weekly summaries of "this week in DataFusion" 🤔 Any takers for doing so?

What changes are included in this PR?

Update quarterly roadmap. See rendered version here: https://github.com/apache/arrow-datafusion/blob/alamb-patch-1/docs/source/contributor-guide/quarterly_roadmap.md

Are these changes tested?

Are there any user-facing changes?

@alamb alamb changed the title Update: Update quarterly_roadmap.md Minor: Update quarterly_roadmap.md Nov 3, 2023
@alamb alamb marked this pull request as ready for review November 3, 2023 17:00
- Improve data output (`COPY`, `INSERT` and DataFrame) output capability [#6569](https://github.com/apache/arrow-datafusion/issues/6569)
- Implementation of `ARRAY` types and related functions [#6980](https://github.com/apache/arrow-datafusion/issues/6980)
- Write an industrial paper about DataFusion for SIGMOD [#6782](https://github.com/apache/arrow-datafusion/issues/6782)
- Faster Merging with Parallel Cascaded Merge: [#7181](https://github.com/apache/arrow-datafusion/issues/7181)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI @wiedld

@alamb alamb requested a review from andygrove November 3, 2023 17:03
Copy link
Contributor Author

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are several potential non trivial changes underway that I think would benefit significantly from additional help - both design feedback, as well as code review, and assistance communicating, etc.

I would really love to help them all but I am running out of time, and I want to be as clear as possible that I am unlikely to be able to help with all of this.

My ask for other maintainers to offer to help guide contributors through some of the following efforts, ideally focused on areas that are aligned with their interests


- Improved planning speed, especially for schemas with large numbers of columns [#5637](https://github.com/apache/arrow-datafusion/issues/5637) / [#7698](https://github.com/apache/arrow-datafusion/issues/7698)
- Make it easier to customize DataFusion, with split out function packages, etc. - [#7977](https://github.com/apache/arrow-datafusion/issues/7977)
- User Defined Types [#7923](https://github.com/apache/arrow-datafusion/issues/7923)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fyi @yukkit

- Improved planning speed, especially for schemas with large numbers of columns [#5637](https://github.com/apache/arrow-datafusion/issues/5637) / [#7698](https://github.com/apache/arrow-datafusion/issues/7698)
- Make it easier to customize DataFusion, with split out function packages, etc. - [#7977](https://github.com/apache/arrow-datafusion/issues/7977)
- User Defined Types [#7923](https://github.com/apache/arrow-datafusion/issues/7923)
- ORC Support [#4707](https://github.com/apache/arrow-datafusion/issues/4707)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The following work items are still in the planning / proposal stage.

- Improved planning speed, especially for schemas with large numbers of columns [#5637](https://github.com/apache/arrow-datafusion/issues/5637) / [#7698](https://github.com/apache/arrow-datafusion/issues/7698)
- Make it easier to customize DataFusion, with split out function packages, etc. - [#7977](https://github.com/apache/arrow-datafusion/issues/7977)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@alamb
Copy link
Contributor Author

alamb commented Dec 12, 2023

This seems not to have garnered much attention, so let's just shut it down

@alamb alamb closed this Dec 12, 2023
@alamb alamb deleted the alamb-patch-1 branch January 17, 2024 14:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants