-
Notifications
You must be signed in to change notification settings - Fork 19.8k
Description
What problem does this feature solve?
I'm working with Apache Arrow data via Arquero, and I often deal with BigInt values (Apache Arrow's Int64 type). However, Apache ECharts currently stumbles on BigInt, throwing errors like Uncaught TypeError: Cannot convert a BigInt value to a number. There were some discussions (#13819, #12873) on this, but they didn't move forward.
BigInt support is needed for dealing with high-precision timestamps (sensor data) or large integers in datasets.
Apache Arrow is a portable format for storing column-oriented data, allowing data to be shared as-is between various systems and language runtimes. Arquero can process Arrow data directly and serialize data to the Arrow format. It can read from Apache Arrow columns out-of-the-box, enabling zero-copy query processing over loaded data.
I'm trying to avoid pre-processing the data (for casting BigInt to Number), as it is a big amount of data and I would lose the zero-copy optimizations and end up with multiple copies on memory (I already have to live with the unneeded clone of data).
What does the proposed API look like?
-
Adapt Data Handling: Adjust the data handling parts to play nice with
BigInt, especially around uses of theMathAPI and theisFinitemethod. E.g.detectValueatsourceHelper.jsorparseDataValueatdataValueHelper.js. -
Type Conversion: Provide tools to switch between
BigIntandNumberif needed, while ensuring no loss in precision. However, this might not be necessary as all values of a dimension are of the same type. -
Option API Tweak: We can allow specifying
biginttype in the dataset's dimensions, although the type should be easy to guess by looking at the data.
Thanks.