Pre loader

BigInt support in SciChart for financial data

Welcome to the SciChart Forums!

  • Please read our Question Asking Guidelines for how to format a good question
  • Some reputation is required to post answers. Get up-voted to avoid the spam filter!
  • We welcome community answers and upvotes. Every Q&A improves SciChart for everyone

WPF Forums | JavaScript Forums | Android Forums | iOS Forums

1
0

Hello, has anyone found any solution with regards to using BigInt in SciChartJS. I am building an app that renders time series chart of say last 2 hours of market data activity of a given instrument (ESH3, a CME futures instrument).

The backend sends the data as JSON object in fragments (1000 fragments with 1000 items in an array fragment, so the total data for 2 hours is 1000 x 1000 = 1,000,000 items in an array).

Each item in the array has a time field (for x-axis) and say TradePrice (for y-axis). The time field is a unix time and it just cannot fit into the number field. So we parse it using BigInt as follows:
JSON.parse(rawData, (key, value) => {
if (key.includes('time')) return BigInt(value);
return value;
});

The parsing works, but, now SciChartJS is not happy about that. It needs number to append the data into its XyDataSeries object.

Has anyone come across this situation before and how have you solved this?

I am thinking the following and would appreciate if someone would give their 2nd opinion as well:
1. Convert to BigInt during deserializing
2. Create a global offset value — eg. Offset = nanoseconds since mid-night of 2020
3. Use Value – Offset as x value when appending to XyDataSeries
4. When a data-point is selected in chart, use SelectedTime + Offset for get to the original value from backend

Question for ScichartJS Support:
– Any provision to support BigInt in SciChartJS in near future? This would definitely help those in financial sector with requirements like I metioned above.

Version
2.2+ and 3.0+
  • You must to post comments
1
0

Hi Sachin

Got a few questions for you. Firstly, we do not support BigInt at the moment. However potentially we could

Question – what is BigInt? Is it Int64? Or is it a string format or something.

Underneath we’re using WebAssembly for data storage and mathematical operations. Wasm supports f64 i64 data types as well as f32. There’s no reason why we couldn’t store i64 in a DataSeries as we do something very similar on our WPF (Windows) charts where the DataSeries are generic (templated) and support all kinds of data types.

The problem becomes when you do any maths. Say you have i64 numbers stored as xValies in your DataSeries. When converting to pixel coordinate we have to do mathematical operations. These have to be done using double-precision (f64). You can get some precision loss there even if your data is i64.

It is possible to do all the calculations using a higher precision data type like “long double” (80 or 128bit double) but one doesn’t exist in wasm/js. It would have to be emulated, which means performance drop.

Perhaps there’s another workaround? Achieving high precision data axis should be possible with f64 datatypes but might require some data pre-processing. That’s something we could discuss further

Best regards
Andrew

  • You must to post comments
1
0

Hi Andrew, to answer your question on “What is BigInt in JavaScript”, we have to first look at Number.

Number
* According to https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Number:
* Max integer without losing precision is -(2^53 + 1) to (2^53-1)
* In chrome dev tools: Number.MAX_SAFE_INTEGER = 9_007_199_254_740_991
* Numbers are doubles, that is, 64-bit floating-point values. They can contain any 32-bit integer with full precision, but not all 64-bit ones.

BigInt
* Details at: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/BigInt:
* Google will also lead you to many other docs.
* In a nutshell, it represents integers of arbitrary size so that they can represent 64-bit integers properly.
* BigInts cannot store floats or doubles.
* This has been **officially supported* in chrome v67+ since May 28, 2018 (https://caniuse.com/bigint)

unix timestamp that I am dealing with
* This is one of the timestamp I get from our backend (as string): 1_674_831_606_568_193_791
* Saving this into a number results in rounding: 1_674_831_606_568_193_800
* Saving this into a BigInt is not a problem but presents similar rounding issue with SciChart

Few things that I found on google that might of interest to you as it relates to BigInt and Wasm:
* https://v8.dev/features/wasm-bigint
* https://sauleau.com/notes/wasm-bigint.html

Unfortunately, I don’t have any experience with WebAssembly, so I cannot comment on how easy/hard the implementation would be on SciChart’s side.

Let say, hypothetically, that you could introduce generics in DataSeries to support Number or BigInt (don’t think we need more than these two), what is a realistic time frame for releasing it — 6 months, 1 year, more?
* I won’t hold you on the timeline, just want to get a feel for it so that I can plan accordingly.

As always, really appreciate for your prompt response.

Best Regards,
Sachin Patel.

  • You must to post comments
0
0

Andrew,
Am I correct in parsing your answer to the above question that the LabelProvider classes input argument is a FP32 value?
i.e. if we have xAxis.labelProvider.formatLabel = (timeStamp) => {
that timeStamp is treated as FP32?
Thanks for clarifying,
Mike

  • Andrew Burnett-Thompson
    Hi Mike, the input to a LabelProvider is float64 as this is the only data type in javascript (JS number). For other platforms of scichart (iOS android WPF) we also use f64 as an intermediate type for calculations, although data can be stored in Int64
  • You must to post comments
Showing 3 results
Your Answer

Please first to submit.

Try SciChart Today

Start a trial and discover why we are the choice
of demanding developers worldwide

Start TrialCase Studies