XyDataSeries perfomance, custom IXyDataSeries, and alternatives

Good morning Lorenzo,

Again my sincere apologies for our tardy response on this issue. I’d like to know why you need to throw away the whole DataSeries, however I appreciate that you do and will answer your question as best as I can.

The reason the first method is faster (DataSeries.Clear() than DataSeries.Append()) is internally the memory is not recreated. The memory is cleared but not recreated.

In the second method, new DataSeries(), you are creating a 160MByte buffer (10M points * 2 * 8 bytes) and garbage collecting another one. Despite the operations looking very similar, under the hood there are optimisations when you clear a DataSeries to re-use pooled memory.

When an analysing performance and ‘how can I improve it’ its important to understand the physical limitations of the computer. The memory bandwidth in a modern PC is about 16Gbytes / second. 10Million points of double/double data is 160MBytes, therefore it will take a minimum of 10ms to copy 160MBytes of data into memory, giving you a maximum frame rate of 100FPS. Don’t forget we have to read back this 160MBytes to draw it, so that’s another 10ms dropping the maximum frame rate to 50FPS, unless we do caching internally (which we do if data not changed). This is a physical wall and cannot be circumvented. There is no cheap or easy trick to make the computer outperform memory bandwidth.

So given this, how does DataSeries.Append() work under the hood? If the memory is already allocated (it is, if you clear() – values are cleared but memory is not destroyed, it is reused) then Append() simply fills those buffers again, however it does a little bit more than that. We internally inside DataSeries calculate the distribution of your data. We check if it is sorted ascending, we check if it is evenly spaced. We check if it contains NaN or null or empty points. All of these flags are required for later optimisation of drawing: by knowing the properties of data in advance we are able to select the best performing drawing algorithm for your situation: rather than apply a hammer to everything, we choose the right tool.

Now it is possible to hack (override) the Data Distribution calculation and specify those flags yourself. If you know the distribution of the data, you can tell SciChart and avoid this calculation entirely, making DataSeries.Append much faster. There is an example of this here.

The DataSeries.DataDistributionCalculator is a class which determines
the distribution of your data (sorted in X or not, evenly spaced in X
or not) and the flags are used to determine the correct algorithm(s)
for resampling, hit-testing and indexing of data.

By default this all works automatically, however, if you want to save
a few CPU cycles and you know in advance the distribution of your
data, you can override the flags as follows:

var xyDataSeries = new XyDataSeries<double, double>()
{
    // use this feature to override Data-distribution calculation and
    // provide your own flags to save CPU cycles. Only use if you are certain about the
    // distribution of your data
    DataDistributionCalculator = new UserDefinedDistributionCalculator<double>()
    {
        IsSortedAscending = true,
        IsEvenlySpaced = true,
    }
};

If you would like to discuss further your requirements and how to squeeze precious CPU cycles out of SciChart, I’d welcome you to get in touch with us by using the http://www.scichart.com/contact-us page. We also have a project in-progress now (ETA early September) which is a total performance overhaul, which may help your project get the best from SciChart also.

Best regards,
Andrew