Hello Sci Chart support!
I am attempting to apply a low pass filter to a FIFO chart. I have implemented the FilterAll() function successfully but I was wondering if there are examples on how to override the FilterOnAppend() function or how to make FilterAll() work for a FIFO chart.
Currently, when calling the FilterAll() method the program throws an invalid operation exception:
System.InvalidOperationException: ‘Data has been Appended to a DataSeries which is unsorted in the X-Direction. Unsorted data can have severe performance implications in SciChart.
For maximum performance, please double-check that you are only inserting sorted data to SciChart. Alternatively, to disable this warning and allow unsorted data, please set DataSeries.AcceptsUnsortedData = true. For more info see Performance Tips and Tricks at https://www.scichart.com/documentation/v4.x/webframe.html#Performance_Tips_&_Tricks.html‘
I do not want to allow unsorted data because I know it is not a good practice and even if allowing unsorted data, the chart stops updating after a few seconds.
It may be worth knowing that I am sampling data from a National Instruments’ data acquisition board. Sci Chart is helping me display this data, but I would like to display the filtered data as it is being queued.
If you could help me with this matter I would appreciate it!
Best,
Paola
- Matthew Beatty asked 4 years ago
- You must login to post comments
Hi Matthew,
Take a look at our own Moving Average Low Pass filter implementation below:
/// <summary>
/// The filter for moving average
/// </summary>
internal sealed class MovingAverageFilter<TX> : FilterBase where TX : IComparable
{
private readonly XyDataSeries<TX, double> _filteredDataSeries = new XyDataSeries<TX, double>();
private readonly IDataSeries<TX, double> _originalDataSeries;
private readonly int _length;
/// <summary>
/// Initializes a new instance of <see cref="MovingAverageFilter{TX}"/>
/// </summary>
/// <param name="originalDataSeries">The original data series</param>
/// <param name="length">The length</param>
public MovingAverageFilter(IDataSeries<TX, double> originalDataSeries, int length) : base(originalDataSeries)
{
_originalDataSeries = originalDataSeries;
_length = length;
FilteredDataSeries = _filteredDataSeries;
FilterOnAppend(0);
}
/// <summary>
/// Filters on original data series append
/// </summary>
/// <param name="originalStartIndex">The start index of original data series</param>
protected override void FilterOnAppend(int originalStartIndex)
{
for (int i = originalStartIndex; i < _originalDataSeries.Count; ++i)
{
var averageY = AverageOf(i - _length + 1);
_filteredDataSeries.Append(_originalDataSeries.XValues[i], averageY);
}
}
/// <summary>
/// Filters on original data series update
/// </summary>
/// <param name="originalStartIndex">The start index of original data series</param>
protected override void FilterOnUpdate(int originalStartIndex)
{
for (int i = originalStartIndex; i < _originalDataSeries.Count && originalStartIndex + _length > i; ++i)
{
var averageY = AverageOf(i - _length + 1);
_filteredDataSeries.Update(i, averageY);
}
}
/// <summary>
/// Filters on original data series insert
/// </summary>
/// <param name="originalStartIndex">The start index of original data series</param>
/// <param name="changedItemsCount">The amount of changed items</param>
protected override void FilterOnInsert(int originalStartIndex, int changedItemsCount)
{
for (int i = originalStartIndex; i < _originalDataSeries.Count; ++i)
{
var averageY = AverageOf(i - _length + 1);
if (i - originalStartIndex < changedItemsCount)
{
_filteredDataSeries.Update(i, averageY);
}
else
{
_filteredDataSeries.Append(_originalDataSeries.XValues[i], averageY);
}
}
}
/// <summary>
/// Filters on original data series remove
/// </summary>
/// <param name="originalStartIndex">The start index of original data series</param>
/// <param name="changedItemsCount">The amount of changed items</param>
protected override void FilterOnRemove(int originalStartIndex, int changedItemsCount)
{
var filteredStartIndex = 0;
var count = _filteredDataSeries.Count;
for (int i = originalStartIndex, startIndex = filteredStartIndex; i < count; i++)
{
if (i - originalStartIndex <= changedItemsCount)
_filteredDataSeries.RemoveAt(originalStartIndex);
else
{
var averageY = AverageOf(startIndex - _length + 1);
_filteredDataSeries.Update(startIndex, averageY);
}
}
}
/// <summary>
/// Filters data series. You should implement it if you want to filter in this way for all data series changes.
/// Override append, update, insert, remove methods to optimize filtering.
/// </summary>
public override void FilterAll()
{
using (_filteredDataSeries.SuspendUpdates())
{
_filteredDataSeries.Clear();
FilterOnAppend(0);
}
}
private double AverageOf(int from)
{
double result = double.NaN;
if (from >= 0)
{
result = 0.0;
var list = _originalDataSeries.YValues;
for (int i = from; i < from + _length; i++)
{
result += list[i].ToDouble();
}
}
return result / _length;
}
}
FilterOnAppend() is passed an index to start working from, and you work until DataSeries.Count-1
For example, say the DataSeries has 100 points and your filtered series now has X points.
The user appends 50 points to the DataSeries, now it has 150 points.
FilterOnAppend() is called with originalStartIndex=100, and you need to loop from there to DataSeries.Count-1 or index 149 to update the filter.
Anyway, hopefully its clear from the code provided. You may even be able to modify our MovingAverage filter to create your own low pass filter.
Best regards,
Andrew
- Andrew Burnett-Thompson answered 4 years ago
-
Hi Andrew, This was super helpful. I was able to fix my problem! Thanks,
- You must login to post comments
Hi Matthew,
We have docs and a video on the Filters API over here:
The video goes into detail how this API works: https://www.youtube.com/watch?v=OmgCcY5Lmoo
Regarding the error you see, “Data has been Appended to a DataSeries” this is a warning to make sure you meant to append unsorted data. You didn’t mean to, so it means that when you are filtering, you are creating a DataSeries which has unsorted data. Make sure all the X_values are ascending in X and this error won’t show.
FilterALl() basically is like a reset of the entire filter, you have to clear the underlying DataSeries and recreate all points. Its better to use the FilterOnAppend() and similar functions to do incremental filtering.
Let me know if this helps,
Best regards,
Andrew
- Andrew Burnett-Thompson answered 4 years ago
-
Hi Andrew, I have replied to the thread with more information. Any advice would be welcomed! Thanks,
- You must login to post comments
Hi Andrew,
This is helpful. I am attempting to override FilterOnAppend() but I have trouble defining the index argument. Is there a way to specify the index of the latest item that has been added to the original data series? using originalDataSeries.Count – 1 doesn’t yield the desired result. See the attached image.
Also, if you could provide any additional information about FilterOnAppend() that would be helpful since I don’t know how the function is supposed to work.
A little snippet of what I have so far:
protected override void FilterOnAppend(int index)
{
filteredDataSeries.Append(originalDataSeries.XValues[index], originalDataSeries.YValues[index]);
double cutoffFrequency = 10;
double RC = 1.0 / (2 * Math.PI * cutoffFrequency);
double dt = 1.0 / ConfigInfo.SAMPLE_RATE;
double beta = dt / (RC + dt);
if (index > 1)
{
double xValue = originalDataSeries.XValues[index];
double yValue = beta * originalDataSeries.YValues[index] + (1 - beta) * filteredDataSeries.YValues[index - 1];
filteredDataSeries.Append(xValue, yValue);
}
}
Thanks,
- Matthew Beatty answered 4 years ago
- You must login to post comments
Please login first to submit.