How to make the ChartModifer works in MVVM

Hi I tried to bind the ChartModifer to a property in the VM. But though it build successful, and there is no warning or errors when I run the application , but the ChartModifier does not work.

Here is my code

    <sci:SciChartSurface Grid.Row="1" 
                         YAxes="{sci:AxesBinding YAxes}"
                         XAxes="{sci:AxesBinding XAxes}"
                         RenderableSeries="{sci:SeriesBinding RenderableSeries}"
                         Background="{Binding ChartBackBrush}"
                         Foreground="{Binding ChartForeBrush}"
                         ViewportManager="{Binding ViewportManager}"
                         ChartModifier="{Binding ChartModifiers}">
    </sci:SciChartSurface>

In ViewModel, I created all the child for the ChartModifiers. But it does not work.

        ChartModifiers = new ObservableCollection<IChartModifier>();
        //ModifierGroup.ChildModifiers
        ChartModifiers.Add(rubberBandXyZoomModifier);
        ChartModifiers.Add(new ZoomPanModifier()
        {
            ExecuteOn = ExecuteOn.MouseRightButton,
            ClipModeX = SciChart.Charting.ClipMode.None
        });
        ChartModifiers.Add(new ZoomExtentsModifier());
        ChartModifiers.Add(new MouseWheelZoomModifier());
        rolloverModifer = new RolloverModifier()
        {
            ExecuteOn = ExecuteOn.MouseMove,
            ShowTooltipOn = RolloverVisible
        };
        ChartModifiers.Add(rolloverModifer);

        xAxisDrag = new XAxisDragModifier()
        {
            IsEnabled = true,
            ClipModeX = SciChart.Charting.ClipMode.None,
            DragMode = SciChart.Charting.AxisDragModes.Pan
        };
        yAxisDrag = new YAxisDragModifier()
        {
            IsEnabled = true,
            DragMode = SciChart.Charting.AxisDragModes.Pan
        };

        ChartModifiers.Add(xAxisDrag);
        ChartModifiers.Add(yAxisDrag);