Data Manager and Data Feeds

A chart's candles arrive through data feeds, and are held in its data manager, where studies read them. This page describes that flow, for when you read a chart's data yourself or write a study.

Data Feeds: Where the Candles Come From

A data feed loads one stream of data for a chart from its exchange: the candles for its symbol, interval and period, then live updates. Every chart has a default feed, DEFAULT_FEED_ID ("Default"), with its OHLCV candles; studies that need other data add feeds of their own:

Feed type (`EDataFeedType`)	What it carries	Added by
`OHLCVRealtime`	Candles: history, then live updates	Every chart: the default feed
`Trade`	Individual trades	`LargeTradeStudy`
`OrderBook`	The order book over time	The order book studies; see Order Book Data

setSymbol, changeInterval and createDataFeed replace the default feed; see Changing the Symbol.

The Data Manager: One Column per Value

The data manager holds the chart's data as columns, all index-aligned with the candles: xValues holds each candle's time, in seconds, and each other column one value per candle.

The default feed's columns are Open, High, Low, Close and Volume (EDataSourceStringId).
Each study adds output columns, e.g. the SMA's values, which the legend reads.
Each column is named by a DataSourceId: find the feed's columns by name, and a study's in its outputIds.

import { DEFAULT_FEED_ID, EDataSourceStringId } from "scichart-scitrader";

const dataManager = financeChartPanel.financeApi.dataManager;

const close = dataManager.getSourceIdByName(DEFAULT_FEED_ID, EDataSourceStringId.CLOSE);
const closes = dataManager.getValues<number>(close); // one per candle
const times = dataManager.xValues; // each candle's time, in seconds

const rsiValues = dataManager.getValues<number>(rsiStudy.outputIds[0]); // a study's output

Following Changes: observeChanges and EDataUpdateType

observeChanges(ids) emits every change to the given columns, as an EDataUpdateType:

`EDataUpdateType`	When
`Replace`	A whole new series: the first load, and a new symbol, interval or period
`Append`	A new candle opens
`Update`	A live tick changes the forming candle
`Insert`	Older candles, loaded as the user pans back
`Evict`	Candles dropped from either end to save memory

const subscription = dataManager.observeChanges<number>([close]).subscribe((update) => {
    const { updateType, values, startIndex } = update[close.id];
    console.log(EDataUpdateType[updateType], values.length, "values from index", startIndex);
});
subscription.unsubscribe(); // when done

Studies follow their input columns the same way, and recalculate for each update type; see Creating Custom Studies.

note

Most changes are Update: on a busy market, the forming candle changes many times a second. Keep the work you do for each one small.

How History Loads

The first load fetches the candles for the chart's period, plus a buffer to pan back into, plus the warm-up the studies need before their first value: an SMA of 50 needs 50 earlier candles. See Periods.
Panning back loads older candles as the user nears the start, in pages, until the exchange's history begins (Insert).
Live updates extend the series (Append, Update).
The memory cap. On a long session, or after panning far back, candles far from the view are dropped (Evict), and loaded again if needed.

TypeScript Example: Inside the Data Manager

One-minute BTC-USD candles from Coinbase, with SMA and RSI. The panel on the right lists the data manager's columns, the feed's and the studies', and counts the updates to the close column by type, as they arrive: an Update for each live tick, an Append for each new candle.

The code for this example is found below.

import "scichart-scitrader/scitrader.css";
import {
    CandleInterval,
    CoinbaseExchange,
    DEFAULT_FEED_ID,
    defaultExchangeProvider,
    EDataSourceStringId,
    EDataUpdateType,
    idbPriceStore,
    OHLCStudy,
    RSIStudy,
    SciFinanceChartPanel,
    SMAStudy,
} from "scichart-scitrader";

// This example expects this HTML:
// <div style="display: flex; height: 100%">
//     <div id="chart" style="flex: 1; min-width: 0"></div>
//     <pre id="store-log" style="width: 340px"></pre>
// </div>

// Register a data source, once, at app startup
defaultExchangeProvider.registerExchange(CoinbaseExchange, idbPriceStore);

async function createChart() {
    const panel = document.getElementById("store-log");
    Object.assign(panel.style, logStyle);

    // One-minute candles, so live updates arrive often
    const financeChartPanel = await SciFinanceChartPanel.create(document.getElementById("chart"), {
        exchangeName: CoinbaseExchange.name,
        symbol: "BTC-USD",
        interval: CandleInterval.ONE_MINUTE,
        period: "1H",
        studies: [new OHLCStudy(), new SMAStudy({ length: 20 }), new RSIStudy()],
    });
    const dataManager = financeChartPanel.financeApi.dataManager;

    // Every column is named by a DataSourceId. The default feed's columns are found by name:
    // Open, High, Low, Close and Volume. Studies add their own output columns, e.g. SMA's
    const close = dataManager.getSourceIdByName(DEFAULT_FEED_ID, EDataSourceStringId.CLOSE);

    // observeChanges: every change to the close column, as an EDataUpdateType.
    // Replace: a whole new series. Append: a new candle. Update: a live tick changed the latest candle.
    // Insert: older candles, loaded as you pan back. Evict: candles dropped to save memory
    const counts: Record<string, number> = {};
    const subscription = dataManager.observeChanges<number>([close]).subscribe((update) => {
        const type = EDataUpdateType[update[close.id].updateType];
        counts[type] = (counts[type] ?? 0) + 1;
    });

    // Show the columns and the update counts, once a second
    const render = () => {
        const columns = dataManager
            .getSources()
            .map((id) => `${(id.legendName || id.name).padEnd(10)} ${dataManager.getValues(id)?.length ?? 0} values`)
            .join("\n");
        const latest = dataManager.getValues<number>(close).at(-1);
        panel.textContent =
            `Candles: ${dataManager.xValues.length}, latest close ${latest?.toLocaleString()}\n\n` +
            `Columns (DataSourceId):\n${columns}\n\n` +
            `Updates to Close (EDataUpdateType):\n${JSON.stringify(counts, null, 1).replace(/[{}"]/g, "")}`;
    };
    const timer = setInterval(render, 1000);
    render();

    // When the page goes away: clearInterval(timer); subscription.unsubscribe(); financeChartPanel.cleanup();
    return { financeChartPanel, subscription, timer };
}

const logStyle: Partial<CSSStyleDeclaration> = {
    margin: "0",
    padding: "8px 12px",
    overflowY: "auto",
    background: "var(--st-ui-background, #2b2839)",
    borderLeft: "1px solid var(--st-ui-divider, rgba(255,255,255,0.1))",
    color: "var(--st-ui-text-muted, #9da2bc)",
    font: "12px/1.5 monospace",
    whiteSpace: "pre-wrap",
};

createChart();

React Example: Inside the Data Manager

The same in React: the chart's data manager comes from onInit, and the subscription lives in an effect.

The code for this example is found below.

React

import "scichart-scitrader/scitrader.css";
import * as React from "react";
import {
    CandleInterval,
    CoinbaseExchange,
    DEFAULT_FEED_ID,
    defaultExchangeProvider,
    EDataSourceStringId,
    EDataUpdateType,
    idbPriceStore,
    IDataManager,
    ISciFinanceChartPanelApi,
    ISciFinanceChartPanelConfig,
    OHLCStudy,
    RSIStudy,
    SMAStudy,
} from "scichart-scitrader";
import { SciFinanceChartView } from "scichart-scitrader-react";

// Register a data source, once, at app startup
defaultExchangeProvider.registerExchange(CoinbaseExchange, idbPriceStore);

export function DataManagerChart() {
    const [dataManager, setDataManager] = React.useState<IDataManager>();
    const [text, setText] = React.useState("");

    // One-minute candles, so live updates arrive often. Create the config once per mounted chart
    const config = React.useMemo<ISciFinanceChartPanelConfig>(
        () => ({
            exchangeName: CoinbaseExchange.name,
            symbol: "BTC-USD",
            interval: CandleInterval.ONE_MINUTE,
            period: "1H",
            studies: [new OHLCStudy(), new SMAStudy({ length: 20 }), new RSIStudy()],
        }),
        []
    );

    // Once the chart is ready: count the updates to the close column, and show the columns
    React.useEffect(() => {
        if (!dataManager) return undefined;
        // Every column is named by a DataSourceId. The default feed's columns are found by name
        const close = dataManager.getSourceIdByName(DEFAULT_FEED_ID, EDataSourceStringId.CLOSE);
        // observeChanges: every change to the close column, as an EDataUpdateType: Replace, Append,
        // Update (a live tick), Insert (older candles) or Evict (candles dropped to save memory)
        const counts: Record<string, number> = {};
        const subscription = dataManager.observeChanges<number>([close]).subscribe((update) => {
            const type = EDataUpdateType[update[close.id].updateType];
            counts[type] = (counts[type] ?? 0) + 1;
        });
        const render = () => {
            const columns = dataManager
                .getSources()
                .map((id) => `${(id.legendName || id.name).padEnd(10)} ${dataManager.getValues(id)?.length ?? 0} values`)
                .join("\n");
            const latest = dataManager.getValues<number>(close).at(-1);
            setText(
                `Candles: ${dataManager.xValues.length}, latest close ${latest?.toLocaleString()}\n\n` +
                    `Columns (DataSourceId):\n${columns}\n\n` +
                    `Updates to Close (EDataUpdateType):\n${JSON.stringify(counts, null, 1).replace(/[{}"]/g, "")}`
            );
        };
        const timer = setInterval(render, 1000);
        return () => {
            clearInterval(timer);
            subscription.unsubscribe();
        };
    }, [dataManager]);

    // Fills its parent, so give the parent a height
    return (
        <div style={{ display: "flex", height: "100%" }}>
            <div style={{ flex: 1, minWidth: 0 }}>
                {/* financeApi throws until the chart has initialised, so read it in onInit */}
                <SciFinanceChartView config={config} onInit={(api: ISciFinanceChartPanelApi) => setDataManager(api.financeApi.dataManager)} />
            </div>
            <pre style={logStyle}>{text}</pre>
        </div>
    );
}

const logStyle: React.CSSProperties = {
    width: 340,
    margin: 0,
    padding: "8px 12px",
    overflowY: "auto",
    background: "var(--st-ui-background, #2b2839)",
    borderLeft: "1px solid var(--st-ui-divider, rgba(255,255,255,0.1))",
    color: "var(--st-ui-text-muted, #9da2bc)",
    font: "12px/1.5 monospace",
    whiteSpace: "pre-wrap",
};

Data Manager API

The IDataManager interface, with its TSDoc

interface IDataManager {
    xId: DataSourceId;
    xValues: number[];
    yEntries: {
        id: DataSourceId;
        values: any[];
    }[];
    getSources(): DataSourceId[];
    getValues<T>(id: DataSourceId): T[];
    getSourceId(id: string): DataSourceId;
    getSourceIdByName(feedId: string, name: string): DataSourceId;
    getOrCreateSourceId(id: string, builder: () => DataSourceId): DataSourceId;
    replaceValues(ids: DataSourceId[], values: any[][]): void;
    appendValues(ids: DataSourceId[], values: any[][], startIndex: number): void;
    insertValues(ids: DataSourceId[], values: any[][], startIndex: number): void;
    updateValues(ids: DataSourceId[], values: any[], startIndex: number): void;
    /**
     * Drop `leadingCount` entries from the front and `trailingCount` from
     * the back of every registered source, emitting one `Evict` update per
     * source.
     */
    evict(leadingCount: number, trailingCount: number): void;
    observeChanges<T>(ids: DataSourceId[]): Observable<TDataUpdate<T>>;
    removeSources(ids: DataSourceId[]): void;
}

The DataSourceId class, EDataSourceStringId, EDataFeedType and DEFAULT_FEED_ID, with their TSDoc

class DataSourceId {
    static DEFAULT_X_ID: DataSourceId;
    static create(dataManager: IDataManager, name?: string, legendName?: string, studyId?: StudyId, feedId?: string): DataSourceId;
    readonly feedId: string;
    readonly name: string;
    readonly legendName: string;
    readonly id: string;
    toString(): string;
}

enum EDataSourceStringId {
    X = "X",
    OPEN = "Open",
    HIGH = "High",
    LOW = "Low",
    CLOSE = "Close",
    VOLUME = "Volume"
}

enum EDataFeedType {
    OHLCVRealtime = "OHLCVRealtime",
    OHLCV1second = "OHLCV1second",
    Trade = "Trade",
    UserOrder = "UserOrder",
    OrderBook = "OrderBook"
}

const DEFAULT_FEED_ID = "Default";

The EDataUpdateType enum, IDataUpdate interface and TDataUpdate type, with their TSDoc

enum EDataUpdateType {
    Replace = 0,
    Append = 1,
    Insert = 2,
    Update = 3,
    /**
     * Candles were removed from one or both ends of the source array
     * `values` is an empty array (no new data); the
     * `eviction` field carries the per-edge drop counts. Subscribers that
     * mirror data into renderable series should shrink their copy
     * symmetrically; subscribers that recompute outputs from inputs (e.g.
     * indicator studies) can treat Evict like a Replace-shrunken — values
     * for surviving indices are unchanged, so no recomputation is needed
     * beyond keeping the output series the same length as the input.
     */
    Evict = 4
}
interface IDataUpdate<T> {
    id: DataSourceId;
    values: T[];
    updateType: EDataUpdateType;
    startIndex?: number;
    /**
     * Populated only for {@link EDataUpdateType.Evict}. Reports how many
     * elements were dropped from each edge of the source array, so
     * subscribers can mirror the eviction without recomputing indices.
     */
    eviction?: {
        leadingDrop: number;
        trailingDrop: number;
    };
}
type TDataUpdate<T> = Record<string, IDataUpdate<T>>;

Where to read next

Creating Custom Studies: a study that reads and writes columns.
Candle Stores and Caching: where the candles come from.
Periods: how much history a chart loads.

Data Feeds: Where the Candles Come From​

The Data Manager: One Column per Value​

Following Changes: observeChanges and EDataUpdateType​

How History Loads​

TypeScript Example: Inside the Data Manager​

React Example: Inside the Data Manager​

Data Manager API​

Where to read next​