Candle Stores and Caching
Market data APIs are slow and rate-limited, so SciTrader caches candles at two levels:
- In memory, for the session. Each registered exchange's client keeps the candles it has loaded for each symbol and interval, as ranges. Panning back over loaded history, or switching back to an interval, needs no request.
- In a candle store, between sessions. The store you register the exchange with,
registerExchange(exchange, store), keeps candles across page loads.idbPriceStorekeeps them in the browser's IndexedDB, so a reload draws from the cache and fetches only newer candles.
How a Load Uses the Cache
When a chart needs candles for a range, the client:
- reads the store for the candles it has in that range;
- fetches the gaps from the exchange, in requests of up to its
candleLimitcandles, padded to whole requests, since exchanges charge per request, not per candle; - writes what it fetched to the store;
- remembers where history starts. When an exchange returns fewer candles than asked for, its history starts there. The store keeps this listing-date floor, so a later load doesn't ask for older candles that don't exist;
- remembers empty ranges, such as a halted market, so it doesn't ask for them again.
The example below logs each of these as they happen.
The IndexedDB Store: idbPriceStore
idbPriceStore is the built-in store, an IdbCandleStore. It stores candles in compact binary chunks, keyed by exchange, symbol and interval, so two exchanges that use the same symbol never share candles.
| Member | What it does |
|---|---|
clear | Deletes every cached candle, e.g. for a "Clear cache" button. The next load fetches again |
get | The most candles kept for an interval |
set | Changes it, and saves the choice in local. Number. for no cap |
reset | Back to the default caps |
subscribe | Called after old candles are deleted to keep the store's size down |
clean | How often old candles are deleted, in ms. Default: 24 hours |
How much is kept. By default, up to 50,000 candles per symbol at intervals from 1m to 30m, and every candle at 1h and longer. Older candles beyond the cap are deleted, at most once a day (cleanInterval).
When the cache is cleared for you. The cache is versioned with the SDK: after you update the SDK, the old cache is deleted on the first load.
IndexedDB is shared by every page of a site. If two pages that use the cache, such as two docs pages or two tabs of your app, are open at once, opening the cache can fail in one of them, and its chart stays empty. A fix that falls back to no cache is planned. Until then, if your app opens charts in several tabs, use nullCandleStore in all but one.
Not Caching: nullCandleStore
nullCandleStore stores nothing: every page load starts cold, though the in-memory cache still works within a session. Use it for:
- data that changes between loads, such as CSV data moved to the current date, or generated data.
registerCSVExchangeuses it; - finding a caching problem: if a chart shows wrong or missing candles, switch to
nullCandleStoreto see whether the cache is the cause.
isCandleCacheEnabled() reads the page's URL: ?cache=0 turns the cache off for that page load. The SciTrader app chooses its store this way:
import { idbPriceStore, isCandleCacheEnabled, nullCandleStore } from "scichart-scitrader";
const store = isCandleCacheEnabled() ? idbPriceStore : nullCandleStore; // ?cache=0 turns it off
defaultExchangeProvider.registerExchange(CoinbaseExchange, store);
Writing Your Own Candle Store: ICandleStore
A store is any object that implements ICandleStore, so you can cache candles anywhere, e.g. on your server, so users share a cache, or in memory only. Every call says which exchange, symbol and interval it is for; times are in milliseconds.
| ICandleStore member | Required | What it does |
|---|---|---|
get | Yes | The stored candles with start , oldest first |
insert | Yes | Stores candles, merging with any already stored |
get, set | Yes | The listing-date floor: where the exchange's history starts |
get, set | No | Time ranges with no candles, so they aren't asked for again |
subscribe | No | Lets the in-memory cache know when stored candles are deleted |
The example wraps idbPriceStore in a store that logs every call:
// A candle store that logs what the SDK asks of it, and passes each call to idbPriceStore, the
// built-in IndexedDB store. Every ICandleStore call says which exchange, symbol and interval it is for
class LoggingCandleStore implements ICandleStore {
constructor(private readonly log: (line: string) => void) {}
// The cache asks the store first: candles it has for the requested range
async getCandles(exchange: string, symbol: string, interval: CandleInterval, start?: number, end?: number) {
const candles = await idbPriceStore.getCandles(exchange, symbol, interval, start, end);
this.log(`read ${symbol} ${interval}: ${candles.length} cached candles`);
return candles;
}
// Then it fetches what was missing from the exchange, and stores it here
async insertCandles(exchange: string, symbol: string, interval: CandleInterval, candles: ExchangeRestCandle[]) {
this.log(`write ${symbol} ${interval}: ${candles.length} candles from ${exchange}`);
return idbPriceStore.insertCandles(exchange, symbol, interval, candles);
}
// Where the exchange's history starts, once found, so it isn't asked for again
getEarliestCandleTime(exchange: string, symbol: string, interval: CandleInterval) {
return idbPriceStore.getEarliestCandleTime(exchange, symbol, interval);
}
setEarliestCandleTime(exchange: string, symbol: string, interval: CandleInterval, time: number) {
this.log(`floor ${symbol} ${interval}: history starts ${new Date(time).toISOString().slice(0, 10)}`);
return idbPriceStore.setEarliestCandleTime(exchange, symbol, interval, time);
}
// Optional: time spans with no candles, so they aren't asked for again
getKnownEmptyRanges(exchange: string, symbol: string, interval: CandleInterval) {
return idbPriceStore.getKnownEmptyRanges(exchange, symbol, interval);
}
setKnownEmptyRanges(exchange: string, symbol: string, interval: CandleInterval, ranges: Array<{ start: number; end: number }>) {
return idbPriceStore.setKnownEmptyRanges(exchange, symbol, interval, ranges);
}
// Optional: tells the cache when old candles are deleted to keep the store's size down
subscribeToEvictions(callback: () => void) {
return idbPriceStore.subscribeToEvictions(callback);
}
}
TypeScript Example: Watching the Candle Cache
BTC-USD from Coinbase, registered with the logging store, in a provider of the chart's own. The log on the right shows each store call. Switch to 6h and back to 1h: the second time, the 1h candles come from the store, with no request. Clear the cache calls idbPriceStore.clearAll(). Reload the page, and the candles come from IndexedDB.
The code for this example is found below.
- TS
import "scichart-scitrader/scitrader.css";
import {
CandleInterval,
CoinbaseExchange,
createExchangeProvider,
DataFeedManager,
ExchangeRestCandle,
ICandleStore,
idbPriceStore,
OHLCStudy,
SciFinanceChartPanel,
VolumeStudy,
} from "scichart-scitrader";
// This example expects this HTML:
// <div style="display: flex; flex-direction: column; height: 100%">
// <div id="toolbar"></div>
// <div style="display: flex; flex: 1; min-height: 0">
// <div id="chart" style="flex: 1; min-width: 0"></div>
// <pre id="store-log" style="width: 340px"></pre>
// </div>
// </div>
// A candle store that logs what the SDK asks of it, and passes each call to idbPriceStore, the
// built-in IndexedDB store. Every ICandleStore call says which exchange, symbol and interval it is for
class LoggingCandleStore implements ICandleStore {
constructor(private readonly log: (line: string) => void) {}
// The cache asks the store first: candles it has for the requested range
async getCandles(exchange: string, symbol: string, interval: CandleInterval, start?: number, end?: number) {
const candles = await idbPriceStore.getCandles(exchange, symbol, interval, start, end);
this.log(`read ${symbol} ${interval}: ${candles.length} cached candles`);
return candles;
}
// Then it fetches what was missing from the exchange, and stores it here
async insertCandles(exchange: string, symbol: string, interval: CandleInterval, candles: ExchangeRestCandle[]) {
this.log(`write ${symbol} ${interval}: ${candles.length} candles from ${exchange}`);
return idbPriceStore.insertCandles(exchange, symbol, interval, candles);
}
// Where the exchange's history starts, once found, so it isn't asked for again
getEarliestCandleTime(exchange: string, symbol: string, interval: CandleInterval) {
return idbPriceStore.getEarliestCandleTime(exchange, symbol, interval);
}
setEarliestCandleTime(exchange: string, symbol: string, interval: CandleInterval, time: number) {
this.log(`floor ${symbol} ${interval}: history starts ${new Date(time).toISOString().slice(0, 10)}`);
return idbPriceStore.setEarliestCandleTime(exchange, symbol, interval, time);
}
// Optional: time spans with no candles, so they aren't asked for again
getKnownEmptyRanges(exchange: string, symbol: string, interval: CandleInterval) {
return idbPriceStore.getKnownEmptyRanges(exchange, symbol, interval);
}
setKnownEmptyRanges(exchange: string, symbol: string, interval: CandleInterval, ranges: Array<{ start: number; end: number }>) {
return idbPriceStore.setKnownEmptyRanges(exchange, symbol, interval, ranges);
}
// Optional: tells the cache when old candles are deleted to keep the store's size down
subscribeToEvictions(callback: () => void) {
return idbPriceStore.subscribeToEvictions(callback);
}
}
async function createChart() {
const toolbar = document.getElementById("toolbar");
Object.assign(toolbar.style, toolbarStyle);
const logPanel = document.getElementById("store-log");
Object.assign(logPanel.style, logStyle);
const log = (line: string) => (logPanel.textContent = `${new Date().toLocaleTimeString()} ${line}\n` + logPanel.textContent);
// Register Coinbase with the logging store, in a provider of this chart's own
const provider = createExchangeProvider();
provider.registerExchange(CoinbaseExchange, new LoggingCandleStore(log));
const financeChartPanel = await SciFinanceChartPanel.create(document.getElementById("chart"), {
exchangeName: CoinbaseExchange.name,
symbol: "BTC-USD",
interval: CandleInterval.ONE_HOUR,
period: "1W",
services: { dataFeedManager: new DataFeedManager(provider) }, // the chart's data comes from `provider`
studies: [new OHLCStudy(), new VolumeStudy()],
});
const chart = financeChartPanel.financeApi;
// Change interval and back: the second time, the candles come from the store.
// Reload the page: they come from IndexedDB
toolbar.append(
buttonGroup(CoinbaseExchange.getSupportedIntervals(), (i) => i, CandleInterval.ONE_HOUR, (interval) =>
chart.changeInterval(interval)
),
// clearAll deletes every candle in IndexedDB. The next load fetches everything again
buttonGroup(["Clear the cache"], (s) => s, "", async () => {
await idbPriceStore.clearAll();
log("cleared IndexedDB");
})
);
// When the page goes away: financeChartPanel.cleanup();
return financeChartPanel;
}
// A row of toggle buttons: one per value, the selected one highlighted
function buttonGroup<T>(values: readonly T[], label: (v: T) => string, selected: T, onSelect: (v: T) => void) {
const group = document.createElement("div");
Object.assign(group.style, { display: "flex", gap: "4px" });
const render = (current: T) =>
group.replaceChildren(
...values.map((value) => {
const b = document.createElement("button");
b.textContent = label(value);
Object.assign(b.style, buttonStyle(value === current));
b.onclick = () => {
render(value);
onSelect(value);
};
return b;
})
);
render(selected);
return group;
}
// The toolbar uses SciTrader's UI colours: its --st-ui-* CSS variables, with the stylesheet's defaults
const toolbarStyle: Partial<CSSStyleDeclaration> = {
display: "flex",
flexWrap: "wrap",
alignItems: "center",
gap: "8px 16px",
padding: "8px",
background: "var(--st-ui-background, #2b2839)",
borderBottom: "1px solid var(--st-ui-divider, rgba(255,255,255,0.1))",
color: "var(--st-ui-text-muted, #9da2bc)",
fontSize: "13px",
};
const buttonStyle = (active = false): Partial<CSSStyleDeclaration> => ({
padding: "4px 10px",
border: "1px solid var(--st-ui-divider, rgba(255,255,255,0.1))",
borderRadius: "6px",
background: active ? "var(--st-ui-selection, #6c63ff)" : "var(--st-ui-surface, #373447)",
color: "var(--st-ui-text, #ffffff)",
font: "inherit",
fontSize: "13px",
cursor: "pointer",
});
const logStyle: Partial<CSSStyleDeclaration> = {
margin: "0",
padding: "8px 12px",
overflowY: "auto",
background: "var(--st-ui-background, #2b2839)",
borderLeft: "1px solid var(--st-ui-divider, rgba(255,255,255,0.1))",
color: "var(--st-ui-text-muted, #9da2bc)",
font: "12px/1.5 monospace",
whiteSpace: "pre-wrap",
};
createChart();
React Example: Watching the Candle Cache
The same chart in React. The log is React state.
The code for this example is found below.
- React
import "scichart-scitrader/scitrader.css";
import * as React from "react";
import {
CandleInterval,
CoinbaseExchange,
createExchangeProvider,
DataFeedManager,
ExchangeRestCandle,
ICandleStore,
idbPriceStore,
ISciFinanceChartPanelApi,
ISciFinanceChartPanelConfig,
OHLCStudy,
VolumeStudy,
} from "scichart-scitrader";
import { SciFinanceChartView } from "scichart-scitrader-react";
// A candle store that logs what the SDK asks of it, and passes each call to idbPriceStore, the
// built-in IndexedDB store. Every ICandleStore call says which exchange, symbol and interval it is for
class LoggingCandleStore implements ICandleStore {
constructor(private readonly log: (line: string) => void) {}
// The cache asks the store first: candles it has for the requested range
async getCandles(exchange: string, symbol: string, interval: CandleInterval, start?: number, end?: number) {
const candles = await idbPriceStore.getCandles(exchange, symbol, interval, start, end);
this.log(`read ${symbol} ${interval}: ${candles.length} cached candles`);
return candles;
}
// Then it fetches what was missing from the exchange, and stores it here
async insertCandles(exchange: string, symbol: string, interval: CandleInterval, candles: ExchangeRestCandle[]) {
this.log(`write ${symbol} ${interval}: ${candles.length} candles from ${exchange}`);
return idbPriceStore.insertCandles(exchange, symbol, interval, candles);
}
// Where the exchange's history starts, once found, so it isn't asked for again
getEarliestCandleTime(exchange: string, symbol: string, interval: CandleInterval) {
return idbPriceStore.getEarliestCandleTime(exchange, symbol, interval);
}
setEarliestCandleTime(exchange: string, symbol: string, interval: CandleInterval, time: number) {
this.log(`floor ${symbol} ${interval}: history starts ${new Date(time).toISOString().slice(0, 10)}`);
return idbPriceStore.setEarliestCandleTime(exchange, symbol, interval, time);
}
// Optional: time spans with no candles, so they aren't asked for again
getKnownEmptyRanges(exchange: string, symbol: string, interval: CandleInterval) {
return idbPriceStore.getKnownEmptyRanges(exchange, symbol, interval);
}
setKnownEmptyRanges(exchange: string, symbol: string, interval: CandleInterval, ranges: Array<{ start: number; end: number }>) {
return idbPriceStore.setKnownEmptyRanges(exchange, symbol, interval, ranges);
}
// Optional: tells the cache when old candles are deleted to keep the store's size down
subscribeToEvictions(callback: () => void) {
return idbPriceStore.subscribeToEvictions(callback);
}
}
const INTERVALS = CoinbaseExchange.getSupportedIntervals();
export function CandleStoreChart() {
const chartRef = React.useRef<ISciFinanceChartPanelApi>(null);
const [lines, setLines] = React.useState<string[]>([]);
const [interval, setInterval] = React.useState(CandleInterval.ONE_HOUR);
const log = (line: string) => setLines((previous) => [`${new Date().toLocaleTimeString()} ${line}`, ...previous]);
// Register Coinbase with the logging store, in a provider of this chart's own. Create the config
// once per mounted chart: studies are live objects bound to one chart
const config = React.useMemo<ISciFinanceChartPanelConfig>(() => {
const provider = createExchangeProvider();
provider.registerExchange(CoinbaseExchange, new LoggingCandleStore(log));
return {
exchangeName: CoinbaseExchange.name,
symbol: "BTC-USD",
interval: CandleInterval.ONE_HOUR,
period: "1W",
services: { dataFeedManager: new DataFeedManager(provider) }, // the chart's data comes from `provider`
studies: [new OHLCStudy(), new VolumeStudy()],
};
}, []);
// Fills its parent, so give the parent a height
return (
<div style={{ display: "flex", flexDirection: "column", height: "100%" }}>
<div style={toolbarStyle}>
{/* Change interval and back: the second time, the candles come from the store */}
<ButtonGroup
values={INTERVALS}
selected={interval}
onSelect={(i) => {
setInterval(i);
if (chartRef.current?.isInitialized) chartRef.current.financeApi.changeInterval(i);
}}
/>
{/* clearAll deletes every candle in IndexedDB. The next load fetches everything again */}
<button style={buttonStyle()} onClick={() => idbPriceStore.clearAll().then(() => log("cleared IndexedDB"))}>
Clear the cache
</button>
</div>
<div style={{ display: "flex", flex: 1, minHeight: 0 }}>
<div style={{ flex: 1, minWidth: 0 }}>
<SciFinanceChartView ref={chartRef} config={config} />
</div>
<pre style={logStyle}>{lines.join("\n")}</pre>
</div>
</div>
);
}
// A row of toggle buttons: one per value, the selected one highlighted
function ButtonGroup<T extends string>(props: { values: readonly T[]; selected: T; onSelect: (v: T) => void }) {
return (
<div style={{ display: "flex", gap: 4 }}>
{props.values.map((value) => (
<button key={value} style={buttonStyle(value === props.selected)} onClick={() => props.onSelect(value)}>
{value}
</button>
))}
</div>
);
}
// The toolbar uses SciTrader's UI colours: its --st-ui-* CSS variables, with the stylesheet's defaults
const toolbarStyle: React.CSSProperties = {
display: "flex",
flexWrap: "wrap",
alignItems: "center",
gap: "8px 16px",
padding: 8,
background: "var(--st-ui-background, #2b2839)",
borderBottom: "1px solid var(--st-ui-divider, rgba(255,255,255,0.1))",
color: "var(--st-ui-text-muted, #9da2bc)",
fontSize: 13,
};
const buttonStyle = (active = false): React.CSSProperties => ({
padding: "4px 10px",
border: "1px solid var(--st-ui-divider, rgba(255,255,255,0.1))",
borderRadius: 6,
background: active ? "var(--st-ui-selection, #6c63ff)" : "var(--st-ui-surface, #373447)",
color: "var(--st-ui-text, #ffffff)",
font: "inherit",
fontSize: 13,
cursor: "pointer",
});
const logStyle: React.CSSProperties = {
width: 340,
margin: 0,
padding: "8px 12px",
overflowY: "auto",
background: "var(--st-ui-background, #2b2839)",
borderLeft: "1px solid var(--st-ui-divider, rgba(255,255,255,0.1))",
color: "var(--st-ui-text-muted, #9da2bc)",
font: "12px/1.5 monospace",
whiteSpace: "pre-wrap",
};
The ICandleStore interface, with its TSDoc
/**
* A local candle cache, addressed by `(exchange, symbol, interval)`.
*
* ### Why `exchange` is part of every key
*
* One store can be shared by every exchange registered with it (see `registerExchange`).
* With the exchange in every key, two exchanges that use the same symbol string never share
* candles.
*/
interface ICandleStore {
getCandles(exchange: string, symbol: string, interval: CandleInterval, start?: number, end?: number): Promise<ExchangeRestCandle[]>;
insertCandles(exchange: string, symbol: string, interval: CandleInterval, prices: ExchangeRestCandle[]): Promise<void>;
/**
* Open time (ms) of the oldest candle the remote exchange has ever returned
* for this (exchange, symbol, interval), or `undefined` if no listing-date floor has
* been observed yet. Persisted across sessions so subsequent cold loads
* that target dates below the floor can short-circuit without a wasted
* REST round-trip.
*/
getEarliestCandleTime(exchange: string, symbol: string, interval: CandleInterval): Promise<number | undefined>;
/**
* Persist a discovered listing-date floor. Called by the cache when
* a remote fetch returns fewer candles than requested — the partial result
* implies the remote has no data older than its earliest returned candle.
*/
setEarliestCandleTime(exchange: string, symbol: string, interval: CandleInterval, time: number): Promise<void>;
/**
* Time spans the remote has confirmed it has no candles for — halts, outages, buckets in
* which the instrument simply did not trade.
*
* Without them, a cache that stores only candles can't tell a range it never fetched from
* a range with no candles, and asks the exchange again for every gap.
*
* Optional: a store that omits these works, at the cost of those extra requests.
*/
getKnownEmptyRanges?(exchange: string, symbol: string, interval: CandleInterval): Promise<Array<{
start: number;
end: number;
}>>;
setKnownEmptyRanges?(exchange: string, symbol: string, interval: CandleInterval, ranges: Array<{
start: number;
end: number;
}>): Promise<void>;
/**
* Register a callback fired after the store deletes candles, e.g. to
* keep its size down. Optional: stores that never delete candles omit it.
*/
subscribeToEvictions?(callback: () => void): () => void;
}
The IdbCandleStore class, with its TSDoc
declare class IdbCandleStore implements ICandleStore {
/** How often old candles are deleted, in ms. Default: 24 hours. */
cleanInterval: number;
/**
* Register a callback to fire after each successful `cleanCacheAll`.
* Returns an unsubscribe function. The callback is invoked even when
* no candles were deleted, so keep it cheap.
*/
subscribeToEvictions(cb: TEvictionSubscriber): () => void;
/**
* The candle cap for an interval with no default cap of its own. The
* default caps: 50,000 candles from 1m to 30m, unlimited from 1h.
*/
maxCandles: number;
minDensity: number;
constructor();
/**
* Returns the active candle cap for `interval`: the override set with
* {@link setIntervalCap} if there is one, otherwise the default.
*/
getIntervalCap(interval: CandleInterval): number;
/**
* Set a per-interval cap override and persist it. Pass a non-negative
* finite number (use `Number.POSITIVE_INFINITY` for "no cap"), or call
* {@link resetIntervalCaps} to drop all overrides.
*/
setIntervalCap(interval: CandleInterval, cap: number): void;
/** Drop every per-interval cap override; future caps fall back to defaults. */
resetIntervalCaps(): void;
/**
* Wipe the entire IndexedDB database. The next load fetches candles
* from the exchange again.
*
* Fires the eviction subscribers, so the in-memory cache drops its
* candles too.
*
* Interval-cap overrides survive — they're a UI preference, not
* cached data.
*/
clearAll(): Promise<void>;
/**
* Writes candles as binary chunks, merging into whatever each chunk already holds.
*
* Read-modify-write per touched chunk rather than a blind put, because a fetch almost never
* lines up with a chunk boundary: appending the live bar or back-filling a gap lands inside
* a chunk that already has candles, and a blind put would drop them.
*/
insertCandles(exchange: string, symbol: string, interval: CandleInterval, candles: ExchangeRestCandle[]): Promise<void>;
getEarliestCandleTime(exchange: string, symbol: string, interval: CandleInterval): Promise<number | undefined>;
setEarliestCandleTime(exchange: string, symbol: string, interval: CandleInterval, time: number): Promise<void>;
/**
* Known-empty spans for this series, or `[]` when none have been recorded.
*/
getKnownEmptyRanges(exchange: string, symbol: string, interval: CandleInterval): Promise<Array<{
start: number;
end: number;
}>>;
setKnownEmptyRanges(exchange: string, symbol: string, interval: CandleInterval, ranges: Array<{
start: number;
end: number;
}>): Promise<void>;
getCandles(exchange: string, symbol: string, interval: CandleInterval, start?: number, end?: number): Promise<ExchangeRestCandle[]>;
/** Delete candles beyond each interval's cap now, rather than waiting for `cleanInterval`. */
cleanCacheAll(): Promise<void>;
}
The NullCandleStore class, isCandleCacheEnabled, idbPriceStore and nullCandleStore, with their TSDoc
declare class NullCandleStore implements ICandleStore {
/** Always a miss, so `DataCacheLayer` treats the whole requested range as a gap. */
getCandles(): Promise<ExchangeRestCandle[]>;
/** Discards. */
insertCandles(): Promise<void>;
/**
* Always `undefined` — with nothing persisted there is no observed listing-date floor, so
* the cache layer probes the remote exactly as it would on a cold first-ever load.
*/
getEarliestCandleTime(): Promise<number | undefined>;
/** Discards. */
setEarliestCandleTime(): Promise<void>;
}
/**
* Whether the IndexedDB candle cache should be used.
*
* `?cache=0` (or `?cache=off`) disables it for that page load only — nothing is persisted, so
* removing the parameter restores normal behaviour with the previous cache contents intact.
*/
function isCandleCacheEnabled(search?: string): boolean;
const idbPriceStore: IdbCandleStore;
const nullCandleStore: NullCandleStore;
Where to read next
- Building a Custom Exchange: the exchange the store caches for.
- Data Manager and Data Feeds: what the chart does with the candles.
- Data Sources: registering exchanges.