Custom Data Sources

Custom data sources let you integrate any external data into your strategies — from DeFi protocols to sentiment feeds, on-chain metrics, or proprietary signals.

Overview

Custom data sources are code-first definitions that:

Fetch data from external APIs
Transform responses into a standardized format
Run in a secure sandboxed environment
Support backtesting and live trading

Key feature: Network domains and secrets are auto-detected from your code — no need to declare them separately!

Definition Structure

A custom data source is an IIFE (Immediately Invoked Function Expression) that returns a definition object:

(() => {
  const name = "My Data Source";
  const description = "What this data source provides";

  // Parameters define the URI structure
  const params = {
    symbol: "symbol",      // Built-in: BTC-USD format
    timeframe: "timeframe" // Built-in: 1m, 5m, 1h, 4h, 1D, 1W
  };

  const capabilities = { historical: true };

  // ⚠️ Helper functions MUST be defined inside fetchData
  async function fetchData({ symbol, timeframe, startDate, endDate, secrets }) {
    // Convert symbol format (BTC-USD → exchange format)
    function convertSymbol(sym) {
      return sym.replace("-", "").replace("USD", "USDT");
    }
    
    const exchangeSymbol = convertSymbol(symbol);
    
    // Domain is auto-detected from URL
    const res = await fetch(`https://api.example.com/data?symbol=${exchangeSymbol}`);
    if (!res.ok) throw new Error(`API error: ${res.status}`);
    
    const json = await res.json();
    return json.map(d => ({
      timestamp: new Date(d.time),
      value: d.value,
    }));
  }

  // Optional: Pre-built analysis expressions (flat key:value format)
  const analysis = {
    value: "value",           // numeric output
    high: "value > 70"        // boolean output (signal)
  };

  return { name, description, params, capabilities, fetch: fetchData, analysis };
})()

⚠️Important

Helper functions must be defined inside fetchData, not at the IIFE's top level. Only fetchData is extracted and executed at runtime.

Fetch Function Inputs

The fetchData function receives:

Property	Type	Description
`symbol`	`string`	Symbol in `BASE-QUOTE` format (e.g., `BTC-USD`)
`timeframe`	`string`	Timeframe (`1m`, `5m`, `1h`, `4h`, `1D`, `1W`)
`startDate`	`Date`	Start of requested date range
`endDate`	`Date`	End of requested date range
`secrets`	`Record<string, string>`	User-configured API keys (auto-detected)
your params	varies	Any custom parameters you define

Symbol Format Conversion

The symbol parameter always comes in BASE-QUOTE format (e.g., BTC-USD). You must convert it to the exchange's expected format:

// Inside fetchData:
function convertSymbol(sym) {
  // BTC-USD → BTCUSDT (Binance)
  return sym.replace("-", "").replace("USD", "USDT");
}

// Or for other formats:
// BTC-USD → BTC/USDT
// BTC-USD → btc
// BTC-USD → bitcoin (requires mapping table)

Fetch Function Output

Return an array of row objects with a timestamp column:

return [
  { timestamp: new Date("2025-01-01"), value: 52, label: "Neutral" },
  { timestamp: new Date("2025-01-02"), value: 48, label: "Fear" },
];

Column Types

Custom data sources can return these column types:

Type	Description	Example
`datetime`	Timestamp (required as `timestamp`)	`new Date('2024-01-01')`
`number`	Numeric values	`100.5`, `42`
`string`	Text values	`'bullish'`, `'high'`
`boolean`	True/false	`true`, `false`

Parameter Types

Parameters define the URI structure. Use shorthand for built-in types or objects for custom types:

const params = {
  // Built-in types (shorthand)
  symbol: "symbol",        // Platform symbol selector (BTC-USD, ETH-USD)
  timeframe: "timeframe",  // Timeframe selector (1m, 5m, 1h, 4h, 1D, 1W)
  
  // Custom types (object format)
  wallet: { type: "text" },
  network: { type: "text", default: "mainnet" },
  limit: { type: "number" },
  period: { type: "number", default: 14 },
  include_history: { type: "boolean" },
  aggregate: { type: "boolean", default: true },
  chain: { type: "list", options: ["ethereum", "solana", "arbitrum"] },
};

Type	Description
`"symbol"`	Platform symbol selector (BTC-USD format)
`"timeframe"`	Timeframe selector (1m, 5m, 15m, 1h, 4h, 1D, 1W)
`{ type: "text" }`	Free-form text input
`{ type: "number" }`	Numeric input
`{ type: "boolean" }`	Toggle switch
`{ type: "list", options: [...] }`	Dropdown with fixed options

📝Note

Order matters: Parameters appear in the URI in the order defined.

Auto-Detection (No Manual Config Needed!)

Network Domains

Network domains are automatically extracted from fetch() URLs in your code:

// This URL's domain (api.example.com) is auto-detected
const res = await fetch(`https://api.example.com/data?symbol=${symbol}`);

Secrets

Secrets are automatically extracted from secrets.XXX patterns:

// API_KEY is auto-detected and users will be prompted to configure it
const apiKey = secrets.API_KEY;

const res = await fetch(`https://api.example.com/data`, {
  headers: { 'Authorization': `Bearer ${apiKey}` }
});

Users configure secret values in Settings → Credentials.

Available Global Functions

Function	Description
`fetch(url, options)`	Make HTTP requests (same as browser fetch)
`sleep(ms)`	Pause execution for ms milliseconds (max 10 seconds)
`console.log/warn/error`	Log messages (shown in the Test tab)

Rate Limiting with sleep()

Use sleep() between API calls to avoid rate limiting:

async function fetchData({ startDate, endDate }) {
  const allData = [];
  let cursor = startDate.getTime();
  
  while (cursor < endDate.getTime()) {
    // Rate limit: wait 300ms between requests
    if (allData.length > 0) await sleep(300);
    
    const res = await fetch(`https://api.example.com/data?after=${cursor}&limit=1000`);
    const batch = await res.json();
    
    if (batch.length === 0) break;
    allData.push(...batch);
    
    cursor = batch[batch.length - 1].timestamp + 1;
  }
  
  return allData.map(d => ({
    timestamp: new Date(d.timestamp),
    value: d.value,
  }));
}

Console Logging

Use console.log(), console.warn(), etc. for debugging:

async function fetchData({ symbol }) {
  console.log(`Fetching data for ${symbol}`);
  // ...
  console.warn('API returned partial data');
}

Logs are captured and shown in the Test tab.

Event-Based vs OHLCV Data

OHLCV Data (With Timeframe)

If your URI includes a timeframe parameter, it's treated as regular OHLCV data:

custom://myexchange/?symbol=BTC-USD&timeframe=4h

Can be used as execution data source
Standard multi-timeframe alignment applies
Suitable for: exchange data, custom aggregations

Event-Based Data (No Timeframe)

If your URI has no timeframe, it's treated as event-based data:

custom://wallet_tracker/?address=0x123...

Timestamps are normalized to execution timeframe boundaries
Events align to the next bar (to avoid look-ahead bias)
Cannot be used as execution data source
Suitable for: wallet trackers, on-chain events, signals

Handling Large Date Ranges (Pagination)

🚨Important for OHLCV Data

When backtesting, the system may request months or years of data. Most APIs limit results (e.g., 1000 candles per request). Your fetch function must implement pagination internally.

async function fetchData({ symbol, timeframe, startDate, endDate, secrets }) {
  // Helper function INSIDE fetchData
  function convertSymbol(sym) {
    return sym.replace("-", "").replace("USD", "USDT");
  }
  
  const allCandles = [];
  let cursor = startDate.getTime();
  const endMs = endDate.getTime();
  const limit = 1000;
  
  while (cursor < endMs) {
    // Rate limit between requests
    if (allCandles.length > 0) await sleep(300);
    
    const url = `https://api.exchange.com/klines?symbol=${convertSymbol(symbol)}&startTime=${cursor}&limit=${limit}`;
    const res = await fetch(url, {
      headers: { 'X-API-Key': secrets.API_KEY }
    });
    
    if (!res.ok) throw new Error(`API error: ${res.status}`);
    const batch = await res.json();
    
    if (batch.length === 0) break;
    allCandles.push(...batch);
    
    // Move cursor past last candle
    cursor = batch[batch.length - 1][0] + 1;
    
    // Safety limit
    if (allCandles.length > 500000) break;
  }
  
  return allCandles.map(c => ({
    timestamp: new Date(c[0]),
    open: Number(c[1]),
    high: Number(c[2]),
    low: Number(c[3]),
    close: Number(c[4]),
    volume: Number(c[5]),
  }));
}

Examples

Fear & Greed Index

(() => {
  const name = "Fear & Greed Index";
  const description = "Crypto market sentiment index (0–100)";
  const params = { symbol: "symbol", timeframe: "timeframe" };
  const capabilities = { historical: true };

  // Domain (api.alternative.me) auto-detected from fetch URL
  async function fetchData({ startDate, endDate }) {
    const days = Math.ceil((endDate.getTime() - startDate.getTime()) / (24 * 60 * 60 * 1000));
    const limit = Math.min(Math.max(days, 30), 365);

    const res = await fetch(`https://api.alternative.me/fng/?limit=${limit}&format=json`);
    if (!res.ok) throw new Error(`API error (${res.status})`);

    const json = await res.json();
    if (!Array.isArray(json?.data)) throw new Error("Unexpected response format");

    return json.data.map(d => ({
      timestamp: new Date(Number(d.timestamp) * 1000),
      value: Number(d.value),
      label: String(d.value_classification ?? ""),
    }));
  }

  const analysis = {
    fng: "value",
    extreme_fear: "value < 25",
    extreme_greed: "value >= 75"
  };

  return { name, description, params, capabilities, fetch: fetchData, analysis };
})()

Usage in strategy:

extreme_fear: fng.value < 25
extreme_greed: fng.value > 75
greed_zone: fng.label == "Greed"

DeFiLlama TVL

(() => {
  const name = "DeFiLlama TVL";
  const description = "Total Value Locked from DeFiLlama API";
  const params = {
    protocol: { type: "text" }  // Protocol slug (e.g., "aave", "uniswap")
  };
  const capabilities = { historical: true };

  async function fetchData({ protocol, startDate, endDate }) {
    const res = await fetch(`https://api.llama.fi/protocol/${protocol}`);
    if (!res.ok) throw new Error(`API error: ${res.status}`);
    
    const data = await res.json();
    
    return (data.tvl || [])
      .filter(d => {
        const date = new Date(d.date * 1000);
        return date >= startDate && date <= endDate;
      })
      .map(d => ({
        timestamp: new Date(d.date * 1000),
        tvl: d.totalLiquidityUSD,
      }));
  }

  return { name, description, params, capabilities, fetch: fetchData };
})()

Exchange OHLCV (with Pagination)

(() => {
  const name = "OKX Perpetual Futures";
  const description = "OHLCV data from OKX";
  const params = { symbol: "symbol", timeframe: "timeframe" };
  const capabilities = { historical: true };

  async function fetchData({ symbol, timeframe, startDate, endDate }) {
    // Helper functions INSIDE fetchData
    const tfMapping = { "1h": "1H", "4h": "4H", "1D": "1D" };
    function convertSymbol(sym) {
      // BTC-USD → BTC-USDT-SWAP
      return sym.replace("USD", "USDT") + "-SWAP";
    }
    
    const allCandles = [];
    let cursor = endDate.getTime();
    const startMs = startDate.getTime();
    
    while (cursor > startMs) {
      if (allCandles.length > 0) await sleep(150);
      
      const url = `https://www.okx.com/api/v5/market/history-candles?instId=${convertSymbol(symbol)}&bar=${tfMapping[timeframe]}&after=${cursor}&limit=100`;
      const res = await fetch(url);
      const json = await res.json();
      
      if (json.code !== "0" || !json.data?.length) break;
      
      for (const c of json.data) {
        const ts = parseInt(c[0]);
        if (ts >= startMs) {
          allCandles.push({
            timestamp: new Date(ts),
            open: parseFloat(c[1]),
            high: parseFloat(c[2]),
            low: parseFloat(c[3]),
            close: parseFloat(c[4]),
            volume: parseFloat(c[5])
          });
        }
      }
      cursor = parseInt(json.data[json.data.length - 1][0]);
    }
    
    return allCandles.sort((a, b) => a.timestamp - b.timestamp);
  }

  return { name, description, params, capabilities, fetch: fetchData };
})()

Wallet Tracker (Event-Based)

(() => {
  const name = "Wallet Tracker";
  const description = "Track wallet positions for copy trading";
  const params = {
    address: { type: "text" },
    symbol: { type: "text" }
  };
  const capabilities = { historical: true };

  async function fetchData({ address, symbol, startDate, endDate }) {
    const res = await fetch('https://api.hyperliquid.xyz/info', {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify({
        type: 'userFillsByTime',
        user: address,
        startTime: startDate.getTime(),
        endTime: endDate.getTime()
      })
    });
    
    const fills = await res.json();
    
    return fills
      .filter(f => f.coin === symbol)
      .map(f => ({
        timestamp: new Date(f.time),
        action: f.side === 'B' ? 'enter_long' : 'enter_short',
        size: parseFloat(f.sz),
        price: parseFloat(f.px),
      }));
  }

  const analysis = {
    copy_long: "action == 'enter_long'",
    copy_short: "action == 'enter_short'"
  };

  return { name, description, params, capabilities, fetch: fetchData, analysis };
})()

Usage in strategy (string comparisons):

copy_long: wallet.action == "enter_long"
copy_short: wallet.action == "enter_short"

Resource Limits

Resource	Limit
Memory	128 MB
Execution time	120 seconds
Sleep duration	10 seconds max per call
Network	Auto-detected domains only

Best Practices

Handle pagination — Most APIs limit results per request
Rate limit requests — Use sleep() between API calls
Filter by date range — Only return data within startDate to endDate
Sort results — Return rows sorted by timestamp ascending
Validate responses — Check for API errors before processing
Use meaningful columns — Name columns descriptively (tvl, sentiment, not value1)

Reducing the date range
Optimizing pagination
Using smaller batch sizes

"Memory limit exceeded"

Your data source is using too much memory. Try:

Processing data incrementally
Returning fewer rows
Reducing response payload size