PriceCollector

Geo-located Durable Object that polls Hyperliquid every second, stores prices in 10-second binary shards, backfills gaps from peer locations, and generates OHLC candles.

Architecture

One instance per region (wnam, weur, apac, etc.), created via geolocation routing with Cloudflare location hints. Each instance independently collects prices and serves queries for its region. Cross-location backfill ensures data completeness when a region misses updates.

Hyperliquid API
      │
      ▼ (1s poll)
HyperliquidApiManager ──emit "allMids"──▶ PriceStorageManager
                                              │
                                   ┌──────────┼──────────┐
                                   ▼          ▼          ▼
                              In-Memory    SQLite     Callback
                               Cache     (10s shards)    │
                                              │          ▼
                                              │   CandleAggregationManager
                                              │     (1m / 1h candles)
                                              ▼
                                      BackfillManager
                                    (gap detection + fill)
                                         │         │
                                    Peer DOs    Pyth Network
                                   (priority)   (fallback)

Alarm Loop

Runs every 1 second:

1. Run gap detection (cursor scan of recent timestamps)
2. Process backfill queue (10 gaps/batch, non-blocking fire-and-forget)
3. Fetch prices via HyperliquidApiManager.getAllMids() → stored in-memory → complete 10s buckets flushed to SQLite → candle aggregation callback triggered
4. Flush metrics to Analytics Engine (every 60s)

Data Model

Prices are stored as compact binary blobs — one row per 10-second bucket containing all assets. This avoids JSON overhead and allows unchanged asset data to be copied as raw bytes during backfill reconciliation.

Header (12B): magic "PRCE" | assetCount | reserved | dataLength
Index  (~20B/asset): assetId | recordCount | offset | length
Data   (12B/record): timestamp (4B) | scaled int64 price (8B)

SQLite Tables

Table	Purpose
price_data	Binary shards, 1 row per 10s bucket
backfill_gaps	Missing timestamps with status, attempt count, source location
collector_metadata	Single row: location, colo, first alarm time, candle cursors, last cleanup
price_candles_1m	1-minute OHLC candles (30-day retention)
price_candles_1h	1-hour OHLC candles (30-day retention)

Key Constants

Constant	Value
Bucket size	10 seconds
Warm cache on init	6 hours (MAX_CACHE_INTERVAL)
Cache max size	30,000 timestamps
Backfill batch	10 gaps/alarm cycle
Max backfill retries	5 (then permanently failed)
Candle retention	30 days
Valid price threshold	≥13 symbols, all positive
Staleness threshold	3 seconds

Depended By

• PriceAlert — reads price data for alert evaluation and read replica cache
• AccountValueAggregationTracker — reads price ranges for hourly/daily account value calculation
• Admin dashboard — queries health status and candle data