AceTrader Docs

Appearance

DeduplicationManager ​

Prevents duplicate processing of on-chain subscription events using transaction hash and log index.

Purpose ​

Blockchain event polling can return the same event multiple times due to reorgs, overlapping block ranges, or retries. DeduplicationManager maintains a persistent set of already-processed event hashes so that each event is handled exactly once.

High-Level Design ​

Event Identity ​

Each on-chain log entry is uniquely identified by the composite key {transactionHash}-{logIndex}. This is deterministic — the same event always produces the same hash regardless of when or how many times it's fetched.

Data Flow ​

RpcManager fetches events
       │
       ▼
┌──────────────────────┐
│ isEventProcessed()   │──── already seen? ──→ skip
│ (SELECT by hash)     │
└──────────────────────┘
       │ new event
       ▼
EventHandlerManager processes event
       │
       ▼
┌──────────────────────┐
│ markEventProcessed() │
│ (INSERT OR IGNORE)   │
└──────────────────────┘

The watcher calls isEventProcessed() as a pre-filter before handing events to EventHandlerManager. Only after successful processing does it call markEventProcessed(). This ensures failed events are retried on the next alarm cycle.

Bounded Storage via Block-Based Cleanup ​

The processed_events table grows with every new event. To prevent unbounded growth, cleanupOldEvents() runs during each alarm cycle and removes records older than 10,000 blocks behind the current cursor per chain.

The 10,000-block retention window is deliberately large — well beyond typical reorg depths on both Arbitrum and Ethereum — so cleanup never risks re-processing a recent event.

Cleanup relies on CursorStateManager.getWatcherState() to determine the current block cursor. If a chain has no cursor state yet, that chain is silently skipped.

Idempotency ​

INSERT OR IGNORE on the primary key (event_hash) makes markEventProcessed() safe to call multiple times for the same event with no side effects. This is important because the alarm-based retry loop may re-encounter events that were partially processed before a failure.

See Also ​