Build a Rolling Drawdown Circuit Breaker for Crypto Bots in Python
Implements a real-time equity-curve monitor in Python that calculates 1-hour and 24-hour rolling drawdown, trips a circuit breaker at configurable thresholds, and sends a PagerDuty alert before flattening all open positions.
A rolling drawdown circuit breaker for crypto bots in Python is one of those components that looks optional until the day your momentum strategy gets caught in a liquidity vacuum at 3 AM and you wake up down 40%. This is not a theoretical exercise — every bot that runs unsupervised in production needs this, and building it correctly requires thinking carefully about window semantics, threshold calibration, and what "flat" actually means on Solana or Hyperliquid.
Why Rolling Windows Beat Peak-to-Trough Drawdown
Classical drawdown measures the decline from the all-time equity peak. That metric is fine for end-of-day risk reports, but it is useless as a real-time trip signal for a bot that can lose 15% in twelve minutes on a perp delist.
Rolling windows solve this by anchoring the reference point to a fixed lookback — typically 1 hour and 24 hours — and recomputing continuously. The 1-hour window catches acute blowups: a bad fill cascade, a fee miscalculation, a sudden oracle divergence. The 24-hour window catches slow bleeds: a strategy that is losing a consistent 0.5% per hour, which is fine for a few samples and catastrophic by end of day.
The key property you want is independent thresholds on each window. A strategy might legitimately drawdown 8% intraday during a high-volatility session but should never lose 3% in any single hour. Your thresholds are not symmetric, and your code should not treat them as if they are.
Data Structure: Timestamped Equity Snapshots
You need a time-indexed equity series. The simplest production-grade structure is a collections.deque bounded by wall-clock time, not by count, with each entry being a (timestamp_utc, equity_usd) tuple.
import time
import collections
from dataclasses import dataclass, field
from typing import Deque, Tuple
@dataclass
class EquityMonitor:
window_1h_sec: int = 3600
window_24h_sec: int = 86400
threshold_1h: float = 0.03 # 3 % in any rolling hour
threshold_24h: float = 0.08 # 8 % in any rolling 24 h
_series: Deque[Tuple[float, float]] = field(
default_factory=lambda: collections.deque(maxlen=100_000)
)
def record(self, equity: float) -> None:
self._series.append((time.time(), equity))
def _rolling_drawdown(self, lookback_sec: int) -> float:
now = time.time()
cutoff = now - lookback_sec
window = [(t, e) for t, e in self._series if t >= cutoff]
if len(window) < 2:
return 0.0
peak = max(e for _, e in window)
current = window[-1][1]
return (peak - current) / peak
def check(self) -> Tuple[bool, str]:
dd_1h = self._rolling_drawdown(self.window_1h_sec)
dd_24h = self._rolling_drawdown(self.window_24h_sec)
if dd_1h >= self.threshold_1h:
return True, f"1h drawdown {dd_1h:.2%} >= {self.threshold_1h:.2%}"
if dd_24h >= self.threshold_24h:
return True, f"24h drawdown {dd_24h:.2%} >= {self.threshold_24h:.2%}"
return False, ""
Keep the maxlen on the deque large enough that you never silently drop samples during a fast-sampling period. At one sample per second, 100,000 entries covers 27 hours — enough for the 24-hour window with headroom.
Threshold Calibration: Numbers That Come From Data
The values 3%/1h and 8%/24h in the example are starting points, not gospel. The right thresholds come from your strategy's historical equity curve. Run your backtest or paper-trading series through this monitor with different threshold pairs and ask: how many legitimate sessions would have been halted? How many actual blowups would have been caught?
A few empirical rules that hold across most market-making and momentum strategies on Hyperliquid and Polymarket:
- Never set the 1-hour threshold above 5%. A legitimate strategy can almost always tolerate a brief 2-3% adverse run, but if you are losing more than 5% in a single hour you are likely in a regime the strategy was not designed for.
- Set the 24-hour threshold at roughly 2-2.5x your expected daily P&L volatility (1 sigma). If your bot historically makes or loses 2% per day (1 sigma), tripping at 4-5% is a reasonable "three-sigma move" proxy.
- Recalibrate every 30 days. Volatility regimes shift, especially on Solana where compute unit fees and slot latency introduce noise that does not show up in price data alone.
The Alert and Flatten Sequence
When check() returns True, the order of operations matters. Getting this wrong — alerting after flattening, or trying to flatten while the alert infrastructure is blocking — is a common production bug.
The correct sequence is:
- Set a local
circuit_openflag atomically before touching any network calls. This prevents the main trading loop from sending new orders even if the alert and flatten steps are slow. - Fire the PagerDuty alert (or whatever your on-call tool is) as a non-blocking background task.
- Cancel all open orders, then close all positions. On Hyperliquid this means sending a
cancelAllfollowed by market orders on every open side. On Solana with a custom program you may need to close PDAs before the position account will drain cleanly. - Log the final equity snapshot and the trip reason to your persistent store before the process exits or enters a halted state.
Do not combine the alert and flatten into a single function that can raise mid-execution. Keep them as separate, independently retriable operations. If the position-flatten fails (network partition, RPC node down), you want to retry the flatten without re-sending the alert.
Integrating With the Trading Loop
The monitor runs as a side-car coroutine if you are on asyncio, or in a thread with a shared threading.Event if you are synchronous. The critical constraint is that circuit_open must be readable by your order-dispatch path with zero blocking.
import asyncio
async def monitor_loop(monitor: EquityMonitor, get_equity_fn, on_trip_fn):
circuit_open = False
while not circuit_open:
equity = await get_equity_fn()
monitor.record(equity)
tripped, reason = monitor.check()
if tripped:
circuit_open = True
await on_trip_fn(reason)
break
await asyncio.sleep(5) # sample every 5 seconds
The 5-second sample interval is deliberately conservative. Sampling faster burns RPC quota and adds noise from mid-candle latency spikes. If you are running at high frequency and need sub-second equity estimates, interpolate from fill events rather than polling the account balance.
Trade-offs and Failure Modes
A circuit breaker that is too sensitive will halt a profitable strategy during normal drawdown. A circuit breaker that is too loose will not fire until significant damage is done. Neither failure mode is acceptable, which is why the risk management services for any automated system should treat the circuit breaker configuration as a first-class deliverable, not an afterthought.
Two failure modes deserve special attention. First, equity calculation drift: if your equity function includes unrealized P&L marked at stale prices, the drawdown signal will lag. Use mark prices from the exchange's own feed, not a secondary source, especially on Hyperliquid where mark and index can diverge during high funding periods. Second, re-entry after a trip: decide upfront whether the circuit can reset automatically (after a cooldown period and equity recovery) or requires a manual restart. Automatic resets are operationally convenient and operationally dangerous. Most production systems require a human to restart the bot after a 24-hour drawdown trip.
If you want a circuit breaker built correctly and calibrated against real strategy data from day one, talk to us — this is the kind of production risk infrastructure we wire into every bot we ship.
Need a bot like this built?
We design, build and run trading bots on Solana, Hyperliquid and Polymarket.
Start a projectMore from the blog
Public vs Paid RPC: How to Run a Fair Benchmark
A fair comparison of public and paid RPC endpoints must account for quotas, workload, freshness and support guarantees.
Read articleRPC Benchmark Percentiles Explained: p50, p95 and p99
Why averages hide the latency spikes that break trading bots, wallets and production blockchain applications.
Read articleHyperliquid REST vs WebSocket Benchmark: What to Measure
A benchmark plan for Hyperliquid market data that separates request latency from streaming freshness and recovery.
Read article