🤖 Kabobagool — Polymarket Market-Neutral Arbitrage

Production-ready, high-frequency bot that captures YES/NO mispricings on Polymarket. Designed for sub-microsecond strategy ticks, fill-or-kill execution, and multi-market orchestration via a supervisor.

---

📊 Strategy Overview

• Binary markets satisfy YES + NO ≈ $1.00; dislocations create risk-free spread.
• Bot buys the cheaper side first, then hedges the opposite side while keeping fee/edge-adjusted projected pair cost under the effective limit: effective_pair_cost(avg_yes, avg_no, TAKER_FEE_BPS) <= (MAX_PAIR_COST - EDGE_BUFFER) after the simulated fill.
• All orders are Fill-Or-Kill (FOK) to avoid resting quotes; DRY_RUN mode simulates fills.
• Hedging: if inventory imbalance exceeds HEDGE_TOLERANCE, force-buy the lighter side.

Example

YES = 0.52, NO = 0.45  → pair_cost = 0.97
Buy 100 NO @ 0.45 → later YES = 0.48
Buy 100 YES @ 0.48 → payout $100, cost $93 → +$7 locked profit

---

🏗️ Architecture

Data Flow (single-market)

WebSocket prices (wss://ws-subscriptions-clob.polymarket.com/ws/market)
┌────────────────────────────────────────────────┐
│                   MarketFeed                   │
└───────────────────────────┬────────────────────┘
                            │ prices dict
                            ▼
┌────────────────────────────────────────────────┐
│               KabobagoolStrategy               │
│          (orchestration + risk limits)         │
└───────────────────────────┬────────────────────┘
                            ▼
┌────────────────────────────────────────────────┐
│     PositionTracker + decision.py (pure logic) │
└───────────────────────────┬────────────────────┘
                            │ validated order
                            ▼
┌────────────────────────────────────────────────┐
│     ExecutionClient (py-clob-client, FOK only) │
└───────────────────────────┬────────────────────┘
                            ▼
┌────────────────────────────────────────────────┐
│           Polymarket CLOB (fills/trades)       │
└────────────────────────────────────────────────┘

Startup reconciliation: recent BUY fills for PROXY_ADDRESS → PositionTracker
Observability (sidecar): Feed/Strategy → RuntimeSnapshot → Health/Metrics (/health, /status, /metrics)

Data Flow (Hydra multi-market)

WebSocket prices (one shared connection)
┌────────────────────────────────────────────────┐
│              MultiplexedMarketFeed             │
│        (fan-out; never awaits strategy)        │
└───────────────────────────┬────────────────────┘
                            │ best-ask updates
                            ▼
┌────────────────────────────────────────────────┐
│   ConflatingPriceBuffer (per market; YES+NO)   │
└───────────────────────────┬────────────────────┘
                            ▼
┌────────────────────────────────────────────────┐
│            MarketEngine (per market task)      │
└───────────────────────────┬────────────────────┘
                            ▼
┌────────────────────────────────────────────────┐
│               KabobagoolStrategy               │
│   (per market) + PositionTracker + decision.py │
└───────────────────────────┬────────────────────┘
                            │ order request
                            ▼
┌────────────────────────────────────────────────┐
│ ExecutionClient (global queue + min interval)  │
│    (wallet-level serialization + FOK only)     │
└───────────────────────────┬────────────────────┘
                            ▼
┌────────────────────────────────────────────────┐
│           Polymarket CLOB (fills/trades)       │
└────────────────────────────────────────────────┘

Startup reconciliation (per engine): recent BUY fills for PROXY_ADDRESS → PositionTracker
Observability (single server): Engines/Feed snapshots → task_health.py (/health, /status, /metrics)

Key Modules (production path)

• config.py — env validation, logging, reconciliation & health toggles.
• feed.py — WebSocket to wss://ws-subscriptions-clob.polymarket.com/ws/market, auto-reconnect, cache reset to 1.0 on disconnect.
• strategy.py — orchestration; asyncio lock guards; caches config for hot path.
• decision.py — pure, deterministic selection/validation functions.
• position.py — inventory & locked-profit math (@dataclass(slots=True) + __slots__).
• client.py — async wrapper over py-clob-client, executes all blocking calls in run_in_executor, always uses OrderType.FOK.
• reconcile.py — replays historical BUY trades (default 72h) so restarts don’t double-buy.
• health.py — FastAPI health/metrics server (Prometheus-friendly gauges and counters).

Supervisor/Discovery (multi-market)

• discovery.py — pulls Gamma markets + order books; filters by glob patterns, volume, and time-to-expiry; normalizes token IDs (decimal strings).
• market_supervisor.py — orchestrates per-market engines, shared feed, unified health/metrics, and Prometheus file_sd targets.
• supervisor_config.py — typed supervisor config loader with profile + local override support.
• multiplex_feed.py / market_engine.py / task_health.py — Hydra shared feed, per-market engines, unified health/metrics.

Project Layout

kabobagool/
├── src/
│   ├── main.py                # Single-market entrypoint
│   ├── config.py              # Env + defaults (fails fast on missing secrets)
│   ├── client.py              # ExecutionClient async wrapper
│   ├── feed.py                # MarketFeed websocket handler
│   ├── strategy.py            # KabobagoolStrategy orchestration
│   ├── decision.py            # Pure strategy logic
│   ├── position.py            # PositionTracker + Side enum
│   ├── reconcile.py           # State restore from historical trades
│   ├── health.py              # /health, /status, /metrics endpoints
│   ├── discovery.py           # Market discovery (Gamma + books)
│   ├── transport.py           # Swappable HTTP layer (aiohttp / curl_cffi TLS impersonation)
│   ├── market_supervisor.py   # Multi-market supervisor entrypoint
│   ├── supervisor_config.py   # Typed config loader for supervisor
│   ├── multiplex_feed.py      # Shared WebSocket + per-market ConflatingPriceBuffer (Hydra)
│   ├── market_engine.py       # Per-market asyncio task runner (Hydra)
│   ├── task_health.py         # Unified health/metrics server (Hydra)
│   ├── bot_config.py          # Immutable per-market config dataclass
│   ├── benchmark.py           # Performance harness
│   ├── test_strategy.py       # Strategy unit tests (29+)
│   ├── test_reconcile.py      # Reconciliation regression tests
│   ├── test_transport.py      # Transport layer tests
│   └── test_hydra.py          # Hydra tests
├── scripts/
│   ├── load-profile.sh        # Merge base + profile into .env
│   ├── run-local.sh           # Run supervisor locally (Hydra)
│   └── switch-profile.sh      # Switch active .env profile
├── docs/
│   ├── DEPLOYMENT.md          # Production deployment notes
│   ├── PROFILES.md            # Strategy profiles
│   ├── hosting_plan.md        # Deployment/infra notes
│   ├── metrics_guidelines.md  # Prometheus/Grafana tips
│   └── events.md              # Sample markets
├── tests/                     # pytest-based tests/utilities
├── docker-compose.yml         # Supervisor service definition
├── Dockerfile                 # Supervisor image
└── README.md

Operational Guarantees

• Non-blocking concurrency: all signing/HTTP in thread executor; WS heartbeat never starves.
• Atomic execution: FOK orders only; no resting limits.
• Defensive behavior: reconnect on WS drop, clear stale prices, 0.5s cooldown after failed order, reconciliation before live trading.

---

🚀 Performance

Run python3 src/benchmark.py and confirm the hot path stays under ~1 µs/tick and sustains >1M ticks/s (machine-dependent).

---

📦 Installation & Environment

Prerequisites

• Python 3.10+ (3.11+ recommended for local dev scripts)
• Docker (optional, for running the supervisor via docker-compose)
• Polymarket credentials: PRIVATE_KEY, PROXY_ADDRESS
• Optional: curl_cffi for TLS impersonation (USE_BROWSER_IMPERSONATION=true)

Setup

python3 -m venv venv
source venv/bin/activate
pip install -r src/requirements.txt              # single-market bot
pip install -r supervisor-requirements.txt       # supervisor (Hydra)

# Prepare credential & profile files (do not hand-edit .env directly)
cp .env.profiles/base.env.template .env.profiles/base.env   # add PRIVATE_KEY/PROXY_ADDRESS
# Choose a profile: 15min | crypto | sports | test | conservative | hydra-test
./scripts/load-profile.sh test   # merges base + profile into .env (backs up previous)

Legacy single-market fallback (not recommended): cp .env.example .env then set TOKEN_YES/TOKEN_NO and thresholds.

---

🎮 Running (single market)

# Dry run (simulated fills), INFO logs
DRY_RUN=true LOG_LEVEL=INFO python src/main.py

# Live trading, minimal logging
DRY_RUN=false LOG_LEVEL=WARNING python src/main.py

# Debug everything (app + websockets)
LOG_LEVEL=DEBUG WEBSOCKETS_LOG_LEVEL=DEBUG python src/main.py

Observability

• Health server on :8080 by default: /health (liveness with staleness), /status, /metrics (Prometheus). Toggle via ENABLE_HEALTH_SERVER / ENABLE_METRICS.
• Reconciliation: ENABLE_RECONCILIATION=true replays last RECONCILIATION_LOOKBACK_HOURS (default 72) hours of BUY trades for the proxy wallet; set RECONCILIATION_REQUIRED=true to fail startup on errors.

---

🐙 Supervisor (multi-market, Hydra)

Purpose: auto-discover high-volume Polymarket markets and run each market as an asyncio task in a single supervisor process (shared WebSocket, global order queue, unified health/metrics).

Quick start

# Local testing (no Docker required)
./scripts/run-local.sh hydra-test

# Or via docker-compose (supervisor container)
docker compose up -d supervisor

Core mechanics

• Shared WebSocket: MultiplexedMarketFeed maintains one connection, fans out best-ask updates to per-market ConflatingPriceBuffers.
• Per-market runner: MarketEngine consumes price updates and calls strategy logic.
• Wallet fairness: ExecutionClient uses a global order queue with GLOBAL_ORDER_MIN_INTERVAL_S spacing to avoid wallet-level rate limits.
• Unified health: Single /health, /status, /metrics endpoint with market label per engine.

Note on Docker reachability

• In docker compose, the health server binds inside the supervisor container; publish the chosen port (default HEALTH_PORT_BASE) if you want to scrape it from outside Docker.

Supervisor-specific envs

• MARKET_PATTERNS: comma-separated globs for market slugs (e.g., US*Election*).
• MIN_VOLUME: minimum 24h volume filter (e.g., 5000).
• MIN_DURATION_HOURS / MAX_DURATION_HOURS: expiry window; defaults exclude <5m and >720h.
• MAX_MARKETS: cap on concurrent markets (default 20).
• ENABLE_BOOK_SCREEN=true: require order-book pair cost ≤ SPAWN_PAIR_COST (default 1.02) before spawning.
• HEALTH_PORT_BASE: port for the supervisor health server (default 8000).
• TARGETS_PATH: Prometheus target file (default /var/lib/kabobagool/targets.json).
• GLOBAL_ORDER_MIN_INTERVAL_S: seconds between orders across all markets (default: 0.5).

---

⚙️ Configuration Reference (single bot)

• Required: PRIVATE_KEY, PROXY_ADDRESS
• Market: TOKEN_YES, TOKEN_NO (decimal strings)
• Strategy: MAX_PAIR_COST (default 0.98), EDGE_BUFFER (0.0), TAKER_FEE_BPS (1), BATCH_SIZE (50), HEDGE_TOLERANCE (100), FORCE_HEDGE_IMBALANCE (0)
• Rate limiting: COOLDOWN_BASE_S, COOLDOWN_MAX_S, COOLDOWN_BACKOFF, GLOBAL_ORDER_MIN_INTERVAL_S (supervisor)
• Safety: DRY_RUN (true/false)
• Logging: LOG_LEVEL (WARNING/INFO/DEBUG), WEBSOCKETS_LOG_LEVEL
• Network: CLOB_PROXY_URL for geo-bypass (optional), HOST (default https://clob.polymarket.com), CHAIN_ID (137), USE_BROWSER_IMPERSONATION (false) for TLS fingerprint bypass via curl_cffi
• Observability: ENABLE_HEALTH_SERVER, ENABLE_METRICS, HEALTH_CHECK_PORT (8080), HEALTH_PRICE_STALENESS_THRESHOLD (60s)
• State restore: ENABLE_RECONCILIATION, RECONCILIATION_LOOKBACK_HOURS (72), RECONCILIATION_REQUIRED

---

🧪 Testing & Verification

# Unit tests
python3 src/test_strategy.py      # Strategy + decision logic (29+ tests)
python3 src/test_reconcile.py     # Reconciliation regression tests
python3 src/test_transport.py     # Transport layer (aiohttp/curl_cffi)
python3 src/test_hydra.py         # Hydra components

# Optional: pytest suite in ./tests (requires pytest installed)
python3 -m pytest -q

# Benchmarks
python3 src/benchmark.py

# Syntax check
python3 -m py_compile src/*.py

---

🐛 Troubleshooting

• Missing env vars: ensure .env.profiles/base.env has credentials then rerun ./scripts/load-profile.sh <profile>.
• Frequent WS drops: network issues or rate limits; bot auto-reconnects and resets prices to 1.0 to avoid stale trades.
• No fills: lower MAX_PAIR_COST, confirm token IDs are decimal strings, and verify proxy wallet is funded.
• Supervisor spawns nothing: adjust MARKET_PATTERNS, MIN_VOLUME, or raise MAX_MARKETS; check book-screen threshold SPAWN_PAIR_COST.

---

🔒 Security Notes

• Secrets live only in .env.profiles/base.env and generated .env (git-ignored). Do not commit.
• All orders are FOK to avoid resting exposure; DRY_RUN defaults to true for safety in development.
• Proxy/private keys should use a dedicated funded wallet; consider hardware wallet or secrets manager in production.