Production · Read-only Research & Risk Tool

Market Calibration
Agent

Trust Scores for Prediction Market Probabilities.
Quantify how reliable each Polymarket probability actually is.

A production-grade system that assesses prediction market reliability through calibration analysis, statistical scoring, and Time Series Foundation Model inference — with safeguards for rate-limiting, caching, and circuit breakers.

View on GitHub ↗ Explore Features →
// What It Measures
Statistical calibration,
made actionable.

The agent quantifies market signal reliability through four core statistical metrics — turning abstract probabilities into trustworthy, auditable Trust Scores.

Brier Proper scoring rule
ECE Expected Calibration Error
Log-Loss Probabilistic accuracy
Drift Probability deviation alerts
Trust Score Composite reliability signal
// Key Features
Everything you need to
trust a probability.
🔗
Multi-source Data Integration

Integrates REST APIs, WebSocket feeds, and subgraph queries from Polymarket to maintain a real-time, unified view of market data.

📐
Calibration Analysis Pipeline

Computes Brier score, log-loss, and Expected Calibration Error per market — generating composite Trust Scores with postmortem-ready audit trails.

🚨
Configurable Alert Generation

YAML-defined alert rules trigger on probability deviation thresholds. Alert feeds are exposed as read-only API endpoints for downstream consumers.

🧠
TSFM Inference with Safeguards

Integrates Time Series Foundation Model inference with production-grade safeguards: rate-limiting, response caching, and circuit breakers to prevent cascade failures.

🖥️
Streamlit Operational Dashboard

Real-time monitoring dashboard showing market summaries, Trust Score distributions, active alerts, and calibration postmortems — designed for ops and quant teams.

📡
Read-only API Services

Exposes market summaries, alert feeds, and performance postmortems as structured, read-only API endpoints. No write operations — safe for research and risk workflows.

⚙️
YAML-based Configuration

Fully configurable via YAML — alert thresholds, model settings, API targets, and caching behavior are all declared without code changes.

🧩
Modular Architecture

Cleanly separated connectors, calibration pipelines, and API handlers. Add new market sources or scoring metrics without touching core logic.

Production Python,
modular by design.

Built with uv as package manager with a frozen lockfile for reproducible installs. Python 95%+ of the codebase.

  • Python Primary runtime (95.6% of codebase)
  • uv Package manager with frozen lockfile
  • YAML Configuration — alerts, models, API targets
  • Streamlit Operational monitoring dashboard
  • Polymarket APIs REST + WebSocket + subgraph
  • TSFM Time Series Foundation Model inference
json · /api/market/{id}/trust-score GET
// Market calibration result
{
  "market_id": "0x1a2b...",
  "question": "Will X happen by...",
  "current_probability": 0.74,
  "trust_score": 0.81,
  "metrics": {
    "brier_score": 0.142,
    "log_loss": 0.318,
    "ece": 0.047
  },
  "alerts": [],
  "tsfm_signal": {
    "forecast": [0.72, 0.75, 0.71],
    "cached": false
  }
}
// Target Users
Built for the people who
need to trust the signal.
📊
Quantitative Researchers

Use calibration metrics and TSFM signals to evaluate prediction market quality before incorporating them into models or strategies.

🛡️
Operations Teams

Monitor real-time alert feeds and Trust Score distributions to catch signal degradation before it affects downstream decisions.

🔧
Platform Engineers

Integrate the read-only API endpoints into existing risk pipelines, dashboards, or agent workflows with minimal configuration.

🧪
Model Teams

Access structured postmortem data and calibration history to evaluate how prediction markets perform across different market regimes.

// Get Started

Know when to trust
the market signal.

Production-ready for quantitative research and risk workflows.

View on GitHub ↗ ← Back to Tollama AI