● Ship · Fixes

Troubleshooting
errors and their fixes.

First the diagnostic steps for the integrations that fail most often, with the real log lines and commands. Then messages you may see, what they mean and what to do. Tip: paste the exact error into the search box.

Which terminal

The commands on this page are written for the shell of macOS and Linux (bash or zsh). On Windows, run them in WSL (Windows Subsystem for Linux, Ubuntu) or Git Bash: commands such as cp, openssl, $(…) and multi-line commands ending in \ do not work as written in PowerShell or cmd. The npm and node commands work in any of them.

01Start here: health check and logs

Two things answer most questions: the health endpoint and the server log.

curl -s http://127.0.0.1:8787/api/health          # on your computer
curl -s https://<your-domain>/api/health           # online

A fresh install with no model key and no arena file answers (captured from a local run):

{"ok":true,"version":"1.1.0","uptimeSec":1,"serverTime":…,
 "configSource":"arena.config.example.json","operatorTokenSet":false,
 "providers":{"anthropic":false,"openai":false,"google":false,"xai":false,"deepseek":false},
 "streamClients":0,"league":{"enabled":true,"lastRefresh":null,"lastError":null,"marketsInPlay":0}}

Field	Look for
`configSource`	`arena.config.json` once you have your own file; `arena.config.example.json` means the server did not find it.
`providers`	`true` for each provider whose key comes from the environment.
`league.lastError`	The last Polymarket error, or `null`.
`operatorTokenSet`	Must be `true` before a real-money season.

Where the log is: the terminal running npm run dev or npm start; journalctl -u pit under systemd; pm2 logs pit under pm2. Each line reads <time> <LEVEL> [<area>] <message>, for example INFO [http] GET /api/health 200 3ms. A healthy start contains:

INFO  [paper] paper league engine started (season s1)
INFO  [pit] providers with keys: none (competitors are idle, nothing is simulated)
INFO  [pit] listening on http://127.0.0.1:8787 (development, API only) · season s1
INFO  [paper-feed] allMids websocket connected (mainnet)

Then open the console → Status: each check names what is wrong.

Status page on the demo console: one blocker (no model key is set there) and passing checks for builder fee, Stripe, USDC, Ride, Hyperliquid prices and PIT_SECRET — Status on the demo console, where no model key is set.

02The server stops at start with a variable named

On every start PIT checks the environment. A value it cannot use stops the start, with one line per problem, so a typo never runs silently. The value itself is never printed. Example with PORT=abc and RIDE_ENABLED=maybe:

ERROR [pit] PORT must be a whole number from 1 to 65535.
ERROR [pit] RIDE_ENABLED must be true or false.
ERROR [pit] fix the environment (.env) and start again; see .env.example

Fix the named variables in .env (compare with .env.example) and start again. Lines marked WARN (for example ADMIN_SECRET_KEY is not set) do not stop the start; they tell you which optional part is off.

03AI provider errors

Is there a key?Arena banner No model key configured, competitor label "No model key", or the run state no key on the Paper league page: the server has no key for that provider. Add it in Integrations.
Does the key work?Integrations → the provider → Test connection. It asks the provider for its model list. "Key works: N models available." means the key is fine. Anything else is the provider's own answer.
Read the failed decisionA failed model call is stored as a failed decision and its error shows as the note on Radio and on the entrant page, in the form <provider>: HTTP <status>: <provider's message>. Prediction-league failures are logged as [league] forecast failed.

You see	Meaning	Do
`HTTP 401` or `HTTP 403`	The key is wrong, revoked, or not allowed to use that model.	Create a new key, paste it, Test connection.
`HTTP 404` or a "model" error	The model id does not exist (any more) for your key.	Fix the id on Competitors.
`HTTP 429`	Rate limit or no credit left. PIT already retried (3 attempts, honouring `retry-after`).	Add credit or raise your provider limit; lengthen the decision interval.
`HTTP 5xx`, `network error: …`	Provider outage or no route from your server. Also retried.	Wait; check the server can reach the internet.
`timed out after Ns`	The model answered too slowly. Not retried.	Raise the timeout on Model settings.
`model returned invalid output (stop reason: max_tokens)`	A reasoning model ran out of output tokens.	Raise Max output tokens on Model settings.
Banner Daily model budget reached, run state budget	The budget guard: today's estimated paper spend reached the daily LLM budget. Not an error.	Wait for 00:00 UTC or raise the budget on Paper league.

04Stripe webhook signature failures

Stripe signs every webhook with your endpoint's signing secret. PIT checks it before anything else (server/money/stripe.ts) and answers with a reason. These answers were captured from a running server:

Answer	Meaning	Fix
`400 {"error":"signature check failed (no_header)","code":"bad_signature"}`	No `Stripe-Signature` header: not a Stripe call, or a proxy dropped the header.	Let the proxy pass all headers.
`signature check failed (malformed)`	The header has no timestamp or no `v1` signature.	Same as above.
`signature check failed (too_old)`	The signature is more than 300 seconds from the server's clock.	Sync the server clock (NTP).
`signature check failed (mismatch)`	The secret in PIT is not the secret of this endpoint, or the body was changed on the way.	Copy the signing secret of this endpoint (test and live mode differ; `stripe listen` has its own). Make sure no proxy rewrites the request body.
`503 … "webhook secret not configured"`	No webhook secret in Integrations or `STRIPE_WEBHOOK_SECRET`.	Enter it.

Steps:

Is the address reachable?
```
curl -s -X POST -H 'content-type: application/json' --data '{}' https://<your-domain>/api/money/webhook/stripe
```
Expected: the no_header answer above. A 404, an HTML page or a timeout means the address or the proxy is wrong.
Read the logEach delivery is one line: [http] POST /api/money/webhook/stripe 400 2ms (refused) or … 200 … (accepted). In the Stripe Dashboard, the endpoint's delivery list shows PIT's answer body.
Test locallyWith the Stripe CLI: stripe listen --forward-to localhost:8787/api/money/webhook/stripe, use the secret it prints, then stripe trigger checkout.session.completed (details).
Checkout does not startThe visitor sees "could not start the card checkout, try again" and the log has [money] stripe checkout failed: Stripe: <Stripe's message>. Test the secret key in Integrations.

05Hyperliquid prices not arriving

The paper league needs live prices. The server subscribes to Hyperliquid's allMids WebSocket, reconnects with a back-off of up to 30 seconds, and polls the HTTP API when the socket has been silent for 10 seconds (server/paper/feed.ts). It uses the network in season.network.

What you see: arena banner Waiting for live Hyperliquid prices with "Price feed: <error>. Retrying."; Paper league run state waiting for prices; Status check "Paper league has live Hyperliquid prices" failing with the error or "No fresh Hyperliquid prices: no paper decision runs."
Healthy: the log has [paper-feed] allMids websocket connected (mainnet) and Status says, for example, "235 coins via ws". "via http" means the socket is down and the HTTP fallback carries the prices.
Other log lines: [paper-feed] metaAndAssetCtxs failed: … (asset data) and market snapshot failed for <entrant>: … — retrying in 60 s.

Test from the server itself (mainnet; for testnet use api.hyperliquid-testnet.xyz):

curl -s -X POST https://api.hyperliquid.xyz/info -H 'content-type: application/json' -d '{"type":"allMids"}' | head -c 200

A JSON object of coin prices means the route works. An error or a timeout means your host or its firewall blocks outbound HTTPS or WebSockets to Hyperliquid. If visitors' charts stay empty but the server is fine, a browser extension or a custom CSP_CONNECT_EXTRA may be the cause: the site's policy allows Hyperliquid's API and WebSocket hosts.

06Polymarket unreachable

What you see: the prediction league shows its empty state; /api/health → league.lastError; Status check "Polymarket answers (prediction league)" with the same text.
Log, captured on a network that blocks Polymarket:
```
WARN  [league] refresh had errors: active markets: Gamma unreachable (gamma-api.polymarket.com): ERR_SSL_PACKET_LENGTH_TOO_LONG
```
A whole failed cycle logs ERROR [league] cycle failed: …. Gamma HTTP <status> for /markets means Polymarket answered with an error (rate limit or outage).
Test: curl -sS -m 10 "https://gamma-api.polymarket.com/markets?limit=1" from the server. Some internet providers and countries block it.
Fix: host the server where Polymarket is reachable, set POLYMARKET_GAMMA_URL to a mirror you trust, or turn the prediction league off. The rest of PIT is unaffected.

07ADMIN_SECRET_KEY and the console

Log line or answer	Meaning	Fix
`WARN [pit] ADMIN_SECRET_KEY is not set: the operator console (/admin) is off. Set it: openssl rand -hex 32` and `/admin` answers `503 admin_disabled`	No key	`openssl rand -hex 32`, set it, restart
`ERROR [pit] operator console: ADMIN_SECRET_KEY must be 32 bytes: openssl rand -hex 32`, then the server exits	Wrong format (captured from a run with a short key)	64 hex characters, or base64 of 32 bytes
`[admin] secret "…" cannot be decrypted: was the secretKey changed?`	The key differs from the one the secrets were saved with	Recovery
A line starting `[admin] No owner account yet`	No owner exists yet. Enter your `ADMIN_SETUP_TOKEN` at `/admin`; without a token, any printed setup code works until one is created.	Open `/admin` and create the owner

08SQLite and disk permissions

Error: unable to open database file (code ERR_SQLITE_ERROR) at start: the folder of PIT_DB (default data/, /data in Docker) does not exist and cannot be created, or the server's user cannot write to it. Captured from a run with a read-only folder.
attempt to write a readonly database: the database file itself is read-only for the server's user.
Check with ls -l data/ (or ls -l /data in the container). In Docker the server runs as the node user; the entrypoint fixes ownership of /data, pit.db, its -wal/-shm files, arena.config.json and backups/, but not of a file you upload as root later, such as admin.sqlite. Fix it with chown node:node /data/<file>.
Never run two instances on the same files, and never copy pit.db or admin.sqlite without their -wal file while the server runs: use deploy/backup.mjs.

09Sign-in behind a reverse proxy

Symptom	Cause	Fix
Wallet sign-in fails with `the message was issued for a different site` (`wrong_domain`)	The sign-in message is bound to `PUBLIC_URL`'s host (or the request's `Host`). Visitors use another address, for example `www.` versus no `www.`, or the `.fly.dev` name.	Set `PUBLIC_URL` to exactly the address visitors use, redirect the others to it, restart.
Writes refused with `cross-origin write refused` (`csrf`)	The proxy changes the `Host` header, so the browser's origin matches neither `Host` nor `PUBLIC_URL`.	Pass the original `Host` (nginx: `proxy_set_header Host $host;`) and set `PUBLIC_URL`.
Viewers sign in but are signed out at once	In production the viewer cookie is `Secure`, so a browser drops it over plain `http://`.	Serve the site over HTTPS. `ADMIN_COOKIE_SECURE=false` is only for a local test.
Ride says country unknown; per-visitor limits cannot tell visitors apart	Country and client-IP headers are trusted only with `TRUSTED_PROXY_SECRET`.	Make the proxy send `x-pit-proxy-secret` with that value; set `CLIENT_IP_HEADER` and `RIDE_COUNTRY_HEADER` if your proxy uses other header names.
Radio or the arena update late	The proxy buffers the live stream. PIT already sends `X-Accel-Buffering: no`, which nginx respects.	Turn off response buffering for `/api/stream` in your proxy. Check with `curl -N https://<your-domain>/api/stream`: it should print `retry: 5000` and a `hello` event at once.

The console cookie is separate: it is httpOnly, SameSite=Strict, and marked Secure when the request arrives over HTTPS or with X-Forwarded-Proto: https.

10Server and console

npm install fails on better-sqlite3 or @resvg/resvg-js

Both are native modules. npm downloads a prebuilt binary for common systems (macOS, Linux x64/arm64, Windows x64 on Node 22). If no prebuilt binary matches, better-sqlite3 compiles from source and needs build tools: on Debian/Ubuntu sudo apt install python3 make g++, on macOS xcode-select --install, on Windows the "Desktop development with C++" workload of Visual Studio Build Tools. Check node -v is 22.13 or newer, delete node_modules and run npm ci again.

"dist/ is missing"

Run npm run build before npm start.

ExperimentalWarning: SQLite is an experimental feature

Node prints this notice the first time a program uses its built-in node:sqlite module. npm start and npm test pass --disable-warning=ExperimentalWarning, so you only see it when you start build/server.mjs another way; add the same flag there. It is harmless.

/admin answers 503 admin_disabled

Set ADMIN_SECRET_KEY (openssl rand -hex 32) and restart.

Server exits: "ADMIN_SECRET_KEY must be 32 bytes"

Use 64 hex characters or base64 of 32 bytes.

Lost the setup code

Set ADMIN_SETUP_TOKEN in .env and restart, or restart without it: every start before the owner exists prints a new code. A wrong token is refused (403).

Sign-in refused with HTTP 429

5 wrong passwords for that email lock it for 15 minutes.

Console secrets unreadable after a move or restore

ADMIN_SECRET_KEY differs from the one they were saved with. Restore the old key, or enter the secrets again in Integrations.

A console setting seems to have no effect

Status shows a failing check with the reason (for example a builder address that is not 0x…). PIT does not apply a value it cannot use.

Status: "Read from the local secrets store"

A key is read from PIT's local encrypted store instead of the console. Enter it once in Integrations.

Status warning on "PUBLIC_URL is set (sign-in domain, share cards, Stripe return links)"

Set PUBLIC_URL to your public https:// address and restart (why).

A smart-contract wallet (Safe, Argent) cannot sign in

Add an RPC for that wallet's chain to SIWE_RPC_URLS (chainId=url) and restart. A wallet that is not yet deployed on that chain cannot sign in. Plain wallets need nothing.

Members cannot save an alert webhook ("the server has no encryption key")

Set PIT_SECRET and restart.

11Arena and leagues

Competitor shows "No model key"

No key for its provider. Add it in Integrations and press Test connection.

Paper league stops deciding for the day

Today's estimated LLM spend reached the daily budget. It resumes at 00:00 UTC, or raise the budget on the Paper league page.

"model returned invalid output (stop reason: max_tokens)"

A reasoning model ran out of output tokens. Raise Max output tokens for that provider on Model settings.

League empty, last error "Gamma unreachable … ERR_SSL_PACKET_LENGTH_TOO_LONG"

Your internet provider blocks Polymarket. Host the server elsewhere, or set POLYMARKET_GAMMA_URL to a mirror you trust.

Ride says "country unknown"

No trusted proxy. See Ride → Country gate. This is the safe default.

12Runner (real-money season)

"agent key … no longer approved (pruned, replaced or expired)"

The master has no funds, the key expired, or another approval replaced it. Put USDC back on the master and press Authorise agent again.

"not approved to trade sub-account 0x…"

Connect that sub-account's master wallet and authorise again.

"builder … has $x perps account value; Hyperliquid requires at least 100 USDC"

Fund the builder address with 100+ USDC in perps, or switch the builder fee off.

"the master has approved x% for the builder; orders carry y%"

Press Approve in Setup → Builder fee. If a sub-account's orders are still refused, approve from that account.

Competitor shows "Awaiting account"

Its address is empty. Add the sub-account on the Competitors page.

Competitor shows "Runner offline"

No heartbeat for two season intervals: the runner tab is closed, asleep or lost the server. Reopen it, paste the token and start.

HTTP 503 operator_token_unset

Set OPERATOR_TOKEN and restart, or generate one on Operator & runner.

HTTP 429 "too soon"

Decisions faster than half the interval are refused. Wait, or lower the interval.

"arena config changed — reload the runner and start again"

The arena config was saved after this tab loaded it (HTTP 409 config_changed). All loops stop on purpose. Reload the runner tab and start again.

"server has no model key for this competitor"

The server has no key for that competitor's provider. Add it in Integrations, then start the competitor again.

"The PIT server is not reachable"

Start the server and press Retry.

Operator and runner page: the operator token, per-competitor runner heartbeats and the server kill switch — Operator & runner: generate a token, read the last heartbeat per competitor, stop everything.

Troubleshootingerrors and their fixes.