HTTP API

Every node answers HTTP on its --addr (default 127.0.0.1:8080). The Python and JavaScript clients, the shell and the console all use these endpoints, and so can curl. Any node takes any request. A request that changes a table, view, task or function goes to the leader, and a follower forwards it there.

Roles

A node started without tokens is open. With --read-token, --write-token and --admin-token (or the PONDRA_*_TOKEN variables), a request carries Authorization: Bearer <token>, and the token’s role decides what it may do. A role covers the ones before it: admin, then write, then read.

Role	May
read	run queries, look up keys, watch, live queries, read files, the Iceberg catalog
write	also append rows, insert, put files, and write with POST /sql
admin	also create tables, views, tasks and functions, run DDL (CREATE, ALTER, DROP, COPY … TO) and DO blocks in POST /sql, and tier

A request with no token is answered 401 (sign in: a token, or a user's name and password), with a wrong one 401 too, and with a token or user without enough rights 403 (this needs more rights than this token's or user's). Any other error is 500 with its message as plain text.

curl -s -H "Authorization: Bearer $PONDRA_TOKEN" -X POST http://localhost:8080/sql -d "SELECT 1"

Run SQL

Method and path	Role	Meaning
POST /sql	read (write and admin for statements that change things)	run one or more statements; answer with the last one’s rows or outcome
GET /sql/pages/{id}?from=&rows=	read	more rows of an answer of more than 10,000 (format=typed’s pages), kept on the node a while: 410 once it isn’t

The body is SQL text. To send parameters, send JSON with Content-Type: application/json: {"sql": "…", "params": {"min": 5}}. Parameters are $name in the SQL; a value is JSON, or {"sql": "DATE '2026-09-27'"} for one that JSON has no type for. views (a map of a name to a query) and tables are for clients that send frames and data along; the clients build them.

Query parameter	Meaning
format	json (the default), table (a text table), arrow (an Arrow IPC stream, for pandas, Polars and DuckDB) or typed (below); or a file of every row to download: csv, tsv, ndjson (JSON lines), parquet or xlsx (an Excel workbook: one sheet, the names in bold, at most 1,048,575 rows)
after	wait until this node has seen this log segment (from an append’s seg), for reading your own writes
spread	1 runs the query across the cluster even for small tables, 0 only here
stale_ms	accept a remembered answer up to this many milliseconds old
job	for a write: a retry with the same job id is applied once

A query answers a JSON array of rows, one object each. A column that is null in a row is left out of its object. A statement that isn’t a query answers a JSON object, such as {"rows": 2} or {"table": "t", "publish": []}. Several statements in one request run in order.

format=typed is what the console uses: {"columns": [{"name", "type"}], "rows": [[…], …], "total": N, "pages": "…"}. Each row is a list in the columns’ order, so two columns may share a name (a join’s). It holds the first 10,000 rows (&rows=N for another number, 1 to 100,000: the console’s rows a page), and total says how many there were. Decimals come as text with all their digits ("1.50"), and so do integers past 2^53, which JavaScript can’t hold exactly.

An answer of more rows than that is kept on the node a while, and pages is its id (else null). GET /sql/pages/{id}?from=10000&rows=10000 answers more of its rows, in the same form: the same rows in the same order, without running the query again. The node keeps such answers within PONDRA_PAGES_MB (256) in all, for 20 minutes after each was last read; one no longer kept answers 410, and the query can be run again (LIMIT … OFFSET …). The id is random: only whoever was sent the answer knows it, and reading a page needs the read token, as /sql does.

import json, urllib.request

def post(sql):
    return json.load(urllib.request.urlopen(urllib.request.Request("http://localhost:8080/sql?format=typed", sql.encode(), method="POST")))

first = post("SELECT value AS n FROM range(0, 25000)")
more = json.load(urllib.request.urlopen(f"http://localhost:8080/sql/pages/{first['pages']}?from=20000&rows=10000"))
print(first["total"], len(first["rows"]), len(more["rows"]), more["rows"][0])

25000 10000 5000 [20000]

The header x-pondra-notices on an answer carries what procedures and DO blocks printed, as a JSON list of lines. Send x-pondra-session: <id> (8 to 128 letters, digits, - or _) to use temporary tables that only that id sees, DO LANGUAGE python blocks that share their variables (Functions and procedures), and a transaction: BEGIN, then statements, then COMMIT, each request with the same id (Transactions). An error is status 500 with its message as the body, and x-pondra-sqlstate, Postgres’s code for it (error codes).

curl -s -X POST http://localhost:8080/sql -d "CREATE TABLE events (user VARCHAR, amount BIGINT)"
curl -s -X POST http://localhost:8080/sql -d "INSERT INTO events VALUES ('ann', 5), ('bo', 7)"
curl -s -X POST http://localhost:8080/sql -d "SELECT user, sum(amount) AS total FROM events GROUP BY user ORDER BY user"
curl -s -X POST http://localhost:8080/sql -H 'content-type: application/json' -d '{"sql": "SELECT * FROM events WHERE amount > $min", "params": {"min": 5}}'
curl -s -X POST 'http://localhost:8080/sql?format=table' -d "SELECT * FROM events ORDER BY user"

{"table":"events","publish":[]}
{"rows":2}
[{"user":"ann","total":5},{"user":"bo","total":7}]
[{"user":"bo","amount":7}]
+------+--------+
| user | amount |
+------+--------+
| ann  | 5      |
| bo   | 7      |
+------+--------+

Write rows

Method and path	Role	Meaning
POST /append/{table}	write	append rows exactly once
POST /insert/{table}	write	run a query and write its rows as files, in bulk

POST /append/{table}?producer=NAME&seq=N[&prev=M]: the body is newline-delimited JSON, one row per line, or an Arrow IPC stream with Content-Type: application/vnd.apache.arrow.stream. A column left out takes its DEFAULT, or null. producer names the writer and seq counts its appends, from 1, so a retry with the same producer and seq is recognised and not applied twice. prev is a compare-and-swap: the append applies only if the producer’s last seq is prev. Without producer, an append is applied each time it is sent (at least once), as a Kafka producer that isn’t idempotent. The answer is:

Field	Meaning
seg	the log segment that holds the rows
duplicate	true if this seq was already applied, so nothing was written
conflict	true if prev didn’t match: read again and retry
row	where the rows start among the table’s rows in seg
ms	the commit time, for a reservation

POST /insert/{table}?job=ID: the body is a query, such as SELECT * FROM events. The node runs it and writes the rows as Parquet, and the table is made if it doesn’t exist. The answer is {"rows": N}, or {"duplicate": true} if this job was already applied.

curl -s -X POST 'http://localhost:8080/append/events?producer=p1&seq=1' -H 'content-type: application/x-ndjson' --data-binary $'{"user":"cy","amount":3}\n{"user":"di","amount":4}\n'
curl -s -X POST 'http://localhost:8080/append/events?producer=p1&seq=1' --data-binary $'{"user":"cy","amount":3}\n'
curl -s -X POST 'http://localhost:8080/insert/events_copy?job=copy-1' -d "SELECT * FROM events"
curl -s -X POST 'http://localhost:8080/insert/events_copy?job=copy-1' -d "SELECT * FROM events"

{"seg":3,"duplicate":false,"conflict":false,"row":0,"ms":0}
{"seg":0,"duplicate":true,"conflict":false,"row":0,"ms":0}
{"rows":4}
{"duplicate":true}

Tables, views and tasks

Method and path	Role	Meaning
POST /tables/{name}	admin	make a table from a JSON definition, or change an existing one (add columns at the end, publish, cluster_by, ttl, order_by)
POST /views/{name}	admin	make a materialized view; the body is its query, and its options are query parameters
POST /tasks/{name}	admin	make a task that runs a query over a source table into a target, as it arrives

The body of POST /tables/{name} is either a list of [name, type] pairs, with Arrow’s type names (Utf8, Int64, Float64, Boolean, Timestamp(Microsecond, None)), or an object with columns and the options: key (a list), merge (an object of column to function), publish and cluster_by (lists), partition_by, ttl and order_by (strings). See table options. The answer is {"table": name, "publish": […]}.

POST /views/{name}?window=w&size_secs=60&lateness_secs=10 takes any of the options of CREATE MATERIALIZED VIEW as query parameters. The answer is {"view": name, "materialized": true}.

POST /tasks/{name} takes JSON: source and target (table names), sql (the query over the source), key (the target’s primary key, to make it an upsert table), shards (default 1) and shard_by (a column, needed when shards is more than 1). The answer is {"task": name}.

curl -s -X POST http://localhost:8080/tables/points -d '{"columns": [["id", "Int64"], ["label", "Utf8"]], "key": ["id"]}'
curl -s -X POST http://localhost:8080/tables/ticks -d '[["ts", "Timestamp(Microsecond, None)"], ["amount", "Float64"]]'
curl -s -X POST 'http://localhost:8080/views/spend?window=w&size_secs=3600' -d "SELECT date_bin(INTERVAL '1 hour', ts) AS w, sum(amount) AS total FROM ticks GROUP BY 1"
curl -s -X POST http://localhost:8080/tasks/per_user -d '{"source": "events", "target": "per_user", "sql": "SELECT user, sum(amount) AS total FROM events GROUP BY user", "key": ["user"]}'

{"table":"points","publish":[]}
{"table":"ticks","publish":[]}
{"view":"spend","materialized":true}
{"task":"per_user"}

Read and follow

Method and path	Role	Meaning
GET /lookup/{table}/{key}	read	the current row of one key of a keyed table
GET /watch/{table}	read	a stream of a table’s new rows as they commit
GET /live?sql=… or POST /live	read	a stream of a query’s answer, again each time a commit changes it

GET /lookup/{table}/{key} answers a JSON array with the row, or []. A composite key is the values, comma-separated, in the key’s column order. It is an error for a table with no key.

GET /watch/{table} streams newline-delimited JSON, one row per line, for as long as you keep the connection open. Its parameters: after=N (replay from this position; the default is from now on, and a replay goes as far back as the log is kept), marks=true (after each batch of rows, a line {"_after": N}: resume with ?after=N) and changes=true (the change feed: UPDATE’s and DELETE’s rows too, each with _change_type, _row_id, _version, _created_at and _updated_at).

/live answers a query now, and again each time a commit changes a table it reads, as lines of {"at": COMMIT, "rows": […]}. GET /live?sql=… takes the query in the URL, and POST /live takes a body as POST /sql does, with $name parameters. every_ms sets the shortest time between queries on a busy table (default 100). An answer that comes out the same isn’t sent again. A line {"error": …} ends the stream. The query runs while the connection is open, and stops when it closes.

Several live queries share one connection with POST /live and a JSON body {"queries": [{"id": "a", "sql": "…", "params": {…}, "session": "…"}, …]}: each line names its query’s id, a query that can’t be watched sends its own {"id": …, "error": …} line while the others go on, and session is the x-pondra-session that query runs in. A browser opens six connections to a host at most, so a page that shows several live answers (the console’s) needs this.

curl -s http://localhost:8080/lookup/points/1
curl -s --max-time 2 'http://localhost:8080/watch/events?after=0&marks=true' || true
curl -s --max-time 2 -G http://localhost:8080/live --data-urlencode 'sql=SELECT count(*) AS n FROM events' || true
curl -s --max-time 2 http://localhost:8080/live -H 'content-type: application/json' \
  -d '{"queries": [{"id": "n", "sql": "SELECT count(*) AS n FROM events"}, {"id": "one", "sql": "SELECT 1 AS one"}]}' || true

[]
{"user":"ann","amount":5}
{"user":"bo","amount":7}
{"user":"cy","amount":3}
{"user":"di","amount":4}
{"_after":6}
{"at":7,"rows":[{"n":4}]}

Files

The lake’s own files, under files/: notebooks, SQL and Python files, data files, images. The console’s Workspace shows them and edits them in place.

Method and path	Role	Meaning
PUT /files/{path}	write	store the body at files/{path}; a new file, or with If-Match the file replaced
GET /files/{path}	read	the file’s bytes, its version as etag, and a content type by its name
DELETE /files/{path}	write	remove it (its versions stay)
GET /files/{path}?versions	read	its versions, newest first: [{"id", "at", "who", "bytes"}]
GET /files/{path}?version={id}	read	one version’s bytes
POST /files/{path}?restore={id}	write	that version made the file again (and kept as the newest)

• A new file: a PUT with no If-Match to a path that exists is refused with 409, so a file is never overwritten blind.
• Replacing one: send the etag a GET gave as If-Match. If someone saved the file since, the PUT is refused with 412, and theirs is kept: read it again, or save under another name.
• The answer to PUT is {"path": "files/{path}", "bytes": N, "version": "…"}, the version the next If-Match needs. A GET of a path with no file is 404.
• Every node reads a replaced file’s new bytes at its next statement: nothing caches files/ (not the SSD tier either). The rest of the lake is never overwritten, so it is cached.
• Every save is kept as a version, with who made it and when, in files/.versions/ (which files() doesn’t list): the newest 50 for 90 days (PONDRA_FILE_VERSIONS, PONDRA_FILE_VERSIONS_DAYS), files up to 64 MB. A deleted file keeps its versions, so it can be restored. The console’s Versions… shows what changed and restores one.

Read them from SQL with files() and file_read(), see functions.

curl -s -X PUT http://localhost:8080/files/notes/a.txt --data-binary 'hello'
curl -s http://localhost:8080/files/notes/a.txt; echo
v=$(curl -sI http://localhost:8080/files/notes/a.txt | tr -d '\r' | sed -n 's/^etag: //ip')
curl -s -X PUT http://localhost:8080/files/notes/a.txt -H "If-Match: $v" --data-binary 'hello again'
curl -s -X PUT http://localhost:8080/files/notes/a.txt -H "If-Match: $v" --data-binary 'too late'; echo
curl -s -X DELETE http://localhost:8080/files/notes/a.txt
first=$(curl -s "http://localhost:8080/files/notes/a.txt?versions" | python3 -c "import json, sys; print(json.load(sys.stdin)[-1]['id'])")
curl -s -X POST "http://localhost:8080/files/notes/a.txt?restore=$first" > /dev/null
curl -s http://localhost:8080/files/notes/a.txt; echo

{"path":"files/notes/a.txt","bytes":5,"version":"…"}
hello
{"path":"files/notes/a.txt","bytes":11,"version":"…"}
files/notes/a.txt: the file changed since it was opened: open it again, or save under another name
{"removed":"files/notes/a.txt"}
hello

Functions, routines and secrets

Method and path	Role	Meaning
POST /functions/{name}	admin	register a function run by your own Arrow Flight server
DELETE /functions/{name}	admin	remove it
GET /functions	admin	the lake’s registered Flight functions, as an object by name
GET /routines	read	the lake’s SQL and Python functions and procedures, as an object by name
GET /secrets/{name}	none	a secret’s values: only for the token a running Python procedure is lent

The body of POST /functions/{name} is {"flight": "http://host:port", "args": ["Utf8"], "returns": "Utf8", "rows": N}, where rows is optional. See functions. The answer is {"function": name}, and {"dropped": name} for DELETE.

GET /secrets/{name} is what pondra.secret(name) calls inside a Python procedure. Any other token is refused.

The server

Method and path	Role	Meaning
GET /stats	none	health and position, as JSON
GET /healthz	none	ok (200) while the process runs: a liveness check
GET /ready	none	ready (200) when the node serves; 503 while it stops, or just after it started and is still catching up with its leader: a readiness check
POST /login	none	{"user", "password"}: a session for the user, {"token": "ps_…", "until": ms}, to send as Authorization: Bearer … until it ends (PONDRA_SESSION_HOURS)
GET /whoami	none	who the request signs in as: {"user", "role", "superuser", "open"}
GET /objects	read	each table and view of this lake and the lakes attached to it: catalog, schema, name, kind (table, view, materialized view, or files for a view of files), and for a table its rows and bytes in files, files, key, partition, cluster, publish, not_null, defaults; for a view its sql. Also files: where the lake’s own files are, for read_csv('…'). What the console’s catalog shows
GET /kinds	read	every kind of object a lake holds, [{"kind", "family", "statements"}], as pondra.kinds lists them
GET /metrics	read	Prometheus metrics, as text
POST /tier	admin	write the log into Parquet now: {"rows_tiered": N}
DELETE /sessions/{id}	read	end a session: its temporary tables and views go, and so does its Python: {"ended": true}
GET /sessions/{id}/python	admin	the session’s Python (a console page’s cells): {"running", "busy", "variables": [{"name", "type", "size", "look"}], "python": {"python", "version"}}
POST /sessions/{id}/python	admin	interrupt its running cell: it ends with KeyboardInterrupt and the variables stay ({"done": "interrupted"}); on Windows Python restarts instead ("restarted")
DELETE /sessions/{id}/python	admin	restart it: its variables go, its temporary tables stay
GET /python	admin	the Python workers run with: {"python", "version"}; with ?all=1, also pythons: each Python this machine has, tried (path, version, pondra, pyarrow, ok, error)
PUT /python	admin, from this machine	{"path": "…"}: run workers with that Python from now on (it must have pondra and pyarrow), and next time too. Workers running now finish with the Python they have
POST /python/format	admin	{"code": "…"}: Python code formatted by the node’s Python, as ruff (or black, if it has no ruff) formats it: {"code": "…"}. The code is only read, never run
GET /	none	the console: a web page, with no data in it until you use it
GET /console/console.js, /console/console.css, /console/ext/{n}.js	none	the console’s scripts and style (the modules it loads, some the first time they are needed), and the extensions the node was given (PONDRA_CONSOLE_EXTENSIONS); each with an ETag, 304 when unchanged
GET /console/settings	none	the console’s settings kept on this machine (theme, colours, layout), as JSON: {} if none
PUT /console/settings	none, from this machine	keep them: a JSON object, at most 64 KB. From a page on another machine it is refused, and that page keeps its own

GET /stats needs no token, so a load balancer can poll it. It answers lake (its name), role (leader, follower or reader), leader (its address), term, nodes, hwm (the newest commit), shard_runs, python_workers, live_queries, version (the node’s release) and format (the lake’s, see upgrades). On the leader it also has untiered_rows, commits, commit_ms_p50, commit_ms_p95 and releases (each live node’s release, by address).

A node that is stopping answers every other path with 503 and Retry-After: 1: send the request to another node.

curl -s http://localhost:8080/stats
curl -s http://localhost:8080/metrics | head -3
curl -s -X POST http://localhost:8080/tier

{"lake":"lake","role":"leader","leader":"127.0.0.1:8080","term":1,"nodes":["127.0.0.1:8080"],"hwm":7,"shard_runs":1,"python_workers":0,"live_queries":0,"version":"0.30.0","format":1,"untiered_rows":0,"commits":9,"commit_ms_p50":1.69,"commit_ms_p95":2.98,"releases":{"127.0.0.1:8080":"0.30.0"}}
# HELP pondra_role 1 for this node's role
# TYPE pondra_role gauge
pondra_role{role="leader"} 1
{"rows_tiered":0}

A folder of lakes

pondra serve on a folder of lakes answers on one port for every database in it:

Method and path	Role	Meaning
/db/{name}/…	as on a node	database name’s whole API: POST /db/sales/sql, GET /db/sales/stats, …
any other path	as on a node	the default database’s (--default)
GET /databases	read	the databases: [{"name", "running", "default"}]
POST /databases	admin	make one: {"name": "hr"}, as CREATE DATABASE hr
DELETE /databases/{name}	admin	drop one, folder and all, as DROP DATABASE; ?if_exists=true is quiet about one that isn’t there
GET /	none	the console, listing every database

Other doors on the same port

Path	Role	What it is
POST /mcp	read (writes need more)	the MCP server: see MCP
/v1/...	read; write for POST	the Iceberg REST catalog: see Iceberg REST

Between nodes

Nodes in a cluster call each other on these paths. You don’t call them, but they are why a firewall between nodes must let HTTP through. Each call carries the caller’s release and the newest lake format it knows (x-pondra-version, x-pondra-format), so the leader knows when every node can read a newer format. They need an admin token, except commit and files, which need a write token, and leader, which needs none.

Path	Meaning
POST /cluster/commit	a follower’s flush, to be sequenced by the leader
POST /cluster/files	record the files an INSERT wrote, on the leader
POST /cluster/ddl, /cluster/change, /cluster/iceberg	forward a schema change, an UPDATE, DELETE or MERGE, or another engine’s Iceberg append to the leader
POST /cluster/stage, /cluster/copy, /cluster/job	this node’s share of a spread query, COPY … TO, or the leader’s data work
GET /cluster/shuffle	a shuffle bucket this node keeps for another node
GET /cluster/log, /cluster/replica, POST /cluster/ack	the commit stream, and replicated commits
POST /cluster/beat	a heartbeat
GET /cluster/leader, /cluster/kafka	who leads, and where the Kafka port is
GET /cluster/probe	measure the link between two nodes

Caution

A request body is limited to 1 GiB. The body of an append or an insert must fit in memory on the node that takes it, so send big loads in batches.