Environment variables

Set these in the environment of the process that runs pondra (or the client that starts a node). Most have a flag as well, and the flag wins. The ones you are likely to need are the tokens, the memory and cache sizes, the AI endpoint, and PONDRA_SECRET_KEY. The rest tune how queries spread across nodes and how the node cleans up after itself.

A number that is left unset, or set to something that is not a number, takes its default.

Access

Variable	Default	Meaning
PONDRA_READ_TOKEN	none	the read token, when --read-token is not given. Reading needs any token
PONDRA_WRITE_TOKEN	none	the write token, when --write-token is not given
PONDRA_ADMIN_TOKEN	none	the admin token, when --admin-token is not given. A node also sends it when it calls other nodes
PONDRA_TOKEN	none	the token a program presents: pondra run (--token), a write made with pondra sql, and a node’s calls to other nodes, which use it before the admin token
PONDRA_SECRET_KEY	one made for the machine	the master key that wraps each secret’s own key. Every node of a cluster needs the same one. Without it, a key is made in ~/.pondra/secret.key (from HOME or USERPROFILE), which is enough for nodes that share that machine’s home
PONDRA_SECRET_KEY_PREVIOUS	none	the master key before it changed: secrets’ keys it wrapped still open, and the leader rewraps them with the new one when it starts
PONDRA_KMS_COMMAND	none	a key service in place of the master key: a command run with wrap or unwrap and a data key on its input, answering on its output (a script over aws kms, gcloud kms, az keyvault key or Vault’s transit)
PONDRA_SESSION_HOURS	12	how long a user’s session lasts (POST /login, a Flight handshake)
PONDRA_TLS_CERT, PONDRA_TLS_KEY	none	a PEM certificate and its key, when --tls-cert and --tls-key are not given: every door takes TLS, and nodes call each other over HTTPS
PONDRA_TLS_CA	none	the authority that signed the nodes’ certificates, when --tls-ca is not given: nodes show theirs to each other (mutual TLS), and the nodes’ own key is taken only with one it signed
PONDRA_TLS	plain from this machine only	optional: a node with a certificate takes plain connections from anywhere too
PONDRA_AUDIT	role,ddl,function	what the audit log keeps, by class: role, ddl, function, write, read, misc, all, or off. Refusals are always kept
PONDRA_AUDIT_DAYS	90	how many days pondra.audit keeps. Read when the log is first made
PONDRA_HISTORY	on	off: this node writes nothing to pondra.history, the row every statement leaves
PONDRA_HISTORY_RATE	500	at most this many rows a second a node in pondra.history; past it, fast statements that went well are counted in one row (class = 'skipped'). Slow and failed ones are always written
PONDRA_HISTORY_DAYS	7	how many days pondra.history keeps. Read when it is first made
PONDRA_SLOW_MS	1000	a statement this slow keeps its plan and trace in pondra.history and writes a line to the node’s log
PONDRA_LEARN_MS	100	a query this slow notes in pondra.history how far its joins’ rows were from what was expected (misestimate), ten times off keeping its plan too, and the filters it found 2× or more off (learned, pondra.learned)
PONDRA_USER_QUERIES	none	how many statements a user without its own MAX_QUERIES runs at once on a node (quotas)
PONDRA_USER_TIMEOUT	none	how many seconds a statement of a user without its own STATEMENT_TIMEOUT may run

An empty value counts as not set. See HTTP for what each token may do.

PONDRA_ADMIN_TOKEN="$ADMIN" PONDRA_WRITE_TOKEN="$WRITE" PONDRA_READ_TOKEN="$READ" \
PONDRA_SECRET_KEY="$SECRET" pondra serve lake

Finding things

Variable	Default	Meaning
PONDRA_BIN	the binary the package installed, or the one on PATH	the pondra executable that the Python and JavaScript clients start with local()
PONDRA_PYTHON	none	the first Python that --python auto tries. It needs the pondra package and pyarrow. auto then tries the one chosen in the console, those beside the pondra binary, those on the PATH, Windows’ py launcher’s, and Anaconda’s and Miniconda’s (their environments too), all at once, and takes the first that works
PONDRA_PYTHON_PROBE_SECS	15	how long --python auto (and the console’s Choose the Python…) waits for each Python it tries. One that doesn’t answer in time, such as a broken install, is passed over
PONDRA_WORKER_START_SECS	60	how long a new Python worker has to start and answer. Past it, the error says so, with what the worker printed

Memory and disk

Variable	Default	Meaning
PONDRA_MEMORY_GB	a third of the machine’s memory	memory for queries, when --memory-gb is not given. A sort, join or aggregation that needs more spills to disk
PONDRA_CACHE_DIR	<temp dir>/pondra-cache	where a node keeps its local SSD copy of a lake’s recent objects, what a shuffle spills, and a follower’s copy of the log with --ack replicated. The same as --cache-dir
PONDRA_CACHE_GB	20, or a quarter of the free disk if that is less	the size of that SSD copy. 0 turns it off. The same as --cache-gb
PONDRA_CACHE_MB	1024	the query read cache in memory
PONDRA_HOT_GB	a quarter of the query memory	the most the cache of hot columns may hold
PONDRA_FILES_CACHE_MB	512	the memory cache for files read from outside the lake, such as attached Delta and Iceberg tables and URLs
PONDRA_FUNCTION_CACHE_MB	256	the memory for answers a function keeps with cache = '…'
PONDRA_TEMP_MB	1024	the memory all temporary tables on a node may hold together. A change past it is refused
PONDRA_PAGES_MB	256	the memory for answers of more than 10,000 rows that a node keeps, so the console turns their pages (GET /sql/pages/{id}) without running them again. Each is kept 20 minutes after it was last read; past this size, the least lately read go first
PONDRA_CONFIG_DIR	%APPDATA%\pondra on Windows, ~/Library/Application Support/pondra on a Mac, ~/.config/pondra elsewhere	where a node keeps what it keeps for its machine: the console’s settings (console.json) and the Python chosen in it (python.txt)
PONDRA_CONSOLE_EXTENSIONS	none	scripts the console loads after its own, separated as PATH is: an extension adds sections, tabs, views of answers and actions. Served at /console/ext/{n}.js
PONDRA_SESSION_IDLE_SECS	3600	a session idle this long ends. Its temporary tables go, and so does its Python, with the variables its DO LANGUAGE python blocks made
PONDRA_TXN_IDLE_SECS	600	a transaction idle this long ends, rolled back (Transactions)
PONDRA_TRACE_PG	unset	1: where each Postgres statement’s time goes, on the node’s stderr
PONDRA_DATABASE_IDLE_SECS	600	serving a folder of lakes: a database’s node with no connection open and no request on its way, that nothing has used for this long, and that has nothing left to tier, stops. The next query starts it again
PONDRA_CODEC	lz4	how data files are compressed: lz4 (fastest to read), zstd (smaller files), snappy or none

Queries across nodes

A query is spread over the nodes of a cluster when that is cheaper than running it on one.

Variable	Default	Meaning
PONDRA_CORES	this machine’s cores	the number of cores a node counts: a query runs one partition for each. Set it to stand for another machine’s count
PONDRA_SPREAD_MB	256	tables smaller than this are not spread over nodes
PONDRA_BROADCAST_MB	64	in a shuffle, tables smaller than this are read whole by every node, so a join broadcasts them. 0 slices every table
PONDRA_SKEW_MB	64	a partition bigger than this, and than twice the average, is shared out among nodes
PONDRA_SPILL_MB	64	rows a shuffle holds in memory before they go to disk
PONDRA_LINK	measured	the network between nodes as ms,MB/s (0.5,1000), instead of measuring it every 10 minutes
PONDRA_JOIN_ORDER	2	how much cheaper a new join order must look before it is taken. 0 turns join reordering off
PONDRA_SQL_OPTIONS	none	DataFusion settings, key=value separated by commas, applied after Pondra’s own defaults

Stopping

Variable	Default	Meaning
PONDRA_DRAIN_GRACE_SECS	0	a node told to stop answers GET /ready with 503 for this long before it turns requests away, so a load balancer notices first (upgrades)
PONDRA_DRAIN_SECS	30	how long a stopping node waits for the requests that were running when it was told to stop to finish. Then a leader steps down and the node stops

Housekeeping

Variable	Default	Meaning
PONDRA_GC_SECS	60	how often the catalog’s write-ahead-log objects and old manifests are cleared, and how old they must be
PONDRA_RUNS_DAYS	30	how many days the run log, pondra.runs, keeps. Read when the log is first made
PONDRA_BUCKET_REQUESTS	256	how many requests a node sends a bucket at once to start with. It halves when the bucket answers 503 or 429, and grows back, up to four times this
PONDRA_FILE_VERSIONS	50	how many versions of each of the lake’s files are kept (versions)
PONDRA_FILE_VERSIONS_DAYS	90	how many days a file’s versions are kept (its newest always is)
PONDRA_PURGE_ROWS	100000	after UPDATE, DELETE and MERGE on an append table, its files are rewritten without the old rows once this many, or a tenth of the table, are waiting. A table that publishes a format is rewritten every round
PONDRA_INLINE_KB	64, or 1024 with --ack replicated and --replicas above 1	a write smaller than this is carried inside the commit. A bigger one is written as its own object first

Python workers

Variable	Default	Meaning
PONDRA_PROCEDURES	four times the cores	how many Python procedures a node runs at once. Functions run one for each core
PONDRA_WORKER_MB	2048	a Python worker that has grown past this many MB is stopped after its job. It also governs a session’s Python: past it, the session’s next cell starts empty, and a notice says so
PONDRA_WORKER_IDLE_SECS	60	a Python worker idle this long is stopped

AI functions

Variable	Default	Meaning
PONDRA_AI_URL	http://127.0.0.1:11434/v1	the OpenAI-compatible endpoint that ai_complete and ai_embed call
PONDRA_AI_KEY	none	sent as a bearer token to that endpoint, if it wants one
PONDRA_AI_MODEL	llama3.2	the model ai_complete uses when the call names none
PONDRA_EMBED_MODEL	nomic-embed-text	the model ai_embed uses when the call names none
PONDRA_AI_CALLS	16	requests a node has in flight to that endpoint at most, from all its queries together
PONDRA_AI_TIMEOUT_SECS	300	how long one request may take before its rows are NULL

Writes with pondra sql

PONDRA_JOB is the id of a write made with pondra sql. A write repeated with the same id is applied once and answers {"duplicate":true}. Left unset, each write gets an id of its own.

pondra sql --dir jobs-lake "CREATE TABLE loads (n BIGINT)"
PONDRA_JOB=load-1 pondra sql --dir jobs-lake "INSERT INTO loads VALUES (1), (2)"
PONDRA_JOB=load-1 pondra sql --dir jobs-lake "INSERT INTO loads VALUES (1), (2)"
pondra sql --dir jobs-lake "SELECT count(*) AS n FROM loads"

{"table":"loads","publish":[]}
{"committed":true}
{"duplicate":true}
+---+
| n |
+---+
| 2 |
+---+

Object stores

A lake on object storage reads its credentials and endpoint from the environment, as each cloud’s own tools do: AWS_* for s3:// (AWS_ENDPOINT for R2 and MinIO), GOOGLE_* for gs://, and AZURE_* for az://, abfs:// and abfss://. Pondra does not read them any other way.

Set by Pondra

Do not set these. Pondra sets them for its own processes, and some are copies of flags.

Variable	Set by	Meaning
PONDRA_OWNER_KEY	the shell, pondra.local() and local() in JavaScript	a random key that lets the program that started a node read files on this machine through it. Nobody else may
PONDRA_PUBLISH, PONDRA_CHANGELOG_SECS, PONDRA_FSYNC, PONDRA_REPLICAS	pondra serve, from --publish, --changelog-secs, --fsync, --ack and --replicas	the flags’ values, for the rest of the node. Setting them in the environment has no effect: serve overwrites them. Use the flags
PONDRA_TRIES	pondra sql	how many times a write has started over after another node took the lake
PONDRA_SERVER_URL	pondra serve on a folder of lakes, for each database’s node	where the process that serves the folder is, for DROP DATABASE

Caution

PONDRA_CRASH, PONDRA_DROP_BEATS, PONDRA_NO_DIRECT, PONDRA_DEBUG_SPREAD (why each query ran on one node or across them, with the estimate that decided), PONDRA_DEBUG_JOIN_ORDER (each join order considered: the rows the order written moves, the rows the order chosen moves, and each input’s size) and PONDRA_DEBUG_PG (which prints each statement the Postgres port gets, as it arrives and as it runs) are for tests and debugging. PONDRA_CRASH aborts the process at chosen points, at random, to test recovery. Never set it on a node that matters.