Performance Tuning#

Tips for maximising throughput and getting clean benchmark results.

Batch Size#

For seed operations using exec_batch, the size field controls how many rows are grouped into each SQL statement. This has a significant impact on insert throughput:

Start with size: 1000 and adjust based on your row width and database. Wide rows (many columns, large strings) benefit from smaller batches.

seed {
  populate_users(count: 1000000, size: 5000) `
    INSERT INTO users (email)
    __values__
  ` (gen('email'))
}

Seed Workers#

For seed operations, batch generation and query execution are pipelined: one goroutine generates batches while worker goroutines execute them concurrently. This keeps memory bounded (at most 2 batches in memory regardless of total row count) and lets you saturate database throughput with multiple inserters.

Use the --workers flag on the seed command or the per-query workers field in your config:

edg seed --workers 4 --driver pgx --config workload.edg --url ${DATABASE_URL}

seed {
  populate_users(count: 1000000, size: 5000, workers: 4) `
    INSERT INTO users (email)
    __values__
  ` (gen('email'))
}

The --workers CLI flag overrides per-query workers values. Only exec_batch queries support concurrent workers; query_batch queries with RETURNING data always run with a single worker.

Workers vs Pool Size#

The --workers flag controls concurrency (goroutines), while --pool-size controls the number of database connections. This applies to both run and seed commands:

For benchmarks measuring peak throughput, leave pool-size at the default. For benchmarks simulating production conditions, set pool-size to match your application’s connection pool.

Connections are recycled after --max-conn-lifetime (default 30m) plus up to --max-conn-lifetime-jitter (default 5m). For a long run against a cluster this keeps connections rebalancing across nodes; for a short benchmark neither will fire. See Connection Pool for driver support.

Prepared Statements#

Setting prepared: true on run queries reduces server-side parse overhead by caching the query plan per worker:

run {
  lookup_product(prepared: true) `
    SELECT id, name, price FROM product WHERE id = $1
  ` (ref('fetch_products').id)
}

The benefit scales with query complexity. Simple point lookups see minimal improvement, but multi-table joins with aggregations can see 20-30% latency reduction.

Prepared statements are not compatible with batch types (query_batch, exec_batch) or queries inside transactions.

Warmup Period#

Database caches, JIT compilation, and connection establishment all affect early query latencies. Use --warmup-duration to run the workload for a period before collecting metrics:

edg run \
  --warmup-duration 30s \
  --duration 5m \
  -w 10 \
  ...

Workers run during warmup but results are discarded. This produces more representative p90/p95/p99 numbers by excluding cold-start outliers.

Deterministic Seeding#

Use --rng-seed to make generated data deterministic. This eliminates variability between runs, making it easier to compare benchmark results:

edg all --rng-seed 42 --driver pgx --config workload.edg --url ${DATABASE_URL} -w 10 -d 5m

Two runs with the same seed produce identical seed data and identical random selections during the run phase.

Distribution Selection#

Choosing the right distribution for your workload matters more than raw throughput numbers:

Zipfian distributions with s=1.1 to s=2.0 are the most common for realistic OLTP benchmarks. Higher s values create more contention.

Run Weights#

Use run_weights to control the read/write mix rather than duplicating queries:

weights {
  read_order = 80
  update_status = 15
  insert_order = 5
}

This produces 80% reads, 15% updates, and 5% inserts, a realistic OLTP mix. Without run_weights, all queries execute sequentially on every iteration.

Metrics Samples#

Percentile calculations (p90, p95, p99) use a sliding window of the most recent latency samples. By default, edg keeps the last 10,000 samples per query. For long-running benchmarks this keeps memory bounded while still producing accurate percentiles.

Increase the window if you need percentiles calculated over a larger history, or decrease it to reduce memory usage with many concurrent queries:

edg run \
  --metrics-samples 50000 \
  --driver pgx \
  --config workload.edg \
  --url ${DATABASE_URL} \
  -w 100 \
  -d 1h

Count, average, QPS, and error metrics are always exact regardless of this setting.

Monitoring During Runs#

Use --metrics-addr to expose Prometheus metrics for real-time monitoring:

edg run --metrics-addr :9090 --driver pgx --config workload.edg --url ${DATABASE_URL} -w 10 -d 5m

This lets you correlate edg metrics with database-side metrics in Grafana. See Observability for dashboard setup.

Stages for Load Profiling#

Use stages to vary worker counts over time and observe how the database responds to changing load. Each stage can also override run_weights to shift the workload mix per phase:

stages {
  ramp(workers: 1, duration: 30s, weights: {read_order: 95, insert_order: 5})
  low(workers: 10, duration: 2m)
  peak(workers: 100, duration: 5m, weights: {read_order: 60, update_status: 25, insert_order: 15})
  cooldown(workers: 10, duration: 2m)
}

weights {
  read_order = 80
  update_status = 15
  insert_order = 5
}

Stages without run_weights fall back to the top-level weights. Combine with Prometheus metrics to produce load-vs-latency curves that reveal database scaling characteristics.

QPS Rate Limiting#

Use the qps field on a stage to cap total queries per second across all workers. This is useful for testing how your database behaves at a specific throughput target rather than running flat out:

stages {
  steady(workers: 50, duration: 5m, qps: 1000)
  burst(workers: 50, duration: 2m, qps: 5000)
}

Without qps, workers fire queries as fast as the database can respond. With qps set, workers wait for a shared rate limiter before each iteration, producing a steady and predictable load. This makes it easier to correlate latency changes with a known request rate.

QPS rate limiting applies per stage. Set workers high enough that they can sustain the target rate - if you set qps: 5000 with 2 workers, you won’t reach 5,000 QPS if each query takes more than 400 microseconds.

QPS Ramping#

Use ramp(start, end, duration) to linearly increase QPS over time instead of jumping straight to the target rate:

stages {
  warmup(workers: 50, duration: 5m, qps: ramp(10, 1000, 60s))
  sustain(workers: 50, duration: 30m, qps: 1000)
}

This ramps QPS from 10 to 1,000 over the first 60 seconds, then holds at 1,000 for the remainder. All workers start immediately; only the rate limit changes.

QPS ramping via ramp() is independent of ramp_duration. When both are set, ramp_duration controls worker staggering while ramp() controls the QPS curve.

Breakpoint Testing PRO#

Use edg perf breakpoint to automatically find the point at which your database starts to degrade. It starts with one worker and adds another every interval, monitoring p99 latency, per-worker QPS, and error count. When metrics degrade past a threshold, it warns you. When absolute limits are hit, it stops.

edg perf breakpoint \
  --driver pgx \
  --config workload.edg \
  --url ${DATABASE_URL} \
  --ramp-interval 10s \
  --threshold 30 \
  --stop-p99 100ms

This tells you how many concurrent workers your database can handle before latency becomes unacceptable. Combine with --pool-size to test under realistic connection limits.

See Breakpoint Testing for full flag reference and examples.