Performance Tuning#

Tips for maximising throughput and getting clean benchmark results.

Batch Size#

For seed operations using exec_batch, the size field controls how many rows are grouped into each SQL statement. This has a significant impact on insert throughput:

Start with size: 1000 and adjust based on your row width and database. Wide rows (many columns, large strings) benefit from smaller batches.

seed {
  populate_users(count: 1000000, size: 5000) `
    INSERT INTO users (email)
    __values__
  ` (gen('email'))
}

seed:
  - name: populate_users
    type: exec_batch
    count: 1000000
    size: 5000
    args:
      - gen('email')
    query: |-
      INSERT INTO users (email)
      __values__

Seed Workers#

For seed operations, batch generation and query execution are pipelined: one goroutine generates batches while worker goroutines execute them concurrently. This keeps memory bounded (at most 2 batches in memory regardless of total row count) and lets you saturate database throughput with multiple inserters.

Use the --workers flag on the seed command or the per-query workers field in your config:

edg seed --workers 4 --driver pgx --config workload.edg --url ${DATABASE_URL}

seed {
  populate_users(count: 1000000, size: 5000, workers: 4) `
    INSERT INTO users (email)
    __values__
  ` (gen('email'))
}

seed:
  - name: populate_users
    type: exec_batch
    count: 1000000
    size: 5000
    workers: 4
    args:
      - gen('email')
    query: |-
      INSERT INTO users (email)
      __values__

The --workers CLI flag overrides per-query workers values. Only exec_batch queries support concurrent workers; query_batch queries with RETURNING data always run with a single worker.

Workers vs Pool Size#

The --workers flag controls concurrency (goroutines), while --pool-size controls the number of database connections. This applies to both run and seed commands:

For benchmarks measuring peak throughput, match pool-size to workers or leave it at the default. For benchmarks simulating production conditions, set pool-size to match your application’s connection pool.

Prepared Statements#

Setting prepared: true on run queries reduces server-side parse overhead by caching the query plan per worker:

run {
  lookup_product(prepared: true) `
    SELECT id, name, price FROM product WHERE id = $1
  ` (ref_rand('fetch_products').id)
}

run:
  - name: lookup_product
    type: query
    prepared: true
    args:
      - ref_rand('fetch_products').id
    query: SELECT id, name, price FROM product WHERE id = $1

The benefit scales with query complexity. Simple point lookups see minimal improvement, but multi-table joins with aggregations can see 20-30% latency reduction.

Prepared statements are not compatible with batch types (query_batch, exec_batch) or queries inside transactions.

Warmup Period#

Database caches, JIT compilation, and connection establishment all affect early query latencies. Use --warmup-duration to run the workload for a period before collecting metrics:

edg run \
  --warmup-duration 30s \
  --duration 5m \
  -w 10 \
  ...

Workers run during warmup but results are discarded. This produces more representative p50/p95/p99 numbers by excluding cold-start outliers.

Deterministic Seeding#

Use --rng-seed to make generated data deterministic. This eliminates variability between runs, making it easier to compare benchmark results:

edg all --rng-seed 42 --driver pgx --config workload.edg --url ${DATABASE_URL} -w 10 -d 5m

Two runs with the same seed produce identical seed data and identical random selections during the run phase.

Distribution Selection#

Choosing the right distribution for your workload matters more than raw throughput numbers:

Zipfian distributions with s=1.1 to s=2.0 are the most common for realistic OLTP benchmarks. Higher s values create more contention.

Run Weights#

Use run_weights to control the read/write mix rather than duplicating queries:

weights {
  read_order = 80
  update_status = 15
  insert_order = 5
}

run_weights:
  read_order: 80
  update_status: 15
  insert_order: 5

This produces 80% reads, 15% updates, and 5% inserts, a realistic OLTP mix. Without run_weights, all queries execute sequentially on every iteration.

Metrics Samples#

Percentile calculations (p50, p95, p99) use a sliding window of the most recent latency samples. By default, edg keeps the last 10,000 samples per query. For long-running benchmarks this keeps memory bounded while still producing accurate percentiles.

Increase the window if you need percentiles calculated over a larger history, or decrease it to reduce memory usage with many concurrent queries:

edg run \
  --metrics-samples 50000 \
  --driver pgx \
  --config workload.edg \
  --url ${DATABASE_URL} \
  -w 100 \
  -d 1h

Count, average, QPS, and error metrics are always exact regardless of this setting.

Monitoring During Runs#

Use --metrics-addr to expose Prometheus metrics for real-time monitoring:

edg run --metrics-addr :9090 --driver pgx --config workload.edg --url ${DATABASE_URL} -w 10 -d 5m

This lets you correlate edg metrics with database-side metrics in Grafana. See Observability for dashboard setup.

Stages for Load Profiling#

Use stages to vary worker counts over time and observe how the database responds to changing load. Each stage can also override run_weights to shift the workload mix per phase:

stages {
  ramp(workers: 1, duration: 30s, weights: {read_order: 95, insert_order: 5})
  low(workers: 10, duration: 2m)
  peak(workers: 100, duration: 5m, weights: {read_order: 60, update_status: 25, insert_order: 15})
  cooldown(workers: 10, duration: 2m)
}

weights {
  read_order = 80
  update_status = 15
  insert_order = 5
}

stages:
  - name: ramp
    workers: 1
    duration: 30s
    run_weights:
      read_order: 95
      insert_order: 5
  - name: low
    workers: 10
    duration: 2m
  - name: peak
    workers: 100
    duration: 5m
    run_weights:
      read_order: 60
      update_status: 25
      insert_order: 15
  - name: cooldown
    workers: 10
    duration: 2m

run_weights:
  read_order: 80
  update_status: 15
  insert_order: 5

Stages without run_weights fall back to the top-level weights. Combine with Prometheus metrics to produce load-vs-latency curves that reveal database scaling characteristics.

QPS Rate Limiting#

Use the qps field on a stage to cap total queries per second across all workers. This is useful for testing how your database behaves at a specific throughput target rather than running flat out:

stages {
  steady(workers: 50, duration: 5m, qps: 1000)
  burst(workers: 50, duration: 2m, qps: 5000)
}

stages:
  - name: steady
    workers: 50
    duration: 5m
    qps: 1000
  - name: burst
    workers: 50
    duration: 2m
    qps: 5000

Without qps, workers fire queries as fast as the database can respond. With qps set, workers wait for a shared rate limiter before each iteration, producing a steady and predictable load. This makes it easier to correlate latency changes with a known request rate.

QPS rate limiting applies per stage. Set workers high enough that they can sustain the target rate - if you set qps: 5000 with 2 workers, you won’t reah 5,000 QPS if each query takes more than 400 microseconds.

Breakpoint Testing PRO#

Use edg perf breakpoint to automatically find the point at which your database starts to degrade. It starts with one worker and adds another every interval, monitoring p99 latency, per-worker QPS, and error count. When metrics degrade past a threshold, it warns you. When absolute limits are hit, it stops.

edg perf breakpoint \
  --driver pgx \
  --config workload.edg \
  --url ${DATABASE_URL} \
  --ramp-interval 10s \
  --threshold 30 \
  --stop-p99 100ms

This tells you how many concurrent workers your database can handle before latency becomes unacceptable. Combine with --pool-size to test under realistic connection limits.

See Breakpoint Testing for full flag reference and examples.