Configuration#

Workloads are defined across one or more config files. YAML (.yaml) and edg-lang (.edg) are fully supported and produce the same result. edg-lang is more compact and offers syntax highlighting and autocompletion. See the edg-lang page for a full syntax guide.

# Variables available in all expressions.
let name = value

# User-defined expression functions.
expr name = expression

# Reusable arg templates for queries.
object name {
  field = expr
}

# Named query templates for reducing boilerplate.
template name(options)

# Static datasets available to ref_* functions.
ref name [{key: val}, {key: val}]

# Named auto-incrementing sequences.
seq name(start: 1, step: 1)

# Staged workload execution.
stages {
  name(workers: N, duration: D [, ramp_duration: R] [, qps: Q])
}

# Schema creation queries.
up { query_name `SQL` }

# Data population queries.
seed { query_name `SQL` (args) }

# Data cleanup queries.
deseed { query_name `SQL` }

# Schema teardown queries.
down { query_name `SQL` }

# Per-worker initialisation queries.
init { query_name `SQL` }

# Weighted transaction mix.
weights { name = N }

# Workload queries.
run { query_name `SQL` (args) }

# Background queries on a fixed schedule.
workers { name(rate: 1/10s) `SQL` }

# Post-run assertions.
expect { metric_name < threshold }

# Variables available in all expressions.
globals:

# User-defined expression functions.
expressions:

# Reusable arg templates for queries.
objects:

# Named query templates for reducing boilerplate.
templates:

# Static datasets available to ref_* functions without a database query.
reference:

# Named auto-incrementing sequences shared across all workers.
seq:

# Staged workload execution (overridden by explicit -w and -d flags).
stages:

# Schema creation queries.
up:

# Data population queries.
seed:

# Data cleanup queries.
deseed:

# Schema teardown queries.
down:

# Per-worker initialisation queries (run before workload).
init:

# Weighted transaction mix (optional).
run_weights:

# Workload queries (standalone or grouped into transactions).
run:

# Background queries that run independently on a fixed schedule.
workers:

# Post-run assertions for CI/CD (exit non-zero on failure).
expectations:

Globals#

The globals section defines top-level variables available in all expressions:

let warehouses = 1
let districts = 10
let customers = 30000
let items = 100000

globals:
  warehouses: 1
  districts: 10
  customers: 30000
  items: 100000

These can be referenced directly in arg expressions, including in arithmetic:

query_name `SELECT ... WHERE district_id = $1 AND warehouse_id = $2` (
  customers / districts, # evaluates to 3000
  warehouses * 10        # evaluates to 10
)

args:
  - customers / districts # evaluates to 3000
  - warehouses * 10       # evaluates to 10

Expression-valued globals#

Global values can be expressions, including references to other globals and environment variables. String values are compiled as expressions; if compilation fails (e.g. a plain string like "new york"), the value is kept as a literal.

Globals are evaluated in the order they appear in config, so later globals can reference earlier ones:

let warehouses = 1
let districts = warehouses * 10
let customers = districts * 3000

globals:
  warehouses: 1
  districts: warehouses * 10  # evaluates to 10
  customers: districts * 3000 # evaluates to 30000

Globals from environment variables#

Use env() to read a required environment variable, or env_nil() for an optional one. Combine env_nil() with coalesce() to provide a default:

# Required (fails at startup if DB_BATCH_SIZE is not set).
let batch_size = env('DB_BATCH_SIZE')

# Optional (falls back to 10000 if CUSTOMERS is not set).
let customers = int(coalesce(env_nil('CUSTOMERS'), 10000))

globals:
  # Required (fails at startup if DB_BATCH_SIZE is not set).
  batch_size: env('DB_BATCH_SIZE')

  # Optional (falls back to 10000 if CUSTOMERS is not set).
  customers: int(coalesce(env_nil('CUSTOMERS'), 10000))

env() returns a string. Wrap with int() or float() if arithmetic is needed downstream.

Objects#

The objects section defines reusable named arg templates. Each key is an object name, and the value is a map of field names to expressions. Queries reference an object using the object field instead of duplicating args. Fields are bound to query parameters in declaration order ($1, $2, …):

object customer {
  email = gen('email')
  name = gen('name')
  created_at = timestamp('2020-01-01T00:00:00Z', '2024-01-01T00:00:00Z')
}

objects:
  customer:
    email: gen('email')
    name: gen('name')
    created_at: timestamp('2020-01-01T00:00:00Z', '2024-01-01T00:00:00Z')

See the Objects page for full documentation, including field(), obj(), sub-objects, aggregate functions, and captured sub-items.

Templates#

The templates section defines named query templates that reduce boilerplate. A template is a partial query definition; any query can reference it with template: and inherit its fields. Fields set on the query override the template.

template batch_insert(count: batch_size, size: 1000)

seed {
  seed_users(template: batch_insert, object: user)
    `INSERT INTO users __columns__ __values__`

  seed_orders(template: batch_insert, object: order)
    `INSERT INTO orders __columns__ __values__`
}

templates:
  batch_insert:
    count: batch_size
    size: 1000

seed:
  - name: seed_users
    template: batch_insert
    object: user
    query: INSERT INTO users __columns__ __values__

  - name: seed_orders
    template: batch_insert
    object: order
    query: INSERT INTO orders __columns__ __values__

Template fields work as defaults - any field set on the query wins. Referencing an unknown template name is a validation error. Templates can be used in any section (seed, run, workers, transactions, etc.).

Reference#

The reference section defines static datasets that are loaded into the environment at startup, making them available to ref_rand, ref_same, ref_perm, and ref_diff functions without needing an init query or database connection. Each key is a dataset name, and the value is a list of row objects:

ref regions [
  {name: "us", cities: ["a", "b", "c"]}
  {name: "eu", cities: ["d", "e", "f"]}
  {name: "ap", cities: ["g", "h", "i"]}
]

reference:
  regions:
    - {name: us, cities: [a, b, c]}
    - {name: eu, cities: [d, e, f]}
    - {name: ap, cities: [g, h, i]}

Reference datasets work exactly like datasets populated by init queries. You can use them in any arg expression:

query_name `SELECT ... WHERE region = $1 AND city = $2` (
  # Random region row, access the 'name' field.
  ref_rand('regions').name,

  # Same row reused across all ref_same calls in this query execution.
  ref_same('regions').name,
  set_rand(ref_same('regions').cities, [])
)

args:
  # Random region row, access the 'name' field.
  - ref_rand('regions').name

  # Same row reused across all ref_same calls in this query execution.
  - ref_same('regions').name
  - set_rand(ref_same('regions').cities, [])

This is useful when your lookup data is small and known ahead of time, avoiding the need for a database round-trip.

Loading from CSV files#

Instead of inlining reference data in config, you can load it from CSV files at startup using the --csv-file and --csv-directory flags. Each CSV file becomes a reference dataset named after its filename (minus the extension). The header row defines the field names.

edg seed \
  --config workload.edg \
  --csv-file data/regions.csv \
  --csv-file data/translations.csv \
  --url $DATABASE_URL

Or load all CSV files from a directory (non-recursive):

edg seed \
  --config workload.edg \
  --csv-directory data/ \
  --url $DATABASE_URL

A file named regions.csv with this content:

code,name,currency
us,United States,USD
gb,United Kingdom,GBP
de,Germany,EUR

is equivalent to this inline reference block:

ref regions [
  {code: "us", name: "United States", currency: "USD"}
  {code: "gb", name: "United Kingdom", currency: "GBP"}
  {code: "de", name: "Germany", currency: "EUR"}
]

reference:
  regions:
    - {code: us, name: United States, currency: USD}
    - {code: gb, name: United Kingdom, currency: GBP}
    - {code: de, name: Germany, currency: EUR}

Both flags are repeatable and work with all commands (up, seed, run, stage, repl, etc.). If a CSV filename collides with an existing reference: dataset name, edg returns an error.

See examples/csv/ for a complete working example.

Seq#

The seq section defines named auto-incrementing sequences that are shared across all workers. Unlike seq(start, step) which maintains a separate counter per worker, seq_global("name") returns globally unique values using atomic counters.

seq order_id(start: 1, step: 1)
seq line_item_id(start: 1000, step: 5)
seq sku_code(length: 3)

seq:
  - name: order_id
    start: 1
    step: 1
  - name: line_item_id
    start: 1000
    step: 5
  - name: sku_code
    length: 3

Each entry must use either start/step (numeric) or length (alpha), not both.

Use seq_global("name") in any arg expression:

seed {
  seed_orders(count: 1000)
    `INSERT INTO orders (id, name) VALUES ($1, $2)` (
    seq_global("order_id"),
    gen('firstname')
  )
}

seed:
  - name: seed_orders
    type: exec
    count: 1000
    args:
      - seq_global("order_id")
      - gen('firstname')
    query: INSERT INTO orders (id, name) VALUES ($1, $2)

Sequences work in all sections (seed, run, workers) and produce gap-free, globally unique values across concurrent workers. The same sequence can be used in both seed and run; the counter continues from where seeding left off.

To reference a random value that has already been generated by a sequence, use the distribution functions:

These functions compute valid values algebraically from the sequence’s start, step, and current counter without generating and storing each sequence in memory. This is useful for foreign key references where the referenced column was populated by seq_global.

See examples/global_sequences/ for a complete working example but here’s a quick introduction.

Create two tables, one to store 1,000 orders and one to store references to them (using each of the sequence generators):

let orders = 1000
let samples = 10000
let batch_size = 100

seq order_id(start: 1, step: 1)

up {
  create_orders `CREATE TABLE IF NOT EXISTS orders (
    id INT PRIMARY KEY,
    customer STRING NOT NULL
  )`

  create_samples `CREATE TABLE IF NOT EXISTS samples (
    id INT PRIMARY KEY DEFAULT unique_rowid(),
    uniform_val INT NOT NULL,
    zipf_val INT NOT NULL,
    norm_val INT NOT NULL,
    exp_val INT NOT NULL,
    lognorm_val INT NOT NULL
  )`
}

seed {
  seed_orders(count: orders, size: batch_size)
    `INSERT INTO orders (id, customer) __values__` (
    seq_global("order_id"),
    gen('firstname') + ' ' + gen('lastname')
  )

  seed_samples(count: samples, size: batch_size)
    `INSERT INTO samples (uniform_val, zipf_val, norm_val, exp_val, lognorm_val) __values__` (
    seq_rand("order_id"),
    seq_zipf("order_id", 1.1, 1.0),
    seq_norm("order_id", 500, 150),
    seq_exp("order_id", 0.01),
    seq_lognorm("order_id", 5.5, 0.5)
  )
}

globals:
  orders: 1000
  samples: 10000
  batch_size: 100

seq:
  - name: order_id
    start: 1
    step: 1

up:

  - name: create_orders
    query: |-
      CREATE TABLE IF NOT EXISTS orders (
        id INT PRIMARY KEY,
        customer STRING NOT NULL
      )

  - name: create_samples
    query: |-
      CREATE TABLE IF NOT EXISTS samples (
        id INT PRIMARY KEY DEFAULT unique_rowid(),
        uniform_val INT NOT NULL,
        zipf_val INT NOT NULL,
        norm_val INT NOT NULL,
        exp_val INT NOT NULL,
        lognorm_val INT NOT NULL
      )

seed:

  - name: seed_orders
    type: exec_batch
    count: orders
    size: batch_size
    args:
      - seq_global("order_id")
      - gen('firstname') + ' ' + gen('lastname')
    query: |-
      INSERT INTO orders (id, customer)
      __values__

  - name: seed_samples
    type: exec_batch
    count: samples
    size: batch_size
    args:
      - seq_rand("order_id")
      - seq_zipf("order_id", 1.1, 1.0)
      - seq_norm("order_id", 500, 150)
      - seq_exp("order_id", 0.01)
      - seq_lognorm("order_id", 5.5, 0.5)
    query: |-
      INSERT INTO samples (uniform_val, zipf_val, norm_val, exp_val, lognorm_val)
      __values__

After seeding, the samples table will show various distributions of order ids (all with complete referential integrity to the orders table) and can be queried to show their distribution as follows:

-- Uniform distribution.
SELECT
  div(uniform_val - 1, 50) * 50 + 1 AS bucket,
  count(*) AS total,
  repeat('█', (count(*) * 50 / max(count(*)) OVER ())::INT) AS histogram
FROM samples
GROUP BY 1
ORDER BY 1;

  bucket | total |                     histogram
---------+-------+-----------------------------------------------------
       1 |   481 | █████████████████████████████████████████████
      51 |   484 | █████████████████████████████████████████████
     101 |   496 | ███████████████████████████████████████████████
     151 |   465 | ████████████████████████████████████████████
     201 |   492 | ██████████████████████████████████████████████
     251 |   510 | ████████████████████████████████████████████████
     301 |   516 | ████████████████████████████████████████████████
     351 |   513 | ████████████████████████████████████████████████
     401 |   471 | ████████████████████████████████████████████
     451 |   533 | ██████████████████████████████████████████████████
     501 |   519 | █████████████████████████████████████████████████
     551 |   498 | ███████████████████████████████████████████████
     601 |   522 | █████████████████████████████████████████████████
     651 |   518 | █████████████████████████████████████████████████
     701 |   492 | ██████████████████████████████████████████████
     751 |   473 | ████████████████████████████████████████████
     801 |   522 | █████████████████████████████████████████████████
     851 |   515 | ████████████████████████████████████████████████
     901 |   472 | ████████████████████████████████████████████
     951 |   508 | ████████████████████████████████████████████████

-- Normal distribution.
SELECT
  div(norm_val - 1, 50) * 50 + 1 AS bucket,
  count(*) AS total,
  repeat('█', (count(*) * 50 / max(count(*)) OVER ())::INT) AS histogram
FROM samples
GROUP BY 1
ORDER BY 1;

  bucket | total |                     histogram
---------+-------+-----------------------------------------------------
       1 |    10 |
      51 |    23 | █
     101 |    59 | ██
     151 |   142 | █████
     201 |   254 | █████████
     251 |   390 | ██████████████
     301 |   667 | ████████████████████████
     351 |   905 | █████████████████████████████████
     401 |  1159 | ██████████████████████████████████████████
     451 |  1380 | ██████████████████████████████████████████████████
     501 |  1362 | █████████████████████████████████████████████████
     551 |  1162 | ██████████████████████████████████████████
     601 |   917 | █████████████████████████████████
     651 |   652 | ████████████████████████
     701 |   440 | ████████████████
     751 |   263 | ██████████
     801 |   128 | █████
     851 |    57 | ██
     901 |    27 | █
     951 |     3 |

-- Exponential distribution.
SELECT
  div(exp_val - 1, 50) * 50 + 1 AS bucket,
  count(*) AS total,
  repeat('█', (count(*) * 50 / max(count(*)) OVER ())::INT) AS histogram
FROM samples
GROUP BY 1
ORDER BY 1;

  bucket | total |                     histogram
---------+-------+-----------------------------------------------------
       1 |  3961 | ██████████████████████████████████████████████████
      51 |  2436 | ███████████████████████████████
     101 |  1475 | ███████████████████
     151 |   782 | ██████████
     201 |   532 | ███████
     251 |   329 | ████
     301 |   207 | ███
     351 |   111 | █
     401 |    66 | █
     451 |    40 | █
     501 |    26 |
     551 |    19 |
     601 |     7 |
     651 |     1 |
     701 |     1 |
     751 |     5 |
     851 |     1 |
     901 |     1 |

-- Log-normal distribution.
SELECT
  div(lognorm_val - 1, 50) * 50 + 1 AS bucket,
  count(*) AS total,
  repeat('█', (count(*) * 50 / max(count(*)) OVER ())::INT) AS histogram
FROM samples
GROUP BY 1
ORDER BY 1;

  bucket | total |                     histogram
---------+-------+-----------------------------------------------------
       1 |    10 |
      51 |   342 | █████████
     101 |  1293 | ███████████████████████████████████
     151 |  1836 | ██████████████████████████████████████████████████
     201 |  1699 | ██████████████████████████████████████████████
     251 |  1425 | ███████████████████████████████████████
     301 |  1024 | ████████████████████████████
     351 |   767 | █████████████████████
     401 |   516 | ██████████████
     451 |   335 | █████████
     501 |   238 | ██████
     551 |   147 | ████
     601 |   121 | ███
     651 |    81 | ██
     701 |    71 | ██
     751 |    38 | █
     801 |    25 | █
     851 |    15 |
     901 |    13 |
     951 |     4 |

-- Zipfian distribution.
SELECT
  div(zipf_val - 1, 50) * 50 + 1 AS bucket,
  count(*) AS total,
  repeat('█', (count(*) * 50 / max(count(*)) OVER ())::INT) AS histogram
FROM samples
GROUP BY 1
ORDER BY 1;

  bucket | total |                     histogram
---------+-------+-----------------------------------------------------
       1 |  6904 | ██████████████████████████████████████████████████
      51 |   778 | ██████
     101 |   458 | ███
     151 |   312 | ██
     201 |   255 | ██
     251 |   177 | █
     301 |   146 | █
     351 |   112 | █
     401 |   106 | █
     451 |    77 | █
     501 |   104 | █
     551 |    76 | █
     601 |    79 | █
     651 |    87 | █
     701 |    56 |
     751 |    64 |
     801 |    66 |
     851 |    41 |
     901 |    54 |
     951 |    48 |

Stages#

The stages section defines a sequence of workload phases, each with its own worker count and duration. Explicitly passing -w or -d on the command line overrides the stages section, falling back to single-stage mode with the provided values.

stages {
  ramp(workers: 1, duration: 10s)
  steady(workers: 10, duration: 30s, ramp_duration: 5s)
  cooldown(workers: 2, duration: 10s)
}

stages:
  - name: ramp
    workers: 1
    duration: 10s
  - name: steady
    workers: 10
    duration: 30s
    ramp_duration: 5s
  - name: cooldown
    workers: 2
    duration: 10s

Each stage runs sequentially. When a stage completes (its duration expires), the next stage starts immediately with a new set of workers. The init section runs once before the first stage, and its results are shared across all stages.

Per-stage run weights#

Each stage can optionally define its own run_weights to change the workload mix for that phase. When a stage omits run_weights, the top-level run_weights apply. When neither exists, all run items execute sequentially.

stages {
  ramp(workers: 1, duration: 10s, weights: {check_balance: 90, credit_account: 5, make_transfer: 5})
  steady(workers: 10, duration: 30s)
  # steady falls back to top-level weights
}

weights {
  check_balance = 50
  credit_account = 5
  make_transfer = 45
}

stages:
  - name: ramp
    workers: 1
    duration: 10s
    run_weights:
      check_balance: 90
      credit_account: 5
      make_transfer: 5
  - name: steady
    workers: 10
    duration: 30s
    # Falls back to top-level run_weights

run_weights:
  check_balance: 50
  credit_account: 5
  make_transfer: 45

When ramp_duration is set, the stage linearly ramps up to full capacity over that period instead of starting at full blast. For stages with qps, all workers start immediately and QPS increases linearly. For stages without qps, workers are spawned incrementally over the ramp period.

This is useful for simulating ramp-up patterns, sustained load tests, or multi-phase benchmarks without running separate commands.

See examples/stages/ for basic staged execution and examples/stages_run_weights/ for per-stage weight overrides.

Includes / Imports#

Both formats support splitting configs into reusable fragments:

• YAML uses !include to inline fragments from other YAML files.
• edg-lang uses include for flat file merging (like YAML !include), and import / pub for namespaced sharing across .edg files.

Includes (flat merging)#

Use include (edg-lang) or !include (YAML) to merge another file’s contents directly (without namespacing or visibility restrictions):

include 'shared/globals.edg'
include 'shared/schema.edg'

seed {
  populate_users(count: count, size: batch_size)
    `INSERT INTO users (email) __values__` (gen('email'))
}

globals: !include shared/globals.yaml
up: !include shared/schema.yaml

Imports (namespaced sharing)#

In edg-lang, use import / pub to share declarations with namespacing. The filename (minus .edg) becomes the namespace:

import 'top.edg'
import 'objects/customer.edg' as cust

seed {
  populate_customer(count: top.customers, size: top.batch_size, object: cust.customer)
    `INSERT INTO customer (email) __values__ __columns__`
}

Use as to override the default namespace:

import 'shared/long_filename.edg' as cfg
# now use cfg.customers, cfg.batch_size, etc.

The pub keyword works with: let, object, ref, expr, seq, and template. Only pub-marked declarations are visible to importing files. Imported sections (up, seed, down, init, deseed, run, workers, weights) are also merged, with query names namespaced under the import alias.

YAML !include splicing#

In YAML, !include can replace an entire section or splice items into a list:

up: !include shared/schema.yaml

run:
  - name: local_query
    type: query
    query: SELECT 1
  - !include shared/extra_queries.yaml

Rules#

• Includes and imports must appear before all other declarations.
• Circular includes/imports are detected and produce an error.
• Paths are resolved relative to the file containing the directive; nested includes are supported.

See examples/imports/ (edg-lang imports) and examples/includes/ (YAML includes) for complete working examples.

CSV data (edg-lang only)#

Use csv to declare CSV file dependencies directly in your .edg config. Each CSV file becomes a reference dataset (like ref), named after its filename stem:

csv 'data/regions.csv'
csv 'data/translations.csv'

seed {
  populate_customer(type: exec_batch, count: 10, size: 10)
    `INSERT INTO customer (region, currency, greeting) __values__` (
      ref_same('regions').code,
      ref_same('regions').currency,
      ref_rand('translations').greeting
    )
}

You can also point to a directory to load all .csv files inside it:

csv 'data/'

CSV files must have a header row and at least one data row. The header columns become the field names accessible via dot notation (e.g., .code, .currency).

For those wishing to configure edg workloads with YAML, use the --csv-file and --csv-directory CLI flags to make the edg runtime away of the CSV files. For edg-lang files, The LSP recognises csv directives, so you get:

• Go-to-definition on dataset names (e.g., ctrl-click 'regions' in ref_same('regions') opens the CSV file)
• Path autocompletion when typing the CSV path

See examples/csv/ for a complete working example.