edg-lang Specification#
Version: 1.0 Status: Draft
edg-lang is a domain-specific language for defining database workloads: schema setup, data generation, query execution, and load testing. It compiles to an internal Request structure that drives the edg runtime.
On this page: 1. Lexical Structure · 2. Source File Structure · 3. Includes and Imports · 4. Declarations · 5. Sections · 6. Queries · 7. Transactions · 8. Control Flow · 9. Weights · 10. Expectations · 11. Stages · 12. Workers · 13. Stats · 14. Trees · 15. Complete (AI Generation) · 16. Expressions · 17. Built-in Functions · 18. Query Type Inference · 19. Visibility and pub · EBNF Grammar · Keywords
1. Lexical Structure#
1.1 Source Encoding#
Source files are UTF-8 encoded. The lexer operates on byte sequences but supports Unicode letters in identifiers.
1.2 Line Termination#
Newlines (\n, \r\n, \r) are significant tokens - they terminate expressions and declarations. Newlines inside parentheses, brackets, braces, and backtick-delimited strings are consumed as whitespace.
1.3 Whitespace#
Spaces and tabs between tokens are insignificant and ignored (except inside strings and raw strings).
1.4 Comments#
Line comments begin with # and extend to the end of the line:
# This is a comment
let batch_size = 1000 # inline commentThere are no block comments.
1.5 Identifiers#
identifier = letter { letter | digit | "_" } .
letter = "a"…"z" | "A"…"Z" | "_" | <Unicode letter> .
digit = "0"…"9" .Identifiers name variables, objects, queries, fields, and other declarations. Keywords may be used as identifiers in positions where identifiers are expected (query names, field names, option keys).
1.6 Numeric Literals#
number = [ "-" ] digit { digit } [ "." digit { digit } ] .Numbers are signed integers or floating-point values. A leading - is part of the number token only when followed immediately by a digit.
Examples: 42, -1, 3.14, -0.5
1.7 String Literals#
Strings are delimited by single quotes (') or double quotes ("). Backslash escapes (\\, \', \") are supported. Strings may span multiple lines.
'hello world'
"it's a string"
'escape \'this\''1.8 Raw Strings (SQL)#
Backtick-delimited strings preserve all content verbatim, including whitespace and newlines. They are used for SQL, CQL, and MongoDB query bodies.
`SELECT id, name FROM users WHERE id = $1`Multi-line raw strings are automatically dedented: the minimum leading whitespace of non-first lines is removed.
1.9 Operators#
Single-character: +, -, *, /, %, <, >, !, &, |, ^
Multi-character: <=, >=, !=, ==, &&, ||
Operators appear inside expressions, which are evaluated by the expr-lang engine at runtime.
1.10 Punctuation#
| Token | Meaning |
|---|---|
{ } | Block delimiters (sections, objects, ref rows, weights, etc.) |
( ) | Options, arguments, function calls |
[ ] | Array/list delimiters (ref data, tree levels) |
= | Assignment (let, field, weight) |
, | Separator (arguments, options, ref fields) |
: | Key-value separator (options, ref rows, named args) |
. | Member access (in expressions) |
; | Expression separator |
2. Source File Structure#
An edg source file consists of the following sections, in order:
- Includes / Imports - must appear before all other declarations
- Declarations -
let,object,ref,seq,expr,signal,season,template - Sections -
up,seed,deseed,down,init,run - Configuration blocks -
weights,expect,stages,workers,tree,complete
Declarations and configuration blocks may be interleaved after includes/imports, but includes and imports must precede everything else.
Each section may appear multiple times; entries are appended.
edg fmt rewrites a file into a single canonical block order. See CLI Reference > Block order.
3. Includes and Imports#
3.1 Includes#
include "path/to/file.edg"pathis relative to the including file.- All declarations from the included file are merged directly (no namespacing or
pubrequired). - Merged sections:
let,object,ref,expr,signal,season,seq,template,up,seed,deseed,down,init,run,workers,weights,stages,expect,tree. - Circular includes are detected and rejected.
- All includes must appear before any declarations.
Use include for flat file splitting where you want everything merged as-is.
3.2 Imports#
import "path/to/file.edg"
import "shared.edg" as sharedpathis relative to the importing file.- Only
pub-marked declarations from the imported file are visible. - With
as alias, imported names are prefixed:alias.name. - Without
as, the filename stem (without.edg) is used as the namespace prefix. - Imported sections (
up,seed,down,init,deseed,run,workers,weights) are namespaced with the import alias. - Circular imports are detected and rejected.
- All imports must appear before any declarations.
4. Declarations#
4.1 let - Variables#
let name = value
pub let name = valueDefines a global variable. The value is a scalar expression evaluated at parse time. Values may be integers, floats, booleans, or expression strings.
let batch_size = 1000
let warehouses = int(coalesce(env_nil('WAREHOUSES'), 1))Variables are available in all subsequent expressions via their name.
4.2 object - Data Generators#
object name {
field = expression
sub {
field = expression
}
}Defines a named data generator with typed fields. Each field is an expression evaluated per-row.
The sub block contains fields that generate nested/array-valued data. Sub-fields are evaluated before regular fields.
object customer {
id = uuid_v4()
email = gen('email')
name = gen('name')
sub {
orders = obj_n('order', 1, 5)
}
}4.3 ref - Reference Data#
ref name [
{key: value, key: value}
{key: value, key: value}
]Defines inline static reference data as a list of records. Values may be strings, numbers, booleans (true/false), or arrays ([...]). Identifiers in value position are treated as constants.
ref regions [
{id: 1, name: "US", weight: 60}
{id: 2, name: "EU", weight: 30}
{id: 3, name: "APAC", weight: 10}
]Array values are enclosed in square brackets with comma-separated elements:
ref regions [
{name: "us", cities: ["new york", "chicago", "la"]}
{name: "eu", cities: ["london", "paris", "berlin"]}
]Fields within a row are separated by commas. Rows are separated by newlines.
4.4 seq - Sequences#
seq name(start: N, step: M)
seq name(start: N, step: M, length: L)Defines a named numeric sequence.
| Parameter | Required | Description |
|---|---|---|
start | Yes | Initial value |
step | Yes | Increment per call |
length | No | If set, creates a bounded alphabetic sequence |
Without length: monotonic counter (start, start+step, start+2*step, …).
With length: generates alphabetic sequences of the given length (0-based letter sequences).
seq order_id(start: 1, step: 1)
seq sku_code(start: 0, step: 1, length: 3)4.5 expr - Named Expressions#
expr name = expressionDefines a reusable expression that can be referenced by name in other expressions, templates, or query arguments.
expr is_active = result().status == 'active'
expr order_total = result().quantity * result().price4.6 signal - Pre-computed Signal Buffers PRO#
signal name(length: N) { expression }
signal name(from: 'RFC3339', to: 'RFC3339', interval: 'duration') { expression }Defines a named pre-computed signal buffer for correlated temporal patterns across tables. The expression is evaluated for each index i (0 to length-1) at init time.
Length-based:
signal promo_boost(length: 2016) {
floor(50 * pow(cos(pi * mod(i, 500) / 500), 20))
}Duration-based (length auto-computed from time range):
signal traffic(from: '2024-01-01T00:00:00Z', to: '2024-01-08T00:00:00Z', interval: '5m') {
1000 + 400 * sin(2 * pi * i / 288)
}| Option | Required | Description |
|---|---|---|
length | One of | Number of data points |
from | One of | Start timestamp (RFC3339) |
to | One of | End timestamp (RFC3339) |
interval | One of | Time step between data points |
Three functions consume signals:
signal('name')- value at currentiter(), wrapping aroundsignal_at('name', index)- value at explicit indexsignal_correlated('name', lag, correlation)- lagged value with correlated noise (lag is int or duration string)
4.7 season - Weighted Timestamp Distributions PRO#
season name(from: 'RFC3339', to: 'RFC3339', bucket: 'duration') { terms }Defines a named timestamp distribution over [from, to). The range is divided into buckets, each bucket is weighted by the declared terms, and season('name') draws a bucket in proportion to its weight and then picks a uniform instant inside it.
Every term is a multiplier, so they compose: a December month weight of 2.5 and an 18:00 hour weight of 1.8 make 18:00 on a December day 4.5 times as likely as a baseline hour.
season retail(from: '2023-01-01T00:00:00Z', to: '2025-01-01T00:00:00Z', bucket: '1h') {
months = [0.8, 0.7, 0.9, 0.9, 1, 0.9, 0.9, 1, 1, 1.2, 1.8, 2.5]
weekdays = [1.1, 1, 1, 1, 1, 1.2, 1.3]
hours = [0.2, 0.1, 0.1, 0.1, 0.1, 0.2, 0.4, 0.7, 1, 1.2, 1.3, 1.3, 1.2, 1.2, 1.3, 1.3, 1.4, 1.6, 1.8, 1.7, 1.4, 1, 0.6, 0.3]
recency = { half_life: '8760h' }
spikes = [
{ at: '2023-11-24T00:00:00Z', width: '96h', mag: 12 },
{ at: '2024-11-29T00:00:00Z', width: '96h', mag: 12 }
]
weight = month == 12 && day > 25 ? 0.3 : 1
}| Option | Required | Description |
|---|---|---|
from | Yes | Start timestamp (RFC3339), inclusive |
to | Yes | End timestamp (RFC3339), exclusive |
bucket | No | Bucket width. Defaults to the coarsest width that still resolves every declared term |
At least one term must be declared:
| Term | Shape | Description |
|---|---|---|
months | 12 weights | Month of year, index 0 is January |
weekdays | 7 weights | Day of week, index 0 is Sunday |
hours | 24 weights | Hour of day, index 0 is midnight |
business_hours | { from, to, off } | Hours in [from, to) weigh 1, all others weigh off |
recency | { half_life } | Exponential decay towards the past, halving every half_life |
spikes | array of { at, width, mag } | Peaks at mag on at, tapering linearly to 1 over width either side |
weight | expression | Arbitrary weight evaluated once per bucket |
business_hours shapes the day only and is deliberately blind to the day of week, so that it multiplies with weekdays rather than fighting it. Express quiet weekends by weighting Saturday and Sunday down. off is a multiplier like every other term, so { from: 8, to: 20, off: 0.05 } makes an out-of-hours bucket 20 times less likely than a business hour, and off: 0 suppresses out-of-hours traffic entirely.
A mag below 1 turns a spike into a lull, which models a holiday shutdown.
The weight expression is evaluated at each bucket’s start with these variables in scope, alongside globals and the usual functions:
| Variable | Type | Description |
|---|---|---|
t | time | Bucket start |
year | int | Calendar year |
month | int | 1 to 12 |
day | int | Day of month |
yearday | int | 1 to 366 |
weekday | int | 0 is Sunday |
hour | int | 0 to 23 |
Two functions consume seasons:
season('name')- an RFC3339 timestamp drawn from the distributionseason_weight('name', ts)- the intensity atts, normalised so that the mean across the range is 1.0
bucket sets the finest shape the season can express, so it’s validated against the terms: an hours term with a 24h bucket is rejected rather than silently ignored. The cumulative-weight table is capped at 1,048,576 buckets; a bucket too fine for the range is widened to fit and a warning is logged.
4.8 template - Query Templates#
template name(options)
template name(options) `SQL`Defines a reusable query template with default options and optional SQL. Queries referencing a template inherit its settings; query-level settings take precedence.
template batch_insert(count: 10000, size: 1000, type: exec_batch)
seed {
populate_users(template: batch_insert)
`INSERT INTO users (id, email) __values__`
(uuid_v4(), gen('email'))
}5. Sections#
Sections group queries for different lifecycle phases. Each section is a keyword followed by a brace-delimited block of queries.
| Section | Purpose | Execution |
|---|---|---|
up | Schema creation (CREATE TABLE, etc.) | Sequential, once |
seed | Data population | Sequential, once |
deseed | Data cleanup before down | Sequential, once |
down | Schema teardown (DROP TABLE, etc.) | Sequential, once |
init | Pre-run initialization (SELECT for reference data) | Sequential, once |
run | Main workload (queries, transactions) | Concurrent, repeated |
up {
create_users `CREATE TABLE users (id UUID PRIMARY KEY, email TEXT)`
}
seed {
populate_users(count: 10000, size: 100)
`INSERT INTO users (id, email) __values__`
(uuid_v4(), gen('email'))
}
init {
fetch_users `SELECT id FROM users`
}
run {
read_user `SELECT * FROM users WHERE id = $1`
(ref('fetch_users').id)
}The run section is special: it supports transactions, if/match control flow, and noop/rollback statements in addition to regular queries.
6. Queries#
6.1 Syntax#
name [(options)] [`SQL`] [(args)]A query has a name, optional parenthesized options, an optional backtick-delimited SQL body, and optional parenthesized arguments.
6.2 Options#
Options are comma-separated key: value pairs inside parentheses.
| Option | Type | Description |
|---|---|---|
assert | expr | Inline assertion (run section only, fail-fast) |
batch_format | string | Batch output format |
count | expr | Number of rows/iterations to generate |
delay | duration | Delay between executions (workers block) |
ignore | bool | Ignore errors |
object | ident | Object definition to use for field generation |
post_print | expr or {key: K, value: E, agg: A, series: value, window: D} | Print values after execution |
prepared | bool | Use prepared statements |
print | expr or {key: K, value: E, agg: A, series: value, window: D} | Print values during execution |
rate | rate | Execution rate (workers block) |
request_timeout | duration | Per-query timeout |
rollback_if | expr | Condition to trigger rollback |
size | expr | Batch size for batch types |
template | ident | Template to inherit defaults from |
type | query | exec | query_batch | exec_batch | line | bar | Query execution mode, or chart type in a stats block |
wait | duration | Delay before execution |
workers | int | Parallel workers for this query |
Duration format: Go duration strings - 1s, 500ms, 5m, 1h30m.
Rate format: times/interval - 1/10s, 100/1m.
Print compound form fields:
| Field | Type | Description |
|---|---|---|
key | string | Label to display the value under |
value | expr | The expression to print (alias of expr, used when key is set) |
expr | expr | The expression to print |
agg | expr | Aggregation expression over min, max, avg, sum, count, freq |
series | value | Plot one chart line per distinct value returned |
window | duration | Aggregate over a sliding window instead of the whole run. Rounded to the nearest multiple of --print-interval (default 1s), minimum one interval, maximum 3600 intervals |
6.3 SQL Body#
SQL is enclosed in backticks. It can be any SQL, CQL, or MongoDB query text. Multi-line SQL is supported and automatically dedented.
Inline arguments: ${expr} inside SQL is replaced with a positional placeholder ($N) and the expression is added to the argument list.
insert_user `INSERT INTO users (id, email) VALUES (${uuid_v4()}, ${gen('email')})`Batch marker: __values__ in INSERT statements is replaced at runtime with generated VALUES clauses for batch inserts.
populate_users(count: 10000, size: 100)
`INSERT INTO users (id, email) __values__`
(uuid_v4(), gen('email'))6.4 Arguments#
Arguments are either positional or named, enclosed in parentheses after the SQL body.
Positional:
(uuid_v4(), gen('email'), uniform.int(18, 65))Named:
(id: uuid_v4(), email: gen('email'), age: uniform.int(18, 65))Each argument is an expression evaluated per-row at runtime.
6.5 Query-Scoped Locals#
let bindings may appear between the SQL body and the arguments block. They are evaluated per-row, in declaration order, before the arguments.
query_name(count: 1000, size: 100)
`INSERT INTO t (a, b, c) __values__`
let x = ref('data.cities')
let y = uniform.int(local('x').lat_min, local('x').lat_max)
(local('x').name, local('y'), gen('email'))Each let creates a local variable accessible via local('name'). Later let bindings can reference earlier ones. Query-scoped locals follow the same clearOneCache boundary as arguments: ref_rand in a local picks a fresh row per data row, while ref_same returns the same row within a single data row.
7. Transactions#
Transactions group queries inside an explicit BEGIN/COMMIT boundary.
transaction name [(options)] {
[let local_var = expr]*
query*
}Options:
| Option | Type | Description |
|---|---|---|
ignore_nested | bool | Ignore errors in nested queries |
ignore | bool | Ignore errors |
wait | duration | Delay before execution |
Local variables: let inside a transaction defines a transaction-scoped variable accessible via local('name').
transaction transfer(wait: 100ms) {
let amount = uniform.int(1, 100)
read_balance `SELECT balance FROM accounts WHERE id = $1`
(ref('fetch_accounts').id)
if result().balance > local('amount') {
debit `UPDATE accounts SET balance = balance - $2 WHERE id = $1`
(ref_same('read_balance').id, local('amount'))
}
}8. Control Flow#
Control flow statements may appear inside run sections and transaction blocks.
8.1 if / else#
if expr {
(let | query)*
} else {
(let | query)*
}The expression is evaluated at runtime. If truthy, the then block executes; otherwise the optional else block executes. let statements inside branches set transaction-scoped locals that persist after the conditional.
8.2 match / when / default#
match expr {
when value {
(let | query)*
}
when value {
(let | query)*
}
default {
(let | query)*
}
}Evaluates the expression and executes the matching when branch. If no branch matches, default executes (if present). let statements inside branches set transaction-scoped locals that persist after the conditional.
8.3 noop#
A no-operation statement. Takes up a slot in the query list without executing anything.
8.4 rollback / rollback_if#
rollback- unconditionally rolls back the current transaction.rollback_if expr- rolls back if the expression evaluates to true.
9. Weights#
weights {
query_name = weight
query_name = weight
}Assigns relative weights to queries in the run section for random selection. Weights are positive integers.
weights {
read_user = 70
update_user = 20
delete_user = 10
}10. Expectations PRO#
expect {
name [`SQL`] expr
}Post-run assertions. Each expectation has a name, an optional SQL query for context, and a boolean expression.
If SQL is provided, its result columns are available as variables in the expression.
expect {
error_rate < 1
row_count `SELECT count(*) as cnt FROM users` cnt > 0
}11. Stages#
stages {
name(workers: N, duration: D [, ramp_duration: R] [, qps: Q | ramp(S, E, D)] [, weights: {name: N, ...}])
}Defines sequential load stages with varying worker counts and durations.
| Option | Type | Description |
|---|---|---|
duration | duration | How long the stage runs |
qps | int | ramp(start, end, duration) | Optional queries-per-second limit. Use ramp() to linearly increase QPS from start to end over duration. |
ramp_duration | duration | Optional linear ramp-up period (must be less than duration) |
weights | {name: N} | Stage-specific query weight overrides |
workers | int | Number of concurrent workers |
When ramp_duration is set on a stage with a static qps, all workers start immediately and QPS increases linearly from near-zero to the target rate over the ramp period. On a stage without qps, workers are spawned incrementally over the ramp period.
When qps: ramp(start, end, duration) is used, QPS ramps linearly from start to end over duration. This is independent of ramp_duration, which continues to control worker staggering.
stages {
warmup(workers: 5, duration: 30s)
ramp(workers: 50, duration: 2m, qps: ramp(10, 1000, 30s))
sustain(workers: 100, duration: 10m, qps: 1000)
cooldown(workers: 10, duration: 1m, weights: {read_user: 100})
}12. Workers#
workers {
name [(options)] `SQL` [(args)]
}Background workers run continuously in their own goroutine. Worker queries use the same syntax as regular queries, with additional options:
| Option | Type | Description |
|---|---|---|
rate | rate | Execution rate (e.g. 1/10s) |
delay | duration | Fixed delay between executions |
workers {
heartbeat(rate: 1/10s) `SELECT 1`
cleanup(delay: 5m) `DELETE FROM sessions WHERE expires_at < now()`
}13. Stats#
stats [(options)] {
name [(options)] [(print options)] `SQL` [(args)]
}Periodic observability queries that run alongside the main workload. Unlike workers (one goroutine per query), stats queries that share the block rate run sequentially on each tick.
Block options:
| Option | Type | Description |
|---|---|---|
include | bool | Show stats queries in the progress table and TUI chart (default false) |
rate | rate | Required. Default execution rate for queries that don’t set their own rate |
Query options: stats queries accept all query options, plus:
| Option | Type | Description |
|---|---|---|
rate | rate | Per-query execution rate, overriding the block rate. The query gets its own goroutine and ticker |
type | line | bar | Chart type for the inline TUI chart (default line) |
A stats query with a print or post_print option renders an inline chart in the TUI Stats view. line plots a numeric value over time; bar shows the frequency distribution of a categorical value.
stats (rate: 1 / 1s) {
node (type: bar) (
post_print: { key: 'id', value: result().node_id }
) `SHOW node_id`
}Multiple stats blocks may be declared; each has its own rate and include setting.
stats (rate: 1/5s, include: true) {
mvcc(
post_print: { key: 'old_mvcc_mb', value: result().old_mvcc_mb, agg: max }
) `SELECT round(sum(range_size_mb), 2) AS old_mvcc_mb FROM crdb_internal.ranges`
version_over_time(
type: line,
rate: 20 / 1s,
post_print: { key: 'version', value: result().version, series: value, window: 10s }
) `SELECT split_part(version(), ' ', 3) AS version`
}Print values from stats queries always appear in the TUI Stats view, regardless of include. In staged mode, stats run for the whole duration rather than restarting per stage. See TUI for chart behaviour.
14. Trees#
tree name [(options)] `SQL` [(args)] [{ level_overrides }]Generates hierarchical data with parent-child relationships.
Options:
| Option | Type | Description |
|---|---|---|
batch_format | string | Batch format |
dag | bool | Allow multiple parents (DAG mode) |
id_column | ident | Column name for the node ID |
levels | [N, N, ...] | Number of children per level |
parent_column | ident | Column name for the parent reference |
prepared | bool | Use prepared statements |
size | expr | Batch size |
template | ident | Template reference |
type | query type | Execution mode |
wait | duration | Delay between level inserts |
Level overrides: per-level argument overrides in a brace block.
tree org(levels: [1, 3, 5], id_column: id, parent_column: parent_id)
`INSERT INTO employees (id, name, parent_id) __values__` (
id: seq_global('emp_id'),
name: gen('name'),
parent_id: const(null)
) {
level 0 { name: const('CEO') }
level 1 { name: template('VP of %s', gen('buzzword')) }
}15. Complete (AI Generation)#
complete {
tool name [(system: "prompt")] {
property [(type: "T", required: true)]
property [(type: "T")]
}
}Defines an AI tool-calling schema for structured data generation via LLM.
complete {
tool product(system: "Generate realistic product data") {
name(type: "string", required: true)
price(type: "number", required: true)
category(type: "string")
}
}Used with the complete('tool_name') and complete_array('tool_name', N) functions.
16. Expressions#
Expressions in edg-lang are evaluated by the expr-lang engine. They support:
16.1 Operators#
| Category | Operators |
|---|---|
| Arithmetic | +, -, *, /, % |
| Comparison | <, >, <=, >=, ==, != |
| Logical | &&, ||, ! |
| Member access | .field, [index] |
16.2 Literals#
- Integers:
42,-1 - Floats:
3.14,-0.5 - Strings:
'hello',"world" - Booleans:
true,false - Null:
null,nil - Arrays:
[1, 2, 3]
16.3 Function Calls#
function_name(arg1, arg2, ...)All built-in functions (see Section 17) and user-defined globals are accessible in expressions.
16.4 Member Access#
ref('fetch_users').id
result().balance
point(51.5, -0.1, 10).lat16.5 Expression Contexts#
Expressions appear in several contexts:
letvalues:let x = 42 * 3exprdefinitions:expr total = result().qty * result().price- Query arguments:
(ref('users').id, gen('email')) - Query options:
count: warehouses * districts ifconditions:if result().balance > local('amount')matchvalues:match result().statuswhenvalues:when 'active'rollback_ifconditions:rollback_if result().balance < 0expectassertions:error_rate < 1print/post_printvalues:print: result().count- Object fields:
id = uuid_v4()
17. Built-in Functions#
17.1 Data Generation#
| Function | Signature | Description |
|---|---|---|
array | array(min, max, pattern) | PostgreSQL array literal |
bit | bit(n) | Fixed-length bit string |
blob | blob(n) | Raw random bytes ([]byte) |
bool | bool() | Random boolean |
bytes | bytes(n) | Hex-encoded random bytes |
gen_batch | gen_batch(total, size, pattern) | Generate N values in batches |
gen_locale | gen_locale(locale, pattern) | Locale-aware PII generation |
gen | gen(pattern) | Generate random value via gofakeit pattern |
hash | hash(value, algo) | Unkeyed hex digest (md5, sha1, sha256, crc32) |
inet | inet(cidr) | Random IP in CIDR block |
ltree | ltree(parts...) | PostgreSQL ltree path |
mask | mask(key, value) | Deterministic PII pseudonymization |
objectid | objectid() | MongoDB ObjectID |
regex | regex(pattern) | String matching regex |
ulid | ulid() | 26-character Crockford base32 ULID (48-bit ms timestamp + 80 random bits) |
uuid_v1 | uuid_v1() | Version 1 UUID |
uuid_v4 | uuid_v4() | Version 4 UUID (random) |
uuid_v6 | uuid_v6() | Version 6 UUID |
uuid_v7 | uuid_v7() | Version 7 UUID |
varbit | varbit(n) | Variable-length bit string |
17.2 Numeric Distributions#
| Function | Signature | Description |
|---|---|---|
beta.float | beta.float(alpha, beta, min, max, precision) | Beta distribution (float). min/max scale the [0, 1] draw rather than clamping it |
beta.n | beta.n(alpha, beta, min, max, minN, maxN) | N unique Beta values |
binomial.n | binomial.n(n, p, minN, maxN) | N unique Binomial values |
binomial | binomial.int(n, p) | Binomial-distributed integer |
empirical.float | empirical.float(samples, precision) | Empirical CDF with precision |
empirical.n | empirical.n(samples, minN, maxN) | N unique empirical CDF values |
empirical | empirical.int(samples) | Sample from empirical CDF |
exp.float | exp.float(rate, min, max, precision) | Exponential (float) |
exp.n | exp.n(rate, min, max, minN, maxN) | N unique exponential values |
exp | exp(rate, min, max) | Exponential distribution |
gamma.float | gamma.float(shape, rate, min, max, precision) | Gamma distribution (float) |
gamma.n | gamma.n(shape, rate, min, max, minN, maxN) | N unique Gamma values |
lognorm.float | lognorm.float(mu, sigma, min, max, precision) | Log-normal (float) |
lognorm.n | lognorm.n(mu, sigma, min, max, minN, maxN) | N unique log-normal values |
lognorm | lognorm(mu, sigma, min, max) | Log-normal distribution |
markov | markov(group, states, matrix) | Stateful Markov chain transition |
mvnorm | mvnorm(group, index, means, stddevs, correlations) | Correlated multivariate normal |
norm.float | norm.float(mean, stddev, min, max, precision) | Normal distribution (float) |
norm.n | norm.n(mean, stddev, min, max, minN, maxN) | N unique normal values |
norm | norm(mean, stddev, min, max) | Normal distribution (int) |
nurand_n | nurand_n(A, min, max, minN, maxN) | N unique NURand values |
nurand | nurand(A, min, max) | TPC-C Non-Uniform Random |
pareto.float | pareto.float(alpha, min, max, precision) | Continuous Pareto float in [min, max] |
pareto.n | pareto.n(alpha, imax, minN, maxN) | N unique Pareto values |
pareto | pareto.int(alpha, max) | Pareto distribution |
poisson.n | poisson.n(lambda, minN, maxN) | N unique Poisson values |
poisson | poisson.int(lambda) | Poisson-distributed integer |
rwalk_f | rwalk_f(group, start, drift, volatility, precision) | Random walk with precision |
rwalk | rwalk(group, start, drift, volatility) | Random walk step |
uniform.float | uniform.float(min, max, precision) | Uniform float with decimal places |
uniform.n | uniform.n(min, max, minN, maxN) | N unique uniform values |
uniform | uniform.int(min, max) | Uniform random float |
weibull.float | weibull.float(shape, scale, min, max, precision) | Weibull distribution (float) |
weibull.n | weibull.n(shape, scale, min, max, minN, maxN) | N unique Weibull values |
zipf.n | zipf.n(s, v, imax, minN, maxN) | N unique Zipfian values |
zipf | zipf.int(s, v, max) | Zipfian distribution |
Every .n function returns a comma-separated string. Its final two arguments are always minN, maxN: a count is chosen uniformly at random in [minN, maxN] and that many distinct values are drawn. It is an error if minN < 1, if maxN < minN, or if that many distinct values can’t be found within 10,000 draws.
There is deliberately no binomial.float, poisson.float or zipf.float - those distributions are integer-valued by definition. Use beta.float or gamma.float for a continuous skewed value.
17.3 Temporal#
| Function | Signature | Description |
|---|---|---|
date_offset | date_offset(duration) | Now ± duration |
date | date(format, min, max) | Random date with custom format |
duration | duration(min, max) | Random Go duration string |
season_weight | season_weight(name, ts) | Season intensity at a timestamp, mean 1.0 PRO |
season | season(name) | Timestamp drawn from a named season PRO |
time | time(min, max) | Random time of day (HH:MM:SS) |
timestamp_step | timestamp_step() | Next monotonic timestamp |
timestamp_steps | timestamp_steps(min, max, interval_or_count) | Evenly spaced timestamps |
timestamp | timestamp(min, max) | Random RFC3339 timestamp |
timez | timez(min, max) | Random time with timezone |
17.4 Geographic#
| Function | Signature | Description |
|---|---|---|
geo_bearing | geo_bearing(lat1, lon1, lat2, lon2) | Initial compass bearing in degrees, [0, 360) |
geo_distance | geo_distance(lat1, lon1, lat2, lon2) | Great-circle distance in kilometres (haversine) |
point_wkt | point_wkt(lat, lon, radiusKM) | Random point → WKT |
point | point(lat, lon, radiusKM) | Random point → {lat, lon} |
polygon_wkt | polygon_wkt(lat, lon, minKM, maxKM, points) | Polygon → WKT |
polygon | polygon(lat, lon, minKM, maxKM, points) | Polygon vertices |
17.5 JSON#
| Function | Signature | Description |
|---|---|---|
json_arr | json_arr(min, max, pattern) | JSON array of generated values |
json_obj | json_obj(key, val, ...) | JSON object string |
17.6 Sequences#
| Function | Signature | Description |
|---|---|---|
beta.seq | beta.seq(name, alpha, beta) | Beta from sequence |
binomial.seq | binomial.seq(name, n, p) | Binomial from sequence |
empirical.seq | empirical.seq(name, samples) | Empirical CDF from sequence |
exp.seq | exp.seq(name, rate) | Exponential from sequence |
gamma.seq | gamma.seq(name, shape, rate) | Gamma from sequence |
lognorm.seq | lognorm.seq(name, mu, sigma) | Log-normal from sequence |
norm.seq | norm.seq(name, mean, stddev) | Normal from sequence |
pareto.seq | pareto.seq(name, alpha) | Pareto from sequence |
poisson.seq | poisson.seq(name, lambda) | Poisson from sequence |
seq_alpha_global | seq_alpha_global(name) | Shared alpha sequence |
seq_alpha | seq_alpha(length) | Per-worker alpha sequence |
seq_global | seq_global(name) | Shared auto-increment across workers |
seq | seq(start, step) | Per-worker auto-increment |
uniform.seq | uniform.seq(name) | Random value from sequence range |
weibull.seq | weibull.seq(name, shape, scale) | Weibull from sequence |
zipf.seq | zipf.seq(name, s, v) | Zipfian from sequence |
17.7 References#
| Function | Signature | Description |
|---|---|---|
beta.ref | beta.ref(name, alpha, beta) | Beta row selection |
binomial.ref | binomial.ref(name, p) | Binomial row selection |
empirical.ref | empirical.ref(name, samples) | Empirical CDF row selection |
exp.ref | exp.ref(name, rate) | Exponential row selection |
gamma.ref | gamma.ref(name, shape, rate) | Gamma row selection |
lognorm.ref | lognorm.ref(name, mu, sigma) | Log-normal row selection |
norm.ref | norm.ref(name, mean, stddev) | Normal row selection |
pareto.ref | pareto.ref(name, alpha) | Pareto row selection |
poisson.ref | poisson.ref(name, lambda) | Poisson row selection |
ref_cursor | ref_cursor(query, size, col, repeat?) | Keyset-paginated cursor over SQL query. Optional repeat count for exact cardinality |
ref_diff | ref_diff(name) | Different row than previous |
ref_each | ref_each(name, repeat?) | Cycle through rows sequentially. Optional repeat count for exact cardinality |
ref_n | ref_n(name, field, n) | N unique random field values |
ref_perm | ref_perm(name) | Same random row for worker lifetime |
ref_same | ref_same(name) | Same row as previous access |
ref_weighted | ref_weighted(name, weights) | Weighted row selection with explicit integer weights |
ref | ref(name) | Random row (uniform) |
weibull.ref | weibull.ref(name, shape, scale) | Weibull row selection |
zipf.ref | zipf.ref(name, s, v) | Zipfian row selection |
17.8 Set Selection#
| Function | Signature | Description |
|---|---|---|
beta.set | beta.set(items, alpha, beta) | Beta selection |
binomial.set | binomial.set(items, p) | Binomial selection |
empirical.set | empirical.set(items, samples) | Empirical CDF selection |
exp.set | exp.set(items, rate) | Exponential selection |
gamma.set | gamma.set(items, shape, rate) | Gamma selection |
lognorm.set | lognorm.set(items, mu, sigma) | Log-normal selection |
norm.set | norm.set(items, mean, stddev) | Normal selection |
pareto.set | pareto.set(items, alpha) | Pareto selection |
poisson.set | poisson.set(items, lambda) | Poisson selection |
set | set(items [, weights]) | Random item (uniform or weighted) |
weibull.set | weibull.set(items, shape, scale) | Weibull selection |
zipf.set | zipf.set(items, s, v) | Zipfian selection |
17.9 Objects#
| Function | Signature | Description |
|---|---|---|
<dist>.obj_n | e.g. poisson.obj_n(name, lambda, min, max) | Generate N object instances using distribution-controlled count |
field | field(name) | Access field from current object. Bare field names also work: quantity * price instead of field('quantity') * field('price') |
obj_n | obj_n(name, min, max) | Generate N object instances (uniform, alias for uniform.obj_n) |
obj | obj(name) | Evaluate object fields |
17.10 Runtime Context#
| Function | Signature | Description |
|---|---|---|
arg | arg(index_or_name) | Previously evaluated argument. With named args, bare names also work: first_name + " " + last_name instead of arg('first_name') + " " + arg('last_name') |
batch | batch(n) | Batch indices [0, n) |
env_nil | env_nil(name) | OS environment variable (nil if missing) |
env | env(name) | OS environment variable (error if missing) |
global_iter | global_iter() | Monotonic counter across all workers |
global | global(name) | Global config variable |
iter | iter() | 1-based row counter (batch queries) |
local | local(name) | Transaction-scoped variable |
result | result() | Last query result row |
results | results() | All rows from last query result |
17.11 Aggregation#
| Function | Signature | Description |
|---|---|---|
avg | avg(dataset, field) | Average numeric field |
count | count(dataset) | Row count |
distinct | distinct(dataset, field) | Distinct value count |
max | max(dataset, field) | Maximum value |
median | median(dataset, field) | Median, equivalent to percentile(dataset, field, 50) |
min | min(dataset, field) | Minimum value |
percentile | percentile(dataset, field, p) | pth percentile, linearly interpolated between the two nearest ranks. p is in [0, 100] |
stddev | stddev(dataset, field) | Population standard deviation (divides by N) |
sum | sum(dataset, field) | Sum numeric field |
variance | variance(dataset, field) | Population variance (divides by N) |
Every aggregate (sum, avg, min, max, median, stddev, variance, percentile) also accepts a single-array form: sum([1, 2, 3]), median(prices), stddev(prices), percentile(prices, 95). These names shadow expr-lang’s builtin aggregates, so edg reimplements them and both call shapes work. For percentile, p is always the last argument, so the array form takes 2 arguments and the dataset form takes 3. Non-numeric elements in an array are skipped, and empty input returns 0 (including for min and max).
17.12 Utilities#
| Function | Signature | Description |
|---|---|---|
coalesce | coalesce(a, b, ...) | First non-nil value |
cond | cond(pred, trueVal, falseVal) | Conditional value |
const | const(value) | Literal constant |
expr | expr(value) | Evaluate arithmetic expression |
fail | fail(message) | Graceful worker error |
fatal | fatal(message) | Terminate process |
nullable | nullable(probability, value) | NULL with given probability |
template | template(format, args...) | sprintf-style formatting |
uniq_across | uniq_across(pool, expr...) | Cross-query uniqueness pool |
uniq | uniq(expr...) | Retry until unique value |
17.13 Math#
| Function | Signature | Description |
|---|---|---|
abs | abs(x) | Absolute value |
acos | acos(x) | Arccosine |
asin | asin(x) | Arcsine |
atan | atan(x) | Arctangent |
atan2 | atan2(y, x) | Two-argument arctangent |
ceil | ceil(x) | Ceiling |
cos | cos(x) | Cosine |
floor | floor(x) | Floor |
log | log(x) | Natural log |
log10 | log10(x) | Base-10 log |
mod | mod(x, y) | Modulo |
pi | pi | Pi constant |
pow | pow(x, y) | Power |
round | round(x, n) | Round to n decimals |
sin | sin(x) | Sine |
sqrt | sqrt(x) | Square root |
tan | tan(x) | Tangent |
17.14 Vectors and Embeddings#
| Function | Signature | Description |
|---|---|---|
beta.vector | beta.vector(dims, centroids, noise, alpha, beta) | Beta centroid |
binomial.vector | binomial.vector(dims, centroids, noise, p) | Binomial centroid |
embed | embed(text) | Real vector embedding via API |
empirical.vector | empirical.vector(dims, centroids, noise, samples) | Empirical centroid |
exp.vector | exp.vector(dims, centroids, noise, rate) | Exponential centroid |
gamma.vector | gamma.vector(dims, centroids, noise, shape, rate) | Gamma centroid |
lognorm.vector | lognorm.vector(dims, centroids, noise, mu, sigma) | Log-normal centroid |
norm.vector | norm.vector(dims, centroids, noise, mean, stddev) | Normal centroid |
pareto.vector | pareto.vector(dims, centroids, noise, alpha) | Pareto centroid |
poisson.vector | poisson.vector(dims, centroids, noise, lambda) | Poisson centroid |
vector | vector(dims, centroids) | Clustered vector (uniform) |
weibull.vector | weibull.vector(dims, centroids, noise, shape, scale) | Weibull centroid |
zipf.vector | zipf.vector(dims, centroids, s, v) | Zipfian centroid |
17.15 AI Generation#
| Function | Signature | Description |
|---|---|---|
complete | complete(tool_name) | Structured data via LLM tool call |
complete_array | complete_array(tool_name, n) | N items in single LLM call |
17.16 Distribution Helpers#
| Function | Signature | Description |
|---|---|---|
distribute_sum | distribute_sum(total, n) | Partition total into N random parts |
distribute_weighted | distribute_weighted(total, weights) | Proportional partition with noise |
weighted_sample_n | weighted_sample_n(dataset, field, weight_field, n) | N weighted random values |
18. Query Type Inference#
When type is not explicitly specified, edg infers the query type from the SQL verb:
| SQL Prefix | Inferred Type |
|---|---|
CREATE, DROP, ALTER, TRUNCATE | exec |
INSERT/UPDATE/DELETE ... OUTPUT INSERTED/DELETED | query |
INSERT/UPDATE/DELETE ... RETURNING | query |
INSERT, UPDATE, DELETE, UPSERT | exec |
SELECT | query |
MongoDB verbs:
| Verb | Inferred Type |
|---|---|
find, aggregate, count, distinct | query |
insert, update, delete, create, drop, createindexes | exec |
19. Visibility and pub#
The pub keyword exports a declaration for use by importing files.
Supported with: let, expr, seq, ref, object, template.
pub let batch_size = 1000
pub expr active = result().status == 'active'
pub seq counter(start: 1, step: 1)
pub ref regions [ ... ]
pub object user { ... }
pub template batch(count: 1000, type: exec_batch)Declarations without pub are file-private and invisible to importers.
Appendix A: EBNF Grammar#
(* === Top-level === *)
SourceFile = { Include | Import } { Declaration } .
Include = "include" STRING NEWLINE .
Import = "import" STRING [ "as" IDENT ] NEWLINE .
Declaration = LetDecl
| ObjectDecl
| RefDecl
| SeqDecl
| ExprDecl
| SignalDecl
| SeasonDecl
| TemplateDecl
| Section
| RunSection
| WeightsBlock
| ExpectBlock
| StagesBlock
| WorkersBlock
| StatsBlock
| TreeDecl
| CompleteBlock .
(* === Declarations === *)
LetDecl = [ "pub" ] "let" IDENT "=" Expr NEWLINE .
ExprDecl = [ "pub" ] "expr" IDENT "=" Expr NEWLINE .
SeqDecl = [ "pub" ] "seq" IDENT "(" SeqOptions ")" .
SeqOptions = SeqOption { "," SeqOption } .
SeqOption = ( "start" | "step" | "length" ) ":" NUMBER .
ObjectDecl = [ "pub" ] "object" IDENT "{" { ObjectField | SubBlock } "}" .
ObjectField = IDENT "=" Expr NEWLINE .
SubBlock = "sub" "{" { ObjectField } "}" .
RefDecl = [ "pub" ] "ref" IDENT "[" { RefRow } "]" .
RefRow = "{" RefField { "," RefField } "}" .
RefField = IDENT ":" Value .
Value = STRING | NUMBER | "true" | "false" | IDENT .
SignalDecl = [ "pub" ] "signal" IDENT "(" SignalOptions ")" "{" Expr "}" .
SignalOptions = SignalOption { "," SignalOption } .
SignalOption = ( "length" | "from" | "to" | "interval" ) ":" Value .
SeasonDecl = [ "pub" ] "season" IDENT "(" SeasonOptions ")" "{" { SeasonTerm } "}" .
SeasonOptions = SeasonOption { "," SeasonOption } .
SeasonOption = ( "from" | "to" | "bucket" ) ":" STRING .
SeasonTerm = ( "months" | "weekdays" | "hours" ) "=" Array NEWLINE
| ( "business_hours" | "recency" ) "=" RefRow NEWLINE
| "spikes" "=" "[" { RefRow [ "," ] } "]" NEWLINE
| "weight" "=" Expr NEWLINE .
Array = "[" [ Value { "," Value } ] "]" .
TemplateDecl = [ "pub" ] "template" IDENT [ "(" QueryOptions ")" ]
[ RAWSTRING ] .
(* === Sections === *)
Section = SectionKW "{" { Query } "}" .
SectionKW = "up" | "seed" | "deseed" | "down" | "init" .
RunSection = "run" "{" { RunItem } "}" .
RunItem = Transaction | SpecialQuery | Query .
(* === Queries === *)
Query = IDENT [ "(" QueryOptions ")" ]
[ RAWSTRING ]
[ "(" Args ")" ] .
QueryOptions = QueryOption { "," QueryOption } .
QueryOption = IDENT ":" OptionValue .
OptionValue = Expr
| "{" BraceMap "}" (* print compound form *)
| "{" InlineWeights "}" . (* stage weights *)
Args = PositionalArgs | NamedArgs .
PositionalArgs = Expr { "," Expr } .
NamedArgs = NamedArg { "," NamedArg } .
NamedArg = IDENT ":" Expr .
SpecialQuery = "noop"
| "rollback"
| "rollback_if" Expr NEWLINE
| IfBlock
| MatchBlock .
(* === Transactions === *)
Transaction = "transaction" IDENT [ "(" TxOptions ")" ]
"{" { TxLocal | SpecialQuery | Query } "}" .
TxOptions = TxOption { "," TxOption } .
TxOption = ( "wait" | "ignore" | "ignore_nested" ) ":" Expr .
TxLocal = "let" IDENT "=" Expr NEWLINE .
(* === Control Flow === *)
IfBlock = "if" Expr "{" { SpecialQuery | Query } "}"
[ "else" "{" { SpecialQuery | Query } "}" ] .
MatchBlock = "match" Expr "{"
{ WhenClause }
[ DefaultClause ]
"}" .
WhenClause = "when" Expr "{" { SpecialQuery | Query } "}" .
DefaultClause = "default" "{" { SpecialQuery | Query } "}" .
(* === Configuration Blocks === *)
WeightsBlock = "weights" "{" { IDENT "=" NUMBER NEWLINE } "}" .
ExpectBlock = "expect" "{" { Expectation } "}" .
Expectation = IDENT [ RAWSTRING ] Expr NEWLINE .
StagesBlock = "stages" "{" { StageEntry } "}" .
StageEntry = IDENT "(" StageOptions ")" .
StageOptions = StageOption { "," StageOption } .
StageOption = "workers" ":" NUMBER
| "duration" ":" DURATION
| "ramp_duration" ":" DURATION
| "qps" ":" ( NUMBER | "ramp" "(" NUMBER "," NUMBER "," DURATION ")" )
| "weights" ":" "{" InlineWeights "}" .
InlineWeights = IDENT ":" NUMBER { "," IDENT ":" NUMBER } .
WorkersBlock = "workers" "{" { Query } "}" .
StatsBlock = "stats" [ "(" StatsOptions ")" ] "{" { Query } "}" .
StatsOptions = StatsOption { "," StatsOption } .
StatsOption = "rate" ":" RATE
| "include" ":" ( "true" | "false" ) .
TreeDecl = "tree" IDENT [ "(" TreeOptions ")" ]
[ RAWSTRING ]
[ "(" Args ")" ]
[ "{" { LevelOverride } "}" ] .
TreeOptions = TreeOption { "," TreeOption } .
TreeOption = "levels" ":" "[" NUMBER { "," NUMBER } "]"
| "id_column" ":" IDENT
| "parent_column" ":" IDENT
| "dag" ":" ( "true" | "false" )
| QueryOption .
LevelOverride = "level" NUMBER "{" { IDENT ":" Expr } "}" .
CompleteBlock = "complete" "{" { ToolDecl } "}" .
ToolDecl = "tool" IDENT [ "(" ToolOptions ")" ]
"{" { PropertyDecl } "}" .
ToolOptions = "system" ":" STRING .
PropertyDecl = IDENT [ "(" PropOptions ")" ] .
PropOptions = PropOption { "," PropOption } .
PropOption = ( "type" | "required" | IDENT ) ":" Expr .
(* === Expressions === *)
Expr = (* expr-lang expression: arithmetic, comparisons,
logical operators, function calls, member access,
array literals, string literals, numbers, booleans,
nil/null *) .
(* === Lexical Tokens === *)
IDENT = letter { letter | digit | "_" } .
NUMBER = [ "-" ] digit { digit } [ "." digit { digit } ] .
STRING = "'" { char | escape } "'"
| '"' { char | escape } '"' .
RAWSTRING = "`" { any } "`" .
NEWLINE = "\n" | "\r\n" | "\r" .
DURATION = NUMBER ( "ns" | "us" | "µs" | "ms" | "s" | "m" | "h" ) .
RATE = NUMBER "/" DURATION .Appendix B: Keywords#
The following identifiers are reserved keywords:
| Keyword | Category |
|---|---|
complete | Configuration |
default | Control flow |
deseed | Section |
down | Section |
else | Control flow |
expect | Configuration |
expr | Declaration |
if | Control flow |
import | Module |
include | Module |
init | Section |
let | Declaration |
match | Control flow |
object | Declaration |
pub | Modifier |
ref | Declaration |
run | Section |
season | Declaration |
seed | Section |
seq | Declaration |
signal | Declaration |
stages | Configuration |
stats | Configuration |
sub | Object block |
template | Declaration |
transaction | Run block |
tree | Configuration |
up | Section |
weights | Configuration |
when | Control flow |
workers | Configuration |
Keywords may be used as identifiers in positions where identifiers are expected (e.g. query names, field names, option keys), per the parser’s expectIdent function which accepts both TokenIdent and keyword tokens.