edg-lang Specification#

Version: 1.0 Status: Draft

edg-lang is a domain-specific language for defining database workloads: schema setup, data generation, query execution, and load testing. It compiles to an internal Request structure that drives the edg runtime.

1. Lexical Structure#

1.1 Source Encoding#

Source files are UTF-8 encoded. The lexer operates on byte sequences but supports Unicode letters in identifiers.

1.2 Line Termination#

Newlines (\n, \r\n, \r) are significant tokens - they terminate expressions and declarations. Newlines inside parentheses, brackets, braces, and backtick-delimited strings are consumed as whitespace.

1.3 Whitespace#

Spaces and tabs between tokens are insignificant and ignored (except inside strings and raw strings).

1.4 Comments#

Line comments begin with # and extend to the end of the line:

# This is a comment
let batch_size = 1000  # inline comment

There are no block comments.

1.5 Identifiers#

identifier = letter { letter | digit | "_" } .
letter     = "a"…"z" | "A"…"Z" | "_" | <Unicode letter> .
digit      = "0"…"9" .

Identifiers name variables, objects, queries, fields, and other declarations. Keywords may be used as identifiers in positions where identifiers are expected (query names, field names, option keys).

1.6 Numeric Literals#

number = [ "-" ] digit { digit } [ "." digit { digit } ] .

Numbers are signed integers or floating-point values. A leading - is part of the number token only when followed immediately by a digit.

Examples: 42, -1, 3.14, -0.5

1.7 String Literals#

Strings are delimited by single quotes (') or double quotes ("). Backslash escapes (\\, \', \") are supported. Strings may span multiple lines.

'hello world'
"it's a string"
'escape \'this\''

1.8 Raw Strings (SQL)#

Backtick-delimited strings preserve all content verbatim, including whitespace and newlines. They are used for SQL, CQL, and MongoDB query bodies.

`SELECT id, name FROM users WHERE id = $1`

Multi-line raw strings are automatically dedented: the minimum leading whitespace of non-first lines is removed.

1.9 Operators#

Single-character: +, -, *, /, %, <, >, !, &, |, ^

Multi-character: <=, >=, !=, ==, &&, ||

Operators appear inside expressions, which are evaluated by the expr-lang engine at runtime.

1.10 Punctuation#

2. Source File Structure#

An edg source file consists of the following sections, in order:

1. Includes / Imports - must appear before all other declarations
2. Declarations - let, object, ref, seq, expr, signal, template
3. Sections - up, seed, deseed, down, init, run
4. Configuration blocks - weights, expect, stages, workers, tree, complete

Declarations and configuration blocks may be interleaved after includes/imports, but includes and imports must precede everything else.

Each section may appear multiple times; entries are appended.

3. Includes and Imports#

3.1 Includes#

include "path/to/file.edg"

• path is relative to the including file.
• All declarations from the included file are merged directly (no namespacing or pub required).
• Merged sections: let, object, ref, expr, signal, seq, template, up, seed, deseed, down, init, run, workers, weights, stages, expect, tree.
• Circular includes are detected and rejected.
• All includes must appear before any declarations.

include is the edg-lang equivalent of YAML’s !include. Use it for flat file splitting where you want everything merged as-is.

3.2 Imports#

import "path/to/file.edg"
import "shared.edg" as shared

• path is relative to the importing file.
• Only pub-marked declarations from the imported file are visible.
• With as alias, imported names are prefixed: alias.name.
• Without as, the filename stem (without .edg) is used as the namespace prefix.
• Imported sections (up, seed, down, init, deseed, run, workers, weights) are namespaced with the import alias.
• Circular imports are detected and rejected.
• All imports must appear before any declarations.

4. Declarations#

4.1 let - Variables#

let name = value
pub let name = value

Defines a global variable. The value is a scalar expression evaluated at parse time. Values may be integers, floats, booleans, or expression strings.

let batch_size = 1000
let warehouses = int(coalesce(env_nil('WAREHOUSES'), 1))

Variables are available in all subsequent expressions via their name.

4.2 object - Data Generators#

object name {
  field = expression
  sub {
    field = expression
  }
}

Defines a named data generator with typed fields. Each field is an expression evaluated per-row.

The sub block contains fields that generate nested/array-valued data. Sub-fields are evaluated before regular fields, matching YAML Object evaluation order.

object customer {
  id = uuid_v4()
  email = gen('email')
  name = gen('name')
  sub {
    orders = obj_n('order', 1, 5)
  }
}

4.3 ref - Reference Data#

ref name [
  {key: value, key: value}
  {key: value, key: value}
]

Defines inline static reference data as a list of records. Values may be strings, numbers, booleans (true/false), or arrays ([...]). Identifiers in value position are treated as constants.

ref regions [
  {id: 1, name: "US", weight: 60}
  {id: 2, name: "EU", weight: 30}
  {id: 3, name: "APAC", weight: 10}
]

Array values are enclosed in square brackets with comma-separated elements:

ref regions [
  {name: "us", cities: ["new york", "chicago", "la"]}
  {name: "eu", cities: ["london", "paris", "berlin"]}
]

Fields within a row are separated by commas. Rows are separated by newlines.

4.4 seq - Sequences#

seq name(start: N, step: M)
seq name(start: N, step: M, length: L)

Defines a named numeric sequence.

Without length: monotonic counter (start, start+step, start+2*step, …).

With length: generates alphabetic sequences of the given length (0-based letter sequences).

seq order_id(start: 1, step: 1)
seq sku_code(start: 0, step: 1, length: 3)

4.5 expr - Named Expressions#

expr name = expression

Defines a reusable expression that can be referenced by name in other expressions, templates, or query arguments.

expr is_active = result().status == 'active'
expr order_total = result().quantity * result().price

4.6 signal - Pre-computed Signal Buffers PRO#

signal name(length: N) { expression }
signal name(from: 'RFC3339', to: 'RFC3339', interval: 'duration') { expression }

Defines a named pre-computed signal buffer for correlated temporal patterns across tables. The expression is evaluated for each index i (0 to length-1) at init time.

Length-based:

signal promo_boost(length: 2016) {
  floor(50 * pow(cos(pi * mod(i, 500) / 500), 20))
}

Duration-based (length auto-computed from time range):

signal traffic(from: '2024-01-01T00:00:00Z', to: '2024-01-08T00:00:00Z', interval: '5m') {
  1000 + 400 * sin(2 * pi * i / 288)
}

Three functions consume signals:

• signal('name') - value at current iter(), wrapping around
• signal_at('name', index) - value at explicit index
• signal_correlated('name', lag, correlation) - lagged value with correlated noise (lag is int or duration string)

4.7 template - Query Templates#

template name(options)
template name(options) `SQL`

Defines a reusable query template with default options and optional SQL. Queries referencing a template inherit its settings; query-level settings take precedence.

template batch_insert(count: 10000, size: 1000, type: exec_batch)

seed {
  populate_users(template: batch_insert)
    `INSERT INTO users (id, email) __values__`
    (uuid_v4(), gen('email'))
}

5. Sections#

Sections group queries for different lifecycle phases. Each section is a keyword followed by a brace-delimited block of queries.

up {
  create_users `CREATE TABLE users (id UUID PRIMARY KEY, email TEXT)`
}

seed {
  populate_users(count: 10000, size: 100)
    `INSERT INTO users (id, email) __values__`
    (uuid_v4(), gen('email'))
}

init {
  fetch_users `SELECT id FROM users`
}

run {
  read_user `SELECT * FROM users WHERE id = $1`
    (ref_rand('fetch_users').id)
}

The run section is special: it supports transactions, if/match control flow, and noop/rollback statements in addition to regular queries.

6. Queries#

6.1 Syntax#

name [(options)] [`SQL`] [(args)]

A query has a name, optional parenthesized options, an optional backtick-delimited SQL body, and optional parenthesized arguments.

6.2 Options#

Options are comma-separated key: value pairs inside parentheses.

Duration format: Go duration strings - 1s, 500ms, 5m, 1h30m.

Rate format: times/interval - 1/10s, 100/1m.

6.3 SQL Body#

SQL is enclosed in backticks. It can be any SQL, CQL, or MongoDB query text. Multi-line SQL is supported and automatically dedented.

Inline arguments: ${expr} inside SQL is replaced with a positional placeholder ($N) and the expression is added to the argument list.

insert_user `INSERT INTO users (id, email) VALUES (${uuid_v4()}, ${gen('email')})`

Batch marker: __values__ in INSERT statements is replaced at runtime with generated VALUES clauses for batch inserts.

populate_users(count: 10000, size: 100)
  `INSERT INTO users (id, email) __values__`
  (uuid_v4(), gen('email'))

6.4 Arguments#

Arguments are either positional or named, enclosed in parentheses after the SQL body.

Positional:

(uuid_v4(), gen('email'), uniform(18, 65))

Named:

(id: uuid_v4(), email: gen('email'), age: uniform(18, 65))

Each argument is an expression evaluated per-row at runtime.

6.5 Query-Scoped Locals#

let bindings may appear between the SQL body and the arguments block. They are evaluated per-row, in declaration order, before the arguments.

query_name(count: 1000, size: 100)
  `INSERT INTO t (a, b, c) __values__`
  let x = ref_rand('data.cities')
  let y = uniform(local('x').lat_min, local('x').lat_max)
  (local('x').name, local('y'), gen('email'))

Each let creates a local variable accessible via local('name'). Later let bindings can reference earlier ones. Query-scoped locals follow the same clearOneCache boundary as arguments: ref_rand in a local picks a fresh row per data row, while ref_same returns the same row within a single data row.

7. Transactions#

Transactions group queries inside an explicit BEGIN/COMMIT boundary.

transaction name [(options)] {
  [let local_var = expr]*
  query*
}

Options:

Local variables: let inside a transaction defines a transaction-scoped variable accessible via local('name').

transaction transfer(wait: 100ms) {
  let amount = uniform(1, 100)

  read_balance `SELECT balance FROM accounts WHERE id = $1`
    (ref_rand('fetch_accounts').id)

  if result().balance > local('amount') {
    debit `UPDATE accounts SET balance = balance - $2 WHERE id = $1`
      (ref_same('read_balance').id, local('amount'))
  }
}

8. Control Flow#

Control flow statements may appear inside run sections and transaction blocks.

8.1 if / else#

if expr {
  (let | query)*
} else {
  (let | query)*
}

The expression is evaluated at runtime. If truthy, the then block executes; otherwise the optional else block executes. let statements inside branches set transaction-scoped locals that persist after the conditional.

8.2 match / when / default#

match expr {
  when value {
    (let | query)*
  }
  when value {
    (let | query)*
  }
  default {
    (let | query)*
  }
}

Evaluates the expression and executes the matching when branch. If no branch matches, default executes (if present). let statements inside branches set transaction-scoped locals that persist after the conditional.

8.3 noop#

A no-operation statement. Takes up a slot in the query list without executing anything.

8.4 rollback / rollback_if#

• rollback - unconditionally rolls back the current transaction.
• rollback_if expr - rolls back if the expression evaluates to true.

9. Weights#

weights {
  query_name = weight
  query_name = weight
}

Assigns relative weights to queries in the run section for random selection. Weights are positive integers.

weights {
  read_user = 70
  update_user = 20
  delete_user = 10
}

10. Expectations#

expect {
  name [`SQL`] expr
}

Post-run assertions. Each expectation has a name, an optional SQL query for context, and a boolean expression.

If SQL is provided, its result columns are available as variables in the expression.

expect {
  error_rate < 1
  row_count `SELECT count(*) as cnt FROM users` cnt > 0
}

11. Stages#

stages {
  name(workers: N, duration: D [, ramp_duration: R] [, qps: Q] [, weights: {name: N, ...}])
}

Defines sequential load stages with varying worker counts and durations.

When ramp_duration is set on a stage with qps, all workers start immediately and QPS increases linearly from near-zero to the target rate over the ramp period. On a stage without qps, workers are spawned incrementally over the ramp period.

stages {
  warmup(workers: 5, duration: 30s)
  ramp(workers: 50, duration: 2m, ramp_duration: 30s, qps: 1000)
  sustain(workers: 100, duration: 10m)
  cooldown(workers: 10, duration: 1m, weights: {read_user: 100})
}

12. Workers#

workers {
  name [(options)] `SQL` [(args)]
}

Background workers run continuously in their own goroutine. Worker queries use the same syntax as regular queries, with additional options:

workers {
  heartbeat(rate: 1/10s) `SELECT 1`
  cleanup(delay: 5m) `DELETE FROM sessions WHERE expires_at < now()`
}

13. Trees#

tree name [(options)] `SQL` [(args)] [{ level_overrides }]

Generates hierarchical data with parent-child relationships.

Options:

Level overrides: per-level argument overrides in a brace block.

tree org(levels: [1, 3, 5], id_column: id, parent_column: parent_id)
  `INSERT INTO employees (id, name, parent_id) __values__` (
    id: seq_global('emp_id'),
    name: gen('name'),
    parent_id: const(null)
  ) {
    level 0 { name: const('CEO') }
    level 1 { name: template('VP of %s', gen('buzzword')) }
  }

14. Complete (AI Generation)#

complete {
  tool name [(system: "prompt")] {
    property [(type: "T", required: true)]
    property [(type: "T")]
  }
}

Defines an AI tool-calling schema for structured data generation via LLM.

complete {
  tool product(system: "Generate realistic product data") {
    name(type: "string", required: true)
    price(type: "number", required: true)
    category(type: "string")
  }
}

Used with the complete('tool_name') and complete_array('tool_name', N) functions.

15. Expressions#

Expressions in edg-lang are evaluated by the expr-lang engine. They support:

15.1 Operators#

15.2 Literals#

• Integers: 42, -1
• Floats: 3.14, -0.5
• Strings: 'hello', "world"
• Booleans: true, false
• Null: null, nil
• Arrays: [1, 2, 3]

15.3 Function Calls#

function_name(arg1, arg2, ...)

All built-in functions (see Section 16) and user-defined globals are accessible in expressions.

15.4 Member Access#

ref_rand('fetch_users').id
result().balance
point(51.5, -0.1, 10).lat

15.5 Expression Contexts#

Expressions appear in several contexts:

• let values: let x = 42 * 3
• expr definitions: expr total = result().qty * result().price
• Query arguments: (ref_rand('users').id, gen('email'))
• Query options: count: warehouses * districts
• if conditions: if result().balance > local('amount')
• match values: match result().status
• when values: when 'active'
• rollback_if conditions: rollback_if result().balance < 0
• expect assertions: error_rate < 1
• print/post_print values: print: result().count
• Object fields: id = uuid_v4()

16. Built-in Functions#

16.1 Data Generation#

16.2 Numeric Distributions#

16.3 Temporal#

16.4 Geographic#

16.5 JSON#

16.6 Sequences#

16.7 References#

16.8 Set Selection#

16.9 Objects#

16.10 Runtime Context#

16.11 Aggregation#

16.12 Utilities#

16.13 Math#

16.14 Vectors and Embeddings#

16.15 AI Generation#

16.16 Distribution Helpers#

17. Query Type Inference#

When type is not explicitly specified, edg infers the query type from the SQL verb:

MongoDB verbs:

18. Visibility and pub#

The pub keyword exports a declaration for use by importing files.

Supported with: let, object, ref, expr, seq, template.

pub let batch_size = 1000
pub object user { ... }
pub ref regions [ ... ]
pub expr active = result().status == 'active'
pub seq counter(start: 1, step: 1)
pub template batch(count: 1000, type: exec_batch)

Declarations without pub are file-private and invisible to importers.

Appendix A: EBNF Grammar#

(* === Top-level === *)

SourceFile     = { Include | Import } { Declaration } .

Include        = "include" STRING NEWLINE .
Import         = "import" STRING [ "as" IDENT ] NEWLINE .

Declaration    = LetDecl
               | ObjectDecl
               | RefDecl
               | SeqDecl
               | ExprDecl
               | TemplateDecl
               | Section
               | RunSection
               | WeightsBlock
               | ExpectBlock
               | StagesBlock
               | WorkersBlock
               | TreeDecl
               | CompleteBlock .

(* === Declarations === *)

LetDecl        = [ "pub" ] "let" IDENT "=" Expr NEWLINE .

ExprDecl       = [ "pub" ] "expr" IDENT "=" Expr NEWLINE .

SeqDecl        = [ "pub" ] "seq" IDENT "(" SeqOptions ")" .
SeqOptions     = SeqOption { "," SeqOption } .
SeqOption      = ( "start" | "step" | "length" ) ":" NUMBER .

ObjectDecl     = [ "pub" ] "object" IDENT "{" { ObjectField | SubBlock } "}" .
ObjectField    = IDENT "=" Expr NEWLINE .
SubBlock       = "sub" "{" { ObjectField } "}" .

RefDecl        = [ "pub" ] "ref" IDENT "[" { RefRow } "]" .
RefRow         = "{" RefField { "," RefField } "}" .
RefField       = IDENT ":" Value .
Value          = STRING | NUMBER | "true" | "false" | IDENT .

TemplateDecl   = [ "pub" ] "template" IDENT [ "(" QueryOptions ")" ]
                 [ RAWSTRING ] .

(* === Sections === *)

Section        = SectionKW "{" { Query } "}" .
SectionKW      = "up" | "seed" | "deseed" | "down" | "init" .

RunSection     = "run" "{" { RunItem } "}" .
RunItem        = Transaction | SpecialQuery | Query .

(* === Queries === *)

Query          = IDENT [ "(" QueryOptions ")" ]
                 [ RAWSTRING ]
                 [ "(" Args ")" ] .

QueryOptions   = QueryOption { "," QueryOption } .
QueryOption    = IDENT ":" OptionValue .
OptionValue    = Expr
               | "{" BraceMap "}"              (* print compound form *)
               | "{" InlineWeights "}" .       (* stage weights *)

Args           = PositionalArgs | NamedArgs .
PositionalArgs = Expr { "," Expr } .
NamedArgs      = NamedArg { "," NamedArg } .
NamedArg       = IDENT ":" Expr .

SpecialQuery   = "noop"
               | "rollback"
               | "rollback_if" Expr NEWLINE
               | IfBlock
               | MatchBlock .

(* === Transactions === *)

Transaction    = "transaction" IDENT [ "(" TxOptions ")" ]
                 "{" { TxLocal | SpecialQuery | Query } "}" .
TxOptions      = TxOption { "," TxOption } .
TxOption       = ( "wait" | "ignore" | "ignore_nested" ) ":" Expr .
TxLocal        = "let" IDENT "=" Expr NEWLINE .

(* === Control Flow === *)

IfBlock        = "if" Expr "{" { SpecialQuery | Query } "}"
                 [ "else" "{" { SpecialQuery | Query } "}" ] .

MatchBlock     = "match" Expr "{"
                 { WhenClause }
                 [ DefaultClause ]
                 "}" .
WhenClause     = "when" Expr "{" { SpecialQuery | Query } "}" .
DefaultClause  = "default" "{" { SpecialQuery | Query } "}" .

(* === Configuration Blocks === *)

WeightsBlock   = "weights" "{" { IDENT "=" NUMBER NEWLINE } "}" .

ExpectBlock    = "expect" "{" { Expectation } "}" .
Expectation    = IDENT [ RAWSTRING ] Expr NEWLINE .

StagesBlock    = "stages" "{" { StageEntry } "}" .
StageEntry     = IDENT "(" StageOptions ")" .
StageOptions   = StageOption { "," StageOption } .
StageOption    = "workers" ":" NUMBER
               | "duration" ":" DURATION
               | "ramp_duration" ":" DURATION
               | "qps" ":" NUMBER
               | "weights" ":" "{" InlineWeights "}" .
InlineWeights  = IDENT ":" NUMBER { "," IDENT ":" NUMBER } .

WorkersBlock   = "workers" "{" { Query } "}" .

TreeDecl       = "tree" IDENT [ "(" TreeOptions ")" ]
                 [ RAWSTRING ]
                 [ "(" Args ")" ]
                 [ "{" { LevelOverride } "}" ] .
TreeOptions    = TreeOption { "," TreeOption } .
TreeOption     = "levels" ":" "[" NUMBER { "," NUMBER } "]"
               | "id_column" ":" IDENT
               | "parent_column" ":" IDENT
               | "dag" ":" ( "true" | "false" )
               | QueryOption .
LevelOverride  = "level" NUMBER "{" { IDENT ":" Expr } "}" .

CompleteBlock  = "complete" "{" { ToolDecl } "}" .
ToolDecl       = "tool" IDENT [ "(" ToolOptions ")" ]
                 "{" { PropertyDecl } "}" .
ToolOptions    = "system" ":" STRING .
PropertyDecl   = IDENT [ "(" PropOptions ")" ] .
PropOptions    = PropOption { "," PropOption } .
PropOption     = ( "type" | "required" | IDENT ) ":" Expr .

(* === Expressions === *)

Expr           = (* expr-lang expression: arithmetic, comparisons,
                    logical operators, function calls, member access,
                    array literals, string literals, numbers, booleans,
                    nil/null *) .

(* === Lexical Tokens === *)

IDENT          = letter { letter | digit | "_" } .
NUMBER         = [ "-" ] digit { digit } [ "." digit { digit } ] .
STRING         = "'" { char | escape } "'"
               | '"' { char | escape } '"' .
RAWSTRING      = "`" { any } "`" .
NEWLINE        = "\n" | "\r\n" | "\r" .
DURATION       = NUMBER ( "ns" | "us" | "µs" | "ms" | "s" | "m" | "h" ) .

Appendix B: Keywords#

The following identifiers are reserved keywords:

Keywords may be used as identifiers in positions where identifiers are expected (e.g. query names, field names, option keys), per the parser’s expectIdent function which accepts both TokenIdent and keyword tokens.