Expectations PRO#

The expectations section defines assertions that are evaluated after the workload finishes. Each expectation is a boolean expression checked against the collected run metrics. If any expectation fails, edg prints the results and exits with a non-zero status code, making it suitable for CI/CD pipelines.

expect {
  error_rate < 1
  check_balance.p99 < 100
  tpm > 5000
}

Expressions use the same expr engine as arg expressions and must evaluate to a boolean.

Naming expectations#

Give an expectation a name to make log output easier to identify. The name replaces the raw expression in the check field of log messages:

expect {
  low_errors error_rate < 1
  fast_balance_check check_balance.p99 < 100
}

Output with names:

INFO expectation passed check=low_errors
INFO expectation passed check=fast_balance_check

On failure, the expression and resolved values are printed to aid debugging:

ERRO expectation failed check=low_errors expr="error_rate < 1" values="error_rate = 2.3"

Names are optional. Without a name, the expression itself is shown as the check label (the original behaviour).

Referencing globals#

Expectations can reference any variable defined in the globals section. This avoids hardcoding values that already exist in your config:

let accounts = 10000
let max_error_pct = 5

expect {
  error_rate < max_error_pct
  account_count `SELECT COUNT(*) AS cnt FROM account` cnt == accounts
}

Global names must not collide with built-in metrics (error_rate, error_count, success_count, tpm). edg will reject the config at startup if they do.

Database-backed expectations#

An expectation can include a SQL query that runs after the workload finishes. The first-row columns are available in the expression alongside globals and metrics. In edg-lang, place the query in backticks between the name and the expression:

expect {
  account_count `SELECT COUNT(*) AS cnt FROM account` cnt == accounts
  positive_balance `SELECT SUM(balance) AS total FROM account` total > 0
}

Available Metrics#

Expectations have access to globals, global metrics, and per-query metrics. All latency values are in milliseconds and all error rates are percentages (0-100).

Global metrics#

Per-query metrics#

Per-query metrics are accessed using dot notation with the query name, e.g. check_balance.p99. The query name must match the name field in your run section.

Examples#

Error rate threshold#

Ensure the overall error rate stays below 1%:

expect {
  error_rate < 1
}

Per-query latency and error check#

Ensure a specific query’s p99 latency stays under 100ms and its error rate is zero:

expect {
  check_balance.p99 < 100
  check_balance.errors == 0
}

Compound conditions#

Combine multiple conditions in a single expression:

expect {
  error_rate < 0.5 && tpm > 10000
}

Parameterised SLA thresholds#

Use globals to centralise thresholds so they can be overridden per-environment without editing expressions:

let max_error_pct = 5
let min_tpm = 1000

expect {
  error_rate < max_error_pct
  tpm > min_tpm
}

Zero-tolerance critical path#

Some queries must never fail. Use an exact count check instead of a rate:

expect {
  payment_process.error_count == 0
}

Latency ratio guard#

Ensure one query path isn’t disproportionately slower than another:

expect {
  slow_query.p99 / fast_query.p99 < 3
}

Tail latency stability#

A high p99/p90 ratio signals unpredictable outlier spikes:

expect {
  insert_order.p99 / insert_order.p90 < 5
}

Percentile cliff detection#

When p99 is close to p95, the tail is predictable. A large gap means something is falling off a cliff:

expect {
  insert_order.p99 - insert_order.p95 < 10
}

Throughput balance#

Verify that parallel read/write workloads stay roughly balanced, catching unexpected skew:

expect {
  reads.qps / writes.qps > 0.8
  reads.qps / writes.qps < 1.2
}

Data integrity after generation#

Verify the workload left the database in a valid state:

expect {
  no_negative `SELECT COUNT(*) AS cnt FROM account WHERE balance < 0` cnt == 0
}

Referential integrity#

Confirm foreign-key relationships survived the workload:

expect {
  no_orphans `SELECT COUNT(*) AS orphans FROM orders o LEFT JOIN customers c ON o.customer_id = c.id WHERE c.id IS NULL` orphans == 0
}

Cardinality check#

Verify uniqueness constraints hold after data generation:

expect {
  unique_emails `SELECT COUNT(DISTINCT email) AS uniq, COUNT(*) AS total FROM users` uniq == total
}

Data distribution#

Ensure data has sufficient variety (values are spread across expected categories):

expect {
  region_spread `SELECT COUNT(DISTINCT region) AS regions FROM warehouses` regions >= 4
}

Write amplification#

Detect unexpected triggers or cascades by comparing row counts against operation counts:

expect {
  write_amp `SELECT COUNT(*) AS rows FROM audit_log` rows <= success_count * 2
}

Migration validation#

Run the same workload before and after a schema change against identical thresholds. If expectations pass in both runs, the migration didn’t regress performance:

expect {
  error_rate < 1
  insert_account.p99 < 50
  tpm > 5000
  has_accounts `SELECT COUNT(*) AS cnt FROM account` cnt > 0
}

Comparing queries#

Per-query metrics can be compared against each other, not just against constants. This is useful for A/B-style tests; for example, verifying that an indexed lookup outperforms an unindexed one:

run {
  lookup_with_index `SELECT id, payload, created_at
    FROM event_indexed
    WHERE category = $1::INT` (
    gen('number:1,' + string(categories))
  )

  lookup_without_index `SELECT id, payload, created_at
    FROM event_unindexed
    WHERE category = $1::INT` (
    gen('number:1,' + string(categories))
  )
}

expect {
  error_rate < 1
  lookup_with_index.avg < lookup_without_index.avg
  lookup_with_index.p99 < lookup_without_index.p99
}

When the workload is finished and the expectations have been performed, you’ll see something similar to this:

INFO expectation passed check="lookup_with_index.avg < lookup_without_index.avg" lookup_with_index.avg=19.33 lookup_without_index.avg=36.42
INFO expectation passed check="lookup_with_index.p99 < lookup_without_index.p99" lookup_with_index.p99=35.00 lookup_without_index.p99=60.11

See the full index comparison example for a runnable config that creates, seeds, and compares indexed vs unindexed tables.

Variable values are printed alphabetically after the expression. Only variables that appear in the expression are shown and per-query metrics like check_balance.p99 are matched precisely, so a reference to check_balance.error_count will not also print the global error_count.

If any expectation fails, edg exits with status code 1 and reports the number of failures. When using the all command, teardown (down) still runs before the non-zero exit, so your database is left clean regardless of expectation results.

CI/CD Usage#

A typical CI pipeline step runs the workload and relies on the exit code to gate the build:

edg all \
  --driver pgx \
  --config workload.edg \
  --url "$DATABASE_URL" \
  -w 50 \
  -d 5m

If any expectation defined in workload.edg fails, the command exits with code 1, failing the pipeline step.

For a complete guide to using edg as an integration testing tool, see Integration Testing.

Inline Assertions (Assert)#

For fail-fast checks during execution rather than after, use the assert inline arg on individual queries. Inline assertions evaluate a boolean expression after each query execution in the run section. If the assertion fails, the entire run is cancelled immediately.

run {
  seed_users (assert: cnt > 0)
    `INSERT INTO users (email) VALUES ($1) RETURNING id AS cnt` (gen('email'))

  read_back (assert: success_count > 0 && error_rate < 5)
    `SELECT 1`
}

The assert expression can reference:

Per-query metrics track the current worker’s local counters, not the aggregate across all workers. Latency metrics (avg, p90, p95, p99) are in milliseconds.