Expectations#

The expectations section defines assertions that are evaluated after the workload finishes. Each expectation is a boolean expression checked against the collected run metrics. If any expectation fails, edg prints the results and exits with a non-zero status code, making it suitable for CI/CD pipelines.

expectations:
  - error_rate < 1
  - check_balance.p99 < 100
  - tpm > 5000

Expressions use the same expr engine as arg expressions and must evaluate to a boolean.

Naming expectations#

Give an expectation a name to make log output easier to identify. The name replaces the raw expression in the check field of log messages:

expectations:
  - name: low_errors
    expr: error_rate < 1
  - name: fast_balance_check
    expr: check_balance.p99 < 100

Output with names:

INFO expectation passed check=low_errors
INFO expectation passed check=fast_balance_check

On failure, the expression and resolved values are printed to aid debugging:

ERRO expectation failed check=low_errors expr="error_rate < 1" values="error_rate = 2.3"

Names are optional. Without a name, the expression itself is shown as the check label (the original behaviour).

Referencing globals#

Expectations can reference any variable defined in the globals section. This avoids hardcoding values that already exist in your config:

globals:
  accounts: 10000
  max_error_pct: 5

expectations:
  - error_rate < max_error_pct
  - query: SELECT COUNT(*) AS cnt FROM account
    expr: cnt == accounts

Global names must not collide with built-in metrics (error_rate, error_count, success_count, tpm). edg will reject the config at startup if they do.

Database-backed expectations#

An expectation can be an object with query and expr fields. The SQL query runs after the workload finishes and its first-row columns are available in the expression alongside globals and metrics:

expectations:
  - name: account_count
    query: SELECT COUNT(*) AS cnt FROM account
    expr: cnt == accounts
  - name: positive_balance
    query: SELECT SUM(balance) AS total FROM account
    expr: total > 0

Available Metrics#

Expectations have access to globals, global metrics, and per-query metrics. All latency values are in milliseconds and all error rates are percentages (0–100).

Global metrics#

Per-query metrics#

Per-query metrics are accessed using dot notation with the query name, e.g. check_balance.p99. The query name must match the name field in your run section.

Examples#

Error rate threshold#

Ensure the overall error rate stays below 1%:

expectations:
  - error_rate < 1

Per-query latency and error check#

Ensure a specific query’s p99 latency stays under 100ms and its error rate is zero:

expectations:
  - check_balance.p99 < 100
  - check_balance.errors == 0

Compound conditions#

Combine multiple conditions in a single expression:

expectations:
  - error_rate < 0.5 && tpm > 10000

Parameterised SLA thresholds#

Use globals to centralise thresholds so they can be overridden per-environment without editing expressions:

globals:
  max_error_pct: 5
  min_tpm: 1000

expectations:
  - error_rate < max_error_pct
  - tpm > min_tpm

Zero-tolerance critical path#

Some queries must never fail. Use an exact count check instead of a rate:

expectations:
  - payment_process.error_count == 0

Latency ratio guard#

Ensure one query path isn’t disproportionately slower than another:

expectations:
  - slow_query.p99 / fast_query.p99 < 3

Tail latency stability#

A high p99/p50 ratio signals unpredictable outlier spikes:

expectations:
  - insert_order.p99 / insert_order.p50 < 5

Percentile cliff detection#

When p99 is close to p95, the tail is predictable. A large gap means something is falling off a cliff:

expectations:
  - insert_order.p99 - insert_order.p95 < 10

Throughput balance#

Verify that parallel read/write workloads stay roughly balanced, catching unexpected skew:

expectations:
  - reads.qps / writes.qps > 0.8
  - reads.qps / writes.qps < 1.2

Data integrity after generation#

Verify the workload left the database in a valid state:

expectations:
  - query: SELECT COUNT(*) AS cnt FROM account WHERE balance < 0
    expr: cnt == 0

Referential integrity#

Confirm foreign-key relationships survived the workload:

expectations:
  - query: >-
      SELECT COUNT(*) AS orphans
      FROM orders o
      LEFT JOIN customers c ON o.customer_id = c.id
      WHERE c.id IS NULL
    expr: orphans == 0

Cardinality check#

Verify uniqueness constraints hold after data generation:

expectations:
  - query: SELECT COUNT(DISTINCT email) AS uniq, COUNT(*) AS total FROM users
    expr: uniq == total

Data distribution#

Ensure data has sufficient variety (values are spread across expected categories):

expectations:
  - query: SELECT COUNT(DISTINCT region) AS regions FROM warehouses
    expr: regions >= 4

Write amplification#

Detect unexpected triggers or cascades by comparing row counts against operation counts:

expectations:
  - query: SELECT COUNT(*) AS rows FROM audit_log
    expr: rows <= success_count * 2

Migration validation#

Run the same workload before and after a schema change against identical thresholds. If expectations pass in both runs, the migration didn’t regress performance:

expectations:
  - error_rate < 1
  - insert_account.p99 < 50
  - tpm > 5000
  - query: SELECT COUNT(*) AS cnt FROM account
    expr: cnt > 0

Comparing queries#

Per-query metrics can be compared against each other, not just against constants. This is useful for A/B-style tests; for example, verifying that an indexed lookup outperforms an unindexed one:

run:
  - name: lookup_with_index
    type: query
    args:
      - gen('number:1,' + string(categories))
    query: |-
      SELECT id, payload, created_at
      FROM event_indexed
      WHERE category = $1::INT

  - name: lookup_without_index
    type: query
    args:
      - gen('number:1,' + string(categories))
    query: |-
      SELECT id, payload, created_at
      FROM event_unindexed
      WHERE category = $1::INT

expectations:
  - error_rate < 1
  - lookup_with_index.avg < lookup_without_index.avg
  - lookup_with_index.p99 < lookup_without_index.p99

When the workload is finished and the expectations have been performed, you’ll see something similar to this:

INFO expectation passed check="lookup_with_index.avg < lookup_without_index.avg" lookup_with_index.avg=19.33 lookup_without_index.avg=36.42
INFO expectation passed check="lookup_with_index.p99 < lookup_without_index.p99" lookup_with_index.p99=35.00 lookup_without_index.p99=60.11

See the full index comparison example for a runnable config that creates, seeds, and compares indexed vs unindexed tables.

Variable values are printed alphabetically after the expression. Only variables that appear in the expression are shown and per-query metrics like check_balance.p99 are matched precisely, so a reference to check_balance.error_count will not also print the global error_count.

If any expectation fails, edg exits with status code 1 and reports the number of failures. When using the all command, teardown (deseed and down) still runs before the non-zero exit, so your database is left clean regardless of expectation results.

CI/CD Usage#

A typical CI pipeline step runs the workload and relies on the exit code to gate the build:

edg all \
  --driver pgx \
  --config workload.yaml \
  --url "$DATABASE_URL" \
  -w 50 \
  -d 5m

If any expectation defined in workload.yaml fails, the command exits with code 1, failing the pipeline step.

For a complete guide to using edg as an integration testing tool, see Integration Testing.