edg#

edg is an expression-based data generator that can be driven with YAML or a purpose-built DSL (Domain-Specific Language) called edg-lang.

It was built to help you understand and prepare for how your database will behave when production traffic arrives and prevent against logic and performance regressions. Connect to one of the supported databases, create database objects, seed lots of realistic data quickly, and define workloads (transactional or otherwise). Concurrent workers and real-time throughput reporting will take of the rest.

Query arguments are written as expressions compiled at startup, giving you access to global constants, random data generation, reference lookups, and a bunch of random distributions.

Quick start#

Install with the install script.

curl -fsSL https://raw.githubusercontent.com/codingconcepts/edg-releases/main/install.sh | sh

Install via Homebrew.

brew install codingconcepts/tap/edg

Run all of the config steps for the built-in TPC-C workload (one of 16 built-in workloads).

edg workload tpcc all \
  --driver <DRIVER> \
  --url <CONNECTION_STRING> \
  --workers 100 \
  --duration 10m

Run a custom workload.

edg all \
  --driver <DRIVER> \
  --url <CONNECTION_STRING> \
  --config <FILE/HTTP PATH TO CONFIG>.yaml \
  --workers 10 \
  --duration 10m

Who is edg for?#

If you interact with databases in any way, edg will have a place in your arsenal. Whether you’re a database administrator, a developer, or a product manager, edg:

  • Gives you a reproducible tool for testing workloads; even with complex relationships.
  • Can compare the impact of migrations from one database to another.
  • Can compare the impact of schema changes on database performance.
  • Can reconcile ETL pipelines betweend databases for consistency.
  • Can stress test your database, with high concurrency to isolate performance bottlenecks.
  • Allows you to run complex workloads and assert the performance and end result (e.g. consistency).

Supported databases#

DatabaseDriverURL (example)
PostgreSQLpgxpostgres://root@localhost:26257/db?sslmode=disable
MySQLmysqluser:password@tcp(host:port)/db?parseTime=true
Cassandracassandracassandra://user:pass@host1[,host2,host3]:9042/keyspace
MongoDBmongodbmongodb://localhost:27017/db
Aurora DSQL
PRO
dsqlclusterid.dsql.us-east-1.on.aws
Spanner
PRO
spannerprojects/PROJECT/instances/INSTANCE/databases/DATABASE
MSSQL
PRO
mssqlsqlserver://user:password@host:port?database=db&encrypt=disable
Oracle
PRO
oracleoracle://system:password@localhost:1521/db

Supported features#

Featurepgxmysqlmongodbcassandramssqloracledsqlspanner
up / seed / run / deseed / down☑️☑️☑️☑️☑️☑️☑️☑️
scaffold☑️☑️☑️☑️☑️☑️☑️☑️
sync (run / down / verify)☑️☑️☑️☑️☑️☑️☑️☑️
jobs (serve / submit / stream)☑️☑️☑️☑️☑️☑️☑️☑️
Prometheus metrics☑️☑️☑️☑️☑️☑️☑️☑️
Embeddings☑️☑️☑️☑️☑️☑️☑️☑️
LLM completions☑️☑️☑️☑️☑️☑️☑️☑️
init (schema introspection)☑️☑️⚠️🚫☑️☑️☑️☑️
Batch operations☑️☑️☑️☑️☑️☑️☑️☑️
Expectations☑️☑️☑️☑️☑️☑️☑️☑️
Plugins (WASM)☑️☑️☑️☑️☑️☑️☑️☑️
Prepared statements☑️☑️⚠️🚫☑️☑️☑️☑️
Stages☑️☑️☑️☑️☑️☑️☑️☑️
Transactions☑️☑️☑️ 1☑️ 2☑️☑️☑️☑️
Workers☑️☑️☑️☑️☑️☑️☑️☑️

☑️ Supported · 🚫 Not yet supported · ⚠️ Not applicable


  1. Implemented using multi-document transactions via client sessions. ↩︎

  2. Implemented using logged batches, which guarantee atomicity but not isolation. ↩︎