edg#
edg is an expression-based data generator that can be driven with YAML or a purpose-built DSL (Domain-Specific Language) called edg-lang.
It was built to help you understand and prepare for how your database will behave when production traffic arrives and prevent against logic and performance regressions. Connect to one of the supported databases, create database objects, seed lots of realistic data quickly, and define workloads (transactional or otherwise). Concurrent workers and real-time throughput reporting will take of the rest.
Query arguments are written as expressions compiled at startup, giving you access to global constants, random data generation, reference lookups, and a bunch of random distributions.
Quick start#
Install with the install script.
curl -fsSL https://raw.githubusercontent.com/codingconcepts/edg-releases/main/install.sh | shInstall via Homebrew.
brew install codingconcepts/tap/edgRun all of the config steps for the built-in TPC-C workload (one of 16 built-in workloads).
edg workload tpcc all \
--driver <DRIVER> \
--url <CONNECTION_STRING> \
--workers 100 \
--duration 10mRun a custom workload.
edg all \
--driver <DRIVER> \
--url <CONNECTION_STRING> \
--config <FILE/HTTP PATH TO CONFIG>.yaml \
--workers 10 \
--duration 10mWho is edg for?#
If you interact with databases in any way, edg will have a place in your arsenal. Whether you’re a database administrator, a developer, or a product manager, edg:
- Gives you a reproducible tool for testing workloads; even with complex relationships.
- Can compare the impact of migrations from one database to another.
- Can compare the impact of schema changes on database performance.
- Can reconcile ETL pipelines betweend databases for consistency.
- Can stress test your database, with high concurrency to isolate performance bottlenecks.
- Allows you to run complex workloads and assert the performance and end result (e.g. consistency).
Supported databases#
| Database | Driver | URL (example) |
|---|---|---|
| PostgreSQL | pgx | postgres://root@localhost:26257/db?sslmode=disable |
| MySQL | mysql | user:password@tcp(host:port)/db?parseTime=true |
| Cassandra | cassandra | cassandra://user:pass@host1[,host2,host3]:9042/keyspace |
| MongoDB | mongodb | mongodb://localhost:27017/db |
| Aurora DSQL PRO | dsql | clusterid.dsql.us-east-1.on.aws |
| Spanner PRO | spanner | projects/PROJECT/instances/INSTANCE/databases/DATABASE |
| MSSQL PRO | mssql | sqlserver://user:password@host:port?database=db&encrypt=disable |
| Oracle PRO | oracle | oracle://system:password@localhost:1521/db |
Supported features#
| Feature | pgx | mysql | mongodb | cassandra | mssql | oracle | dsql | spanner |
|---|---|---|---|---|---|---|---|---|
| up / seed / run / deseed / down | ☑️ | ☑️ | ☑️ | ☑️ | ☑️ | ☑️ | ☑️ | ☑️ |
| scaffold | ☑️ | ☑️ | ☑️ | ☑️ | ☑️ | ☑️ | ☑️ | ☑️ |
| sync (run / down / verify) | ☑️ | ☑️ | ☑️ | ☑️ | ☑️ | ☑️ | ☑️ | ☑️ |
| jobs (serve / submit / stream) | ☑️ | ☑️ | ☑️ | ☑️ | ☑️ | ☑️ | ☑️ | ☑️ |
| Prometheus metrics | ☑️ | ☑️ | ☑️ | ☑️ | ☑️ | ☑️ | ☑️ | ☑️ |
| Embeddings | ☑️ | ☑️ | ☑️ | ☑️ | ☑️ | ☑️ | ☑️ | ☑️ |
| LLM completions | ☑️ | ☑️ | ☑️ | ☑️ | ☑️ | ☑️ | ☑️ | ☑️ |
| init (schema introspection) | ☑️ | ☑️ | ⚠️ | 🚫 | ☑️ | ☑️ | ☑️ | ☑️ |
| Batch operations | ☑️ | ☑️ | ☑️ | ☑️ | ☑️ | ☑️ | ☑️ | ☑️ |
| Expectations | ☑️ | ☑️ | ☑️ | ☑️ | ☑️ | ☑️ | ☑️ | ☑️ |
| Plugins (WASM) | ☑️ | ☑️ | ☑️ | ☑️ | ☑️ | ☑️ | ☑️ | ☑️ |
| Prepared statements | ☑️ | ☑️ | ⚠️ | 🚫 | ☑️ | ☑️ | ☑️ | ☑️ |
| Stages | ☑️ | ☑️ | ☑️ | ☑️ | ☑️ | ☑️ | ☑️ | ☑️ |
| Transactions | ☑️ | ☑️ | ☑️ 1 | ☑️ 2 | ☑️ | ☑️ | ☑️ | ☑️ |
| Workers | ☑️ | ☑️ | ☑️ | ☑️ | ☑️ | ☑️ | ☑️ | ☑️ |
☑️ Supported · 🚫 Not yet supported · ⚠️ Not applicable