Embed#

embed(text...) generates real vector embeddings via an OpenAI-compatible embedding API. Unlike vector/vector_zipf/vector_norm which produce synthetic clustered vectors, embed calls a live embedding model so that similarity search reflects actual semantic relationships.

The function is variadic - multiple arguments are joined with a space before embedding.

args:
  - embed('hello world')
  - embed(field('name'), field('description'))

Configuration#

Flag	Env Var	Default
`--embed-api-key`	`EDG_EMBED_API_KEY`	(required)
`--embed-url`	`EDG_EMBED_URL`	`https://api.openai.com/v1/embeddings`
`--embed-model`	`EDG_EMBED_MODEL`	`text-embedding-3-small`
`--embed-dimensions`	`EDG_EMBED_DIMENSIONS`	`1536`
`--embed-max-batch`	`EDG_EMBED_MAX_BATCH`	`0` (unlimited)

Any OpenAI-compatible API works (Ollama, vLLM, Azure OpenAI, etc.) - set --embed-url to point at your endpoint.

Execution Modes#

Immediate mode (exec/query)#

Each embed() call makes a separate API request:

run:
  - name: insert_product
    type: exec
    args:
      - ref_same('product_catalog').name
      - ref_same('product_catalog').description
      - embed(ref_same('product_catalog').name, ref_same('product_catalog').description)
    query: |-
      INSERT INTO product (name, description, embedding)
      VALUES ($1, $2, $3::VECTOR)

With 100 iterations, this makes 100 API calls (one per row).

Deferred mode (exec_batch/query_batch)#

In batch queries, embed() calls are deferred - placeholders are inserted during arg evaluation, then all pending texts are resolved together at the end of each batch:

seed:
  - name: populate_product
    type: exec_batch
    count: 100
    size: 50
    args:
      - ref_each(product_catalog).name
      - ref_each(product_catalog).description
      - embed(ref_each(product_catalog).name, ref_each(product_catalog).description)
    query: |-
      INSERT INTO product (name, description, embedding)
      SELECT n, d, e::VECTOR
      FROM unnest(ARRAY[$1], ARRAY[$2], ARRAY[$3]) AS t(n, d, e)

With count: 100 and size: 50, there are 2 batches of 50. Each batch collects 50 texts, then resolves them in a single API call - 2 API calls instead of 100.

Use --embed-max-batch to cap texts per API call. For example, --embed-max-batch 30 on a 50-row batch produces 2 API calls (30+20) per batch, or 4 total.

Embedding values are substituted as strings in batch SQL. Cast to your vector type in the query: e::VECTOR.

Example#

A full working example is available at _examples/embed/.

globals:
  products: 50
  batch_size: 10

reference:
  product_catalog: !include products.yaml

up:
  - name: create_product
    query: |-
      CREATE TABLE IF NOT EXISTS product (
        id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
        name STRING NOT NULL,
        description STRING NOT NULL,
        embedding VECTOR(512) NOT NULL
      )

seed:
  - name: populate_product
    type: exec_batch
    count: products
    size: batch_size
    args:
      - ref_each(product_catalog).name
      - ref_each(product_catalog).description
      - embed(ref_each(product_catalog).name, ref_each(product_catalog).description)
    query: |-
      INSERT INTO product (name, description, embedding)
      SELECT n, d, e::VECTOR
      FROM unnest(ARRAY[$1], ARRAY[$2], ARRAY[$3]) AS t(n, d, e)

run:
  - name: find_similar
    type: query
    args:
      - embed(ref_rand('product_catalog').name)
    query: |-
      SELECT id, name, embedding <=> $1::VECTOR AS distance
      FROM product
      ORDER BY embedding <=> $1::VECTOR
      LIMIT 10

Using with Ollama#

edg seed \
  --embed-api-key ollama \
  --embed-url http://localhost:11434/v1/embeddings \
  --embed-model nomic-embed-text \
  --embed-dimensions 768 \
  ...

When changing --embed-dimensions, update the VECTOR(N) column type in your YAML to match.