Embed#

embed(text...) generates real vector embeddings via an OpenAI-compatible embedding API. Unlike vector/zipf.vector/norm.vector which produce synthetic clustered vectors, embed calls a live embedding model so that similarity search reflects actual semantic relationships.

The function is variadic - multiple arguments are joined with a space before embedding.

(embed('hello world'))
(embed(field('name'), field('description')))

Configuration#

Any OpenAI-compatible API works (Ollama, vLLM, Azure OpenAI, etc.) - set --embed-url to point at your endpoint.

Execution Modes#

Immediate mode (exec/query)#

Each embed() call makes a separate API request:

run {
  insert_product
    `INSERT INTO product (name, description, embedding)
     VALUES ($1, $2, $3::VECTOR)` (
    ref_same('product_catalog').name,
    ref_same('product_catalog').description,
    embed(ref_same('product_catalog').name, ref_same('product_catalog').description)
  )
}

With 100 iterations, this makes 100 API calls (one per row).

Deferred mode (exec_batch/query_batch)#

In batch queries, embed() calls are deferred - placeholders are inserted during arg evaluation, then all pending texts are resolved together at the end of each batch:

seed {
  populate_product(count: 100, size: 50)
    `INSERT INTO product (name, description, embedding)
     SELECT n, d, e::VECTOR
     FROM unnest(ARRAY[$1], ARRAY[$2], ARRAY[$3]) AS t(n, d, e)` (
    ref_each(product_catalog).name,
    ref_each(product_catalog).description,
    embed(ref_each(product_catalog).name, ref_each(product_catalog).description)
  )
}

With count: 100 and size: 50, there are 2 batches of 50. Each batch collects 50 texts, then resolves them in a single API call - 2 API calls instead of 100.

Use --embed-max-batch to cap texts per API call. For example, --embed-max-batch 30 on a 50-row batch produces 2 API calls (30+20) per batch, or 4 total.

Embedding values are substituted as strings in batch SQL. Cast to your vector type in the query: e::VECTOR.

Example#

A full working example is available at examples/embed/.

import 'products.edg'
# products.edg exports: pub ref product_catalog [...]

let products = 50
let batch_size = 10

up {
  create_product `CREATE TABLE IF NOT EXISTS product (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    name STRING NOT NULL,
    description STRING NOT NULL,
    embedding VECTOR(512) NOT NULL
  )`
}

seed {
  populate_product(count: products, size: batch_size)
    `INSERT INTO product (name, description, embedding)
     SELECT n, d, e::VECTOR
     FROM unnest(ARRAY[$1], ARRAY[$2], ARRAY[$3]) AS t(n, d, e)` (
    ref_each(product_catalog).name,
    ref_each(product_catalog).description,
    embed(ref_each(product_catalog).name, ref_each(product_catalog).description)
  )
}

run {
  find_similar
    `SELECT id, name, embedding <=> $1::VECTOR AS distance
     FROM product
     ORDER BY embedding <=> $1::VECTOR
     LIMIT 10` (
    embed(ref('product_catalog').name)
  )
}

Using with Ollama#

edg seed \
  --embed-api-key ollama \
  --embed-url http://localhost:11434/v1/embeddings \
  --embed-model nomic-embed-text \
  --embed-dimensions 768 \
  ...

When changing --embed-dimensions, update the VECTOR(N) column type in your config to match.