Milvus Migrations

Milvus Upgrade

OpenRAG now uses Milvus 3.0.1 with PyMilvus 3.0.1. Existing Milvus 2.6 deployments can keep their data, but they must prepare the Milvus data directory before starting the new image because Milvus 3 runs as a non-root user.

Fresh installations need no manual ownership step. The Compose stack initializes empty bind mounts and named volumes before the server starts. If existing files are not already owned by the Milvus 3 user, startup stops with a migration error instead of risking a partially writable deployment. The procedure below remains mandatory for existing data in either storage mode because all files created by Milvus 2.6 must be transferred to the Milvus 3 user.

Temporal support introduced in 2.6.x

Milvus 2.6.6+ introduced the TIMESTAMPTZ field type, which enables:

Comparison and range filtering using standard operators (=, !=, <, >, etc.)
Interval arithmetic — add or subtract durations (days, hours, minutes) directly in filter expressions
Time-based indexing for faster temporal queries
Combined filtering — pair timestamp conditions with vector similarity search

Example — basic comparison:

expr = "tsz != ISO '2025-01-03T00:00:00+08:00'"
results = client.query(
    collection_name,
    filter=expr,
    output_fields=["id", "tsz"],
    limit=10
)

Example — interval arithmetic:

expr = "tsz + INTERVAL 'P1D' > ISO '2025-01-03T00:00:00+08:00'"
results = client.query(
    collection_name,
    filter=expr,
    output_fields=["id", "tsz"],
    limit=10
)

INTERVAL values follow ISO 8601 duration syntax:

P1D = 1 day

PT3H = 3 hours

P2DT6H = 2 days and 6 hours.

Milvus Version Upgrade Steps

For the full official reference, see the Milvus upgrade guide.

Upgrade from Milvus 2.6.x to 3.0.1

Milvus 2.6 containers ran as root, so existing bind-mounted files and Docker named-volume contents are commonly owned by root:root. Milvus 3 runs as UID/GID 999:999 and cannot start until it owns the directory mounted at /var/lib/milvus.

The message-queue type must also remain unchanged during the upgrade. Before stopping Milvus 2.6, inspect its startup logs and identify the effective mqType or walName:

docker compose logs --no-color milvus \
  | grep -E 'mqType=|walName=' \
  | tail -20

Set MILVUS_MQ_TYPE in the Compose .env file to the detected value, such as rocksmq or woodpecker. If the logs do not show a clear value, confirm it in the Milvus WebUI configuration view before continuing. Do not combine the version upgrade with a queue migration; use the separate Milvus queue-switch procedure after version 3 is healthy.

Run the procedure for the storage profile selected in infra/compose/.env.

Bind-mounted storage

While the Milvus 2.6 stack still exists, confirm that /var/lib/milvus is a bind mount:

MILVUS_CONTAINER="$(docker compose ps -q milvus)"
docker inspect "$MILVUS_CONTAINER" \
  --format '{{range .Mounts}}{{if eq .Destination "/var/lib/milvus"}}type={{.Type}} source={{.Source}}{{end}}{{end}}'
# Expected: type=bind

Stop the stack, take the backup described above, and then transfer ownership from a temporary service container. Running the change through Docker makes it apply to the daemon-side mount and preserves the correct ownership mapping with remote, rootless, and user-namespaced Docker. Do not add --volumes to the shutdown command.

docker compose down
docker compose run --rm --no-deps --user 0:0 \
  --entrypoint chown milvus -R 999:999 /var/lib/milvus

Docker named-volume storage

Docker Desktop and remote Docker contexts do not expose a named volume’s daemon-side path to the local machine. Stop the stack without removing volumes, take the backup described above, and transfer ownership from a temporary service container instead:

docker compose down
docker compose run --rm --no-deps --user 0:0 \
  --entrypoint chown milvus -R 999:999 /var/lib/milvus

The command must run with MILVUS_COMPOSE=milvus/milvus.named-volumes.yaml, matching the existing deployment. Do not use docker compose down --volumes during this procedure.

Update OpenRAG, then pull and start the dependencies followed by Milvus:

docker compose pull milvus
docker compose up -d etcd minio
docker compose up -d milvus

Do not restart OpenRAG until Milvus is healthy and the logs contain no permission or migration errors:

docker compose ps milvus
docker compose logs --tail 100 milvus
docker inspect "$(docker compose ps -q milvus)" --format '{{.Config.Image}}'
# Expected: milvusdb/milvus:v3.0.1

Once those checks pass, start the complete stack:

docker compose up -d

Helm deployments

The bundled Helm chart now installs the Milvus 3-compatible chart and image. Before upgrading an existing release, stop writes and back up the MinIO and etcd volumes, together with any standalone or log volume enabled through custom values.

The upstream repository index does not yet publish chart 5.0.26, so OpenRAG currently uses chart 5.0.25 with the Milvus 3 image selected explicitly. Resources rendered by that chart can therefore retain the label app.kubernetes.io/version: "2.6.21". This label describes the chart’s default application version, not the running server; verify the container image as shown below.

Milvus 3 runs with GID 999. The chart sets fsGroup: 999 with the Always policy so Kubernetes recursively makes supported mounted volumes writable before Milvus starts. The first startup can therefore take longer when a volume contains many files.

Some storage drivers do not support fsGroup ownership changes. Check the driver’s fsGroupPolicy before upgrading. If it is None, migrate the affected Milvus volume to group 999 using the storage provider’s procedure before starting Milvus 3; otherwise the pods can fail with permission errors.

OpenShift’s restricted security context constraints normally assign an fsGroup from the project’s permitted range and can reject the fixed group 999. On those clusters, set milvus.securityContext to null in the OpenRAG values so admission can assign the allowed group. Confirm the admitted pod security context and volume ownership in a staging namespace before upgrading production.

After the upgrade, confirm that every Milvus workload is ready and uses the expected image:

kubectl get pods -n <namespace> -l app.kubernetes.io/name=milvus
kubectl get pods -n <namespace> -l app.kubernetes.io/name=milvus \
  -o jsonpath='{range .items[*]}{.metadata.name}{"\t"}{.spec.containers[*].image}{"\n"}{end}'
# Expected Milvus image: milvusdb/milvus:v3.0.1

Legacy path: upgrade Milvus 2.5.x to 2.6.11

This intermediate path is only for deployments upgrading from OpenRAG <= 1.1.7 whose current Milvus server is 2.5.x. Do not jump directly from Milvus 2.5.x to 3.0.1. Earlier Milvus versions can require additional intermediate or metadata migrations that are outside this procedure.

Step 1 — Upgrade Milvus to 2.5.16

Milvus requires an intermediate upgrade to v2.5.16 before jumping to 2.6.x. This step must be done manually before updating OpenRAG.

Temporarily edit infra/compose/milvus/milvus.yaml to set the intermediate Milvus image:

milvus:
  image: milvusdb/milvus:v2.5.4
  image: milvusdb/milvus:v2.5.16

Then restart Milvus and wait for it to be healthy:

docker compose down
docker compose up milvus -d

Verify it is running and healthy before continuing:

docker inspect milvus-standalone --format '{{ .Config.Image }}'
# Expected: milvusdb/milvus:v2.5.16

Preserve the message queue

OpenRAG 2.5 standalone deployments normally use RocksMQ, but verify the effective source queue using the log check in the 2.6-to-3.0 procedure above. Keep that detected value in every Compose configuration used for the 2.6.11 and 3.0.1 steps.

Before starting the intermediate OpenRAG release, add MQ_TYPE: <detected-value> to the Milvus service’s environment. When moving to the current release, set the corresponding value in the Compose .env file and keep it until the upgrade is validated:

MILVUS_MQ_TYPE=<detected-value>

Step 2 — Upgrade Milvus to 2.6.11

Once Milvus 2.5.16 is healthy, stop all services and move to an OpenRAG release that still packages Milvus 2.6.11, such as OpenRAG 2.1.0. This intermediate release also provides the required MinIO and etcd versions.

docker compose down

Verify that all containers are stopped:

docker ps | grep milvus

Pull or checkout the new OpenRAG release, then start the stack:

docker compose up -d

Confirm the running Milvus version:

docker inspect milvus-standalone --format '{{ .Config.Image }}'
# Expected: milvusdb/milvus:v2.6.11

After verifying that the collections are readable and searchable on 2.6.11, continue with the 2.6-to-3.0 procedure above.

Schema Migrations

OpenRAG ships a generic migration runner that discovers and applies all pending Milvus schema migrations in order. You never need to invoke individual migration scripts by hand.

Migrations are versioned. The runner reads the current schema version stored in the collection’s properties and only applies scripts that bring the collection forward (or backward) from that version.

Version 2 — case-insensitive BM25 analyzer

Schema version 2 makes two changes to the text field:

adds the lowercase filter to the analyzer, so BM25 stops treating Rapport and rapport as different terms. Milvus applies the analyzer to the query as well as to the indexed text, so before this fix a case mismatch scored zero on the lexical leg;
drops enable_match. Nothing in OpenRAG issues a TEXT_MATCH query, and while it is set Milvus refuses to alter the analyzer.

It rebuilds the collection rather than altering it, because both changes are refused in place: analyzer_params cannot change while text match is enabled, and enable_match itself cannot be turned off. The migration copies the rows into <collection>_v2_rebuild, checks the counts, then swaps the names, keeping the old collection as <collection>_v1_backup.

This is the last rebuild. Once enable_match is gone, a later analyzer change is three calls on the live collection with the sparse column backfilled by compaction — provided Storage V3 is enabled.

Disk — both copies exist at once; allow roughly twice the collection size until the backup is dropped.
Memory — checking the row counts loads both collections, and a rename does not unload one, so the migration releases the backup from query-node memory after the swap. The doubling above is disk, not RAM.
No re-embedding — dense vectors are copied verbatim and the sparse ones are regenerated server-side on insert, which is what applies the new analyzer.
Chunk IDs are preserved — _id is auto_id, which normally refuses caller-supplied keys, so the rebuild collection carries allow_insert_auto_id for the duration of the copy and drops it again before the swap. A chunk ID handed out earlier by the MCP get_chunk_by_id tool still resolves.
Stop OpenRAG first, as the banner above says — the copy reads a snapshot, so a row written while it runs would be missing from the rebuild. The migration re-counts the source afterwards and aborts, leaving the original untouched, if the count moved.
hybrid_search: false deployments have no BM25 leg and are only re-stamped — no rebuild, and no backup collection, in either direction.
A failed copy leaves the original untouched; drop the partial <collection>_v2_rebuild and retry.
The version check runs on the indexing path only. An un-migrated deployment still boots and still answers searches — with the old analyzer — and fails the first upload with Collection ... is at schema version 1 but the application requires version 2.

Once validated, drop the backup:

from pymilvus import MilvusClient

MilvusClient(uri="http://localhost:19530").drop_collection("<collection>_v1_backup")

The commands below run from infra/compose/. From the repo root, point Compose at the file (and at your project name, if you use one):

docker compose -p <project> -f infra/compose/docker-compose.yaml run --no-deps --rm --build --entrypoint "" openrag \
    uv run python services/persistence/migrations/milvus/migrate.py --dry-run

The migration runner is a file, not a package — pass the full path to migrate.py, and keep Milvus running.

Step 1 — Start only the Milvus container

docker compose up -d milvus

Wait until Milvus is healthy:

docker compose ps milvus

Step 2 — Dry-run (inspect, no changes)

docker compose run --no-deps --rm --build --entrypoint "" openrag \
    uv run python services/persistence/migrations/milvus/migrate.py --dry-run

docker compose --profile cpu run --no-deps --rm --build --entrypoint "" openrag-cpu \
    uv run python services/persistence/migrations/milvus/migrate.py --dry-run

Review the output to confirm which migrations are pending and what changes they would apply. The migration scripts ship inside the image, which is why --build is there: if the Discovered N migration(s) line does not list the migration you expect, the image is stale — and on a shared host another checkout may have rebuilt the same linagoraai/openrag tag, so add --build to the apply step too.

Step 3 — Apply all pending migrations

docker compose run --no-deps --rm --build --entrypoint "" openrag \
    uv run python services/persistence/migrations/milvus/migrate.py

docker compose --profile cpu run --no-deps --rm --build --entrypoint "" openrag-cpu \
    uv run python services/persistence/migrations/milvus/migrate.py

The runner will apply each pending migration in order. For the add_temporal_fields migration (v0 → v1) this means:

Adding the nullable TIMESTAMPTZ field created_at
Creating an STL_SORT index on that field
Stamping the collection with schema_version=1 so OpenRAG no longer reports a migration error on startup

Step 4 — Restart OpenRAG

docker compose up --build -d

docker compose --profile cpu up --build -d

Targeting a specific version

To upgrade or downgrade to a specific schema version rather than the latest:

# Upgrade to version 2 only
docker compose run --no-deps --rm --build --entrypoint "" openrag \
    uv run python services/persistence/migrations/milvus/migrate.py --target 2

# Downgrade to version 0 (resets version stamp and drops indexes)
docker compose run --no-deps --rm --build --entrypoint "" openrag \
    uv run python services/persistence/migrations/milvus/migrate.py --downgrade --target 0

# Upgrade to version 2 only
docker compose --profile cpu run --no-deps --rm --build --entrypoint "" openrag-cpu \
    uv run python services/persistence/migrations/milvus/migrate.py --target 2

# Downgrade to version 0 (resets version stamp and drops indexes)
docker compose --profile cpu run --no-deps --rm --build --entrypoint "" openrag-cpu \
    uv run python services/persistence/migrations/milvus/migrate.py --downgrade --target 0

Rollback

What a downgrade can undo depends on the migration:

Version 2 → 1 swaps the <collection>_v1_backup collection back into place and sets the version stamp to 1. The version-2 collection is kept aside as <collection>_v2_rolled_back rather than dropped, so nothing is lost — but this is a point-in-time rollback: rows indexed after the upgrade exist only in the collection set aside.

The rollback needs that backup collection. If it is gone, the downgrade fails rather than reporting a rollback it did not perform, and there are three ways forward:
- Restore <collection>_v1_backup from a Milvus backup and re-run the downgrade.
- Re-create the collection with the v1 schema and re-index it.
- Move the version stamp back by hand — but only if you know this collection was re-stamped rather than rebuilt. A collection that already had the v2 analyzer and no text match before the upgrade was moved to version 2 by the stamp alone, so it correctly has no backup. Afterwards it is indistinguishable from one that was rebuilt and lost its backup, which is why the migration refuses to guess between them.
A hybrid_search: false collection never had a backup — it was only re-stamped on the way up, so the rollback moves its stamp back and nothing else.
Version 1 → 0 only removes indexes and resets the version stamp, since Milvus does not support dropping fields — they remain in the schema but are unused by the application.

docker compose run --no-deps --rm --build --entrypoint "" openrag \
    uv run python services/persistence/migrations/milvus/migrate.py --downgrade

docker compose --profile cpu run --no-deps --rm --build --entrypoint "" openrag-cpu \
    uv run python services/persistence/migrations/milvus/migrate.py --downgrade

To fully remove the fields you would need to recreate the collection from scratch.

Adding a New Migration Script

Migration scripts live in openrag/services/persistence/migrations/milvus/. The runner discovers them automatically — no registration step required.

Naming convention

Files must follow the pattern N.short_description.py, where N is the target schema version as a positive integer:

openrag/services/persistence/migrations/milvus/
  1.add_created_at_temporal_fields.py  ← brings the schema to version 1
  2.rebuild_text_analyzer.py           ← brings the schema to version 2
  3.your_new_migration.py              ← brings the schema to version 3
  migrate.py                           ← generic runner (do not rename)

The numeric prefix determines execution order. Never reuse or change an existing version number.

Required module interface

Each migration script must expose the following at module level:

Name	Type	Description
`TARGET_VERSION`	`int`	The schema version this script brings the collection to
`upgrade(client, collection_name, dry_run)`	`function`	Applies the migration
`downgrade(client, collection_name, dry_run)`	`function`	Reverts the migration, as far as Milvus allows — a field added in `upgrade` cannot be dropped, so such a migration only removes its indexes and resets the stamp

Minimal template

"""
Milvus migration: <description>  (schema version N-1 → N)
"""

from pymilvus import DataType, MilvusClient
from core.utils.logging import get_logger

TARGET_VERSION = N  # replace with the actual version number

FIELDS_2_ADD = [
    {"field_name": "my_field", "data_type": DataType.VARCHAR, "max_length": 256, "nullable": True},
]

INDEXES_2_ADD = [
    # add index specs here if needed
]

logger = get_logger()


def upgrade(client: MilvusClient, collection_name: str, dry_run: bool = False) -> None:
    # Add fields and indexes, then bump the version property.
    ...


def downgrade(client: MilvusClient, collection_name: str, dry_run: bool = False) -> None:
    # Drop indexes and reset the version property.
    # Note: Milvus does not support dropping fields.
    ...

Use 1.add_temporal_fields.py as a reference implementation for the full upgrade/downgrade pattern.