Docker

OpenRAG is most comprehensively deployed using Docker.

Before proceeding, ensure your hardware meets the recommended specifications.
Install Docker.

The OpenRAG docker image is available on DockerHub and the GitHub Container Registry.

Docker Compose

OpenRAG requires several services to run, which can be orchestrated using Docker Compose.

Grafana monitoring

The optional monitoring stack provides ready-to-use dashboards for OpenRAG HTTP traffic, host resources, and NVIDIA GPUs. It is separate from the default stack so deployments that do not need monitoring keep the same footprint.

Before starting it, set a strong GRAFANA_ADMIN_PASSWORD and a random METRICS_TOKEN (openssl rand -hex 16) in .env. The GRAFANA_ADMIN_USER defaults to admin. The bundled Prometheus scrapes GET /metrics on the API over the Compose network with that token, which the overlay hands to it directly; nothing else to mount. See Prometheus metrics to scrape the endpoint from an external Prometheus.

Start OpenRAG with the monitoring overlay:

export SHARED_ENV="$PWD/.env"
docker compose --env-file .env \
  -f docker-compose.yaml \
  -f monitoring.docker-compose.yaml \
  up -d

GRAFANA_URL controls where System > Metrics opens. It must be reachable from the user’s browser. For direct local access, use:

GRAFANA_URL=http://localhost:3000/d/openrag-http/openrag-http-metrics

For remote deployments, expose Grafana through the Admin UI proxy and configure its subpath:

GRAFANA_URL=https://openrag.example.com/grafana/d/openrag-http/openrag-http-metrics
GF_SERVER_ROOT_URL=https://openrag.example.com/grafana/
GF_SERVER_SERVE_FROM_SUB_PATH=true

Grafana has its own login. Its admin password is initialized when the persistent Grafana volume is created; changing the environment variable later does not reset an existing password.

Log shipping to Loki

An opt-in overlay ships the API and worker logs to an external Loki:

docker compose -f docker-compose.yaml -f logging.docker-compose.yaml up -d

Set LOKI_URL (and the optional LOKI_USERNAME, LOKI_PASSWORD, LOKI_TENANT_ID) in .env. See Loki logs.