Quick Start

OpenRAG is an open source Retrieval Augmented Generation (RAG) solution. This guide is a step-by-step walkthrough to help you get started with OpenRAG.

Docker

Prerequisites

• Docker and Docker Compose
• Your hardware should meet these specifications:
  • CPU deployment: Minimum 13 GiB RAM for light PDF parsers (PyMuPDFLoader), or 23 GiB RAM for heavier parsers like MarkerLoader (refer to this section for details)
  • GPU deployment: 16 GB GPU memory recommended (for systems with separate CPU and GPU memory)

Installation and Configuration

1. Clone the repository:

Cloning the OpenRag repository

git clone --recurse-submodules git@github.com:linagora/openrag.git

cd openrag/
git checkout main # or a given release

2. Create a .env File

The Docker Compose stack lives under infra/compose/, next to its .env.example template. Copy it to a .env in the same folder and fill in the blanks (LLM/VLM endpoints, embedder, and the required MinIO / Postgres secrets):

Create your .env from the template

cd infra/compose
cp .env.example .env

Here is the minimal set of variables to get started — see the full environment-variable reference for every other option:

infra/compose/.env

# ============================================================================
# OpenRAG — minimal .env
#
# Only the variables you must set for the default compose stack to boot are
# listed here. Every other knob (PDF/audio loaders, chunking, retriever,
# reranker, Ray Serve, admin UI, OIDC/SSO, rate limiting, MCP server, web
# search, …) has a sensible default and is documented in full at:
#
#   https://linagora.github.io/openrag/documentation/env_vars/
# ============================================================================

# ── LLM (external, OpenAI-compatible) ───────────────────────────────────────
BASE_URL=
API_KEY=
MODEL=
LLM_SEMAPHORE=10

# ── VLM (vision model, used for image understanding) ────────────────────────
# Can reuse the LLM values above if that model accepts images.
VLM_BASE_URL=
VLM_API_KEY=
VLM_MODEL=
VLM_SEMAPHORE=20

# ── Embedder (HuggingFace model served by the bundled vLLM) ─────────────────
# Point these at an external embedding service instead of the bundled vLLM:

EMBEDDER_MODEL_NAME=jinaai/jina-embeddings-v3
# EMBEDDER_BASE_URL=http://vllm:8000/v1
# EMBEDDER_API_KEY=EMPTY
# MAX_MODEL_LEN=2047

# ── Reranker (re-scores retrieved chunks; bundled Infinity server) ───────────
RERANKER_PROVIDER=infinity  # 'infinity' (bundled, default) or 'openai' (external endpoint)
RERANKER_MODEL=Alibaba-NLP/gte-multilingual-reranker-base
RERANKER_ENABLED=true

## Point these at an external reranker instead of the bundled Infinity / VLLM(Openai) server:
# RERANKER_BASE_URL=http://reranker:7997
# RERANKER_API_KEY=EMPTY


# ── PDF parser (default: PyMuPDF — lightweight, CPU-friendly) ────────────────
# Switch to MarkerLoader for OCR / scanned PDFs, complex layouts & embedded
# images (heavier — more RAM/GPU). Other option: DoclingLoader.
PDFLOADER=PyMuPDFLoader
# Marker tuning (only when PDFLOADER=MarkerLoader):
# MARKER_POOL_SIZE=1         # marker worker actors (≈ 1 per cluster node / Machine).
# MARKER_MAX_PROCESSES=2     # concurrent PDFs per worker (raise with more GPU)

# ── Image captioning & chunk contextualization (both ON by default) ──────────
# Both run during indexing and call the VLM/LLM — set to false to index
# faster and cheaper (with some retrieval-quality trade-off).
# IMAGE_CAPTIONING=false       # stop describing images in documents via the VLM
# CONTEXTUAL_RETRIEVAL=false   # stop prepending LLM-generated context to each chunk (Anthropic technique)

# ── Secrets — dev defaults so the stack boots out of the box ─────────────────
# ⚠️  Production: replace these with strong values, e.g. `openssl rand -hex 16`.
# MINIO_* are shared by the minio service and Milvus — both sides must match.
MINIO_ACCESS_KEY=minioadmin
MINIO_SECRET_KEY=minioadmin
POSTGRES_PASSWORD=postgres
# POSTGRES_USER=root

# ── API and Chainlit (chat interface) authentication ──────────────────────────────
# Bearer token that bootstraps the admin user and guards the API.
# ⚠️  Production: replace with a strong value, e.g. `openssl rand -hex 16`.
# For SSO instead, see https://linagora.github.io/openrag/documentation/oidc/
AUTH_TOKEN=or-openrag-1234
# AUTH_MODE=token          # 'token' (default) or 'oidc' for SSO
# SUPER_ADMIN_MODE=true

# # FastAPI port (the API and Chainlit chat interface share the same port).
# APP_PORT=8080

# # Rate limiting, activated by default (recommended).
# RATE_LIMIT_ENABLED=false


# ──────────────── Chainlit: Chat interface ────────────────
# Session-cookie signing secret — required once auth is enabled; keep it stable.
# ⚠️  Production: replace with a generated value:
#   python -c "import secrets; print(secrets.token_urlsafe(32))"
CHAINLIT_AUTH_SECRET=openrag-dev-secret


# ──────────────── Admin / Indexer UI (React SPA) ────────────────
# Host port for the admin UI — the document ingestion, indexing & management
# interface, served at http://<host>:<ADMIN_UI_PORT>/app/. It also proxies the
# API/auth, so it is the OIDC front door. Zero-config otherwise (same-origin, no
# CORS); VITE_* build-time options are documented in the env vars reference.
# ADMIN_UI_PORT=8081



# ── Ray (kept as-is by the compose stack; see the docs for what each does) ───
RAY_DEDUP_LOGS=0
RAY_ENABLE_RECORD_ACTOR_TASK_LOGGING=1
RAY_task_retry_delay_ms=3000
RAY_ENABLE_UV_RUN_RUNTIME_ENV=0

# RAY_memory_monitor_refresh_ms=0

# ── Logging (DEBUG on dev, INFO on prod) ──
LOG_LEVEL=DEBUG

3. File Parser configuration

All supported file format parsers are pre-configured. For PDF processing, PyMuPDFLoader is the default parser — a lightweight, fast, CPU-friendly engine well suited to searchable PDFs and quick local testing.

Important

PyMuPDFLoader cannot process non-searchable (image-based / scanned) PDFs and does not run OCR or extract embedded images.

Note

For OCR-scanned documents, complex layouts, tables, or embedded images, switch to MarkerLoader (heavier; runs on GPU and CPU) by setting the PDFLOADER variable: PDFLOADER=MarkerLoader. Other options: DoclingLoader.

4. Run OpenRAG

Admin UI

The admin UI — a web interface for document ingestion, indexing, and management — ships bundled as the admin-ui service, so there is no separate setup. Once the stack is up it is served at http://localhost:ADMIN_UI_PORT/app/ (default port 8081).

All deployment assets now live under infra/:

• Directoryinfra/

  • Directorycompose/ full stack (recommended)

    • docker-compose.yaml
    • .env.example
    • .env your configured env
  • Directorydocker/ Dockerfiles

    • …
  • Directoryansible/ remote deployment

    • …
• Directoryopenrag/ application code

  • …
• Directoryconf/ YAML configuration

  • …

Run the stack from infra/compose/, where the .env you just created lives. Use the GPU tab on a machine with an NVIDIA GPU, or the CPU tab otherwise.

GPU

cd infra/compose
docker compose up -d

# stop it later with:
# docker compose down

x86 CPU

cd infra/compose
docker compose --profile cpu up -d

# stop it later with:
# docker compose --profile cpu down

Development builds

Add --build to rebuild the images from your working tree: docker compose up --build -d.

Inference services

The LLM and VLM are always external OpenAI-compatible endpoints — set BASE_URL/MODEL/API_KEY (and the VLM_* equivalents) in .env.

The embedder and reranker, by contrast, are bundled: docker compose up -d starts a vllm container serving EMBEDDER_MODEL_NAME and a reranker container (RERANKER_PROVIDER, default Infinity). No extra step is needed to use them.

cd infra/compose
docker compose up -d                   # GPU — core stack + bundled embedder & reranker
# docker compose --profile cpu up -d   # CPU-only host

To use external embedding/reranking instead — reusing existing inference servers and keeping the footprint small:

• Embedder — set EMBEDDER_BASE_URL (+ EMBEDDER_API_KEY) to your endpoint, then stop the local server by commenting out the vllm-gpu / vllm-cpu service in infra/compose/docker-compose.yaml.
• Reranker — set RERANKER_ENABLED=false to skip reranking, or point it at an endpoint with RERANKER_PROVIDER=openai and RERANKER_BASE_URL (+ RERANKER_API_KEY); to also stop the container, comment out the extern/reranker/… line in the include: block.

Once the app is up and running, you can access the provided services — see Default ports below.

Ansible

Clone the repository, then run the deployment script from infra/ansible/ and follow the interactive prompts:

git clone --recurse-submodules https://github.com/linagora/openrag.git
cd openrag/infra/ansible
./deploy.sh

Default ports

Once the stack is up, OpenRAG exposes the following services by default:

Service	Port	Description
API Documentation	8080/docs	Main FastAPI for document ingestion and querying. See this
Chainlit UI	8080/chainlit	User interface for interacting with the RAG system
Ray Dashboard	8265	Ray dashboard for monitoring and managing tasks
Admin UI	8081/app/	Main user interface for indexing and viewing indexed documents

Ray Serve mode

The table above is for the default (uvicorn) deployment, where the Chainlit UI is the /chainlit subroute of the API. When ENABLE_RAY_SERVE=true, the API is served by Ray Serve on RAY_SERVE_PORT and the Chainlit UI runs on its own port, CHAINLIT_PORT (default 8090), instead of the subroute. Uncomment the ${CHAINLIT_PORT}:${CHAINLIT_PORT} mapping in infra/compose/docker-compose.yaml to expose it. See CHAINLIT_PORT.

More information about each service is available in its respective documentation page.