Installation

Prerequisites

Docker

License. Datris is open source under AGPL-3.0. Running an unmodified copy inside your own organization needs nothing more; a commercial license is available for hosted, embedded, and supported deployments — see Licensing & Support.Telemetry. The platform sends no usage data, crash reports, or license checks anywhere. The one outbound request it can make is an on-demand model-catalog refresh from the Configuration tab, which falls back to a built-in list when it fails — see Security Architecture for how to verify.

Quick Start

All you need is Docker — no git checkout, no build tools. The installer pulls the pre-built images from Docker Hub, fetches the few runtime files Compose needs into a ./datris directory, seeds a .env, and starts the full stack:

curl -fsSL https://get.datris.ai/install.sh | sh

The installer walks you through a short setup — press Enter at every prompt for a standard install:

AI keys. Datris can use Anthropic Claude, OpenAI, or Grok (xAI) — the installer prompts for each key in turn; enter any you have (one is enough, each prompt is one Enter to skip). If you prefer Claude through Amazon Bedrock or OpenAI through Azure OpenAI, skip the key prompts and the installer offers those routes next — Azure asks for your key, resource endpoint, and chat deployment name; Bedrock asks for AWS credentials (or Enter to use the host’s IAM role) and a region.
Databases and stores. Pick what to run — each can be bundled (a local container), external (point at a service you already have), or skipped:
- Postgres — structured destination + pgvector (default: bundled)
- Semantic search embeddings — OpenAI text-embedding-3-small (recommended: no 2.2 GB model download, no resident container) or the bundled local TEI server for data that can’t leave the machine
- Vector stores — pgvector by default (included with Postgres, no extra container); optionally add qdrant, weaviate, or chroma
- Kafka — local test broker or external (default: none)
- Snowflake / Databricks — optionally store destination credentials now so pipelines can use them on day one

It prints an install summary, starts the stack, and then checks every store you configured — a mistyped external hostname or credential shows up right there by name, not later mid-pipeline. On first start, vault-init seeds your keys and store credentials into Vault. Changing your mind later is one line in ./datris/.env — for example POSTGRES_ENABLED=0, COMPOSE_PROFILES=qdrant,kafka, or EMBEDDING_PROVIDER=openai — followed by docker compose up -d. Disabling a store removes its container but keeps its data volume, so re-enabling restores the data. The .env.example comments document every knob.

The install.sh installer is a POSIX shell script, so it runs on macOS and Linux. On Windows, run it from a POSIX shell — WSL2 (recommended) or Git Bash — or skip the installer and use the single-file Compose option below, which works natively in PowerShell.

Or, for a fully self-contained single file (no installer) — the init scripts and config are inlined, so nothing else is needed (requires Docker Compose ≥ 2.23). This is the simplest path on Windows:

curl -O https://get.datris.ai/docker-compose.standalone.yml
ANTHROPIC_API_KEY=sk-ant-... docker compose -f docker-compose.standalone.yml up -d

curl.exe -O https://get.datris.ai/docker-compose.standalone.yml
$env:ANTHROPIC_API_KEY="sk-ant-..."
docker compose -f docker-compose.standalone.yml up -d

In PowerShell, use curl.exe (the bundled curl alias maps to Invoke-WebRequest and takes different flags), and set the key with $env: since the inline KEY=value command syntax is bash-only.

Verify

curl http://localhost:8080/api/v1/version

Alternative: install from a git clone

If you’d rather work from a checked-out repository — for example to track the source, customize docker-compose.yml, or contribute — clone the repo and start from the checked-out compose file:

git clone https://github.com/datris/datris-platform-oss.git
cd datris-platform-oss
cp .env.example .env       # add ANTHROPIC_API_KEY and/or OPENAI_API_KEY (or the AZURE_OPENAI_* trio, XAI_API_KEY, or AI_PROVIDER=bedrock)
docker compose up -d

This still pulls the pre-built images from Docker Hub — no build tools required. To build the images yourself, see Building from Source below.

Upgrading

Upgrading pulls the latest images and refreshes the Compose file, and your data survives in Docker volumes. The procedure for each install method, the pre-upgrade self-check, and what to do about stale secrets live on Upgrades & Supported Versions.

Reset everything (destroys all data)

If you don’t care about anything on this machine and want a completely fresh install — same data layout as a brand-new clone — wipe all volumes:

cd datris            # your install dir: ./datris (installer), the standalone file's dir, or your clone
docker compose down -v
docker compose up -d

Warning: docker compose down -v is destructive. The -v flag removes every Docker volume the project owns (both named volumes and the anonymous volumes the data services create automatically). You will lose:

All your pipelines, runs, taps, and metadata (MongoDB internal oss database) and all data landed in Postgres (datris database)
All Vault secrets — API keys, database credentials, AI configuration (will be re-seeded from your .env on next start, but only the defaults — any UI-edited overrides are gone)
All MinIO object storage — raw uploads, configs, temp files, processed outputs
MongoDB destination data
Queued messages and offsets in Kafka, Zookeeper, ActiveMQ
The bundled bge-m3 embedding model in the tei-data volume (will re-download ~2.2 GB on first start)
Cached pip wheels in the pip-cache volume (taps that need extras like yfinance will re-download on first run)

Only run down -v if you are certain none of the above matters, e.g. on a brand-new dev machine, after exporting anything you needed, or on a CI runner. Never run down -v on a production or shared instance. After docker compose up -d, vault-init.sh re-seeds the AI configuration secrets from your .env, MinIO buckets are recreated, Postgres starts empty, and the bundled embedding service re-downloads bge-m3 (a few minutes one-time).

Note for production deployments: the deploy/docker-compose.prod.yml file used by dedicated production installs uses bind mounts to host directories under /data/* instead of Docker volumes, so docker compose -f docker-compose.prod.yml down -v does not wipe the data — the host directories survive. To reset a prod install, you’d need to also delete the relevant /data/* directories on the host, which is a much riskier operation and not recommended outside of disaster recovery.

Volumes

All stateful services use named Docker volumes, created automatically by docker compose up — no user action required:

Volume	Mounted at	Purpose
`postgres-data`	postgres `/var/lib/postgresql/data`	Pipeline data landed in Postgres + pgvector collections
`mongodb-data` / `mongodb-configdb`	mongodb `/data/db`, `/data/configdb`	All pipeline, tap, and platform configuration
`minio-data`	minio `/data`	Object storage — uploads, configs, pipeline output
`vault-data`	vault `/vault/file`	Secrets (persist across restarts)
`tei-data`	tei `/data`	The bundled embedding model (~2.2 GB)
`pip-cache`	datris `/root/.cache/pip`	Caches downloaded pip wheel files so tap-installed packages don’t have to be re-downloaded

Because these are named volumes, they re-attach whenever a container is recreated or a store is disabled (POSTGRES_ENABLED=0) and later re-enabled — disable → re-enable is lossless. If you upgraded from a version where the data services used anonymous volumes, the first up -d after this change starts the stores on fresh named volumes; the installer-era data lives on in the old anonymous volumes (visible via docker volume ls -f dangling=true) and can be copied across with docker run --rm -v <old>:/src:ro -v <new>:/dst alpine cp -a /src/. /dst/. Commonly used packages (requests, beautifulsoup4, pandas, lxml, feedparser, boto3, google-cloud-storage, azure-storage-blob, openpyxl, pyyaml, python-dateutil, pytz) are baked into the image. When a tap needs something extra (e.g. yfinance), pip downloads it on first run (~30 seconds) and caches the wheel in pip-cache. Subsequent container restarts re-run pip install for those extras, but the install is near-instant because the wheel is already cached locally.

Services

Service	Port	Purpose
Pipeline Server	8080	REST API and data processing
Pipeline UI	4200	Web dashboard
MCP Server	3000	AI agent integration (MCP protocol)
MinIO	9000 (API), 9001 (Console)	Object storage
MongoDB	27017	Configuration and status store
ActiveMQ	61616 (broker), 8161 (console)	Message queue and notifications
Vault	8200	Secrets management
PostgreSQL	5432	Database destination + pgvector — optional (`POSTGRES_ENABLED=0` to disable, or point at an external Postgres)
TEI	11434	Bundled embedding server — optional (`TEI_ENABLED=0` to disable; recommended alternative: OpenAI embeddings)
Qdrant	6333/6334	Vector store — opt-in (`COMPOSE_PROFILES=qdrant`)
Weaviate	8079, 50051	Vector store — opt-in (`COMPOSE_PROFILES=weaviate`)
Chroma	8000	Vector store — opt-in (`COMPOSE_PROFILES=chroma`)
Kafka	9092	Streaming — opt-in (`COMPOSE_PROFILES=kafka`; includes Zookeeper on 2181 and Kafka UI on 8085)

Optional vs opt-in. Services that ship enabled (Postgres, TEI) are disabled with an explicit *_ENABLED=0 in .env — an .env without these lines keeps them running, so upgrades never silently drop a service you were using. Brand-new services (vector stores, Kafka) activate via COMPOSE_PROFILES. Datris bundles kafka-clients, so pipelines that talk to an external Kafka don’t need the local broker at all — set KAFKA_BOOTSTRAP_SERVERS in .env instead. One teardown nuance: docker compose down ignores inactive profiles, so for a full teardown use docker compose --profile "*" down.

Web UIs

UI	URL	Credentials
Datris Platform UI	http://localhost:4200	none
Datris Platform API	http://localhost:8080	none
MCP Server (SSE)	http://localhost:3000/sse	none
MinIO Console	http://localhost:9001	`minioadmin` / `minioadmin`
ActiveMQ Console	http://localhost:8161	`admin` / `admin`
Kafka UI (opt-in)	http://localhost:8085	none — only with `COMPOSE_PROFILES=kafka`
Vault UI	http://localhost:8200	Token: run `docker compose exec -T datris cat /vault-token/token` (random per install)

API Keys and AI Providers

Datris supports six AI providers. Set your keys in .env:

Provider	Environment Variable	Used For
Anthropic Claude	`ANTHROPIC_API_KEY`	AI data quality, transformations, error explanation, schema generation, profiling
OpenAI	`OPENAI_API_KEY`	Same as above, plus embeddings for vector database / RAG
Azure OpenAI	`AZURE_OPENAI_API_KEY` + `AZURE_OPENAI_ENDPOINT` + `AZURE_OPENAI_MODEL`	Same as OpenAI, served from your Azure resource (models addressed by deployment name)
Amazon Bedrock	`AI_PROVIDER=bedrock` + `AWS_REGION` (AWS keys optional — IAM role works)	Claude served through your AWS account with IAM auth and AWS billing
Grok (xAI)	`XAI_API_KEY`	Chat and CodeGen on Grok via xAI’s OpenAI-compatible API (no embeddings)
Ollama (local)	`OLLAMA_MODEL` (requires uncommenting the `ollama` service in `docker-compose.yml`)	Same as above — no API key needed, runs locally

At least one AI provider key is required for AI features. The embedding provider for RAG is configured via Vault secrets — see AI Configuration for details.

Infrastructure Details

MinIO

The minio-init container automatically creates the required buckets:

{env}-raw - File upload staging
{env}-temp - Temporary processing files
{env}-data - Pipeline output (object store destination)
{env}-config - Configuration files (validation schemas)

Where {env} is the environment name (default: oss). See Configuration Reference for the environment setting.

Vault

On first boot, the vault-init container seeds Vault with default secrets for the bundled services (MinIO, ActiveMQ, MongoDB, PostgreSQL) plus your AI provider API keys and any external-store credentials (Kafka, vector stores, Snowflake, Databricks) from .env. Vault uses durable file storage on the vault-data volume, so secrets — including any you add later in the Configuration tab or via taps — persist across restarts and rebuilds. .env is the first-boot seed only; after that the Configuration tab is the source of truth. See How Configuration Persists for details and the clean-reset path.

Vector Databases

pgvector is included whenever Postgres is enabled (bundled or external). The others are one line in .env:

Qdrant — high-performance vector database: COMPOSE_PROFILES=qdrant
Weaviate — open-source vector database: COMPOSE_PROFILES=weaviate
Chroma — lightweight, single container: COMPOSE_PROFILES=chroma
Milvus — scalable vector database; runs external-only (it needs its own etcd/minio stack — see the Milvus standalone install), then set MILVUS_HOST/MILVUS_PORT in .env

Combine profiles with commas: COMPOSE_PROFILES=qdrant,kafka. To use a managed/cloud vector store instead of a bundled one, skip the profile and set the store’s *_HOST / *_PORT / *_API_KEY variables in .env (first boot seeds the credentials into Vault; afterwards manage them in Configuration → Secrets).

External databases and stores

Every optional store can point at infrastructure you already run instead of a bundled container — the installer asks, or set the variables in .env before first boot: POSTGRES_JDBC_URL/POSTGRES_USER/POSTGRES_PASSWORD (with POSTGRES_ENABLED=0), KAFKA_BOOTSTRAP_SERVERS, vector-store *_HOST variables, and SNOWFLAKE_* / DATABRICKS_* destination credentials. These are first-boot seeds: vault-init writes them into Vault create-if-absent, and from then on Configuration → Secrets is the source of truth. The platform’s availability probes treat external stores exactly like bundled ones — the pipeline wizard and the Assistant offer whatever is actually reachable.

Configuration

The pipeline server reads configuration from application.yaml, mounted from docker/config/application.yaml. See Configuration Reference for the full list of properties.

JVM Heap Sizing

The datris service runs a Spring Boot JVM. Its heap is governed by the JAVA_OPTS environment variable, passed in via docker-compose.yml. The default is sized to fit comfortably on an 8 GB host alongside the bundled TEI embedder, Postgres, MongoDB, MinIO, ActiveMQ, Vault, the UI, and the MCP server:

JAVA_OPTS=-Xms512m -Xmx2g   # default

Override in your .env file on larger hosts. Suggested sizings:

Host RAM	Suggested `-Xmx`
16 GB	`-Xmx4g`
24 GB	`-Xmx8g`
32 GB+	`-Xmx12g`

Example for a 24 GB host:

# .env
JAVA_OPTS=-Xms1g -Xmx8g

Then restart the service:

docker compose up -d datris

Docker Desktop note (macOS / Windows): Docker Desktop runs containers inside a Linux VM with its own RAM allocation, which may be much lower than your host’s total RAM. Check the VM ceiling with:

docker info --format '{{.MemTotal}}' | awk '{printf "%.1f GB\n", $1/1024/1024/1024}'

If the VM is capped below what you need, raise it in Docker Desktop → Settings → Resources → Memory. The JVM -Xmx must be smaller than the Docker VM’s allocation, otherwise the kernel inside the VM will OOM-kill containers under load.

Building from Source

For development or contributing:

Prerequisites

Java 17+
SBT

Build and run

# Build the server JAR
sbt clean assembly

# Start with local builds (edit docker-compose.yml to uncomment build: lines)
docker compose up --build

In docker-compose.yml, uncomment the build: lines and comment out the image: lines for the services you want to build locally:

datris:
  # image: datrisai/datris-server:latest
  build: .  # Build from source

Getting Started

Connect your agent

Assistant

Catalog

Taps

Ingestion

Destinations

Data Quality

Transformation

AI Features

Configuration

Governance & Security

Production

Integrations

Examples

Prerequisites

Quick Start

Verify

Alternative: install from a git clone

Upgrading

Reset everything (destroys all data)

Volumes

Services

Web UIs

API Keys and AI Providers

Infrastructure Details

MinIO

Vault

Vector Databases

External databases and stores

Configuration

JVM Heap Sizing

Building from Source

Prerequisites

Build and run

​Prerequisites

​Quick Start

​Verify

​Alternative: install from a git clone

​Upgrading

​Reset everything (destroys all data)

​Volumes

​Services

​Web UIs

​API Keys and AI Providers

​Infrastructure Details

​MinIO

​Vault

​Vector Databases

​External databases and stores

​Configuration

​JVM Heap Sizing

​Building from Source

​Prerequisites

​Build and run

Prerequisites

Quick Start

Verify

Alternative: install from a git clone

Upgrading

Reset everything (destroys all data)

Volumes

Services

Web UIs

API Keys and AI Providers

Infrastructure Details

MinIO

Vault

Vector Databases

External databases and stores

Configuration

JVM Heap Sizing

Building from Source

Prerequisites

Build and run