Skip to main content
The Assistant tab is an in-product agent that wires up taps and pipelines for you. You describe what data you want; the agent finds a source, generates the tap script, creates the pipeline, links them, and shows you the result — all in one chat. It uses the same MCP tools that external agents (Claude Desktop, Cursor) see when they connect to your Datris MCP server. There is one canonical workflow, shared between in-product and external agents.

What you can ask

The Assistant is open-ended. Some examples:
  • “Find a source of US public-company SEC filings and build a tap and pipeline.”
  • “Pull current weather data from a public API and create a pipeline for it.”
  • “Ingest the most recent NYC taxi trip data into a pipeline.”
  • “List the taps that are currently scheduled and tell me which haven’t run in over a week.”
  • “Show me the rows in the weather pipeline.” (works for MongoDB, PostgreSQL, Snowflake, Databricks, SQL Server, Object Store (Parquet / ORC), or vector destinations — the agent picks the right query tool for the destination)
The agent will reach for tools as it needs them: web search to find a source, list_taps / list_pipelines to check for existing work, create_tap_secret when credentials are needed, create_tap, test_tap, propose_pipeline_schema, create_pipeline, and so on.

Preview and confirm the schema

When the Assistant builds a tap and a pipeline together, it builds the tap first and takes the pipeline’s column types from rows the tap actually returned:
  1. Creates and tests the tap. The test pulls a short preview (20 rows by default) and lands nothing.
  2. Shows the proposed schema. The proposal uses the rows that test just returned, so the tap is not run a second time (asking for more rows runs it again). Each column gets a recommended type with its evidence: sample values, plus a reason when it stays text. Recommendations are cautious. A column with fewer than 5 values, only empty values, or a value that doesn’t fit a type (such as N/A in a number column) stays text and says why.
  3. Asks you to confirm. Accept the types, change any of them (“make that column an integer”), or ask for more rows to get more evidence.
  4. Asks one follow-up question. It offers data-quality checks, transformations, and field protection for the confirmed columns. “None” is a fine answer. If you pick data-quality checks or transformations, the Assistant proposes a short list of them for you to review before it builds anything.
  5. Creates the pipeline with the confirmed types and attaches the tap. On Postgres, Snowflake, Databricks, SQL Server, and object store destinations, the first run lands typed columns, not text.
If the proposal fails (for example, the test returned no rows), the Assistant tells you what failed and what to do next, and does not ask about column types. That is at most two questions in total. The destination, key columns, and schedule are still agreed in the plan the Assistant states up front. MongoDB and vector-store destinations have no column types, so the card shows sample rows only and needs no click; the Assistant carries on in the same reply. MongoDB still asks the follow-up question; vector stores skip it, because the pipeline chunks and embeds documents as they load, so there are no checks, transformations or protection to choose. In the Assistant tab the proposal appears as a card in the chat, not as a table in the reply. The card shows the first few preview rows, then one line per column with the recommended type in a dropdown and sample values. When a column stayed text, or had too few values to type, a short note under its dropdown says why. Change any type in its dropdown, then click Confirm types, or click Keep all as text. Pull more rows runs the tap test again with five times as many rows (up to 500), shows the rows it fetched in the preview, and updates the samples and recommendations on the card without a new reply from the Assistant. Any type you already changed stays as you set it. When the tap has no more rows to give, or the 500-row limit is reached, the card says so next to the button. Your choice goes back to the Assistant as your next message, and the card shrinks to a one-line summary. Other MCP clients, such as Claude Desktop, work as before: the agent shows the proposal as a table and you answer in chat. When you choose data-quality checks or transformations at the follow-up question, the Assistant shows what it proposes as a second card: one row per check (or transformation), each with a checkbox, an editable text box, and the column it covers. Untick a row to leave it out, edit its wording, or click Add rule to add your own. Click Clear all to untick every row at once (it becomes Select all when nothing is ticked); the rows stay for you to re-tick or edit. Click Confirm to build with exactly the ticked rows, or Skip to build without them. The data-quality card also asks what a failing row does: Reject fails the run (the default), Warn only logs the failing rows and keeps loading. If you chose both, the data-quality card comes first and the transformations card follows once you answer it. Nothing you did not confirm is added, and field protection is still handled as before. You can also ignore the card and reply in your own words. Other MCP clients list the proposed checks in the chat and ask you to confirm them, and whether a failing row should warn only or reject the run, before the pipeline is created. Pipelines built from a dropped file or from rows you paste into the chat still use a small sample, and their columns are stored as text. To type those columns after the first load, click the text badge next to the pipeline’s destination on the Pipelines tab.

Drop a file to build a pipeline

You can drag a file straight into the chat — or attach one with the file picker — and the Assistant will build a pipeline around it for you. Drop a CSV, JSON, XML, or document file onto the chat window (or click the attach control and pick it), then describe what you want done with it. When you send a message with a file attached, the agent:
  1. Reads a sample of the file — a text extract from the attached content — so it can see the columns and data shape.
  2. Proposes a destination, defaulting sensibly for the data and naming the alternatives (Postgres, MongoDB, an object store, Snowflake, Databricks, SQL Server, or a vector store) so you can pick a different one.
  3. Confirms before building — it tells you what it’s about to create and waits for your go-ahead.
  4. Asks about an upsert key when relevant, so repeated loads update rows in place instead of duplicating them.
  5. Creates the pipeline, loads the data, and reports the result — including how many rows landed in the destination.
This is the fastest path from “here’s a file” to a working pipeline: no wizard, no manual schema entry — just drop it in and confirm.

What you’ll see in real time

Three live layers in every assistant turn:
  1. 💭 Thinking — the model’s chain-of-thought. On Anthropic it streams character-by-character; other providers deliver it at the end of each turn (see the provider table below). Anthropic and Bedrock show the full reasoning; OpenAI shows a high-level summary instead. Click the block to expand.
  2. Tool cards — inline within the response. Each card shows a human-friendly label (”🔍 Searching the web for SEC EDGAR API…”, ”🛠 Creating tap sec-edgar…”) with a spinner while running, then a ✓ or ✕. Click to expand the raw input and result.
  3. Streaming prose — the response text appears as the model writes it on Anthropic, and at the end of each turn on other providers, with tool calls inlined where the model invoked them.
When the agent needs answers from you (which source, how often to run, where the data should land, which locations to cover), it asks on a question card instead of a numbered list. The card has one tab per question. A question with choices shows up to eight of them, each with a one-line description, plus an Other box for your own answer; an open-ended question shows a text box. When the agent asks where the data should land, every destination available on your install is its own choice. Picking a single choice moves to the next tab. On a pick-any question, an option that means none or no change cannot be combined with the others: ticking it clears them, and ticking another clears it. Click Submit on the last tab; the answers are posted as your next message and the agent carries on. You can also ignore the card and type a reply, or click Answer in chat instead. When the agent finishes, Open tap and Open pipeline links appear at the bottom of the turn so you can jump straight to anything it created.

Safety

The agent has access to every MCP tool, including destructive ones (delete_tap, delete_pipeline, delete_tap_secret, update_secret). It is instructed to never invoke a destructive tool without asking you first — if you say “delete the broken tap,” the agent will name what it’s about to delete and wait for your confirmation in the chat before proceeding. The Stop button at the bottom of the chat aborts the loop server-side at the next checkpoint, so you can interrupt anytime. Stop halts the in-flight model call within a fraction of a second — cancelled responses cost only what was already streamed.

Platform-tab secrets are visible but read-only

When a pipeline destination needs operator-owned credentials (AWS S3, an external Postgres, a vector-store endpoint, etc.), the agent can discover secrets you’ve created on Configuration → Secrets → Platform and verify their field shape — but it cannot create, modify, or delete them. If nothing suitable exists, the agent will name the field shape the destination needs and ask you to add the secret in the Secrets tab, then give it the name to use. Secret values are never exposed to the agent.

Long-running runs are carried to completion

When the agent runs a tap (run_tap or upload_data), it owns that work all the way to a terminal success / warning / error outcome. It will not kick off the job and tell you to check back later — that defeats the point of delegating. The polling pattern it uses:
  1. Poll get_pipeline_status immediately after the run is submitted (most small structured taps finish in 1–5 seconds; no point sleeping first).
  2. If still in progress, sleep with wait_seconds using exponential backoff (5s → 10s → 20s → 30s → 60s → 60s …, capped at 60s normally, 120s only if things are genuinely glacial).
  3. Reset to a short wait (5–15s) the moment a poll shows new jobs flipping to a terminal state — progress means it’s about to finish.
  4. Emit a one-line progress update each cycle (“12 of 28 jobs done, 16 still processing — checking again in 30s”) so you can see the run advance.
  5. After ~20 polls, if the run is still in progress, pause and ask whether to keep waiting, kill the run, or check later — never silently stop.
The same pattern applies to upload_data → get_job_status. If the chat seems to be “hanging” mid-run, it’s waiting between polls on purpose.

Reasoning visibility by provider

The Assistant names the chat provider and model it is using (for example Anthropic · claude-fable-5-1 · extended thinking on) on the empty conversation and in the top bar once you start chatting, with a link to Configuration → AI Providers where you change it. The line refreshes when you return to the Assistant tab or start a new chat, so it follows a change you just saved there. Set ai.extendedThinking: "false" in application.yaml to disable the reasoning block platform-wide. Default is on. Extended thinking adds roughly $0.05/turn and 2–8s of latency on Opus — cheap insurance for transparency on an agent that’s writing production-bound config.

Configuration

The Assistant uses your tenant’s AI primary config (the chat slot). Tap script generation, AI data quality, and AI transformations use the separate codegen slot. On a fresh install the Anthropic default is Claude Opus 5.5 (claude-opus-5-5) for both slots; the OpenAI default is gpt-5.6-sol. There is no per-tenant Assistant configuration to manage beyond what you already have for AI primary. See AI Configuration for the underlying AI primary / codegen / web search / embedding settings.

Authentication

The Assistant inherits the identity of the user running it. How that maps to permissions depends on which auth mode the platform is running in:
  • USE_USER_AUTH=true (recommended): the Assistant runs as your logged-in user. Audit log entries and capability checks both show session:<your username>. Permissions follow your role — admin gets full access, editor gets create/update/run on data resources, viewer gets read-only. See User Authentication for the role-to- capability mapping.
  • USE_USER_AUTH=false + USE_API_KEYS=true: the Assistant uses whatever value you pasted into the Connect prompt (typically the seeded ui key). Same identity for everyone using the browser.
  • USE_API_KEYS=false: no auth on tool calls; the Assistant runs as the platform’s anonymous identity.
The Assistant doesn’t have its own dedicated API key — its server-side MCP-bound tool calls use the same identity the chat endpoint was authenticated under. Want to give the Assistant narrower permissions than the human user has? That’s not exposed today; the workaround is to log in as a different user (an editor or viewer) when running the Assistant, so the role mapping applies. See API Keys for the full capability model.

Limits

  • 50 iterations per turn. The agent loop is hard-capped to bound cost. If the agent runs out of iterations, it stops and asks you to send a follow-up message to continue.
  • No server-side conversation history. The transcript lives in the browser tab’s session storage — it survives a refresh but is gone once the tab is closed. Durable persistence is on the roadmap.
  • Earlier tool calls travel with each message. Every new message sends the tool calls from earlier turns in the same chat (name, input and a shortened result), so the Assistant remembers what it already listed, created and tested instead of starting over. Long results are cut to about 1 KB each (4 KB for pipeline and tap listings) and 8 KB per turn, and all earlier turns together share a 32 KB budget, newest first; older turns keep only the names of the tools called. Passwords, tokens, keys and other credential fields in tool inputs are replaced with [redacted] before they are re-sent. The Assistant calls a tool again when it needs the full output.
  • 32K output tokens per LLM call. Sufficient for thinking + reasoning + tool selection on every public model we support.

How it relates to other tabs

  • Chat panels — four panels, one job each: the Assistant (build taps and pipelines), Ops chat (runs, failures, incidents — see Incidents), Catalog chat (curation — see Data Catalog) and the Configuration assistant (settings — see Configuration Assistant).
  • MCP tab — static catalog of the tools the Assistant uses. Read this if you want to understand what each tool does.
  • Agent Monitor (pop-out) — read-only activity monitor of external MCP clients (Claude Desktop, Cursor, scripts) calling your server. Opens in its own browser window via the pop-out icon (open-in-new) in the Agent Monitor header under MCP → Activity, so you can keep an eye on Connections and the Activity Log while you work elsewhere. The Assistant is the in-product counterpart — same tools, different surface.
  • Catalog tab — anything the Assistant creates (taps and pipelines) lands here, grouped by catalog. The ”→ Open tap” / ”→ Open pipeline” links at the end of each conversation route you there.