Beta. The Taps API is stable but may add fields or refine response shapes in upcoming releases. Existing endpoints will remain backward-compatible where possible.
List Taps
Get Tap
Response (success):
scriptMissing only appears when scriptPath is set but the object is absent — typically after an interrupted edit or re-generation wiped the old script before the new one landed. Distinguish from “never generated” (scriptPath null, script null, no scriptMissing key). The Edit Tap UI uses this flag to render a recoverable-state banner instead of the generic “Generate a script first” validation.
Create or Update Tap
Document-tap compatibility rule. When
tapType == "document" and targetPipeline is non-empty, the server validates that the pipeline has:
source.fileAttributes.unstructuredAttributes != null- A vector-store destination (one of
qdrant,pgvector,weaviate,milvus,chroma)
TapRunner.feedDocumentPipeline in case the pipeline is reshaped after the tap is saved.
Test-before-cron rule (HTTP 409). A tap may only carry a cronExpression once the exact script (or, for HTTP taps, the exact endpointUrl) that will run on that schedule has passed a test run (POST /api/v1/tap/run with mode: "test"). The save is refused with HTTP 409 and nothing is persisted (no new version is minted) when either:
- the
cronExpressionis being set or changed and the stored tap has no successful test for the script being saved, or - the
cronExpressionis unchanged but the script (newscriptPath/scriptCommitSha/endpointUrl) differs from the one that last passed a test.
cronExpression; run it with mode: "test" until it succeeds (the server stamps the tested script on the tap as lastTestRunScriptId); then save again with the cronExpression. Clearing a cron, and cadence-only or metadata edits (description, tags, targetPipeline, enabled) on a tap whose current script has passed a test, are always accepted. Test state is read from the stored tap only — lastTestRun* fields in the request body are ignored. Pre-existing scheduled taps keep running and stay editable; the first cron or script change on such a tap is accepted if the tap has ever tested or run successfully, otherwise it needs a test run first. The same rule applies to POST /api/v1/tap/version/restore. POST /api/v1/tap/script applies the rule up front: storing different bytes for a tap that currently has a cronExpression is refused with the same HTTP 409 body before anything is written to the object store or code repository (so the repository never drifts ahead of the tap’s pinned commit). Clear the cron first (save with cronExpression: null), then store, test, and set the cron. Storing identical bytes, or storing for an unscheduled tap, is always accepted. The AI script endpoints (/tap/generate, /tap/fix, /tap/review, /tap/optimize) still return the new script and its scriptPath with HTTP 200, but do not move a scheduled tap onto the new bytes; when that happens the body also carries the gate message under error.
Delete Tap
Brainstorm (AI Chat)
- Asks one focused clarifying question at a time
- Suggests specific data sources (e.g., yfinance, Alpha Vantage, Open-Meteo)
- Recognizes Datris platform tables and uses the metadata/query endpoints
- Returns an updated instruction draft on every turn so the UI can keep the instruction box in sync
- Returns suggestedEnvVars when an external API requires authentication
Response:
DATRIS_POSTGRES_DATABASE, DATRIS_MONGODB_DATABASE, DATRIS_PLATFORM_HOST, or DATRIS_PLATFORM_PORT in suggestedEnvVars — those are always injected by the platform.
Generate Script (AI)
fetch() script from a plain-English description. The system prompt branches on tapType — document taps get instructions to return {uri, filename, content} dicts and a different set of rules (no chunking, no embedding, no local-filesystem fallback).
Body:
Response:
injectedPrompts lists the names of the tap prompt fragments that were injected into the generation prompt (empty when none matched).
Fix Script (AI Diagnosis)
Test Tap
Body: A TapConfig JSON object (same as Create).
Structured response (
dataType: "csv" shown):
dataType: "document"):
aiExplanation contains an AI-generated diagnosis. durationMs is the end-to-end test wall time in milliseconds — this is what the Optimize Script endpoint uses as the baseline timing for its perf rewrite.
For dataType: "csv", column names in columns and the keys inside each record in records are normalized by the platform: lowercase, [a-z0-9_] only, with % rewritten to percent. Source data with names like EPS Estimate or Surprise(%) will appear as eps_estimate and surprise_percent. JSON/XML results destined for MongoDB are not normalized. See Schema Definition → Column Naming Rules. Document-tap records pass through unchanged — the platform never rewrites URI, filename, content, or metadata keys.
Review Script (Post-Run)
time.sleep, switch to the recommended endpoint, add pagination, update parsing for renamed fields) and persisted to MinIO as a new version. When no signal is found, the script is returned unchanged.
The UI invokes this automatically before Optimize Script — if the reviewer rewrites the script, the optimizer is skipped on that pass (correctness from output outranks speed). Callers integrating directly should follow the same order: call /tap/review first, re-test on rewritten=true, and only call /tap/optimize when rewritten=false.
Body:
Response:
rewritten is false, script / scriptPath match the input and changes is empty — the reviewer found nothing in the output worth acting on. Prompt fragments configured via Tap Prompt Fragments are auto-injected into the reviewer’s system prompt when their key or any alias appears in the script.
Optimize Script
ThreadPoolExecutor, reuse a requests.Session(), or drop unnecessary time.sleep() — while preserving correctness (fetch() signature, DATRIS_TAP_TEST_LIMIT handling, Vault env-var reads, retry/backoff behavior, and raise_for_status calls).
The UI invokes this automatically after a green test in the Create Tap wizard, with a regression guard that auto-reverts if the rewrite runs ≥20% slower. Callers integrating this endpoint directly should implement the same re-test + revert pattern.
Body:
Response:
changes is an empty array and script / scriptPath match the input. In that case no re-test is needed.
Run Tap
Response:
columns lists the column names the script produced (normalized for dataType: "csv", see Test Tap); it is null when the run produced no records.
mode=run does not return the records themselves — the data is in transit to targetPipeline, and the agent / caller should verify completion via get_pipeline_status, not read from the response body. recordCount summarizes how many records were submitted. To preview what a script produces, call the same endpoint with mode=test, which pulls only the first 20 records by default (raise or remove the cap with testLimit) and returns up to 20 sample rows in a records array (with recordsTruncated: true set when the script produced more than that). Because of the cap, a test’s recordCount is the preview size, not what a mode=run call will produce.
persisted: true means the records were submitted to targetPipeline. When persisted: false, a persistedReason field names the cause — one of test_mode, run_error, no_records, no_target_pipeline, or debounced. On persisted runs, publisherToken groups every ingestion job this run submitted; pipelineTokens lists each. Document taps fan out to many pipelineTokens but share one publisherToken.
Run debounce (mode=run only)
/tap/run debounces mode=run requests per tap on a 5-second window to suppress accidental duplicates — agents that emit parallel tool_use blocks, UI buttons that get double-clicked, or transport-level retries. If a second run hits within the window, the response is HTTP 200 with status: "skipped", persisted: false, persistedReason: "debounced", and an error string explaining how long ago the previous run started. The previous run keeps executing — pivot via get_tap_logs and get_pipeline_status to track its outcome. mode=test is read-only and is never debounced.
Watching a tap run
A/tap/run response comes back as soon as records are handed off to the async ingestion pipeline — the actual load is still in progress. Use publisherToken to poll:
{rollup, events} wrapper covering every ingestion job this tap run submitted (for structured taps that’s one job; for document taps it’s one per submitted document). Poll until rollup.allDone is true, then read rollup.status for the outcome and rollup.jobs[].lastError for any failure. See the Pipeline Status API → Rollup Response for the full shape.
Generate CRON Expression (AI)
Run History
Document Ledger
Document taps track which files they’ve already processed in a ledger — a MongoDB collection keyed by{tapName}|{uri}. See Document Taps → The Document Ledger for the concept; these endpoints manage it.
Read ledger
Delete one entry (force re-process)
Clear the entire ledger (force full re-scan)
- The tap is deleted (
DELETE /api/v1/tap). - The target pipeline’s data is cleared (
DELETE /api/v1/pipeline?pipeline=X&deleteData=true). Without this, the tap would skip docs it already “processed” and the pipeline would stay empty.
Available Vector Stores
[qdrant, weaviate, pgvector, milvus, chroma] whose Vault secret is present and whose service is currently reachable. Used by the document-tap pipeline wizard to drive the store picker. Secret presence alone isn’t sufficient — the dev stack seeds placeholder secrets for every store, so the endpoint actually probes each service.
Response:
/api/v1/health/services, with a 2-second timeout per store (~10 s worst case if everything is unreachable).
Tap Prompt Fragments
User-configured system prompt fragments that auto-inject into tap AI flows when the user’s text matches the fragment’s key or any alias. See Tap Prompt Fragments for the full concept; this section covers the REST endpoints. Fragments are per-tenant, stored in MongoDB at{env}-tap-prompt, and matched case-insensitively with word boundaries.
List Fragments
TapPromptFragment objects.
Get Fragment
{key} is not found. {key} should be URL-encoded.
Create or Update Fragment
key already exists, it is overwritten. createdAt is preserved across updates; updatedAt is refreshed on every write. Writing through this endpoint invalidates the in-process fragment cache immediately, so the next tap generation / fix / optimize / brainstorm call sees the change without restart.
Body:
Response:
{"status": "ok"} on success.
Delete Fragment
{key}. Returns {"status": "ok"}.
Suggest Fragment Content
content is non-empty in the request, the LLM is instructed to refine/expand it rather than replace.
Body:
Authentication
All endpoints require thex-api-key header if API key authentication is enabled.