> ## Documentation Index
> Fetch the complete documentation index at: https://docs.datris.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# API File Upload

> Ingest data via API file upload

Upload data files directly to the pipeline through the REST API. The pipeline accepts any file type (CSV, JSON, XML, Excel, PDF, Word, PowerPoint, HTML, email, EPUB, plain text, and archives), handles decompression automatically, and returns a tracking token.

## Endpoint

```
POST /api/v1/pipeline/upload
```

## Request

The request uses `multipart/form-data` encoding with the following parts:

| Part             | Required | Description                                                            |
| ---------------- | -------- | ---------------------------------------------------------------------- |
| `file`           | Yes      | The data file to upload                                                |
| `pipeline`       | Yes      | Name of the target pipeline configuration                              |
| `publishertoken` | No       | Opaque token identifying the data publisher; used for lineage tracking |

### Example

```bash theme={null}
curl -X POST "http://localhost:8080/api/v1/pipeline/upload" \
  -F "file=@transactions_2025.csv" \
  -F "pipeline=transactions" \
  -F "publishertoken=finance-team-a"
```

## Response

A successful upload returns HTTP 200 with a **bare string** body — the pipeline token itself (a unique UUID generated for the job), not a JSON object:

```
d4f8e1a2-7b3c-4e9f-a5d6-1c2b3e4f5a6b
```

Use this token to track the job's processing status, view errors, or cancel the job.

When a compressed archive contains multiple inner files that are submitted as separate jobs, the response is instead the literal string:

```
3 file(s) submitted
```

## Compressed Files

When the uploaded file has a compressed extension, the archive is extracted **in memory** and processed **synchronously** — nothing is staged to MinIO:

| Extension | Handling               |
| --------- | ---------------------- |
| `.zip`    | Extracted in memory    |
| `.gz`     | Decompressed in memory |
| `.tar`    | Extracted in memory    |
| `.jar`    | Extracted in memory    |

Compressed archives may contain multiple data files. Handling depends on the pipeline type:

* **CSV pipelines** — when an archive contains multiple inner files, they are concatenated into a single job. The header is kept from the first file and stripped from the 2nd and later files. The response is the single pipeline token.
* **Non-CSV pipelines, or single-file archives** — each inner file is submitted as its own job. With more than one file, the response is the `"N file(s) submitted"` string.

## Uncompressed Files

Files without a recognized compressed extension (e.g., `.csv`, `.json`, `.xml`, `.xls`, `.pdf`, `.docx`, `.txt`, `etc`) are read into memory and submitted directly. They are not staged to MinIO.

## Processing Flow

1. The client sends the multipart request.
2. The endpoint validates that the named `pipeline` is registered.
3. The file bytes are read into memory and the extension is inspected.
4. **Compressed path**: the archive is decompressed in memory. For CSV pipelines with multiple inner files, the files are concatenated (headers stripped after the first) and submitted as one job; otherwise each inner file is submitted individually.
5. **Uncompressed path**: the file contents are passed directly to the ingestion pipeline.
6. The endpoint returns the pipeline token (or the `"N file(s) submitted"` string).

## Error Responses

| HTTP Status | Cause                                                                      |
| ----------- | -------------------------------------------------------------------------- |
| 400         | Missing `pipeline` request parameter (Spring binding error)                |
| 500         | Pipeline not registered, empty file, internal error, or processing failure |

The 500 response covers all errors handled by the controller, including an empty file and an unregistered pipeline. There is no explicit `413` handler; exceeding the configured multipart size limit produces Spring's default multipart-size error.

## Size Limits

The maximum upload size is controlled by `spring.servlet.multipart.max-file-size` in `application.yaml`. The default is 1 GB. Adjust this value if your files exceed the limit:

```yaml theme={null}
spring:
  servlet:
    multipart:
      max-file-size: 1GB
```