BrainAPI
BrainAPI
Build

For the complete BrainAPI documentation index, see llms.txt. A markdown version of any docs page is available by appending .md to its URL. Docs MCP: /docs/mcp.

Ingestion tasks

Track asynchronous ingestion, publication, and cancellation

Text, file, URL, and structured ingest endpoints accept work asynchronously. They return 202 Accepted with a task_id. Use the tasks API to wait for the import to finish.

For agents

  • After ingest: read task_id from 202 body
  • Poll GET /tasks/{task_id} with BrainPAT (+ brain headers) until a terminal status
  • 404 = task unknown — do not treat as "still running"
  • Optional on text ingest: Task-Identifier header to choose the text ingestion task identifier

How to poll a task

  1. Submit an ingest request and read task_id from the 202 body.
  2. Poll GET /tasks/{task_id} until status is completed, failed, partial_failed, cancelled, or rollback_failed.
  3. Treat 404 as "task unknown" — not "still pending".
# 1) Ingest
RESP=$(curl -s -X POST "<DEPLOYMENT_URL>/ingest/" \
  -H "Content-Type: application/json" \
  -H "BrainPAT: YOUR_BRAIN_PAT" \
  -H "X-Brain-ID: example01" \
  -d '{
    "data": { "data_type": "text", "text_data": "Alice met Bob at Acme in 2024." },
    "brain_id": "example01"
  }')
echo "$RESP"
TASK_ID=$(echo "$RESP" | jq -r .task_id)

# 2) Poll
curl -s "<DEPLOYMENT_URL>/tasks/$TASK_ID" \
  -H "BrainPAT: YOUR_BRAIN_PAT" \
  -H "X-Brain-ID: example01"

Optional stable task id (text ingest)

For POST /ingest/, you may send a Task-Identifier header. Use the returned identifier to correlate the request and poll its status.

curl -X POST "<DEPLOYMENT_URL>/ingest/" \
  -H "Content-Type: application/json" \
  -H "BrainPAT: YOUR_BRAIN_PAT" \
  -H "X-Brain-ID: example01" \
  -H "Task-Identifier: my-stable-job-001" \
  -d '{
    "data": { "data_type": "text", "text_data": "Alice met Bob at Acme in 2024." },
    "brain_id": "example01"
  }'

Structured and file ingest always mint a new UUID task_id in the 202 body.

Endpoints that return 202

EndpointMessageNotes
POST /ingest/Ingestion acceptedHonors Task-Identifier
POST /ingest/structuredStructured ingestion acceptedNew UUID each call
POST /ingest/structured/batchStructured ingestion acceptedOne root task for a staged batch
POST /ingest/fileFile ingestion acceptedNew UUID each call
POST /ingest/urlURL ingestion acceptedNew UUID each call; poll the returned root task

Queue unavailable → 503 Task queue unavailable.

Task status API

GET /tasks/{task_id}

Returns the task ID and status, plus progress or cancellation metadata when available. Staged runs can be polled from durable state even when a cache entry is missing.

HTTPMeaning
200Task record found
404Task not found — id never queued, expired from cache, or wrong brain

Older clients that treated a missing task as soft pending will break. On 404, stop polling and decide whether to re-submit.

GET /tasks/

Lists known tasks for the brain (from the task key index). Useful for the console and ops dashboards.

Cancel an ingestion

In the hosted Data page, choose Cancel ingestion on an active import. The dialog offers the modes supported by that run. For direct engine requests, check the root task's cancellation_modes before submitting a cancellation. Use your deployment's engine URL for cancellation and activity requests.

ModeResult
keep_savedStop remaining work. Keep previously saved data and publish completed staged units when the run uses staging.
rollbackStop remaining work and discard this run's staged data. Earlier imports remain intact.
curl --fail-with-body -X POST "<DEPLOYMENT_URL>/tasks/$TASK_ID/cancel" \
  -H "BrainPAT: YOUR_BRAIN_PAT" \
  -H "X-Brain-ID: example01" \
  -H "Content-Type: application/json" \
  -d '{"mode":"rollback"}'

HTTP 202 returns task_id, mode, and the current cancellation status. Repeat the same mode to read the current state. A different accepted mode or a run past its publication cutoff returns 409. An unknown or out-of-scope task returns 404.

Keep polling through cancelling until the task reports cancelled or rollback_failed. A run can return rollback_failed when cleanup retries are exhausted. Staged data stays private until publication, which can briefly return 503 with Retry-After on other brain operations. Honor that header.

Progressive URL activity

For hosted progressive URL imports, GET /tasks/{task_id}/activity?offset=0&limit=50 reports pages, sections, progress, and cancellation state once discovery has produced activity. Follow next_offset for more sections. Use the root task endpoint to poll the overall outcome.

Troubleshooting

SymptomFix
Immediate 404 after 202Wrong X-Brain-ID / brain header vs the brain used at ingest
Stuck on queuedWorker not running or Redis/Celery down
503 on ingestBroker unavailable — retry with backoff
File ingest 202 but no progressCheck OCR mode (docling vs docparser) and worker logs
Edit on GitHub

Last updated on

On this page