For the complete BrainAPI documentation index, see llms.txt. A markdown version of any docs page is available by appending .md to its URL. Docs MCP: /docs/mcp.
Ingestion tasks
Track asynchronous ingestion, publication, and cancellation
Text, file, URL, and structured ingest endpoints accept work asynchronously. They return 202 Accepted with a task_id. Use the tasks API to wait for the import to finish.
For agents
- After ingest: read
task_idfrom 202 body - Poll
GET /tasks/{task_id}withBrainPAT(+ brain headers) until a terminalstatus - 404 = task unknown — do not treat as "still running"
- Optional on text ingest:
Task-Identifierheader to choose the text ingestion task identifier
How to poll a task
- Submit an ingest request and read
task_idfrom the 202 body. - Poll
GET /tasks/{task_id}untilstatusiscompleted,failed,partial_failed,cancelled, orrollback_failed. - Treat 404 as "task unknown" — not "still pending".
# 1) Ingest
RESP=$(curl -s -X POST "<DEPLOYMENT_URL>/ingest/" \
-H "Content-Type: application/json" \
-H "BrainPAT: YOUR_BRAIN_PAT" \
-H "X-Brain-ID: example01" \
-d '{
"data": { "data_type": "text", "text_data": "Alice met Bob at Acme in 2024." },
"brain_id": "example01"
}')
echo "$RESP"
TASK_ID=$(echo "$RESP" | jq -r .task_id)
# 2) Poll
curl -s "<DEPLOYMENT_URL>/tasks/$TASK_ID" \
-H "BrainPAT: YOUR_BRAIN_PAT" \
-H "X-Brain-ID: example01"Optional stable task id (text ingest)
For POST /ingest/, you may send a Task-Identifier header. Use the returned identifier to correlate the request and poll its status.
curl -X POST "<DEPLOYMENT_URL>/ingest/" \
-H "Content-Type: application/json" \
-H "BrainPAT: YOUR_BRAIN_PAT" \
-H "X-Brain-ID: example01" \
-H "Task-Identifier: my-stable-job-001" \
-d '{
"data": { "data_type": "text", "text_data": "Alice met Bob at Acme in 2024." },
"brain_id": "example01"
}'Structured and file ingest always mint a new UUID task_id in the 202 body.
Endpoints that return 202
| Endpoint | Message | Notes |
|---|---|---|
POST /ingest/ | Ingestion accepted | Honors Task-Identifier |
POST /ingest/structured | Structured ingestion accepted | New UUID each call |
POST /ingest/structured/batch | Structured ingestion accepted | One root task for a staged batch |
POST /ingest/file | File ingestion accepted | New UUID each call |
POST /ingest/url | URL ingestion accepted | New UUID each call; poll the returned root task |
Queue unavailable → 503 Task queue unavailable.
Task status API
GET /tasks/{task_id}
Returns the task ID and status, plus progress or cancellation metadata when available. Staged runs can be polled from durable state even when a cache entry is missing.
| HTTP | Meaning |
|---|---|
| 200 | Task record found |
| 404 | Task not found — id never queued, expired from cache, or wrong brain |
Older clients that treated a missing task as soft pending will break.
On 404, stop polling and decide whether to re-submit.
GET /tasks/
Lists known tasks for the brain (from the task key index). Useful for the console and ops dashboards.
Cancel an ingestion
In the hosted Data page, choose Cancel ingestion on an active import. The dialog offers the modes supported by that run. For direct engine requests, check the root task's cancellation_modes before submitting a cancellation. Use your deployment's engine URL for cancellation and activity requests.
| Mode | Result |
|---|---|
keep_saved | Stop remaining work. Keep previously saved data and publish completed staged units when the run uses staging. |
rollback | Stop remaining work and discard this run's staged data. Earlier imports remain intact. |
curl --fail-with-body -X POST "<DEPLOYMENT_URL>/tasks/$TASK_ID/cancel" \
-H "BrainPAT: YOUR_BRAIN_PAT" \
-H "X-Brain-ID: example01" \
-H "Content-Type: application/json" \
-d '{"mode":"rollback"}'HTTP 202 returns task_id, mode, and the current cancellation status. Repeat the same mode to read the current state. A different accepted mode or a run past its publication cutoff returns 409. An unknown or out-of-scope task returns 404.
Keep polling through cancelling until the task reports cancelled or rollback_failed. A run can return rollback_failed when cleanup retries are exhausted. Staged data stays private until publication, which can briefly return 503 with Retry-After on other brain operations. Honor that header.
Progressive URL activity
For hosted progressive URL imports, GET /tasks/{task_id}/activity?offset=0&limit=50 reports pages, sections, progress, and cancellation state once discovery has produced activity. Follow next_offset for more sections. Use the root task endpoint to poll the overall outcome.
Troubleshooting
| Symptom | Fix |
|---|---|
| Immediate 404 after 202 | Wrong X-Brain-ID / brain header vs the brain used at ingest |
Stuck on queued | Worker not running or Redis/Celery down |
| 503 on ingest | Broker unavailable — retry with backoff |
| File ingest 202 but no progress | Check OCR mode (docling vs docparser) and worker logs |
Related
Last updated on
