What is a Source?
A Source represents a single uploaded document. When you upload a file, the platform creates a Source record, stores the original file, and kicks off an asynchronous extraction pipeline. The pipeline extracts text content from the document, handling PDFs, Word documents, images (via OCR), and other file types. The result is clean, structured text organized by page.Supported File Types
Extraction Models (PDF)
For PDFs you can choose how the text is extracted by passing extraction_model at upload time or via POST /api/sources/{id}/reextract. If you don’t pass one, the pipeline uses auto, which tries mistral → opendataloader → fitz → pdfplumber in order until one succeeds. paddleocr and lighton are not part of the auto chain — request them explicitly.Extraction Pipeline
When you upload a file, it goes through several stages: the file is stored in project storage, a Celery worker picks up the extraction task, the appropriate strategy is selected based on file type, and the extracted content is stored as derivatives (page texts, markdown, per-page images) associated with the source. This runs asynchronously, so poll the source status to check progress.
Document extraction pipeline from upload to structured derivatives
Status Lifecycle
Every source goes through an extraction_status lifecycle. After upload the source is pending; a worker picks it up and it moves to extracting; on success it becomes extracted. If some pages fail but others succeed the status is attention_required (a partial success that’s still indexable). Terminal states are extracted, failed, attention_required, and cancelled.Storage Integration
You can also import files that are already in your project’s storage buckets using the import-from-storage endpoint. This avoids re-uploading files and is useful when you have an existing storage workflow. The extraction process is the same regardless of whether the file was uploaded directly or imported from storage.Next Steps
Upload a Document
Step-by-step guide to uploading and extracting.
Knowledge Bases & Indexing
Turn extracted text into searchable vectors.
Sources API Reference
Full endpoint documentation.