> ## Documentation Index
> Fetch the complete documentation index at: https://docs.powabase.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# Sources & Extraction
> Sources are the entry point for documents in the platform. Upload files, and the extraction pipeline converts them into structured text that can be indexed into knowledge bases.
## What is a Source?
A Source represents a single uploaded document. When you upload a file, the platform creates a Source record, stores the original file, and kicks off an asynchronous extraction pipeline. The pipeline extracts text content from the document, handling PDFs, Word documents, images (via OCR), and other file types. The result is clean, structured text organized by page.
## Supported File Types
| Type | Extensions | Extraction Method |
| ---------- | ------------------------------------- | ------------------------------------------------------------- |
| PDF | .pdf | Multiple extractors — see 'Extraction Models' below |
| Word | .docx, .doc | Structured text extraction preserving headings and formatting |
| Images | .png, .jpg, .jpeg, .webp, .gif, .tiff | OCR (Optical Character Recognition) |
| Text | .txt, .md, .csv | Direct text reading |
| PowerPoint | .pptx | Slide-by-slide text extraction (REST API only) |
| Excel | .xlsx | Sheet-by-sheet cell content extraction (REST API only) |
| URLs | http(s):// | Fetched via URL import — single URLs, crawl, or sitemap |
## Extraction Models (PDF)
For PDFs you can choose how the text is extracted by passing extraction\_model at upload time or via POST /api/sources/\{id}/reextract. If you don't pass one, the pipeline uses auto, which tries mistral → opendataloader → fitz → pdfplumber in order until one succeeds. paddleocr and lighton are not part of the auto chain — request them explicitly.
| Model | What it does | Requires |
| -------------- | --------------------------------------------------------------------- | ----------------------------------------- |
| auto | Default fallback chain (mistral → opendataloader → fitz → pdfplumber) | — |
| mistral | Mistral OCR — scanned PDFs, image-heavy documents | MISTRAL\_API\_KEY |
| paddleocr | PaddleOCR-VL API — strong non-English support and layout detection | PADDLEOCR\_API\_KEY, PADDLEOCR\_BASE\_URL |
| lighton | LightOn OCR API | LIGHTON\_API\_KEY, LIGHTON\_BASE\_URL |
| opendataloader | Local high-accuracy structural extraction (tables, headings, layout) | None (local) |
| fitz | PyMuPDF — fast, text-based extraction | None (local) |
| pdfplumber | Reliable fallback for complex tables and unusual layouts | None (local) |
## Extraction Pipeline
When you upload a file, it goes through several stages: the file is stored in project storage, a Celery worker picks up the extraction task, the appropriate strategy is selected based on file type, and the extracted content is stored as derivatives (page texts, markdown, per-page images) associated with the source. This runs asynchronously, so poll the source status to check progress.
## Status Lifecycle
Every source goes through an extraction\_status lifecycle. After upload the source is pending; a worker picks it up and it moves to extracting; on success it becomes extracted. If some pages fail but others succeed the status is attention\_required (a partial success that's still indexable). Terminal states are extracted, failed, attention\_required, and cancelled.
| Status | Meaning |
| ------------------- | ------------------------------------------------------------------------------ |
| pending | Uploaded but not yet picked up by a worker |
| extracting | Currently being extracted |
| extracted | Extraction finished — derivatives available, source is indexable |
| attention\_required | Partial success — some pages failed. error\_message explains. Still indexable. |
| failed | Extraction failed — check error\_message for details |
| cancelled | User cancelled via POST /api/sources/{id}/cancel |
## Storage Integration
You can also import files that are already in your project's storage buckets using the import-from-storage endpoint. This avoids re-uploading files and is useful when you have an existing storage workflow. The extraction process is the same regardless of whether the file was uploaded directly or imported from storage.
## Next Steps
Step-by-step guide to uploading and extracting.
Turn extracted text into searchable vectors.
Full endpoint documentation.