Skip to main content

Data Source Monitoring

Monitor document ingestion health and source status from the Sources Dashboard at /sources/dashboard.


Ingestion health at a glance

The four stat cards on the Sources Dashboard give a 30-day snapshot:

MetricDescription
Documents Processed TodayDocuments that completed ingestion today
Documents Processed (30 Days)Total successful ingestions in the last 30 days
Error Documents (30 Days)Documents that failed ingestion in the last 30 days
Total TokensTokens consumed by ingestion in the last 30 days

Document Processing Queue

The queue section shows documents that are stuck or actively being processed:

Queue stateWhat it means
FailedIngestion attempted but failed — available for retry
WaitingQueued and waiting for the backend to process

For failed documents:

  • The Attempts column shows how many times ingestion has been tried.
  • Click Retry to re-enqueue the document immediately.

See Ingestion Queue for the full Retry flow and state transitions.


Monitoring a specific source

Each source has its own detail page at /sources/{sourceCode}. The stats panel shows:

StatDescription
Total DocumentsDocuments currently in the source
Tokens ConsumedLifetime tokens used for this source's ingestion
EnabledWhether the source is active for RAG retrieval
StatusCurrent operational state
Last ProcessedWhen ingestion last completed
Last ScannedWhen the source was last scanned for new documents
Last ModifiedWhen the source configuration was last changed

Recent Ingestion Sessions

The Recent Ingestion Sessions table on the Sources Dashboard shows the last ~10 ingestion events across all sources. Columns: Document, Source, User, Tokens, Start Time.

Use this table to quickly identify which user triggered a large ingestion, spot sources with high token usage, or find a recent document by filename.