Offline AI Mode

Complete privacy with local models

Offline AI mode runs a language model directly on your Mac. Your documents, your questions, and the AI responses all stay on your device. No internet connection is required after the model is downloaded.

Requirements

• Apple Silicon Mac (M1, M2, M3, M4, or later)
• At least 8 GB of unified memory (16 GB recommended for larger models)
• 4 GB of free storage for the default model

Setting up offline AI

1. Open Elephas Settings
2. Go to the AI Provider tab
3. Select Offline AI
4. Choose a model size. Smaller models are faster; larger models give better answers
5. Click Download. The model downloads once and is stored locally

When to use offline AI

• Working with confidential or sensitive documents
• No internet access available
• You want zero data transmission to external services
• Testing or evaluating Elephas without connecting to any API

Tradeoffs

Local models are smaller than cloud models and may produce less detailed answers for complex questions. For most document Q&A tasks, the quality is sufficient. You can switch between offline and cloud modes at any time to compare results.

Choosing offline when you create a Brain

You can set a Brain to be fully offline at creation: in Create Brain > Compatibility & control, choose an offline indexing approach and set the chat mode to Offline. An Offline badge then confirms indexing and chat are running locally.

Using LM Studio or Jan.ai

Besides the built-in models, Elephas can connect to a local model served by LM Studio or Jan.ai:

• LM Studio (Apple Silicon M1/M2/M3 only): install it, search and download a model (e.g. a Llama 3 8B GGUF), load/select the model, and start its local server. Then point Elephas at it in the Offline / provider settings.
• Jan.ai (works on Intel and Apple Silicon): install it, open the Hub, download a model, run it, then connect Elephas to Jan's local server.