AI Assistant & LLM Configuration

Nocarta includes a built-in AI Assistant powered by Large Language Models (LLMs). It works out of the box with Nocarta AI — no setup required. You can also bring your own LLM provider for additional flexibility.

Overview

The AI Assistant can help you with questions about Nocarta features, troubleshooting, and general tasks. It works with multiple LLM providers:

Ollama - Run models locally on your own machine (fully private)
OpenAI - GPT models via the OpenAI API
Anthropic - Claude models via the Anthropic API
Google Gemini - Gemini models via the Google AI API
Custom (OpenAI-compatible) - Any server that implements the OpenAI-compatible API (vLLM, LM Studio, etc.)

Nocarta AI

Nocarta AI is the built-in assistant that works immediately — no API keys, no external accounts, no configuration needed. Every Nocarta account has access to it from day one.

In the LLM Configuration section of your Settings, Nocarta AI appears with a SYSTEM badge, indicating it is provided by Nocarta and always available.

How It Works with Your Own LLMs

No LLM configured: Nocarta AI is used automatically for both the General Assistant and the Automation Assistant.
Your own LLMs configured: Your configurations take priority. Nocarta AI remains available as a fallback — it appears last in the LLM selector.
Switching: You can manually select Nocarta AI from the assistant dropdown at any time, even if you have your own LLMs configured.

Fair Use Limits

To ensure reliable service for all users, Nocarta AI has fair use limits on message frequency. If you reach these limits, you will be asked to wait before sending more messages. The limits are generous for normal usage and are designed to prevent abuse, not restrict legitimate work.

Configuring Your Own LLM Provider

To set up your first LLM configuration:

Go to Settings → Preferences
Scroll to the LLM Configuration section
Click the Add LLM button
Choose a provider from the dropdown: Ollama, OpenAI, Anthropic, Google Gemini, or Custom
Enter a configuration name (e.g., "My GPT-4") and optionally an alias
The API endpoint URL is auto-filled based on the selected provider. Adjust it if you use a custom endpoint.
Enter the model name, or click the Discover button to fetch available models (see below)
If required, enter your API key. It will be stored securely in your Secrets Vault.
Set a response timeout (how long to wait for the LLM to respond)
Click Test Connection to verify everything works
Click Save

Model Discovery

Instead of typing a model name manually, you can use the Discover button next to the model field. This queries the LLM server directly from your browser and displays all available models as clickable chips.

Discovery works with all supported providers:

Ollama - Queries /api/tags to list locally installed models
OpenAI / Custom / vLLM - Queries /v1/models using the standard OpenAI-compatible API
Anthropic - Queries /v1/models with Anthropic-specific headers
Google Gemini - Queries /v1beta/models for available Gemini models

Click on any discovered model chip to select it automatically.

Multiple Configurations

You can add multiple LLM configurations, each using a different provider and model. This is useful if you want to:

Switch between a local Ollama model (for privacy) and a cloud model (for quality)
Have different models for different tasks
Keep a backup configuration if one provider is unavailable

One configuration is always set as the default. The AI Assistant will use the default configuration unless you switch to another one.

Using Your Own LLM for Document Interpretation

Beyond the chat assistant, your LLM configurations can run the interpretation step of document processing — turning extracted text into structured values in Scan Areas, bulk instance fills, and post-processing. Text extraction (image → text) always runs on Nocarta's infrastructure; interpretation (text → meaning) is the step you can route to your own LLM — for cost control, model choice, or privacy.

Two switches per configuration

"Use as my default for interpretation" — an explicit opt-in. Document interpretation runs on this LLM instead of Nocarta AI. Merely creating a configuration does not re-route anything; only one configuration can be the interpretation default. Scan-area pipelines can also bind a specific configuration per step, which wins over the default.
"Fall back to Nocarta AI on failure" — off by default (private). Off: if your LLM fails, you get an honest error with the real reason and your document text is never sent to Nocarta AI for interpretation. On: failed interpretations retry on Nocarta AI and the result is labeled accordingly.

Where the call runs

Your configuration	Where interpretation executes
Cloud endpoint + Tier 1 key (server-encrypted)	Nocarta's servers call your endpoint directly — works even with all your devices offline. Only the extracted text is sent, never the image; calls are rate-limited per configuration so a bulk run can't burst against your account.
Localhost endpoint (Ollama, local vLLM) or Tier 2 key (vault)	Your Nocarta Desktop app executes it. Jobs queue durably and your desktop picks them up — see below.

The desktop LLM worker

Nocarta's servers can't reach localhost and can't decrypt Tier 2 keys — by design. Interpretations bound to such configurations are placed in a durable queue, and Nocarta Desktop drains it whenever it's open:

Dispatch from anywhere. Trigger a bulk fill from any browser; the affected scan areas show "queued — waiting for your desktop." Nothing fails while the desktop is offline.
Consent at app open. Jobs that accumulated while the desktop was offline never run silently: at app open a dialog lists exactly what is waiting — template, scan area, instance, how it was dispatched and when, and which LLM will run it — with Run queued jobs / Not now.
Queue interface. Settings → Preferences → LLM Configuration shows the queue from any device: each job's status, target LLM, and error, with a Cancel action for waiting jobs and the live worker state inside the desktop app.
Recovery built in. If the desktop crashes mid-job, the job is re-queued and retried. Jobs that wait longer than 24 hours expire with a clear message (or retry on Nocarta AI if you enabled fallback).
Privacy. Document text is wiped from the queue as soon as a job finishes; Tier 2 keys are decrypted only on your machine, in your unlocked vault.

Every interpreted result records which LLM produced it — the scan-area result page shows "Interpreted by: your-llm (your LLM, on your desktop)" or "Nocarta AI (fallback)" so provenance is always auditable.

Security & Privacy

Nocarta takes your privacy seriously when it comes to AI interactions:

Assistant chat with your own LLM — browser-direct: Interactive chat requests go directly from your browser to your provider.
Document interpretation with your own LLM: Runs server-side only for cloud endpoints with Tier 1 keys (extracted text only, never images, never your Tier 2 keys); localhost and vault-protected configurations execute exclusively on your desktop.
Nocarta AI — server-processed: When using Nocarta AI, requests are processed through Nocarta's servers. Your conversations are not stored or used for training.
Secure key storage: API keys for your own providers are stored in the Secrets Vault using Tier 1 (server-encrypted) or Tier 2 (master-password-protected) encryption.
Local option: Use Ollama to run models entirely on your own machine.

Troubleshooting

CORS Issues with Ollama

If you see CORS errors when connecting to a local Ollama instance, you need to allow browser access. Set the environment variable before starting Ollama, restricted to the Nocarta origin:

OLLAMA_ORIGINS=https://app.nocarta.ai ollama serve

Connection Failures

Verify the API endpoint URL is correct
Make sure the LLM server is running and reachable from your network
Check your firewall settings if connecting to a local server
Use the Test Connection button to diagnose issues

Authentication Errors

Double-check that your API key is correct and has not expired
Ensure the API key has the necessary permissions for the selected model
For Anthropic, verify your API key starts with sk-ant-
For OpenAI, verify your API key starts with sk-

Slow Responses

Increase the response timeout in your LLM configuration
For local models (Ollama), ensure your hardware meets the model requirements
Try a smaller or faster model if response times are too long