LMS
Back to home
AI features

AI models

A superadmin-only page (/ai-models) for setting which provider and model each AI activity uses platform-wide - the platform-default half of ADR-066's two-tier model. A tenant can separately override any of these for their own workspace from their Feature settings page; a tenant's own override always wins when set, this page's defaults apply everywhere else.

The page is three tabs:

  • AI models - the per-activity provider/model defaults described below.
  • Provider Pool - the "Provider keys" section (ADR-067): each provider's pool of API keys.
  • Model catalog - the curated list of "known-good" models behind every "Catalog…" quick-fill dropdown (ADR-069), superadmin-managed and stored in the database instead of hardcoded in frontend source.

What's configurable

The platform's AI calls are grouped into six activities, by actual capability need rather than by which feature is calling:

  • Embeddings - knowledge-base document/lesson indexing and retrieval (RAG). Only Gemini has an embeddings API, so this activity's provider is fixed to Gemini; only the model name can be changed.
  • Course Builder chat - the AI Course Builder's tool-calling chat loop.
  • Course Q&A - answers a learner's question about a course, grounded in its indexed material.
  • Handwriting transcription - transcribes a photographed page of handwritten notes for the "Import handwritten notes" flow. Needs a vision-capable model.
  • Structured generation - short, structured drafts: code, diagrams, matching pairs, exam questions, vocabulary definitions.
  • Long-form generation - full lesson/blog content drafts and module/lesson name suggestions.

Each row lets a superadmin pick a provider, then pick a model from a dropdown of that provider's Model catalog entries, then Save - there's no free-text model field on this tab, only what's in the catalog for the selected provider and activity (the Model catalog tab is where a superadmin adds one). If a provider/activity combination has no catalog entries yet, that row's model dropdown is disabled with an explanatory placeholder until one is added. Switching a row's provider clears its model selection, since a different provider has a different catalog. An already-configured row whose saved model isn't (yet) in the catalog still shows correctly, labeled "(current, not in catalog)" - resaving it unchanged is allowed even before its catalog entry exists; picking anything else requires the catalog. This restriction is enforced server-side too, not just hidden client-side. A tenant's own Feature settings model-override row is unaffected by this - it still offers the catalog as a quick fill next to a free-text field, exactly as before. A "Clear" action reverts an activity to whatever the constructing code's own built-in default is - there's always a working fallback, so a platform default is optional, not required, for every activity to function.

A provider only appears as a real choice when this deployment actually has an API key configured for it - an unconfigured provider shows in the dropdown but disabled, with "(no API key configured)" next to it, rather than letting a superadmin pick something guaranteed to fail at call time. If no configured provider can serve an activity at all (e.g. embeddings, when no Gemini key exists), that row's whole picker is disabled with a short explanation. The same check runs server-side too (PUT /ai/model-configs/{activity_group} rejects an unavailable provider outright), not just hidden client-side.

The provider list itself (GET /ai/providers) isn't hardcoded to a fixed pair - it's a small backend registry (app/infrastructure/ai/providers.py), currently Anthropic, Gemini, and OpenAI, so adding a fourth provider later needs no change to this page, the resolution logic, or the tenant override UI.

Provider Pool

The "Provider Pool" tab (ADR-067) lets a superadmin manage each provider's own pool of API keys - add a key, see the existing ones (label + last4 only, the value is never shown again after saving), and disable one. Adding a provider's first key is what actually flips that provider from "no API key configured" to available above and in a tenant's own override picker - there's no separate enable switch. Adding a second key for a provider that already has one gives real redundancy: if a call to the currently active key gets rejected as unauthorized or rate-limited, the platform automatically retries with the pool's next key, in the same request, with no visible error. A key that's actually bad still needs a superadmin to disable it from this page by hand - this only covers that call recovering on its own, not the platform noticing and disabling a failing key for you.

Every key lives in its own Secrets Manager secret - never in this app's own Postgres database, which only ever stores a label/last4/ which-secret-it-is. Disabling a key is two steps: it stops being usable immediately (the very next AI call resolves a different key or falls back to the environment variable), and its Secrets Manager secret is then deleted with AWS's normal recovery window, not force-deleted - an accidental disable stays recoverable by an operator with direct AWS access even though this page itself has no undo.

Fallback models

A third piece, underneath each activity's own row (ADR-068): an ordered list of fallback models, tried in order when the currently active one hits an actual quota/capacity problem - either every key in its pool comes back rate-limited (429), or the model itself reports being overloaded (503). Neither a bad key nor any other kind of failure (a malformed request, a content-policy rejection) triggers a fallback - those aren't fixed by switching models, and silently retrying anyway would hide a real misconfiguration from whoever needs to notice and fix it.

A fallback candidate can be any provider, not just the one the primary uses - e.g. a Gemini primary with an Anthropic or OpenAI fallback, each drawing from its own key pool independently. Add one with the same provider/model picker the primary uses, right below it; remove one with its own "Remove" action (there's no reordering control - removing and re-adding is how the order changes). A fallback chain only exists once the activity's own primary is configured, and clearing the primary clears its whole chain too, no separate cleanup needed. Not available for Embeddings - a fixed-width vector column and no guarantee two models share a dimensionality or semantic space rules out an automatic fallback there.

Model catalog

The "Model catalog" tab (ADR-069) is where a superadmin curates the "known-good" models behind every model picker that reads it - this page's own "AI models" tab (where it's the only way to set a model, see above) and the "Catalog…" quick-fill dropdown on a tenant's own Feature settings model-override row (still a convenience there, alongside a free-text field). Each entry is a provider, a model id, and a display label (e.g. provider gemini, model gemini-3.6-flash, label "Gemini 3.6 Flash"), plus an "Embeddings only" checkbox for entries that are only ever valid for the Embeddings activity (a vision/chat model would never show up there, and an embeddings model would never show up anywhere else). Adding an entry doesn't require that provider to already have a configured key - a superadmin can stock a provider's catalog ahead of time. Removing an entry only affects the dropdown going forward; it never touches a model name a superadmin already typed or picked into an activity row or fallback slot.

This replaced a hardcoded list that used to live in the frontend's own source (MODEL_CATALOG in apps/web/lib/ai-activity-groups.ts) - adding or updating a catalog entry no longer needs a code change and deploy, just this tab. On the Feature settings model-override row it's still a pure convenience next to a free-text field, and nothing there is validated against at call time; on this page's own "AI models" tab it's the enforced source of truth for what a row can be set to, both client-side and server-side (see above).

How it's stored

Platform defaults live in their own table, ai_model_configs, keyed by activity group - not a JSONB column on tenants like flags/ feature_settings/branding, since this configuration isn't per-tenant at all. A tenant's own override, by contrast, does live in that tenant's feature_settings.ai_model_overrides (see Feature settings) - same free-form JSONB shape the rest of that column already uses.

Resolution order for any single AI call: the calling tenant's own override (if AI features are on for them and they've set one for that activity) → this page's platform default (if set) → the calling code's own hardcoded fallback - that's which model is used. See docs/ADR.md's ADR-066 for the full design.

Provider keys (ADR-067) live in a separate table, ai_provider_keys - metadata only (label/last4/which Secrets Manager secret backs the row), never the raw value. Which key is actually used for a call is resolved independently of the model: the pool's oldest active key for that provider if one exists, otherwise the provider's own environment variable, exactly as before this table existed. See docs/ADR.md's ADR-067 for the full design.

Fallback chains (ADR-068) live in a third table, ai_model_config_fallbacks - one row per (activity group, rank), FK'd to ai_model_configs with cascading delete. Resolving a call now means resolving a list of provider/model candidates (the platform default followed by its fallbacks in rank order, or a tenant override alone with no chain beneath it), each with its own independently resolved key pool - the first candidate whose key pool doesn't come back with a quota/capacity error wins. See docs/ADR.md's ADR-068 for the full design.

The model catalog (ADR-069) lives in a fourth table, ai_model_catalog_entries - also platform-wide, not tenant-scoped, and read-open (a tenant's own Feature settings page reads it under its own role, not superadmin) with superadmin-only writes. It's read by GET /ai/model-catalog, and it's never joined against ai_model_configs or ai_model_config_fallbacks at the database level - the AI models tab's PUT/POST handlers query it separately at request time to check catalog membership before writing, rather than a foreign key enforcing it, since a catalog entry can be freely removed later without needing to touch (or invalidate) whatever activity/fallback rows were set from it while it existed. Removing a catalog entry has no cascading effect on an already-configured activity, fallback, or tenant override - see docs/ADR.md's ADR-069 for the full design, including its addendum on the AI models tab's catalog-only enforcement.