Skip to content

CRD Reference ​

The LiteLLM Operator defines nine Custom Resource Definitions in the litellm.palena.ai/v1alpha1 API group.

Overview ​

CRDShort NameScopeDescription
LiteLLMInstanceliNamespacedPrimary CRD. Deploys LiteLLM proxy infrastructure
LiteLLMOrganizationloNamespacedCreates an organization for multi-tenant isolation
LiteLLMModellmNamespacedRegisters an AI model with the proxy
LiteLLMTeamltNamespacedCreates a team with budget and member management
LiteLLMUserluNamespacedCreates a user (non-SSO environments)
LiteLLMCustomerlcustNamespacedCreates an external end-user with budgets and rate limits
LiteLLMCredentiallcNamespacedDeclares a reusable provider credential materialized into credential_list
LiteLLMGuardraillgNamespacedDeclares a content moderation / safety guardrail materialized into guardrails
LiteLLMBudgetlbNamespacedDeclares a reusable budget / rate-limit tier via /budget/*, referenced by budgetId
LiteLLMVirtualKeylkNamespacedGenerates a scoped API key

Relationship Diagram ​

text
LiteLLMInstance
├── LiteLLMOrganization (instanceRef → LiteLLMInstance)
│   └── LiteLLMTeam     (organizationRef → LiteLLMOrganization)
├── LiteLLMCredential   (instanceRef → LiteLLMInstance) — reusable provider creds
├── LiteLLMGuardrail    (instanceRef → LiteLLMInstance) — content moderation / safety
├── LiteLLMModel        (instanceRef → LiteLLMInstance, credentialRef → LiteLLMCredential)
├── LiteLLMTeam         (instanceRef → LiteLLMInstance) — guardrails: [lg-name, ...]
├── LiteLLMUser         (instanceRef → LiteLLMInstance, teamRef → LiteLLMTeam)
├── LiteLLMCustomer     (instanceRef → LiteLLMInstance) — external end-users
└── LiteLLMVirtualKey   (instanceRef → LiteLLMInstance, teamRef → LiteLLMTeam, userRef → LiteLLMUser) — guardrails: [lg-name, ...]

All secondary CRDs reference a LiteLLMInstance via spec.instanceRef, in their own namespace unless instanceRef.namespace says otherwise; an instance must opt in to references from other namespaces with spec.allowedNamespaces, which admits only LiteLLMVirtualKey unless it lists other kinds. Every other reference (teamRef, userRef, organizationRef, credentialRef) stays in the referencing resource's namespace. Teams can optionally reference a LiteLLMOrganization via spec.organizationRef. Models can reference a LiteLLMCredential via spec.litellmParams.credentialRef to reuse shared provider credentials instead of inlining API keys. Teams and virtual keys can opt in to specific LiteLLMGuardrail resources by name via spec.guardrails (enterprise feature). The operator resolves these references to find the LiteLLM API endpoint, master key, and organization ID.

Common Types ​

These types are shared across multiple CRDs:

SecretKeyRef ​

References a specific key within a Kubernetes Secret.

yaml
secretRef:
  name: my-secret    # Secret name
  key: my-key        # Key within the Secret

InstanceRef ​

References a LiteLLMInstance, by default in the same namespace.

yaml
instanceRef:
  name: my-gateway   # LiteLLMInstance name
  namespace: llm     # Optional; defaults to the resource's own namespace

An instance in another namespace must allow both the reference's namespace and its kind through spec.allowedNamespaces; otherwise the resource reports Synced=False with reason RefNotPermitted and nothing is created in LiteLLM. Only LiteLLMVirtualKey is admitted unless kinds lists others. LiteLLMModel, LiteLLMCredential and LiteLLMGuardrail cannot set namespace: each changes what the whole proxy does for every tenant, so they must live beside their instance.

TeamRef / UserRef ​

Reference a LiteLLMTeam or LiteLLMUser in the same namespace as the referencing resource.

yaml
teamRef:
  name: engineering

Quick Reference ​

bash
# List all resources
kubectl get li,lo,lm,lt,lu,lcust,lc,lg,lk

# Watch a specific type
kubectl get litellmmodels -w

# Describe a resource
kubectl describe litellminstance my-gateway

Released under the Apache 2.0 License.