Installation
The LiteLLM Operator can be installed three ways depending on your cluster setup.
OLM (OpenShift / OperatorHub)
For clusters with the Operator Lifecycle Manager installed (all OpenShift clusters, or vanilla Kubernetes with OLM):
operator-sdk run bundle ghcr.io/palenaai/litellm-operator-bundle:v0.5.0Verify the installation:
kubectl get csv -n operators
kubectl get crd | grep litellmTo uninstall:
operator-sdk cleanup litellm-operatorOLM v1 catalog (GitOps)
Each release publishes a versioned file-based catalog (FBC) at ghcr.io/palenaai/litellm-operator-catalog:vX.Y.Z. It contains only OLM package/channel metadata and a SHA256-pinned reference to the public bundle. Maintainers must mark this GHCR package public once so the cluster's catalogd can pull it without registry credentials.
apiVersion: olm.operatorframework.io/v1
kind: ClusterCatalog
metadata:
name: litellm-operator
spec:
source:
type: Image
image:
ref: ghcr.io/palenaai/litellm-operator-catalog:vX.Y.Z
---
apiVersion: olm.operatorframework.io/v1
kind: ClusterExtension
metadata:
name: litellm-operator
spec:
namespace: litellm-operator-system
source:
sourceType: Catalog
catalog:
packageName: litellm-operator
version: X.Y.Z
selector:
matchLabels:
olm.operatorframework.io/metadata.name: litellm-operatorUpgrading
Each release's catalog holds only that release, so to upgrade, point the ClusterCatalog at the new tag and set the ClusterExtension's version to match (or omit version to take the newest version the catalog serves). Every bundle's channel entry carries skipRange: "<X.Y.Z", which tells OLM it may replace any earlier version. That keeps OLM v1's default CatalogProvided upgrade policy working even though the older bundle is not in the new catalog, so you do not need upgradeConstraintPolicy: SelfCertified, which would also permit downgrades.
Helm Chart
For vanilla Kubernetes, k3s, RKE2, and other clusters without OLM:
helm install litellm-operator deploy/charts/litellm-operator/To customize values:
helm install litellm-operator deploy/charts/litellm-operator/ \
--set image.repository=ghcr.io/palenaai/litellm-operator \
--set image.tag=v0.5.0To uninstall:
helm uninstall litellm-operatorDirect (Makefile)
For development and CI/CD:
# Install CRDs only
make install
# Deploy the operator to the cluster
make deploy IMG=ghcr.io/palenaai/litellm-operator:v0.5.0
# Or run locally against your kubeconfig cluster
make runTo uninstall:
make undeploy
make uninstallSingle YAML Install
For a standalone install without Helm or OLM:
# Build the installer manifest
make build-installer IMG=ghcr.io/palenaai/litellm-operator:v0.5.0
# Apply it
kubectl apply -f dist/install.yamlNamespace-Scoped Watching
By default, the operator watches all namespaces. To restrict it to specific namespaces:
Helm
helm install litellm-operator deploy/charts/litellm-operator/ \
--set watchNamespaces="team-a,team-b"Flag
Pass --watch-namespaces to the manager binary:
/manager --watch-namespaces=team-a,team-bOLM
When installed via OLM in OwnNamespace or SingleNamespace mode, OLM automatically sets the WATCH_NAMESPACE environment variable. The operator reads this variable and scopes its watches accordingly — no additional configuration needed.
Makefile (development)
Set the environment variable before running:
WATCH_NAMESPACE=default make runOperator Log Level
The operator logs at info in JSON encoding. Raise the verbosity when debugging.
Helm
helm upgrade litellm-operator deploy/charts/litellm-operator/ \
--set logging.level=debug| Value | Default | Description |
|---|---|---|
logging.level | info | debug, info, error, or a positive integer for V-level logging. 1 enables the operator's own per-reconcile diagnostics |
logging.development | false | Console encoding, debug level and stacktraces on warnings. Overrides logging.level; intended for local runs, not clusters |
logging.encoder | "" | json or console. Empty follows the mode above |
extraArgs | [] | Extra manager flags the values above do not model, e.g. --zap-stacktrace-level=panic |
Flag
/manager --zap-log-level=debugAll of controller-runtime's zap flags are available (--zap-devel, --zap-encoder, --zap-stacktrace-level, --zap-time-encoding).
WARNING
debug is genuinely verbose: the operator emits a V(1) line per instance reconcile, so one instance produces roughly 100 lines an hour. Prefer raising the level temporarily while diagnosing rather than leaving it on.
Verify Installation
After installation, verify the operator is running:
# Check the operator pod
kubectl get pods -n litellm-operator-system
# Check CRDs are installed
kubectl get crd | grep litellm
# Expected CRDs:
# litellminstances.litellm.palena.ai
# litellmmodels.litellm.palena.ai
# litellmteams.litellm.palena.ai
# litellmusers.litellm.palena.ai
# litellmvirtualkeys.litellm.palena.ai