Krypton Runtime

Agent CRD

Mon, 01 Jan 0001 00:00:00 +0000

apiVersion: krypton.ai/v1alpha1, kind: Agent, namespaced (short name ag).

Minimal example

apiVersion: krypton.ai/v1alpha1
kind: Agent
metadata:
 name: travel
 namespace: agents
spec:
 image: ghcr.io/org/travel-agent:latest

That’s the smallest valid Agent. Everything else has a default.

Full example

apiVersion: krypton.ai/v1alpha1
kind: Agent
metadata:
 name: travel
 namespace: agents
spec:
 # Container
 image: ghcr.io/org/travel-agent:latest
 imagePullPolicy: IfNotPresent
 imagePullSecrets:
 - name: ghcr-secret

 # Metadata (informational — surfaced in UI + metrics)
 runtime: python
 framework: langgraph

 # Wire protocol the agent speaks
 protocol: a2a  # a2a | mcp | http

 # Mode
 mode: always-on  # always-on (default); scale-from-zero is on the roadmap

 # Scaling
 minReplicas: 1
 maxReplicas: 10
 concurrency: 8 # in-flight requests per pod

 # Networking
 port: 8080
 invocationPath: /a2a

 # Pod
 resources:
 requests: { cpu: 100m, memory: 256Mi }
 limits: { cpu: 1000m, memory: 1Gi }
 env:
 - { name: LOG_LEVEL, value: info }
 envFrom:
 - secretRef: { name: travel-secrets }
 serviceAccountName: "" # blank = auto-create

 # Lifecycle
 timeout: 60s  # per-invocation
 startupTimeout: 30s  # cold-start grace

Spec reference

Container

Field	Type	Default	Notes
`image`	string (required)	—	Container image, including registry + tag
`imagePullPolicy`	string	`IfNotPresent`	Standard K8s pull policy
`imagePullSecrets`	`[]corev1.LocalObjectReference`	—	Same shape as a pod’s `imagePullSecrets`

Metadata

Field	Type	Default	Notes
`runtime`	string	—	Informational (`python`, `node`, `go`, …)
`framework`	string	—	Informational (`langgraph`, `crewai`, …)
`protocol`	string	`a2a`	One of `a2a`, `mcp`, `http`

Mode + scaling

Field	Type	Default	Notes
`mode`	string	`always-on`	Currently `always-on` only; scale-from-zero is on the roadmap
`minReplicas`	int32 (≥ 1)	`1`	Always-on floor
`maxReplicas`	int32 (≥ 1)	`10`	Must be ≥ `minReplicas`
`concurrency`	int32 (≥ 1)	`8`	In-flight requests per pod cap (enforced by sidecar)

Networking

Field	Type	Default	Notes
`port`	int32 (1–65535)	`8080`	User container’s listen port
`invocationPath`	string	`/`	Path the gateway forwards invocations to (prefix-stripped)

Pod

Field	Type	Default	Notes
`resources`	`corev1.ResourceRequirements`	—	Same as a pod’s `resources`
`env`	`[]corev1.EnvVar`	—	Passed to user container
`envFrom`	`[]corev1.EnvFromSource`	—	Same
`serviceAccountName`	string	`""`	Empty = auto-created SA with minimal permissions

Lifecycle

Field	Type	Default	Notes
`timeout`	duration	`60s`	Bounds a single invocation
`startupTimeout`	duration	`30s`	Activator’s cold-start grace; gateway returns 504 if exceeded

Status (read-only)

Field	Type	Written by
`phase`	enum	Manager
`replicas`	int32	Manager
`readyReplicas`	int32	Manager
`desiredReplicas`	int32	Scaler + activator
`url`	string	Manager
`lastInvocationAt`	time	Gateway
`observedGeneration`	int64	Manager
`conditions`	`[]Cond.`	Manager

Phases

Phase	Meaning
`Pending`	At least one desired replica, none ready yet
`Ready`	At least one replica ready, OR scaled to zero
`Scaling`	(reserved; not currently emitted)
`Failed`	Persistent reconcile errors (e.g. crashloop)

Validation

Beyond OpenAPI defaults, the (optional) validating webhook enforces:

Architecture

Mon, 01 Jan 0001 00:00:00 +0000

Krypton is composed of four binaries:

Component	Role
Manager	Kubernetes operator. Reconciles `Agent` CRs → Deployments + Services + ServiceAccounts. Runs the scaling decider.
Control plane	Read-only HTTP API + the operator UI. Optionally mirrors agents into Postgres for offline querying.
Gateway	Public ingress. Reverse-proxies invocations to the agent’s in-cluster Service.
Sidecar	Per-pod `krypton-proxy`. Enforces concurrency, surfaces in-flight count, exposes Prometheus metrics.

High-level diagram

%%{init: {"theme": "base", "flowchart": {"nodeSpacing": 70, "rankSpacing": 80, "diagramPadding": 24}, "themeVariables": {"fontFamily": "Inter, ui-sans-serif, system-ui, sans-serif", "primaryColor": "#eef2ff", "primaryTextColor": "#1f2937", "primaryBorderColor": "#6366f1", "lineColor": "#64748b", "secondaryColor": "#ecfeff", "tertiaryColor": "#f8fafc"}}}%%
flowchart TB
 client["Client"]
 ui["Krypton UI<br/>Operator console"]
 cp["Control plane<br/>REST API and cache"]
 gw["Gateway<br/>Ingress and activator"]
 mgr["Manager<br/>Controller"]
 scaler["Scaler<br/>Replica decisions"]

 subgraph pod["Agent pod"]
 proxy["krypton-proxy<br/>Concurrency and metrics"]
 app["User agent container"]
 proxy -->|"proxy"| app
 end

 ui -->|"REST"| cp
 client -->|"invoke"| gw
 gw -->|"proxy request"| proxy
 cp -->|"watch"| mgr
 mgr -->|"owns"| pod
 scaler -->|"scale"| pod
 scaler -. "in-flight" .-> proxy

 classDef external fill:#f8fafc,stroke:#94a3b8,color:#0f172a;
 classDef control fill:#eef2ff,stroke:#6366f1,color:#312e81;
 classDef traffic fill:#ecfeff,stroke:#0891b2,color:#164e63;
 classDef runtime fill:#f0fdf4,stroke:#16a34a,color:#14532d;
 classDef podbox fill:#ffffff,stroke:#cbd5e1,color:#0f172a;
 class client external;
 class ui,cp,mgr,scaler control;
 class gw,proxy traffic;
 class app runtime;
 class pod podbox;

Where state lives

State	Source of truth
Agent desired spec	The `Agent` CR (Kubernetes etcd)
`status.phase`, `replicas`	Manager writes; readers consume
`status.desiredReplicas`	Scaler (in manager)
`status.lastInvocationAt`	Gateway writes after each invocation
In-flight count	Sidecar’s `/_krypton/inflight` endpoint
Invocation history (later)	Postgres

CRDs are the source of truth. Postgres is a write-through mirror — the API serves directly from the informer cache (fresher, no DB hop).

Bring your own ingress

Mon, 01 Jan 0001 00:00:00 +0000

The krypton-gateway Service exposes plain HTTP on port 8080. Production installs put their own L7 ingress in front for TLS termination, auth, and rate limiting. The path prefix you route is always /v1/agents — the gateway handles everything under it (see Ports & endpoints).

Whichever ingress you pick, two settings are non-negotiable:

Disable response buffering — SSE / chunked HTTP need to flush as the agent emits, not at EOF.
Bump the read timeout above your agents’ worst-case response time (default 60s on most ingresses is too low for an LLM-backed agent).

Gateway API

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
 name: krypton-public
 namespace: krypton-system
spec:
 parentRefs:
 - name: my-gateway
 hostnames: ["agents.example.com"]
 rules:
 - matches:
 - path: { type: PathPrefix, value: /v1/agents }
 backendRefs:
 - name: krypton-gateway
 port: 8080

Nginx Ingress

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
 name: krypton-public
 namespace: krypton-system
 annotations:
 nginx.ingress.kubernetes.io/proxy-buffering: "off"
 nginx.ingress.kubernetes.io/proxy-read-timeout: "300"
spec:
 ingressClassName: nginx
 rules:
 - host: agents.example.com
 http:
 paths:
 - path: /v1/agents
 pathType: Prefix
 backend:
 service:
 name: krypton-gateway
 port: { number: 8080 }

Webhooks (optional)

Validating + defaulting webhooks are off by default because they require TLS plumbing (cert-manager or hand-minted certs). The CRD’s OpenAPI validation catches most spec mistakes either way.

Deploying your first Agent

Mon, 01 Jan 0001 00:00:00 +0000

This guide deploys a custom agent into a running Krypton cluster, verifies the gateway route, and shows where scaling signals come from. Krypton treats your container as a black box that speaks A2A, MCP, or plain HTTP.

Ports & endpoints — the two-minute mental model

Krypton exposes two HTTP services. Talk to the right one:

Port	Service	What lives here
8080	Gateway	All agent traffic. Anything under `/v1/agents/{ns}/{name}/*` is reverse-proxied to your pod — the protocol RPC at `/`, the A2A card at `/.well-known/agent-card.json`, OAuth callbacks at `/oauth/...`, MCP SSE streams, anything. The gateway strips the `/v1/agents/{ns}/{name}` prefix; your container sees the original sub-path.
8090	Control plane	Web UI and introspection APIs (`/v1/agents`, `/v1/agents/{ns}/{name}/mcp/tools`, `/v1/agents/{ns}/{name}/status`). Operator tooling only — never the path your clients use to invoke an agent.

Rule of thumb: if it’s a normal A2A / MCP / HTTP client, point it at :8080. If it’s a browser or kubectl-adjacent tool, :8090.

HTTP server

Mon, 01 Jan 0001 00:00:00 +0000

If your MCP server speaks MCP’s streamable-HTTP transport natively (JSON-RPC 2.0 over HTTP POST), Krypton treats it like any other Agent.

Ports refresher: client traffic to :8080 (gateway), introspection / UI on :8090 (control plane). See Ports & endpoints for the full breakdown.

Container contract

Your container needs to:

Listen on spec.port (default 8080)
Accept POST at spec.invocationPath (default /) with JSON-RPC 2.0 bodies
Implement initialize, notifications/initialized, tools/list, tools/call
Honor Accept: text/event-stream if you want strict MCP clients (Postman MCP, MCP Inspector) — return SSE frames when requested, plain JSON otherwise

Deploy

apiVersion: krypton.ai/v1alpha1
kind: Agent
metadata:
 name: my-mcp
 namespace: agents
spec:
 image: my-registry/my-mcp-server:v1
 imagePullPolicy: IfNotPresent
 protocol: mcp
 port: 8080
 invocationPath: /
 mode: always-on
 minReplicas: 1
 maxReplicas: 3

kubectl apply -f my-mcp.yaml

The pod starts, the Service exposes it inside the cluster, the operator UI surfaces its tools.

Installation

Mon, 01 Jan 0001 00:00:00 +0000

Krypton ships as a single Helm chart with four components: manager (controller + scaler), control plane (REST API + UI), gateway (public ingress + activator), and a per-pod proxy sidecar the manager injects into agent pods.

If you just want to kick the tyres on a laptop, jump to Local testing — that wraps everything below in make deploy-dev.

1. Install the chart

helm install krypton oci://ghcr.io/kryptonhq/charts/krypton \
 --namespace krypton-system \
 --create-namespace \
 --set controlPlane.databaseUrl="postgres://user:pass@host:5432/db"

This installs the most recent chart; its appVersion pins the matching image tags. To pin a specific release, pass --version 0.0.4 (latest stable is v0.0.4).

Metrics

Mon, 01 Jan 0001 00:00:00 +0000

Every Krypton component emits Prometheus metrics. The chart ships ServiceMonitor + PodMonitor resources gated behind opt-in flags so the integration is one --set away.

Quick setup with kube-prometheus-stack

If you don’t already have Prometheus Operator on the cluster:

helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update
helm install prom prometheus-community/kube-prometheus-stack \
 --namespace monitoring --create-namespace

Then enable Krypton’s monitors:

helm upgrade --install krypton oci://ghcr.io/kryptonhq/charts/krypton \
 --namespace krypton-system --create-namespace \
 --set serviceMonitor.enabled=true \
 --set podMonitor.enabled=true \
 --set serviceMonitor.labels.release=prom \
 --set podMonitor.labels.release=prom

The labels.release=prom matches kube-prometheus-stack’s default serviceMonitorSelector (replace prom with your stack’s release name). Without it, the operator’s Prometheus instance ignores the new monitors.

Components

Mon, 01 Jan 0001 00:00:00 +0000

Manager

Source: cmd/manager + internal/controller.

Standard controller-runtime operator. Reconciles Agent CRs by owning three child resources per agent: a Deployment, a Service, and a ServiceAccount. The Deployment is injected with the krypton-proxy sidecar at template-render time.

Also runs the scaling decider as a manager.Runnable.

Key behavior:

CreateOrUpdate semantics: each child resource uses controllerutil.CreateOrUpdate wrapped in retry.RetryOnConflict so spec drift converges without hot-looping when the apps controller concurrently updates Deployment status.
Status writes use Patch with MergeFrom, not Update, so they don’t conflict with the scaler/gateway’s concurrent writes to other status fields.
Finalizer krypton.ai/cleanup blocks deletion until child resources have drained.

Control plane

Source: cmd/control-plane + internal/controlplane.

Deploy Your First LLM

Mon, 01 Jan 0001 00:00:00 +0000

Krypton lets you serve an LLM the same way you manage the rest of your cluster: declare a resource, let the controller create the workload, and send traffic through the gateway.

Krypton serves Hugging Face GGUF models with llama.cpp. A Model resource points at a repo and file, the controller creates a Deployment and Service, and Krypton exposes the model through OpenAI-compatible endpoints:

GET /v1/models
POST /v1/chat/completions
POST /v1/completions
POST /v1/embeddings

Any OpenAI SDK can use the gateway as its base_url.

Model CRD

Mon, 01 Jan 0001 00:00:00 +0000

apiVersion: krypton.ai/v1alpha1, kind: Model, namespaced (short name mdl).

The Model CRD declares “host this Hugging Face GGUF as an OpenAI-compatible model named X”. The controller turns it into a Deployment running llama.cpp’s HTTP server, plus a Service. The gateway aggregates every Model in the cluster at /v1/models and routes incoming /v1/chat/completions (etc.) requests by the model field in the body.

Minimal example

apiVersion: krypton.ai/v1alpha1
kind: Model
metadata:
 name: qwen2-0-5b
 namespace: models
spec:
 source:
 huggingface: Qwen/Qwen2.5-0.5B-Instruct-GGUF
 file: qwen2.5-0.5b-instruct-q4_k_m.gguf

That’s the smallest valid Model — every other field has a default.

STDIO server

Mon, 01 Jan 0001 00:00:00 +0000

Most published MCP servers (the official @modelcontextprotocol/server-* packages, plenty of community ones) only speak stdio. Krypton can host them via mcp-stdio-bridge — a tiny HTTP-in-front-of-stdio shim that runs alongside your server inside the same container.

How it works

%%{init: {"theme": "base", "flowchart": {"nodeSpacing": 60, "rankSpacing": 70, "diagramPadding": 24}, "themeVariables": {"fontFamily": "Inter, ui-sans-serif, system-ui, sans-serif", "primaryColor": "#eef2ff", "primaryTextColor": "#1f2937", "primaryBorderColor": "#6366f1", "lineColor": "#64748b", "secondaryColor": "#ecfeff", "tertiaryColor": "#f8fafc"}}}%%
flowchart LR
 request["HTTP POST /<br/>JSON-RPC"] --> bridge["mcp-stdio-bridge"]
 bridge --> stdin["Child stdin"]
 stdout["Child stdout"] --> bridge
 bridge --> response["HTTP response<br/>id-routed"]

 subgraph child["Long-lived MCP process"]
 stdin --> process["Server command"]
 process --> stdout
 end

 classDef traffic fill:#ecfeff,stroke:#0891b2,color:#164e63;
 classDef bridgeNode fill:#eef2ff,stroke:#6366f1,color:#312e81;
 classDef processNode fill:#f0fdf4,stroke:#16a34a,color:#14532d;
 class request,response traffic;
 class bridge bridgeNode;
 class stdin,stdout,process processNode;

The bridge forks $MCP_COMMAND once at container startup.
HTTP requests are serialized to the child’s stdin.
Responses are demultiplexed by JSON-RPC id, so concurrent HTTP requests don’t cross-talk.
Notifications (no id) are fire-and-forget.
One-shot stdio servers (process-per-call) aren’t supported — the child must be long-lived.

Deploy

The published bridge image is a pure-Go binary on distroless/static. That’s enough for stdio binaries that are themselves statically linked. For runtime-dependent servers (Node, Python), layer the runtime onto a derived image first.

CLI flags

Mon, 01 Jan 0001 00:00:00 +0000

Every binary uses Go’s standard flag package. --help on any of them prints the full list.

Manager (`cmd/manager`)

The controller + scaler.

Flag	Default	Notes
`--metrics-bind-address`	`:8080`	Prometheus scrape target
`--health-probe-bind-address`	`:8081`	`/healthz`, `/readyz`
`--leader-elect`	`false`	Required when running multiple replicas
`--enable-webhooks`	`true`	Set false to skip webhook server (Helm chart does)
`--proxy-image`	`ghcr.io/kryptonhq/krypton-proxy:latest`	Sidecar image injected into agent pods
`--enable-scaler`	`true`	Disable to run controller only
`--scaler-interval-ms`	`1000`	Tick interval for the scaling decider
`--scaler-stable-window-ms`	`60000`	Hysteresis: refuse scale-down within this window
`--sidecar-port`	`8888`	Where the scaler probes `/_krypton/inflight`

Control plane (`cmd/control-plane`)

REST API + UI.

Request lifecycle

Mon, 01 Jan 0001 00:00:00 +0000

This page walks through what happens between curl POST .../invocations and the JSON coming back.

Hot path (pod already running)

%%{init: {"theme": "base", "themeVariables": {"fontFamily": "Inter, ui-sans-serif, system-ui, sans-serif", "primaryColor": "#eef2ff", "primaryTextColor": "#1f2937", "primaryBorderColor": "#6366f1", "lineColor": "#64748b", "secondaryColor": "#ecfeff", "tertiaryColor": "#f8fafc"}}}%%
sequenceDiagram
 autonumber
 participant Client
 participant Gateway
 participant Cache as Informer cache
 participant KubeProxy as kube-proxy
 participant Sidecar as krypton-proxy
 participant Agent as User container
 participant Status as Agent status

 Client->>Gateway: POST /v1/agents/agents/echo/foo
 Gateway->>Cache: Resolve Agent and ready Endpoints
 Cache-->>Gateway: echo.agents.svc:8080
 Gateway->>Gateway: Strip prefix, preserve traceparent, enable streaming flush
 Gateway->>KubeProxy: Proxy /foo
 KubeProxy->>Sidecar: Route to ready Endpoint
 Sidecar->>Sidecar: Check shutdown and acquire concurrency slot
 alt capacity available
 Sidecar->>Agent: Reverse proxy to 127.0.0.1:<spec.port>
 Agent-->>Sidecar: Streaming response
 Sidecar-->>Gateway: Release slot and forward response
 Gateway-->>Client: Response stream
 Gateway->>Status: Patch lastInvocationAt asynchronously
 else concurrency cap reached
 Sidecar-->>Gateway: 503 with Retry-After
 Gateway-->>Client: 503 with Retry-After
 end

Typical latency: P50 ~50ms, P95 ~200ms for a 100ms user-handler.

Troubleshooting & FAQ

Mon, 01 Jan 0001 00:00:00 +0000

Troubleshooting

Pods can’t pull images (`ImagePullBackOff`)

In kind, confirm the image was loaded:

kind load docker-image --name krypton-dev <image>:<tag>

Image policy must allow local images:

imagePullPolicy: IfNotPresent  # or Never for strict local-only

Cold start times out (504)

The gateway timed out waiting for an Endpoint to become ready. Common causes:

Cause	What to check
Image pull is slow (uncached)	`kubectl describe pod -n <ns> <pod>`
Agent container fails readiness	`kubectl logs -n <ns> <pod> -c agent`
Sidecar `/readyz` failing	`kubectl logs -n <ns> <pod> -c krypton-proxy`
Wrong `spec.port` (agent listens elsewhere)	Look for “connect: connection refused” in sidecar logs

Bump the timeout if your agent is slow to start:

Helm values

Mon, 01 Jan 0001 00:00:00 +0000

Chart source: deploy/helm/krypton.

Common overrides

# values.yaml
image:
 registry: ghcr.io/kryptonhq
 tag: v0.0.4

controlPlane:
 databaseUrl: "postgres://user:pass@host:5432/db"

manager:
 enableWebhooks: false
 enableScaler: true

gateway:
 service:
 type: ClusterIP  # operators add their own ingress in front

Full reference

`image`

Default image base for all components. Per-component overrides under images.* take precedence.

image:
 registry: ghcr.io/kryptonhq
 tag: "" # empty → falls back to .Chart.AppVersion
 pullPolicy: IfNotPresent

Leaving tag empty makes the chart pin images at the same version as the chart release (the helper falls back to .Chart.AppVersion), so helm install Just Works without explicit --set image.tag=....

Local testing with kind

Mon, 01 Jan 0001 00:00:00 +0000

End-to-end loop for running Krypton on your laptop. Tested on macOS (OrbStack and Docker Desktop) and Linux.

Prerequisites

Docker (or OrbStack)
kind ≥ 0.24 — brew install kind
Helm ≥ 3.14 — brew install helm
kubectl — brew install kubectl
Node 22 + pnpm 10 (for the UI build)
Go 1.25 (only required for unit tests)

One-liner on macOS:

brew install kind helm kubectl

One-shot install

make deploy-dev

What that runs (hack/local-up.sh):

Roadmap

Mon, 01 Jan 0001 00:00:00 +0000

A forward-looking view of where Krypton is headed. Order within each bucket suggests rough priority; nothing is a commitment and dates aren’t promises.

Recently shipped

Item	Notes
Built-in model hosting (`Model` CRD)	Host LLMs by Hugging Face name via llama.cpp, aggregated behind one OpenAI-compatible endpoint. See first LLM.

High priority

Item	Notes
GPU-aware scheduling	`spec.gpu` on `Model` / `Agent`; multi-GPU and MIG slicing
More inference runtimes	vLLM and TGI behind the same `Model` CRD
Model autoscaling	Concurrency-aware scaling for `Model` pods (today they’re always-on)
Secure sandbox runtime	`Sandbox` CRD with gVisor / Kata runtime classes
AI-native observability	OTEL GenAI conventions; token-usage roll-ups
OpenTelemetry tracing	OTLP exporter; `traceparent` propagation across hops

Medium priority

Item	Notes
Serverless mode (GA)	Cold-start activator path, off by default today
Invocation history (Postgres)	Queryable history, surfaced in UI
Schema-driven MCP tool forms	Auto-generated inputs per tool from its JSON schema
MCP resources + prompts	`resources/` and `prompts/` method support
Per-agent network policies	Egress / ingress rules generated from spec
TCP-dial readiness check	Closes the kube-proxy race on first cold-start

Exploring

Item	Notes
Multi-cluster federation	Federated control plane; cross-cluster routing
Authentication & multi-tenancy	OIDC-backed UI/API; per-tenant namespaces

Out of scope

Things people ask about that we deliberately won’t build:

Krypton Runtime

Agent CRD

Minimal example

Full example

Spec reference

Container

Metadata

Mode + scaling

Networking

Pod

Lifecycle

Status (read-only)

Phases

Validation

Architecture

High-level diagram

Where state lives

Bring your own ingress

Gateway API

Nginx Ingress

Webhooks (optional)

Deploying your first Agent

Ports & endpoints — the two-minute mental model

HTTP server

Container contract

Deploy

Installation

1. Install the chart

Metrics

Quick setup with kube-prometheus-stack

Components

Manager

Control plane

Deploy Your First LLM

Model CRD

Minimal example

STDIO server

How it works

Deploy

CLI flags

Manager (cmd/manager)

Control plane (cmd/control-plane)

Request lifecycle

Hot path (pod already running)

Troubleshooting & FAQ

Troubleshooting

Pods can’t pull images (ImagePullBackOff)

Cold start times out (504)

Helm values

Common overrides

Full reference

image

Local testing with kind

Prerequisites

One-shot install

Roadmap

Recently shipped

High priority

Medium priority

Exploring

Out of scope

Manager (`cmd/manager`)

Control plane (`cmd/control-plane`)

Pods can’t pull images (`ImagePullBackOff`)

`image`