# 0013. CNI presets: the interface, custom's contract, Hubble's certificates, and where the guards live

Status: Accepted; superseded in part by ADR 0099 · Date: 2026-10-25 · Design section: CNI

Decided: Settled the `cni.Preset` interface, `custom`'s required fields, Hubble's certificate handling, and where the two immutability guards live

## Context

The design names six responsibilities a CNI preset carries and says a new
CNI is a new implementation of that interface and nothing else: Talos-side
patches, manifest rendering for the bootstrap, an Argo CD Application
template, whether it can replace kube-proxy, a consistency check against the
live configuration, and the workloads that signal a successful hand-over.
Three things needed deciding to build it: the interface's exact shape, what
`custom` makes the user declare, and where two "refuse this" rules actually
live, since the obvious place for one of them does not work.

## Decision: the interface

`pkg/cni.Preset` has exactly the six methods the design names, one per
responsibility, taking `KubeProxyMode` ("enabled" or "replaced") where the
mode changes the answer:

```go
type Preset interface {
	TalosPatches(kubeProxy KubeProxyMode) ([]ichorv1alpha1.TalosConfigPatchDocument, error)
	BootstrapManifest(kubeProxy KubeProxyMode) ([]byte, error)
	ArgoCDApplication(kubeProxy KubeProxyMode) (*gitops.Application, error)
	SupportsKubeProxyReplacement() bool
	CheckConsistency(ctx context.Context, live LiveConfig, kubeProxy KubeProxyMode) error
	HandoverWorkloads() []WorkloadRef
}
```

No extra methods, no constructor argument beyond what each preset needs
internally. `TalosPatches` returns `TalosConfigPatchDocument` directly --
the same type a cluster-layer `TalosConfigPatch` already carries -- so a
future caller applies it exactly the way `pkg/render` already applies any
other patch, instead of inventing a second patch shape.

**Update (ADR 0019's `ichor init` work):** `ArgoCDApplication` originally
returned `*unstructured.Unstructured`, for the reason above -- pulling in
`argoproj/argo-cd`'s Go module for one struct's field names is a dependency
for a few lines, and `unstructured.Unstructured` was already a transitive
one. That reasoning held right up until a caller needed to *use* the
result: `ichor init --gitops-dir` writes every Argo CD Application it
builds through `pkg/gitops.Skeleton`, and `pkg/gitops` is where the actual
protections against writing a credential into Git live -- the `RepoURL`
userinfo check (twice: once in `gitops.New`, once again in
`Skeleton.Write` for an `Application` that reached it some other way) and
the refusal to write a `*corev1.Secret`. An `*unstructured.Unstructured`
carries none of that; a caller handed one has to re-derive whatever
`pkg/gitops` already guarantees, or (worse) skip it. `ArgoCDApplication`
now returns `*gitops.Application` -- `pkg/cni` depends on `pkg/gitops`,
which is the right direction: the package that builds an Argo CD
Application for a specific CNI depends on the package that models what an
Argo CD Application is, not the other way around. `pkg/gitops` gained one
field it didn't have yet to carry this (`ApplicationSourceHelm.Values`,
`spec.source.helm.values` per argo-cd.readthedocs.io's Helm page, retrieved
2026-10-25) -- Cilium's rendered values document goes there now, exactly
as it did in the unstructured map before.

`flannel.Preset` is the zero-value proof that "Ichor does nothing" is
literal: every method either returns `nil, nil` or, where the design implies
a hard "no" (kube-proxy replacement), an error. There is nothing to
configure, so there is no constructor argument and no state.

`cilium.Preset` renders Helm values, not Helm's expanded chart templates.
Actually running Helm -- fetching `cilium/cilium` and expanding its
templates -- needs a network call and is bootstrap wiring for the unit that
calls this package (`ichor init`, explicitly out of this change's scope).
What `BootstrapManifest` and `ArgoCDApplication` both render today is the
resolved values document the chart would be installed with
(`pkg/cni/cilium/values.go`), plus the chart's coordinates
(`helm.cilium.io`, chart `cilium`, version `1.18.0` -- pinned to the version
the sourced Talos values below were written against). That is honest about
what this phase delivers and is exactly the "declare values and chart
coordinates, and be able to render" boundary the unit was scoped to.

**Sources, not memory**, since both are Ichor dependencies this change
touches directly:

- Talos's recommended Cilium values -- `ipam.mode: kubernetes`,
  `kubeProxyReplacement`, `k8sServiceHost: localhost` /
  `k8sServicePort: 7445` (KubePrism) for the kube-proxy-free mode, the
  `securityContext.capabilities` lists for `ciliumAgent` and
  `cleanCiliumState`, and `cgroup.autoMount.enabled: false` /
  `cgroup.hostRoot: /sys/fs/cgroup` -- are from
  `docs.siderolabs.com/kubernetes-guides/cni/deploying-cilium`, the Talos
  v1.14 docs referencing Cilium 1.18.0, retrieved 2026-10-25.
- `hubble.tls.auto.method: cronJob` (see below) is from
  `docs.cilium.io/en/stable/helm-reference/`, retrieved 2026-10-25 (current
  stable reference, ~1.20.4).
- `ServerSideApply=true` as the large-CRD-set sync option, and the
  `source`/`destination`/`syncPolicy.syncOptions` Application shape, are
  from `argo-cd.readthedocs.io`'s sync-options and
  `operator-manual/application.yaml` pages, retrieved 2026-10-25.
- The Cilium agent/operator workload names (`cilium` DaemonSet,
  `cilium-operator` Deployment, both in `kube-system`) returned by
  `HandoverWorkloads` are Cilium's own standard install layout, confirmed
  against `docs.cilium.io`'s current Helm install docs, retrieved
  2026-10-25.

No Sidero Omni material was read for any of this.

## Decision: what `custom` makes the user declare

`TalosClusterCNICustom` holds exactly the two things the design says a
`custom` preset cannot infer on its own: `documents`
(`[]TalosConfigPatchDocument`, the Talos-side settings the CNI needs, e.g.
disabling kube-proxy) and `source` (`repoURL`, one of `chart` or `path`, and
`targetRevision` -- the same four names Argo CD's own `ApplicationSource`
uses). There is no `pkg/cni` `Preset` implementation for `custom`: the
design's interface is for CNIs Ichor knows how to render something for.
`custom` is the opposite case -- the user's chart/manifests and the user's
declared patches pass straight through to the reconciler and the Argo CD
Application it builds, with nothing for a `Preset` to compute. Building a
no-op `custom.Preset` that exists only to satisfy an interface no caller
needs yet would be the exact kind of extension point ponytail's ladder says
not to add.

## Decision: Hubble's certificates

> Update (ADR 0055): re-checked against the Cilium v1.20.0 chart; `cronJob` kept, and the Application also ignores `CiliumIdentity`.

Cilium's Helm chart defaults `hubble.tls.auto.method` to `helm`, which
regenerates the mTLS certificate as a Helm hook on every install/upgrade --
and so on every Argo CD sync, which runs `helm template`/`helm upgrade`
repeatedly, means a permanent diff. Setting `hubble.tls.auto.method:
cronJob` is enough: it is a plain Helm value, already inside
`pkg/cni/cilium/values.go`'s `Values.Hubble`, carried into both
`BootstrapManifest`'s rendered values and `ArgoCDApplication`'s
`spec.source.helm.values`. No separate CronJob manifest needed to be
hand-authored; the chart renders its own CronJob once this value is set.

## Decision: where the two "refuse this" rules live

**Preset is immutable.** `TalosClusterCNI`'s own
`+kubebuilder:validation:XValidation:rule="self.preset == oldSelf.preset"`
rejects the change. This is a CRD-embedded CEL rule, like every other
invariant already in `api/v1alpha1` (ADR 0003's allow-list check, the
patch/target discrimination rules) -- not a webhook, both because
`internal/webhook/` was out of this change's scope and because a rule that
only ever compares two fields on the same object needs nothing a webhook
would add. It was generated and confirmed against envtest: changing
`preset` on an existing `TalosCluster` is rejected; every other update
(including one that changes something else entirely) is accepted.

**Switching `kubeProxy` needs the
`ichor.maesh.dev/confirm-kube-proxy-switch` annotation.** This one could
not be a CRD-embedded rule, which the `preset` rule above is. CRD
`x-kubernetes-validations` only exposes `metadata.name` and
`metadata.generateName` on `self`/`oldSelf` -- confirmed the hard way: a
rule written as `self.metadata.annotations` failed CRD creation outright
with `undefined field 'annotations'` against envtest's real apiserver, not
a lint warning. There is no CRD-schema-only way to gate on an annotation.
The remaining options were a controller-runtime webhook
(`internal/webhook/`, explicitly out of this change's scope -- another unit
owns it) or a `ValidatingAdmissionPolicy`: a built-in, declarative,
CEL-evaluated admission rule (stable since Kubernetes v1.30; this project's
pinned k8s.io/api is v0.36, i.e. envtest 1.36) that evaluates against the
full admission request -- `object`/`oldObject`, metadata included -- rather
than the CRD's own schema. It is still CEL, still no Go admission code, and
it does not touch `internal/webhook/`.

`config/admission/kubeproxy-switch-policy.yaml` carries the
`ValidatingAdmissionPolicy` and its `ValidatingAdmissionPolicyBinding`,
matching `UPDATE`s to `talosclusters.ichor.maesh.dev` and denying any
update that changes `spec.cni.kubeProxy` unless the annotation is present.
It is wired into `config/default/kustomization.yaml`, so a kustomize-based
deploy installs it, and mirrored -- identical below the
`apiVersion`/`kind`/`metadata.name` lines -- into
`dist/chart/templates/admission/kubeproxy-switch-policy.yaml`, gated by a
new `admission.enabled` value (default `true`, independent of
`webhook.enabled`: this guard is enforced by the apiserver itself, not the
controller-runtime webhook server, so it still holds while the controller
is down). `dist/chart/chart_test.go`'s `checkKubeProxySwitchPolicy` guards
both halves the same way the existing checks guard the webhook
configuration: it fails if either the `ValidatingAdmissionPolicy` or its
`ValidatingAdmissionPolicyBinding` is missing from the rendered chart, and
fails if the Binding's `spec.policyName` doesn't name the Policy this chart
ships -- a Policy with no Binding, or a Binding naming the wrong Policy,
guards nothing. Verified by deliberately breaking each half (removing the
template; retargeting `policyName` to a name that doesn't exist) and
confirming `go test ./dist/chart/...` fails with a message naming the exact
drift, then restoring it and confirming the suite is green again -- the
same bar `dist/chart/chart_test.go`'s own doc comment sets, and the one the
CRD-`documents[]` chart-drift incident shows matters: an invariant nobody
has watched fail is not a tested invariant. `api/v1alpha1/suite_test.go`
also loads `config/admission/kubeproxy-switch-policy.yaml` directly (not a
copy) before the CNI specs run, so the kustomize path is tested too.

Because a cluster below Kubernetes 1.30 would otherwise install this chart
successfully with the guard silently absent, `dist/chart/Chart.yaml` now
declares `kubeVersion: ">=1.30.0-0"` -- `ValidatingAdmissionPolicy` is
stable, enabled by default, from exactly that version
(kubernetes.io/docs/reference/access-authn-authz/validating-admission-policy/,
retrieved 2026-10-25), so an older cluster now fails the install outright
instead of silently shipping without the guard.

**The annotation unblocks the API-level guard; it does not run the safe
switch.** `ichor.maesh.dev/confirm-kube-proxy-switch` only lets the
`kubeProxy` update through admission -- nothing in this change (or
anywhere else in the codebase yet) brings the new side up on every node,
verifies it, and then turns the old side off, which is the one safe order
the design requires once a switch is allowed. Until that exists (the
reconciler's job, out of this change's scope -- the bar here was "declare
the hand-over workloads and the consistency check," not "execute the
switch"), adding the annotation and flipping `kubeProxy` only clears the
admission guard; actually carrying out the switch safely is a manual,
unassisted operation. Do not read the annotation's existence as a
statement that switching is safe to automate yet.

## Consequences

- Adding a CNI (`calico`, later) means writing one `pkg/cni/<name>`
  package implementing the six methods, plus whatever `TalosCluster`
  CRD validation its own quirks need -- no interface change.
- `pkg/cni/cilium`'s values are pinned to a Cilium/Talos-doc version pair
  (1.18.0) that will drift from upstream recommendations over time; bumping
  it is a values.go change plus re-reading the same two source pages, not a
  structural one.
- The kube-proxy-switch guard now ships identically through both install
  paths -- `make deploy`/`kustomize build config/default` and the Helm
  chart -- and `dist/chart/chart_test.go` fails loudly if the two drift
  again, the same way it already does for the CRDs.
- A chart install against a pre-1.30 Kubernetes now fails outright
  (`kubeVersion`), rather than silently shipping without the guard.
- The annotation is a necessary, not sufficient, condition for a safe
  `kubeProxy` switch: it only clears admission. Nothing executes the
  design's required new-side-up/verify/old-side-down sequence yet, so a
  switch today is a manual operation even once the annotation is set.

- Superseded in part by ADR 0066: the new-side-up, verify, old-side-down sequence is now executed by a `TalosOperation` of kind `cni`.