Ichor

Ichor manages Talos Linux clusters from Kubernetes. Nodes, machine configuration, upgrades and etcd backups are Kubernetes resources, and you work with them through the ichor CLI, the ichor-ui web UI, or by writing the resources yourself.

Status: alpha (v0.1.0). Exercised end to end on Talos v1.14 test clusters, not yet on bare metal.

The commands in these docs are written as if their tools are on your PATH. Run nix develop in a checkout to get them all, or install them yourself: Before you begin lists which and at what versions.

What Ichor manages

  • Clusters and machines. A node that sits in Talos maintenance mode is discovered, approved by a person in the UI or the CLI, installed and joined to a cluster. Existing Talos clusters can be adopted, and machines that another cluster configured can be reclaimed. See Clusters and machines.
  • Machine configuration. Patches are typed resources built on the Talos machinery types, with no raw YAML. The controller renders and applies them and reports drift.
  • Operations. Talos, Kubernetes and Ichor upgrades, reboot-requiring configuration changes, node actions, shutdown, etcd restore, and rotation of CAs and keys are TalosOperation resources that run one node at a time with etcd quorum in mind. See Operations and CA and key rotation.
  • etcd backups. Scheduled or on-demand, encrypted, stored in S3-compatible storage; restore into a running cluster, or rebuild a cluster from the secrets bundle and a snapshot alone.
  • Cilium. The only CNI Ichor supports, kept as a Helm release. It is the one thing Ichor installs on the cluster besides itself. See CNI and Cilium.

Ichor does not boot machines (get them to maintenance mode by ISO, PXE or whatever you like), does not manage storage or monitoring, knows nothing about GitOps tools, and does not target air-gapped installs.

How the pieces fit

Architecture of Ichor. The browser, the ichor CLI and kubectl reach the Kubernetes API of the cluster that Ichor manages. Inside that cluster, in the ichor-system namespace, run ichor-ui, the ichor-controller with its admission webhooks, and the installer Jobs; they read and write Ichor's custom resources through the API. The API server holds Ichor's custom resources, Secrets and Leases. Only the controller configures the Talos machines, over the Talos API; a new machine waits in maintenance mode until you approve it, and then becomes a Talos node, a control plane or a worker.

Ichor runs inside the cluster it manages, and every change goes through the Kubernetes API as a custom resource.

PieceRole
Custom resourcesThe API. TalosCluster, TalosMachine, TalosConfigPatch, TalosOperation, TalosEtcdBackup, TalosStorage, MachineReclaim, TalosCredentialRequest, IchorInstallation and IchorUser. See the CRD reference.
ichor-controllerThe only component that writes to Talos nodes. It reconciles the resources, keeps the state machine of long-running work in resource status, and never changes spec.
ichor CLIBootstraps the first node (ichor init), adopts existing clusters, and does everything the UI does. See the CLI reference.
ichor-uiA Preact web UI with its Go backend. It changes the world only by writing resources, reads node state from the Talos API read-only, and holds no CA key.

Every write, from the UI, the CLI or your own tooling, is a change to a custom resource, so validation, RBAC and audit are the same whichever way it arrived. The design is in the architecture document, and every decision is recorded.

What is in v0.1.0

  • Discovery and approval of machines, with NIC, addressing and disk roles set at approval.
  • Typed machine configuration with drift reporting, and reboot-requiring changes rolled out as one operation.
  • Talos, Kubernetes, Cilium and Ichor upgrades, with the pause on failure and resume model described in Operations. A first install uses Talos v1.14.5, Kubernetes v1.36.6 and Cilium 1.20.4.
  • CA rotation (Talos API, Kubernetes API, and optionally etcd and aggregator) and service-account and secretbox key rotation.
  • etcd backup, restore and full-loss recovery.
  • Optional Ichor-owned users with the roles viewer, operator and admin, and personal kubeconfigs with no secret in them.
  • Admission that enforces the UI's checks on hand-written resources, an admin-only right to change Cilium's values, and an encrypted local secrets bundle. See Security model.
  • Typed, off-by-default hardening for the clusters Ichor creates (SecureBoot, kernel lockdown, kubelet serving certificates).

The complete list is in the changelog.

Where to go next

Next
Contributing to the docs

Search the docs

Search pages and headings