# 0093. The UI hands over a CA rotation's new credentials to administrators who hold the download verb Status: Accepted · Date: 2026-10-27 · Design section: UI backend, Trust, Secrets · Amends: 0091 (Decision 3, "the UI has no download") Decided: `OperationService` gains `DownloadCARotationCredentials` (server-streaming, one zip with `kubeconfig` and `talosconfig`, the files `ichor cluster credentials` writes) and `AcknowledgeCARotationCredentials` (builds the new kubeconfig in memory, checks that it authenticates to the cluster's Kubernetes API, then sets `ichor.maesh.dev/credentials-handoff` with `CARotationHandoffToken`); the download follows ADR 0084's authorization model: the Kubernetes RBAC verb `download` on the subresource `talosclusters/credentials`, checked with a `SelfSubjectAccessReview` as the caller, granted to nobody by default (the chart's `ichorUI.credentials.groups`, or `serviceAccount: true` if the operator insists), a Warning Event `CredentialsDownload` on the TalosCluster before any byte is produced and a refusal if it cannot be recorded; the download is allowed from `AwaitingCredentials` to `Finalizing`, the acknowledgement only in `AwaitingCredentials`; the acknowledgement needs no download and no new RBAC (it is an `update` of the operation, like pausing); the operation page offers "Download new credentials" (only to a caller who holds the verb) and "Continue rotation" (behind a confirmation); the credential builder is the shared package `internal/credentials`, used by the CLI and the UI backend ## Context ADR 0091 made the CLI the only way to leave `AwaitingCredentials`, on the argument that the credentials are cluster-admin private keys and the keys should not travel through ichor-ui or the browser. In practice the operator who starts a rotation from the UI has no `kubectl` at hand, or is not the person who holds the Secret-reading kubeconfig; the rotation waits until somebody opens a terminal. The rotation was only half doable in the UI. The premise "the keys never pass through ichor-ui" is also weaker than it read. The ichor-ui backend already reads the secrets bundle for every Talos connection, and ADR 0084 lets it mint a Talos client certificate per call and hand the result to an administrator, behind a verb nobody has by default and an audit Event. The CA rotation keys are the same class of secret (they are in the same Secret, `rotation.yaml`), the audience is the same (an administrator), and the gate on the transfer is the same mechanism. So the reversal is narrow: it does not add a new place where keys are stored or a broader permission; it adds one more audited, opt-in download. ## Decision **Shared builder.** The kubeconfig and talosconfig are made by `internal/credentials` (`NewSources`, `Load`, `Sources.Kubeconfig`, `Sources.Talosconfig`, `VerifyKubeconfig`), moved out of `internal/cli` without a behaviour change; the CLI keeps what is about files and the operator's own kubeconfig (output paths, `--bundle`, the offline path, `bundle-export` comparison). The UI backend has no kubeconfig to copy a server from, so the Kubernetes server is the cluster's declared endpoint, `https://:6443` (`ServerConfig`, the same address `pkg/render` gives the machines), and the Talos endpoints are the control-plane machines' `DialEndpoint()` as in the CLI. The context, cluster and user names are the cluster's name and `admin`. Content is never logged, put in an error or in an Event; error text is cut at 300 bytes. **Download.** `DownloadCARotationCredentials(operation)` answers one stream of chunks (`filename`, `content_type` and `data`, the file name and type on the first chunk, as ADR 0084; its own response message). The archive is a zip (what the support bundle already is, and what a browser and every OS opens), files `kubeconfig` and `talosconfig` with mode 0600 in the zip header, named `credentials--.zip`. Both trust the old and the new CAs and are signed by the new ones while the rotation is under way, exactly as the CLI writes them. The backend reads the Secret with its own ServiceAccount (it has `get` on Secrets in its namespace already), so the caller needs no Secret access; the caller's permission is the verb alone. **Stage window.** The download is refused unless the operation is a `ca-rotation` at `AwaitingCredentials`, `Issuing`, `Dropping` or `Finalizing`. Not before: until `Accepting` is verified the new keys are not yet trusted by any node, so files made then would not work. Not after `Done`: the rotation is over, the bundle holds one CA, and `ichor cluster credentials` writes the single-CA files from the Secret; the UI does not become a general credentials page. Why past the gate rather than only at it: an operator who continued without downloading (the UI allows it, below) loses the old kubeconfig and talosconfig at `Issuing` and would otherwise be locked out of a rotation they started; the dual-CA files still work through `Dropping`. At `Finalizing` the Secret may already hold the new CAs alone (`rotation.yaml` removed), and the files then trust that one CA, which is what is true then. **Authorization.** The verb is `download`, the resource `talosclusters`, the subresource `credentials`, the name the operation's cluster, in the cluster's namespace. A subresource so that a role with `*` on `talosclusters` (the `taloscluster_*_role.yaml` helpers) does not include it, as for the diagnostics subresources. The check is a `SelfSubjectAccessReview` made as the caller (the impersonated user with OIDC on; the backend's ServiceAccount with OIDC off, and then it holds nothing by default; whoever the kubeconfig is in the CLI's in-process fallback). Chart: a `ichor-credentials-role` ClusterRole (a helper role under `rbac.helpers.enabled`, and rendered when a grant is asked for) and a binding rendered only when `ichorUI.credentials.groups` (OIDC groups, prefix applied) or `ichorUI.credentials.serviceAccount: true` is set; both empty by default, `ichor-ui-role` unchanged. `config/rbac/credentials_role.yaml` is the kustomize copy. It is a separate role from ADR 0084's: whoever may take a support bundle does not thereby get cluster-admin keys. `GetCARotationCredentialsPermission` returns the answer so the UI shows the button only to those who may; the download asks again. **Audit.** Before any key is read, after the stage and the verb are checked, the backend creates a Warning Event with reason `CredentialsDownload` on the TalosCluster, with the backend's own client (users may not create Events, ADR 0065): who, when and the operation's name, no content. If the Event cannot be created the download is refused (`Internal`). A refused request (wrong stage, no verb) leaves no Event. **Acknowledgement.** `AcknowledgeCARotationCredentials(operation)` is allowed only at `AwaitingCredentials`. The backend builds the same kubeconfig in memory, asks the cluster's Kubernetes API who it is (`SelfSubjectReview`, the CLI's check; a seam `Server.VerifyKubeconfig` for tests, 30 s timeout), and only if that succeeds sets the annotation with `CARotationHandoffToken` of the operation's status, as the caller, by an `update` of the TalosOperation (what `SetOperationPaused` needs: `update` on `talosoperations`, which `ichor-ui-role` and an operator who can pause already have; the CLI uses `patch`, the UI does not need it). If the check fails the answer is `FailedPrecondition` saying nothing was set, so a Talos build where an accepted CA does not authenticate clients leaves the rotation waiting with the old credentials intact, as in 0091. The Talos check is not made (the backend's route to the nodes is the controller's, not the operator's; the CLI only warned on it). The response carries an `AuditRecord`, like the other operation writes; no Event is added, since nothing leaves the backend. **Continuing without a download is allowed.** The acknowledgement does not require a prior download, and the server cannot know whether one happened (nothing is stored). An operator who only uses the UI may accept losing their old files, or may fetch the new ones later (the stage window above), or may use the CLI. The UI says it in the confirmation: from `Issuing` the current kubeconfig and talosconfig stop working; the downloaded files work with both CAs and after the old one is dropped. **UI.** On the operation page, inside the `AwaitingCredentials` alert: "Download new credentials" (an `alert-dialog` that says the zip holds cluster-admin private keys and that the download is recorded; rendered only when the permission probe says yes) and "Continue rotation" (an `alert-dialog` that states what stops working; the server's refusal is shown inside the dialog). During `Issuing`, `Dropping` and `Finalizing` a plain note offers the download alone. The file is saved like the diagnostics downloads (the stream is read into memory, `lib/download.ts`). Results show as toasts. The start dialog's text and the awaiting alert describe both ways, the UI and the CLI command; the sentence that credentials are not offered for download is gone, as is the CLI plan's "The UI cannot hand them over". ## Consequences - A default install can download nothing; ADR 0091's safety argument for the CLI-only hand-off now rests on RBAC, as ADR 0084's does. An operator who binds `ichor-credentials-role` (or sets `ichorUI.credentials.serviceAccount`, which lets anyone who can reach ichor-ui get cluster-admin credentials) accepts that the UI backend and the browser carry the key once per download. - The UI backend's reach grows by one capability: it uses the CA rotation private keys (read from the Secret it could already read) to mint a cluster-admin kubeconfig. A compromised backend could do that before this change too (it holds the bundle); the new RPC adds a path, not a privilege. `rbac_enforced_test.go` runs the build, the Event and the acknowledgement as the generated ServiceAccount role and requires the download to be denied. - The Kubernetes server in the downloaded kubeconfig is the declared endpoint, which can differ from the address an operator's own kubeconfig uses (a load balancer name, say); `ichor cluster credentials` copies the operator's own. The backend's acknowledgement check also goes to the declared endpoint from inside the cluster, so a cluster whose endpoint pods cannot reach is acknowledged through the CLI. - Not built: downloading after `Done`, a Talos API check on acknowledgement, a way for the server to know a download happened, a `--skip` of the hand-off, and the CLI streaming through ichor-ui (`ichor cluster credentials` still reads the Secret itself and works offline; the in-process OperationService answers `Unimplemented` to the stream). - Not verified on hardware: nothing here has run against a real rotation; the tests use a fake for the authentication check (a fresh CA is not trusted by the envtest API server), and the first run on a disposable cluster should read its result, as for 0091. Amendment: `GetCARotationCredentialsPermission` is removed; `DiagnosticsService.GetMyPermissions(cluster)` returns the same `download`-on-`talosclusters/credentials` answer as `cluster_credentials`, together with ADR 0084's flags.