# 0094. Ichor users, its own OIDC issuer, roles and per-user kubeconfigs

Status: Accepted · Date: 2026-10-27 · Design section: Trust, Secrets, UI backend, Deployment · Extends: 0058 · Touches: 0084, 0091, 0065 (override)
Decided: Ichor keeps its own users as `IchorUser` CRs (namespace `ichor-system`, spec = display name, e-mail, `roles[]`, `disabled`; never a password) with argon2id hashes in one Secret `ichor-user-credentials`, and ichor-ui becomes a minimal stateless OIDC issuer (hand-written on `go-jose/v4`, RS256, authorization code + PKCE for the browser and a loopback redirect for the CLI, 5 minute ID tokens, 12 hour non-rotating refresh tokens, signing keys kept and rotated by the controller in a Secret); the controller renders both Ichor's issuer (reached by the kube-apiserver through `discoveryURL` on a ClusterIP with an Ichor-made CA) and the existing external IdP as two `jwt[]` entries of Talos's `KubeAuthenticationConfig` document (Talos v1.14 contract and later), usernames `ichor:<name>` and `oidc:<name>`; permissions are plain Kubernetes RBAC: three built-in ClusterRoles `ichor-viewer`, `ichor-operator`, `ichor-admin` bound to the groups `ichor:viewer|operator|admin`, with new verbs for destructive operation starts, user administration and the trust configuration; a user's kubeconfig holds no secret and runs `ichor auth token` as a client-go exec plugin; no user ever receives a talosconfig, and the two paths that hand out Talos credentials (ADR 0091's CLI hand-off, ADR 0093's UI download) stay admin-only; the whole feature is off by default and the secrets-bundle admin certificate stays as override

## Context

ADR 0058 lets ichor-ui verify an ID token from one external IdP and call Kubernetes as that user. It has no users of its own, so a site without an IdP has no way to give two people different rights, and nothing makes the cluster's kube-apiserver accept those people: the user's own `kubectl` still needs a certificate or a token someone minted by hand. The decision asked for: Ichor manages its own users and the external OIDC login keeps working beside them; permissions are roles; each user can get a kubeconfig with no secret in it; users never receive a talosconfig; one cluster only (self-managed, no hub mode).

Everything below was checked against the module cache: Talos `pkg/machinery` v1.14.3, `k8s.io/apiserver` v0.36.0, `k8s.io/client-go` v0.36.0, `github.com/coreos/go-oidc/v3` v3.21.0, `golang.org/x/crypto` v0.58.0. Paths are relative to each module root.

## Decision 1: users

**CR.** `IchorUser` (`ichor.maesh.dev/v1alpha1`, namespaced, only in `ichor-system`, scaffolded with `kubebuilder create api`). The name is the identity: lower-case DNS label, at most 63 characters, immutable, and it becomes `sub` and the Kubernetes user `ichor:<name>`. Spec: `displayName`, `email` (informational), `roles []string` (each `^[a-z][a-z0-9-]{0,30}$`, at most 16), `disabled bool`. Status (written by the controller only): conditions `Ready`, `PasswordSet`. There is no password, hash, token or session field, so a user CR can live in Git next to the other CRs. A CR from Git without a password is a user who cannot log in (`PasswordSet=False`) until an admin sets one; that is the GitOps story, not a gap.

**Hash.** One Secret `ichor-user-credentials` in `ichor-system`, key = user name, value = JSON `{"v":1,"hash":"$argon2id$v=19$m=65536,t=3,p=4$…","notBefore":"RFC3339","mustChange":bool}`. argon2id with the parameters `golang.org/x/crypto/argon2` itself gives as RFC 9106's second recommendation (`argon2/argon2.go:29-33`: time=3, memory=64 MiB, threads=4; `IDKey` at `argon2.go:101`). `golang.org/x/crypto v0.58.0` is already in go.mod, as `// indirect` (go.mod:161); it becomes a direct requirement, nothing new to download. The PHC string carries the parameters, so a login with older parameters rehashes. 64 MiB per hash means ichor-ui admits at most two hashes at once (a semaphore; the rest wait), and a wrong or unknown user costs the same (verify against a dummy hash) so response time does not reveal which users exist. Minimum password length 12, maximum 128 bytes, no composition rules.

**Who writes what.** The UI backend and the CLI write the CR (as the caller, so Kubernetes RBAC decides) and the Secret entry (as ichor-ui's ServiceAccount, after a `SelfSubjectAccessReview` as the caller, ADR 0084's pattern). The controller never writes `spec`; it writes `IchorUser.status`, deletes Secret entries whose CR is gone, and owns the signing Secret (Decision 2). ichor-ui's Role gets `get`, `update`, `patch` on exactly the Secret named `ichor-user-credentials` (resourceNames); it cannot create other Secrets. A Secret entry changes through a JSON merge patch of one key, so two replicas do not overwrite each other.

**Operations.** `UserService` (Connect, `proto/ichor/v1alpha1/user.proto`): `ListUsers`, `GetUser`, `CreateUser`, `UpdateUser` (roles, disabled, display data), `DeleteUser`, `SetPassword` (admin, optional generated password returned once, sets `mustChange`), `ChangeOwnPassword` (old and new password; needs no RBAC, it is the user's own account), `RevokeSessions` (sets `notBefore` to now), `GetMyKubeconfig` (Decision 5), `WhoAmI`. RBAC for the admin ones: `ichorusers` verbs through the impersonated client; `SetPassword` and `RevokeSessions` additionally need verb `update` on subresource `ichorusers/password`. Changing a password sets `notBefore`, which kills every refresh token issued earlier. Disable is `spec.disabled: true`; nothing else is needed. Refusing to disable, delete or demote the last enabled admin is a guard in the RPC, not a security boundary (the override certificate exists).

**First admin.** `ichor init --admin-user NAME` (default `admin`; the flag `--admin-password-file` reads one instead) runs after Ichor is installed. It writes the `IchorUser` (role `admin`) and the Secret entry with `mustChange:true`, using the same package (`pkg/ichorauth`) ichor-ui uses, with the operator's own kubeconfig. A generated password (24 characters, `crypto/rand`) is printed once to stdout and stored nowhere; the first login forces a change. Without `--ui-url` (Decision 3) init creates nothing and users stay off.

**Override.** The secrets-bundle `admin` client certificate (O=`system:masters`, what `ichor init` writes into `~/.ichor/clusters/<name>/kubeconfig`) and the Talos `os:admin` identity are untouched. They do not depend on any JWT authenticator, so a broken issuer, a stopped ichor-ui or a deleted user cannot lock the operator out of Kubernetes, and the controller dials Talos with the bundle, not through Kubernetes. The docs call this the override credential and say to keep it offline once users exist.

## Decision 2: the issuer

**Which component.** ichor-ui, because it already authenticates browsers, is stateless, runs two replicas and has the login UI. It gets a second package `internal/oidcissuer` and two listeners: the existing one (port 8080) gains `/.well-known/openid-configuration`, `/oidc/jwks`, `/oidc/authorize`, `/oidc/login`, `/oidc/token`; a new HTTPS listener (port 8444) serves only the discovery document and the JWKS for the kube-apiserver (Decision 3). The login page is a route of the preactcn SPA (`/login`), not server-rendered HTML.

**Library: hand-written on `github.com/go-jose/go-jose/v4`, not `zitadel/oidc`.** go-jose v4.1.4 is already a direct requirement (go.mod:10) and go-oidc v3.21.0 (client side, unchanged) already uses it. `zitadel/oidc` v3 is the credible OP library, but it is not in go.mod or the module cache, and its OP is built on a `Storage` that persists authorization requests, codes, access tokens and refresh tokens and resolves them by lookup (`CreateAuthRequest`, `AuthRequestByCode`, `SaveAuthCode`, `CreateAccessToken`, `TokenRequestByRefreshToken`, `RevokeToken`, `TerminateSession`; docs: https://github.com/zitadel/oidc/blob/main/_autodocs/api-reference/op-storage.md). Behind two replicas that means a shared store, so every login and every five-minute token refresh would write to etcd, and the stateless backend of ADR 0058 would stop being stateless. The surface needed here is small and fixed (two clients, one flow, one refresh grant), so the hand-written issuer is stateless by design:
- authorization code = a JWS signed by the issuer key (typ `ichor-code+jwt`, 60 s) carrying client_id, redirect_uri, sub, nonce, `code_challenge`, audience; the token endpoint checks PKCE `S256` (`plain` is refused) and the exact redirect URI. Single use is enforced per replica from an in-memory `jti` set only; a replay to the other replica within 60 s still needs the `code_verifier`, which only the client holds. This is a stated weakening of strict OAuth, accepted.
- the authorize request is carried to the login form as a signed 5 minute ticket, so no cookies and no server session exist and a cross-site POST has no ambient authority.
- refresh token = a JWS (typ `ichor-refresh+jwt`) with sub, client, `iat` (session start) and `exp` (absolute lifetime). It is valid while the user exists, is not disabled and `iat >= notBefore`. It is not rotated: nothing has to be stored, and concurrent `kubectl` processes cannot race each other into a revoked token.
- no implicit or hybrid flow, no password grant, no client secret, no dynamic registration, no userinfo, introspection, revocation or end-session endpoint (logout discards the tokens; "sign out everywhere" is `RevokeSessions`).
- the risk of hand-writing a security component is covered by tests that run the **real** consumers against it: go-oidc's `NewProvider` and `Verify`, and `k8s.io/apiserver/plugin/pkg/authenticator/token/oidc.New` (the code the kube-apiserver runs; `oidc.go:269`) in an `httptest` TLS server, plus negative tests (wrong audience, expired, tampered, replayed code, wrong PKCE verifier, unknown redirect URI, disabled user, `iat < notBefore`).

**Clients (fixed, configured in the chart, not CRs).** `ichor-ui`: public, redirect URI exactly `<externalURL>/auth/callback`. `ichor-cli`: public, redirect `http://127.0.0.1:<any port>/callback` (RFC 8252 section 7.3: the server MUST allow any port for loopback redirects; the literal `127.0.0.1`, not `localhost`, section 8.3). Both use `response_type=code`, `S256`, `state`, `nonce`, scope `openid` (+ `offline_access` for a refresh token).

**Audiences.** The ID token's `aud` is chosen by the request: `ichor-ui` for the SPA, and for the CLI either `ichor-kube` (the one the kube-apiserver accepts) or `ichor-ui`, asked for by an `audience` parameter on the refresh-token request. The kube-apiserver trusts only `ichor-kube`, so the token that sits in the browser's `sessionStorage` (ADR 0058 point 6) is not a Kubernetes credential, which keeps ADR 0058's statement true.

**Claims and prefixes.** `iss` = the external URL, `sub` = user name, `aud`, `exp`, `iat`, `groups` = the user's `roles` (for example `["admin"]`). The kube-apiserver maps `sub` with prefix `ichor:` and `groups` with prefix `ichor:`, giving user `ichor:alice` and group `ichor:admin`. An external IdP is mapped with prefix `oidc:` (ADR 0058's default), so the two cannot collide, and `system:` is blocked by a user validation rule on both. The webhook refuses an external prefix that equals `ichor:`, starts with it, or is empty.

**TTLs.** ID token 5 minutes (chart `ichorUI.users.tokenTTL`, 1 to 15 min). Refresh token absolute lifetime 12 hours (`refreshLifetime`, 1 hour to 30 days). The kube-apiserver does not check anything against Ichor after a token is issued, so the token TTL is the worst-case delay for "disable".

**Disable, role change, password change.** ichor-ui re-reads the `IchorUser` for every RPC (an in-memory cache of at most 10 s), refuses a disabled user, and impersonates with the roles *from the CR*, not from the token. So in the UI a disable or a role change takes effect within 10 s. For `kubectl` it takes effect at the next token refresh, at most 5 minutes later, because the token's `groups` claim is old until then and the refresh endpoint re-reads the CR and `notBefore`. That is "promptly" and is stated as a number rather than "immediately".

**Signing keys.** RSA-3072, RS256 only. RS256 is the one algorithm OIDC requires and the kube-apiserver authenticator defaults to (`k8s.io/apiserver/plugin/pkg/authenticator/token/oidc/oidc.go:281-286`); whether structured configuration widens that list is a claim in Talos's own doc comment (`config/types/k8s/authentication.go:59-61`) that was not verified in kube-apiserver source, so nothing here depends on it. Keys live in the Secret `ichor-oidc-signing` (`keys.json`: kid, PEM, `createdAt`, `activeFrom`, `retireAfter`). The **controller** (leader-elected, so one writer) creates the first key, adds a new key every 30 days, makes it the signing key 10 minutes after it is published (covering the kubelet's Secret-volume sync lag in ichor-ui), and drops a key 2 hours after the last token signed with it can have expired. ichor-ui mounts the Secret read-only and reloads on change; it never writes keys. The kube-apiserver does not need to be told about a rotation: go-oidc's key set caches keys in memory and re-fetches the JWKS when it sees an unknown `kid` (`oidc/jwks.go:57-58, 168-174`), so no machine-config change and no restart follow a rotation.

## Decision 3: making the kube-apiserver trust both issuers

**What Talos can carry (verified).** In the v1.14 contract Talos has a typed document `KubeAuthenticationConfig` (`config/types/k8s/authentication.go`) whose `configuration` is the literal Kubernetes `AuthenticationConfiguration`, i.e. a list `jwt[]` of authenticators, each with its own issuer. Its doc comment says kube-apiserver refuses any `--oidc-*` flag when `--authentication-config` is set "and Talos always sets it", and `KubeAPIServerConfig.extraArgs` rejects `oidc-*`, `anonymous-auth` and `authentication-config` (`config/types/k8s/apiserver.go:178-180`). Talos's generator adds the default document with an empty `jwt: []` and anonymous access limited to `/livez`, `/readyz`, `/healthz` (`config/generate/kubernetes.go:120`, `authentication.go:84-107`). It is used only with the multi-document form: the legacy v1alpha1 shim returns `UseAuthenticationConfig() == false` (`config/types/v1alpha1/v1alpha1_k8s_bridge.go:141-144`), the modern document returns true (`apiserver.go:293-296`), and multi-document Kubernetes configuration is generated when the contract is greater than 1.13 (`config/contract.go:282-285`). So:
- **contract v1.14 and later**: multiple issuers, typed. This is the only supported mode for Ichor's issuer.
- **earlier contracts**: only the legacy `--oidc-*` flag set, i.e. exactly one issuer, no `discoveryURL`, no per-issuer CA. Ichor's issuer cannot be used there (the apiserver would need a CA file and a reachable external URL), and a cluster below v1.14 keeps ADR 0058 as it is. The webhook refuses `spec.authentication` unless `versionContract >= v1.14`.

**How Ichor stays typed.** `KubeAuthenticationConfig` is not on the mirror generator's kind allow-list (`internal/tools/mirrorgen/main.go`, `roots`) and its field is a `meta.Unstructured`, so no `TalosConfigPatch` can set it (ADR 0003). That is right: the document is owned by the renderer, as the control-plane metrics documents are (`pkg/render/controlplanemetrics.go`). The renderer builds `k8s.io/apiserver/pkg/apis/apiserver/v1.AuthenticationConfiguration` (a typed Go struct, `types.go:56`, already a dependency through the ADR 0003 overrides), validates it with the apiserver's own `validation.ValidateAuthenticationConfiguration` (`pkg/apis/apiserver/validation/validation.go:47`) before it ever reaches a node (an invalid file makes kube-apiserver fail to start, and this is the single document whose breakage affects every control plane), converts it through the existing `mirrorToMachineryUnstructured` helper and applies it as a strategic merge patch. The field is `merge:"replace"` (`authentication.go:70`), so the rendered `configuration` must contain the default `anonymous` block as well, or health probes lose anonymous access. No raw YAML is accepted anywhere.

**CR.** `TalosCluster.spec.authentication` (immutable-name fields, CEL-validated; changes to it need `update` on `talosclusters/authentication`, Decision 4):
- `ichor`: `{enabled, url}` — `url` is the external https URL of ichor-ui and is the issuer (`iss`).
- `external[]`: `{issuerURL, audiences[], usernameClaim (email), groupsClaim (groups), usernamePrefix ("oidc:"), groupsPrefix ("oidc:"), certificateAuthority (PEM, public) }` — at most 3. The ichor-ui `ichorUI.oidc.*` values (ADR 0058) are what the UI uses; this list is what the apiserver uses; `ichor init` and the chart docs say to keep the two equal, and the UI shows a warning when they differ.
Each entry becomes one `jwt[]` element: issuer URL, audiences, claim mappings with the prefixes above, `userValidationRules: [!user.username.startsWith('system:')]` and `!user.groups.exists(g, g.startsWith('system:'))`. Issuer URLs must be unique across entries (`validation.go:61-70`).

**Reaching the issuer from a host-network kube-apiserver.** The apiserver fetches discovery and JWKS from the issuer URL, but the spec allows `discoveryURL` for "the well-known and jwks endpoints hosted at a different location than the issuer (such as locally in the cluster)", with `certificateAuthority` to verify that connection, and requires the document's `issuer` to equal `url` (`apiserver/v1/types.go:139-166`; `discoveryURL` must be https and different from `url`, `validation.go:149-154, 170`). Ichor uses it:
- `url` = `https://ichor.example.com` (external, what browsers and the CLI use, and the `iss` of every token).
- `discoveryURL` = `https://<ClusterIP of ichor-ui-service>:8444/.well-known/openid-configuration`, served by the internal listener, whose discovery document says `issuer` = the external URL and `jwks_uri` = the internal address. The cluster does not need the external URL to be reachable from the control planes, nor DNS, nor a load balancer that itself runs on the cluster it authenticates.
- `certificateAuthority` = an Ichor-made CA (RSA, 10 years) kept by the controller in `ichor-oidc-ca` with a leaf (1 year, renewed at two thirds) in `ichor-oidc-tls`, SANs: the ClusterIP and `ichor-ui-service.ichor-system.svc`. The controller publishes the public parts (`discoveryURL`, the CA PEM) in the ConfigMap `ichor-oidc-trust`, which is a render input (ADR 0061 watches render inputs), so a changed CA or ClusterIP is a desired-state change like any other. A leaf renewal touches nothing in machine config. It is not the Kubernetes or Talos CA, so ADR 0063/0091 rotations do not interact with it.
- `egressSelectorType` stays unset (no custom dialer; `types.go:190-200`).
That a control plane's host network can reach a Service ClusterIP is not verified (see "Not verified").

**Bootstrap and when ichor-ui is down.** The authenticator is built per entry asynchronously and polls every 10 s until discovery works; "this enables self-hosted providers, providers that run on top of Kubernetes itself" (`oidc.go:423-430`). The apiserver starts without the issuer, and until discovery succeeds, tokens of that issuer are rejected (`oidc.go:412`, `867`) while certificate authentication is unaffected. Once running, keys are cached in memory and only re-fetched on an unknown `kid` (`jwks.go:57-58`), so with ichor-ui down, already-issued tokens keep working until they expire (5 minutes) and nothing new can be issued. After an apiserver restart with ichor-ui still down, Ichor-issuer tokens fail until ichor-ui is back, and the external IdP's entries are independent. First boot has no `jwt` entries: `ichor init` creates the cluster, installs Ichor, creates the admin, then sets `spec.authentication` (Decision 7).

**Rollout.** The rendered document changes for every control plane. `pkg/fieldmeta` has no rule for `KubeAuthenticationConfig`, so the default applies: reboot-required, and it is rolled out by the `config` operation (ADR 0080), one control plane at a time with etcd checks. Whether Talos v1.14.3 applies this document live (and so a rule `{prefix: "KubeAuthenticationConfig", safe: true}` like the one for `KubeAPIServerConfig`, `fieldmeta.go:212`) was not checked, because Talos controller source is not in `pkg/machinery`; the first implementation unit must read it and add the rule only if it is true. The kube-apiserver is assumed to restart on a change; the reload behaviour of `--authentication-config` is in `k8s.io/kubernetes`, not in the module cache, and was not verified.
**Adopt.** A cluster adopted with non-empty `jwt` in its `KubeAuthenticationConfig` would have those entries replaced by the render. ADR 0075's adopt names the document as unsupported today; it must refuse (not drop) a non-empty `jwt`, and say to move the IdP into `spec.authentication.external`.

## Decision 4: roles

No new binding CR. Roles are Kubernetes RBAC objects, so they are the same for the UI (impersonation, ADR 0058), for `kubectl` with the user's kubeconfig, and for a Git-managed setup. Three ClusterRoles ship in the chart and kustomize (`config/rbac/ichor_*_role.yaml`), each bound with a ClusterRoleBinding to the group of the same name (`ichor-viewer` to Group `ichor:viewer`, and so on); `IchorUser.spec.roles` names the groups a user is in, so a custom role is a ClusterRole plus a binding to `ichor:<name>`, and a one-off grant is a binding to User `ichor:<name>`. Role names `viewer`, `operator`, `admin` are reserved. ClusterRoleBindings, because the product uses one namespace and the roles also need `nodes` and `pods`; a site that wants less writes its own namespaced binding.

| Role | Gets |
|---|---|
| `ichor-viewer` | `get list watch` on `talosclusters talosmachines talosconfigpatches talosoperations talosstorages talosetcdbackups ichorinstallations`; core `nodes pods` `get list` (ADR 0082); in `ichor-system`: `services` and `pods` `get list`, `pods/portforward create` (for `ichor ui`). No `secrets`, no `events`, no `ichorusers`. |
| `ichor-operator` | viewer, plus `create update patch delete` on `talosconfigpatches`; `update patch` on `talosmachines` (approve, reject, node settings) and on `talosclusters` and `ichorinstallations` (versions, settings; not `spec.authentication`); `create update patch delete` on `talosoperations`. Because `create` on `talosoperations` cannot say which kind, the destructive kinds are gated separately (below). |
| `ichor-admin` | operator, plus `delete` on `talosmachines` and `talosclusters`; all verbs on `ichorusers`; verb `update` on `ichorusers/password` and `talosclusters/authentication`; verb `start` on `talosoperations/shutdown`, `/ca-rotation`, `/etcd-restore`, `/node-reset`, `/node-shutdown`; verb `override` on `talosclusters talosconfigpatches talosmachines` (ADR 0065); verb `download` on `talosmachines/support-bundle`, `/etcd-snapshot`, `/packet-capture` (ADR 0084); verb `download` on `talosclusters/credentials` (the name ADR 0093 chose replaces this one; the rule below is what binds). |

**Destructive starts (new).** A `TalosOperation` of kind `shutdown`, `ca-rotation`, `etcd-restore` or `node` with action `reset` or `shutdown` needs verb `start` on the subresource of that name, asked with a `SelfSubjectAccessReview` by `OperationService.CreateOperation` as the caller and by the validating webhook (a `SubjectAccessReview` on the request's user, the way the override webhook already gates a reset and an etcd restore, `internal/webhook/v1alpha1/override_webhook.go`) so a hand-written CR obeys it too. The controller's own identity is exempt, as in the override webhook. The override requirements of ADR 0065/0089 stay on top. The same webhook pattern gates edits to `spec.authentication` (`update` on `talosclusters/authentication`), because an operator who could repoint the IdP could break sign-in for everyone.

**Not in any built-in role:** `secrets` (so nobody gets the secrets bundle or the credentials Secret through Ichor's roles), `impersonate`, `*`. `cluster-admin` is **not** part of `ichor-admin`; a site that wants its Ichor admins to run arbitrary `kubectl` binds `cluster-admin` to Group `ichor:admin` itself (chart value `ichorUI.users.clusterAdminRoles: []`, empty by default). Doing so also lets them read the secrets bundle, which is the source of Talos credentials, so the statement "users never get Talos access" in Decision 6 holds for the built-in roles only.

**Enforcement.** Unchanged from ADR 0058, extended: the Authenticator accepts tokens from the local issuer (verified in process against its own keys, `aud=ichor-ui`, `iss` = the configured URL) and from the external one (go-oidc); it routes by the unverified `iss`, and each path then verifies fully. Both produce `Identity{User, Groups}` and the same impersonating client. The `impersonate` grant is rendered when either source is on; `groups` narrow to the built-in group names plus configured ones when only Ichor users are on.

## Decision 5: kubeconfig issuance

**No secret in the file.** The kubeconfig has the API server URL, the cluster CA (public), and a user with an `exec` block: `apiVersion: client.authentication.k8s.io/v1` (in client-go's accepted list, `plugin/pkg/client/auth/exec/exec.go:76-79`), `command: ichor`, `args: [auth, token, --issuer, <url>, --client-id, ichor-cli, --audience, ichor-kube]`, `interactiveMode: IfAvailable` (required for the v1 API; `tools/clientcmd/api/types.go:260-270`), `provideClusterInfo: false`. The plugin prints an `ExecCredential` whose `status.token` is the ID token and `status.expirationTimestamp` is its `exp`; client-go caches it until then (`exec.go:386-390`, `496-497`) and drops it on a 401 (`exec.go:378`).

**`ichor auth token`.** (1) a cached ID token with 30 s left: print it. (2) a cached refresh token: call the token endpoint, store the new ID token, print. (3) otherwise log in: listen on `127.0.0.1:<random>`, open the authorize URL (`S256` PKCE, `state`, `nonce`, `offline_access`), exchange the code, store, print. Diagnostics go to stderr; stdout carries only the ExecCredential. A file lock on the cache serialises concurrent `kubectl` runs. Cache: `~/.ichor/auth/<sha256(issuer,client)>.json`, directory 0700, file 0600, written atomically by `bundlefile.go`'s writer; it holds a refresh token, so it is a credential at rest on that user's machine, as any CLI login cache is, and the kubeconfig itself stays secret-free. `ichor auth login`, `ichor auth logout` (deletes the cache entry) and `ichor auth status` complete the set. Not built: device authorization grant, so a headless box without a browser cannot log in (it can copy a refresh cache from another machine); the password is never handled by the CLI, only by the browser's login form.

**Getting the file.** `ichor auth kubeconfig [--issuer URL] [-o FILE]`: logs in if needed, calls `UserService.GetMyKubeconfig` on ichor-ui (authenticated with the `aud=ichor-ui` token), which fills the server URL from `TalosCluster.spec.endpoint` and the cluster CA from the bundle (public certificate only) and returns the YAML; the UI has "Download my kubeconfig" on the account menu calling the same RPC. It requires `ichor-viewer` or better, and never contains anything but public data. After a Kubernetes CA rotation (ADR 0091) an old file has the old CA and fails TLS until it is fetched again; the rotation plan text says so.

**CLI to ichor-ui.** With the Ichor issuer the CLI can now present a token to the backend (`ichor auth token --audience ichor-ui`), which is the "flag that would carry an ID token" ADR 0058 point 5a left for later. It does this only when the current kubeconfig user is an Ichor exec-plugin user; it still never forwards kubeconfig credentials, and an external-IdP user keeps the in-process fallback.

## Decision 6: Talos

Users never get a talosconfig, and nothing in this ADR issues one: no per-user Talos identity exists, so there is nothing to revoke when a user is disabled. Every Talos action stays a CR written by someone with the role for it and carried out by the controller, or one of ADR 0084's three read-only downloads, which the backend performs with a per-call certificate after Kubernetes RBAC allows the signed-in user. The remaining paths that put Talos credentials in someone's hands:
- `ichor init` writes the operator's kubeconfig and talosconfig locally and `~/.ichor/clusters/<name>/` holds the bundle (the override credentials; unchanged).
- `ichor cluster credentials`, `bundle-export` and `rotate-ca` (ADR 0091) read the bundle Secret with the operator's own kubeconfig. They stay admin-only without a new gate because no built-in role can `get secrets`; only override or `cluster-admin` holders can run them.
- The UI download ADR 0093 builds is new surface: it hands an `os:admin` identity to a browser. This ADR requires of it: verb `download` on a `talosclusters` subresource (placeholder `credentials`) held by `ichor-admin` only; a Warning Event written before any byte is read and a refusal if the Event cannot be written (ADR 0084's rule); a short-lived certificate (24 hours) rather than ADR 0091's one year; and a confirm dialog that says the file is the cluster's root identity and is not attributable to a person. Where ADR 0093 differs, the differing point is for that ADR to change or to supersede this paragraph.
The CLI's and the backend's Talos code paths are unchanged.

## Decision 7: ADR 0058's flags, migration, defaults

`ichorUI.oidc.*` and the `--oidc-*` flags keep their meaning and defaults; an install that uses neither feature is byte-for-byte as before apart from the new CRD (`ichorusers`). New values: `ichorUI.users.enabled` (default **false**), `.externalURL` (required when enabled, https), `.tokenTTL`, `.refreshLifetime`, `.clusterAdminRoles`, and `ichorUI.service.type` (ClusterIP by default; exposing ichor-ui over https is the operator's job through an Ingress, Gateway or LoadBalancer, or `ichorUI.tls.secretName` to terminate TLS in ichor-ui; Ichor installs no ingress controller). Built-in users are off by default because the feature needs three things a default install does not have: an https URL reachable by browsers and CLIs, a Talos v1.14 contract, and a config rollout on the control planes. `ichor init --ui-url https://…` turns it on: it sets `ichorUI.users.*`, creates the first admin, sets `spec.authentication.ichor` and prints that `ichor config apply` rolls the control planes. With both `ichorUI.oidc.enabled` and `users.enabled`, `/auth/config` returns `providers: [{id,issuer,clientId}]` and keeps the old `enabled/issuer/clientId` keys, so an older CLI (`oidcEnabled`, `internal/cli/api.go:379`) still sees "enabled" and the SPA shows both sign-in choices. With neither, ADR 0058 point 4's warning and banner remain. Upgrading from an OIDC-off install changes nothing until `users.enabled` is set; turning it on makes every RPC require a token, as turning on OIDC does, so the install notes say to create the first admin before the switch.

## Not built

Hub mode or several clusters; a device-code login; a user directory (LDAP), SSO session cookie, MFA, password reset by e-mail, lockout state shared between replicas (failed-login back-off is per replica and in memory: five failures per user and per source IP back off exponentially, so the effective limit is twice that with two replicas); per-token revocation; a refresh-token rotation; group nesting; an Ichor-run IdP for other applications; Ichor installing an ingress; per-user Talos identities; hardware-checked behaviour (below).

- (Built; see "Amendment: the apiserver restart rolls out one control plane at a time" below.) An apiserver-restart rollout for `KubeAuthenticationConfig`.

## Not verified

- (Resolved by the hardware amendment below.) That a host-network kube-apiserver on a Talos control plane reaches a Service ClusterIP (kube-proxy or Cilium on the node). The design relies on it for `discoveryURL`; the first e2e must read the authenticator's state, and if it fails the fall-back is the external URL for discovery too.
- (Resolved by the hardware amendment below.) Whether a stopped ichor-ui (authenticator uninitialised, `oidc.go:412`) can ever make the kube-apiserver's `/readyz` fail. `HealthCheck()` exists on the authenticator (`oidc.go:974`) but its caller is in `k8s.io/kubernetes`, which is not in the module cache. If it does, the load balancer pulls the control plane out when ichor-ui is down; the e2e must stop ichor-ui and read `/readyz`.
- Whether kube-apiserver accepts algorithms other than RS256 under structured configuration (irrelevant here; RS256 is used).
- ADR 0093's names and certificate lifetime (Decision 6); it was not on this branch.

## Implementation units

Each is one PR, with the listed tests, and scaffolded with the tools named in CLAUDE.md. Order: 1 first; 2, 3, 6 and 9 after 1 and in parallel; 4 after 2 and 3; 5 after 1 and 3; 7 after 4; 8 after 4 and 5; 10 after 7 and 8; 11 last.

1. **`pkg/ichorauth` and the `IchorUser` API.** `kubebuilder create api` for `IchorUser` (`api/v1alpha1/ichoruser_types.go`, CEL for names and roles, samples, CRD in `config/crd` and the chart); `pkg/ichorauth` (argon2id hash/verify with a dummy-hash path and rehash, PHC encode/decode, credentials-Secret entry type, password policy, token and ticket claim structs). Tests: hash round-trip, parameter upgrade, constant-time shape, envtest for the CRD validation.
2. **Issuer core** `internal/oidcissuer`: discovery (external and internal variants), JWKS, authorize/login ticket, code, token (code and refresh grants), key loading and rotation-aware signing, per-replica code jti set and login back-off. Tests: the go-oidc client and the real `k8s.io/apiserver/.../oidc.New` authenticator against an `httptest` TLS server, plus the negative list in Decision 2. Depends on 1.
3. **Controller: signing keys and internal TLS** (`internal/controller/issuer_trust.go`): Secrets `ichor-oidc-signing`, `ichor-oidc-ca`, `ichor-oidc-tls`, ConfigMap `ichor-oidc-trust`, rotation timeline, `IchorUser.status`, credentials-Secret garbage collection, RBAC markers and chart Roles (resourceNames), metrics (issuer certificate expiry is exported as a metric only; no TalosCluster condition, because both certificates renew themselves). Tests: envtest reconcile, fake clock for rotation and renewal, ClusterIP change reissues the leaf.
4. **ichor-ui integration**: new listeners and flags, `Authenticator` for two sources, 10 s `IchorUser` cache, `UserService` (RBAC-checked with `SelfSubjectAccessReview`, Secret writes through the SA), `/auth/config` providers, chart values and `ichor-ui` role changes (`ichorusers` get, the credentials Secret by name, Secret volumes). Extends `rbac_enforced_test.go` so the generated role is the only thing under test. Depends on 2, 3.
5. **Render and webhook** (`pkg/render/authentication.go`, `api/v1alpha1/taloscluster_types.go`, `internal/webhook`): `spec.authentication`, the typed `AuthenticationConfiguration` builder, `ValidateAuthenticationConfiguration`, the contract gate, the prefix rules, `update` on `talosclusters/authentication`, the `start` check on `talosoperations/*` in the webhook and in `OperationService.CreateOperation`, `fieldmeta` classification (after reading Talos source), adopt refusing non-empty `jwt`. Tests: render golden files per contract, envtest webhook with real roles (the shape of `override_reset_test.go`), fieldmeta rule test. Depends on 1, 3.
6. **Roles**: `config/rbac/ichor_{viewer,operator,admin}_role.yaml`, chart templates, bindings to the three groups, `ichorUI.users.clusterAdminRoles`; `dist/chart/chart_test.go` pins that no role contains `secrets`, `impersonate` or `*`, and that the defaults render nothing. Independent of the rest except the names in Decision 4.
7. **CLI** `ichor auth {token,login,logout,status,kubeconfig}` and `ichor user {list,create,update,delete,set-password,revoke-sessions}`: loopback + PKCE, cache and lock, ExecCredential output, bearer to ichor-ui when the user is an exec-plugin user (`internal/cli/api.go`). Tests: fake issuer, golden ExecCredential, 0600/0700 checks, no token in logs. Depends on 4.
8. **UI**: login route, account menu (change password, download kubeconfig), Users page (list, create, roles, disable, reset password), both sign-in choices from `/auth/config`; preactcn components only, scaffolded with `npx shadcn@latest add @preactcn/…`; English and Japanese strings (ADR 0064). Tests: vitest for the PKCE helper and the login state machine. Depends on 4, 5 for the final shape.
9. **`ichor init` bootstrap** (`internal/cli/init_ichor.go`): `--ui-url`, `--admin-user`, `--admin-password-file`, first admin with `mustChange`, `spec.authentication.ichor`. Tests: `init_order_test.go`-style ordering, the password is never written to a file or a log (`init_leak_test.go` pattern). Depends on 1 and the CR in 5.
10. **E2E**: Talos v1.14 cluster, users on, log in with the CLI, `kubectl` as an operator and as a viewer, disable and see the 5 minute bound, stop ichor-ui and check `/readyz` and cached-key behaviour, restart an apiserver with ichor-ui down, rotate a signing key without restart, a rollout through `ichor config apply`. This is where the "Not verified" items are decided. Depends on 7, 8.
11. **Docs**: architecture.md "Trust" (replace "OIDC login... comes later" with this), "Deployment", the Amendment in ADR 0058 (done with this ADR), the user guide for exposing ichor-ui over https.

## Consequences

- A compromised ichor-ui pod can now read password hashes and the signing keys and so mint a token for any user, including an `ichor:admin` that the apiserver accepts. ADR 0058 already gives that pod `impersonate` on any non-`system:` user, so the class of risk is the same, but the pod is now also the root of a login system. Two replicas, `resourceNames` on every Secret rule, no `secrets` list, and short tokens bound it; they do not remove it.
- The issuer needs an https URL for browsers and CLIs that Ichor does not provide. A lab without one stays on ADR 0058 or on certificates.
- The controller gains Secret writes in `ichor-system` for four named objects and a reconciler; the Talos config for control planes gains one more rendered document, and every change to it is a control-plane rollout.
- The `ichor-operator` role can start the routine kinds of `TalosOperation` (upgrades, config, CNI, reboot, service restart) but not shutdown, CA rotation, restore, reset: those are separate, named, auditable grants.
- Kubeconfigs downloaded before a Kubernetes CA rotation must be fetched again.

## Amendments

- Unit 2: the `audience` parameter (only `ichor-kube` or `ichor-ui`; `ichor-ui` client only `ichor-ui`) is accepted on the authorize request and on the refresh request. It is carried in the code and in the refresh token, so a refresh without the parameter keeps the audience chosen at login; absent everywhere, `ichor-ui` client gets `ichor-ui` and `ichor-cli` gets `ichor-kube`. A refresh grant returns no new refresh token (not rotated).
- Unit 2: the kube-apiserver authenticator uses go-oidc v2.5.0, not v3 as the ADR cites for rotation: it refetches the JWKS on an unknown `kid` only after the cache expiry given by the response headers. Discovery and JWKS are therefore served with `Cache-Control: no-cache` (no positive max-age), which makes rotation pickup immediate; a test pins it.
- Unit 2/3 integration: the issuer reads the controller's `keys.json` (`{"v":1,"keys":[...]}`, `internal/oidctrust`) through the shared `oidctrust.ParseKeys`, `Signing` and `Published`; it has no format of its own. Keys are PKCS#8 only. The retire boundary is the controller's: a key is dropped at `retireAfter` exactly, not after it. A test feeds the issuer a document made by the controller's own rotation code.

- Unit 7: the `ichor user` commands and `ichor auth status|kubeconfig` do not use the port-forward or the in-process fallback of ADR 0058/0047. With users on, ichor-ui refuses an unauthenticated tunnel, and `ichor auth kubeconfig` has to work for a person with no cluster access yet. They call `UserService` on ichor-ui at `--issuer` (the external URL is ichor-ui's own URL) over https with a bearer ID token for audience `ichor-ui` from the login cache, and refuse plain http except to the loopback. `-o FILE` of the ADR text is `--out FILE` (`-o` is the global output format). The `--audience` default of `ichor auth login|token` is `ichor-kube`.
- Unit 7: `--ca-file` (env `ICHOR_CA_FILE`) adds a PEM CA to the system roots for the issuer and ichor-ui connections. `GetMyKubeconfigRequest.ca_file` makes the server add `--ca-file <path>` to the exec args; unset adds nothing.
- Unit 9: `ichor init` installs from the embedded kustomize manifest, not the chart, so `--ui-url` edits the ichor-ui Deployment (flags, internal port, optional signing and tls volumes) and Service in place and creates the empty `ichor-user-credentials` Secret, in the order Secret, first admin, switch, then sets `spec.authentication.ichor` by merge patch once the cluster exists. `--admin-password-file` is not built; the password is generated and printed once to stderr, also when a later step fails.
- Unit 9: `ichor user create NAME --role admin --generate --bootstrap` is the first admin on an existing install: the caller's own kubeconfig writes the `IchorUser` and the Secret entry (merge patch of one key, hashing with `pkg/ichorauth`). It refuses an existing user and a missing Secret.

## Amendment: Hardware results

A spike on real hardware (Talos v1.14.3, Kubernetes v1.37.1, one control plane; the change was `talosctl patch machineconfig --mode=auto`, with nginx as the issuer on ClusterIP 10.251.0.50:8443) settled the two "Not verified" items about Decision 3 and replaces the reboot-required default of its "Rollout" paragraph.

1. **Applying `KubeAuthenticationConfig` needs no reboot, but restarts the kube-apiserver.** `--mode=auto` answered "Applied configuration without a reboot" and the boot ID stayed the same. Talos re-renders the kube-apiserver static pod and the apiserver restarts on every change; with a single control plane `/readyz` was down for 15 to 45 s per change. `pkg/fieldmeta` therefore has `{prefix: "KubeAuthenticationConfig", safe: true}`. The intended rollout is an apiserver restart, not a reboot: one control plane at a time, wait for that node's `/readyz` before the next, no drain. The machine controller today applies every `safe` change to every machine at once, and the `config` operation (ADR 0080) only rolls reboot-required changes, so the ordering and the `/readyz` wait were not built at this point; the amendment below builds them, and `KubeAuthenticationConfig` is no longer `safe` but the new `APIServerRestart` class.
2. **Talos does not validate the document.** An invalid one was accepted ("applied without a reboot") and then crash-looped kube-apiserver for about 6 minutes, which also put kube-controller-manager into CrashLoopBackOff. The trigger was two audiences without `audienceMatchPolicy`: `jwt[0].issuer.audienceMatchPolicy: Invalid value: "": audienceMatchPolicy must be MatchAny for multiple audiences`. So `validation.ValidateAuthenticationConfiguration` is mandatory before rendering and any error stops the render, and the renderer always sets `audienceMatchPolicy: MatchAny` when an issuer has more than one audience.
3. **Verified, no change needed.** A ClusterIP plus `discoveryURL` plus an Ichor CA with an IP SAN works from the host-network kube-apiserver; `/readyz` stays ok while the issuer is down, before and after an apiserver restart; two `jwt[]` issuers work together; an unknown `kid` makes the apiserver re-fetch the JWKS at once (about 3 s).

## Amendment: the apiserver restart rolls out one control plane at a time

Replaces the "Not built" line about the apiserver-restart rollout. `pkg/fieldmeta` has a third class beside safe and reboot-required, `APIServerRestart` (ADR 0080's amendment has the whole design and the audit of which fields belong to it). `KubeAuthenticationConfig` is in it, so a change to `spec.authentication` is no longer applied by the machine controller to every control plane at once: the machines report `Ready=False/APIServerRestartRequired` and `ichor config apply` (or the cluster page) rolls it, one control plane at a time with no drain and no reboot, waiting for that node's kube-apiserver `/readyz` before the next. `ichor init --ui-url` prints that command, as before. The plan says "Restarts the Kubernetes API server on each control plane, one at a time; no reboot." On a cluster with one control plane the plan also warns that the API is unavailable while it restarts. Not verified on hardware.

## Amendment: must-change is enforced server-side; the kubeconfig server is configurable

Hardware testing found that `mustChange` only stopped the SPA. Now `AuthInterceptor` refuses every RPC except `WhoAmI` and `ChangeOwnPassword` with PermissionDenied "change your password first" for an Ichor user whose credentials entry has `mustChange` (read through `UserCache.MustChange`, cached at most 10 s like the IchorUser, and dropped on this replica when the password is written; a read failure refuses the token). The issuer, via `oidcissuer.User.MustChange`, issues no `ichor-kube` token (code or refresh) to such a user and says why; `ichor-ui` tokens still work, so the user can change the password. `GetMyKubeconfig` used the cluster `spec.endpoint`, which is a node address, not what users reach through a load balancer: `ichorUI.users.kubeAPIServerURL` (`--kube-api-server-url`) replaces the server, and `ichorUI.users.kubeAPITLSServerName` (`--kube-api-tls-server-name`) sets `tls-server-name` when that host is not in the apiserver certificate SANs. Both are empty by default, which keeps the old behaviour.


Amendment (names): object names differ by install path (kustomize `namePrefix: ichor-` gives `ichor-ichor-ui` and `ichor-ichor-ui-service`), so `ichor init` and the controller find the ichor-ui Deployment and Service by the label `app.kubernetes.io/component=ui` (`--ui-service-name` stays as an override), init also adds `--user-issuer` and `--oidc-refresh-lifetime` to the controller before creating the admin, and gives ichor-ui `--internal-jwks-uri` from the Service ClusterIP instead of the kubelet `*_SERVICE_HOST` variable, whose name follows the Service name.

## Amendment: accepted after the hardware run

Amendment (2026-10-28): verified end to end on hardware; `ichor init --ui-url` works as shipped, and must-change, roles, disable, issuer-down and the one-at-a-time apiserver rollout were each checked, so Status is Accepted.

Amendment (CLI bearer): when ichor-ui's `/auth/config` says auth is required, the CLI sends `Authorization: Bearer <ID token>` on its Connect calls through the port-forward, using the cached `ichor-ui`-audience login for one of the advertised issuers (refreshed with the cached refresh token when near expiry, never printed); with none cached it keeps the in-process fallback and prints `ichor auth login --issuer <url> --audience ichor-ui`. A TLS verification failure now says so in the fallback line. `ichor auth logout` removes every cached audience for the issuer unless `--audience` is given.

Amendment (events): the operation page's activity log lists Events as the signed-in user, so the namespaced `ichor-user-access` Role (not the cluster-wide roles) also grants `get list watch` on `events` (core and `events.k8s.io`) in `ichor-system`; "no `events`" in the viewer row means no cluster-wide events.

Amendment (ClusterIP changes): a deleted and recreated ichor-ui Service gets a new ClusterIP. Everything on the controller side already followed: `IssuerTrustReconciler` watches the Service and reissues only the leaf (IP SAN) and the `ichor-oidc-trust` discoveryURL, and the machine reconciler maps that ConfigMap to every machine of an Ichor-issuer cluster, so the apiserver config re-renders through the normal rollout. What went stale was the internal discovery document: `ichor init` pinned `--internal-jwks-uri` to the ClusterIP in the ichor-ui Deployment, so the apiserver was sent to the old address for the JWKS. Decision: ichor-ui now advertises the jwks_uri on the host the caller connected to (`oidcissuer.Config.InternalJWKSFromHost`), and init no longer passes `--internal-jwks-uri` (a re-run removes it from an existing Deployment; the flag stays as an explicit override). The apiserver reaches the internal listener only through an address in the serving certificate SAN, so that host is always one the controller has issued for. A fixed ClusterIP was rejected: it needs a chosen free address in the Service CIDR on every install path and does not help the Service-recreated-by-someone-else case.

Amendment (2026-10-29): in-page sign-in. The SPA shows one card, "Sign in to Ichor", with the user name and password form and, when the external provider of ADR 0058 is also configured, a "Sign in with your identity provider" button below a separator; there is no provider-choice screen and no page transition for the password sign-in. `GET /oidc/authorize` answers `{"ticket": "..."}` (200) instead of the 302 to `/login` when the request carries `Accept: application/json`; every check is the same as on the redirect path (client, exact redirect URI, S256 PKCE, state, nonce, audience), and a failure is a 400 JSON error with no redirect. The SPA then posts the ticket and credentials to `/oidc/login`, checks that the returned `redirect_uri` is its own `/auth/callback` with its own `state`, and exchanges the code at the token endpoint with `fetch` (verifier and nonce kept in memory, not in `sessionStorage`). Why: the second card and two navigations added nothing the PKCE and ticket checks needed, and the full-page redirects made the flow feel broken. The redirect path and the `/login?ticket=` page stay for `ichor auth login` (the CLI opens a browser at the issuer); that page renders the same card. The external IdP button still redirects, necessarily.

Amendment 2026-10-29: served certificates are reloaded on change. ichor-ui's HTTPS listener (`--tls-cert-file`), its internal listener on 8444 and the controller's metrics server follow their certificate files through controller-runtime's `certwatcher` (file events plus a 10 s poll), so a cert-manager renewal reaches new connections without a restart; a pair that fails to load keeps the previous certificate serving and is logged without key material. The webhook server already used `certwatcher` through its `CertDir`.

Amendment 2026-10-29 (ADR 0105): ichor-ui's main listener is HTTPS only, on 8443, with the operator's certificate or the Ichor leaf (`ichor-oidc-tls`, now issued for every install, not only with `--user-issuer`); the 8444 issuer listener is unchanged and keeps the Ichor leaf. The controller publishes the CA as `ichor-ui-ca`. Token and refresh lifetimes are runtime settings in `IchorInstallation` (`spec.users`), read per issued token; the flags are the fallback. The guide's "Expose ichor-ui over https" now carries the Ingress-NGINX, `BackendTLSPolicy` and passthrough recipes.

Amendment 2026-11-04: a password change keeps the browser tab signed in. ADR 0108 made a password change move `notBefore` and drop every refresh family, which ends the tab's own ID token too, and the SPA then called `signOut()`, so a user forced through the must-change screen (or choosing a new password in the account menu) had to sign in again by hand. Decision: after `ChangeOwnPassword` succeeds the SPA signs in again with the new password through the in-page flow above (`signInAgain` in `ui/src/lib/login.ts`: `/auth/config` for the `ichor` provider, the user name from the `sub` claim of the stored token, then authorize, login and token as for any sign-in), and reloads. No new server surface and no new token-minting path: the password is in memory for that one call, the new token is issued by the normal login, and `notBefore` and the families stay as ADR 0108 left them, so every other session and refresh-token family of the user is still revoked. The new login's `iat` is not before `notBefore` (whole seconds, compared with `<`), so it is accepted. If the silent sign-in fails for any reason the tab clears its token and shows the sign-in card, as before. Rejected: returning tokens from `ChangeOwnPassword` (a second minting path next to the issuer's) and sparing the caller's own family (the SPA has no refresh token, and the server could not tell which family is the caller's). The CLI has no change-own-password command (`ichor user set-password` is the administrator's and revokes the user's sessions by design), so it is unaffected.

Amendment 2026-11-04 (browser session): the SPA signs in with a refresh token, so a tab stays signed in up to the issuer's absolute refresh lifetime instead of 5 minutes. The cause of the short session was in the SPA alone: it asked for `openid email profile groups`, and the issuer issues a refresh token only for `offline_access` (`codeGrant`), so the tab held an ID token of `DefaultTokenTTL` (5 minutes) and nothing to renew it with. The issuer already allowed the refresh grant for `ichor-ui` (same rotation, reuse detection and absolute lifetime as the CLI's, ADR 0108 F7; audience stays `ichor-ui`, never `ichor-kube`), so it has no change; a test now pins the browser client's rotation.

Decision.
- The ichor sign-in (redirect and in-page) adds `offline_access`. The external IdP request is unchanged: its refresh behaviour is the provider's, not ours.
- The refresh token is kept in `sessionStorage` (`ichor.refresh`, with the token endpoint and client id) beside the ID token, not in `localStorage`, not in a cookie. Reasoning against XSS: the ID token is already there and is the bearer for every Ichor call, so a script that can read the storage can already act as the user for its 5 minutes; the refresh token extends that to the session lifetime, which is the new exposure. It is bounded by what is already in place: the CSP is `script-src 'self'` with no inline script, there is no HTML injection surface in the pages, the token is tab-scoped and gone when the tab closes, it is single-use with rotation and reuse detection (a thief's first use makes the owner's next refresh fail and kills the family, so the theft is visible as a sign-out), the lifetime is absolute (rotation never extends it), the audience is `ichor-ui` only so it is not a Kubernetes credential, and RevokeSessions, a password change and disabling the user end the family. In-memory only was rejected: a reload would sign the user in again, which is the problem being fixed. A cookie (httpOnly) would need a server-side session, which the issuer deliberately does not have, and would add CSRF surface.
- Refresh runs one at a time per tab (single flight), a minute before the ID token ends (a third of its life for tokens under 3 minutes, never more often than every 10 s), when a hidden tab becomes visible, before a request that would carry an ID token about to end, and once on a 401 for a unary call, which is retried once. A refresh the issuer refuses (400/401/403) ends the session: tokens dropped, tab back to the sign-in card. A network or 5xx failure keeps the session and retries every 15 s. A server stream that ends with "your sign-in expired" (the log view) is reopened from its resume token after a refresh, within its reconnect budget.
- Two tabs: each tab signs in separately and has its own family (sessionStorage is per tab, so no tokens are shared and no cross-tab protocol is needed). The one way two tabs end up with one family is the browser's "duplicate tab" and `window.open` without `noopener`, which copy sessionStorage; the first rotation would then make the copy present a retired token and revoke the family for both. A tab that has a family holds a Web Lock named after it (`ichor-session-<fam>`) for as long as its page lives; a copy cannot take it within 1.5 s, drops the copied tokens and shows the sign-in card, so the original keeps its session. Without Web Locks, or if the lock call fails for another reason, the tab goes ahead (the failure is then the reuse detection's: both tabs sign in again). Coordinating one family across tabs (BroadcastChannel plus a lock) was rejected as more code and more exposure (tokens on a channel) for no gain over a second sign-in.
- Password change: `signInAgain` goes through the normal flow, so it asks for `offline_access` and stores the new refresh token; the old family died with the change (ADR 0108), as before.
- No idle timeout. An open tab refreshes by itself, so the session ends at the absolute lifetime (`ichorUI.users.refreshLifetime`, runtime-settable, default 12h, 1h to 30d) and not before, like the CLI. An idle timeout would need either a server-side last-seen (the issuer is stateless apart from the family entry) or browser activity tracking that a hidden, throttled tab defeats; an administrator who wants a shorter exposure lowers `refreshLifetime`. Not added; the guide says so.
- Sign-out drops the tokens in the tab; the issuer has no revocation endpoint, so a copy taken earlier lives to the end of its lifetime unless the user's sessions are revoked. Same as `ichor auth logout`. A revocation endpoint (RFC 7009) would need a `RefreshStore.Revoke` and is not built.
- Superseded: the "A refresh grant returns no new refresh token (not rotated)" amendment of Unit 2 (rotation came with ADR 0108), and the sentence in 2026-11-04's password-change amendment that the SPA has no refresh token.
- Known limit: if the answer to a refresh is lost after the issuer rotated, the next try presents the retired token and the family is revoked: the tab signs in again. Rotation without a grace window makes this unavoidable.

Follow-up 2026-11-04 (sign-out revokes): the open issue above is closed. The issuer has `POST /oidc/revoke`, the RFC 7009 revocation endpoint for refresh tokens, advertised as `revocation_endpoint` in the external discovery document (not on the internal listener, which serves only discovery and the JWKS). Request: form `token` (required), `client_id` (`ichor-ui` or `ichor-cli`, public clients, no other authentication; same checks and the same per-source rate limit as `/oidc/token`), optional `token_type_hint=refresh_token`. Behaviour: a valid refresh token deletes its whole family (`RefreshStore.Revoke`, an optimistic-lock patch of the user's credentials entry like rotation), so that token, its predecessors and its successor all stop refreshing; a token that is not a valid refresh token (garbage, expired, an ID token, family already gone) answers 200 with no body, as RFC 7009 section 2.2 requires, so the endpoint is no oracle; a valid refresh token issued to another client is refused with 400 `invalid_grant` and not revoked (section 2.1); a missing or repeated `token`, or a body that is not a form, is 400 `invalid_request`; an unknown `client_id` is 401 `invalid_client`; `token_type_hint=access_token` is 400 `unsupported_token_type`, because ID tokens (the bearer) are not revocable and end within `TokenTTL`; a store failure is 500 `server_error`. Sign-out: the SPA's `signOut` posts the refresh token to the revocation endpoint it saved from discovery with the session (`revocationEndpoint` beside the token endpoint in `ichor.refresh`; `keepalive`, 3 s timeout) before it clears the tokens, and `ichor auth logout` posts every cached login's refresh token (discovering the endpoint, 10 s timeout) before it deletes the cache. Both are best effort: a failure, a refusal or an issuer without the endpoint only warns (CLI) or is ignored (SPA), and the sign-out completes, so the token then lasts to its lifetime as before. A tab signed in before this change has no stored endpoint and only discards its tokens. A tab that lost a family to the duplicate-tab check or a refused refresh has already cleared its tokens and posts nothing, so a copied tab never revokes the original's family.

Amendment 2026-11-05 (ADR 0129): the last-enabled-admin guard and the role-grant guard also hold for a `kubectl` write, through a validating webhook on `ichorusers`. System identities and `system:masters` (the override certificate) are exempt, so the override credential stays the recovery path described above.