hive reads the forge for nix as its own account, not core
hive-c0re copies the hive's store read token to /run/hyperhive/forge-hive-token for nix's git credential helper, and no longer mints, reads or falls back to the core admin token. drops the core user sweep, avatar upload, webhook cleanup and hive-priv's RunForgeAdmin.
This commit is contained in:
atlas
parent
32a4821cb4
commit
b79896c237
23 changed files with 236 additions and 1118 deletions
|
|
@ -157,7 +157,7 @@ and for a new agent, and sends the deploy only when that update succeeded.
|
|||
a custom agent from the config commit at the head of `agent-configs/<n>`
|
||||
`main`, an archetype agent from the archetype it has now. An agent switched
|
||||
between custom and archetype, or to another archetype, fails that test.
|
||||
A hive without the `core` token file that nix's git reads, a repo with no
|
||||
A hive without the forge credential file that nix's git reads, a repo with no
|
||||
commit, a head that lacks the agent and a head behind the config each fail
|
||||
the build with an error that names the agent. `MetaSync` records the commit,
|
||||
with the hyperhive and config commits the head pins, in `hive-c0re`'s own state,
|
||||
|
|
@ -175,7 +175,8 @@ and for a new agent, and sends the deploy only when that update succeeded.
|
|||
`{"op":"update_container","name":"atlas","stream":true,"meta_rev":"<40 hex>"}`.
|
||||
- **Fetching.** nix fetches the forge's inputs with `git`. Both `hive-c0re`
|
||||
and `hive-priv` carry a `$HOME/.gitconfig` that answers for the forge host
|
||||
through `git-credential-hive-forge` (the `core` token file) and rewrites the
|
||||
through `git-credential-hive-forge` (the hive's forge read token, published to
|
||||
`/run/hyperhive/forge-hive-token`) and rewrites the
|
||||
forge's `https` URL, which `internal/meta`'s lock names its inputs by, to
|
||||
`http`.
|
||||
- **Hyperhive.** The commit pins one hyperhive rev for the swarm, so an agent
|
||||
|
|
@ -258,7 +259,7 @@ agent can't fast-forward its own config or self-merge its PR (see
|
|||
[Config changes](#config-changes) above). hive-c0re passes the credential to
|
||||
each `git` call inline, never writing it into
|
||||
`applied/<n>/.git/config`; that repo is RO-bind-mounted into the root
|
||||
agent, and a stored token would leak core's admin credential to an
|
||||
agent, and a stored token would leak the hive's forge credential to an
|
||||
agent.
|
||||
|
||||
The dashboard deep-links into this org — a `config repo` link
|
||||
|
|
|
|||
|
|
@ -16,7 +16,7 @@ each agent on relevant activity.
|
|||
|
||||
## Token scopes
|
||||
|
||||
Three scope sets:
|
||||
Two scope sets:
|
||||
|
||||
**`AGENT_TOKEN_SCOPES`** (swarm-controller's `forge::agent_token`, pinned
|
||||
by a test). swarm-controller mints every agent token with it:
|
||||
|
|
@ -32,15 +32,6 @@ by a test). swarm-controller mints every agent token with it:
|
|||
| `read:notification` | Poll `GET /notifications` for unread events. |
|
||||
| `write:notification` | Mark notifications read via `PATCH /notifications/threads/{id}`. |
|
||||
|
||||
**`CORE_TOKEN_SCOPES`** (`hive-c0re::forge`, for its own `core` user): the
|
||||
same set as `AGENT_TOKEN_SCOPES`, with no admin scope. Forgejo's token scope
|
||||
gate runs before the user-permission check, so this token gets
|
||||
`403 Forbidden` on `/api/v1/admin/*` even though `core` is a site admin.
|
||||
hive-c0re only mints the token when the file at
|
||||
`/var/lib/hyperhive/forge-core-token` is missing or the forge rejects it, so
|
||||
a token minted with admin scopes keeps them until an operator deletes that
|
||||
file and restarts hive-c0re.
|
||||
|
||||
**`HIVE_TOKEN_SCOPES`** (swarm-controller's `forge::hive_token`, pinned by a
|
||||
test): `read:repository` alone, for each hive's read-only account (next
|
||||
section).
|
||||
|
|
@ -68,9 +59,8 @@ agent's account to the team or mints a token for it. An agent's own forge
|
|||
account setup refuses the hive's account the same way.
|
||||
|
||||
hive-c0re reads the token with `forge::reader_token()`, under the hive's own
|
||||
store certificate, every time it reads the forge. Where the store holds none,
|
||||
it falls back to the `core` token file, which only the host running the forge
|
||||
container has. These reads go through it, and none of them needs a local
|
||||
store certificate, every time it reads the forge. A hive whose store holds
|
||||
none reads nothing. These reads go through it, and none of them needs a local
|
||||
forge container:
|
||||
|
||||
- fetching a merged config commit for a swarm deploy, and
|
||||
|
|
@ -79,16 +69,19 @@ forge container:
|
|||
- seeding `proposed` from `agent-configs/<agent>` on first deploy
|
||||
- cloning and pulling `/knowledge`
|
||||
|
||||
The builds themselves read as `core`. nix fetches the config inputs and
|
||||
The builds read as the hive's account too. nix fetches the config inputs and
|
||||
`internal/meta` with `git`, and `git` gets its credential from
|
||||
`git-credential-hive-forge`, which reads the `core` token file. A hive without
|
||||
that file fetches nothing private through nix, whatever its own token reads, so
|
||||
its builds fail: they have no other flake than `internal/meta`.
|
||||
`git-credential-hive-forge`, which reads `/run/hyperhive/forge-hive-token`.
|
||||
hive-c0re writes that file (account on the first line, token on the second)
|
||||
from the hive's store token at each boot and each `internal/meta` read, when its
|
||||
content differs. A hive whose store holds no token has no such file, fetches
|
||||
nothing private through nix and fails its builds: they have no other flake than
|
||||
`internal/meta`. This also holds on a hive with no forge container.
|
||||
|
||||
Roll a change to any of this out store host first (its granter writes the
|
||||
controller's grant on `swarm/hives/+/forge/token`), then the controller host,
|
||||
then the hives. A hive that starts before its token exists reads with the
|
||||
core file, or not at all, until the controller's next pass.
|
||||
then the hives. A hive that starts before its token exists reads nothing, and
|
||||
builds nothing, until the controller's next pass.
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -108,7 +101,7 @@ the avatar sync read it from there, falling back to `<state>/forge-token`,
|
|||
the file hive-c0re wrote before. hive-c0re no longer creates agent users or
|
||||
mints agent tokens. swarm-controller deletes the `hyperhive-<unix-seconds>`
|
||||
tokens it left behind, for each agent whose current `swarm-agent` token is
|
||||
live; `core`'s `hyperhive`-named admin token is never touched.
|
||||
live; tokens of other names are never touched.
|
||||
|
||||
Two things live in the `agent-configs` Forgejo organization:
|
||||
|
||||
|
|
|
|||
|
|
@ -72,7 +72,7 @@ Cheap — no build slot:
|
|||
| `SetWanted` | write the durable power intent (`wanted = Up`/`Offline`) as the head node of a power-op DAG. Takes the agent lease even though it's a store write, so the intent write and the tail `Reconcile` are atomic per-agent — two racing power ops can't clobber each other's intent before either reconciles |
|
||||
| `EmitRebuilt` | tail of a rebuild — emit the agent's `Rebuilt` manager event (ok/fail per outcome, nothing on cancel). One node per agent _and_ per outcome |
|
||||
| `WriteDropin` | `set_nspawn_flags` + `set_resource_limits` + daemon-reload |
|
||||
| `ForgeSweep` | one-shot boot-time forge user/token sweep for every container (`forge::ensure_all`) as a first-class node, so it shows as real work on the dashboard instead of running invisibly in a bare `tokio::spawn`. Agentless |
|
||||
| `ForgeSweep` | one-shot boot-time publish of the hive's forge read token for nix (`forge::ensure_all`) as a first-class node, so it shows as real work on the dashboard instead of running invisibly in a bare `tokio::spawn`. Agentless |
|
||||
| `MatrixSweep` | matrix user/space sweep (`matrix::ensure_all`): the boot-time instance, plus one every 30 min from a loop in `main.rs`. Holds `Resource::MatrixSweep` (capacity 1), so two passes never overlap; each tick queues its own pass, which waits for the resource if one is already live. Agentless |
|
||||
| `KnowledgePull` | `/knowledge` pull (`knowledge::pull`): at boot (commits that landed while `hive-c0re` was down), on the swarm knowledge-changed event, and hourly as a fallback. Holds `Resource::KnowledgeTree` (capacity 1), so two pulls never overlap on the working tree; each trigger queues its own pass, which waits for the resource if one is already live. Agentless |
|
||||
| `WantedPull` | pull of the agent set the swarm controller declares for this hive (`wanted::pull`), converging the agents it names. Runs at boot, waiting up to a minute in total for the queue connection, and again after every attach of the wanted-state watch (which retries with a backoff when the attach fails or the stream ends); the deploy event (`swarm_status`) and the watch are the fast path, this repairs a missed one. A first deploy already queued or running for an agent isn't queued again. Agentless |
|
||||
|
|
|
|||
|
|
@ -477,8 +477,7 @@ and with what the swarm controller sees. `hyperhive.hive.warnings` is the
|
|||
detail behind it — `warn`-level entries mean "an operator should look" and do
|
||||
**not** set `degraded`. `hyperhive.hive.warning` names the cause: the `kind`
|
||||
label is one of a fixed set in the source (for example `disk_pressure`, a
|
||||
failing sweep such as `swarm_status_publish`, `pending_logins`,
|
||||
`forge_ensure_core_user`), and the message lives in the
|
||||
failing sweep such as `swarm_status_publish`, `pending_logins`), and the message lives in the
|
||||
`hive warning raised` line hive-c0re logs when it raises the warning.
|
||||
|
||||
The swarm Grafana (`nix/host-modules/swarm-grafana.nix`) alerts on these:
|
||||
|
|
|
|||
|
|
@ -255,12 +255,13 @@ such as the root token.
|
|||
|
||||
## Hive-level — one of each per hive
|
||||
|
||||
| secret | generated by | lives at |
|
||||
| ---------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| hive leaf certs | `hive-tls.nix`, issued by the secret store's `pki` mount | `<deploy.hive-controller.tls.stateDir>/<name>.pem` |
|
||||
| the hive's own store identity | `glue-bao-tls.nix` on the store's host; placed by hand on every other host ([`ca.md`](ca.md#the-gateway-leaf-comes-from-the-secret-store)) | `deploy.hive-controller.tls.baoHiveIdentityCertFile` / `baoHiveIdentityKeyFile` |
|
||||
| the forge's copy of its OIDC secret | `swarm-bao-forge-oidc.service` reads it out of the swarm secret store under the forge publisher's leaf, wherever authelia runs | `/run/hive-forge-oidc/<id>.secret` on the host (tmpfs), bound into the forge container, handed to `forgejo-sso-source` through `LoadCredential` |
|
||||
| the homeserver's copy of its OIDC secret | `swarm-bao-matrix-oidc.service` reads it out of the swarm secret store under matrix-ctl's leaf, wherever authelia runs | `/run/hive-matrix-oidc/<id>.secret` on the host (tmpfs), bound into the homeserver container, handed to tuwunel through `LoadCredential` |
|
||||
| secret | generated by | lives at |
|
||||
| ---------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
||||
| hive leaf certs | `hive-tls.nix`, issued by the secret store's `pki` mount | `<deploy.hive-controller.tls.stateDir>/<name>.pem` |
|
||||
| the hive's own store identity | `glue-bao-tls.nix` on the store's host; placed by hand on every other host ([`ca.md`](ca.md#the-gateway-leaf-comes-from-the-secret-store)) | `deploy.hive-controller.tls.baoHiveIdentityCertFile` / `baoHiveIdentityKeyFile` |
|
||||
| the forge's copy of its OIDC secret | `swarm-bao-forge-oidc.service` reads it out of the swarm secret store under the forge publisher's leaf, wherever authelia runs | `/run/hive-forge-oidc/<id>.secret` on the host (tmpfs), bound into the forge container, handed to `forgejo-sso-source` through `LoadCredential` |
|
||||
| the homeserver's copy of its OIDC secret | `swarm-bao-matrix-oidc.service` reads it out of the swarm secret store under matrix-ctl's leaf, wherever authelia runs | `/run/hive-matrix-oidc/<id>.secret` on the host (tmpfs), bound into the homeserver container, handed to tuwunel through `LoadCredential` |
|
||||
| this hive's forge read token | `swarm-controller`, into the secret store at `swarm/hives/<hive>/forge/token`, for the `hive-<hive>` account; a pass at start and every five minutes re-mints a missing or stale token | `hive-c0re` reads it under the hive's own store identity, and copies it (account on the first line, token on the second) to `/run/hyperhive/forge-hive-token` (tmpfs, `0600` `hive-core`) for nix's `git`, which `hive-priv` also runs. Each `internal/meta` read and each boot rewrites the copy when it differs; where they differ after a rotation, a build that starts before the next read fails to authenticate, and a retry reads the new token |
|
||||
|
||||
The forge and the homeserver read their copies out of the store, the way
|
||||
Grafana does below: the forge under the forge publisher's leaf, the
|
||||
|
|
|
|||
|
|
@ -61,9 +61,7 @@ host.
|
|||
|
||||
A hive that stops running the forge keeps the old container's state at
|
||||
`/var/lib/nixos-containers/hive-forge/`. Nothing moves it to the swarm's
|
||||
forge: push anything worth keeping there by hand. Its
|
||||
`/var/lib/hyperhive/forge-core-token` came from that old forge and
|
||||
fails against the swarm's one.
|
||||
forge: push anything worth keeping there by hand.
|
||||
|
||||
</details>
|
||||
|
||||
|
|
|
|||
|
|
@ -260,7 +260,6 @@ known operations; there is no arbitrary command pass-through:
|
|||
| `WriteResourceLimits` | write `CPUQuota=`/`MemoryMax=`/`CPUWeight=`/`IOWeight=` systemd drop-in for agent container |
|
||||
| `RemoveServiceDropin` | remove `container@<name>.service.d/` drop-in on destroy |
|
||||
| `DaemonReload` | `systemctl daemon-reload` |
|
||||
| `RunForgeAdmin` | `nixos-container run hive-forge -- runuser -u forgejo -- forgejo admin <args>` |
|
||||
| `ControlInfraContainer` | `systemctl <action> nginx.service` for the gateway — the `InfraContainer` enum is the allowlist, and serde rejects unknown names at the wire boundary (`hive-c0re`, the forge and the matrix have no variant, so no request can name them) |
|
||||
| `SyncAgentTmpfiles` | unlink `/etc/tmpfiles.d/hyperhive-agents.conf` and return `Ok` |
|
||||
| `SetAgentPaused` | create / remove the `<state>/<name>/harness/paused` marker that parks an agent's turn loop |
|
||||
|
|
|
|||
Loading…
Reference in a new issue