hive reads the forge for nix as its own account, not core
hive-c0re copies the hive's store read token to /run/hyperhive/forge-hive-token for nix's git credential helper, and no longer mints, reads or falls back to the core admin token. drops the core user sweep, avatar upload, webhook cleanup and hive-priv's RunForgeAdmin.
This commit is contained in:
parent
32a4821cb4
commit
b79896c237
23 changed files with 236 additions and 1118 deletions
|
|
@ -157,7 +157,7 @@ and for a new agent, and sends the deploy only when that update succeeded.
|
||||||
a custom agent from the config commit at the head of `agent-configs/<n>`
|
a custom agent from the config commit at the head of `agent-configs/<n>`
|
||||||
`main`, an archetype agent from the archetype it has now. An agent switched
|
`main`, an archetype agent from the archetype it has now. An agent switched
|
||||||
between custom and archetype, or to another archetype, fails that test.
|
between custom and archetype, or to another archetype, fails that test.
|
||||||
A hive without the `core` token file that nix's git reads, a repo with no
|
A hive without the forge credential file that nix's git reads, a repo with no
|
||||||
commit, a head that lacks the agent and a head behind the config each fail
|
commit, a head that lacks the agent and a head behind the config each fail
|
||||||
the build with an error that names the agent. `MetaSync` records the commit,
|
the build with an error that names the agent. `MetaSync` records the commit,
|
||||||
with the hyperhive and config commits the head pins, in `hive-c0re`'s own state,
|
with the hyperhive and config commits the head pins, in `hive-c0re`'s own state,
|
||||||
|
|
@ -175,7 +175,8 @@ and for a new agent, and sends the deploy only when that update succeeded.
|
||||||
`{"op":"update_container","name":"atlas","stream":true,"meta_rev":"<40 hex>"}`.
|
`{"op":"update_container","name":"atlas","stream":true,"meta_rev":"<40 hex>"}`.
|
||||||
- **Fetching.** nix fetches the forge's inputs with `git`. Both `hive-c0re`
|
- **Fetching.** nix fetches the forge's inputs with `git`. Both `hive-c0re`
|
||||||
and `hive-priv` carry a `$HOME/.gitconfig` that answers for the forge host
|
and `hive-priv` carry a `$HOME/.gitconfig` that answers for the forge host
|
||||||
through `git-credential-hive-forge` (the `core` token file) and rewrites the
|
through `git-credential-hive-forge` (the hive's forge read token, published to
|
||||||
|
`/run/hyperhive/forge-hive-token`) and rewrites the
|
||||||
forge's `https` URL, which `internal/meta`'s lock names its inputs by, to
|
forge's `https` URL, which `internal/meta`'s lock names its inputs by, to
|
||||||
`http`.
|
`http`.
|
||||||
- **Hyperhive.** The commit pins one hyperhive rev for the swarm, so an agent
|
- **Hyperhive.** The commit pins one hyperhive rev for the swarm, so an agent
|
||||||
|
|
@ -258,7 +259,7 @@ agent can't fast-forward its own config or self-merge its PR (see
|
||||||
[Config changes](#config-changes) above). hive-c0re passes the credential to
|
[Config changes](#config-changes) above). hive-c0re passes the credential to
|
||||||
each `git` call inline, never writing it into
|
each `git` call inline, never writing it into
|
||||||
`applied/<n>/.git/config`; that repo is RO-bind-mounted into the root
|
`applied/<n>/.git/config`; that repo is RO-bind-mounted into the root
|
||||||
agent, and a stored token would leak core's admin credential to an
|
agent, and a stored token would leak the hive's forge credential to an
|
||||||
agent.
|
agent.
|
||||||
|
|
||||||
The dashboard deep-links into this org — a `config repo` link
|
The dashboard deep-links into this org — a `config repo` link
|
||||||
|
|
|
||||||
|
|
@ -16,7 +16,7 @@ each agent on relevant activity.
|
||||||
|
|
||||||
## Token scopes
|
## Token scopes
|
||||||
|
|
||||||
Three scope sets:
|
Two scope sets:
|
||||||
|
|
||||||
**`AGENT_TOKEN_SCOPES`** (swarm-controller's `forge::agent_token`, pinned
|
**`AGENT_TOKEN_SCOPES`** (swarm-controller's `forge::agent_token`, pinned
|
||||||
by a test). swarm-controller mints every agent token with it:
|
by a test). swarm-controller mints every agent token with it:
|
||||||
|
|
@ -32,15 +32,6 @@ by a test). swarm-controller mints every agent token with it:
|
||||||
| `read:notification` | Poll `GET /notifications` for unread events. |
|
| `read:notification` | Poll `GET /notifications` for unread events. |
|
||||||
| `write:notification` | Mark notifications read via `PATCH /notifications/threads/{id}`. |
|
| `write:notification` | Mark notifications read via `PATCH /notifications/threads/{id}`. |
|
||||||
|
|
||||||
**`CORE_TOKEN_SCOPES`** (`hive-c0re::forge`, for its own `core` user): the
|
|
||||||
same set as `AGENT_TOKEN_SCOPES`, with no admin scope. Forgejo's token scope
|
|
||||||
gate runs before the user-permission check, so this token gets
|
|
||||||
`403 Forbidden` on `/api/v1/admin/*` even though `core` is a site admin.
|
|
||||||
hive-c0re only mints the token when the file at
|
|
||||||
`/var/lib/hyperhive/forge-core-token` is missing or the forge rejects it, so
|
|
||||||
a token minted with admin scopes keeps them until an operator deletes that
|
|
||||||
file and restarts hive-c0re.
|
|
||||||
|
|
||||||
**`HIVE_TOKEN_SCOPES`** (swarm-controller's `forge::hive_token`, pinned by a
|
**`HIVE_TOKEN_SCOPES`** (swarm-controller's `forge::hive_token`, pinned by a
|
||||||
test): `read:repository` alone, for each hive's read-only account (next
|
test): `read:repository` alone, for each hive's read-only account (next
|
||||||
section).
|
section).
|
||||||
|
|
@ -68,9 +59,8 @@ agent's account to the team or mints a token for it. An agent's own forge
|
||||||
account setup refuses the hive's account the same way.
|
account setup refuses the hive's account the same way.
|
||||||
|
|
||||||
hive-c0re reads the token with `forge::reader_token()`, under the hive's own
|
hive-c0re reads the token with `forge::reader_token()`, under the hive's own
|
||||||
store certificate, every time it reads the forge. Where the store holds none,
|
store certificate, every time it reads the forge. A hive whose store holds
|
||||||
it falls back to the `core` token file, which only the host running the forge
|
none reads nothing. These reads go through it, and none of them needs a local
|
||||||
container has. These reads go through it, and none of them needs a local
|
|
||||||
forge container:
|
forge container:
|
||||||
|
|
||||||
- fetching a merged config commit for a swarm deploy, and
|
- fetching a merged config commit for a swarm deploy, and
|
||||||
|
|
@ -79,16 +69,19 @@ forge container:
|
||||||
- seeding `proposed` from `agent-configs/<agent>` on first deploy
|
- seeding `proposed` from `agent-configs/<agent>` on first deploy
|
||||||
- cloning and pulling `/knowledge`
|
- cloning and pulling `/knowledge`
|
||||||
|
|
||||||
The builds themselves read as `core`. nix fetches the config inputs and
|
The builds read as the hive's account too. nix fetches the config inputs and
|
||||||
`internal/meta` with `git`, and `git` gets its credential from
|
`internal/meta` with `git`, and `git` gets its credential from
|
||||||
`git-credential-hive-forge`, which reads the `core` token file. A hive without
|
`git-credential-hive-forge`, which reads `/run/hyperhive/forge-hive-token`.
|
||||||
that file fetches nothing private through nix, whatever its own token reads, so
|
hive-c0re writes that file (account on the first line, token on the second)
|
||||||
its builds fail: they have no other flake than `internal/meta`.
|
from the hive's store token at each boot and each `internal/meta` read, when its
|
||||||
|
content differs. A hive whose store holds no token has no such file, fetches
|
||||||
|
nothing private through nix and fails its builds: they have no other flake than
|
||||||
|
`internal/meta`. This also holds on a hive with no forge container.
|
||||||
|
|
||||||
Roll a change to any of this out store host first (its granter writes the
|
Roll a change to any of this out store host first (its granter writes the
|
||||||
controller's grant on `swarm/hives/+/forge/token`), then the controller host,
|
controller's grant on `swarm/hives/+/forge/token`), then the controller host,
|
||||||
then the hives. A hive that starts before its token exists reads with the
|
then the hives. A hive that starts before its token exists reads nothing, and
|
||||||
core file, or not at all, until the controller's next pass.
|
builds nothing, until the controller's next pass.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|
@ -108,7 +101,7 @@ the avatar sync read it from there, falling back to `<state>/forge-token`,
|
||||||
the file hive-c0re wrote before. hive-c0re no longer creates agent users or
|
the file hive-c0re wrote before. hive-c0re no longer creates agent users or
|
||||||
mints agent tokens. swarm-controller deletes the `hyperhive-<unix-seconds>`
|
mints agent tokens. swarm-controller deletes the `hyperhive-<unix-seconds>`
|
||||||
tokens it left behind, for each agent whose current `swarm-agent` token is
|
tokens it left behind, for each agent whose current `swarm-agent` token is
|
||||||
live; `core`'s `hyperhive`-named admin token is never touched.
|
live; tokens of other names are never touched.
|
||||||
|
|
||||||
Two things live in the `agent-configs` Forgejo organization:
|
Two things live in the `agent-configs` Forgejo organization:
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -72,7 +72,7 @@ Cheap — no build slot:
|
||||||
| `SetWanted` | write the durable power intent (`wanted = Up`/`Offline`) as the head node of a power-op DAG. Takes the agent lease even though it's a store write, so the intent write and the tail `Reconcile` are atomic per-agent — two racing power ops can't clobber each other's intent before either reconciles |
|
| `SetWanted` | write the durable power intent (`wanted = Up`/`Offline`) as the head node of a power-op DAG. Takes the agent lease even though it's a store write, so the intent write and the tail `Reconcile` are atomic per-agent — two racing power ops can't clobber each other's intent before either reconciles |
|
||||||
| `EmitRebuilt` | tail of a rebuild — emit the agent's `Rebuilt` manager event (ok/fail per outcome, nothing on cancel). One node per agent _and_ per outcome |
|
| `EmitRebuilt` | tail of a rebuild — emit the agent's `Rebuilt` manager event (ok/fail per outcome, nothing on cancel). One node per agent _and_ per outcome |
|
||||||
| `WriteDropin` | `set_nspawn_flags` + `set_resource_limits` + daemon-reload |
|
| `WriteDropin` | `set_nspawn_flags` + `set_resource_limits` + daemon-reload |
|
||||||
| `ForgeSweep` | one-shot boot-time forge user/token sweep for every container (`forge::ensure_all`) as a first-class node, so it shows as real work on the dashboard instead of running invisibly in a bare `tokio::spawn`. Agentless |
|
| `ForgeSweep` | one-shot boot-time publish of the hive's forge read token for nix (`forge::ensure_all`) as a first-class node, so it shows as real work on the dashboard instead of running invisibly in a bare `tokio::spawn`. Agentless |
|
||||||
| `MatrixSweep` | matrix user/space sweep (`matrix::ensure_all`): the boot-time instance, plus one every 30 min from a loop in `main.rs`. Holds `Resource::MatrixSweep` (capacity 1), so two passes never overlap; each tick queues its own pass, which waits for the resource if one is already live. Agentless |
|
| `MatrixSweep` | matrix user/space sweep (`matrix::ensure_all`): the boot-time instance, plus one every 30 min from a loop in `main.rs`. Holds `Resource::MatrixSweep` (capacity 1), so two passes never overlap; each tick queues its own pass, which waits for the resource if one is already live. Agentless |
|
||||||
| `KnowledgePull` | `/knowledge` pull (`knowledge::pull`): at boot (commits that landed while `hive-c0re` was down), on the swarm knowledge-changed event, and hourly as a fallback. Holds `Resource::KnowledgeTree` (capacity 1), so two pulls never overlap on the working tree; each trigger queues its own pass, which waits for the resource if one is already live. Agentless |
|
| `KnowledgePull` | `/knowledge` pull (`knowledge::pull`): at boot (commits that landed while `hive-c0re` was down), on the swarm knowledge-changed event, and hourly as a fallback. Holds `Resource::KnowledgeTree` (capacity 1), so two pulls never overlap on the working tree; each trigger queues its own pass, which waits for the resource if one is already live. Agentless |
|
||||||
| `WantedPull` | pull of the agent set the swarm controller declares for this hive (`wanted::pull`), converging the agents it names. Runs at boot, waiting up to a minute in total for the queue connection, and again after every attach of the wanted-state watch (which retries with a backoff when the attach fails or the stream ends); the deploy event (`swarm_status`) and the watch are the fast path, this repairs a missed one. A first deploy already queued or running for an agent isn't queued again. Agentless |
|
| `WantedPull` | pull of the agent set the swarm controller declares for this hive (`wanted::pull`), converging the agents it names. Runs at boot, waiting up to a minute in total for the queue connection, and again after every attach of the wanted-state watch (which retries with a backoff when the attach fails or the stream ends); the deploy event (`swarm_status`) and the watch are the fast path, this repairs a missed one. A first deploy already queued or running for an agent isn't queued again. Agentless |
|
||||||
|
|
|
||||||
|
|
@ -477,8 +477,7 @@ and with what the swarm controller sees. `hyperhive.hive.warnings` is the
|
||||||
detail behind it — `warn`-level entries mean "an operator should look" and do
|
detail behind it — `warn`-level entries mean "an operator should look" and do
|
||||||
**not** set `degraded`. `hyperhive.hive.warning` names the cause: the `kind`
|
**not** set `degraded`. `hyperhive.hive.warning` names the cause: the `kind`
|
||||||
label is one of a fixed set in the source (for example `disk_pressure`, a
|
label is one of a fixed set in the source (for example `disk_pressure`, a
|
||||||
failing sweep such as `swarm_status_publish`, `pending_logins`,
|
failing sweep such as `swarm_status_publish`, `pending_logins`), and the message lives in the
|
||||||
`forge_ensure_core_user`), and the message lives in the
|
|
||||||
`hive warning raised` line hive-c0re logs when it raises the warning.
|
`hive warning raised` line hive-c0re logs when it raises the warning.
|
||||||
|
|
||||||
The swarm Grafana (`nix/host-modules/swarm-grafana.nix`) alerts on these:
|
The swarm Grafana (`nix/host-modules/swarm-grafana.nix`) alerts on these:
|
||||||
|
|
|
||||||
|
|
@ -256,11 +256,12 @@ such as the root token.
|
||||||
## Hive-level — one of each per hive
|
## Hive-level — one of each per hive
|
||||||
|
|
||||||
| secret | generated by | lives at |
|
| secret | generated by | lives at |
|
||||||
| ---------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------- |
|
| ---------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
||||||
| hive leaf certs | `hive-tls.nix`, issued by the secret store's `pki` mount | `<deploy.hive-controller.tls.stateDir>/<name>.pem` |
|
| hive leaf certs | `hive-tls.nix`, issued by the secret store's `pki` mount | `<deploy.hive-controller.tls.stateDir>/<name>.pem` |
|
||||||
| the hive's own store identity | `glue-bao-tls.nix` on the store's host; placed by hand on every other host ([`ca.md`](ca.md#the-gateway-leaf-comes-from-the-secret-store)) | `deploy.hive-controller.tls.baoHiveIdentityCertFile` / `baoHiveIdentityKeyFile` |
|
| the hive's own store identity | `glue-bao-tls.nix` on the store's host; placed by hand on every other host ([`ca.md`](ca.md#the-gateway-leaf-comes-from-the-secret-store)) | `deploy.hive-controller.tls.baoHiveIdentityCertFile` / `baoHiveIdentityKeyFile` |
|
||||||
| the forge's copy of its OIDC secret | `swarm-bao-forge-oidc.service` reads it out of the swarm secret store under the forge publisher's leaf, wherever authelia runs | `/run/hive-forge-oidc/<id>.secret` on the host (tmpfs), bound into the forge container, handed to `forgejo-sso-source` through `LoadCredential` |
|
| the forge's copy of its OIDC secret | `swarm-bao-forge-oidc.service` reads it out of the swarm secret store under the forge publisher's leaf, wherever authelia runs | `/run/hive-forge-oidc/<id>.secret` on the host (tmpfs), bound into the forge container, handed to `forgejo-sso-source` through `LoadCredential` |
|
||||||
| the homeserver's copy of its OIDC secret | `swarm-bao-matrix-oidc.service` reads it out of the swarm secret store under matrix-ctl's leaf, wherever authelia runs | `/run/hive-matrix-oidc/<id>.secret` on the host (tmpfs), bound into the homeserver container, handed to tuwunel through `LoadCredential` |
|
| the homeserver's copy of its OIDC secret | `swarm-bao-matrix-oidc.service` reads it out of the swarm secret store under matrix-ctl's leaf, wherever authelia runs | `/run/hive-matrix-oidc/<id>.secret` on the host (tmpfs), bound into the homeserver container, handed to tuwunel through `LoadCredential` |
|
||||||
|
| this hive's forge read token | `swarm-controller`, into the secret store at `swarm/hives/<hive>/forge/token`, for the `hive-<hive>` account; a pass at start and every five minutes re-mints a missing or stale token | `hive-c0re` reads it under the hive's own store identity, and copies it (account on the first line, token on the second) to `/run/hyperhive/forge-hive-token` (tmpfs, `0600` `hive-core`) for nix's `git`, which `hive-priv` also runs. Each `internal/meta` read and each boot rewrites the copy when it differs; where they differ after a rotation, a build that starts before the next read fails to authenticate, and a retry reads the new token |
|
||||||
|
|
||||||
The forge and the homeserver read their copies out of the store, the way
|
The forge and the homeserver read their copies out of the store, the way
|
||||||
Grafana does below: the forge under the forge publisher's leaf, the
|
Grafana does below: the forge under the forge publisher's leaf, the
|
||||||
|
|
|
||||||
|
|
@ -61,9 +61,7 @@ host.
|
||||||
|
|
||||||
A hive that stops running the forge keeps the old container's state at
|
A hive that stops running the forge keeps the old container's state at
|
||||||
`/var/lib/nixos-containers/hive-forge/`. Nothing moves it to the swarm's
|
`/var/lib/nixos-containers/hive-forge/`. Nothing moves it to the swarm's
|
||||||
forge: push anything worth keeping there by hand. Its
|
forge: push anything worth keeping there by hand.
|
||||||
`/var/lib/hyperhive/forge-core-token` came from that old forge and
|
|
||||||
fails against the swarm's one.
|
|
||||||
|
|
||||||
</details>
|
</details>
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -260,7 +260,6 @@ known operations; there is no arbitrary command pass-through:
|
||||||
| `WriteResourceLimits` | write `CPUQuota=`/`MemoryMax=`/`CPUWeight=`/`IOWeight=` systemd drop-in for agent container |
|
| `WriteResourceLimits` | write `CPUQuota=`/`MemoryMax=`/`CPUWeight=`/`IOWeight=` systemd drop-in for agent container |
|
||||||
| `RemoveServiceDropin` | remove `container@<name>.service.d/` drop-in on destroy |
|
| `RemoveServiceDropin` | remove `container@<name>.service.d/` drop-in on destroy |
|
||||||
| `DaemonReload` | `systemctl daemon-reload` |
|
| `DaemonReload` | `systemctl daemon-reload` |
|
||||||
| `RunForgeAdmin` | `nixos-container run hive-forge -- runuser -u forgejo -- forgejo admin <args>` |
|
|
||||||
| `ControlInfraContainer` | `systemctl <action> nginx.service` for the gateway — the `InfraContainer` enum is the allowlist, and serde rejects unknown names at the wire boundary (`hive-c0re`, the forge and the matrix have no variant, so no request can name them) |
|
| `ControlInfraContainer` | `systemctl <action> nginx.service` for the gateway — the `InfraContainer` enum is the allowlist, and serde rejects unknown names at the wire boundary (`hive-c0re`, the forge and the matrix have no variant, so no request can name them) |
|
||||||
| `SyncAgentTmpfiles` | unlink `/etc/tmpfiles.d/hyperhive-agents.conf` and return `Ok` |
|
| `SyncAgentTmpfiles` | unlink `/etc/tmpfiles.d/hyperhive-agents.conf` and return `Ok` |
|
||||||
| `SetAgentPaused` | create / remove the `<state>/<name>/harness/paused` marker that parks an agent's turn loop |
|
| `SetAgentPaused` | create / remove the `<state>/<name>/harness/paused` marker that parks an agent's turn loop |
|
||||||
|
|
|
||||||
|
|
@ -20,13 +20,20 @@ const CONFIG_BRANCH: &str = "main";
|
||||||
/// [`MetaSource`] over the forge API.
|
/// [`MetaSource`] over the forge API.
|
||||||
pub(crate) struct ForgeSource {
|
pub(crate) struct ForgeSource {
|
||||||
api: Forgejo,
|
api: Forgejo,
|
||||||
|
/// Whether the credential file nix's `git` reads holds this hive's token.
|
||||||
|
nix_credential: bool,
|
||||||
}
|
}
|
||||||
|
|
||||||
/// A source reading as this hive, or `None` when it holds no forge token.
|
/// A source reading as this hive, or `None` when it holds no forge token. It
|
||||||
|
/// also refreshes the credential file nix's `git` fetches with, which the
|
||||||
|
/// build after this read needs to be current.
|
||||||
pub(crate) async fn internal_meta_source() -> Option<ForgeSource> {
|
pub(crate) async fn internal_meta_source() -> Option<ForgeSource> {
|
||||||
let token: ReaderToken = super::reader_token().await?;
|
let token: ReaderToken = super::reader_token().await?;
|
||||||
match token.api() {
|
match token.api() {
|
||||||
Ok(api) => Some(ForgeSource { api }),
|
Ok(api) => Some(ForgeSource {
|
||||||
|
api,
|
||||||
|
nix_credential: token.publish_for_nix(),
|
||||||
|
}),
|
||||||
Err(e) => {
|
Err(e) => {
|
||||||
tracing::warn!(error = ?e, "internal/meta: cannot build a forge client");
|
tracing::warn!(error = ?e, "internal/meta: cannot build a forge client");
|
||||||
None
|
None
|
||||||
|
|
@ -66,7 +73,7 @@ impl ForgeSource {
|
||||||
|
|
||||||
impl MetaSource for ForgeSource {
|
impl MetaSource for ForgeSource {
|
||||||
fn nix_can_fetch(&self) -> bool {
|
fn nix_can_fetch(&self) -> bool {
|
||||||
super::core_token().is_some()
|
self.nix_credential
|
||||||
}
|
}
|
||||||
|
|
||||||
async fn meta_head(&self) -> anyhow::Result<Option<String>> {
|
async fn meta_head(&self) -> anyhow::Result<Option<String>> {
|
||||||
|
|
@ -106,3 +113,25 @@ impl MetaSource for ForgeSource {
|
||||||
self.branch_head(CONFIG_ORG, agent, CONFIG_BRANCH).await
|
self.branch_head(CONFIG_ORG, agent, CONFIG_BRANCH).await
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
use super::*;
|
||||||
|
use forgejo_api::Auth;
|
||||||
|
|
||||||
|
fn source(nix_credential: bool) -> ForgeSource {
|
||||||
|
let url = url::Url::parse("http://forge.test").expect("url");
|
||||||
|
ForgeSource {
|
||||||
|
api: Forgejo::new(Auth::None, url).expect("client"),
|
||||||
|
nix_credential,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The control (`true`) shows the answer follows the field rather than
|
||||||
|
/// being a constant.
|
||||||
|
#[test]
|
||||||
|
fn nix_fetches_only_where_its_credential_was_published() {
|
||||||
|
assert!(source(true).nix_can_fetch());
|
||||||
|
assert!(!source(false).nix_can_fetch());
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
|
||||||
|
|
@ -1,7 +1,6 @@
|
||||||
//! Optional Forgejo wiring — the `core` admin user and token and its avatar.
|
//! The hive's forge access: reads of the forge go through [`reader_token`] on
|
||||||
//! Those writes, as `core`, run only beside a local `hive-forge`
|
//! every hive, and [`ensure_all`] publishes that token for nix's `git`. The
|
||||||
//! container; reads of the forge go through [`reader_token`] on every hive.
|
//! swarm-wide orgs, teams and repos, and each agent's read grant on
|
||||||
//! The swarm-wide orgs, teams and repos, and each agent's read grant on
|
|
||||||
//! `internal/docs`, are swarm-controller's. Full design:
|
//! `internal/docs`, are swarm-controller's. Full design:
|
||||||
//! `docs/integrations/forge.md`.
|
//! `docs/integrations/forge.md`.
|
||||||
|
|
||||||
|
|
@ -9,24 +8,20 @@ mod internal_meta;
|
||||||
mod reader;
|
mod reader;
|
||||||
mod reconcile;
|
mod reconcile;
|
||||||
mod repos;
|
mod repos;
|
||||||
mod users;
|
|
||||||
|
|
||||||
pub(crate) use internal_meta::internal_meta_source;
|
pub(crate) use internal_meta::internal_meta_source;
|
||||||
pub use reader::reader_token;
|
pub use reader::{publish_nix_credential, reader_token};
|
||||||
pub use reconcile::{fetch_forge_main, reconcile_config_apply, reconcile_config_status};
|
pub use reconcile::{fetch_forge_main, reconcile_config_apply, reconcile_config_status};
|
||||||
pub use repos::{clone_config_into_proposed, fetch_config_main_into_applied};
|
pub use repos::{clone_config_into_proposed, fetch_config_main_into_applied};
|
||||||
pub use users::core_token;
|
|
||||||
|
|
||||||
use std::future::IntoFuture;
|
use std::future::IntoFuture;
|
||||||
use std::sync::OnceLock;
|
use std::sync::OnceLock;
|
||||||
use std::time::{Duration, Instant};
|
use std::time::Duration;
|
||||||
|
|
||||||
use anyhow::{Context, Result};
|
use anyhow::{Context, Result};
|
||||||
use forgejo_api::{Auth, Forgejo};
|
use forgejo_api::{Auth, Forgejo};
|
||||||
use url::Url;
|
use url::Url;
|
||||||
|
|
||||||
use users::{ensure_core_avatar, ensure_core_user_and_token};
|
|
||||||
|
|
||||||
const FORGE_CONTAINER: &str = "hive-forge";
|
const FORGE_CONTAINER: &str = "hive-forge";
|
||||||
|
|
||||||
/// Base HTTP URL for the local Forgejo instance, from `HIVE_FORGE_URL`
|
/// Base HTTP URL for the local Forgejo instance, from `HIVE_FORGE_URL`
|
||||||
|
|
@ -55,18 +50,14 @@ pub(crate) fn forge_http_base() -> &'static str {
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Check the env the forge integration cannot run without: `HIVE_FORGE_URL`
|
/// Check the env the forge integration cannot run without: `HIVE_FORGE_URL`
|
||||||
/// set and a valid URL, `HIVE_ASSETS_DIR` set. Called once from `main`
|
/// set and a valid URL. Called once from `main` before any worker is
|
||||||
/// before any worker is spawned, so a missing or malformed value fails the
|
/// spawned, so a missing or malformed value fails the unit start naming the
|
||||||
/// unit start naming the variable; the lazy accessors then never hit their
|
/// variable; the lazy accessors then never hit their `expect`s.
|
||||||
/// `expect`s.
|
|
||||||
pub(crate) fn validate_env() -> Result<()> {
|
pub(crate) fn validate_env() -> Result<()> {
|
||||||
let url = std::env::var("HIVE_FORGE_URL").context(
|
let url = std::env::var("HIVE_FORGE_URL").context(
|
||||||
"HIVE_FORGE_URL is unset (the hyperhive NixOS module sets it on the hive-c0re unit)",
|
"HIVE_FORGE_URL is unset (the hyperhive NixOS module sets it on the hive-c0re unit)",
|
||||||
)?;
|
)?;
|
||||||
Url::parse(&url).with_context(|| format!("HIVE_FORGE_URL is not a valid URL: {url:?}"))?;
|
Url::parse(&url).with_context(|| format!("HIVE_FORGE_URL is not a valid URL: {url:?}"))?;
|
||||||
std::env::var("HIVE_ASSETS_DIR").context(
|
|
||||||
"HIVE_ASSETS_DIR is unset (the hyperhive NixOS module sets it on the hive-c0re unit)",
|
|
||||||
)?;
|
|
||||||
Ok(())
|
Ok(())
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -74,7 +65,7 @@ pub(crate) fn validate_env() -> Result<()> {
|
||||||
///
|
///
|
||||||
/// The token does not go here. A URL is a process argument, and `argv` is
|
/// The token does not go here. A URL is a process argument, and `argv` is
|
||||||
/// world-readable through `/proc/<pid>/cmdline` for as long as the git child
|
/// world-readable through `/proc/<pid>/cmdline` for as long as the git child
|
||||||
/// lives, so a credentialed remote publishes the core admin token to every
|
/// lives, so a credentialed remote publishes the forge token to every
|
||||||
/// local user on the host. Credentials travel in the environment instead, via
|
/// local user on the host. Credentials travel in the environment instead, via
|
||||||
/// [`basic_auth_header`] and [`crate::lifecycle::git_command_authed`] —
|
/// [`basic_auth_header`] and [`crate::lifecycle::git_command_authed`] —
|
||||||
/// `/proc/<pid>/environ` is owner-only.
|
/// `/proc/<pid>/environ` is owner-only.
|
||||||
|
|
@ -101,8 +92,7 @@ fn basic_auth_header(login: &str, token: &str) -> String {
|
||||||
format!("Authorization: Basic {basic}")
|
format!("Authorization: Basic {basic}")
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Forgejo org grouping every agent's config repo. Core is a site admin
|
/// Forgejo org grouping every agent's config repo. Each agent is a **write collaborator
|
||||||
/// and reads + writes every repo here. Each agent is a **write collaborator
|
|
||||||
/// on its own** `agent-configs/<name>` repo — the editable PR surface it
|
/// on its own** `agent-configs/<name>` repo — the editable PR surface it
|
||||||
/// pushes config-change branches to. `main`'s branch protection is
|
/// pushes config-change branches to. `main`'s branch protection is
|
||||||
/// swarm-controller's: merge and approval allowlisted to the `operators` team,
|
/// swarm-controller's: merge and approval allowlisted to the `operators` team,
|
||||||
|
|
@ -122,52 +112,10 @@ pub async fn is_present() -> bool {
|
||||||
stdout.lines().any(|l| l.trim() == FORGE_CONTAINER)
|
stdout.lines().any(|l| l.trim() == FORGE_CONTAINER)
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Name a `forgejo admin` invocation for an error message without
|
|
||||||
/// reproducing its arguments: keep the leading verb path, stop at the
|
|
||||||
/// first flag. `["user", "create", "--username", "iris", "--password",
|
|
||||||
/// "…"]` becomes `forgejo admin user create`.
|
|
||||||
///
|
|
||||||
/// The verbs are an **allowlist**, and that is the whole point. Some
|
|
||||||
/// callers pass a live secret as an argument value (`--password`), so a
|
|
||||||
/// message built from the raw vector puts it in the log. Listing the
|
|
||||||
/// flags to *hide* instead would repeat the bug this guards against: a
|
|
||||||
/// newly added secret-bearing flag would leak until someone remembered
|
|
||||||
/// to extend the list. Verbs are a closed set this crate chooses
|
|
||||||
/// itself; argument values never are.
|
|
||||||
fn describe_forge_admin(args: &[&str]) -> String {
|
|
||||||
let verbs: Vec<&str> = args
|
|
||||||
.iter()
|
|
||||||
.take_while(|a| !a.starts_with('-'))
|
|
||||||
.copied()
|
|
||||||
.collect();
|
|
||||||
if verbs.is_empty() {
|
|
||||||
"forgejo admin".to_owned()
|
|
||||||
} else {
|
|
||||||
format!("forgejo admin {}", verbs.join(" "))
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Run `forgejo admin <args>` inside the hive-forge container as the
|
|
||||||
/// forgejo user (the only uid with write access to the state dir).
|
|
||||||
/// Returns stdout on success; bails with stderr context on failure.
|
|
||||||
async fn forge_admin(args: &[&str]) -> Result<String> {
|
|
||||||
// Route through hive-priv (root helper) because `nixos-container run`
|
|
||||||
// uses nsenter to enter the container's namespaces, which requires root.
|
|
||||||
// hive-c0re runs as the unprivileged `hive-core` user and cannot call
|
|
||||||
// nsenter directly — doing so produces:
|
|
||||||
// nsenter: stat of /proc/<pid>/ns/user failed: Permission denied
|
|
||||||
let (stdout, _stderr) = crate::priv_client::run_forge_admin(args)
|
|
||||||
.await
|
|
||||||
.with_context(|| format!("{} (via hive-priv)", describe_forge_admin(args)))?;
|
|
||||||
Ok(stdout)
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Typed Forgejo API client for the local forge ([`forge_http_base()`]),
|
/// Typed Forgejo API client for the local forge ([`forge_http_base()`]),
|
||||||
/// authenticated as `token`. All Forgejo API calls that don't shell
|
/// authenticated as `token`. All Forgejo API calls go through clients built
|
||||||
/// out to `forgejo admin` go through clients built here — one place
|
/// here — one place for the base URL and auth. The token is passed per call;
|
||||||
/// for the base URL and auth. Tokens differ per call site (core admin
|
/// the base URL is parsed once. Failures surface as
|
||||||
/// token vs per-agent tokens), so the token is passed per call; the
|
|
||||||
/// base URL is parsed once. Failures surface as
|
|
||||||
/// `forgejo_api::ForgejoError`, whose Display carries the HTTP status
|
/// `forgejo_api::ForgejoError`, whose Display carries the HTTP status
|
||||||
/// and the API's error message (e.g. the validation reason on a 422).
|
/// and the API's error message (e.g. the validation reason on a 422).
|
||||||
pub(crate) fn api(token: &str) -> Result<Forgejo> {
|
pub(crate) fn api(token: &str) -> Result<Forgejo> {
|
||||||
|
|
@ -197,169 +145,19 @@ where
|
||||||
.map_err(|_| anyhow::anyhow!("{op}: no answer from the forge within {API_TIMEOUT:?}"))
|
.map_err(|_| anyhow::anyhow!("{op}: no answer from the forge within {API_TIMEOUT:?}"))
|
||||||
}
|
}
|
||||||
|
|
||||||
/// The `core_token.is_some()` half of [`ensure_all`]: the
|
/// Publish this hive's forge read token for nix's `git` at startup, so a build
|
||||||
/// core avatar, and removing this hive's own config-PR hook —
|
/// that starts before any `internal/meta` read still has a credential. Every
|
||||||
/// every step that needs an authenticated forge client. The swarm-wide
|
/// hive runs it: the token is the hive's own, in the store, whether or not a
|
||||||
/// objects (orgs, the `operators` team, mirrors, `internal/docs`,
|
/// forge container is local.
|
||||||
/// `internal/knowledge`, the `agent-configs` avatar) are the
|
|
||||||
/// swarm-controller's, not this hive's. Split out purely to keep
|
|
||||||
/// `ensure_all` under clippy's function-length limit; not meant to be called
|
|
||||||
/// from anywhere else.
|
|
||||||
async fn ensure_all_orgs_and_repos(token: &str) {
|
|
||||||
if let Err(e) = ensure_core_avatar(token).await {
|
|
||||||
tracing::warn!(error = ?e, "forge: ensure_core_avatar failed");
|
|
||||||
crate::warnings::set_boot_warning(
|
|
||||||
"forge_ensure_core_avatar",
|
|
||||||
"warn",
|
|
||||||
format!("forge: core avatar upload failed: {e}"),
|
|
||||||
);
|
|
||||||
}
|
|
||||||
if let Err(e) = remove_config_pr_webhook(token).await {
|
|
||||||
tracing::warn!(error = ?e, "forge: remove_config_pr_webhook failed");
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Delete this hive's own `/webhook/config-pr` hook from the
|
|
||||||
/// `agent-configs` org. No hive serves that route, so such a hook fails on
|
|
||||||
/// every config-repo event; swarm-controller's hook is the one that acts on
|
|
||||||
/// config PRs.
|
|
||||||
///
|
|
||||||
/// Only a hook whose URL equals this hive's exactly is removed, never one
|
|
||||||
/// matching by path: the same path on another base is another hive's hook.
|
|
||||||
/// Idempotent, and a no-op when `HYPERHIVE_HIVE_DOMAIN` is unset.
|
|
||||||
async fn remove_config_pr_webhook(core_token: &str) -> Result<()> {
|
|
||||||
let Some(domain) = std::env::var("HYPERHIVE_HIVE_DOMAIN")
|
|
||||||
.ok()
|
|
||||||
.filter(|v| !v.is_empty())
|
|
||||||
else {
|
|
||||||
return Ok(());
|
|
||||||
};
|
|
||||||
let own_url = config_pr_hook_url(&domain);
|
|
||||||
let client = api(core_token)?;
|
|
||||||
let hooks = bounded(
|
|
||||||
&format!("list webhooks for org {CONFIG_ORG}"),
|
|
||||||
client.org_list_hooks(CONFIG_ORG).send(),
|
|
||||||
)
|
|
||||||
.await?
|
|
||||||
.with_context(|| format!("list webhooks for org {CONFIG_ORG}"))?;
|
|
||||||
for id in hook_ids_at(&hooks, &own_url) {
|
|
||||||
bounded(
|
|
||||||
&format!("delete webhook {id} on org {CONFIG_ORG}"),
|
|
||||||
client.org_delete_hook(CONFIG_ORG, id).send(),
|
|
||||||
)
|
|
||||||
.await?
|
|
||||||
.with_context(|| format!("delete webhook {id} on org {CONFIG_ORG}"))?;
|
|
||||||
tracing::info!(%own_url, id, "forge: removed this hive's config-pr webhook");
|
|
||||||
}
|
|
||||||
Ok(())
|
|
||||||
}
|
|
||||||
|
|
||||||
/// The URL this hive's config-PR hook was registered under.
|
|
||||||
fn config_pr_hook_url(hive_domain: &str) -> String {
|
|
||||||
format!("https://{hive_domain}/webhook/config-pr")
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Ids of the hooks whose URL is exactly `url`.
|
|
||||||
fn hook_ids_at(hooks: &[forgejo_api::structs::Hook], url: &str) -> Vec<i64> {
|
|
||||||
hooks
|
|
||||||
.iter()
|
|
||||||
.filter(|h| {
|
|
||||||
h.config
|
|
||||||
.as_ref()
|
|
||||||
.and_then(|c| c.get("url"))
|
|
||||||
.is_some_and(|u| u == url)
|
|
||||||
})
|
|
||||||
.filter_map(|h| h.id)
|
|
||||||
.collect()
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Poll the local Forgejo API until it answers, with a timeout of 1 minute.
|
|
||||||
/// The whole boot provisioning below hits the API, and [`is_present`] only
|
|
||||||
/// confirms the container exists — not that Forgejo is *listening*. A
|
|
||||||
/// `nixos-rebuild` that restarts hive-forge and hive-c0re together races:
|
|
||||||
/// without this gate every `ensure_*` fires at a refused socket and leaves a
|
|
||||||
/// stale "provisioning failed" warning that never clears (the pass is one-shot).
|
|
||||||
/// Returns whether the endpoint became ready before the deadline; on timeout
|
|
||||||
/// the caller proceeds anyway so a genuinely-down forge still surfaces its real
|
|
||||||
/// errors. Uses the unauthenticated `get_version` endpoint (no token yet).
|
|
||||||
async fn wait_until_ready() -> bool {
|
|
||||||
const READY_TIMEOUT: Duration = Duration::from_mins(1);
|
|
||||||
const POLL_INTERVAL: Duration = Duration::from_millis(500);
|
|
||||||
let Ok(url) = Url::parse(forge_http_base()) else {
|
|
||||||
tracing::warn!("forge: HIVE_FORGE_URL is not a valid URL; skipping readiness wait");
|
|
||||||
return false;
|
|
||||||
};
|
|
||||||
let Ok(client) = Forgejo::new(Auth::None, url) else {
|
|
||||||
tracing::warn!("forge: could not build readiness client; skipping readiness wait");
|
|
||||||
return false;
|
|
||||||
};
|
|
||||||
let deadline = Instant::now() + READY_TIMEOUT;
|
|
||||||
let mut waited = false;
|
|
||||||
loop {
|
|
||||||
if bounded("forge readiness probe", client.get_version())
|
|
||||||
.await
|
|
||||||
.is_ok_and(|probe| probe.is_ok())
|
|
||||||
{
|
|
||||||
if waited {
|
|
||||||
tracing::info!("forge: endpoint ready, proceeding with provisioning");
|
|
||||||
}
|
|
||||||
return true;
|
|
||||||
}
|
|
||||||
if Instant::now() >= deadline {
|
|
||||||
tracing::warn!(
|
|
||||||
"forge: endpoint not ready after {READY_TIMEOUT:?}; provisioning anyway"
|
|
||||||
);
|
|
||||||
return false;
|
|
||||||
}
|
|
||||||
waited = true;
|
|
||||||
tokio::time::sleep(POLL_INTERVAL).await;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Sweep the forge at startup: seed the `core` admin user (hive-c0re's own
|
|
||||||
/// identity for driving the API, and the one nix's git reads as) and its token,
|
|
||||||
/// then its avatar. Agent accounts,
|
|
||||||
/// tokens and `agent-configs/<name>` repos are the swarm-controller's, as is
|
|
||||||
/// the `agent-configs` org; this sweep touches none of them.
|
|
||||||
/// Called once at hive-c0re startup. Per-step failures are logged
|
|
||||||
/// but don't abort the sweep.
|
|
||||||
pub async fn ensure_all() {
|
pub async fn ensure_all() {
|
||||||
if !is_present().await {
|
if publish_nix_credential().await {
|
||||||
tracing::debug!("forge: hive-forge container absent, skipping user sweep");
|
tracing::debug!("forge: nix's forge credential is current");
|
||||||
return;
|
|
||||||
}
|
|
||||||
// The container exists, but its HTTP may not be up yet (a rebuild restarts
|
|
||||||
// hive-forge + hive-c0re together). Wait for the API to answer before the
|
|
||||||
// provisioning pass so a boot race doesn't leave stale failure warnings.
|
|
||||||
wait_until_ready().await;
|
|
||||||
let core_token = match ensure_core_user_and_token().await {
|
|
||||||
Ok(t) => Some(t),
|
|
||||||
Err(e) => {
|
|
||||||
tracing::warn!(error = ?e, "forge: ensure_core_user_and_token failed");
|
|
||||||
crate::warnings::set_boot_warning(
|
|
||||||
"forge_ensure_core_user",
|
|
||||||
"crit",
|
|
||||||
format!("forge: core user/token provisioning failed: {e}"),
|
|
||||||
);
|
|
||||||
None
|
|
||||||
}
|
|
||||||
};
|
|
||||||
if let Some(token) = core_token.as_deref() {
|
|
||||||
ensure_all_orgs_and_repos(token).await;
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
#[cfg(test)]
|
#[cfg(test)]
|
||||||
mod tests {
|
mod tests {
|
||||||
use super::{
|
use super::{API_TIMEOUT, bounded, git_url_with_base};
|
||||||
API_TIMEOUT, bounded, config_pr_hook_url, describe_forge_admin, git_url_with_base,
|
|
||||||
hook_ids_at,
|
|
||||||
};
|
|
||||||
|
|
||||||
fn hook(id: i64, url: &str) -> forgejo_api::structs::Hook {
|
|
||||||
serde_json::from_value(serde_json::json!({ "id": id, "config": { "url": url } }))
|
|
||||||
.expect("hook json")
|
|
||||||
}
|
|
||||||
|
|
||||||
/// A forge that never answers surfaces as an error naming the operation,
|
/// A forge that never answers surfaces as an error naming the operation,
|
||||||
/// at exactly [`API_TIMEOUT`] — not as a hang.
|
/// at exactly [`API_TIMEOUT`] — not as a hang.
|
||||||
|
|
@ -389,21 +187,6 @@ mod tests {
|
||||||
assert_eq!(failed, Err("409"));
|
assert_eq!(failed, Err("409"));
|
||||||
}
|
}
|
||||||
|
|
||||||
/// The same path on another base is another hive's hook, and removing it
|
|
||||||
/// would cut that hive off. Only an exact URL match is this hive's.
|
|
||||||
#[test]
|
|
||||||
fn only_this_hives_own_config_pr_hook_is_removed() {
|
|
||||||
let own = config_pr_hook_url("hive.example");
|
|
||||||
let hooks = [
|
|
||||||
hook(1, &own),
|
|
||||||
hook(2, "https://other-hive.example/webhook/config-pr"),
|
|
||||||
hook(3, "http://127.0.0.1:7000/webhook/config-pr"),
|
|
||||||
hook(4, "https://swarm.example/webhook/forge/config-pr"),
|
|
||||||
];
|
|
||||||
assert_eq!(hook_ids_at(&hooks, &own), vec![1]);
|
|
||||||
assert!(hook_ids_at(&hooks[1..], &own).is_empty());
|
|
||||||
}
|
|
||||||
|
|
||||||
/// The remote carries **no credential**. `argv` is world-readable through
|
/// The remote carries **no credential**. `argv` is world-readable through
|
||||||
/// `/proc/<pid>/cmdline`, so a token spliced in here would be published to
|
/// `/proc/<pid>/cmdline`, so a token spliced in here would be published to
|
||||||
/// every local user for the life of the git child.
|
/// every local user for the life of the git child.
|
||||||
|
|
@ -426,58 +209,22 @@ mod tests {
|
||||||
}
|
}
|
||||||
|
|
||||||
/// The credential goes in an `Authorization` header instead — decodable
|
/// The credential goes in an `Authorization` header instead — decodable
|
||||||
/// back to `core:<token>`, so the swap is genuinely equivalent auth and
|
/// back to `<login>:<token>`, so the swap is genuinely equivalent auth and
|
||||||
/// not a silent downgrade to anonymous.
|
/// not a silent downgrade to anonymous.
|
||||||
#[test]
|
#[test]
|
||||||
fn basic_auth_header_is_basic_core_token() {
|
fn basic_auth_header_is_basic_login_token() {
|
||||||
use base64::Engine as _;
|
use base64::Engine as _;
|
||||||
let header = super::basic_auth_header("core", "s3cret");
|
let header = super::basic_auth_header("hive-alpha", "s3cret");
|
||||||
let b64 = header
|
let b64 = header
|
||||||
.strip_prefix("Authorization: Basic ")
|
.strip_prefix("Authorization: Basic ")
|
||||||
.expect("basic auth header");
|
.expect("basic auth header");
|
||||||
let decoded = base64::engine::general_purpose::STANDARD
|
let decoded = base64::engine::general_purpose::STANDARD
|
||||||
.decode(b64)
|
.decode(b64)
|
||||||
.expect("valid base64");
|
.expect("valid base64");
|
||||||
assert_eq!(String::from_utf8_lossy(&decoded), "core:s3cret");
|
assert_eq!(String::from_utf8_lossy(&decoded), "hive-alpha:s3cret");
|
||||||
assert!(
|
assert!(
|
||||||
!header.contains("s3cret"),
|
!header.contains("s3cret"),
|
||||||
"token not in the clear: {header}"
|
"token not in the clear: {header}"
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn describe_keeps_the_verb_path_and_drops_every_value() {
|
|
||||||
assert_eq!(
|
|
||||||
describe_forge_admin(&[
|
|
||||||
"user",
|
|
||||||
"create",
|
|
||||||
"--username",
|
|
||||||
"iris",
|
|
||||||
"--email",
|
|
||||||
"iris@hyperhive.local",
|
|
||||||
"--random-password",
|
|
||||||
]),
|
|
||||||
"forgejo admin user create"
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn describe_does_not_reproduce_a_password_argument() {
|
|
||||||
let described = describe_forge_admin(&[
|
|
||||||
"user",
|
|
||||||
"change-password",
|
|
||||||
"--username",
|
|
||||||
"iris",
|
|
||||||
"--password",
|
|
||||||
"correct-horse-battery-staple",
|
|
||||||
]);
|
|
||||||
assert_eq!(described, "forgejo admin user change-password");
|
|
||||||
assert!(!described.contains("correct-horse-battery-staple"));
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn describe_survives_a_leading_flag_and_an_empty_vector() {
|
|
||||||
assert_eq!(describe_forge_admin(&["--help"]), "forgejo admin");
|
|
||||||
assert_eq!(describe_forge_admin(&[]), "forgejo admin");
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -1,16 +1,16 @@
|
||||||
//! The credential this hive reads the forge with: its own read-only account,
|
//! The credential this hive reads the forge with: its own read-only account,
|
||||||
//! whose token `swarm-controller` mints into the store at
|
//! whose token `swarm-controller` mints into the store at
|
||||||
//! `swarm/hives/<hive>/forge/token`, and the core token file where the store
|
//! `swarm/hives/<hive>/forge/token`.
|
||||||
//! holds none.
|
|
||||||
//!
|
//!
|
||||||
//! The file is the fallback so a hive deployed before the controller mints its
|
//! nix's `git` cannot reach the store, so [`ReaderToken::publish_for_nix`]
|
||||||
//! token keeps the reads it had: the forge host holds the file, no other hive
|
//! copies the token to a runtime file that `git-credential-hive-forge`
|
||||||
//! does.
|
//! (`nix/host-modules/lib/hive-forge-git.nix`) reads.
|
||||||
|
|
||||||
use super::{basic_auth_header, core_token};
|
use std::path::Path;
|
||||||
|
|
||||||
/// The basic-auth user [`ReaderToken::auth_header`] names for the core token.
|
use anyhow::Result;
|
||||||
const CORE_LOGIN: &str = "core";
|
|
||||||
|
use super::basic_auth_header;
|
||||||
|
|
||||||
/// A forge token and the account it belongs to.
|
/// A forge token and the account it belongs to.
|
||||||
///
|
///
|
||||||
|
|
@ -39,37 +39,59 @@ impl ReaderToken {
|
||||||
pub(crate) fn api(&self) -> anyhow::Result<forgejo_api::Forgejo> {
|
pub(crate) fn api(&self) -> anyhow::Result<forgejo_api::Forgejo> {
|
||||||
super::api(&self.value)
|
super::api(&self.value)
|
||||||
}
|
}
|
||||||
}
|
|
||||||
|
|
||||||
/// The token to read the forge with, or `None` when this hive has neither.
|
/// Copy this token to the file nix's `git` authenticates from
|
||||||
///
|
/// ([`crate::paths::FORGE_HIVE_TOKEN`]). Whether the file now holds it; a
|
||||||
/// 🩸 Logs the store path and which source won, never a value.
|
/// failed write is logged and leaves the file as it was.
|
||||||
pub async fn reader_token() -> Option<ReaderToken> {
|
pub(crate) fn publish_for_nix(&self) -> bool {
|
||||||
pick(stored_reader_token().await, core_token())
|
let path = Path::new(crate::paths::FORGE_HIVE_TOKEN);
|
||||||
}
|
match self.write_credential(path) {
|
||||||
|
Ok(()) => true,
|
||||||
/// The store's token wins over the core file's.
|
Err(e) => {
|
||||||
fn pick(stored: Option<ReaderToken>, core: Option<String>) -> Option<ReaderToken> {
|
tracing::warn!(path = %path.display(), error = ?e, "forge: could not write the nix forge credential");
|
||||||
let picked = stored.or_else(|| {
|
false
|
||||||
core.map(|value| ReaderToken {
|
|
||||||
login: CORE_LOGIN.to_owned(),
|
|
||||||
value,
|
|
||||||
})
|
|
||||||
});
|
|
||||||
if let Some(t) = &picked {
|
|
||||||
tracing::debug!(login = %t.login, "forge: reading as this account");
|
|
||||||
} else {
|
|
||||||
tracing::debug!("forge: no read token in the store or the core token file");
|
|
||||||
}
|
}
|
||||||
picked
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Login on the first line, token on the second. Written only when the
|
||||||
|
/// content differs, so a build's resolve is not a disk write.
|
||||||
|
fn write_credential(&self, path: &Path) -> Result<()> {
|
||||||
|
let content = format!("{}\n{}\n", self.login, self.value);
|
||||||
|
if std::fs::read(path).is_ok_and(|have| have == content.as_bytes()) {
|
||||||
|
return Ok(());
|
||||||
|
}
|
||||||
|
crate::secret_file::write_secret_0600(path, content.as_bytes())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The token to read the forge with, or `None` when the store holds none for
|
||||||
|
/// this hive.
|
||||||
|
///
|
||||||
|
/// 🩸 Logs the store path, never a value.
|
||||||
|
pub async fn reader_token() -> Option<ReaderToken> {
|
||||||
|
let token = stored_reader_token().await;
|
||||||
|
if let Some(t) = &token {
|
||||||
|
tracing::debug!(login = %t.login, "forge: reading as this account");
|
||||||
|
}
|
||||||
|
token
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Publish this hive's read token for nix; see [`ReaderToken::publish_for_nix`].
|
||||||
|
/// Whether the file holds a token.
|
||||||
|
pub async fn publish_nix_credential() -> bool {
|
||||||
|
let Some(token) = reader_token().await else {
|
||||||
|
tracing::debug!("forge: no hive read token in the store; nix has no forge credential");
|
||||||
|
return false;
|
||||||
|
};
|
||||||
|
token.publish_for_nix()
|
||||||
}
|
}
|
||||||
|
|
||||||
/// This hive's token from the store, under the hive's own identity.
|
/// This hive's token from the store, under the hive's own identity.
|
||||||
///
|
///
|
||||||
/// `None`, never an error, for every way it can come up empty — no hive name,
|
/// `None`, never an error, for every way it can come up empty — no hive name,
|
||||||
/// no `BAO_*` identity, an unreachable store, nothing at the path — the same
|
/// no `BAO_*` identity, an unreachable store, nothing at the path — the same
|
||||||
/// as `matrix::stored_sender_token`, and for the same reason: each sends the
|
/// as `matrix::stored_sender_token`.
|
||||||
/// caller to the fallback.
|
|
||||||
async fn stored_reader_token() -> Option<ReaderToken> {
|
async fn stored_reader_token() -> Option<ReaderToken> {
|
||||||
let hive = std::env::var("HYPERHIVE_HIVE_NAME")
|
let hive = std::env::var("HYPERHIVE_HIVE_NAME")
|
||||||
.ok()
|
.ok()
|
||||||
|
|
@ -110,6 +132,7 @@ async fn stored_reader_token() -> Option<ReaderToken> {
|
||||||
#[cfg(test)]
|
#[cfg(test)]
|
||||||
mod tests {
|
mod tests {
|
||||||
use super::*;
|
use super::*;
|
||||||
|
use std::os::unix::fs::PermissionsExt as _;
|
||||||
|
|
||||||
fn stored() -> ReaderToken {
|
fn stored() -> ReaderToken {
|
||||||
ReaderToken {
|
ReaderToken {
|
||||||
|
|
@ -118,23 +141,71 @@ mod tests {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
fn mode(path: &Path) -> u32 {
|
||||||
fn the_store_wins_over_the_core_file() {
|
std::fs::metadata(path).expect("stat").permissions().mode() & 0o777
|
||||||
let t = pick(Some(stored()), Some("from-the-file".to_owned())).expect("a token");
|
|
||||||
assert_eq!(t.value, "from-the-store");
|
|
||||||
assert_eq!(t.login, "hive-alpha");
|
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn the_core_file_is_the_fallback() {
|
fn the_nix_credential_is_the_hives_login_then_its_token_at_0600() {
|
||||||
let t = pick(None, Some("from-the-file".to_owned())).expect("a token");
|
let dir = tempfile::tempdir().expect("tempdir");
|
||||||
assert_eq!(t.value, "from-the-file");
|
let path = dir.path().join("forge-hive-token");
|
||||||
assert_eq!(t.login, CORE_LOGIN);
|
stored().write_credential(&path).expect("written");
|
||||||
|
assert_eq!(
|
||||||
|
std::fs::read_to_string(&path).expect("read"),
|
||||||
|
"hive-alpha\nfrom-the-store\n"
|
||||||
|
);
|
||||||
|
assert_eq!(mode(&path), 0o600);
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn neither_is_none() {
|
fn a_rotated_token_replaces_the_nix_credential() {
|
||||||
assert!(pick(None, None).is_none());
|
let dir = tempfile::tempdir().expect("tempdir");
|
||||||
|
let path = dir.path().join("forge-hive-token");
|
||||||
|
stored().write_credential(&path).expect("first");
|
||||||
|
let rotated = ReaderToken {
|
||||||
|
login: "hive-alpha".to_owned(),
|
||||||
|
value: "rotated".to_owned(),
|
||||||
|
};
|
||||||
|
rotated.write_credential(&path).expect("second");
|
||||||
|
assert_eq!(
|
||||||
|
std::fs::read_to_string(&path).expect("read"),
|
||||||
|
"hive-alpha\nrotated\n"
|
||||||
|
);
|
||||||
|
assert_eq!(mode(&path), 0o600);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn an_unchanged_credential_is_not_rewritten() {
|
||||||
|
let dir = tempfile::tempdir().expect("tempdir");
|
||||||
|
let path = dir.path().join("forge-hive-token");
|
||||||
|
stored().write_credential(&path).expect("first");
|
||||||
|
// A rewrite goes through a sibling temp file and a rename, which
|
||||||
|
// gives the file a new inode.
|
||||||
|
let before = std::fs::metadata(&path).expect("stat");
|
||||||
|
stored().write_credential(&path).expect("second");
|
||||||
|
let after = std::fs::metadata(&path).expect("stat");
|
||||||
|
assert_eq!(
|
||||||
|
std::os::unix::fs::MetadataExt::ino(&before),
|
||||||
|
std::os::unix::fs::MetadataExt::ino(&after)
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// `hive-forge-git.nix`'s credential helper names this path as a literal.
|
||||||
|
#[test]
|
||||||
|
fn the_nix_credential_lives_on_the_runtime_tmpfs_where_the_helper_reads_it() {
|
||||||
|
assert_eq!(
|
||||||
|
crate::paths::FORGE_HIVE_TOKEN,
|
||||||
|
"/run/hyperhive/forge-hive-token"
|
||||||
|
);
|
||||||
|
assert!(crate::paths::FORGE_HIVE_TOKEN.starts_with(crate::paths::RUNTIME_ROOT));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_path_that_cannot_be_written_is_an_error_not_a_panic() {
|
||||||
|
let dir = tempfile::tempdir().expect("tempdir");
|
||||||
|
let file = dir.path().join("a-file");
|
||||||
|
std::fs::write(&file, "").expect("file");
|
||||||
|
assert!(stored().write_credential(&file.join("sub")).is_err());
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
|
|
|
||||||
|
|
@ -1,389 +0,0 @@
|
||||||
//! The bootstrap `core` admin user: its account, access-token lifecycle
|
|
||||||
//! and avatar upload. The typed API-client constructor + `forgejo admin`
|
|
||||||
//! helpers live in the module root (`super`).
|
|
||||||
|
|
||||||
use std::path::Path;
|
|
||||||
|
|
||||||
use anyhow::{Context, Result};
|
|
||||||
use base64::Engine;
|
|
||||||
use forgejo_api::structs::UpdateUserAvatarOption;
|
|
||||||
use forgejo_api::{ApiErrorKind, ForgejoError};
|
|
||||||
use reqwest::StatusCode;
|
|
||||||
|
|
||||||
use super::{api, bounded, forge_admin};
|
|
||||||
|
|
||||||
const TOKEN_NAME_PREFIX: &str = "hyperhive";
|
|
||||||
// Where the host-side `core` admin token lives. Used by hive-c0re itself
|
|
||||||
// to drive admin API calls and as the credential nix's git reads. Root-only. Literal lives
|
|
||||||
// in `crate::paths`; aliased here under the long-standing name.
|
|
||||||
use crate::paths::FORGE_CORE_TOKEN as CORE_TOKEN_PATH;
|
|
||||||
// The `forge/core-avatar-set` marker lives in `crate::paths` — a one-shot
|
|
||||||
// guard: the upload runs once, the marker is written, subsequent startups
|
|
||||||
// skip. Delete it to force a re-upload.
|
|
||||||
// The avatar PNG path is resolved by `core_avatar_png_path` below —
|
|
||||||
// only-used-here, so it lives in this module rather than the shared
|
|
||||||
// `hive_sh4re::assets` helpers (which now hold only `prompt_template`, the
|
|
||||||
// one asset path every crate needs — the agent icon, the last other one, moved off the
|
|
||||||
// shared-asset model entirely; see `hive-agent::web_ui::screen`).
|
|
||||||
|
|
||||||
/// `$HIVE_ASSETS_DIR/branding/hyperhive.png` — the core mark,
|
|
||||||
/// rasterised. Not independently configurable — override the whole
|
|
||||||
/// `services.hyperhive.c0re.assets` package to change it.
|
|
||||||
fn core_avatar_png_path() -> std::path::PathBuf {
|
|
||||||
let dir = std::env::var("HIVE_ASSETS_DIR").expect(
|
|
||||||
"HIVE_ASSETS_DIR is unset — the hyperhive NixOS module sets it from \
|
|
||||||
services.hyperhive.c0re.assets on the hive-c0re unit, so this \
|
|
||||||
process was started outside that unit",
|
|
||||||
);
|
|
||||||
std::path::PathBuf::from(dir).join("branding/hyperhive.png")
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Bootstrap `core` token scopes — the same set as swarm-controller's
|
|
||||||
/// `AGENT_TOKEN_SCOPES`, with no admin scope: nothing in hive-c0re calls
|
|
||||||
/// `/api/v1/admin/*`. See `docs/integrations/forge.md::Token scopes`.
|
|
||||||
const CORE_TOKEN_SCOPES: &str = "read:user,write:user,read:notification,write:notification,write:repository,write:issue,write:organization,write:misc";
|
|
||||||
|
|
||||||
/// Pull the access token out of forgejo's success message. Format
|
|
||||||
/// has shifted across versions (table form vs. "Access token was
|
|
||||||
/// successfully created: `<hex>`"), so just hunt the output for the
|
|
||||||
/// first long hex-looking word.
|
|
||||||
fn extract_token(output: &str) -> Option<String> {
|
|
||||||
output
|
|
||||||
.split(|c: char| c.is_whitespace() || c == ',' || c == ':')
|
|
||||||
.find(|w| w.len() >= 32 && w.chars().all(|c| c.is_ascii_hexdigit()))
|
|
||||||
.map(str::to_owned)
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Canonical email address for a hive agent's Forgejo account.
|
|
||||||
/// Must match the `user.email` set by `hive_meta_flake::render_flake` so commits
|
|
||||||
/// by the agent link back to their Forgejo profile page.
|
|
||||||
fn agent_email(name: &str) -> String {
|
|
||||||
format!("{name}@hyperhive.local")
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Ensure a forgejo user named `name` exists. Idempotent: forgejo
|
|
||||||
/// returns a "user already exists" error which we treat as success.
|
|
||||||
/// `admin` adds `--admin` (site admin) — used for the bootstrap
|
|
||||||
/// `core` user that drives the API. The password is random and never
|
|
||||||
/// read: the account authenticates by token.
|
|
||||||
async fn ensure_user_exists(name: &str, admin: bool) -> Result<()> {
|
|
||||||
let email = agent_email(name);
|
|
||||||
let mut args = vec![
|
|
||||||
"user",
|
|
||||||
"create",
|
|
||||||
"--username",
|
|
||||||
name,
|
|
||||||
"--email",
|
|
||||||
&email,
|
|
||||||
"--random-password",
|
|
||||||
"--must-change-password=false",
|
|
||||||
];
|
|
||||||
if admin {
|
|
||||||
args.push("--admin");
|
|
||||||
}
|
|
||||||
let result = forge_admin(&args).await;
|
|
||||||
match result {
|
|
||||||
Ok(_) => {
|
|
||||||
tracing::info!(%name, "forge: created user");
|
|
||||||
Ok(())
|
|
||||||
}
|
|
||||||
Err(e) => {
|
|
||||||
// Forgejo's "already exists" error wording varies; just
|
|
||||||
// try the next step and let token issuance surface a
|
|
||||||
// real failure if the user truly isn't there.
|
|
||||||
let msg = format!("{e:#}");
|
|
||||||
if msg.contains("already exists") || msg.contains("user already") {
|
|
||||||
tracing::debug!(%name, "forge: user already exists");
|
|
||||||
Ok(())
|
|
||||||
} else {
|
|
||||||
tracing::warn!(%name, error = %msg, "forge: user create unclear; trying token anyway");
|
|
||||||
Ok(())
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Mint a fresh access token for `name`. Token name is suffixed with
|
|
||||||
/// a monotonic clock so re-issuing doesn't collide with an existing
|
|
||||||
/// token of the same name in the DB. `scopes` is the scope string
|
|
||||||
/// passed to `forgejo admin user generate-access-token --scopes`;
|
|
||||||
/// the bootstrap `core` user, its one caller, passes `CORE_TOKEN_SCOPES`.
|
|
||||||
async fn mint_token(name: &str, scopes: &str) -> Result<String> {
|
|
||||||
let token_name = format!(
|
|
||||||
"{TOKEN_NAME_PREFIX}-{}",
|
|
||||||
std::time::SystemTime::now()
|
|
||||||
.duration_since(std::time::UNIX_EPOCH)
|
|
||||||
.map_or(0, |d| d.as_secs())
|
|
||||||
);
|
|
||||||
let stdout = forge_admin(&[
|
|
||||||
"user",
|
|
||||||
"generate-access-token",
|
|
||||||
"--username",
|
|
||||||
name,
|
|
||||||
"--token-name",
|
|
||||||
&token_name,
|
|
||||||
"--scopes",
|
|
||||||
scopes,
|
|
||||||
])
|
|
||||||
.await?;
|
|
||||||
// Deliberately does NOT include `stdout`: on this path forgejo has
|
|
||||||
// already minted a live token and printed it, and an error string
|
|
||||||
// propagates into logs the same way any other message does. The
|
|
||||||
// shape of the output is enough to diagnose a version drift.
|
|
||||||
let token = extract_token(&stdout).with_context(|| {
|
|
||||||
format!(
|
|
||||||
"no token-shaped word (>= 32 hex chars) in forgejo output \
|
|
||||||
({} bytes, {} lines); output withheld, it carries the token",
|
|
||||||
stdout.len(),
|
|
||||||
stdout.lines().count()
|
|
||||||
)
|
|
||||||
})?;
|
|
||||||
tracing::debug!(%name, %token_name, "forge: minted access token");
|
|
||||||
Ok(token)
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Mint a fresh Forgejo access token for the `core` admin user and
|
|
||||||
/// write it directly to `path`. Unlike agent tokens this path is owned
|
|
||||||
/// by hive-c0re itself (under `/var/lib/hyperhive/`), so a direct
|
|
||||||
/// write is both correct and necessary (no priv round-trip).
|
|
||||||
async fn mint_and_persist_core_token(path: &Path) -> Result<()> {
|
|
||||||
let token = mint_token("core", CORE_TOKEN_SCOPES).await?;
|
|
||||||
crate::secret_file::write_secret_0600(path, format!("{token}\n").as_bytes())
|
|
||||||
.with_context(|| format!("write core token to {}", path.display()))?;
|
|
||||||
tracing::info!(path = %path.display(), "forge: persisted core access token");
|
|
||||||
Ok(())
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Set `core`'s Forgejo avatar to the hyperhive logo once, then
|
|
||||||
/// remember it so subsequent startups don't re-upload. Best-effort
|
|
||||||
/// — any non-2xx is logged at the caller; the project runs fine
|
|
||||||
/// with the default hash identicon.
|
|
||||||
pub(super) async fn ensure_core_avatar(token: &str) -> Result<()> {
|
|
||||||
let marker = crate::paths::forge_core_avatar_marker();
|
|
||||||
if marker.exists() {
|
|
||||||
return Ok(());
|
|
||||||
}
|
|
||||||
let png_path = core_avatar_png_path();
|
|
||||||
let png_bytes = tokio::fs::read(&png_path)
|
|
||||||
.await
|
|
||||||
.with_context(|| format!("read core avatar PNG from {}", png_path.display()))?;
|
|
||||||
// The raw-HTTP predecessor POSTed the admin endpoint
|
|
||||||
// `/admin/users/core/avatar`, which forgejo-api has no method for.
|
|
||||||
// `token` IS the core user's own token though, so updating "the
|
|
||||||
// current user's avatar" (`POST /user/avatar`) is behaviorally
|
|
||||||
// identical.
|
|
||||||
bounded(
|
|
||||||
"set core avatar",
|
|
||||||
api(token)?.user_update_avatar(UpdateUserAvatarOption {
|
|
||||||
image: Some(base64::engine::general_purpose::STANDARD.encode(&png_bytes)),
|
|
||||||
}),
|
|
||||||
)
|
|
||||||
.await?
|
|
||||||
.context("set core avatar")?;
|
|
||||||
if let Err(e) = write_marker(&marker) {
|
|
||||||
tracing::warn!(marker = %marker.display(), error = ?e, "forge: could not write the avatar marker; the upload repeats on the next start");
|
|
||||||
}
|
|
||||||
tracing::info!("forge: set core user avatar to hyperhive logo");
|
|
||||||
Ok(())
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Create the one-shot marker file, and its directory if needed.
|
|
||||||
fn write_marker(marker: &Path) -> std::io::Result<()> {
|
|
||||||
if let Some(parent) = marker.parent() {
|
|
||||||
std::fs::create_dir_all(parent)?;
|
|
||||||
}
|
|
||||||
std::fs::write(marker, "")
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Outcome of probing whether the persisted core token still works
|
|
||||||
/// against the *current* forge. Existence on disk is not validity: a
|
|
||||||
/// token minted before a forge rebuild / re-provision is unknown to the
|
|
||||||
/// new forge's DB and 401s on every call.
|
|
||||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
|
||||||
enum CoreTokenCheck {
|
|
||||||
/// Token authenticated successfully — keep using it.
|
|
||||||
Valid,
|
|
||||||
/// Forge explicitly rejected the token (401/403) — re-mint.
|
|
||||||
Invalid,
|
|
||||||
/// Couldn't determine (forge unreachable / 5xx). Don't re-mint on a
|
|
||||||
/// transient: keep the existing token and let a later ensure pass
|
|
||||||
/// re-check once the forge is responsive. Re-minting here would both
|
|
||||||
/// fail (mint needs the forge too) and churn tokens needlessly.
|
|
||||||
Indeterminate,
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Map a failed token-probe call to a [`CoreTokenCheck`]. Only a
|
|
||||||
/// definitive auth rejection (401/403 — surfaced by the typed client
|
|
||||||
/// as `Unauthorized`/`Forbidden` API errors, or defensively as bare
|
|
||||||
/// `UnexpectedStatusCode`s) is `Invalid`; anything else (transport,
|
|
||||||
/// 5xx, unexpected shapes) is `Indeterminate`. Pure so the decision
|
|
||||||
/// logic is unit-testable without a live forge.
|
|
||||||
fn classify_core_token_error(e: &ForgejoError) -> CoreTokenCheck {
|
|
||||||
match e {
|
|
||||||
ForgejoError::ApiError(api) => match api.error_kind() {
|
|
||||||
ApiErrorKind::Unauthorized | ApiErrorKind::Forbidden => CoreTokenCheck::Invalid,
|
|
||||||
_ => CoreTokenCheck::Indeterminate,
|
|
||||||
},
|
|
||||||
ForgejoError::UnexpectedStatusCode(s)
|
|
||||||
if *s == StatusCode::UNAUTHORIZED || *s == StatusCode::FORBIDDEN =>
|
|
||||||
{
|
|
||||||
CoreTokenCheck::Invalid
|
|
||||||
}
|
|
||||||
_ => CoreTokenCheck::Indeterminate,
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Probe whether `token` is still accepted by the current forge with a
|
|
||||||
/// cheap authenticated `user_get_current` (`GET /api/v1/user`, covered
|
|
||||||
/// by the core token's `read:user` scope). See [`CoreTokenCheck`] for
|
|
||||||
/// how the outcome is interpreted.
|
|
||||||
async fn check_core_token(token: &str) -> CoreTokenCheck {
|
|
||||||
let Ok(client) = api(token) else {
|
|
||||||
return CoreTokenCheck::Indeterminate;
|
|
||||||
};
|
|
||||||
let Ok(probe) = bounded("probe core token", client.user_get_current()).await else {
|
|
||||||
tracing::debug!("forge: core-token probe timed out; treating as indeterminate");
|
|
||||||
return CoreTokenCheck::Indeterminate;
|
|
||||||
};
|
|
||||||
match probe {
|
|
||||||
Ok(_) => CoreTokenCheck::Valid,
|
|
||||||
Err(e) => {
|
|
||||||
let outcome = classify_core_token_error(&e);
|
|
||||||
if outcome == CoreTokenCheck::Indeterminate {
|
|
||||||
tracing::debug!(
|
|
||||||
error = %e,
|
|
||||||
"forge: core-token probe inconclusive (unreachable / unexpected response); \
|
|
||||||
treating as indeterminate"
|
|
||||||
);
|
|
||||||
}
|
|
||||||
outcome
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Ensure the bootstrap `core` admin user + a token at
|
|
||||||
/// `CORE_TOKEN_PATH`. The token is what hive-c0re uses for forgejo
|
|
||||||
/// API calls (meta-repo push, collaborator grants).
|
|
||||||
/// Returns the token.
|
|
||||||
///
|
|
||||||
/// Idempotent, but validity-aware: when a token file is already present
|
|
||||||
/// it is **probed against the current forge** before being trusted. A
|
|
||||||
/// token persisted before a forge rebuild / re-provision is stale (the
|
|
||||||
/// new forge DB doesn't know it) and would 401 every caller — so on a
|
|
||||||
/// definitive rejection the token is re-minted. A merely-unreachable
|
|
||||||
/// forge leaves the existing token in place (a later ensure pass
|
|
||||||
/// re-checks) rather than churning tokens on a transient.
|
|
||||||
pub(super) async fn ensure_core_user_and_token() -> Result<String> {
|
|
||||||
let path = std::path::Path::new(CORE_TOKEN_PATH);
|
|
||||||
if let Ok(existing) = std::fs::read_to_string(path) {
|
|
||||||
let trimmed = existing.trim().to_owned();
|
|
||||||
if !trimmed.is_empty() {
|
|
||||||
match check_core_token(&trimmed).await {
|
|
||||||
CoreTokenCheck::Valid | CoreTokenCheck::Indeterminate => return Ok(trimmed),
|
|
||||||
CoreTokenCheck::Invalid => {
|
|
||||||
tracing::warn!(
|
|
||||||
path = %path.display(),
|
|
||||||
"forge: persisted core token rejected by forge (stale after rebuild?); \
|
|
||||||
re-minting"
|
|
||||||
);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
ensure_user_exists("core", true).await?;
|
|
||||||
mint_and_persist_core_token(path).await?;
|
|
||||||
let raw = std::fs::read_to_string(path)
|
|
||||||
.with_context(|| format!("read {CORE_TOKEN_PATH} after mint"))?;
|
|
||||||
Ok(raw.trim().to_owned())
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Read the persisted core token, or None when the forge isn't
|
|
||||||
/// seeded yet. Cheap — just a file read.
|
|
||||||
pub fn core_token() -> Option<String> {
|
|
||||||
std::fs::read_to_string(CORE_TOKEN_PATH)
|
|
||||||
.ok()
|
|
||||||
.map(|s| s.trim().to_owned())
|
|
||||||
.filter(|s| !s.is_empty())
|
|
||||||
}
|
|
||||||
|
|
||||||
#[cfg(test)]
|
|
||||||
mod tests {
|
|
||||||
use super::{CoreTokenCheck, classify_core_token_error, write_marker};
|
|
||||||
use forgejo_api::{ApiError, ApiErrorKind, ForgejoError};
|
|
||||||
use reqwest::StatusCode;
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn write_marker_creates_the_missing_parent_dir() {
|
|
||||||
let dir = tempfile::tempdir().expect("tempdir");
|
|
||||||
let marker = dir.path().join("not-yet").join("nested").join("avatar-set");
|
|
||||||
write_marker(&marker).expect("write");
|
|
||||||
assert!(marker.is_file());
|
|
||||||
}
|
|
||||||
|
|
||||||
/// A parent that is a regular file makes `create_dir_all` fail for any
|
|
||||||
/// uid, root included.
|
|
||||||
#[test]
|
|
||||||
fn write_marker_reports_a_path_it_cannot_create() {
|
|
||||||
let dir = tempfile::tempdir().expect("tempdir");
|
|
||||||
let file = dir.path().join("a-file");
|
|
||||||
std::fs::write(&file, "").expect("file");
|
|
||||||
assert!(write_marker(&file.join("sub").join("avatar-set")).is_err());
|
|
||||||
}
|
|
||||||
|
|
||||||
fn api_err(kind: ApiErrorKind) -> ForgejoError {
|
|
||||||
ForgejoError::ApiError(ApiError {
|
|
||||||
message: None,
|
|
||||||
kind,
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn auth_rejection_errors_are_invalid() {
|
|
||||||
// The whole point: a stale token (forge rebuilt out from under it)
|
|
||||||
// 401s, and 401/403 are the only outcomes that trigger a re-mint.
|
|
||||||
assert_eq!(
|
|
||||||
classify_core_token_error(&api_err(ApiErrorKind::Unauthorized)),
|
|
||||||
CoreTokenCheck::Invalid
|
|
||||||
);
|
|
||||||
assert_eq!(
|
|
||||||
classify_core_token_error(&api_err(ApiErrorKind::Forbidden)),
|
|
||||||
CoreTokenCheck::Invalid
|
|
||||||
);
|
|
||||||
// Defensive: the same statuses arriving as bare status codes
|
|
||||||
// (endpoint spec didn't list them) must classify identically.
|
|
||||||
for s in [StatusCode::UNAUTHORIZED, StatusCode::FORBIDDEN] {
|
|
||||||
assert_eq!(
|
|
||||||
classify_core_token_error(&ForgejoError::UnexpectedStatusCode(s)),
|
|
||||||
CoreTokenCheck::Invalid,
|
|
||||||
"status {s} should be invalid"
|
|
||||||
);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn transient_and_unexpected_errors_are_indeterminate() {
|
|
||||||
// Never re-mint on a transient — minting needs the forge too, and
|
|
||||||
// churning tokens on a blip is worse than keeping the existing one.
|
|
||||||
for s in [
|
|
||||||
StatusCode::INTERNAL_SERVER_ERROR,
|
|
||||||
StatusCode::BAD_GATEWAY,
|
|
||||||
StatusCode::SERVICE_UNAVAILABLE,
|
|
||||||
StatusCode::GATEWAY_TIMEOUT,
|
|
||||||
StatusCode::NOT_FOUND,
|
|
||||||
] {
|
|
||||||
assert_eq!(
|
|
||||||
classify_core_token_error(&ForgejoError::UnexpectedStatusCode(s)),
|
|
||||||
CoreTokenCheck::Indeterminate,
|
|
||||||
"status {s} should be indeterminate"
|
|
||||||
);
|
|
||||||
}
|
|
||||||
assert_eq!(
|
|
||||||
classify_core_token_error(&api_err(ApiErrorKind::NotFound { errors: None })),
|
|
||||||
CoreTokenCheck::Indeterminate
|
|
||||||
);
|
|
||||||
assert_eq!(
|
|
||||||
classify_core_token_error(&api_err(ApiErrorKind::Generic)),
|
|
||||||
CoreTokenCheck::Indeterminate
|
|
||||||
);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
@ -119,11 +119,11 @@ pub(super) async fn run_node(
|
||||||
(builder, result)
|
(builder, result)
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Boot-time forge user/token sweep as a DAG node — see
|
/// Boot-time publish of the hive's forge credential for nix, as a DAG node —
|
||||||
/// [`NodeKind::ForgeSweep`]. `forge::ensure_all` already does its own
|
/// see [`NodeKind::ForgeSweep`]. `forge::ensure_all` logs its own outcome and
|
||||||
/// per-step error handling and boot warnings internally (it's
|
/// is best-effort (a build re-publishes the credential before it fetches), so
|
||||||
/// best-effort by design), so this wrapper has nothing left to report;
|
/// this wrapper has nothing left to report; it exists to make the step a
|
||||||
/// it exists purely to make the sweep a visible unit of work.
|
/// visible unit of work.
|
||||||
async fn run_forge_sweep() -> Result<()> {
|
async fn run_forge_sweep() -> Result<()> {
|
||||||
crate::forge::ensure_all().await;
|
crate::forge::ensure_all().await;
|
||||||
Ok(())
|
Ok(())
|
||||||
|
|
|
||||||
|
|
@ -96,8 +96,8 @@ pub enum NodeKind {
|
||||||
/// Write the agent's durable power intent (`wanted = Up` when `up`,
|
/// Write the agent's durable power intent (`wanted = Up` when `up`,
|
||||||
/// else `Offline`) as the head node of a power-op DAG.
|
/// else `Offline`) as the head node of a power-op DAG.
|
||||||
SetWanted { agent: String, up: bool },
|
SetWanted { agent: String, up: bool },
|
||||||
/// One-shot boot-time forge user/token sweep for every existing
|
/// One-shot boot-time publish of the hive's forge read token for nix
|
||||||
/// container (`forge::ensure_all`). Agentless.
|
/// (`forge::ensure_all`). Agentless.
|
||||||
ForgeSweep,
|
ForgeSweep,
|
||||||
/// Matrix user/space sweep (`matrix::ensure_all`): at boot and every 30
|
/// Matrix user/space sweep (`matrix::ensure_all`): at boot and every 30
|
||||||
/// minutes. Agentless.
|
/// minutes. Agentless.
|
||||||
|
|
|
||||||
|
|
@ -259,7 +259,7 @@ async fn cmd_serve(
|
||||||
tracing::warn!(error = ?e, "auto-update task failed");
|
tracing::warn!(error = ?e, "auto-update task failed");
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
// Forge user sweep: now a `NodeKind::ForgeSweep` DAG node (see
|
// Forge credential publish: a `NodeKind::ForgeSweep` DAG node (see
|
||||||
// `workers::auto_update::submit_startup_sweep_nodes`), submitted
|
// `workers::auto_update::submit_startup_sweep_nodes`), submitted
|
||||||
// unconditionally on every boot — moved off a bare `tokio::spawn` so it
|
// unconditionally on every boot — moved off a bare `tokio::spawn` so it
|
||||||
// shows as real work on the dashboard.
|
// shows as real work on the dashboard.
|
||||||
|
|
|
||||||
|
|
@ -19,8 +19,8 @@ use swarm_queue_client::agent_archetype::ArchetypeRef;
|
||||||
|
|
||||||
/// What the forge says about `internal/meta` and an agent's config repo.
|
/// What the forge says about `internal/meta` and an agent's config repo.
|
||||||
pub(crate) trait MetaSource {
|
pub(crate) trait MetaSource {
|
||||||
/// Whether nix's own `git` can authenticate to the forge: it reads as
|
/// Whether nix's own `git` can authenticate to the forge: it reads as this
|
||||||
/// `core`, from the token file, whatever token this hive reads with.
|
/// hive's account, from the credential file the source publishes.
|
||||||
fn nix_can_fetch(&self) -> bool;
|
fn nix_can_fetch(&self) -> bool;
|
||||||
/// The default branch's head commit of `internal/meta`; `None` when the
|
/// The default branch's head commit of `internal/meta`; `None` when the
|
||||||
/// repo is absent or has no commit.
|
/// repo is absent or has no commit.
|
||||||
|
|
|
||||||
|
|
@ -5,9 +5,9 @@
|
||||||
//! modules (`broker.sqlite`, `matrix-sender-token`, `agent-sockets.json`,
|
//! modules (`broker.sqlite`, `matrix-sender-token`, `agent-sockets.json`,
|
||||||
//! …). This module is the **single Rust-side source** for every host
|
//! …). This module is the **single Rust-side source** for every host
|
||||||
//! path — the strictly host-side ones grouped into subdirs (`db/`,
|
//! path — the strictly host-side ones grouped into subdirs (`db/`,
|
||||||
//! `forge/`, `matrix/`, `run/`), plus the **nix-coupled** roots
|
//! `matrix/`, `run/`), plus the **nix-coupled** roots
|
||||||
//! (`agents/`, `applied/`, `meta/`, `shared/`, `gateway/`, `knowledge/`,
|
//! (`agents/`, `applied/`, `meta/`, `shared/`, `gateway/`, `knowledge/`,
|
||||||
//! the register/core tokens, the `/run` runtime dirs).
|
//! the `/run` runtime dirs).
|
||||||
//!
|
//!
|
||||||
//! Nix-coupled paths are NOT excluded (the old stance) — they're moved
|
//! Nix-coupled paths are NOT excluded (the old stance) — they're moved
|
||||||
//! here too, each carrying a `// nix: …` comment naming the module /
|
//! here too, each carrying a `// nix: …` comment naming the module /
|
||||||
|
|
@ -63,18 +63,6 @@ pub fn db_dir() -> PathBuf {
|
||||||
state_root().join("db")
|
state_root().join("db")
|
||||||
}
|
}
|
||||||
|
|
||||||
/// `forge/` — hive-c0re's own forge provisioning markers.
|
|
||||||
#[must_use]
|
|
||||||
pub fn forge_dir() -> PathBuf {
|
|
||||||
state_root().join("forge")
|
|
||||||
}
|
|
||||||
|
|
||||||
/// `forge/core-avatar-set` — marker: core account avatar uploaded.
|
|
||||||
#[must_use]
|
|
||||||
pub fn forge_core_avatar_marker() -> PathBuf {
|
|
||||||
forge_dir().join("core-avatar-set")
|
|
||||||
}
|
|
||||||
|
|
||||||
/// `agent-identity/` — one subdir per agent holding the store identity this
|
/// `agent-identity/` — one subdir per agent holding the store identity this
|
||||||
/// hive collected for it. Staging only: the files exist so `systemd-nspawn`
|
/// hive collected for it. Staging only: the files exist so `systemd-nspawn`
|
||||||
/// has something to `--load-credential` from, and nothing on the host ever
|
/// has something to `--load-credential` from, and nothing on the host ever
|
||||||
|
|
@ -246,12 +234,12 @@ pub fn gateway_agents_conf() -> PathBuf {
|
||||||
gateway_dir().join("agents.conf")
|
gateway_dir().join("agents.conf")
|
||||||
}
|
}
|
||||||
|
|
||||||
/// `forge-core-token` — the hive-c0re forge account API token. A `&str`
|
/// `forge-hive-token` — this hive's read-only forge token (login on the first
|
||||||
/// (used in `Path::new` + user-facing `format!` messages), so it stays a
|
/// line, token on the second), copied from the swarm secret store for nix's
|
||||||
/// const rather than a `PathBuf` fn.
|
/// `git`. Tmpfs: re-published on every boot.
|
||||||
// nix: read at provisioning; the same literal is in
|
// nix: read by `git-credential-hive-forge`; the same literal is in
|
||||||
// `nix/host-modules/hive-c0re/default.nix` — must match.
|
// `nix/host-modules/lib/hive-forge-git.nix` — must match.
|
||||||
pub const FORGE_CORE_TOKEN: &str = "/var/lib/hyperhive/forge-core-token";
|
pub const FORGE_HIVE_TOKEN: &str = "/run/hyperhive/forge-hive-token";
|
||||||
|
|
||||||
/// `/run/hyperhive` — the runtime root (host admin socket + per-agent dirs).
|
/// `/run/hyperhive` — the runtime root (host admin socket + per-agent dirs).
|
||||||
#[must_use]
|
#[must_use]
|
||||||
|
|
@ -282,10 +270,9 @@ pub fn agent_runtime_dir(name: &str) -> PathBuf {
|
||||||
/// within the same filesystem is atomic.
|
/// within the same filesystem is atomic.
|
||||||
pub fn relocate_legacy_state() {
|
pub fn relocate_legacy_state() {
|
||||||
let root = state_root();
|
let root = state_root();
|
||||||
let moves: [(&str, PathBuf); 6] = [
|
let moves: [(&str, PathBuf); 5] = [
|
||||||
("broker.sqlite", db_dir().join("broker.sqlite")),
|
("broker.sqlite", db_dir().join("broker.sqlite")),
|
||||||
("build_logs.sqlite", db_dir().join("build_logs.sqlite")),
|
("build_logs.sqlite", db_dir().join("build_logs.sqlite")),
|
||||||
("forge-core-avatar-set", forge_core_avatar_marker()),
|
|
||||||
("matrix-sender-token", matrix_sender_token()),
|
("matrix-sender-token", matrix_sender_token()),
|
||||||
("matrix-space-room-id", matrix_space_room_id()),
|
("matrix-space-room-id", matrix_space_room_id()),
|
||||||
("agent-sockets.json", agent_sockets_file()),
|
("agent-sockets.json", agent_sockets_file()),
|
||||||
|
|
|
||||||
|
|
@ -344,14 +344,6 @@ pub async fn reload_gateway_nginx() -> Result<()> {
|
||||||
ok(call(&PrivRequest::ReloadGatewayNginx).await?)
|
ok(call(&PrivRequest::ReloadGatewayNginx).await?)
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Run `forgejo admin <args>` inside the `hive-forge` container via
|
|
||||||
/// hive-priv (which runs as root and can nsenter into the container).
|
|
||||||
/// Returns `(stdout, stderr)` on success.
|
|
||||||
pub async fn run_forge_admin(args: &[&str]) -> Result<(String, String)> {
|
|
||||||
let owned: Vec<String> = args.iter().map(|s| (*s).to_owned()).collect();
|
|
||||||
check(call(&PrivRequest::RunForgeAdmin { args: owned }).await?)
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Create (`paused: true`) or remove (`paused: false`) the pause marker in
|
/// Create (`paused: true`) or remove (`paused: false`) the pause marker in
|
||||||
/// `agent_name`'s harness dir via hive-priv (running as root).
|
/// `agent_name`'s harness dir via hive-priv (running as root).
|
||||||
///
|
///
|
||||||
|
|
|
||||||
|
|
@ -361,25 +361,6 @@ pub enum PrivRequest {
|
||||||
/// service.
|
/// service.
|
||||||
ReloadGatewayNginx,
|
ReloadGatewayNginx,
|
||||||
|
|
||||||
// --- Forge admin CLI ---
|
|
||||||
/// Run `forgejo admin <args>` inside the `hive-forge` container as the
|
|
||||||
/// `forgejo` unix user. hive-priv executes:
|
|
||||||
///
|
|
||||||
/// nixos-container run hive-forge -- runuser -u forgejo --
|
|
||||||
/// forgejo --work-path /var/lib/forgejo admin `<args>`
|
|
||||||
///
|
|
||||||
/// `args` must not contain null bytes, newlines, or shell metacharacters;
|
|
||||||
/// hive-priv validates this before spawning the subprocess.
|
|
||||||
///
|
|
||||||
/// This operation requires root (to nsenter into the forge container's
|
|
||||||
/// namespaces); hive-c0re (which runs as `hive-core`) calls it through
|
|
||||||
/// this route instead of spawning `nixos-container run` directly.
|
|
||||||
RunForgeAdmin {
|
|
||||||
/// Argument list appended after `forgejo --work-path /var/lib/forgejo admin`.
|
|
||||||
/// Each element is a separate argv word — no shell expansion occurs.
|
|
||||||
args: Vec<String>,
|
|
||||||
},
|
|
||||||
|
|
||||||
// --- Agent turn-loop pause ---
|
// --- Agent turn-loop pause ---
|
||||||
// (marker filename: `PAUSED_MARKER_FILE`, defined at the crate root)
|
// (marker filename: `PAUSED_MARKER_FILE`, defined at the crate root)
|
||||||
/// Create (`paused: true`) or remove (`paused: false`) the pause marker
|
/// Create (`paused: true`) or remove (`paused: false`) the pause marker
|
||||||
|
|
|
||||||
|
|
@ -1,180 +1,11 @@
|
||||||
//! Host infrastructure ops: `RunForgeAdmin` (`forgejo admin` inside the
|
//! Host infrastructure ops: `ControlInfraContainer` (start/stop of an
|
||||||
//! `hive-forge` container), `ControlInfraContainer` (start/stop of an
|
//! [`InfraContainer`] unit) and `ReloadGatewayNginx`. Neither takes a unit or
|
||||||
//! [`InfraContainer`] unit) and `ReloadGatewayNginx`. None of them takes a
|
//! container name from the caller.
|
||||||
//! unit or container name from the caller.
|
|
||||||
|
|
||||||
use anyhow::{Context as _, Result, bail};
|
use anyhow::{Context as _, Result, bail};
|
||||||
use hive_priv_sock::{InfraAction, InfraContainer};
|
use hive_priv_sock::{InfraAction, InfraContainer};
|
||||||
use tokio::process::Command;
|
use tokio::process::Command;
|
||||||
|
|
||||||
/// `RunForgeAdmin`: every arg must pass [`validate_forge_admin_arg`] before
|
|
||||||
/// the admin CLI ever sees it.
|
|
||||||
pub(crate) async fn exec_forge_admin(args: &[String]) -> Result<(String, String)> {
|
|
||||||
for arg in args {
|
|
||||||
validate_forge_admin_arg(arg)?;
|
|
||||||
}
|
|
||||||
run_forge_admin(args).await
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Validate a single argument destined for `forgejo admin`. Rejects null
|
|
||||||
/// bytes, newlines and carriage returns (which could corrupt the
|
|
||||||
/// subprocess args list or log output).
|
|
||||||
///
|
|
||||||
/// Shell metacharacters are **not** rejected, and do not need to be: the
|
|
||||||
/// command is spawned directly, with no shell to interpret them.
|
|
||||||
fn validate_forge_admin_arg(arg: &str) -> Result<()> {
|
|
||||||
if arg.bytes().any(|b| b == 0 || b == b'\n' || b == b'\r') {
|
|
||||||
bail!("forge admin arg {arg:?} contains null byte or newline");
|
|
||||||
}
|
|
||||||
Ok(())
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Redact a line before it hits the (root-readable, but still
|
|
||||||
/// unnecessarily exposed) host journal.
|
|
||||||
///
|
|
||||||
/// Two independent rules, because the previous single rule failed open.
|
|
||||||
/// It matched only the substring "password", chosen to be robust against
|
|
||||||
/// forgejo *rewording* its password line — and the leak arrived from the
|
|
||||||
/// other axis entirely: a **different kind of secret** on a differently
|
|
||||||
/// worded line. `forgejo admin user generate-access-token` prints
|
|
||||||
/// `Access token was successfully created: <40 hex>`, which contains no
|
|
||||||
/// "password" and went to the journal verbatim for every agent ever
|
|
||||||
/// provisioned.
|
|
||||||
///
|
|
||||||
/// So the second rule matches on **shape, not vocabulary**: a long
|
|
||||||
/// unbroken run of secret-alphabet characters. A new secret type is then
|
|
||||||
/// caught by default rather than by someone remembering to add a keyword.
|
|
||||||
///
|
|
||||||
/// ⚠️ This deliberately over-matches. A nix store hash is also a long
|
|
||||||
/// opaque run and will redact its line. That is the correct direction to
|
|
||||||
/// be wrong in: the cost of a false positive is one less log line, and
|
|
||||||
/// the cost of a false negative is a live credential sitting in the host
|
|
||||||
/// journal, readable by anything with access to it.
|
|
||||||
fn redact_secret_line(line: &str) -> std::borrow::Cow<'_, str> {
|
|
||||||
if line.to_ascii_lowercase().contains("password") {
|
|
||||||
return std::borrow::Cow::Borrowed("[redacted: line mentions a password]");
|
|
||||||
}
|
|
||||||
if contains_secret_shaped_run(line) {
|
|
||||||
return std::borrow::Cow::Borrowed("[redacted: line contains a secret-shaped token]");
|
|
||||||
}
|
|
||||||
std::borrow::Cow::Borrowed(line)
|
|
||||||
}
|
|
||||||
|
|
||||||
/// True when the line contains an unbroken run of at least 32 characters
|
|
||||||
/// from the hex / base64url alphabet. 32 sits below forgejo's 40-hex
|
|
||||||
/// access token and above the ordinary words and path segments that
|
|
||||||
/// appear in `forgejo admin` output.
|
|
||||||
///
|
|
||||||
/// ⚠️ Scans for a RUN, not for a whitespace-delimited word. An earlier
|
|
||||||
/// version split on whitespace and required the whole word to match,
|
|
||||||
/// which a secret with punctuation glued to it defeats: `"<token>,"` and
|
|
||||||
/// `"[<token>]"` both fail an all-chars check on the word while still
|
|
||||||
/// containing the credential in full. Whitespace is not what delimits a
|
|
||||||
/// secret — the alphabet is (thanks @argus for catching it).
|
|
||||||
fn contains_secret_shaped_run(line: &str) -> bool {
|
|
||||||
const MIN: usize = 32;
|
|
||||||
let mut run = 0usize;
|
|
||||||
for b in line.bytes() {
|
|
||||||
if b.is_ascii_alphanumeric() || matches!(b, b'+' | b'/' | b'=' | b'_' | b'-') {
|
|
||||||
run += 1;
|
|
||||||
if run >= MIN {
|
|
||||||
return true;
|
|
||||||
}
|
|
||||||
} else {
|
|
||||||
run = 0;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
false
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Name a `forgejo admin` invocation for an error message without
|
|
||||||
/// reproducing its arguments: keep the leading verb path, stop at the
|
|
||||||
/// first flag. `["user", "change-password", "--username", "iris",
|
|
||||||
/// "--password", "…"]` becomes `forgejo admin user change-password`.
|
|
||||||
///
|
|
||||||
/// Same allowlist as hive-c0re's `describe_forge_admin`
|
|
||||||
/// (`hive-c0re/src/forge/mod.rs`), for the same reason: the verbs are a
|
|
||||||
/// closed set this crate chooses itself, but hive-c0re passes a live
|
|
||||||
/// `--password` value as an argument on some calls, and argument values
|
|
||||||
/// are never safe to assume closed. Redacting the value after
|
|
||||||
/// `--password` instead would repeat the bug `redact_secret_line`'s own
|
|
||||||
/// doc comment describes — a denylist of one flag name fails open the
|
|
||||||
/// moment a differently named secret-bearing flag is added.
|
|
||||||
fn describe_forge_admin(args: &[String]) -> String {
|
|
||||||
let verbs: Vec<&str> = args
|
|
||||||
.iter()
|
|
||||||
.take_while(|a| !a.starts_with('-'))
|
|
||||||
.map(String::as_str)
|
|
||||||
.collect();
|
|
||||||
if verbs.is_empty() {
|
|
||||||
"forgejo admin".to_owned()
|
|
||||||
} else {
|
|
||||||
format!("forgejo admin {}", verbs.join(" "))
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Run `forgejo admin <args>` inside the `hive-forge` container as the
|
|
||||||
/// `forgejo` unix user. Requires root (for nsenter into the container's
|
|
||||||
/// namespaces). Returns `(stdout, stderr)`.
|
|
||||||
async fn run_forge_admin(args: &[String]) -> Result<(String, String)> {
|
|
||||||
let mut cmd_args: Vec<&str> = vec![
|
|
||||||
"run",
|
|
||||||
"hive-forge",
|
|
||||||
"--",
|
|
||||||
"runuser",
|
|
||||||
"-u",
|
|
||||||
"forgejo",
|
|
||||||
"--",
|
|
||||||
"forgejo",
|
|
||||||
"--work-path",
|
|
||||||
"/var/lib/forgejo",
|
|
||||||
"admin",
|
|
||||||
];
|
|
||||||
for a in args {
|
|
||||||
cmd_args.push(a.as_str());
|
|
||||||
}
|
|
||||||
let out = Command::new("nixos-container")
|
|
||||||
.args(&cmd_args)
|
|
||||||
.output()
|
|
||||||
.await
|
|
||||||
.context("invoke nixos-container run hive-forge -- forgejo admin")?;
|
|
||||||
let stdout = String::from_utf8_lossy(&out.stdout).into_owned();
|
|
||||||
let stderr = String::from_utf8_lossy(&out.stderr).into_owned();
|
|
||||||
// stdout at DEBUG, not INFO: on the success path this stream carries
|
|
||||||
// the *product* of the command (the freshly minted token, the created
|
|
||||||
// user's details) and nothing an operator needs at default verbosity.
|
|
||||||
// Redaction stays on as the second layer — the level decides who sees
|
|
||||||
// it, the redactor decides what it says, and neither alone is enough.
|
|
||||||
for line in stdout.lines() {
|
|
||||||
tracing::debug!(target: "forgejo-admin", "{}", redact_secret_line(line));
|
|
||||||
}
|
|
||||||
for line in stderr.lines() {
|
|
||||||
tracing::warn!(target: "forgejo-admin", "{}", redact_secret_line(line));
|
|
||||||
}
|
|
||||||
if !out.status.success() {
|
|
||||||
// Redact here too. The error string is propagated to the caller and
|
|
||||||
// ends up logged; a partial-failure stderr can carry the same
|
|
||||||
// material stdout would have. And name the invocation via
|
|
||||||
// `describe_forge_admin` rather than `args.join(" ")`: some callers
|
|
||||||
// pass a live `--password` value as an argument, so reproducing the
|
|
||||||
// raw vector here is the same "two of three sites" gap that makes
|
|
||||||
// these leaks survive a fix — the log was redacted, but the error
|
|
||||||
// was not.
|
|
||||||
let safe_stderr: String = stderr
|
|
||||||
.lines()
|
|
||||||
.map(|l| redact_secret_line(l).into_owned())
|
|
||||||
.collect::<Vec<_>>()
|
|
||||||
.join("; ");
|
|
||||||
bail!(
|
|
||||||
"{} failed ({}): {}",
|
|
||||||
describe_forge_admin(args),
|
|
||||||
out.status,
|
|
||||||
safe_stderr.trim()
|
|
||||||
);
|
|
||||||
}
|
|
||||||
Ok((stdout, stderr))
|
|
||||||
}
|
|
||||||
|
|
||||||
/// `ControlInfraContainer` — start/stop a hive infrastructure service via
|
/// `ControlInfraContainer` — start/stop a hive infrastructure service via
|
||||||
/// `systemctl <verb> <unit>`. The [`InfraContainer`] enum is the allowlist:
|
/// `systemctl <verb> <unit>`. The [`InfraContainer`] enum is the allowlist:
|
||||||
/// serde already rejected any unknown / unsafe name (hive-c0re, the forge
|
/// serde already rejected any unknown / unsafe name (hive-c0re, the forge
|
||||||
|
|
@ -291,139 +122,12 @@ pub(crate) async fn sync_gateway_nginx() -> Result<(String, String)> {
|
||||||
}
|
}
|
||||||
Ok((String::new(), String::new()))
|
Ok((String::new(), String::new()))
|
||||||
}
|
}
|
||||||
|
|
||||||
#[cfg(test)]
|
#[cfg(test)]
|
||||||
mod tests {
|
mod tests {
|
||||||
use super::{
|
use super::{InfraAction, InfraContainer};
|
||||||
InfraAction, InfraContainer, contains_secret_shaped_run, describe_forge_admin,
|
|
||||||
redact_secret_line,
|
|
||||||
};
|
|
||||||
use hive_priv_sock::PrivRequest;
|
use hive_priv_sock::PrivRequest;
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn redacts_lines_mentioning_password_case_insensitively() {
|
|
||||||
assert_eq!(
|
|
||||||
redact_secret_line("New password: hunter2"),
|
|
||||||
"[redacted: line mentions a password]"
|
|
||||||
);
|
|
||||||
assert_eq!(
|
|
||||||
redact_secret_line("PASSWORD=hunter2"),
|
|
||||||
"[redacted: line mentions a password]"
|
|
||||||
);
|
|
||||||
assert_eq!(
|
|
||||||
redact_secret_line("User \"foo\" was successfully created."),
|
|
||||||
"User \"foo\" was successfully created."
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
/// `run_forge_admin`'s `bail!` names the invocation through this
|
|
||||||
/// function instead of `args.join(" ")` — the regression this test
|
|
||||||
/// guards. A `--password` value never survives to the error string.
|
|
||||||
#[test]
|
|
||||||
fn describe_forge_admin_does_not_reproduce_a_password_argument() {
|
|
||||||
let described = describe_forge_admin(&[
|
|
||||||
"user".to_owned(),
|
|
||||||
"create".to_owned(),
|
|
||||||
"--username".to_owned(),
|
|
||||||
"x".to_owned(),
|
|
||||||
"--password".to_owned(),
|
|
||||||
"S3cret".to_owned(),
|
|
||||||
]);
|
|
||||||
assert_eq!(described, "forgejo admin user create");
|
|
||||||
assert!(!described.contains("S3cret"), "{described}");
|
|
||||||
}
|
|
||||||
|
|
||||||
/// The `--password=value` single-argument form is also excluded: it
|
|
||||||
/// starts with `-`, so it never enters the kept verb path.
|
|
||||||
#[test]
|
|
||||||
fn describe_forge_admin_redacts_the_equals_sign_form_too() {
|
|
||||||
let described = describe_forge_admin(&[
|
|
||||||
"user".to_owned(),
|
|
||||||
"create".to_owned(),
|
|
||||||
"--password=S3cret".to_owned(),
|
|
||||||
]);
|
|
||||||
assert!(!described.contains("S3cret"), "{described}");
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Control: a non-secret argument (the verb path) still appears, so
|
|
||||||
/// the message stays useful for diagnosing which call failed.
|
|
||||||
#[test]
|
|
||||||
fn describe_forge_admin_keeps_the_verb_path() {
|
|
||||||
assert_eq!(
|
|
||||||
describe_forge_admin(&["user".to_owned(), "change-password".to_owned()]),
|
|
||||||
"forgejo admin user change-password"
|
|
||||||
);
|
|
||||||
assert_eq!(
|
|
||||||
describe_forge_admin(&["--help".to_owned()]),
|
|
||||||
"forgejo admin"
|
|
||||||
);
|
|
||||||
assert_eq!(describe_forge_admin(&[]), "forgejo admin");
|
|
||||||
}
|
|
||||||
|
|
||||||
/// The regression this function exists for. The keyword rule passes
|
|
||||||
/// this line straight through — it says nothing about a password — so
|
|
||||||
/// only the shape rule catches it.
|
|
||||||
#[test]
|
|
||||||
fn redacts_access_token_line_which_mentions_no_password() {
|
|
||||||
let line =
|
|
||||||
"Access token was successfully created: 0123456789abcdef0123456789abcdef01234567";
|
|
||||||
assert!(
|
|
||||||
!line.to_ascii_lowercase().contains("password"),
|
|
||||||
"fixture must not contain the keyword, or it proves nothing"
|
|
||||||
);
|
|
||||||
assert_eq!(
|
|
||||||
redact_secret_line(line),
|
|
||||||
"[redacted: line contains a secret-shaped token]"
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn secret_shape_boundaries() {
|
|
||||||
// Ordinary forgejo-admin output survives: no run is long enough.
|
|
||||||
assert_eq!(
|
|
||||||
redact_secret_line("Command 'user' 'create' finished with no errors."),
|
|
||||||
"Command 'user' 'create' finished with no errors."
|
|
||||||
);
|
|
||||||
// 31 chars is below the floor, 32 is at it.
|
|
||||||
assert!(!contains_secret_shaped_run(&"a".repeat(31)));
|
|
||||||
assert!(contains_secret_shaped_run(&"a".repeat(32)));
|
|
||||||
// Punctuation breaks the run — a sentence never trips it however long.
|
|
||||||
assert!(!contains_secret_shaped_run(
|
|
||||||
"this.is.a.very.long.dotted.identifier.but.not.a.secret"
|
|
||||||
));
|
|
||||||
// base64url and hex alphabets both count.
|
|
||||||
assert!(contains_secret_shaped_run(
|
|
||||||
"ZGVhZGJlZWZkZWFkYmVlZmRlYWRiZWVmZGVhZA=="
|
|
||||||
));
|
|
||||||
assert!(contains_secret_shaped_run(
|
|
||||||
"aG93-dy_there-aG93dy1theresomething"
|
|
||||||
));
|
|
||||||
}
|
|
||||||
|
|
||||||
/// argus on the review: a whitespace-delimited check is defeated by
|
|
||||||
/// punctuation glued to the secret — the punctuation joins the "word"
|
|
||||||
/// and fails the alphabet test for the whole run, while the credential
|
|
||||||
/// sits there in full. Scanning for a RUN rather than a WORD closes it.
|
|
||||||
/// These are the shapes that used to slip through.
|
|
||||||
#[test]
|
|
||||||
fn secret_is_caught_with_punctuation_glued_to_it() {
|
|
||||||
const TOK: &str = "0123456789abcdef0123456789abcdef01234567";
|
|
||||||
for line in [
|
|
||||||
format!("token: {TOK},"),
|
|
||||||
format!("token: {TOK}."),
|
|
||||||
format!("using [{TOK}] now"),
|
|
||||||
format!("value=\"{TOK}\""),
|
|
||||||
format!("(created {TOK})"),
|
|
||||||
// no whitespace anywhere -- one glued blob
|
|
||||||
format!("Bearer:{TOK};next"),
|
|
||||||
] {
|
|
||||||
assert_eq!(
|
|
||||||
redact_secret_line(&line),
|
|
||||||
"[redacted: line contains a secret-shaped token]",
|
|
||||||
"leaked through: {line}"
|
|
||||||
);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/// A start/stop naming the forge or matrix container fails to decode, so
|
/// A start/stop naming the forge or matrix container fails to decode, so
|
||||||
/// `run` answers it with a parse error before anything reaches
|
/// `run` answers it with a parse error before anything reaches
|
||||||
/// `systemctl`. Control: the same request naming the gateway decodes.
|
/// `systemctl`. Control: the same request naming the gateway decodes.
|
||||||
|
|
|
||||||
|
|
@ -21,7 +21,7 @@
|
||||||
//! own module: [`container`] (lifecycle), [`container_config`] (nspawn flags,
|
//! own module: [`container`] (lifecycle), [`container_config`] (nspawn flags,
|
||||||
//! systemd drop-ins), [`storage`] (btrfs subvolumes, quotas, snapshots, disk
|
//! systemd drop-ins), [`storage`] (btrfs subvolumes, quotas, snapshots, disk
|
||||||
//! measurement), [`files`] (pause marker, socket dir, legacy tmpfiles) and
|
//! measurement), [`files`] (pause marker, socket dir, legacy tmpfiles) and
|
||||||
//! [`infra`] (gateway nginx, forge admin, infra containers). Helpers more than
|
//! [`infra`] (gateway nginx, infra containers). Helpers more than
|
||||||
//! one family uses are in [`shared`].
|
//! one family uses are in [`shared`].
|
||||||
|
|
||||||
mod container;
|
mod container;
|
||||||
|
|
@ -399,8 +399,6 @@ async fn exec(
|
||||||
|
|
||||||
PrivRequest::ReloadGatewayNginx => infra::sync_gateway_nginx().await,
|
PrivRequest::ReloadGatewayNginx => infra::sync_gateway_nginx().await,
|
||||||
|
|
||||||
PrivRequest::RunForgeAdmin { ref args } => infra::exec_forge_admin(args).await,
|
|
||||||
|
|
||||||
PrivRequest::SetAgentPaused {
|
PrivRequest::SetAgentPaused {
|
||||||
ref agent_name,
|
ref agent_name,
|
||||||
paused,
|
paused,
|
||||||
|
|
|
||||||
|
|
@ -43,7 +43,6 @@ in
|
||||||
HIVE_AGENT_FRONTEND_DIR = "${cfg.servedFrontend}/agent";
|
HIVE_AGENT_FRONTEND_DIR = "${cfg.servedFrontend}/agent";
|
||||||
# Path to the static runtime asset tree (branding + claude
|
# Path to the static runtime asset tree (branding + claude
|
||||||
# prompts). `hive_sh4re::assets::*` reads paths underneath.
|
# prompts). `hive_sh4re::assets::*` reads paths underneath.
|
||||||
# `forge/users.rs` reads the core avatar PNG from here on startup.
|
|
||||||
HIVE_ASSETS_DIR = "${cfg.assets}/share/hyperhive";
|
HIVE_ASSETS_DIR = "${cfg.assets}/share/hyperhive";
|
||||||
}
|
}
|
||||||
// {
|
// {
|
||||||
|
|
|
||||||
|
|
@ -4,9 +4,9 @@
|
||||||
# input, so git's config is the only credential and URL plumbing it has.
|
# input, so git's config is the only credential and URL plumbing it has.
|
||||||
#
|
#
|
||||||
# - `credential`: the forge host answers through `git-credential-hive-forge`,
|
# - `credential`: the forge host answers through `git-credential-hive-forge`,
|
||||||
# which reads the live `core` token file. An empty `helper =` first resets the
|
# which reads the hive's own forge account and token from the file hive-c0re
|
||||||
# list, so no /etc/gitconfig helper (e.g. `oauth`, which waits on a browser)
|
# publishes. An empty `helper =` first resets the list, so no /etc/gitconfig
|
||||||
# runs ahead of ours.
|
# helper (e.g. `oauth`, which waits on a browser) runs ahead of ours.
|
||||||
# - `url`: the swarm's `internal/meta` lock names its inputs by the forge's
|
# - `url`: the swarm's `internal/meta` lock names its inputs by the forge's
|
||||||
# `https` URL, which only the builder's credential helper answers for. They
|
# `https` URL, which only the builder's credential helper answers for. They
|
||||||
# are fetched over the `http` URL the helper above is scoped to.
|
# are fetched over the `http` URL the helper above is scoped to.
|
||||||
|
|
@ -19,16 +19,18 @@
|
||||||
# # `forgeGit.config` into its gitconfig.
|
# # `forgeGit.config` into its gitconfig.
|
||||||
{ pkgs, forgeDomain }:
|
{ pkgs, forgeDomain }:
|
||||||
{
|
{
|
||||||
# Answers only `get`, for the forge `core` user, from the live token file
|
# Answers only `get`. The file (`/run/hyperhive/forge-hive-token`, hive-c0re's
|
||||||
# (`/var/lib/hyperhive/forge-core-token`, hive-c0re's `paths::FORGE_CORE_TOKEN`)
|
# `paths::FORGE_HIVE_TOKEN`) holds the account on its first line and the
|
||||||
# on every invocation, so it never holds a stale copy and survives token
|
# token on its second; it is read on every invocation, so it never holds a
|
||||||
# rotation; prints nothing where the file is absent.
|
# stale copy and survives token rotation. Prints nothing where the file is
|
||||||
|
# absent or incomplete.
|
||||||
credentialHelper = pkgs.writeShellScriptBin "git-credential-hive-forge" ''
|
credentialHelper = pkgs.writeShellScriptBin "git-credential-hive-forge" ''
|
||||||
[ "''${1:-}" = "get" ] || exit 0
|
[ "''${1:-}" = "get" ] || exit 0
|
||||||
if [ -r /var/lib/hyperhive/forge-core-token ]; then
|
[ -r /run/hyperhive/forge-hive-token ] || exit 0
|
||||||
printf 'username=core\n'
|
{ read -r login; read -r token; } < /run/hyperhive/forge-hive-token || true
|
||||||
printf 'password=%s\n' "$(cat /var/lib/hyperhive/forge-core-token)"
|
[ -n "$login" ] && [ -n "$token" ] || exit 0
|
||||||
fi
|
printf 'username=%s\n' "$login"
|
||||||
|
printf 'password=%s\n' "$token"
|
||||||
'';
|
'';
|
||||||
|
|
||||||
config = ''
|
config = ''
|
||||||
|
|
@ -37,7 +39,6 @@
|
||||||
[credential "http://${forgeDomain}"]
|
[credential "http://${forgeDomain}"]
|
||||||
helper =
|
helper =
|
||||||
helper = hive-forge
|
helper = hive-forge
|
||||||
username = core
|
|
||||||
[url "http://${forgeDomain}/"]
|
[url "http://${forgeDomain}/"]
|
||||||
insteadOf = https://${forgeDomain}/
|
insteadOf = https://${forgeDomain}/
|
||||||
'';
|
'';
|
||||||
|
|
|
||||||
|
|
@ -257,6 +257,13 @@ let
|
||||||
in
|
in
|
||||||
helperOn "hive-c0re"
|
helperOn "hive-c0re"
|
||||||
&& helperOn "hive-priv"
|
&& helperOn "hive-priv"
|
||||||
|
# The helper answers from the file hive-c0re publishes the hive's own
|
||||||
|
# token to. The first arm is the control for the second: the lookup
|
||||||
|
# finds the helper's text, so the absent `core` strings are really absent.
|
||||||
|
&& lib.hasInfix "/run/hyperhive/forge-hive-token" git.credentialHelper.text
|
||||||
|
&& !(lib.hasInfix "forge-core-token" git.credentialHelper.text)
|
||||||
|
&& !(lib.hasInfix "username=core" git.credentialHelper.text)
|
||||||
|
&& !(lib.hasInfix "username = core" git.config)
|
||||||
&& !(helperOn "swarm-controller")
|
&& !(helperOn "swarm-controller")
|
||||||
&& lib.hasInfix "[credential \"http://${forge}\"]\nhelper =\nhelper = hive-forge" git.config
|
&& lib.hasInfix "[credential \"http://${forge}\"]\nhelper =\nhelper = hive-forge" git.config
|
||||||
&& lib.hasInfix "[url \"http://${forge}/\"]\ninsteadOf = https://${forge}/" git.config
|
&& lib.hasInfix "[url \"http://${forge}/\"]\ninsteadOf = https://${forge}/" git.config
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue