f4113033ac
The task that runs the server ignores a start while it thinks it is still running, so the cleanup job could stop the server and then fail to start it again: the site had no shell for forty minutes. The start is now tried three times, with the processes of the task killed in between. The job now only counts the established connections when it decides whether anybody is connected - the server leaves the processes of finished sessions behind, and they kept it from ever restarting the server. Those processes are killed with the server, because they hold the runtime and the protected programs open. The pacman calls are bounded, so a database lock cannot keep the server down, and they run before the server is stopped. The exit code of a program started this way is only available when its handle is taken while it runs, and the pattern for the files to overwrite has to be quoted, or the runtime expands it. Co-Authored-By: Claude Code <noreply@anthropic.com>
99 lines
6 KiB
Markdown
99 lines
6 KiB
Markdown
# The shell server
|
|
|
|
The website's terminal connects to a shell server, whose `root` user logs in at `wss://web.nowhere.wirs.ing` and
|
|
gets a shell on the box.
|
|
|
|
## The security model
|
|
|
|
The box is an isolated, disposable VPS at a random hoster. We assume that it can be compromised and that this is
|
|
acceptable: the worst case is that the gimmick on the website stops working. Nothing else is on the box. The
|
|
website and the websocket bridge that carries the connection run on other servers, so a compromised shell server
|
|
cannot get at them.
|
|
|
|
The `root` user on the box is a normal Windows account without administrator rights. It may do a lot of things
|
|
there, including things we did not plan for. We do not try to contain it and we do not try to undo what it does.
|
|
We only keep the basic user journey working.
|
|
|
|
## What we keep working
|
|
|
|
Anything that breaks the following user journey is not acceptable:
|
|
|
|
- The user can connect with the SSH key as `root`.
|
|
- The prompt: `root@khaos-im-kiez:~#`, which `~/.zsh/.zshrc` sets.
|
|
- The home directory the user ends up in: `/home/root`. They cannot change where it is, and they cannot delete it.
|
|
- `cat README`, `cat README.en`, `cat CODE_OF_CONDUCT` and `cat CODE_OF_CONDUCT.en` always work. The four files
|
|
are always there and always regular text files.
|
|
|
|
So in summary running `ssh root@nowhere.wirs.ing cat README` must always succeed and display the contents of the file.
|
|
|
|
Anything thats not specified above (like the content of README for example) is NOT part of the user journey.
|
|
|
|
## What we do not keep working
|
|
|
|
As long as the user story above does not break, we do not guarantee the functionality of anything else on the server. However the cleanup task should still try to best-effort restore everything to a working condition.
|
|
|
|
## Packages
|
|
|
|
The root user owns the whole MSYS2 installation and can install packages with pacman. That means they can also
|
|
replace or delete the very programs the login needs. So the sshd, the login shell `zsh` and `cat` are kept as a
|
|
second, protected copy in `C:\msys2\opt\kik`, which only SYSTEM and the administrators may write. The sshd runs as
|
|
SYSTEM from that copy and the login shell is that copy's `zsh`, with `/opt/kik` first on the path, so the journey
|
|
keeps working no matter what the user does to their own `/usr/bin`. Replacing `/usr/bin/sshd` cannot touch the
|
|
running server, so the key on the website still never grants administrator access.
|
|
|
|
The copy has to be the *same* MSYS2 installation, not a second one: `C:\msys2\opt\kik` is two levels below
|
|
`C:\msys2`, so its runtime resolves the same root and the login shell and the user's own tools share one cygwin
|
|
runtime. A separate installation does not work - two runtimes in one process tree corrupt each other's file
|
|
descriptors and pacman cannot even lock its database.
|
|
|
|
The runtime itself is not among the protected copies. A program and the shell that starts it have to use the same
|
|
runtime DLL, or the program does not run at all - a second copy of it in one process is refused. The login path
|
|
therefore uses the one in `/usr/bin`, next to the user's own programs, and the user may replace and upgrade it.
|
|
What is protected instead is that it is the one the package says it is: pacman checks the files of `msys2-runtime`,
|
|
and the cleanup job puts the package back if it was changed - while the server is stopped, before it starts it
|
|
again, because the server runs as SYSTEM and would otherwise run code the user of the website wrote.
|
|
|
|
`/etc/passwd` (the login shell and home directory) and `/etc/ssh` (the host keys and config) are read-only to the
|
|
user as well; everything else in the installation is theirs.
|
|
|
|
## The cleanup tasks
|
|
|
|
Two scheduled tasks run as SYSTEM, both at startup, and the user cannot stop either:
|
|
|
|
- `KiK sshd` runs the sshd from `C:\msys2\opt\kik`. It replaces the installation's own `msys2-sshd` service, which
|
|
is disabled.
|
|
- `KiK restore shell files` runs every 30 minutes and rebuilds the protected copy from its trusted originals,
|
|
re-applies the permissions, and puts the home directory, the shell configuration, the website's key and the four
|
|
texts back. Its second job is to best-effort revert other mockery and bring a fresh server to the expected state.
|
|
|
|
The cleanup job also restarts the server when nobody is connected, and when the server does not listen at all:
|
|
it comes back by itself if it died, it takes over a server that was left behind, and it picks up the programs that
|
|
were just restored. A session that is open keeps the port, so a server that is started now could not bind it, and
|
|
the job never disturbs a session.
|
|
|
|
The task that runs the server ignores a start while it thinks it is still running, so a start that is dropped
|
|
leaves the site without a shell. The cleanup job therefore tries three times, killing the processes of the task
|
|
in between, before it gives up.
|
|
|
|
The deployment does not restart the server. It runs in a session, and that session keeps the port, so a server that
|
|
starts during a deployment cannot bind it. The new configuration and the restored programs therefore take effect
|
|
at the next run of the cleanup job, which happens within half an hour.
|
|
|
|
The server logs to `C:\ProgramData\kik-restore\sshd.log`. The log is appended to, not overwritten, because it is
|
|
the only trace of what the server did: a server that is restarted would otherwise take the evidence of an outage
|
|
with it. It is rotated when it grows past a megabyte.
|
|
|
|
Only keys may log in over SSH. The box stands in the open internet and both of its accounts have a password.
|
|
|
|
The files that have to stay belong to the administrators, so the user cannot delete them, and they cannot change the
|
|
shell configuration or the key the website logs in with either. The shell reads its configuration from `~/.zsh`, so
|
|
the startup files the user could write in the home directory are ignored.
|
|
|
|
## Installing the cleanup on the server
|
|
|
|
To deploy the cleanup task an the server, run:
|
|
|
|
server/scripts/deploy.sh
|
|
|
|
The script logs in as `Administrator` - whose shell is a Windows one - copies the scripts and the resources to the
|
|
Administrator's home directory and runs the installer there.
|