a813326312
The sshd log is rotated at a hundred megabytes instead of one, and the event logs of windows are made that big as well: they are the only record of what the box itself did, and twenty megabytes are not much with all the connections. Co-Authored-By: Claude Code <noreply@anthropic.com>
108 lines
6.8 KiB
Markdown
108 lines
6.8 KiB
Markdown
# The shell server
|
|
|
|
The website's terminal connects to a shell server, whose `root` user logs in at `wss://web.nowhere.wirs.ing` and
|
|
gets a shell on the box.
|
|
|
|
## The security model
|
|
|
|
The box is an isolated, disposable VPS at a random hoster. We assume that it can be compromised and that this is
|
|
acceptable: the worst case is that the gimmick on the website stops working. Nothing else is on the box. The
|
|
website and the websocket bridge that carries the connection run on other servers, so a compromised shell server
|
|
cannot get at them.
|
|
|
|
The `root` user on the box is a normal Windows account without administrator rights. It may do a lot of things
|
|
there, including things we did not plan for. We do not try to contain it and we do not try to undo what it does.
|
|
We only keep the basic user journey working.
|
|
|
|
## What we keep working
|
|
|
|
Anything that breaks the following user journey is not acceptable:
|
|
|
|
- The user can connect with the SSH key as `root`.
|
|
- The prompt: `root@khaos-im-kiez:~#`, which `~/.zsh/.zshrc` sets.
|
|
- The home directory the user ends up in: `/home/root`. They cannot change where it is, and they cannot delete it.
|
|
- `cat README`, `cat README.en`, `cat CODE_OF_CONDUCT` and `cat CODE_OF_CONDUCT.en` always work. The four files
|
|
are always there and always regular text files.
|
|
|
|
So in summary running `ssh root@nowhere.wirs.ing cat README` must always succeed and display the contents of the file.
|
|
|
|
Anything thats not specified above (like the content of README for example) is NOT part of the user journey.
|
|
|
|
## What we do not keep working
|
|
|
|
As long as the user story above does not break, we do not guarantee the functionality of anything else on the server. However the cleanup task should still try to best-effort restore everything to a working condition.
|
|
|
|
## Packages
|
|
|
|
The root user owns the whole MSYS2 installation and can install packages with pacman. That means they can also
|
|
replace or delete the very programs the login needs. So the sshd, the login shell `zsh` and `cat` are kept as a
|
|
second, protected copy in `C:\msys2\opt\kik`, which only SYSTEM and the administrators may write. The sshd runs as
|
|
SYSTEM from that copy and the login shell is that copy's `zsh`, with `/opt/kik` first on the path, so the journey
|
|
keeps working no matter what the user does to their own `/usr/bin`. Replacing `/usr/bin/sshd` cannot touch the
|
|
running server, so the key on the website still never grants administrator access.
|
|
|
|
The copy has to be the *same* MSYS2 installation, not a second one: `C:\msys2\opt\kik` is two levels below
|
|
`C:\msys2`, so its runtime resolves the same root and the login shell and the user's own tools share one cygwin
|
|
runtime. A separate installation does not work - two runtimes in one process tree corrupt each other's file
|
|
descriptors and pacman cannot even lock its database.
|
|
|
|
The runtime itself is not among the protected copies. A program and the shell that starts it have to use the same
|
|
runtime DLL, or the program does not run at all - a second copy of it in one process is refused. The login path
|
|
therefore uses the one in `/usr/bin`, next to the user's own programs, and the user may replace and upgrade it.
|
|
What is protected instead is that it is the one the package says it is: pacman checks the files of `msys2-runtime`,
|
|
and the cleanup job puts the package back if it was changed - while the server is stopped, before it starts it
|
|
again, because the server runs as SYSTEM and would otherwise run code the user of the website wrote.
|
|
|
|
`/etc/passwd` (the login shell and home directory) and `/etc/ssh` (the host keys and config) are read-only to the
|
|
user as well; everything else in the installation is theirs.
|
|
|
|
## The cleanup tasks
|
|
|
|
Three scheduled tasks run as SYSTEM, all at startup, and the user cannot stop any of them:
|
|
|
|
- `KiK sshd` runs the sshd from `C:\msys2\opt\kik`. It replaces the installation's own `msys2-sshd` service, which
|
|
is disabled.
|
|
- `KiK restore shell files` runs every 30 minutes and rebuilds the protected copy from its trusted originals,
|
|
re-applies the permissions, and puts the home directory, the shell configuration, the website's key and the four
|
|
texts back. Its second job is to best-effort revert other mockery and bring a fresh server to the expected state.
|
|
- `KiK watch the server` runs every two minutes and starts the server when nothing listens on port 22. The task of
|
|
the server only starts it again when it failed, never when it was stopped, so a cleanup job that is killed while
|
|
it has the server stopped would otherwise take the site down with it.
|
|
|
|
The cleanup job also restarts the server when nobody is connected, and when the server does not listen at all:
|
|
it comes back by itself if it died, it takes over a server that was left behind, and it picks up the programs that
|
|
were just restored. A session that is open keeps the port, so a server that is started now could not bind it, and
|
|
the job never disturbs a session.
|
|
|
|
The server is only down for the short part of the job that needs it down: the protected copy cannot be replaced
|
|
while the server holds it open. Everything else - the home directory, the four texts, the permissions of the
|
|
installation - is put back while the server runs, because on a slow disk that takes minutes and the site should
|
|
have a shell the whole time.
|
|
|
|
The task that runs the server ignores a start while it thinks it is still running, so a start that is dropped
|
|
leaves the site without a shell. The cleanup job therefore tries three times, killing the processes of the task
|
|
in between, before it gives up.
|
|
|
|
The deployment does not restart the server. It runs in a session, and that session keeps the port, so a server that
|
|
starts during a deployment cannot bind it. The new configuration and the restored programs therefore take effect
|
|
at the next run of the cleanup job, which happens within half an hour.
|
|
|
|
The server logs to `C:\ProgramData\kik-restore\sshd.log`. The log is appended to, not overwritten, because it is
|
|
the only trace of what the server did: a server that is restarted would otherwise take the evidence of an outage
|
|
with it. It is rotated when it grows past a hundred megabytes, which takes years. The event logs of windows are
|
|
made that big as well, so that they keep the history of the box itself for just as long.
|
|
|
|
Only keys may log in over SSH. The box stands in the open internet and both of its accounts have a password.
|
|
|
|
The files that have to stay belong to the administrators, so the user cannot delete them, and they cannot change the
|
|
shell configuration or the key the website logs in with either. The shell reads its configuration from `~/.zsh`, so
|
|
the startup files the user could write in the home directory are ignored.
|
|
|
|
## Installing the cleanup on the server
|
|
|
|
To deploy the cleanup task an the server, run:
|
|
|
|
server/scripts/deploy.sh
|
|
|
|
The script logs in as `Administrator` - whose shell is a Windows one - copies the scripts and the resources to the
|
|
Administrator's home directory and runs the installer there.
|