@Cernel When the Revised Ladder bots were crashing frequently, I moved them onto Amazon Web Services and reworked how they were launched and maintained.
AWS is a cloud hosted Linux server, so the bots run remotely rather than depending on my own computer, home internet connection, or an open SSH window. I connect to the server through SSH, but the bots themselves run inside persistent tmux sessions.
tmux is useful here because it separates the bot process from the temporary SSH connection. I can disconnect from the server, close my laptop, or lose my local connection without terminating the bot. Each bot has its own named session, which also makes it easier to inspect or restart one without affecting the other.
The main upkeep commands are:
tmux ls
This shows whether the bot sessions are still present.
tmux attach -t revised_ladder
tmux attach -t revised_ladder2
These reconnect me to the live console output for either bot so I can inspect errors, freezes, or lobby disconnects.
To leave a session while keeping the bot running:
Ctrl+B, then D
If the bot is frozen rather than fully gone, I can attach to the session, stop the process with Ctrl+C, and relaunch it using the relevant startup script.
I also set up the launch process so that if a bot process actually disappears or exits, it is automatically started again. In other words, the server does not simply leave the bot offline after a crash. The surrounding startup or monitoring process detects that the bot is no longer running and relaunches it. That does not necessarily fix every possible freeze, since a frozen process may still technically exist, but it does handle ordinary crashes and unexpected exits without requiring me to log in immediately.
So the overall setup has three layers:
AWS provides the continuously available cloud server.
tmux keeps the bot sessions alive independently of SSH.
The restart logic relaunches a bot when its process exits or disappears.
This is what I implemented when the bots were crashing a lot. Since then, the AWS and tmux setup has generally been stable, and the automatic restart behavior has reduced the amount of manual intervention needed.
