<![CDATA[Bot Crash 2026-09-20]]>Picking up from this thread: https://forums.triplea-game.org/topic/4357/triplea-server-costs/40

What happened:

  • infrastructure needed an update to deploy SSH keys, which ran the infrastructure deployment. That is the server configs, all server config settings are applied at that time
  • part of server configs, is server updates, those were run
  • one server update, is to a system (a service) called 'docker', docker is currently running all of the tripleA applications
  • a config setting to automatically restart services when they are updated, or any of their dependencies are updated is live in the system
  • docker was updated, per the config setting, it was full restarted.
  • as a result, all tripleA systems were rebooted (including bots), which would have ejected players from bots.

With the reboot, some bots may have come back up, I expect for some to crash again over time and would still be interested in error reports.

Since then, docker is now on a blacklist for services to be restarted and a secondary config was added that would allow applications running on the docker service to potential survive a reboot of docker as well. 3rd, there is a PR up to autosave when a bot shuts down.

Historically bot restarts had been for more direct causes. This was more "rare" as there are typically no package updates for docker.

I'm taking a quick look at the autosaves to see what the history is there.

]]>https://forums.triplea-game.org/topic/4421/bot-crash-2026-09-20RSS for NodeMon, 05 Oct 2026 11:57:14 GMTMon, 21 Sep 2026 16:04:10 GMT60<![CDATA[Reply to Bot Crash 2026-09-20 on Sat, 26 Sep 2026 23:53:00 GMT]]>Do please let me know about other bot crashes:

  • new post
  • DM me here
  • DM me on discord
  • Create a github issue

Any of those work. I'm locking this thread as it's meant to be specific to the Sep-20 incident.

]]>https://forums.triplea-game.org/post/80036https://forums.triplea-game.org/post/80036Sat, 26 Sep 2026 23:53:00 GMT<![CDATA[Reply to Bot Crash 2026-09-20 on Sat, 26 Sep 2026 22:17:00 GMT]]>The bots were stuck in a shutdown deadlock. A system exit call triggers the shutdown hook, and if that also invokes system exit then the system deadlocks. This could be triggered through the "admin shutdown" of a running game.

This update would fix the situation:
https://github.com/triplea-game/triplea/pull/15017

That update should roll out to bots in between games (there's an automatic restart between games, the bots have pulled the latest code version, they'll activate the new version on the next game restart)

]]>https://forums.triplea-game.org/post/80028https://forums.triplea-game.org/post/80028Sat, 26 Sep 2026 22:17:00 GMT<![CDATA[Reply to Bot Crash 2026-09-20 on Sat, 26 Sep 2026 03:00:14 GMT]]>@LaFayette

Hi
looks as if London Toronto and Cali all down a bot. 10 pm CST 9/25

Screenshot from 2026-09-25 21-58-08.png

]]>https://forums.triplea-game.org/post/80026https://forums.triplea-game.org/post/80026Sat, 26 Sep 2026 03:00:14 GMT<![CDATA[Reply to Bot Crash 2026-09-20 on Fri, 25 Sep 2026 20:17:32 GMT]]>@Cernel thanks, that issue is closed now.

]]>https://forums.triplea-game.org/post/80023https://forums.triplea-game.org/post/80023Fri, 25 Sep 2026 20:17:32 GMT<![CDATA[Reply to Bot Crash 2026-09-20 on Wed, 23 Sep 2026 14:43:33 GMT]]>

Cernel said:

Maybe there is an issue still open in GitHub about this.

https://github.com/triplea-game/triplea/issues/14435

]]>https://forums.triplea-game.org/post/79963https://forums.triplea-game.org/post/79963Wed, 23 Sep 2026 14:43:33 GMT<![CDATA[Reply to Bot Crash 2026-09-20 on Tue, 22 Sep 2026 12:28:52 GMT]]>@LaFayette Cool. However, it does not seem to me that the place chosen for the auto-saves is very polished (kind of buried, and I guess might favour them being overlooked once again as a removed feature rather than a problem as opposed to have a dedicated space if not working). I think it would be better having a "Load Bot-Save" button right under the "Select Game" button.

By the way (minor detail), they are not actually "towards the bottom": they are before all games starting with lower case (which I guess are also something that should not be (?), all names ideally better starting upper case). I mean, this is what I see:
20260922.png

However, I do recall that bots are supposed to be supported only for a few years more and eventually phased out in favour of better (and cheaper) ways, and I'm sure bot-users are just fine as they are (and anyway I'm not a bot-user, so I'm not really the person who should give user-experience suggestions on the matter).

A problem with bots is that the very few users who report any problems are usually persons savvy enough to be able to host, so any problem with bots is likely not getting reported, unfortunately. I remember I was told this problem (auto-saves missing) the days after the release (months ago), and maybe I reported it somewhere or someone else did (don't remember). Maybe there is an issue still open in GitHub about this.

]]>https://forums.triplea-game.org/post/79928https://forums.triplea-game.org/post/79928Tue, 22 Sep 2026 12:28:52 GMT<![CDATA[Reply to Bot Crash 2026-09-20 on Mon, 21 Sep 2026 16:56:45 GMT]]>cc: @beelee / @cernel / @eyeswideshut ^

]]>https://forums.triplea-game.org/post/79908https://forums.triplea-game.org/post/79908Mon, 21 Sep 2026 16:56:45 GMT<![CDATA[Reply to Bot Crash 2026-09-20 on Mon, 21 Sep 2026 16:56:30 GMT]]>Bot Autosaves restored
  • Autosaves in bots were missing in our latest codebase
  • Autosaves were moved from their own dedicated "load autosave" button to the "select game" button in the hosting menu
  • An update to "live-load" that list so that new maps were picked up dynamically regressed the feature and didn't add the autosaves to the "select game" list.
  • As a result, there was no way to select autosaves.

This is now fixed via: https://github.com/triplea-game/triplea/pull/15002

The autosaves now do appear in the "select game" list, towards the bottom:

Screenshot From 2026-09-21 09-56-18.png

]]>https://forums.triplea-game.org/post/79907https://forums.triplea-game.org/post/79907Mon, 21 Sep 2026 16:56:30 GMT