<![CDATA[Scraping Event - Brief Forum Outage - 2026-Sep-26]]>A write up is here: https://github.com/triplea-game/forums/discussions/5

In short, a scraper started sending us 70x traffic and the forum fell over.

In response, made a number of changes that will help us going forward:

  • most categories are now hidden to non-registered users. "News" and the "Bug Report" categories are the only ones publicly visible now. This gives less surface area for a scanner, so they hopefully move on more quickly.

- applied a lot of infrastructure tunings to reduce the CPU cost of serving a forums website request

  • increased connection limits on the NGINX (the web server in front of forums)
  • added a missing database index that caused guest page views to be very expensive for the CPU
  • added site wide rate limits and per-IP rate limits. The site wide rate limits will more help the forums degrade gracefully rather than fully crash.
  • moved assets to be served directly from web cache instead of by the NodeBB forumn software. Requests for assets are much easier on the server now and no longer involve any database writes.
  • turned off the feature to count for the number of non-registered user page views. Simply counting the number of views requires an update to database. Every scraping request added to page counts, and so turning the feature off removes these database writes.

Overall, the updates make each web request a lot cheaper for the server to serve, which means higher capacity. The scraper event stopped at about the same time when we started responding to it. With these updates, we might mostly survive another scraping event (but if the request rate is high enough, we probably wouldn't).

]]>https://forums.triplea-game.org/topic/4429/scraping-event-brief-forum-outage-2026-sep-26RSS for NodeSun, 27 Sep 2026 20:17:41 GMTSun, 27 Sep 2026 16:39:14 GMT60<![CDATA[Reply to Scraping Event - Brief Forum Outage - 2026-Sep-26 on Sun, 27 Sep 2026 19:54:28 GMT]]>@LaFayette

Just curious, can the LLM access the chat also ?

Edit
hmm ... reading up on them a bit, it seems as if it's just ai doing this on it's own, as opposed to some person directly attacking triplea, although someone had to initially send it on it's way, maybe 🙂

Idk. If so, it's already learning to come back with a smaller attack as Dan stated above.

AI ... uggh ... just gotta get used to it, because it's not going away. Try and make it work for you and defend as best you can against it.

Gonna make cell phone introduction look like kids play lol

]]>https://forums.triplea-game.org/post/80059https://forums.triplea-game.org/post/80059Sun, 27 Sep 2026 19:54:28 GMT<![CDATA[Reply to Scraping Event - Brief Forum Outage - 2026-Sep-26 on Sun, 27 Sep 2026 19:31:27 GMT]]>

@Cernel said:

(I'm curious why the hell would anyone do this. What have they to gain?)

Mass scraping of the internet is how LLMs were trained. There's a number of stories of how they use scrapers that do not declare themselves and overload websites (they're trying to scrape the whole internet, which means you need to do that quickly in order to get it done in a reasonable time frame). Though, anyone's guess is really as a good as mine. The scraper had access to a pool of thousands of IP addresses, this implies it's reasonably well set up entity and not a hobbyist.

]]>https://forums.triplea-game.org/post/80057https://forums.triplea-game.org/post/80057Sun, 27 Sep 2026 19:31:27 GMT<![CDATA[Reply to Scraping Event - Brief Forum Outage - 2026-Sep-26 on Sun, 27 Sep 2026 19:28:38 GMT]]>@Cernel Forums is currently "open" to non-registered users, but limited in terms of categories. I agree that we should do a cleanup of categories.

]]>https://forums.triplea-game.org/post/80056https://forums.triplea-game.org/post/80056Sun, 27 Sep 2026 19:28:38 GMT<![CDATA[Reply to Scraping Event - Brief Forum Outage - 2026-Sep-26 on Sun, 27 Sep 2026 19:26:45 GMT]]>+1 for the counter attack.

A scrape essentially means every URL was crawled on the site and everything downloaded. This bot came in several waves, and yesterday they turned up the scrape rate by 10x. We disconnected the forum briefly to allow updates to be sent to the server. The bot went quiet when that happened, but came back 10 minutes later at a lower rate with a changed user-agent and started scraping again .

I can't help but think of the algorithm that was implemented, this persistent attempt, try again but a bit later, except under the radar. I kinda suspect that 10x rate was meant to be as fast as we could serve traffic without completely taking down the site, yet it did. Very likely the full thing was fully automated, someone grabbing data for whatever reason.

]]>https://forums.triplea-game.org/post/80055https://forums.triplea-game.org/post/80055Sun, 27 Sep 2026 19:26:45 GMT<![CDATA[Reply to Scraping Event - Brief Forum Outage - 2026-Sep-26 on Sun, 27 Sep 2026 19:25:16 GMT]]>Once there is the confidence to reopen the forum to non-registered, I definitely suggest keeping the sections "Ladders", "Tournaments", "Play By Forum" and "Play By Email" hidden to guests. Ladders is practically dead (and not sure why it is still there unless it is for use by private ladders too, like the one hosted by @eyeswideshut), Tournaments is very contingent (and currently vacant of organizers as far as I know) and PBF/PBEM are really a thing you don't need to see unless you are part of it, usually.

(I'm curious why the hell would anyone do this. What have they to gain?)

]]>https://forums.triplea-game.org/post/80054https://forums.triplea-game.org/post/80054Sun, 27 Sep 2026 19:25:16 GMT<![CDATA[Reply to Scraping Event - Brief Forum Outage - 2026-Sep-26 on Sun, 27 Sep 2026 18:58:35 GMT]]>@LaFayette

Is "scraping" essentially a massive "bot" attack ? although attack might not be the right description.

Just wondering if what triplea experienced was/is similar to what Djensen experiences at the other site periodically. It basically started after a nodeBB update, which was done here not all that long ago also.

Anyway, he put up some defenses as well, in case you are interested in what he did or he may be interested in what you did.

https://www.axisandallies.org/forums/topic/42468/bot-attacks?page=1

Edit
I just saw the other thread, so basically was a massive ai bot screwing everything up 😞

We need our own bot Army to counterattack 🙂

]]>https://forums.triplea-game.org/post/80053https://forums.triplea-game.org/post/80053Sun, 27 Sep 2026 18:58:35 GMT