AI Development Discussion and Feedback

  • Hey I have an AI suggestion from a statisticians perspective.

    Basically - if you are in a desperate position, attacking when its a 25% chance of victory sort of thing can be a very good idea. If time is not on your side, a desperate attack is good. That said, if you could code hard AI to be more willing to attack when their alliance has lower production than their enemy does or in other definable desperate situations. Example - if you are backed up to your capital (or other strategically important position) and your odds of victory on attack are higher than on defense, attack no matter the odds of victory on attack.

    There's also often situations (when low luck is off) where defeat is probable or expected TUV swing is negative but expected TUV swing in round 1 is positive. In these circumstances an attack and then retreat maneuver is a good idea. This is especially the case with low luck off where if the first round goes very well, the second round might be worth attacking in and you can just keep going until you need to retreat.

    Anything to make hard AI harder

  • Admin

    @scallen1 So at a high level both suggestions are good. Thoughts:

    1. Its very difficult to determine when you are in a desperate position across all rules/maps. And even when you are, its hard to determine if its better to attack or delay. That being said from more limited scope, if say the AI realizes it has no where left to run and has better odds/TUV to attack rather than defend then it probably should. Overall, I don't think this happens that often in games but gathering some specific save games would be helpful.

    2. Yeah, this is usually referred to as 'strafing' by most players. The AI will do this already but only in the case where it is trying to setup one of its allies to then attack the weaken territory. It doesn't do it just for TUV trades yet and is something I'd like to add.

  • @redrum I don't know if I've ever clearly noticed a "nowhere left to run" scenario for the AI but every time I play as germany on any map I'm guaranteed to eventually run into this in the Baltic.

    Desperate may not be the best way to describe things. In a vanilla WW2 game, Germany should see a 50 50 as a time to attack. Russia should see it as a time to defend cause time is not on their side. And I think you can generalise "time is not on my side" when ratio of production is like Idk greater than 5/4. You could put some small tolerance of attack when slightly disadvantaged (say if TUV swing is > -5% of combined TUV under the condition that your PUs per turn are much lower.because this usually means the situation is the best it is going to get.

    However if the AI actively tried to bleed (straffe) the human player you might remove the need for this except in last stand scenarios. And Last stand scenarios aren't things that probably have much impact on the overall game more than very rarely.

  • Admin

    @scallen1 So TUV comparisons are a tough because of the variety of maps. Some maps have lots of infrastructure units or give free units or have units sets that some nations have more efficient units, etc. The TUV of alliances on those maps means very little at a high level. Open up Civil War if you want to see what I mean.

    But yeah straffing is much more applicable and more tactical to figure out.

  • @redrum Ok makes sense. (yes civil war is a great example. I haven't put in the effort to learn that entire game and I imagine the AI is pretty bad at it)

  • Admin

    @scallen1 Yeah, the AI can't really handle Civil War and some of the other complex maps. Mostly it doesn't understand the unit production facilities or limited round battles.

  • Ok, had some odd ai behavior in a couple places while playing the new Middle Earth Battle for Arda map.
    map version 2.0.6 I'm using engine version *.11715
    probably issues due to all the fanciness in the map ofc

    Scenario 1 (with attached save)

    Angmar consistently (both times I tried it) sends two dragons to attack Rivendell; I have plenty of forces in place, so the dragons just die a horrible death while not accomplishing much. When I let the round play out there wasn't an attack from a subsequent nation to take advantage of it either, so it can't just be suiciding to setup another nation's successful attack.
    From the logs it seems to be marking it as a strafing attack though.
    my best guess is that the battle calc is screwing up the ai because you first have the one round air battle, which is just the two dragons and two eagles for one round; and the ai may only be pulling the calc from that battle, without considering the main battle that necessarily follows it. or something else about the air battle system is screwing it up.

    Scenario 2
    save wherein I noticed the issue had occurred: 0_1535911726380_Arda_AI_StarkhornUnsafe1.tsvg

    an earlier autosave which the ai usually repeats the same behavior from 0_1535911719927_Arda_AI_StarkhornUnsafeBeforehand.tsvg

    The ai often leaves some nazgul with inadequate defense in starkhorn. (4-5 nazgul with a couple snaga, while they're adjacent to 15 eagles).

    Looking through the logs, it seems like for some reason the ai doesn't recognize that the 15 eagles adjacent to it could attack starkhorn, and pose the biggest threat. Several other provinces nearby also don't show the prospect of being attacked by 15 eagles; instead the logs just show the biggest threat being some minor attack by Rohan.. so it's improperly marking territories as safe, then landing expensive units there. So the problem is in the routine that gets sets of different nations possible attackers; it's omitting the eagles as eligible attackers for some provinces. There aren't even kamikaze provinces, they're adjacent, so not sure why it's missing them.

  • Admin

    @zlefin There are some issues with AA units not being considered properly when strafing. This impacts BFA much more than most maps since so many units have an AA type attack. I'll look at adjusting this.

  • Admin

    @zlefin This PR should resolve the issues:

    I also made some tweaks to taking empty or lightly guarded territories in certain situations. The AI was trying to sometimes use a very expensive unit like a wizard to take an empty territory then realizing it didn't want to sacrifice that unit so then not attacking it at all.

  • Hey redrum, all, the AAA team has done a great job on AI, thanks for all your hard work! I have several ideas to improve the AI (mostly based on WaW, NWO, TRS maps). I include a note on severity for each, 1-5 with 1 being minor and 5 being a major flaw that causes the AI to lose from that single flaw, 100% of the time.

    1. add consideration for 2-hit units (e.g. I know it does not account for how juicy a target a damaged battleship is). Severity:2

    2. as the previous posts discuss, adding logic for a situation that is growing worse would be great but tremendously difficult to implement. Humans consider a) "strategic retreat" vs b) "hold as long as possible" (e.g. Germany's 6th army at Stalingrad) vs c) "do a risky attack now, before the noose tightens" (e.g. battle of the bulge). I don't think comparing total unit value would get you much closer to a stronger AI decision here. Side note, kudos, the AI already does a decent job of flying in allied airplanes to buy an extra round or two before being overwhelmed (and retreating them to safety when it's truly hopeless).
      Severity: 2

    3. the dream solution is machine learning. Is there a trove of played games somewhere that it could learn from?

    4. the AI frequently leaves high value units where they can be destroyed by infantry/air on my turn. I end up trading infantry for higher value units like cars/tanks/artillery/mech inf, and even planes round after round. This is especially noticeable on the territories adjacent to the main battle "fronts" where 2 large armies are squaring off/rotating around.Severity: 5

    5. related to (4), the AI does not hoard their specialist units (tanks, cars, motor inf, mech inf, etc) in their main battle stack. Any massing of these units is completely incidental to their arriving at the front. As soon as it can, the AI will dive those units deep to grab some territories for a turn (and leave them to die) removing the flexibility their unique abilities added to the battle stack. I'm thinking the AI could try to match an ideal ratio of inf/art/tanks/motor inf/mech etc for stacks, say 20/5/4/3/3. Severity:3

    6. most importantly, the AI does not execute a long term strategy for attacking/taking enemy capitals, or even strategically important targets that are more than a few zones away from them. 3 examples:
      A)in NWO, Germany could have twice as many units in the Eastern front than Russia has, but if the main Russian battle stack rotates around, it won't push to Moscow.
      😎 same thing in TRS, hisaichi/masaharu could have more than double China's/Russia's units, and won't push them past Mongolia.
      C) <all maps> the US builds several multiples more transports than it needs (as little as 5% are used at any time). So it doesn't create a steady stream of units to a front (or even an unsteady stream. The transports just sit there). Severity:5

    Thanks for reading!

  • Moderators

    @luhhlz I'll add a thing I said a while ago; the AI also undervalues double hitpoints units a lot, when deciding purchases. Even in my 270BC Cernel Variant/270BC 40% it almost never buys warelephant units, that about everyone with which I played agree are super good, especially when placements limits matter (like final build up before a major attack or menace of).

    No idea what the AI is doing with battleships purchases in WAW (?), but I believe all players agree battleships in WAW are one of the best units, especially for Americans.

  • @Cernel I'm no AI expert for sure but have battled many forms of it in games and after reading your post it reminded me of the AI choosing more Artillery than my regular human opponents do when making our general purchases which was usually buying the cheaper Infantry ... was a discussion I had with Redrum one time and he may have changed the algorithm (settings) on those Unit Purchases but I'm pretty sure you are talking about a settings tweak and I wonder what Redrum thinks to see if he needs to change something!

    Interesting though

  • Admin

    @luhhlz Good feedback. One thing I'll add is if you have any save games that show really poor AI behavior in these situations those are always super helpful as I can then test and make adjustments based on them. I'll try to provide some thoughts around each of your points:

    1. So valuing attacking damaged BBs should mostly be captured in the TUV calculations. If you have some save games where the AI passes up attacking damaged BBs where it clearly should I'd be interested in taking a look at them. But I haven't really seen this issue much.

    2. Yeah, this one is a tough one. The AI essentially tries to hold out as long as possible for capitals and factories but if it becomes absolutely overwhelming odds then it'll retreat. I think there could be more improvements here but its not the straightforward and usually this isn't losing the AI the game. There is also a lot of subjective opinion on this as different players have different thoughts on when to retreat vs making last stands in places.

    3. Given the complexity and the number of different rules/maps, machine learning is probably impossible given where it is at today. We also don't have nearly enough data and would need a tremendous amount of processing power. Even if you just focused on say revised, it would still be very very difficult and maybe impossible at this point let alone the larger and more complex maps.

    4. This one isn't easy. The AI tries to use cheaper units for trading territories and tries to spread out its planes across them. It generally tries to avoid exposing more expensive units but doesn't always succeed. If you have some good save games showing absolutely terrible trades that its making, I'd be interested in taking a look to see how it can be improved. I think sometimes it just needs to choose to trade less territories if it doesn't have the cheap units and planes.

    5. Purchasing is really hard. Mostly because every map has a different unit set, there are different game situations, and many different factories. The funny part is even good players often disagree on it for most maps. My gut says if you showed that ratio to expert WaW/NWO/TRS players, many of them would tell you its crap and give a completely different ratio. Also the ratio varies a lot when you consider USA vs Germany vs Russia which each are in a very different situation. Generally the AI prefers cheap units until its factories are far from the front then tries higher movement options. In some part because the AI isn't a good at protecting its high cost units and maximizing their impact. I'd actually like to completely rewrite the purchase logic as while it works ok-ish on most maps, I think needs to really overhauled to make much improvement on how it is now.

    6. A/B. The AI should be willing to march towards the opponents capital/factory if it has the numbers. The problem I usually see is it spreads itself too thin and attacks too many places rather than focusing its stack. It tends towards quantity of attacks rather than quality. This could definitely be improved.
      C. Transporting units is probably the weakest part of the AI and also maybe the hardest to do well. Balancing number of transports with available units and coordinating fleets to protect them to actually get to their target is... well... really difficult. I do agree that the AI tends to over buy transports but because it also doesn't use them as efficiently as humans do it does tend to need more. Because of this the AI tends to play land powers much more effectively then naval powers. In NWO, it plays germany/russia somewhat reasonably compared to USA/UK which end up just being all over the place as it tries to coordinate transports, fleets, and air power into something that useful. The other challenges besides transporting just being hard is that there are a lot of different transport rules across maps. If you try playing 270BC vs Global vs WaW, they all do transports a bit differently and the AI has to try to understand each of them.

  • Donators

    @redrum wow Good Action Here !

    Was really hoping #3 was gonna happen, but at least I'll sleep better : )