AI Development Discussion and Feedback

[alkexr]

@redrum Calling it a map would be a bit of an exaggeration... this is a copied version of a map of mine for all sorts of xml experiments. I think I restored all previous experiments before trying this. https://drive.google.com/open?id=1T8QwATSzqb1SQgaSbDeqrZWwF5n2afTQ

[Cernel]

@redrum So now the AI also places movement 0 units? Because legionaire in 270BC can place only forts.

EDIT: Actually, the legio of 270BC would be a peculiar example, in that such unit should really receive no bonus (or maybe like a 0.05) to its value for being able to place units (limited to fort). It is rather the fort that should be valued a little bit more for being placeable by the legio, thus not only in cities (main example is placing some forts in Messana without a city in it). It would be really wrong if the AI would build significantly more legios due to the fact that they have some placement ability, in this case (really, it boosts the fort, not really the legio).

[redrum]

@cernel Yeah, I had it mixed up. I had tested 270bc to make sure the AI didn't place legions as factories since they can only place forts. When purchasing/placing factories, the AI doesn't have a sense of 'what units a factory can build' yet since it was mostly created around A&A maps. Whether a factory is infra shouldn't really matter but that limitation was put in since a few maps have regular units that can build a very limited set of units like legions and I didn't want the AI to see them as factories.

That being said, if it has non-isInfra factory units then it should consider placing units on them so I'll have to look at that.

[Frostion]

@redrum That sounds great and would really mean a lot for future maps and how compatible the AI is.

[redrum]

@alkexr Here is the PR: https://github.com/triplea-game/triplea/pull/3645

I tested your LAB map before the changes where the AI doesn't place any units since cities are non-infra and then after the AI places normally just like if they were infra factories.

I also tested 270BC since that was the main map that had non-infra factories that caused issues. I needed to add some additional checks when purchasing/placing sea units so that each factory adjacent to a sea territory is treated separately instead of just adding their production together. Otherwise the AI thinks legions add 1 production to sea territories to produce boats.

This change also should help support maps that have different types of factories (think barracks and shipyard) adjacent to the same sea territory as the AI should now consider each of their production and what they can produce separately.

[Cernel]

With 1.9.0.0.11638, starting "Red Sun Over China" with Hard AI, the AI always attacks Liaocheng (that I think is a good decision), but does it with insufficient forces (basically a suicide).
I guess the AI correctly sees that is a good attack, with an over +20 swing, but then fails to actually send everything it can, in the moment of actually moving.
On top of that, it also retreats to Tsinan, that is the worst decision.
Not including a savegame as I believe you can see it by just assigning Nationalist to Hard AI and starting.
Probably this is such an unconventional map that people will want to try it out with AI before playing for real, so this may be worth figuring out.
For info, you can (IMO should) go in Liaocheng with:
6 infantry
1 conscript
13 elite
3 light artillery
4 heavy artillery
6 armour
1 fighter
So I believe that's what the AI should do all games.

[redrum]

@cernel I think the AI doesn't handle land canals properly yet. So it ends up not being able to construct appropriate routes to move all the units so some of them fail.

[Cernel]

@redrum That is probably the case for not sending the fighter, but for the remaining part what happens when I test it, nor it should matter in any ways as, even if the AI would know nothing about canals, canals are negative, so when they are all open the AI doesn't need to know they exist (thus only case is the fighter blocked by the rail canals). Also was surprised that the AI (differently from the autoselect) correctly picks the heavy_artillery instead of the elite for land transport.
All the times I've tested it, the issue is that the AI is not sending all units from Tsinan and Lungkow, that I think has nothing at all to do with canals (the canals are not allowing you to jump the city; they are possibly impeding you to do that).
Also I really believe the retreat choice is a separate issue, that can be a problem also in case of correct movement but getting diced, as it seems the AI is unable to see the safest spot to retreat to.

[redrum]

@cernel Yeah, retreat is a separate calculation and does have some logic to pick which territory though never optimized it a ton as the AI doesn't end up retreating that often (usually only when things go really badly). I'll take a look at this eventually. First I want to work on improving the route finder as that impacts PvP games on it then I'll work on improving the AI.

[scallen1]

Hey I have an AI suggestion from a statisticians perspective.

Basically - if you are in a desperate position, attacking when its a 25% chance of victory sort of thing can be a very good idea. If time is not on your side, a desperate attack is good. That said, if you could code hard AI to be more willing to attack when their alliance has lower production than their enemy does or in other definable desperate situations. Example - if you are backed up to your capital (or other strategically important position) and your odds of victory on attack are higher than on defense, attack no matter the odds of victory on attack.

There's also often situations (when low luck is off) where defeat is probable or expected TUV swing is negative but expected TUV swing in round 1 is positive. In these circumstances an attack and then retreat maneuver is a good idea. This is especially the case with low luck off where if the first round goes very well, the second round might be worth attacking in and you can just keep going until you need to retreat.

Anything to make hard AI harder

[redrum]

@scallen1 So at a high level both suggestions are good. Thoughts:

1. Its very difficult to determine when you are in a desperate position across all rules/maps. And even when you are, its hard to determine if its better to attack or delay. That being said from more limited scope, if say the AI realizes it has no where left to run and has better odds/TUV to attack rather than defend then it probably should. Overall, I don't think this happens that often in games but gathering some specific save games would be helpful.

2. Yeah, this is usually referred to as 'strafing' by most players. The AI will do this already but only in the case where it is trying to setup one of its allies to then attack the weaken territory. It doesn't do it just for TUV trades yet and is something I'd like to add.

[scallen1]

@redrum I don't know if I've ever clearly noticed a "nowhere left to run" scenario for the AI but every time I play as germany on any map I'm guaranteed to eventually run into this in the Baltic.

Desperate may not be the best way to describe things. In a vanilla WW2 game, Germany should see a 50 50 as a time to attack. Russia should see it as a time to defend cause time is not on their side. And I think you can generalise "time is not on my side" when ratio of production is like Idk greater than 5/4. You could put some small tolerance of attack when slightly disadvantaged (say if TUV swing is > -5% of combined TUV under the condition that your PUs per turn are much lower.because this usually means the situation is the best it is going to get.

However if the AI actively tried to bleed (straffe) the human player you might remove the need for this except in last stand scenarios. And Last stand scenarios aren't things that probably have much impact on the overall game more than very rarely.

[redrum]

@scallen1 So TUV comparisons are a tough because of the variety of maps. Some maps have lots of infrastructure units or give free units or have units sets that some nations have more efficient units, etc. The TUV of alliances on those maps means very little at a high level. Open up Civil War if you want to see what I mean.

But yeah straffing is much more applicable and more tactical to figure out.

[scallen1]

@redrum Ok makes sense. (yes civil war is a great example. I haven't put in the effort to learn that entire game and I imagine the AI is pretty bad at it)

[redrum]

@scallen1 Yeah, the AI can't really handle Civil War and some of the other complex maps. Mostly it doesn't understand the unit production facilities or limited round battles.

[zlefin]

Ok, had some odd ai behavior in a couple places while playing the new Middle Earth Battle for Arda map.
map version 2.0.6 I'm using engine version *.11715
probably issues due to all the fanciness in the map ofc

Scenario 1 (with attached save)
0_1535910319428_Arda_AI_dragonattackelves.tsvg

Angmar consistently (both times I tried it) sends two dragons to attack Rivendell; I have plenty of forces in place, so the dragons just die a horrible death while not accomplishing much. When I let the round play out there wasn't an attack from a subsequent nation to take advantage of it either, so it can't just be suiciding to setup another nation's successful attack.
From the logs it seems to be marking it as a strafing attack though.
my best guess is that the battle calc is screwing up the ai because you first have the one round air battle, which is just the two dragons and two eagles for one round; and the ai may only be pulling the calc from that battle, without considering the main battle that necessarily follows it. or something else about the air battle system is screwing it up.

Scenario 2
save wherein I noticed the issue had occurred: 0_1535911726380_Arda_AI_StarkhornUnsafe1.tsvg

an earlier autosave which the ai usually repeats the same behavior from 0_1535911719927_Arda_AI_StarkhornUnsafeBeforehand.tsvg

The ai often leaves some nazgul with inadequate defense in starkhorn. (4-5 nazgul with a couple snaga, while they're adjacent to 15 eagles).

Looking through the logs, it seems like for some reason the ai doesn't recognize that the 15 eagles adjacent to it could attack starkhorn, and pose the biggest threat. Several other provinces nearby also don't show the prospect of being attacked by 15 eagles; instead the logs just show the biggest threat being some minor attack by Rohan.. so it's improperly marking territories as safe, then landing expensive units there. So the problem is in the routine that gets sets of different nations possible attackers; it's omitting the eagles as eligible attackers for some provinces. There aren't even kamikaze provinces, they're adjacent, so not sure why it's missing them.

[redrum]

@zlefin There are some issues with AA units not being considered properly when strafing. This impacts BFA much more than most maps since so many units have an AA type attack. I'll look at adjusting this.

[redrum]

@zlefin This PR should resolve the issues: https://github.com/triplea-game/triplea/pull/4110

I also made some tweaks to taking empty or lightly guarded territories in certain situations. The AI was trying to sometimes use a very expensive unit like a wizard to take an empty territory then realizing it didn't want to sacrifice that unit so then not attacking it at all.

[luhhlz]

Hey redrum, all, the AAA team has done a great job on AI, thanks for all your hard work! I have several ideas to improve the AI (mostly based on WaW, NWO, TRS maps). I include a note on severity for each, 1-5 with 1 being minor and 5 being a major flaw that causes the AI to lose from that single flaw, 100% of the time.

1. add consideration for 2-hit units (e.g. I know it does not account for how juicy a target a damaged battleship is). Severity:2

2. as the previous posts discuss, adding logic for a situation that is growing worse would be great but tremendously difficult to implement. Humans consider a) "strategic retreat" vs b) "hold as long as possible" (e.g. Germany's 6th army at Stalingrad) vs c) "do a risky attack now, before the noose tightens" (e.g. battle of the bulge). I don't think comparing total unit value would get you much closer to a stronger AI decision here. Side note, kudos, the AI already does a decent job of flying in allied airplanes to buy an extra round or two before being overwhelmed (and retreating them to safety when it's truly hopeless).
   Severity: 2

3. the dream solution is machine learning. Is there a trove of played games somewhere that it could learn from?

4. the AI frequently leaves high value units where they can be destroyed by infantry/air on my turn. I end up trading infantry for higher value units like cars/tanks/artillery/mech inf, and even planes round after round. This is especially noticeable on the territories adjacent to the main battle "fronts" where 2 large armies are squaring off/rotating around.Severity: 5

5. related to (4), the AI does not hoard their specialist units (tanks, cars, motor inf, mech inf, etc) in their main battle stack. Any massing of these units is completely incidental to their arriving at the front. As soon as it can, the AI will dive those units deep to grab some territories for a turn (and leave them to die) removing the flexibility their unique abilities added to the battle stack. I'm thinking the AI could try to match an ideal ratio of inf/art/tanks/motor inf/mech etc for stacks, say 20/5/4/3/3. Severity:3

6. most importantly, the AI does not execute a long term strategy for attacking/taking enemy capitals, or even strategically important targets that are more than a few zones away from them. 3 examples:
   A)in NWO, Germany could have twice as many units in the Eastern front than Russia has, but if the main Russian battle stack rotates around, it won't push to Moscow.
   B) same thing in TRS, hisaichi/masaharu could have more than double China's/Russia's units, and won't push them past Mongolia.
   C) <all maps> the US builds several multiples more transports than it needs (as little as 5% are used at any time). So it doesn't create a steady stream of units to a front (or even an unsteady stream. The transports just sit there). Severity:5

Thanks for reading!

[Cernel]

@luhhlz I'll add a thing I said a while ago; the AI also undervalues double hitpoints units a lot, when deciding purchases. Even in my 270BC Cernel Variant/270BC 40% it almost never buys warelephant units, that about everyone with which I played agree are super good, especially when placements limits matter (like final build up before a major attack or menace of).

No idea what the AI is doing with battleships purchases in WAW (?), but I believe all players agree battleships in WAW are one of the best units, especially for Americans.