AI Development Discussion and Feedback
-
Hey I have an AI suggestion from a statisticians perspective.
Basically - if you are in a desperate position, attacking when its a 25% chance of victory sort of thing can be a very good idea. If time is not on your side, a desperate attack is good. That said, if you could code hard AI to be more willing to attack when their alliance has lower production than their enemy does or in other definable desperate situations. Example - if you are backed up to your capital (or other strategically important position) and your odds of victory on attack are higher than on defense, attack no matter the odds of victory on attack.
There's also often situations (when low luck is off) where defeat is probable or expected TUV swing is negative but expected TUV swing in round 1 is positive. In these circumstances an attack and then retreat maneuver is a good idea. This is especially the case with low luck off where if the first round goes very well, the second round might be worth attacking in and you can just keep going until you need to retreat.
Anything to make hard AI harder
-
@scallen1 So at a high level both suggestions are good. Thoughts:
-
Its very difficult to determine when you are in a desperate position across all rules/maps. And even when you are, its hard to determine if its better to attack or delay. That being said from more limited scope, if say the AI realizes it has no where left to run and has better odds/TUV to attack rather than defend then it probably should. Overall, I don't think this happens that often in games but gathering some specific save games would be helpful.
-
Yeah, this is usually referred to as 'strafing' by most players. The AI will do this already but only in the case where it is trying to setup one of its allies to then attack the weaken territory. It doesn't do it just for TUV trades yet and is something I'd like to add.
-
-
@redrum I don't know if I've ever clearly noticed a "nowhere left to run" scenario for the AI but every time I play as germany on any map I'm guaranteed to eventually run into this in the Baltic.
Desperate may not be the best way to describe things. In a vanilla WW2 game, Germany should see a 50 50 as a time to attack. Russia should see it as a time to defend cause time is not on their side. And I think you can generalise "time is not on my side" when ratio of production is like Idk greater than 5/4. You could put some small tolerance of attack when slightly disadvantaged (say if TUV swing is > -5% of combined TUV under the condition that your PUs per turn are much lower.because this usually means the situation is the best it is going to get.
However if the AI actively tried to bleed (straffe) the human player you might remove the need for this except in last stand scenarios. And Last stand scenarios aren't things that probably have much impact on the overall game more than very rarely.
-
@scallen1 So TUV comparisons are a tough because of the variety of maps. Some maps have lots of infrastructure units or give free units or have units sets that some nations have more efficient units, etc. The TUV of alliances on those maps means very little at a high level. Open up Civil War if you want to see what I mean.
But yeah straffing is much more applicable and more tactical to figure out.
-
@redrum Ok makes sense. (yes civil war is a great example. I haven't put in the effort to learn that entire game and I imagine the AI is pretty bad at it)
-
@scallen1 Yeah, the AI can't really handle Civil War and some of the other complex maps. Mostly it doesn't understand the unit production facilities or limited round battles.
-
Ok, had some odd ai behavior in a couple places while playing the new Middle Earth Battle for Arda map.
map version 2.0.6 I'm using engine version *.11715
probably issues due to all the fanciness in the map ofcScenario 1 (with attached save)
0_1535910319428_Arda_AI_dragonattackelves.tsvgAngmar consistently (both times I tried it) sends two dragons to attack Rivendell; I have plenty of forces in place, so the dragons just die a horrible death while not accomplishing much. When I let the round play out there wasn't an attack from a subsequent nation to take advantage of it either, so it can't just be suiciding to setup another nation's successful attack.
From the logs it seems to be marking it as a strafing attack though.
my best guess is that the battle calc is screwing up the ai because you first have the one round air battle, which is just the two dragons and two eagles for one round; and the ai may only be pulling the calc from that battle, without considering the main battle that necessarily follows it. or something else about the air battle system is screwing it up.Scenario 2
save wherein I noticed the issue had occurred: 0_1535911726380_Arda_AI_StarkhornUnsafe1.tsvgan earlier autosave which the ai usually repeats the same behavior from 0_1535911719927_Arda_AI_StarkhornUnsafeBeforehand.tsvg
The ai often leaves some nazgul with inadequate defense in starkhorn. (4-5 nazgul with a couple snaga, while they're adjacent to 15 eagles).
Looking through the logs, it seems like for some reason the ai doesn't recognize that the 15 eagles adjacent to it could attack starkhorn, and pose the biggest threat. Several other provinces nearby also don't show the prospect of being attacked by 15 eagles; instead the logs just show the biggest threat being some minor attack by Rohan.. so it's improperly marking territories as safe, then landing expensive units there. So the problem is in the routine that gets sets of different nations possible attackers; it's omitting the eagles as eligible attackers for some provinces. There aren't even kamikaze provinces, they're adjacent, so not sure why it's missing them.
-
@zlefin There are some issues with AA units not being considered properly when strafing. This impacts BFA much more than most maps since so many units have an AA type attack. I'll look at adjusting this.
-
@zlefin This PR should resolve the issues: https://github.com/triplea-game/triplea/pull/4110
I also made some tweaks to taking empty or lightly guarded territories in certain situations. The AI was trying to sometimes use a very expensive unit like a wizard to take an empty territory then realizing it didn't want to sacrifice that unit so then not attacking it at all.
-
Hey redrum, all, the AAA team has done a great job on AI, thanks for all your hard work! I have several ideas to improve the AI (mostly based on WaW, NWO, TRS maps). I include a note on severity for each, 1-5 with 1 being minor and 5 being a major flaw that causes the AI to lose from that single flaw, 100% of the time.
-
add consideration for 2-hit units (e.g. I know it does not account for how juicy a target a damaged battleship is). Severity:2
-
as the previous posts discuss, adding logic for a situation that is growing worse would be great but tremendously difficult to implement. Humans consider a) "strategic retreat" vs b) "hold as long as possible" (e.g. Germany's 6th army at Stalingrad) vs c) "do a risky attack now, before the noose tightens" (e.g. battle of the bulge). I don't think comparing total unit value would get you much closer to a stronger AI decision here. Side note, kudos, the AI already does a decent job of flying in allied airplanes to buy an extra round or two before being overwhelmed (and retreating them to safety when it's truly hopeless).
Severity: 2 -
the dream solution is machine learning. Is there a trove of played games somewhere that it could learn from?
-
the AI frequently leaves high value units where they can be destroyed by infantry/air on my turn. I end up trading infantry for higher value units like cars/tanks/artillery/mech inf, and even planes round after round. This is especially noticeable on the territories adjacent to the main battle "fronts" where 2 large armies are squaring off/rotating around.Severity: 5
-
related to (4), the AI does not hoard their specialist units (tanks, cars, motor inf, mech inf, etc) in their main battle stack. Any massing of these units is completely incidental to their arriving at the front. As soon as it can, the AI will dive those units deep to grab some territories for a turn (and leave them to die) removing the flexibility their unique abilities added to the battle stack. I'm thinking the AI could try to match an ideal ratio of inf/art/tanks/motor inf/mech etc for stacks, say 20/5/4/3/3. Severity:3
-
most importantly, the AI does not execute a long term strategy for attacking/taking enemy capitals, or even strategically important targets that are more than a few zones away from them. 3 examples:
A)in NWO, Germany could have twice as many units in the Eastern front than Russia has, but if the main Russian battle stack rotates around, it won't push to Moscow.
same thing in TRS, hisaichi/masaharu could have more than double China's/Russia's units, and won't push them past Mongolia.
C) <all maps> the US builds several multiples more transports than it needs (as little as 5% are used at any time). So it doesn't create a steady stream of units to a front (or even an unsteady stream. The transports just sit there). Severity:5
Thanks for reading!
-
-
@luhhlz I'll add a thing I said a while ago; the AI also undervalues double hitpoints units a lot, when deciding purchases. Even in my 270BC Cernel Variant/270BC 40% it almost never buys warelephant units, that about everyone with which I played agree are super good, especially when placements limits matter (like final build up before a major attack or menace of).
No idea what the AI is doing with battleships purchases in WAW (?), but I believe all players agree battleships in WAW are one of the best units, especially for Americans.
-
@Cernel I'm no AI expert for sure but have battled many forms of it in games and after reading your post it reminded me of the AI choosing more Artillery than my regular human opponents do when making our general purchases which was usually buying the cheaper Infantry ... was a discussion I had with Redrum one time and he may have changed the algorithm (settings) on those Unit Purchases but I'm pretty sure you are talking about a settings tweak and I wonder what Redrum thinks to see if he needs to change something!
Interesting though
-
@luhhlz Good feedback. One thing I'll add is if you have any save games that show really poor AI behavior in these situations those are always super helpful as I can then test and make adjustments based on them. I'll try to provide some thoughts around each of your points:
-
So valuing attacking damaged BBs should mostly be captured in the TUV calculations. If you have some save games where the AI passes up attacking damaged BBs where it clearly should I'd be interested in taking a look at them. But I haven't really seen this issue much.
-
Yeah, this one is a tough one. The AI essentially tries to hold out as long as possible for capitals and factories but if it becomes absolutely overwhelming odds then it'll retreat. I think there could be more improvements here but its not the straightforward and usually this isn't losing the AI the game. There is also a lot of subjective opinion on this as different players have different thoughts on when to retreat vs making last stands in places.
-
Given the complexity and the number of different rules/maps, machine learning is probably impossible given where it is at today. We also don't have nearly enough data and would need a tremendous amount of processing power. Even if you just focused on say revised, it would still be very very difficult and maybe impossible at this point let alone the larger and more complex maps.
-
This one isn't easy. The AI tries to use cheaper units for trading territories and tries to spread out its planes across them. It generally tries to avoid exposing more expensive units but doesn't always succeed. If you have some good save games showing absolutely terrible trades that its making, I'd be interested in taking a look to see how it can be improved. I think sometimes it just needs to choose to trade less territories if it doesn't have the cheap units and planes.
-
Purchasing is really hard. Mostly because every map has a different unit set, there are different game situations, and many different factories. The funny part is even good players often disagree on it for most maps. My gut says if you showed that ratio to expert WaW/NWO/TRS players, many of them would tell you its crap and give a completely different ratio. Also the ratio varies a lot when you consider USA vs Germany vs Russia which each are in a very different situation. Generally the AI prefers cheap units until its factories are far from the front then tries higher movement options. In some part because the AI isn't a good at protecting its high cost units and maximizing their impact. I'd actually like to completely rewrite the purchase logic as while it works ok-ish on most maps, I think needs to really overhauled to make much improvement on how it is now.
-
A/B. The AI should be willing to march towards the opponents capital/factory if it has the numbers. The problem I usually see is it spreads itself too thin and attacks too many places rather than focusing its stack. It tends towards quantity of attacks rather than quality. This could definitely be improved.
C. Transporting units is probably the weakest part of the AI and also maybe the hardest to do well. Balancing number of transports with available units and coordinating fleets to protect them to actually get to their target is... well... really difficult. I do agree that the AI tends to over buy transports but because it also doesn't use them as efficiently as humans do it does tend to need more. Because of this the AI tends to play land powers much more effectively then naval powers. In NWO, it plays germany/russia somewhat reasonably compared to USA/UK which end up just being all over the place as it tries to coordinate transports, fleets, and air power into something that useful. The other challenges besides transporting just being hard is that there are a lot of different transport rules across maps. If you try playing 270BC vs Global vs WaW, they all do transports a bit differently and the AI has to try to understand each of them.
-
-
@redrum wow Good Action Here !
Was really hoping #3 was gonna happen, but at least I'll sleep better : )
-
Some thoughts about "strafing".
I don't know what's already in the AI and what you already have planned, but not implemented yet. Here is some feedback based on my observations of Hard AI in the latest stabile TripleA version (1.9.0.0.13066).
I'd like to think for AIplayer1 to make a strafing attack (negative TUV attack) to set up a follow-up attack from AIplayer2, there are a few things that the AI could improve on to evaluate if it's a good idea or not. The easiest to implement, I figure would be simulating "total TUV" for all attacks. So, the odds calculator for the AI needs to run the first attack TUV and immediately the 2nd attack TUV (starting with units remaining after 1st attack) and add the TUVs together and then repeat for however many simulations runs the AI uses for it to be statistically significant. The average TUV swing can then be compared with what you'd get if ONLY AIplayer2 attacks. If the total TUV swing is not improved by AIplayer1's potential strafing attack, then the AI should consider not making that strafing attack.
So essentially what I'm saying is there should be an odds calculator that can consider several attacks in succession (or at the very least 2 attacks). I imagine if a bit more generalized this could also be used to consider potential counter-attacks by enemies or even combinations of strafing attacks, friendly follow-up attacks and enemy counter-attacks, but obviously it gets more complicated the more attacks are involved as you for every attack need to predict if that player chooses to either not attack at all or commit some troops elsewhere, or if their troops will even be there come their turn, which might not be easy to determine by just looking at battles in 1 territory.
-
@Quorthon4 I do not know the technicalities of what you posted but I do want to re-mention something I've said a bunch of times about a "learning AI" which for me was simply adding a saved "WinnersCapturedTerritories" file to be used by the AI every game for direction of focus which would ofcourse improve the AI as more and more games are played. This could even be "machine learning" which as I described is not too complicated and not require that much more resources other than a saved file and some memory space ... so as pertaining to your post ... I think giving the AI direction and goals in its movements would improve the AI more than just letting it decide by TUV what move to make next!
This thread is about improving the AI and I've yet to see anyone give a better suggestion than mine but let's wait for the devs response to see if your idea or mine are in the future for this AI!
-
@Captain-Crunch first i heard about that idea but i was a little late to the new forum party, anyway, i like it!
-
@Quorthon4 Good thoughts. The AI does some limiting strafing already and essentially follows a process similar to what you describe. It primarily just looks to use it on high value territories like capitals and factories. If you test revised or v3 and give USA and UK large forces next to Germany's capital where neither is large enough to capture by themselves but if they both attack then they can, you should see the AI strafe to capture it.
There are a few challenges with expanding its usage:
- Its slow as you are running multiple BC runs for a single battle so using it across all potential battles could slow the AI considerably especially on large maps.
- Adding when to retreat into it so that the AI would say just use Player1 to attack for a 1 round to weaken it then have Player2 attack to capture it instead of Player1 attacking til all their units are dead. This quickly multiplies the number of potential scenarios to calculate and complicates which is ideal.
- Strafing has different uses in dice vs LL. In LL, you can guarantee a certain number of hits and always know you can retreat where this isn't the case for dice. This often becomes the case when you have a large unit stack then your opponent but are defending your capital so need to leave most of your units there so you just attack for a round or 2 to inflict casualties then retreat back.
If you have any save games showing scenarios that the AI could greatly benefit from strafing please upload them here so I can take a look.
-
@redrum @Quorthon4 i've seen AI strafe, does it well in 1914 domination map in fact, particularly in the eastern front where Germany and Austria basically need to team up to overpower Russia unless either ignores their western fronts.
-
@redrum I have a saved game where AI uses strafing in a situation where it's highly questionable. Especially look at Austria turn 5 and Ottomans turn 5 (both attacking Warsaw). The intent is clearly to weaken Warsaw for Germany's turn 6. Germany probably should attack Warsaw on its turn 6, but ends up not doing it, I'm not entirely sure why, but I'm guessing for some reason the AI thinks moving troops towards Greece should be a higher priority than attacking Warsaw.
great_war-allies.tsvgSpeed is surely a priority for AI and I get the part of it quickly multiplying and you probably have to put limits to how complicated things the AI is allowed to consider. You might have to cut some corners, but I figure there's certainly room for improvement without slowing the AI down significantly.
