Improving AI

[alkexr]

@LaFayette said in Improving AI:

RE: machine learning, the AI could play itself and generate a dataset that way. Unfortunately short term that is not very feasible as the code is in a messy state which makes big changes or additions prohibitive.

That sort of learning requires even more data, doesn't it? Like tens of thousands, at the very least? I've thought of this idea, but AI self play can take 10-20 minutes on larger maps each turn. Games can last as long as half a day. Even with multiple people running it on their PCs day and night, it seems like way too much time even for a single map. It might be worth trying for the smallest maps, but relying on it - no.

The better option might be "machine teaching", of some sort. A human would be much better at guessing the correct territory values (or any other parameter), and adjusting after playing games against it and seeing it in action.

[redrum]

Wow... quite a thread.

So I'll add my thoughts around a few of the topics here. Is it in theory possible to create some form of simulation/machine learning/learning AI based on trying to generate lots of played games and having it improve based on patterns of those games? Yes.

Now is it feasible to do that with any reasonable amount of resources/time that would actually produce a useful result for a complex TripleA map? Personally, I don't believe that it is at this point.

Breaking that down a bit further, if you say took just 1 TripleA map say revised, removed 95% of the starting units, reduced overall territory production values to 20%, changed all sea territories to land, removed all sea units, and removed all air units then created a base AI that understands the rules then optimized the performance of that AI then acquired significant compute power and data storage then you could probably create a some type of machine learning AI and if anyone really was serious about doing it that is essentially where I would recommend starting with a very stripped down version of a single map to at least prove they can do it with that and see how long it would take.

Even if you achieve an expert level AI on that stripped down map, being able to then do it for Global or Total World War or Civil War would be orders of magnitude more difficult and require a lot more time and resources.

So IMO, that mostly leaves us with trying to make some form of weighted and limited simulation AI which is what we have as a base today. If done right, this could be improved to beat 99% of players and run within a reasonable amount of time. There is already a list of items that could be improved/added and mostly we'd need more people to post examples of where the AI makes poor decisions.

I would love to focus more on improving the AI and that was what originally brought me to developing TripleA as the previous AI was... well... terrible. Ideally I'd really like to just develop the AI and create some maps but given that we only have a few developers working on TripleA in their spare time, I end up working on a lot of other things around the engine and bugs.

[RogerCooper]

Given the complexity of creating an AI for a game like TripleA, the current hard AI performs quite well, especially on a tactical level. I often play 1 country and let the AI take the rest. This generally makes for challenging and entertaining games.

What I would like to see is the ability of scenario designers to give objectives and directives to the base AI. This should be doable without undue effort.

For example, I am reworking the Axis & Allies: D-Day game in TripleA. I found that making the game's 3 objective cities German 'capitals', the AI played significantly better.

[redrum]

@RogerCooper Agree. Per map configuration, is definitely on the list of things to add. The AI also needs to be able to better handle various capital rules and victory cities.

[RogerCooper]

@redrum said in Improving AI:

@RogerCooper Agree. Per map configuration, is definitely on the list of things to add. The AI also needs to be able to better handle various capital rules and victory cities.

The AI handles capitals pretty well, responding strongly the threats against them and making some strategic effort in the direction of the capital. Something as simple as a settable VictoryCity bonus for AI computations of territory value would improve play. A little more complex would be the ability designate a territory as a "StrategicObjective", which the AI would try to attack or defend as if it were a capital.

[Schulz]

@RogerCooper

Roughly how much bid do you give to AI?

[redrum]

@RogerCooper Sorry I meant difference capital rules. The AI treats all its capitals as must hold or it can't produce units and loses all its PUs. But some games have capitals where losing them doesn't actually impact anything. But yeah, right now the AI essentially looks to primarily defend and try to capture capitals and factories. Having a way to designate other important territories for it to focus on would be useful.

@Schulz In my experience, it varies significantly across maps and just how strong of a player you are. You can also instead of a bid or in addition to give the AI bonus flat or % income.

[RogerCooper]

@Schulz I usually don't give the AI a bid, unless the scenario is unbalanced. I take only 1 country, leaving me with AI allies as well as AI foes. We don't want to really lose to the AI.