Proposal to consolidate Unit options into UnitList & Make explicit which units are available to which players

LaFayette

There are two problems today with XML specs for units:

there is nothing that explicitly states which units are available to which players
the unit configuration is scattered & not well modeled in XML

For example, for a fighter unit, today we have:

  <unitList>
    <unit name="fighter"/>
  </unitList>

    <attachment name="unitAttachment" attachTo="fighter" javaClass="games.strategy.triplea.attachments.UnitAttachment" type="unitType">
      <option name="movement" value="4"/>
      <option name="carrierCost" value="1"/>
      <option name="isAir" value="true"/>
      <option name="attack" value="3"/>
      <option name="defense" value="4"/>
      <option name="canScramble" value="true"/>
      <option name="maxScrambleDistance" value="1"/>
      <option name="canIntercept" value="true"/>
      <option name="canEscort" value="true"/>
      <option name="airDefense" value="2"/>
      <option name="airAttack" value="1"/>
      <option name="canBeGivenByTerritoryTo" value="British"/>
    </attachment>

To fix the modeling, the first proposed change would be to consolidate these two XML blocks, EG:

  <unitList>
    <unit name="fighter">
      <option name="movement" value="4"/>
      <option name="carrierCost" value="1"/>
      <option name="isAir" value="true"/>
      <option name="attack" value="3"/>
      <option name="defense" value="4"/>
      <option name="canScramble" value="true"/>
      <option name="maxScrambleDistance" value="1"/>
      <option name="canIntercept" value="true"/>
      <option name="canEscort" value="true"/>
      <option name="airDefense" value="2"/>
      <option name="airAttack" value="1"/>
      <option name="canBeGivenByTerritoryTo" value="British"/>
    </unit>
  </unitList>

Second to solve (1), we need to list the players that are allowed to the specific unit types. I'm thinking to keep it easy, this is an optional list and if not specified then the unit is allowed to all players. So for example, if we want to restrict a unit to a specific player, we could do:

  <unitList>
    <unit name="allied_fighter">
      <option name="movement" value="4"/>
      <option name="carrierCost" value="1"/>
      <option name="isAir" value="true"/>
      <option name="attack" value="3"/>
      <option name="defense" value="4"/>
      <option name="canScramble" value="true"/>
      <option name="maxScrambleDistance" value="1"/>
      <option name="canIntercept" value="true"/>
      <option name="canEscort" value="true"/>
      <option name="airDefense" value="2"/>
      <option name="airAttack" value="1"/>
      <option name="canBeGivenByTerritoryTo" value="British"/>
      <availableTo>
              <player>Americans</player>
              <player>British</player>
              <player>Russians</player>
       </availableTo>             
    </unit>
  </unitList>

Getting this kind of update in would require some gymnastics in the code, specifically I'm thinking we'll need a "XML-Spec" version flag so that the game engine can switch between reading the above tags and also still read all of the existing maps. I think that is do-able.

I'm wondering what people think of this direction?

Cernel

@LaFayette said in Proposal to consolidate Unit options into UnitList & Make explicit which units are available to which players:

There are two problems today with XML specs for units:

there is nothing that explicitly states which units are available to which players

What do you mean, exactly, by "available units"? Currently, on the matter, you define in the game file (XML) what units each player can purchase (production frontier) and has (unit initialize) (sometimes players have units that they cannot purchase).

What would the availableTo thing exactly do? Only what currently the engine already infers by looking at the production frontiers and the units already on the board or something else? I can see a few things, for which it may be useful, but I wonder what you are seeing.

the unit configuration is scattered & not well modeled in XML

The current configuration is fine for me, as long as defining units abilities and production capacities, even though I would prefer if the abilities themselves are the attachments, with the various options contributing to the same ability grouped together (instead of being options of a unit's attachment), assigned to specific units and players (currently, only the support abilities work this way, and I would prefer all units abilities working like the support attachments already do).

wc_sumpton

@LaFayette

Units follow the same pattern as territories, technologies, territory effects, relationships, etc... So I don't see it as a problem, unless everything would be coded to follow the same steps.

Second the use of productionFrontiers tells which units are available to which players. To change a units availability, only the productionFrontier for that player needs to be changed, either directly edited, or within the xml itself. This would seems to add another step.

That being said, could the unit cost also be added to this information?

Just thinking out loud.

Cheers...

LaFayette

Production frontier is not the full story. For example, the neutrals. Those are determined largely by which images are available. Same goes for all players, it is the full set of:

units on board
production frontier
available unit images

The latter is a bit of a problem as the engine should not necessarily be removing units just because an image is removed. Overall the triplet is certainly a bit of a hodge, it is because it's not explicitly stated "This unit belongs to this player"

LaFayette

@wc_sumpton said in Proposal to consolidate Unit options into UnitList & Make explicit which units are available to which players:

Units follow the same pattern as territories, technologies, territory effects, relationships, etc... So I don't see it as a problem, unless everything would be coded to follow the same steps.

Doing an all or none would make it so that nothing happens, I regret we need to be incremental about it to keep the progression of updates managable. The 'pattern' that exists is also not necessarily a helpful one. It's in addition a 'fake' pattern as XML tags are not going to be uniform, but furthermore the modeling is kinda poor and would be excellent to improve.

Cernel

@LaFayette said in Proposal to consolidate Unit options into UnitList & Make explicit which units are available to which players:

Production frontier is not the full story. For example, the neutrals. Those are determined largely by which images are available. Same goes for all players, it is the full set of:

units on board

production frontier

available unit images

The latter is a bit of a problem as the engine should not necessarily be removing units just because an image is removed. Overall the triplet is certainly a bit of a hodge, it is because it's not explicitly stated "This unit belongs to this player"

Ok. Now I get where you are aiming at. I tend to think "suggesting" the program what are the "available" units to each player (if it is not feasible for the program infallibly to infer it itself, which would be the best solution), so to give you the correct list in unit help or the like, is something that should rather belong to the skin, not to the game file (xml), even though this has the challenge that different games for the same map may have different available units per player.

LaFayette

@wc_sumpton said in Proposal to consolidate Unit options into UnitList & Make explicit which units are available to which players:

Second the use of productionFrontiers tells which units are available to which players. To change a units availability, only the productionFrontier for that player needs to be changed, either directly edited, or within the xml itself. This would seems to add another step.

A skin really should only have a variant of the images. Putting unit-to-player association in a skin would also fracture that association as well. Really we're just saying "this player can owner this unit type".

I wonder if also if we are forced to have unit ownership be 'square'. IE: all players could potentially own a given unit. Notably for the capture case. What happens for example if somehow neutrals managed to capture a unit they could not own. I think it's probably simplest to know which units are capturable, by whom, and to just deal with that accordingly.

LaFayette

@wc_sumpton said in Proposal to consolidate Unit options into UnitList & Make explicit which units are available to which players:

That being said, could the unit cost also be added to this information?
Just thinking out loud.

That would make sense. So instead of 'availableTo' with 'player' sub-elements, we go further and put the production frontier in the element node. That would reduce repetutation. Generally, in good XML design, elements should only refer to 'parent' elements and avoid referencing 'peer' elements. That would fit with that quite well (EG: here is a unit type, here is a player that can own it (so we expect an image for it), and here is the production rule (frontier) for the player and unit).

LaFayette

This proposal is spawned from a game performance problem that in part stemmed from there being no concrete association between unit types and players.

https://github.com/triplea-game/triplea/issues/7142

@Cernel said there:

For example, this way, in World At War, you would have the "red dot" units in unit help only if you open it after the start of round 4. If you want to know ahead what are the abilities of the units that you will have, it makes sense that the notes should do this (and the notes of World At War do it fully and flawlessly, except only for the fact that they don't display all unit images, but you need this only once the units are actually on the board (at which point you have them in Unit Help)).

Long story short, out of this:
    all units on map
    all production frontier
    all unit artworks
I suggest completely to eliminate the third item.

Unit help is meant to be all units, future and present. This way you can plan which units you want to purchase before they are available.
Game notes is responsible for doing a lot already, avoiding duplication between game notes and the game functionality is a good thing. We want map making to be easy & light weight.
The unit help is tied to the unit stats exports, they're meant to be roughly the same

If we have an explicit tie in XML, then all three mechanisms, units on map, production frontiers, and unit artwork would go away. We'd also be able to determine when a map launches exactly if it has all of the right images. Another relatively major problem is the engine does not know which map images are needed until they are needed. If the engine could determine that at game-launch, it'd remove a class of error and would further improve map making.

Pending further ideas, I'm strongly in favor of XML being explicit about unit-type relationship, it seems to solve a number of problems that I otherwise do not really see how else to cleanly solve.

tvleavitt created this issue in triplea-game/triplea

closed Calculating Data on "Unit Help" takes extraordinarily long time #7142

Cernel

@LaFayette said in Proposal to consolidate Unit options into UnitList & Make explicit which units are available to which players:

Pending further ideas, I'm strongly in favor of XML being explicit about unit-type relationship, it seems to solve a number of problems that I otherwise do not really see how else to cleanly solve.

Would this be merely an "information" feature, irrelevant for gameplay? Am I correct to assume that it would matter only for new or specifically updated games?

alkexr

@LaFayette I have minor disagreements about this syntax. First, the list of availableTo players should be a colon delimited list, as those can generally be edited much quicker than copy-pasting (single-line) xml blocks. But my biggest concern is that moving unit attachments out of the attachmentList means that variables and foreach patterns can no longer be used for units. Now, unit attachments aren't the primary candidates for these, but I would say it's still a bad direction, unless variables are extended to the unitList as well. There is also a case to be made that units belong to players, not the other way around, so instead of availableTo, players should have a list of availableUnits. As a mapmaker handling maps with 50+ different units I would much rather edit one player's available units in one place if possible.

LaFayette

I appreciate the discussion @alkexr , the point of this discussion is feedback and to decide the direction.

First, the list of availableTo players should be a colon delimited list

While that can be the case, I would respectfully point out that is not canonical XML. FWIW, XML models lists as:

<elements>
   <element>value1</element>
   <element>value2</element>
</elements>

You can always convert such a list to a colon delimited list provided that colon is an invalid character in any list entry. One thing you cannot do is then augment your list to have more attributes or values, for example:

<elements>
   <element>
      <name>key1</name>
      <value>value1</value>
   </element>
   <element>
      <name>key2</name>
      <value>value2</value>
   </element>
</elements>

As convenient as it might be, a colon delimited list is arguably bad XML practice as XML can and does model lists. List values are not meant to be special encoded attribute values, attribute values are quite meant to be scalar values.

The above you can copy/paste easily to additional lines, it's debatable if that's easier than finding the right spot to add a new entry.

I suspect this is a pedantic point, but it does kinda speak to the overall XML modelling being very non-canonical. There is a cost there as the less than ideal model makes parsing difficult and constrution of an XML non-trivial as well as there is a lot of cross-reference and duplication.

But my biggest concern is that moving unit attachments out of the attachmentList means that variables and foreach patterns can no longer be used for units

What would imply that? A canonical way to do the variable list would be to make it a first class element and then reference it with a specific tag, eg:

      <playerList name="allies">
              <player>Americans</player>
              <player>British</player>
              <player>Russians</player>
       </playerList>  

      <availableTo>
              <playerList>allies</player>
       </availableTo>

In all practicality we might want to re-use the variable mechanism, though the above is more first class way to support a variable list. From a technical perspective we've been seeing issues parsing XML maps, the variable expansion might not necessary be optimally done, though it could debatably be simpler to re-use that as-is. One downside is you do not get as much of a syntax check. In the above you can get a clear error message if you typo the "allies" player list name for something that does not exist, where-as with variable expansion you won't get the same type guarantees. That does make XML editting more error-prone and harder to debug as the error messages are simply not there.

There is also a case to be made that units belong to players, not the other way around, so instead of availableTo, players should have a list of availableUnits.

I think this is perhaps the primary modeling consideration. It looks like we are going to be forced to duplicate either the unit type names, or the player names. Perhaps we'll need to consider how much more future consolidation and overall direction we would want to go.

I'm only nervous about that approach as a grand-update would make for a much more difficult lift, a steeper learning curve, and make it just less likely we'll complete the effort. Meanwhile it is a problem that the game engine is not told anywhere which units belong to which players (which leaves utterly hacky and incorrect code for the engine to figure that out; it needs to be made explicit).

As a mapmaker handling maps with 50+ different units I would much rather edit one player's available units in one place if possible.

This is an interesting scenario when the unit list is large. I'm not sure if it actually helps that much when you have 50 units listed under each player. This implies as well more of a white list. Would it not help that by default all units are available to all players? The time to specify when a unit is available to specific players is only when that unit is not available to all players. If you add a unit, that is available to all players, it would be just one line. IF there are 20 players, and we list all units, say 30 units, then you've got 600 lines to maintainer. Add a new unit, you add 20 lines, add a new player, you add 30 lines. ON the other hand, if we do units to players, you add a new unit, it's one new line. If you add a new unit that is available to 19 out of 20 players, then you add 1 line for the unit and the 19 player entries (and here we can make use of lists if very likely the unit should only be available to an alliance). IF the unit is player specific, then we're only adding one or two player entries, which keeps it pretty manageable.

Cernel

@LaFayette Variables are terribly inefficient, currently. A game file (xml) with variables parses much slower than the same file without variables and the same functionalities. However, I've always assumed @redrum is aware of this and it doesn't actually matter as long as the file is not too big (probably creating the code from the variables has to be time consuming, I guess).

LaFayette

Three quick observations:

general variable usage disallows type checking. For example, you can use a variable for resource types and use that as a player list. If we were to have a 'playerList' tag with a name, the engine could know that it would be an error to have that list be used for a list of resources.
We currently can't use variable lists for production frontiers
Looking at WorldAtWar xml, I'm seeing neutral production frontiers that are not attached to any player, it looks like we've fallen into a not great situation with unlinked entities.

LaFayette

@Cernel I'm not sure why it would be inefficient. One problem is that variables imply a complicated map, which then implies a long XML. The XMLs are currently longer than they need to be due to the modelling, just not as expressive as it could be (IE: we should not need a 30k line long XML, I understand some maps are complicated, though that is an issue with the XML I think and not the complexity).

Cernel

@LaFayette Well, I only know that if you make a game file with some functionalities and, then, reduce the same file, keeping the same functionalities with less code, by the use of variables, the file using variables is a lot slower to load and parse than the one without variables. I mean "a lot".

LaFayette

@Cernel That's unfortunate. Perhaps there is something that can be readily fixed in there. Otherwise modelling lists "more properly" would resolve that. Essentially instead of a variable one would create an XML element and then reference that element (EG: the playerList element above, instead of a <var name='allies' values='russia,uk'>, you create the <playerList name='allies'> tag instead.

wc_sumpton

@LaFayette said in Proposal to consolidate Unit options into UnitList & Make explicit which units are available to which players:

This implies as well more of a white list.

Yes it does, and a necessary evil with the xml. Take TWW as an example, for Germany you will find germanInfantry, germanTank, german... etc. The same for the Russians, russianInfantry, russianTank, russian... etc. Then looking at the unitAttachment's for germanInfantry, russianInfantry they are the same. It is only during game play when changes are made to one player's unit but not to all players.

Lets say that Germany's Tank will receive an increase in offensive firepower. With techAbilityAttachment that can be done. But techAbilityAttachment is very limited in what can be changed.

If I wanted to give Germany's Infantry the ability to amphibious assault from a German Carrier. Well it can't be done unless you remove both carrier and infantry from the 'white list' and modify germanInfantry and germanCarrier.

So X+ similar unit per player has become the norm.

Cheers...

alkexr

@LaFayette What I meant is that to my knowledge variables can only be used within the attachmentList right now. Also, colon delimited lists are a lot more convenient right now (they accept variables). So I'm approaching this from a baby steps perspective. And yes, I'm prefectly well aware what the ideal xml structure is: it's a remarkably horrible way of representing the newer generations of TripleA maps, even if it used to be just fine for A&A et al. Even the concept of variableList is more or less just an outside hack that I originally implemented as a standalone script to generate the xml file, because I couldn't bear its clumsiness. The ultimate solution would of course be a script language for TripleA, but yeah, I'm not going to create that either (or not at the moment anyway).

LaFayette

@wc_sumpton I think you might be pointing to a problem regardless of how it's done with this proposal. It might be cleaner with the explicit list, but it's probably equivalent.

Either:

<playerList>
   <player name="German">
        <units>
            <unit>GermanTank</unit>
        </units>
   </player>
<playerList>

Or:

<unitList>
   <unit name="GermanTank">
       <allowedPlayers>
             <player>German</player>
       </allowedPlayers>
 </unitList>

In either case IMO the special attribute or unit behavior is not allowed to be player specific, so create a specific unit for it seems to be the core problem. For either modeling variant it's kinda equal.

One difference in the above is how the production rule would be nested (or not nested). That would make the latter more attractive.