Proposal to consolidate Unit options into UnitList & Make explicit which units are available to which players

[Cernel]

@LaFayette said in Proposal to consolidate Unit options into UnitList & Make explicit which units are available to which players:

Pending further ideas, I'm strongly in favor of XML being explicit about unit-type relationship, it seems to solve a number of problems that I otherwise do not really see how else to cleanly solve.

Would this be merely an "information" feature, irrelevant for gameplay? Am I correct to assume that it would matter only for new or specifically updated games?

[alkexr]

@LaFayette I have minor disagreements about this syntax. First, the list of availableTo players should be a colon delimited list, as those can generally be edited much quicker than copy-pasting (single-line) xml blocks. But my biggest concern is that moving unit attachments out of the attachmentList means that variables and foreach patterns can no longer be used for units. Now, unit attachments aren't the primary candidates for these, but I would say it's still a bad direction, unless variables are extended to the unitList as well. There is also a case to be made that units belong to players, not the other way around, so instead of availableTo, players should have a list of availableUnits. As a mapmaker handling maps with 50+ different units I would much rather edit one player's available units in one place if possible.

[LaFayette]

I appreciate the discussion @alkexr , the point of this discussion is feedback and to decide the direction.

First, the list of availableTo players should be a colon delimited list

While that can be the case, I would respectfully point out that is not canonical XML. FWIW, XML models lists as:

<elements>
   <element>value1</element>
   <element>value2</element>
</elements>

You can always convert such a list to a colon delimited list provided that colon is an invalid character in any list entry. One thing you cannot do is then augment your list to have more attributes or values, for example:

<elements>
   <element>
      <name>key1</name>
      <value>value1</value>
   </element>
   <element>
      <name>key2</name>
      <value>value2</value>
   </element>
</elements>

As convenient as it might be, a colon delimited list is arguably bad XML practice as XML can and does model lists. List values are not meant to be special encoded attribute values, attribute values are quite meant to be scalar values.

The above you can copy/paste easily to additional lines, it's debatable if that's easier than finding the right spot to add a new entry.

I suspect this is a pedantic point, but it does kinda speak to the overall XML modelling being very non-canonical. There is a cost there as the less than ideal model makes parsing difficult and constrution of an XML non-trivial as well as there is a lot of cross-reference and duplication.

But my biggest concern is that moving unit attachments out of the attachmentList means that variables and foreach patterns can no longer be used for units

What would imply that? A canonical way to do the variable list would be to make it a first class element and then reference it with a specific tag, eg:

      <playerList name="allies">
              <player>Americans</player>
              <player>British</player>
              <player>Russians</player>
       </playerList>  

      <availableTo>
              <playerList>allies</player>
       </availableTo>  

In all practicality we might want to re-use the variable mechanism, though the above is more first class way to support a variable list. From a technical perspective we've been seeing issues parsing XML maps, the variable expansion might not necessary be optimally done, though it could debatably be simpler to re-use that as-is. One downside is you do not get as much of a syntax check. In the above you can get a clear error message if you typo the "allies" player list name for something that does not exist, where-as with variable expansion you won't get the same type guarantees. That does make XML editting more error-prone and harder to debug as the error messages are simply not there.

There is also a case to be made that units belong to players, not the other way around, so instead of availableTo, players should have a list of availableUnits.

I think this is perhaps the primary modeling consideration. It looks like we are going to be forced to duplicate either the unit type names, or the player names. Perhaps we'll need to consider how much more future consolidation and overall direction we would want to go.

I'm only nervous about that approach as a grand-update would make for a much more difficult lift, a steeper learning curve, and make it just less likely we'll complete the effort. Meanwhile it is a problem that the game engine is not told anywhere which units belong to which players (which leaves utterly hacky and incorrect code for the engine to figure that out; it needs to be made explicit).

As a mapmaker handling maps with 50+ different units I would much rather edit one player's available units in one place if possible.

This is an interesting scenario when the unit list is large. I'm not sure if it actually helps that much when you have 50 units listed under each player. This implies as well more of a white list. Would it not help that by default all units are available to all players? The time to specify when a unit is available to specific players is only when that unit is not available to all players. If you add a unit, that is available to all players, it would be just one line. IF there are 20 players, and we list all units, say 30 units, then you've got 600 lines to maintainer. Add a new unit, you add 20 lines, add a new player, you add 30 lines. ON the other hand, if we do units to players, you add a new unit, it's one new line. If you add a new unit that is available to 19 out of 20 players, then you add 1 line for the unit and the 19 player entries (and here we can make use of lists if very likely the unit should only be available to an alliance). IF the unit is player specific, then we're only adding one or two player entries, which keeps it pretty manageable.

[Cernel]

@LaFayette Variables are terribly inefficient, currently. A game file (xml) with variables parses much slower than the same file without variables and the same functionalities. However, I've always assumed @redrum is aware of this and it doesn't actually matter as long as the file is not too big (probably creating the code from the variables has to be time consuming, I guess).

[LaFayette]

Three quick observations:

• general variable usage disallows type checking. For example, you can use a variable for resource types and use that as a player list. If we were to have a 'playerList' tag with a name, the engine could know that it would be an error to have that list be used for a list of resources.
• We currently can't use variable lists for production frontiers
• Looking at WorldAtWar xml, I'm seeing neutral production frontiers that are not attached to any player, it looks like we've fallen into a not great situation with unlinked entities.

[LaFayette]

@Cernel I'm not sure why it would be inefficient. One problem is that variables imply a complicated map, which then implies a long XML. The XMLs are currently longer than they need to be due to the modelling, just not as expressive as it could be (IE: we should not need a 30k line long XML, I understand some maps are complicated, though that is an issue with the XML I think and not the complexity).

[Cernel]

@LaFayette Well, I only know that if you make a game file with some functionalities and, then, reduce the same file, keeping the same functionalities with less code, by the use of variables, the file using variables is a lot slower to load and parse than the one without variables. I mean "a lot".

[LaFayette]

@Cernel That's unfortunate. Perhaps there is something that can be readily fixed in there. Otherwise modelling lists "more properly" would resolve that. Essentially instead of a variable one would create an XML element and then reference that element (EG: the playerList element above, instead of a <var name='allies' values='russia,uk'>, you create the <playerList name='allies'> tag instead.

[wc_sumpton]

@LaFayette said in Proposal to consolidate Unit options into UnitList & Make explicit which units are available to which players:

This implies as well more of a white list.

Yes it does, and a necessary evil with the xml. Take TWW as an example, for Germany you will find germanInfantry, germanTank, german... etc. The same for the Russians, russianInfantry, russianTank, russian... etc. Then looking at the unitAttachment's for germanInfantry, russianInfantry they are the same. It is only during game play when changes are made to one player's unit but not to all players.

Lets say that Germany's Tank will receive an increase in offensive firepower. With techAbilityAttachment that can be done. But techAbilityAttachment is very limited in what can be changed.

If I wanted to give Germany's Infantry the ability to amphibious assault from a German Carrier. Well it can't be done unless you remove both carrier and infantry from the 'white list' and modify germanInfantry and germanCarrier.

So X+ similar unit per player has become the norm.

Cheers...

[alkexr]

@LaFayette What I meant is that to my knowledge variables can only be used within the attachmentList right now. Also, colon delimited lists are a lot more convenient right now (they accept variables). So I'm approaching this from a baby steps perspective. And yes, I'm prefectly well aware what the ideal xml structure is: it's a remarkably horrible way of representing the newer generations of TripleA maps, even if it used to be just fine for A&A et al. Even the concept of variableList is more or less just an outside hack that I originally implemented as a standalone script to generate the xml file, because I couldn't bear its clumsiness. The ultimate solution would of course be a script language for TripleA, but yeah, I'm not going to create that either (or not at the moment anyway).

[LaFayette]

@wc_sumpton I think you might be pointing to a problem regardless of how it's done with this proposal. It might be cleaner with the explicit list, but it's probably equivalent.

Either:

<playerList>
   <player name="German">
        <units>
            <unit>GermanTank</unit>
        </units>
   </player>
<playerList>

Or:

<unitList>
   <unit name="GermanTank">
       <allowedPlayers>
             <player>German</player>
       </allowedPlayers>
 </unitList>

In either case IMO the special attribute or unit behavior is not allowed to be player specific, so create a specific unit for it seems to be the core problem. For either modeling variant it's kinda equal.

One difference in the above is how the production rule would be nested (or not nested). That would make the latter more attractive.

[LaFayette]

@alkexr I'm wondering if actually maybe a grand re-write is actually the way to go. Supporting many small XML spec variants would not be that easy.

RE: colon lists

I'm on the fence here. On the one side it is convenient, on the other it's debatable and there is so much less than ideal structuring that it would be nice to draw a line and start doing canonical XML. Canonical XML is quite verbose, but it "keeps you out of trouble" and keeps parsing more sane and generally more performant. The encoding of attributes via colons makes for not great code and there are numerous places where you don't get any error message at all for invalid configuration. A more canonical structure would get us there.

[LaFayette]

Here is an example of latest proposal:

Example of current:

  <unitList>
    <unit name="Stuka"/>
  </unitList>

  <production>
    <productionRule name="buyStuka">
      <cost resource="PUs" quantity="8"/>
      <result resourceOrUnit="Stuka" quantity="1"/>
    </productionRule>
  <production>

  <productionFrontier name="GermansFrontier">
    <frontierRules name="buyInfantry"/>
  </productionFrontier>

  <playerProduction player="Germans" frontier="GermansFrontier"/>


  <attachmentList>
    <attachment name="unitAttachment" attachTo="Stuka" javaClass="games.strategy.triplea.attachments.UnitAttachment" type="unitType">
      <option name="movement" value="4"/>
      <option name="attack" value="3"/>
      <option name="defense" value="1"/>
      <option name="artillery" value="true"/>
      <option name="isAir" value="true"/>
    </attachment>
  </attachmentList>

Example of proposed:

  <unitList>
    <unit name="Stuka">
        <allowedPlayers*>
           <player>Germany</player>
           <playerList>Axis<playerList>
        </allowedPlayers>

        <productionRules>
            <productionRule>
               <costs>
                  <cost resource="PUs">8</cost>
               </costs>
               <players*>
                 <player>Germany</player>
                 <playerList>Axis<playerList>
               </players>
            </productionRule>
        </productionRules>

        <stats>
          <movement>4</movement>
          <attack>4</attack>
          <defense>1</attack>
        </stats>

        <options*>
           <option name="artillery"/>
           <option name="isAir"/>
        </options>

    </unit>
  </unitList>

(* = optional attributes)

Comments/notes:

• using player list we can consolidate all production rules to just the above, the current example would more properly bring in every player that has the production frontier
• the above has a lot less cross-references, no need to name frontiers and then link them. No need to cross-reference the name of a unit.
• we do have cross-references to players and the playerlist. I suspect the above would scale pretty okay if you add either a player or a unit (notably, adding a unit, it's just a single block of XML that is added).

If we go with the grand re-write, YAML is an option. In YAML the equivalent configuration would be:

unitList:
  - name: stuka
    allowedPlayers*:
       players: [Germany]
       playerLists: [axis]
    productionRules:
      - name: stukaPurchase
        costs: 
           - {resource: PUs, quantity 8}
        players*: [Germany]
        playerLists*: [axis]
    stats: {movement: 4, attack: 4, defense: 1}
    options*: [ isArtillery, isAir ]

[Cernel]

@LaFayette You should also be able to purchase resources or a mix of units and resources.

For example, with 1 PUs you purchase a "stuka" unit and 1 "a" resource. You can also purchase 2 PUs with 1 PUs, that would equal an investment with a 100% interest rate over 1 round.

[alkexr]

@LaFayette The YAML is about ten times more readable. Stats or options being in a single line isn't that great though. Think some unit with AA attacks and AA attack max dice sides and bombing damage and all that, I'd definitely want to group AA related stats/options or bombing related ones or construction related ones together.

Also consider that some maps use multiple production frontiers and swap them during game using triggers.

[LaFayette]

@Cernel Costs being a list allowed for multiple cost elements, eg:

      costs: 
        - {resource: PUs, quantity: 8}
        - {resource: Oil, quantity: 2}

For purchases resources, good consideration if we are going to remove the production frontier tag. To replace it we'd want to update the resource list and add a similar production rule to that.

@All/

At this point, despite my earlier reservation, I think there are two larger questions at play now:

• 1. Do we do a grand update of the map spec? Is a much larger project in order?
• 2. Do we consider using YAML as the updated spec? Splitting the game engine parsing logic to keep the existing XMLs the same and to have a new parsing logic module for YAML would arguably be easier than having multiple XML versions.

[LaFayette]

@alkexr good considerations. We can certainly add/create attributes for AA, eg:

   attributes: [ artillery, air ]
   aa_attacks: { count: 3, dice_sides: 6 }
   bombing_damage: {dice_sides: 10, number_rolls: 3}

It's a bit limited to make all attributes just be an 'option' tag. The XML doctype makes that kind of move attractive. Moving to YAML makes that a lot more fluid, we could model options much more accurately.

For frontiers being turned on and off, we could add a triggers attribute to the production rule, and/or add a name field to it so it can still be referenced from triggers.

[Cernel]

@LaFayette said in Proposal to consolidate Unit options into UnitList & Make explicit which units are available to which players:

Splitting the game engine parsing logic to keep the existing XMLs the same and to have a new parsing logic module for YAML would arguably be easier than having multiple XML versions.

This seems the best solution to me. Hopefully the YAML would be faster to parse and load (also for the battlecalculator)?

As long as all that it is possible to do now with the XML will still be possible in the new format, it should be fine for me.

[Cernel]

@LaFayette As I said, I would advice any such things working like the support attachments currently do (this way you could also have multiple different AA abilities per unit). Meaning you have the ability that you define and give it to a list of units and a list of players.

[LaFayette]

If we are going to seriously consider YAML, I suppose it deserves its own discussion thread.

We'd have to be sure we would want to do that. I think there is a lot of good flexibility it gives us. Though we'd probably want to 'freeze' the XML spec of maps and just keep that working while the YAML version gets new attributes over time.