Should we support YAML for Maps?
-
Actually, the above example is not fair to XML as I wrote out the XML in it's most verbose form. If we condense it using attributes, it gets a lot nicer:
<units> <unit> <name>Infantry</name> <costs><cost resource="PU" quantity="2"/></costs> <image>infantry.png</image> <initialPlacements> <initialPlacement territory="Western USA" quantity="2" /> <initialPlacement territory="Western USA" quantity="2" owner="British" /> <initialPlacement territory="Eastern USA" quantity="2"/> </initialPlacements> </unit> <unit> <name>Stuka</name> <costs> <cost resource="PU" quantity="8"/> <cost resource="Oil" quantity="1"/> </costs> <availableTo> <player>Germany</player> </availableTo> <image>stuka.png</image> <initialPlacements> <initialPlacement territory="Southern Europe" quantity="2" /> <initialPlacement territory="Germany" quantity="10" /> </initialPlacements> </unit> </unit>There is a con list for us if we adopt YAML:
- we'll have two different major specification types
- introduces complexity in the engine
- not everyone will be able to give as much help while we learn YAML
- we'll always be asking which spec someone is using, whether XML or YAML
- there will be complaints when features are available in one format and not another
- there is a cost to migrate XML files to YAML and we'll be doing that for a while
- there will be errors with indentations (potentially painful in large files)
- generally there is a learning curve for people (map makers) to switch
IMO the real benefit we get going forward is being able to write XML blocks that look like the above rather than what we have today. YAML and XML are equivalent, if we keep the XML to as nice as form as possible, the benefits of YAML become more marginal. We also would save a lot of work just by sticking to XML. It will also be likely painful when there are feature differences when YAML gets the newest and XML is left to stagnate.
I'm starting to think that if this were a fresh project, YAML would be a good contender, but the indentation problem would need to be seriously considered. For example, if a map has tabs and spaces in it, you then get into true hell as the indentation could look right, but would not be.
Therefore, IMHO I'm starting to land that unless we have a converter app to get everything into YAML then it would be a stronger consideration. If we can't cleanly migrate, I kinda think it'll just be the start of a lot of headaches for the benefit of a nicer syntax. Meanwhile the real problem is that the XML we have defined is not expressive and we can fix that without switching to YAML.
- we'll have two different major specification types
-
I advise against moving around stuff, like that thing of having the cost in the "units" attachment, instead of in the production rules. In the example, I'm not seeing how that would allow players having different costs for the same unit.
I'm in favour, or at least not against, this general change only as long as everything stays as it is, beside changing the format (then, of course, once the new format is made and stable, specific changes can be considered).
-
@Cernel said in Should we support YAML for Maps?:
I advise against moving around stuff, like that thing of having the cost in the "units" attachment, instead of in the production rules. In the example, I'm not seeing how that would allow players having different costs for the same unit.
That is the exact problem the we have now. How do we change hitPoints for a unit for a player. What @LaFayette is advocating is that everything for a given item, territories (or 'zones'), and units be located in one place in the xml (or YAML). And which language to use, xml or YAML, to generate the map.
@LaFayette said in Should we support YAML for Maps?:
Meanwhile the real problem is that the XML we have defined is not expressive and we can fix that without switching to YAML.
I think this line says it all IMHO.
Cheers...
-
@wc_sumpton That's hardly related. The unit's cost is not a charateristic of the unit itself. It is the mean, or should I say one of the means, by which the unit is placed on the "gameboard". It is more related to whatever trigger placing one or more units on the gameboard than it is to the options these units have in their attachments.
To make an example. I can make 1 single production rule saying that you spend 5 PUs plus 3 Mana plus 125 Bananas and receive 8 PUs plus 1 tech token plus 3 infantry units plus 15 dragon units. The PUs and the tech token are added to your resources stock, while the infantry and the dragon units are added to your inventory (to be likely, but not necessarily, eventually placed on the gameboard). There is no way you can say how many bananas an infantry cost, because you are using them, together with other resources, to buy both infantry and dragon units.
If this won't be anymore possible in the new format, then I oppose this format, as well as opposing if it is not absolutely assured that
everything currently possible will be still possible and loading the game from the game file (YAML) will not be slower
.
I'm not against removing possibilities. But this should be considered individually, not by going for a new and different system that we are not sure whether or not it is supporting in full what it is currently possible.
-
@Cernel said in Should we support YAML for Maps?:
I can make 1 single production rule saying that you spend 5 PUs plus 3 Mana plus 125 Bananas and receive 8 PUs plus 1 tech token plus 3 infantry units plus 15 dragon units.
125 bananas and you only get 3 infantry and 15 dragons? That is way over priced!
-
@Cernel The XML is there as an example to demonstrate what it would look like potentially, it's meant to be a realistic future example that is apples-to-apples between XML and YAML. I literally spent less 5 minutes on it, the focus is on YAML vs XML.
I can't help myself, it'd be easy to make it player specific:
<costs> <cost player="Russia"> <resource>PU</resource> <quantity>2</quantity> </cost> <cost player="Germany"> <resource>PU</resource> <quantity>3</quantity> </cost> </costs> -
@Cernel said in Should we support YAML for Maps?:
If this won't be anymore possible in the new format, then I oppose this format, as well as opposing if it is not absolutely assured that
I think there is a misunderstanding here. XML and YAML are essentially equivalent. The structure is going to change, but we'll still support the old structure for the sake of not breaking maps. The question is whether to go forward with an updated structure in XML or in YAML.
-
@Cernel said in Should we support YAML for Maps?:
everything currently possible will be still possible and loading the game from the game file (YAML) will not be slower
About this, yeah, we're not going to break maps. I don't think we'll have make any changes again where old maps are required to be updated. That has been a disaster every time we do it, we still have issues with maps using "attatchment" years after we "fixed" that.
Will updates to parsing be slower? Perhaps, it might very well be that your average map instead of loading in 600ms might load in 800ms or even 1.1s. Ideally we'll maintain parsing speed, but we've got issues with some maps crashing on parse due to memory and not because of speed. Going forward it's more important that we use less memory so that maps don't crash on parse due to OOM and we can restore XSS back to default which I suspect has caused further memory issues (particularly in bots).
War of Relics is a good benchmark, but it's an example where the XML structure we have is not expressive. Eventually if you are forced to copy/paste things too many times (my understanding is a macro was written to generate that as IIRC it's over 50k lines long), eventually you hit a limit.. So in part, the goal of more expressive XML is so you don't have to generate such macros and can nest things more properly so you don't have duplicate tags repeatedly to apply them to multiple entities.
-
Hehe I enjoy tripleA for making me feel slightly less clueless on some stuff, but I'd have no clue on this one. Only time I've ever opened an xml file is to mess with this game, only time I've seen a yaml file was to play master of orion lol. Which approach do you think best La Fayette?
For someone who doesn't know thing one, the YAML postings certainly seem easier on the eyes haha. Wish I knew more though to have an opinion. Thumbs up though to whatever works!
Best Elk -
I always dislike relying on indentation for structures. I prefer explicit structural elements.
-
i will say that yaml does look a bit easier to learn and more sensible in general i understand the hesitation about indents (which i personally like actually) but doesn't the shorthand version somewhat resolve that issue?
-
Thanks for the feedback all. I think time to put this one to bed. Having both YAML and XML in the mix is really disruptive, the answer to mixing the two from me is "no". If we have a choice to migrate all XML to YAML, the answer from me is then "probably not".
Reasoning:
- there is a lot of expertise around XML built up, this would be lost going to YAML
- a cleaner syntax is nice, but not compelling
- all of the tooling around XML would be lost, the list of attributes, etc, all of the existing examples, would all have to be redone
- XML vs YAML is not hurting us so much where migrating is going to solve a lot of problems. We have limited resources and need to get LOTS of other things done. Unless we're solving a big problem, it's not worth it.
- Similar to saving effort, we might as well save the development effort of switching code
- The chance for indentation errors is high, I just foresee dos line endings and/or tabs giving someone a really bad day. While nasty XML is unpleasing to the eye and lengthy, at least it's easier to diagnose a problem there (and most tools will tell you where and which tags are unmatched or badly structured)
- Having two different formats would be no end to headaches, both in terms of which options they support, hurting people, coding, everything.
-
@LaFayette said in Should we support YAML for Maps?:
In essence, XML was an early standardized format that grew out of HTML that was meant for structuring data for one computer to send data to another. It was never really intended for humans to interact with it and write it by hand (the idea was there would be GUIs to help you do that).
Most likely this has always been the case for TripleA too. Ideally, you would have a map creator covering everything, and you would use that for creating and modifying the XML, never actually opening it with a text editor or anything. I believe this was the plan, except that it never happened (there have been several aborted attempts over the years).
-
Not to bring up a topic that was essentially decided, but I would love to throw my vote behind sticking with XML.
I love XML. For one reason and one reason alone- XSD (namely, well supported schema). We would lose that with YAML. I see YAML as short term gains but long term losses. For example, I've written a tool that takes an Excel sheet with units, their attachments, territories, starting placements and bi-bidirectionally "syncs" it with the game file using JAXB and Apache POI. This is the general direction I would love to help go- using powerful map-making UIs (and how much more power can you get with Excel/LibreOffice/standard spreadsheet UIs?) built on top of the most powerful/structured data interchange format. XSD supports inheritance, inclusion, polymorphism, unions, enums, etc. etc. and can be edited by a human in a pinch, but I'm envisioning a future world where map tools edit and the game engine parses and humans touch the game files less and less.
-
Agree on having humans touch the XML less and less. My long term vision is to have all the map making tools be built into the actual game, just like 'edit' mode, there would be a 'map making' mode that would enable additional options. Such an example would be an attachment editor to configure the attachments, a UI to adjust unit stats, etc..
FWIW, YAML does have a XSD like schema validation, more like DTD I suppose. We run that check against changes to the map.yaml file, adding unrecognized YAML attributes would fail that check and the build. This largely catches typo's or missing attributes.
-
I love building UIs and I love data interchange and would love to help with such a UI
once I get some free time which might not be for awhile...
Hello! It looks like you're interested in this conversation, but you don't have an account yet.
Getting fed up of having to scroll through the same posts each visit? When you register for an account, you'll always come back to exactly where you were before, and choose to be notified of new replies (either via email, or push notification). You'll also be able to save bookmarks and upvote posts to show your appreciation to other community members.
With your input, this post could be even better 💗
Register Login