Magic Files for Map Indexing

  • Admin

    I'm working on a way to avoid having to update: triplea_maps..yaml by instead embedding the needed information in the map bundle itself.

    We still need to supply the same information, but in an alternative location. Today the map folder structure at a top level is just:

    - map/

    The data needed from each map is:

    • map name
    • version (download version, used to notify users of updates)
    • preview image location (the link to this image is in triplea_maps.yaml, typically the image is stored in the map repository)
    • map description (currently this lives in triplea_maps.yaml, usually as a CDATA block.

    The map name and version will be found in map.yml file. The structure of that file is described in the map.yml forum thread. The description and preview I think can just be files. That would make the top level of a map look like this:

    - map/
    - map.yml       ## contains 'map-name' + 'version'
    - preview.png      ## this will be the preview image
    - description.html    ## contains the description

    And there you have it, once those files are added, the engine would be able to pull information automatically without needing an entry in triplea_maps.yml.

    I hope for this feature to be live for 2.6, until we get users migrated to 2.6 the triplea_maps.yml file will need an entry. Meanwhile, we can certainly add the above files and 2.5 or less will ignore them and 2.6 or greater will pick them up.

    Please provide any feedback that comes up, I'm particularly interested in the file naming and if this all seems easy enough.

  • @LaFayette currently the yaml file also has a category. That could maybe go in the map.yml file?
    For future proofing, we could maybe allow multiple categories but only use the first one in a list for now, or just look for certain categories.

    Something else that i have been thinking about with different map versions is that it would be nice to only download files that have changed instead of a whole map. Could this use the map version number as a kind of benchmark, and just download the files that are different between those versions? In my limited programming experience, that seems like something that would use some of the same files, so it might involve less work if it were to be done at the same time. I don't know if it is worth doing, but downloading the warcraft map in its entirety takes a while now (i think it is 500mb) and is kind of admitting if it is just for a few xml edits.

  • Admin

    The category information needs to live outside of the map entirely. If the category changes we should not need to change the map contents and that would require users to re-download the map. I'm thinking the category would live in the maps-server database. The maps-server can query github to get a list of all repositories, from there it can then scrape the 'map.yml' file in each one and create a listing of available maps. This database will replace the main triplea_maps.yaml file. Then the game client when requesting the list of available maps would send a request to the server which would then query it's database to return results.

    Partial downloads is not on the radar right now. It'll be quite complex. I'd rather we first do other projects to reduce the size of the download. Like specifically I'd like to see maps broken up and the XML distributed apart from the assets. This would essentially create a pool of assets such as images, maps, and XMLs would be independent and reference the assets by ID. When downloading an XML, the engine would figure out which asset IDs are required and download those. I don't know if we'll ever get to that model, so discussing it now is perhaps premature, but that's more or less the long-term radar on where maps could go.

  • Could the version number of the map be automatically calculated?
    Something like;
    Date+Time stamp YYYY-MM-DD---HH:MM:SS

    So the person uploading the map does not have to worry about the version?

    This also has the advantage, it shows when the map was last changed/updated/fixed.

  • Admin

    If so, then what would the version be used for @TheDog ?

    There is a commit 'sha' attached to any change, every version of a map is already uniquely identified by that.

  • We humans can easily read the Date+Time stamp and interpret it. It was just an idea.

  • Admin

    All good, just trying to explore the idea in case there is something I'm missing.

    While a date and timestamp are easier to know if one version is newer than another, who is to know if that has significance? Does the newer version imply it is incompatible, or just has some newer images maybe? What if the XML is simplified in one, and not the other, does that make a difference?

    In the lobby, the game version is horribly cut off, so the current value is not very useful. Even if we made the engine aware of map versions and then complain if versions did not align, would that lead to excessive issues for map versions that are actually compatible?

  • Admin

    @TheDog , I had my threads crossed. For this version number, it is never displayed to anyone at all. It is just used by the engine to know when to notify a user that their map is out of date. When to trigger that notification is up to the map maker.

  • Admin

    There certainly is a discussion to be had about map versioning.

    The engine is not designed to handle different map versions, it is designed to assume that they all interoperate and users will be prompted to and always would download the latest. There are lots of questions on how to handle this. Though, that discussion has very little to do with the new 'magic files' that will be required as part of all map repositories going forward.

Log in to reply