Storing Maps as Folders only rather than as either Zip or Folder
-
As a thought, what if we unzipped map zips after downloading?
This seemingly would simplify a whole lot of code that handles either folders or zipped files. We would not need to do as much zip file work within the game engine and would be able to use the file system API to more easily traverse map contents. Furthermore, when unzipping, we could rename the map folder and remove the '-master' suffix (which would further simplify the map content search path).
For backward compatibility and conversion, the engine would likely first scan on start up for any map zips and then extract them. Then after any downloads, a last step would be to unzip.
Any thoughts to this?
-
Maps I like, I always unzip to see what is new and how they were constructed and perhaps tinker with the xml code, so it works for me.
-
In general, I'm 100% a fan of simplifying map storage, organization, and loading/parsing.
Is there any downside as far as bogging down a file system with a) longer paths (probably not - the map folders are relatively shallow, though perhaps some unit names are long) b) a much vaster number of individual files (unit and tile images)?
I'm almost certain that is premature optimization, but that's the only downside I could think of
-
@LaFayette What was the reason to zip the files? It cannot be saving storage as the difference unzipped versus zipped is neglectable.
But there must have been something behind zipping them? Is it only because of how Github provides the files?
-
Zips Provide Faster Downloads
Downloading a single file is far faster than multiple files. For download, that makes a lot of sense.
Zips are not saving very much disk space
Some XMLs are relatively large, they will compress to perhaps around 10% of their original size. Images and sound, essentially no savings. Map disk space usage is dominated by images and sound files, so a zip is not saving much.
Faster file system access unzipped, less CPU usage
Reading compressed files has gotten pretty fast, but the system does need to spend extra CPU cycles to uncompress a zip and maybe need additional hard disk access to access the zip.
Code Complexity
Perhaps the biggest difference is here. There is quite a bit of code to handle map files being in either a folder or a zip and this will be the major difference as far as I can see. Meanwhile on the downside we'd need to have some new code to unzip and ensure that any new zips are also unzipped.
-
@LaFayette i agree folder file structure is generally better.
-
Another advantage of unzipping is that unzipped files don't have case sensitivity problems with file names while zipped files do.
-
@RogerCooper That's just windows FYI- file names are case sensitive on non-Windows OSes
-
@RogerCooper said in Storing Maps as Folders only rather than as either Zip or Folder:
Another advantage of unzipping is that unzipped files don't have case sensitivity problems with file names while zipped files do.
You have just pointed out what I rather believe is a huge disadvantage of dealing only with non-zipped files: it would most likely cause a lot of wrong casing for map-makers using case-insensitive systems, by removing the ability of at least partially checking for case inconsistencies by playing the game using the zipped map's folder.
Unless a way for checking for case consistency while using case-insensitive systems is provided (The only, and only partial, way I know is by zipping the map's folder.), removing the primacy or, even more so, the support of zipped maps' folders would be disastrous (You would most likely end up with the majority of maps made by map-makers on case-insensitive systems being riddled by wrong casing and, therefore, unplayable for normal users on case-sensitive systems.).
-
@Cernel As we unzip a file, we will be doing a lot of checks (are all the required folders there? is there a
centers.txt, etc.?) and as part of that process, we could enforce a casing convention.I have a tool that fixes capitalization. When I got image assets from @GenerationKILL sometimes the casing was a bit off, so I would just run this tool and it would conform every unit to a convention like this:
Unit-Name.pngregardless of the OS and regardless of the original naming.We could implement a similar normalization process at the map parsing level (though it might be hard to get map makers on the same page about which casing convention should be used).
