Add PCKReader class

[yvie-k]

Adds a PCKReader class to read the contents of pck files without loading them.

This class exposes the same interface as the ZipReader class. Therefore we now have the same interface as ZIPPacker/ZIPReader for dealing with pck files.

To do so, the PCK implementation of PackedSourcePCK has been extracted into a new PCKFile class. I also split the loading of normal pck files and the loading of embedded pck files. This seemed clearer to me. If you have any issues regarding this, feel free to tell me.

Something to note would be that you cannot load the pck file embedded in the binary, as the PCKReader does not try to open the embedded file. But I'm not sure whether this should be supported as it feels quite hacky.

Furthermore, the current implementation loads the file list to memory when opening. This way we do not need to parse the file list on each access.

Resolves godotengine/godot-proposals#1212

[yvie-k]

On a side note, I also rewrote the embedded loading of pck files here: yvie-k@a5d5ec8

This would only try to load the embedded pck of the executable and not every opened pack file

I didn't include it as it is a) out of scope for this PR and b) I'm not sure if this is actually wanted

[MewPurPur]

Full sentences should use punctuation.

[yvie-k]

I can't manage to build the crashing CI release locally.

The compiler error mesage is unknown pseudo-op: .siz

Does anybody know what could be causing this?

There's also a warning: end of file not at end of a line; newline inserted

Anyways I'll be going to bed now. I will try to continue debugging the crash tomorrow.

[bruvzg]

I didn't include it as it is a) out of scope for this PR and b) I'm not sure if this is actually wanted

Unlikely this is usefull for non-executable, and if somone realy want's it it's always possible to load it using offset.

CI crash is not related to your PR, we have it for some time but so far were not able to pinpoint the issue. So if it's happen, it's safe to restart the failed CI jobs.

[bruvzg]

Since PCK reader is user accessible, it might be worth adding the encryption key to the open arguments (and fall back to the embedded key if it's empty).

[ghmart]

Hi, Dragoncraft89!

Built your fork and discovered a bunch of problems with it:

1. When saving scene it didn't give it default filename like "main.tscn" if root node called "Main", for example, just empty line (irrelevant to this PR, but still strange). Known bug, already fixed.

2. When exporting PCK it makes it almost 2x size bigger than official version of Godot (or custom builds from master). For example, official version makes PCK of size 4,877,872 against 8,372,048 using this fork (same settings). Sorry, my bad, forgot to exclude several resources.

3. When exporting PCK it didn't see many folders where it could be stored, as if they're absent in filesystem.

4. PCKReader didn't see some files (only text files mostly), for example *.png.
   EDIT: Is it possible to load one of the images using PCKReader without loading whole PCK, and, if yes, how?
   EDIT2: Found a way to get image from PCK: use "Keep file (no import)" import option for image in exported PCK, then use appropriate function such as "load_png_from_buffer" to load it after reading.

5. PCKReader.open() function don't give "offset" parameter default value which should be 0, imo.

My system is Debian 11 (bullseye) if that helps. Tried with original PCK and exported using this fork.

[AThousandShips]

When saving scene it didn't give it default filename like "main.tscn" if root node called "Main", for example, just empty line (irrelevant to this PR, but still strange).

That is a known and solved bug, rebase your branch and test again:

• Fix EditorFileDialog clears the file name on changing directory #81226

[yvie-k]

@ghmart Thanks for testing!

Can you share a PCK file that does not work for you so that I can take a look? I'm not sure how I'd recreate your issue

Regarding 4: I think godot does rename some files on export, so the PCKReader will only see the renamed filenames. Furthermore, text files are not packed into the pck by default. You may have to check your export settings whether txt files are included.

Unfortunately I'm quite busy at the moment, so it'll take a couple of days for me to take a look at this.

[ghmart]

Can you share a PCK file that does not work for you so that I can take a look? I'm not sure how I'd recreate your issue

"3." - not about problem inside PCK. That's when you press "Export PCK/ZIP" and "Save a File" dialog opens up - it shows only several folders (random at a glance), not all that exists at that path or when you choose other path in filesystem (file / folder permissions are all identical).

Regarding 4: I think godot does rename some files on export, so the PCKReader will only see the renamed filenames. Furthermore, text files are not packed into the pck by default. You may have to check your export settings whether txt files are included.

You were right, thank you. Resources are stored in PCK in their import form. "Text files" - all *.tscn, *.gd and *.tres files can be loaded using "get_string_from_utf8". Can't figure out how to make "Image" or "ImageTexture" object from *.ctex or *s3tc.ctex file loaded into PackedByteArray by PCKReader.

[yvie-k]

@ghmart I don't think 3 is due to my patch. As far as I know this is how Godot export always looked like. I think you have to disable the file filter on the export dialogue to see all files

[yvie-k]

Updated the commit with the suggestion from @ghmart (setting default offset for the open function to 0)

Rebased onto latest master, tested and still working.

@ghmart I think I see what the issue with the file dialogue is. It treats directories the same as files for me.

I couldn't find an open issue for that (or a mention somewhere that this was deliberate). So I will do a git bisect and open an issue for this.

[ObaniGemini]

Any news on this?