Resources text file unable to be opened in Windows, but works fine in Linux

tranxxenolab

@JonB Unfortunately trying to seek or get the size doesn't work, as the open command fails.

QFileInfo.size() returns 0 for one file, and -1 for another. It returns the correct size on the smaller JSON file.

On Linux the correct size is returned for all of the files.

Right now I am indeed doing the workaround of having the file outside of resources, but it's strange that it would be doing this and I'd like to figure out why :)

@Christian-Ehrlicher I enabled binary transfer using a .gitattributes and ensured that the file was changed and committed, and pulled. Nothing changed.

Both files have the same md5sum and are around 12 megabytes.

I'm stuck.

I'll try in the next day or two to see if I can make a MRE.

Thanks for the suggestions!

tranxxenolab

Hello everyone, I've created a minimum example with my actual data.

This is a simple example that opens a window and tries to read from three different JSON files in the resources. On Ubuntu 24.04 with Pyside 6.9.2, everything runs properly. On Windows and MacOS, reading fails with the larger 2 JSON files. Any help is appreciated!

The archive for the example can be downloaded from my Nextcloud (as I don't seem to have the ability to upload files here): https://nebula.tranxxenolab.net/s/Cob9ttPkZ9NmZzJ

JoeCFD

@tranxxenolab You have confirmed no issues when smaller json files are loaded. Can not you split your big json files into smaller ones? Then merge them after they are loaded. You may need to take a look at the source code and see how the loader works.

hskoglund

Yeah it might be something with the PySide resource system, I created a C++ Qt 6.9.2 Widget test app and got all 3 resource files to read in correctly both on Ubuntu 24.04 and Windows 11.

friedemannkleint

We had some issues with ARM ( QTBUG-140794 ), but plain Windows should work. The issue might be related though.

tranxxenolab

@hskoglund Yes, I've just confirmed that it does seem to be a difference between the way resources are created for PySide6 on Linux versus Windows. Running rcc -g python resource.qrc -o rc_resource.py (for the correct resource file) on Linux creates a very different file from what is created on Windows. In the particular example here, the files differ by almost 190kb between Windows (the larger file) and Linux. The number of lines is vastly different between the two files as well, by over a factor of 2 (as counted by wc, so it doesn't seem to just be a question of line endings).

But indeed, when I run rcc on Windows (which creates the larger file), the JSON files in the resources are read correctly on Windows. And, even more strangely, the resource file created on Windows can be properly read on Linux, but not the other way around.

I have no idea why this might be happening. I've submitted a bug report as it seems like extremely inconsistent behaviour: PYSIDE-3251

Christian Ehrlicher

I would guess a different compression is used which is not available on Windows

tranxxenolab

@Christian-Ehrlicher fair enough, but I haven't seen this documented anywhere, which it should be at least.

Christian Ehrlicher

You normally don't share resources between different operating systems as they are embedded in an executable. Now the problem is that python is somehwat special wrt this so...

tranxxenolab

@Christian-Ehrlicher Sure, although there are certainly use cases for compiling resources on one platform, checking into git, and then using those resources, especially during testing, on another platform. (@friedemannkleint talked about a similar use case.) To have resource files not be cross-platform compatible, but only in certain cases, should be something that is documented.

tranxxenolab

For anyone else who is having issues similar to mine: I have confirmed that by running pyside6-rcc with --compress-algo zlib allows for the creation of a resources file that works cross-platform, at the cost of a larger resources file size. So, if you need cross-platform compatibility for your resources files that are generated on Linux but pulled on other platforms, be sure to use zlib as the compression algorithm, rather than the default of zstd (which isn't even available on Windows).

I'm going to mark this as solved, but will be going through the bug report I mentioned above to hopefully get some documentation written about this inconsistency.