297 Relocate qt.conf after conda-unpack for Windows

[rayg-ssec]

This fixes inability to start the bundled form of SIFT on Windows. Tested in a VM by manually adding to SIFT 1.1.3 test bundle, so it may yet need addition to any manifest for the bundle build (have not found one so far).

• SIFT.bat now uses %~dp0 to include drive letter of the install location.
• qt_conf_fix.py is run by SIFT.bat at the time that conda-unpack is completed (first run).
• qt_conf_fix.py replaces Qt Library locations in qt.conf, required in order to start up.
• qt_conf_fix.py could be used by Unix OSes if also needed there, it takes the install directory base as its singular argument.

[djhoese]

Nice, thanks. Did you ever check if linux has this conf file? I wonder if this is a bug in conda-pack that should be fixed where it isn't detecting the paths in the qt.conf file on Windows.

[djhoese]

So I just downloaded the 1.1.3 tarball, unpacked it, and looked at bin/qt.conf. conda-unpack is actually fixing things on the linux side:

(base) davidh@janet:/tmp/SIFT_1.1.3$ cat bin/qt.conf
[Paths]
Prefix = /opt/anaconda1anaconda2anaconda3
Binaries = /opt/anaconda1anaconda2anaconda3/bin
Libraries = /opt/anaconda1anaconda2anaconda3/lib
Headers = /opt/anaconda1anaconda2anaconda3/include/qt

(base) davidh@janet:/tmp/SIFT_1.1.3$ bin/conda-unpack

(base) davidh@janet:/tmp/SIFT_1.1.3$ cat bin/qt.conf
[Paths]
Prefix = /tmp/SIFT_1.1.3
Binaries = /tmp/SIFT_1.1.3/bin
Libraries = /tmp/SIFT_1.1.3/lib
Headers = /tmp/SIFT_1.1.3/include/qt

[djhoese]

Side question: Where did 297 come from in the title?

[djhoese]

Ok best I can see this is a problem because on Windows qt.conf is in the root directory instead of the bin directory. Here is the conda-unpack portion of conda-pack:

https://github.com/conda/conda-pack/blob/ea74ae159ec0f1937979ac7fa0cff112c59e77d4/conda_pack/core.py#L949-L951

        if ((self.has_dest or
             file_mode == 'unknown' or
             file_mode == 'text' and file.target.startswith(BIN_DIR))):

So it probably detects a text file that isn't in the bin dir and skips over it.

Edit: Top of the module:

BIN_DIR = 'Scripts' if on_win else 'bin'

[djhoese]

So found some new evidence that this may be an unfortunate configuration choice of conda-forge. They seem to create the qt.conf file in <env_base>/Library/bin/ when you activate the environment:

https://github.com/conda-forge/qt-feedstock/blob/master/recipe/write_qtconf.bat

Here is the qt.conf they use for linux but not sure this is the same one that is used on Windows:

https://github.com/conda-forge/qt-feedstock/blob/master/recipe/qt.conf

[rayg-ssec]

297 comes from the ticket number.

Do we know if there's a deliberate reason these files are getting splashed into the base directory when unpacking, e.g. softlinks that are being added, which are getting turned into copies during archival? And if that's the case, is the better solution to ensure that the right binaries, scripts, config files, and DLLs are being used (from Library/bin rather than base)?

[djhoese]

Closes #297.

There now it is linked.

I have a feeling it is softlinks or something of the sort. In what I'm seeing qt.conf is the most out of place thing in the base directory. I think I'll create an issue with the conda-forge qt people and ask about the proper locations for these things or talk to conda-pack about modifying (or allowing additional) BIN search directories. Let me double check conda-pack first.

[djhoese]

It would seem this PR, with the custom solution for updating qt.conf, is not needed and instead we could run:

<env>\Scripts\.qt-post-link.bat

which is the conda-forge activation script for the Qt library. Why it isn't run in the conda-pack environment I'm not sure yet.

[djhoese]

They are apparently supposed to be run as part of conda's activation: https://docs.conda.io/projects/conda-build/en/latest/resources/link-scripts.html

[djhoese]

Ok I have finally figured this out. I couldn't let this problem go. This is not necessary a bug in conda-forge's decisions or in conda-pack but rather in a limitation of conda-pack for handling certain paths on Windows. The paths in qt.conf as-is are:

[Paths] 
Prefix = C:/Users/User/Miniconda3/envs/conda_unpack_test/Library
Binaries = C:/Users/User/Miniconda3/envs/conda_unpack_test/Library/bin
Libraries = C:/Users/User/Miniconda3/envs/conda_unpack_test/Library/lib
Headers = C:/Users/User/Miniconda3/envs/conda_unpack_test/Library/include/qt
TargetSpec = win32-msvc
HostSpec = win32-msvc

conda-pack searches for an old prefix with \\ (a single backslash) but replaces it with a single forward slash. As you can see the existing file produced by the post-link script already has forward slashes. So the find/replace that conda-unpack does never replaces these paths. One solution is to edit or add these lines:

('qt.conf', 'C:\\Users\\User\\Miniconda3\\envs\\conda_unpack_test', 'text'),
('Library\\bin\\qt.conf', 'C:\\Users\\User\\Miniconda3\\envs\\conda_unpack_test', 'text')

with these lines:

('qt.conf', 'C:/Users/User/Miniconda3/envs/conda_unpack_test', 'text'),
('Library\\bin\\qt.conf', 'C:\\Users\\User\\Miniconda3\\envs\\conda_unpack_test', 'text')

In <unpacked_env>/Scripts/conda-unpack-script.py. Doing this allows me to run this without error:

python -c "from PyQt5 import QtGui, QtWidgets; app = QtWidgets.QApplication([]); w = QtWidgets.QWidget(); w.show(); app.exec_()"

[djhoese]

Or...since we have control of the environment before conda-packing it, we could find/replace these exact paths in qt.conf to use \\ before running conda-pack. That way conda-unpack would find the prefix it is looking for. I will file a bug with conda-pack to ask for other ideas.

[djhoese]

conda/conda-pack#141

[djhoese]

Supersceded by #302