Qt 5.7: QtWebEngineWidgets not working on Linux

[Stefan Scherfke]

Hi all,

I am currently trying to build Qt5 + QtWebEngine for PyQt5 in
a conda environment.

Most components (e.g., QtWidgets) work as expected but I have problems
getting QtWebEngine(Widgets) to work.

I will first show the errors I got and then how I built Qt5.7.

My issues

The first issue I had was that the file icudtl.dat was missing. It should
have been installed into share/qt5/resources/icudtl.dat but was not. I got
this error and then the app crashed:

Qt WebEngine ICU data not found at /home/stefan/pydist/envs/qttest/share/qt5/resources. Trying parent directory...
Qt WebEngine ICU data not found at /home/stefan/pydist/envs/qttest/share/qt5. Trying application directory...
Qt WebEngine ICU data not found at /home/stefan/pydist/envs/qttest/bin. Trying fallback directory... The application MAY NOT work.
Installed Qt WebEngine locales directory not found at location /home/stefan/pydist/envs/qttest/share/qt5/translations/qtwebengine_locales. Trying application directory...
Qt WebEngine locales directory not found at location /home/stefan/pydist/envs/qttest/bin/qtwebengine_locales. Trying fallback directory... Translations MAY NOT not be correct.
[0822/102244:ERROR:icu_util.cc(162)] Invalid file descriptor to ICU data received.
[0822/102244:FATAL:content_main_runner.cc(714)] Check failed: base::i18n::InitializeICU().

After I copied the file from
qt-everywhere-opensource-src-5.7.0/qtwebengine/src/3rdparty/chromium/third_party/icu/source/data/in/icudtl.dat,
I got a new errors. The app did not crash but instead of the rendered HTML
I more or less get the raw HTML (see this screenshot:
https://imgur.com/a/CKRpQ):

Installed Qt WebEngine locales directory not found at location /home/stefan/pydist/envs/qttest/share/qt5/translations/qtwebengine_locales. Trying application directory...
Qt WebEngine locales directory not found at location /home/stefan/pydist/envs/qttest/bin/qtwebengine_locales. Trying fallback directory... Translations MAY NOT not be correct.
Qt WebEngine resources not found at /home/stefan/pydist/envs/qttest/share/qt5/resources. Trying parent directory...
Qt WebEngine resources not found at /home/stefan/pydist/envs/qttest/share/qt5. Trying application directory...
Qt WebEngine resources not found at /home/stefan/pydist/envs/qttest/bin. Trying fallback directory... The application MAY NOT work.
[0822/104459:ERROR:resource_bundle.cc(720)] Failed to load /home/stefan/.qttest.py/qtwebengine_resources.pak
Some features may not be available.
[0822/104459:ERROR:resource_bundle.cc(720)] Failed to load /home/stefan/.qttest.py/qtwebengine_resources_100p.pak
Some features may not be available.
[0822/104459:ERROR:resource_bundle.cc(720)] Failed to load /home/stefan/.qttest.py/qtwebengine_resources_200p.pak
Some features may not be available.
[0822/104459:ERROR:resource_bundle.cc(720)] Failed to load /home/stefan/.qttest.py/qtwebengine_devtools_resources.pak
Some features may not be available.
[0822/104459:WARNING:resource_bundle.cc(311)] locale_file_path.empty() for locale
[0822/104459:ERROR:browser_main_loop.cc(217)] Running without the SUID sandbox! See https://chromium.googlesource.com/chromium/src/+/master/docs/linux_suid_sandbox_development.md for more information on developing with the sandbox on.
Installed Qt WebEngine locales directory not found at location /home/stefan/pydist/envs/qttest/share/qt5/translations/qtwebengine_locales. Trying application directory...
Qt WebEngine locales directory not found at location /home/stefan/pydist/envs/qttest/lib/qt5/libexec/qtwebengine_locales. Trying fallback directory... Translations MAY NOT not be correct.
Qt WebEngine resources not found at /home/stefan/pydist/envs/qttest/share/qt5/resources. Trying parent directory...
Qt WebEngine resources not found at /home/stefan/pydist/envs/qttest/share/qt5. Trying application directory...
Qt WebEngine resources not found at /home/stefan/pydist/envs/qttest/lib/qt5/libexec. Trying fallback directory... The application MAY NOT work.
[0822/104459:ERROR:resource_bundle.cc(720)] Failed to load /home/stefan/.QtWebEngineProcess/qtwebengine_resources.pak
Some features may not be available.
[0822/104459:ERROR:resource_bundle.cc(720)] Failed to load /home/stefan/.QtWebEngineProcess/qtwebengine_resources_100p.pak
Some features may not be available.
[0822/104459:ERROR:resource_bundle.cc(720)] Failed to load /home/stefan/.QtWebEngineProcess/qtwebengine_resources_200p.pak
Some features may not be available.
[0822/104459:ERROR:resource_bundle.cc(720)] Failed to load /home/stefan/.QtWebEngineProcess/qtwebengine_devtools_resources.pak
Some features may not be available.
[0822/104459:WARNING:resource_bundle.cc(311)] locale_file_path.empty() for locale

It looks again as if some required data files did not get installed, although
this time, the errors don’t lead to a crash. There’s also the error
[0822/104459:ERROR:browser_main_loop.cc(217)] Running without the SUID sandbox! See https://chromium.googlesource.com/chromium/src/+/master/docs/linux_suid_sandbox_development.md
for more information on developing with the sandbox on.

So my two main issues I could not yet find a solution for:

1. How to I get the required resources to be installed into share/qt5?
2. How to I fix the sandboxing issue with Chromium?

How I built Qt5

My base build system is Centos 7.

I installed the following requirements system system wide:

$ yum groupinstall 'Development Tools'
$ yum install mesa-libGL-devel libX11-devel libXext-devel libXfixes-devel libXi-devel libXrender-devel libxcb-devel
$ yum install bison flex gperf

The remaining requirements were installed as Conda packages:

• fontconfig
• freetype
• libicu (I tried it with and without this lib as it is also bundled with the
  Chromium that is bundles with Qt 5.7)
• libjpeg
• libmysqlclient
• libpng
• openssl
• xz
• zlib

I applied two patches to Qt5 to fix an issue with libjpeg9 and glibc:

• https://bugs.chromium.org/p/chromium/issues/detail?id=374956
• http://www.linuxfromscratch.org/blfs/view/svn/x/qt5.html

The Conda build.sh (the build script) looks like this (it is largely based on
this recipe):

# Main variables
# --------------
BIN=$PREFIX/lib/qt5/bin
QTCONF=$BIN/qt.conf
VER=$PKG_VERSION
CXXFLAGS=-fno-delete-null-pointer-checks


# Compile
# -------
chmod +x configure

# This might only work on Linux.  Check
# https://github.com/conda/conda-recipes/tree/master/qt5 for OS X support

MAKE_JOBS=$CPU_COUNT

./configure -prefix $PREFIX \
            -libdir $PREFIX/lib \
            -bindir $PREFIX/lib/qt5/bin \
            -headerdir $PREFIX/include/qt5 \
            -archdatadir $PREFIX/lib/qt5 \
            -datadir $PREFIX/share/qt5 \
            -L $PREFIX/lib \
            -I $PREFIX/include \
            -verbose \
            -release \
            -opensource \
            -confirm-license \
            -shared \
            -nomake examples \
            -nomake tests \
            -qt-pcre \
            -qt-xcb \
            -qt-xkbcommon \
            -dbus \
            -optimized-qmake

LD_LIBRARY_PATH=$PREFIX/lib make -j $MAKE_JOBS
make install


# Post build setup
# ----------------

# Remove unneeded files
rm -rf $PREFIX/share/qt5

# Make symlinks of binaries in $BIN to $PREFIX/bin
for file in $BIN/*
do
    ln -sfv ../lib/qt5/bin/$(basename $file) $PREFIX/bin/$(basename $file)-qt5
done

# Remove static libs
rm -rf $PREFIX/lib/*.a

# Add qt.conf file to the package to make it fully relocatable
cp $RECIPE_DIR/qt.conf $BIN/

As I said before, this mostly works quite well except that the ICU data for
Qt5 / QtWebEngine is not packaged/installed and that Chromium has these
sandboxing issues.

Summary

I did a lot of research during the last two weaks and re-compiled qt5 many
times. I made some progress and got it mostly working, but two issues still
remain:

1. How to I get the required resources to be installed into share/qt5?
2. How to I fix the sandboxing issue with Chromium?

I am stucked and have not yet found any solutions for them.

[Stefan Scherfke]

I finally found the solution:

When QtWebEngine is compiled, it generates the required resources in

• qtwebengine/src/core/Release/icudtl.dat
• qtwebengine/src/core/Release/gen/repack/*.pak
• qtwebengine/src/core/Release/gen/repack/qtwebengine_locales/*

The icudtl.dat and *.pak files have to be copied into share/qt5/resources/ and the qtwebengine_locales folter into share/qt5/translations/.

[Stefan Scherfke]

Hi,

just made some progress: The binary PyQt5-package from PyPI comes with a bundled Qt5.
If I copy the translations and resource folders from it to share/qt5/ of my
self-compiled qt5, it works as expected.

So the question is: How can I create all necessary files in share/qt5 automatically when
I build Qt5?

[Stefan Scherfke]

I finally found the solution:

When QtWebEngine is compiled, it generates the required resources in

• qtwebengine/src/core/Release/icudtl.dat
• qtwebengine/src/core/Release/gen/repack/*.pak
• qtwebengine/src/core/Release/gen/repack/qtwebengine_locales/*

The icudtl.dat and *.pak files have to be copied into share/qt5/resources/ and the qtwebengine_locales folter into share/qt5/translations/.

[zhouyy]

mark
qtwebengine_resources.pak + qtwebengine_resources_100p.pak+ qtwebengine_resources_200p.pak