How to configure Vulkan device when using native rendering in a QQuickItem?

mayart

Our software does all main rendering in background threads via contexts & device objects we maintain, and then we pass the final render to Qt on the main thread. This is done via using a QQuickItem and hooking into &QQuickWindow::beforeRendering to copy to a Qt-managed texture, and passing that texture as a QSGSimpleTextureNode via QQuickItem::updatePaintNode. This QQuickItem is then embedded into a traditional QWidget-based window, as the rest of our application does not use Qt Quick.

This works great for Metal (because MTLCreateSystemDefaultDevice() returns the same object for repeat calls) and OpenGL (we created shared contexts in the background thread derived from Qt's global shared context), but is completely failing as I'm porting our application to Vulkan.

Note: This is on Windows 10, Nvidia RTX 4000, using Qt 6.6.0.

From the documentation, I have tried three techniques.

The first is to let Qt create its own Vulkan instance. In order to transfer our rendered frame for Qt to use, we need to make use of VK_KHR_external_memory+_win32/fd and VK_KHR_external_semaphore+_win32/fd.

In the constructor of our QQuickWidget subclass, I enable the extensions using

if (quickWindow->graphicsApi() == QSGRendererInterface::Vulkan)
{
	QQuickGraphicsConfiguration config;
	config.setDebugLayer(true);
	config.setDeviceExtensions({
		"VK_KHR_timeline_semaphore",
		"VK_KHR_external_memory",
		"VK_KHR_external_semaphore",
#ifdef _WIN32
		"VK_KHR_external_memory_win32",
		"VK_KHR_external_semaphore_win32",
#else
		"VK_KHR_external_memory_fd",
		"VK_KHR_external_semaphore_fd",
#endif
	quickWindow->setGraphicsConfiguration(config);
}
qmlRegisterType<RenderingQQuickItem>("RenderPackage", 1, 0, "RenderingQQuickItem");
this->setSource(QUrl("qrc:/qml/RenderPackage/renderview.qml"));

According to the documentation, setGraphicsConfiguration should be picked up on the first QQuickWindow that instantiates. However, when I attempt at runtime

auto vi = vkInstance().functions();
PFN_vkGetMemoryWin32HandlePropertiesKHR vkGetMemoryWin32HandlePropertiesKHR = reinterpret_cast<PFN_vkGetMemoryWin32HandlePropertiesKHR>(vi->vkGetDeviceProcAddr(vkDevice(), "vkGetMemoryWin32HandlePropertiesKHR"));

then vkGetMemoryWin32HandlePropertiesKHR is NULL, despite the extension being enabled.

The second technique I tried is to pass Qt our own VkDevice, and an unused but available queue index for execution.

if (quickWindow->graphicsApi() == QSGRendererInterface::Vulkan)
{
	QQuickGraphicsDevice device = QQuickGraphicsDevice::fromDeviceObjects(ourPhysicalDevice, ourDevice, queueFamilyIndex, queueIndex);
	quickWindow->setGraphicsDevice(device);
}

With this setup, at runtime we check

VkDevice qtDevice = *reinterpret_cast<VkDevice*>(window->rendererInterface()->getResource(window, QSGRendererInterface::DeviceResource));

then qtDevice ≠ ourDevice. It is clear that calling setGraphicsDevice or setGraphicsConfiguration does not propagate down to QQuickItems contained within the QQuickWindow it's called on.

The third technique I tried is to create a global QVulkanInstance to set on the main window, assuming it'll propagate down to the QQuickWindow even though it has already been constructed. In our main,

if (QQuickWindow::graphicsApi() == QSGRendererInterface::Vulkan)
{
	QVulkanInstance* vkInstance = new QVulkanInstance;
	vkInstance->setLayers({ "VK_LAYER_KHRONOS_validation" });
		vkInstance->setExtensions({
		VK_KHR_TIMELINE_SEMAPHORE_EXTENSION_NAME,
		VK_KHR_EXTERNAL_MEMORY_EXTENSION_NAME,
		VK_KHR_EXTERNAL_SEMAPHORE_EXTENSION_NAME,
#ifdef _WIN32
		"VK_KHR_external_memory_win32",
		"VK_KHR_external_semaphore_win32",
#else
		"VK_KHR_external_memory_fd",
		"VK_KHR_external_semaphore_fd",
#endif
		});
	vkInstance->create();
	mainWindow->setVulkanInstance(vkInstance);
}

This fails to compile because QWindow::setVulkanInstance is hidden behind a QT_CONFIG(vulkan) check, which evidently is false on the installed version of Qt.

I'm assuming the third method would work if we compiled Qt from source, but then we would have to distribute the compiled version to all our developers, rather than just using the binaries from the Qt installer.

Surely there must be a way of configuring Qt to use these additional extensions, could somebody please tell me what I'm doing wrong?