Qt5.7 QVector3D::unproject() mouse picking

[Alkoou]

Hi !
I would like to "select" an object in my OpenGL world but first of all, I need to find the mouse position in the OpengGL world coordinates and I thought that the QVector3D::unproject() function was the best but I'm not getting what I want : the world coordinates I get don't seem to be right as I created a triangle, one of his corner is (-0.5, -0.5, 0.0) and when I click with the mouse on it, I get (1.96532, -3.93444, 4.93216).
I tried to change the z coordinate, move the camera ... but I never get close to a working code.
This is a part of my code so far :

QMatrix4x4 modelView;
modelView.setToIdentity();
modelView.lookAt(m_camera.getPosition(), m_camera.getTarget(), m_camera.getUp());

QMatrix4x4 projection;
projection.perspective(70.0, width() / height(), 0.1, 120.0);

GLint view[4];
glGetIntegerv(GL_VIEWPORT, &view[0]);

QVector3D worldPosition = QVector3D(event->pos().x(), event->pos().y(), 0).unproject(modelView, projection, QRect(view[0], view[1], view[2], view[3]));
qDebug() << worldPosition;

I don't understand what's going wrong. Do you have a clue ?
Thanks in advance

[glararan]

Try and edit following function from my old project: https://github.com/glararan/QEditor/blob/master/src/mapview.cpp#L1175

I guess based on your description, this is what you're looking for.

[Alkoou]

Thank you for the help, I copy / pasted your code just to try it but I get values from (1.9, -3.8, 4.9) at the top left-hand corner of the window to (2.0, -3.9, 4.8) at the bottom right-hand corner of the window.
There is your code with my modifications. Do you see what I'm doing wrong ?

    QMatrix4x4 viewMatrix ;
    viewMatrix.lookAt(m_camera.getPosition(), m_camera.getTarget(), m_camera.getUp());

    QMatrix4x4 projection;
    projection.perspective(70.0, width() / height(), 0.1, 120.0);

    QMatrix4x4 viewportMatrix;
    float w2 = width() / 2.0f;
    float h2 = height() / 2.0f;

    viewportMatrix.setToIdentity();
    viewportMatrix.setColumn(0, QVector4D(w2, 0.0f, 0.0f, 0.0f));
    viewportMatrix.setColumn(1, QVector4D(0.0f, h2, 0.0f, 0.0f));
    viewportMatrix.setColumn(2, QVector4D(0.0f, 0.0f, 1.0f, 0.0f));
    viewportMatrix.setColumn(3, QVector4D(w2, h2, 0.0f, 1.0f));

    QMatrix4x4 modelMatrix;
    modelMatrix.setToIdentity();

    QMatrix4x4 modelViewMatrix = viewMatrix * modelMatrix;
    QMatrix4x4 modelViewProject = projection * modelViewMatrix;
    QMatrix4x4 inverted = viewportMatrix * modelViewProject;

    inverted = inverted.inverted();

    float posZ;
    float posY = height() - y - 1.0f;

    glReadPixels((int)x, (int)posY, 1, 1, GL_DEPTH_COMPONENT, GL_FLOAT, &posZ);

    QVector4D clickedPointOnScreen(x, posY, 2.0f * posZ - 1.0f, 1.0f);
    QVector4D clickedPointIn3DOrgn = inverted * clickedPointOnScreen;

    clickedPointIn3DOrgn /= clickedPointIn3DOrgn.w();

    qDebug() << clickedPointIn3DOrgn.toVector3DAffine();

[glararan]

Try set viewMatrix and projection to setIdentity() but its last thing what can help :D

I checked your edited code and it seems okay. Where do you call it?

[Alkoou]

I did some experimental researches and I finally discovered the problem, it was the z coordinate.
I've found the experimental value (0.961635) but I need the law to make the code work if I move the camera.
This is a new project that I made to see things more clearly :

#include "mainwindow.h"
#include <QVector3D>
#include <QKeyEvent>
#include <qDebug>

MainWindow::MainWindow(QWidget *parent)
    : QOpenGLWidget(parent)
{
}

MainWindow::~MainWindow()
{
}

void MainWindow::initializeGL()
{
    initializeOpenGLFunctions();
    glClearColor(0.7f, 0.7f, 1.0f, 0.0f);

    m_program.addShaderFromSourceFile(QOpenGLShader::Vertex, ":/vertexshader.vsh");
    m_program.addShaderFromSourceFile(QOpenGLShader::Fragment, ":/fragmentshader.fsh");
    m_program.link();

    m_matrix.setToIdentity();

    m_matrix.perspective(70.0, width() / height(), 0.1, 120.0);
    m_matrix.lookAt(QVector3D(0, 0, 5), QVector3D(0, 0, 0), QVector3D(0, 1, 0));
}

void MainWindow::paintGL()
{
    glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);

    m_program.bind();

    m_program.setUniformValue("mvp", m_matrix);

    GLfloat vertices[] = {
        0.0f, 1.0f, 0.0f,
        -0.5f, -0.5f, 0.0f,
        0.5f, -0.5f, 0.0f
    };

    glVertexAttribPointer(m_program.attributeLocation("vertexPosition_modelspace"), 3, GL_FLOAT, GL_FALSE, 0, vertices);

    glEnableVertexAttribArray(0);

    glDrawArrays(GL_TRIANGLES, 0, 3);

    glDisableVertexAttribArray(0);

    m_program.release();
}

void MainWindow::mousePressEvent(QMouseEvent *event)
{
    QMatrix4x4 viewMatrix ;
    viewMatrix.setToIdentity();
    viewMatrix.lookAt(QVector3D(0, 0, 5), QVector3D(0, 0, 0), QVector3D(0, 1, 0));

    QMatrix4x4 projection;
    projection.setToIdentity();
    projection.perspective(70.0, width() / height(), 0.1, 120.0);

    QMatrix4x4 viewportMatrix;
    float w2 = width() / 2.0f;
    float h2 = height() / 2.0f;

    viewportMatrix.setToIdentity();
    viewportMatrix.setColumn(0, QVector4D(w2, 0.0f, 0.0f, 0.0f));
    viewportMatrix.setColumn(1, QVector4D(0.0f, h2, 0.0f, 0.0f));
    viewportMatrix.setColumn(2, QVector4D(0.0f, 0.0f, 1.0f, 0.0f));
    viewportMatrix.setColumn(3, QVector4D(w2, h2, 0.0f, 1.0f));

    QMatrix4x4 modelMatrix;
    modelMatrix.setToIdentity();

    QMatrix4x4 modelViewMatrix = viewMatrix * modelMatrix;
    QMatrix4x4 modelViewProject = projection * modelViewMatrix;
    QMatrix4x4 inverted = viewportMatrix * modelViewProject;

    inverted = inverted.inverted();

    float posZ;
    float posY = height() - event->pos().y() - 1.0f;

    glReadPixels((int)event->pos().x(), (int)posY, 1, 1, GL_DEPTH_COMPONENT, GL_FLOAT, &posZ);

    qDebug() << 2.0f * posZ - 1.0f;
    QVector4D clickedPointOnScreen(event->pos().x(), posY, 0.961635, 1.0f);
    QVector4D clickedPointIn3DOrgn = inverted * clickedPointOnScreen;

    clickedPointIn3DOrgn /= clickedPointIn3DOrgn.w();

    qDebug() << clickedPointIn3DOrgn.toVector3DAffine();
}

[Alkoou]

After some testing, it turned out that I need the projected z coordinate to unproject the x and y but in order to get it, I need the unprojected x and y ... it's an infinite loop !
Here's the code I came up with :

QMatrix4x4 viewMatrix ;
viewMatrix.lookAt(m_camera.getPosition(), m_camera.getTarget(), m_camera.getUp());

QMatrix4x4 projection;
projection.perspective(70.0, width() / height(), 0.1, 120.0);

QVector3D Z(0, 0, 0); // instead of 0 for x and y i need worldPosition.x() and worldPosition.y() ....
Z = Z.project(viewMatrix, projection, QRect(0, 0, width(), height()));

QVector3D worldPosition = QVector3D(x, height() - y, Z.z()).unproject(viewMatrix, projection, QRect(0, 0, width(), height()));
qDebug() << worldPosition;

[MicPiano]

I was searching for a way the pass few days. This reference is the most detailed I have seen.
Long story short, screen coordinates (x, y, 0) will transfer into the near field z value, and (x, y, 1) will transfer into far field z value. If you want the values in between a specific position, linear interpolation will help.

[BPrasad]

@Alkoou

Hi Alkoou,
can you share your full code? I am currently working on the same topic.. It would be really helpful.