Help with Autotest for Gerrit submission

[oblivioncth]

Hi,

I currently have two changes on Gerrit that I need to make autotests for and given that I've only ever deployed to Windows and been the lone programming in applications I've written I have never actually needed to make a test before.

https://doc.qt.io/qt-5/qtest-overview.html#creating-a-test is very helpful in general and makes it clear that the test's purpose to is demonstrate the exact functionality of some piece of code while providing a clear and concrete way for others to see exactly how you tested something, instead of relying on a written summary, so they can ensure it covers the correct cases and if desired run it themselves, potentially on other platforms, to check for completeness.

For my first submission that only affects the behavior of a signal I think I understand how I should write the test application:

• Locally compile qtbase with my changes and then create an auto-test project configured to use my local compilation as a kit
• Use booleans or an int value (or what is described in Chapter 2: Data Driven Testing if possible) to keep track of when the signal I changed has fired
• Make required connections in initTestCase()
• Create tests slots that use QTest::KeyAction & QTest:MouseAction to interact with the widget I changed in each possible way the user can within the scope of this signal
• Show using the above and QVERIFY/QCOMPARE that the signal I changed fires as and only when intended in all cases
• Break the connections and perform any other cleanup required in cleanupTestCase() so that the test can be run repeatedly

However for the second submission I'm a bit stuck: https://codereview.qt-project.org/c/qt/qtbase/+/312237/3

This adds a property that lets you control whether QCoreApplication::processEvents() is called on a modal QProgressDialog when setValue() is used. I can more or less use the same procedure as above and clearly can show a test with the property set to true and one with the property set to false, but I am unsure how I can demonstrate if processEvents() is triggered or not since it is part of the internal QProgressDialog implementation and there is no external feedback that setValue returns, nor is the a signal I can connect to that is emitted when processEvents() fires. The test could show that with the value set to false the progress bar remains frozen which is indicative of processEvents() not being called, but that is a qualitative indicator not a quantitative one (which seems to be the entire focus of autotests). I could also change my local implementation of my changes to add a signal or qDebug() statement that prints right after processEvents() is called but from what I understand these tests are supposed to be self-contained and apply directly to the exact implementation you are submitting (i.e. no hacks to make the test possible).

Basically, I'm just not sure how to check for if processEvents fired so I can use the result in a demonstrative autotest, even though I've checked that the behavior is correct directly using the debugger.

Finally, once these have been created, how do I submit the autotest code to Gerrit for the reviewers?

Thanks.

[SGaist]

Hi,

You are complicating stuff :-)

Qt already has lots of tests, you should add your new test cases to the unit tests that already exists for the class you modify.

Your second change shall avoid triggering a stack overflow, so you should write the test so that it triggers the overflow without your patch and pass with it.

[oblivioncth]

@SGaist Excellent, perfect suggestion for the second submission especially.

So I should just add the cases locally, ensure they test correctly, and then commit --amend & push the changed test file as part of the rest of my submission?

[SGaist]

@oblivioncth said in Help with Autotest for Gerrit submission:

So I should just add the cases locally, ensure they test correctly, and then commit --amend & push the changed test file as part of the rest of my submission?

That's correct yes.