QList need to know all modifications

[JonB]

Hmm :) I have a class for a QList of (pointers to) elements of a particular class of mine, like

class MyClassList: public QList<const MyClass *>

(I don't think the const in the pointer is important, would be same if it were QList<MyClass *>.)

For internal reasons/debugging/my requirements I need that list-class to know whenever the list is modified (insert(), remove(), clear(), list[i] = ... etc.) --- I will be doing a bit of extra internal work (after calling base QList method to modify) on every alteration. On the other hand, I do not need to know about any read accesses, I want all of them to be passed straight through to underlying QList implementation.

As we know, QList does not provide any signals and does not supply any virtual methods to override. How best to go about this? I am open to any suggestions. It would be nice if it were fairly generic, but (especially for simplicity) I would consider an approach where I must only call a certain set of methods in the subclass to do modifications even if that is not all the ways the list can be altered.

My initial thoughts:

Approach #1:

• Derive from public QList (as shown above). Define new methods with same name/parameters as the modification methods, and have them call the base QList implementation

void MyClassList::append(const MyClass *t)
{
    QList::append(t);
    myExtraCode();
}

To be "complete" and "safe" I would need to define a new method for every existing modifying method.

Approach #2:

• Derive from private QList (or maybe protected). Same code as above for new definitions of modifying methods. But now I don't have to worry about redefining every write-method as only the ones I have defined are publicly available.

• In this case I would want all read-methods to be publicly available somehow. (I regard redefining every read method as too onerous.) Is there perhaps a C++ way of having a const method of MyClassList which returns something like a const &QList<const MyClass *> , something like

const QList<const MyClass *t> &MyClassList::list() const
{
    return *this;
}

This (I hope) gives access to the QList as a const reference so I should be able to call all const/read methods on the QList. In this case I would not want to have to write e.g. myListLinstance.list().at(i) where I used to write just myListLinstance.at(i), so would there then be a way of telling C++ that every time I write myListInstance.someMethod() where someMethod() is const it should "auto-convert" to the list() const method whenever I try to call a method on MyClassList which does not have an explicit, public declaration for the method (which my modification methods but not my read methods would have)?

Or any other way!? But please note I am not prepared to write reams of code for this, e.g. QList has 50+ methods, I am not going to write a new definition for all of these....

[JonB]

In the interests of transparency, I probably ought to admit why I want this functionality.

Using Creator debugger for gdb, I actually want a particular string representation for the whole list to be visible any time I look at the list e.g. as a local variable. While something like a Python debugger would allow me to call a function to return the string in a debugger pane it seems that gdb/C++ does not allow this, for understandable reasons. Where I want this for a simple element I have defined a class for that with a QString _debugStr member and I maintain/update that string every time the value in the element is altered. I am trying to achieve anything similar in the case of a QList of the elements.

While my QList subclass question is useful in its own right, if anyone knows of a way acceptable to gdb/Creator of making it show a QList of my elements the way I want as a string I generate in code please feel free to let me know!

P.S.
gdb has "pretty printers" per https://sourceware.org/gdb/current/onlinedocs/gdb.html/Pretty-Printing.html. However it says

GDB provides a mechanism to allow pretty-printing of values using Python code. It greatly simplifies the display of complex objects. This mechanism works for both MI and the CLI.

I am thinking that Creator visual debugger (e.g. Locals/Watch/Expression pane) is not "CLI" and I don't know what "MI" is. I'm fearing it does its own display of objects and does not use a gdb string pretty printer if defined?

P.P.S.
To my surprise for Creator debugger I see in https://doc.qt.io/qtcreator/creator-expressions-view.html#c-and-c-expressions

GDB, LLDB and CDB support the evaluation of simple C and C++ expressions. Functions can be called only if they are actually compiled into the debugged executable or a library used by the executable.

Well, my experience is that you cannot call any function, even in your own code, in say the Watch/Expression pane of Creator. You basically can only display variables, not call functions. Anyone find differently?

Ohhh, I have tried this now on the latest version of Creator. It does allow you to call non-inlined functions, I am pretty sure it did not used to! OK, this is not as desirable as being able to display what I want as e.g. a variable inside the class which shows automatically in say the Locals pane, I have to enter an expression each time, but it's good to know.

[jsulm]

@JonB If it does not have to be QList you could implement your class without subclassing QList, use QList as private member and implement all the interfaces you need. Of course you will not be able to use your class where QList is expected.

[JonB]

@jsulm said in QList need to know all modifications:

use QList as private member and implement all the interfaces you need

How does this relate to my

On the other hand, I do not need to know about any read accesses, I want all of them to be passed straight through to underlying QList implementation.

and

But please note I am not prepared to write reams of code for this, e.g. QList has 50+ methods, I am not going to write a new definition for all of these....

Of course I can always use encapsulation of any class if I am prepared to redefine/relay every single existing public method the base encapsulated class has so that there is a public equivalent on the wrapper encapsulating class. QList has at least 50 methods, most of which are const/read-only, I don't expect to have to define 50+ methods each relaying to the base one in this situation when I only need to alter (a certain number) of the write-methods, that is the goal of the question.

[jsulm]

@JonB Well, you don't have to implement all these 50 methods, only those you need.
I don't see any other solutions besides the two you mentioned.

"so would there then be a way of telling C++" - I don't think so.

[JonB]

@jsulm said in QList need to know all modifications:

I don't see any other solutions besides the two you mentioned.

Well that's fair enough, if that's the answer!

[Pl45m4]

@JonB said in QList need to know all modifications:

I need that list-class to know whenever the list is modified (insert(), remove(), clear(), list[i] = ... etc.) --- I will be doing a bit of extra internal work (after calling base QList method to modify) on every alteration.

Do you want to know IF the list has changed or also WHAT has changed?

During research for one of my projects I came across this lately.
(Approach to detect whenever an object changes, using hashes)

Maybe it helps

In general, if I had to do it, without further help and any fancy, hacky, magic I would do it like @jsulm said in QList need to know all modifications:

If it does not have to be QList you could implement your class without subclassing QList, use QList as private member and implement all the interfaces you need. Of course you will not be able to use your class where QList is expected.

With that you can design your API yourself and feed the internal QList<T> through your wrapper class.

and if you need to pass your list somewhere, you could simply add a getter to obtain the basic list type from your custom class.

QList<T*>& ObservableList::getQList() { return m_list; }

Or what about modifying your QList source code?
I mean, as long as it's for yourself only, and you don't need to built/share apps based on your custom Qt version, it's okay I guess (open source license-wise)

[SimonSchroeder]

Your best bet is to use C++29 😉. C++26 brings first parts for reflection in C++. However, there will be no way, yet, to synthesize member functions for a class. I hope for this to be included in C++29. Then you could automatically rewrite every method for your derived class.

On a more serious note: My impression is that the currently preferred solution would be to use composition (encapsulation) instead of inheritance.

Concerning your question about const QList<const MyClass *t> &MyClassList::list() const you can write it as a cast operator:

MyClass::operator(const QList<const MyClass *>&) const
{
    return *this; // or equivalent for composition
}

Then you'd be allowed to write something like this:

MyClassList list;
...
const QList<const MyClass*> &listRef = list;

I also think about using operator-> to simplify a few things. It would allow to not have to reimplement all methods, but maybe the syntax (and cognitive load) is a little bad:

const QList<const MyClass*> *MyClassList::operator->() const
{
    return this;
}

Then you could use a mixed syntax:

MyClassList list;
list.append({}); // use '.' because it is reimplemented in MyClassList
list->front(); // use '->' to forward to the underlying implementation of QList
list->at(0);

I don't think it is particularly beautiful to mix . and -> for the same object in this way (but it works). Also, you only get const access to anything contained in the list, i.e. you cannot change any of its contained items directly.

[JonB]

Thank you all for your various posts.

September was a long time ago now so I have moved on from this (without resolution, but that's OK now) :)

I do realise that in practice encapsulation is probably the best approach. It's just that I didn't want to do that, I wanted existing variables which are QList to remain as QList rather than having to call a method to access the list and/or having to write loads of methods on the encapsulator to access the QList member's methods. But, not surprisingly, nothing quite

It sounds like C++ 2129 might have features I need ;-)

Just musing here. C++ inheritance and virtual methods are great. But I am always disappointed at how many methods are not virtual when I wish they were. Here, for example, if all the base modification QList (and other classes') methods like append() etc. were protected virtual I could do what I want easily. I guess implementation-wise there IS some overhead in the code generated for a virtual method over a fixed one? Or not?

In my last days of using C# they introduced extension methods, whereby you can add your own new methods to existing base classes to extend them with extra functionality. If I use Python, or JavaScript, I can do that naughty "monkey patching" where I can effectively go QList.append = myNewImplementation, changing the actual method used for QList.append() to my own code which does whatever and calls the original base implementation. Naughty/dangerous but nice! But not from C++.

[J.Hilk]

Ok, I'm late to the party, buuuuut :D

You know, QList a very small class, as it is a template class, in fact I think you'll only need to hook into the the cpp file and the even viewer function implementations of QListData (the infamous d-pointer) as that is the actual part that is modifying the data.

once you decide to switch to release, you just link against/ship the original QList impl

[jeremy_k]

@JonB said in QList need to know all modifications:

Or any other way!? But please note I am not prepared to write reams of code for this, e.g. QList has 50+ methods, I am not going to write a new definition for all of these....

Have you considered using with private inheritance instead new definitions?

ie

template<typename T> class List: private QList<T> {
public:
    using QList<T>::append; // pass-thru
    T operator[](qsizetype index) {
        qDebug() << "calling []";
        return QList<T>::operator[](index);
    }
};

[SimonSchroeder]

@JonB said in QList need to know all modifications:

I guess implementation-wise there IS some overhead in the code generated for a virtual method over a fixed one? Or not?

Well, most of the time those functions in QList will be inlined. This opens up the possibility for a lot more optimizations. I would claim that good performance is important for something like QList. With the use of virtual methods in many cases the compiler cannot devirtualize the methodes. Sure, if you declare the variable and call functions within the same scope, nothing is lost. I would guess, however, that it is very common to hand a QList as a parameter to a function as a reference. This called function would have to keep the virtual function call and thus slow down the common case.

@JonB said in QList need to know all modifications:

If I use Python, or JavaScript, I can do that naughty "monkey patching"

Sounds if you would be a little happier with Objective-C (or Objective-C++) instead of C++. Method calls are just strings with a really fast lookup of the correct function. This makes it easy to override a method (you can even replace class implementations system-wide at runtime). I don't know if this is used for containers, though. In the end, it is still a lot like virtual functions and thus has the same drawbacks.

[JonB]

@SimonSchroeder said in QList need to know all modifications:

Method calls are just strings with a really fast lookup of the correct function.

Now hang on! Is that really true? That sounds more like an interpreter. I don't care how fast you claim a lookup might be, this must be orders of magnitude slower than compiling the address of the actual function --- and as I understand it C++ figures the actual correct one to call at compile time, following the inheritance chain at that time not at runtime. So not even one level of indirection. Unless that maybe applies to non-virtual methods only, but I thought it resolves virtual ones too.

[jsulm]

@JonB said in QList need to know all modifications:

So not even one level of indirection

There is for virtual methods. See vtable https://pabloariasal.github.io/2017/06/10/understanding-virtual-tables/
But it is not based on strings :-)

[JonB]

@jsulm
Yes, that's why I suspected indirection would be required in virtual case.

However, for a summary that article still did not make clear (to me) whether this is always a single level of indirection or as many multiple indirections as the inherited class hierarchy have their own overrides of a virtual method? I know what happens when class B overrides class A's virtual method and you write instanceAsA->virtualMethod() --- one indirection.

But what when you have 10 levels of inheritance, A -> B -> C -> D -> ..., each of which override the virtual method.? Does the runtime implementation of instanceAsA->virtualMethod() require 10 vtable/vpointer indirections, via each intermediate class, or does it go straight from A->virtualMethod() to J->virtualMethod()? That is a huge difference in practice when one has a deep class hierarchy?

[jsulm]

@JonB This is a good question. I'm not sure. But there is a way in gdb to inspect the vtable:

info vtbl b

b being an object. This way you could check the pointers in all the vtables and see how those were set up. If I find time I will also do this.
But it looks like the compiler puts the pointer to the "most derived function" in the vtable, so there is always one indirection. Search for "most derived function" in this thread: https://www.learncpp.com/cpp-tutorial/the-virtual-table/

[JonB]

@jsulm said in QList need to know all modifications:

the compiler puts the pointer to the "most derived function" in the vtable, so there is always one indirection.

That is what I thought, but did not know. There is a hell of a difference between a single indirection in machine code and a potentially iterative number of indirections for a deeply-nested inheritance...!

[SimonSchroeder]

Each class has its own vtable (it is not per object). Each object then has a pointer to the vtable. When you have a pointer to a base class pointing to an object of a derived class, it just uses the pointer to the vtable and calls the appropriate function. Like mentioned before, this is only one indirection. (It gets more complicated with multiple inheritance.)

Concerning Objective-C: Most strings for message are compile-time known strings. This means that "string comparison" in reality is most of the time just comparison of pointers. The runtime also uses a little cache to speed up the most recent function calls.

BTW: Since we are always talking about performance here. There is a CppCon talk on Youtube "Optimizing Away C++ Virtual Functions May be Pointless" https://youtu.be/i5MAXAxp_Tw?si=ieyiW3G31UMmANV-