Is VirtualCan from SerialBus limited to 1 server on Localhost?

[zeroflagnet]

i am using a wsl for development in Ros and want to acces the can bus. So i wrote with Qt an little programm to which reads/writes on the local canbus, transceives it via virtualCan to an remote destination. So i used qt serialbus to generate with the qt virtualcan a bridge between win and wsl for can.

for this i use 2 param files, one for windows and one for the wsl ubuntu (custom kernel with vcan).
I only got one Server working. but not 1+n
if i use the config with 2 different ports only one Can bridge is working with one can and one server. if i use the same ports. both can bridges are working. but only with one server.

this leaves me to max 2 Can Connections. but i want to have at least 3. and therfor i need atleast a second running virtuacan server.

So my Question, is it possible to run more then one, when using the localhost? If yes, why are am so blind to not find the reason why only one server is generated and working. If no, are there other ways to get 2 servers running so i can have 3 Cans bridged from win to wsl.

attached c code and parameter files.
parameter ubuntu:

{
    "can_buses": [
        {
            "physical_interface": "can1",
            "plugin": "socketcan",
            "parameters": {
                "host": "127.0.0.1",
                "port": 35476,
                "bitrate": 1000000,
                "data_bitrate": 2000000,
                "can_fd": false,
                "name": "can1"
            }
        },
        {
            "physical_interface": "can0",
            "plugin": "socketcan",
            "parameters": {
                "host": "127.0.0.1",
                "port": 35466,
                "bitrate": 1000000,
                "data_bitrate": 2000000,
                "can_fd": false,
                "name": "can0"
            }
        }
    ]
}

parameter windows:

{
    "can_buses": [
        {
            "physical_interface": "usb0",
            "plugin": "peakcan",
            "parameters": {
                "host": "127.0.0.1",
                "port": 35466,
                "bitrate": 1000000,
                "data_bitrate": 2000000,
                "can_fd": false,
                "name": "can0"
            }
        },
        {
            "physical_interface": "usb1",
            "plugin": "peakcan",
            "parameters": {
                "host": "127.0.0.1",
                "port": 35476,
                "bitrate": 1000000,
                "data_bitrate": 2000000,
                "can_fd": false,
                "name": "can1"
            }
        }
    ]
}

c code:

#include <QCoreApplication>
#include <QCanBus>
#include <QCanBusDevice>
#include <QCanBusFrame>
#include <QDebug>
#include <QFile>
#include <QJsonDocument>
#include <QJsonObject>
#include <QJsonArray>
#include <QLoggingCategory>
#include <QThread>
#include <iostream>
#ifdef Q_OS_WIN
    #include <winsock2.h>
#endif

struct CanBusConfig {
    QString physicalInterface;
    QString plugin;
    QVariantMap parameters;
};

QList<CanBusConfig> loadCanBusConfig(const QString &fileName) {
    QList<CanBusConfig> configs;
    QFile file(fileName);
    if (!file.open(QIODevice::ReadOnly | QIODevice::Text)) {
        qCritical() << "Could not open config file:" << fileName;
        return configs;
    }

    QByteArray data = file.readAll();
    QJsonDocument doc = QJsonDocument::fromJson(data);
    QJsonObject json = doc.object();
    QJsonArray canBuses = json["can_buses"].toArray();

    for (const QJsonValue &value : canBuses) {
        QJsonObject obj = value.toObject();
        CanBusConfig config;
        config.physicalInterface = obj["physical_interface"].toString();
        config.plugin = obj["plugin"].toString();
        config.parameters = obj["parameters"].toObject().toVariantMap();
        configs.append(config);
    }

    return configs;
}

QCanBusDevice::ConfigurationKey parseConfigurationkey(const QString &key) {
    static const QMap<QString, QCanBusDevice::ConfigurationKey> keyMap = {
        {"raw_filter", QCanBusDevice::RawFilterKey},
        {"error_filter", QCanBusDevice::ErrorFilterKey},
        {"loopback", QCanBusDevice::LoopbackKey},
        {"receive_own", QCanBusDevice::ReceiveOwnKey},
        {"bitrate", QCanBusDevice::BitRateKey},
        {"user", QCanBusDevice::UserKey},
        {"can_fd", QCanBusDevice::CanFdKey},
        {"data_bitrate", QCanBusDevice::DataBitRateKey},
        {"protocol", QCanBusDevice::ProtocolKey}
    };
    return keyMap.value(key, QCanBusDevice::ConfigurationKey(-1));  
}



QVariant parseConfigurationValue(QCanBusDevice::ConfigurationKey &configKey, const QVariant &value) {
    if (configKey!=QCanBusDevice::ConfigurationKey(-1)) {
        switch (configKey) {
            case QCanBusDevice::RawFilterKey:
            case QCanBusDevice::ErrorFilterKey:
                return QVariant::fromValue(value.toList());
                break;
            case QCanBusDevice::LoopbackKey:
            case QCanBusDevice::ReceiveOwnKey:
            case QCanBusDevice::CanFdKey:
                return QVariant::fromValue(value.toBool());
                break;
            case QCanBusDevice::BitRateKey:
            case QCanBusDevice::DataBitRateKey:
                return QVariant::fromValue(value.toInt());
                break;
            case QCanBusDevice::ProtocolKey:
                return QVariant::fromValue(value.toString());
                break;
            case QCanBusDevice::UserKey:
                return value; // UserKey can be any QVariant
                break;
            default:
                return QVariant();
                break;
        }
    } else {
        qWarning() << "Unknown configuration key:" << configKey;
        return QVariant();
    }
}

void processReceivedFrames(QCanBusDevice *sourceDevice, QCanBusDevice *targetDevice) {
    if (sourceDevice->state() != QCanBusDevice::ConnectedState) {
        qWarning() << "Failed to connect source device:" << sourceDevice->errorString() << "and is in state:" << sourceDevice->state();
    }
    if (targetDevice->state() != QCanBusDevice::ConnectedState) {
        qWarning() << "Failed to connect target device:" << targetDevice->errorString() << "and is in state:" << targetDevice->state();
    }
    while (sourceDevice->framesAvailable()) {
        QCanBusFrame frame = sourceDevice->readFrame();
        // Send the frame to the target CAN device
        targetDevice->writeFrame(frame);
    }
}

bool connectVirtualDevice(QCanBusDevice *virtualDevice) {
    const int maxRetries = 10;
    const int retryDelayMs = 1000; // 1 second

    for (int attempt = 0; attempt < maxRetries; ++attempt) {
        if (virtualDevice->connectDevice()) {
            return true;
        } else {
            QThread::sleep(retryDelayMs / 1000); // Sleep for retryDelayMs milliseconds
        }
    }

    return false;
}

int main(int argc, char *argv[]) {
    QCoreApplication app(argc, argv);
    QLoggingCategory::setFilterRules(QStringLiteral(
        "qt.serialbus.*=false\n"
        "qt.canbus.plugins.virtualcan.*=false\n"
    ));
    
    // Get the directory of the executable for finding the parameters.json file
    QString execDir = QCoreApplication::applicationDirPath();
    QString configPath = execDir + "/parameters.json";
    
    QList<CanBusConfig> configs = loadCanBusConfig(configPath);
    if (configs.isEmpty()) {
        qCritical() << "No CAN bus configurations found in" << configPath;
        return -1;
    }
    
    // Platform-specific plugin check
    QCanBus *canBus = QCanBus::instance();
    if (!canBus) {
        qCritical() << "Could not get QCanBus instance";
        return -1;
    }

    QStringList availablePlugins = QCanBus::instance()->plugins();
    qInfo() << "Available CAN plugins:" << availablePlugins;
    
    #ifdef Q_OS_WIN
    // On Windows, we might use another plugin instead of socketcan
    if (!availablePlugins.contains("peakcan") && !availablePlugins.contains("systeccan") && 
        !availablePlugins.contains("virtualcan")) {
        qWarning() << "No suitable CAN plugin found for Windows. Available plugins:" << availablePlugins;
    }
    #else
    // On Linux, check for socketcan
    if (!availablePlugins.contains("socketcan") && !availablePlugins.contains("virtualcan")) {
        qWarning() << "Neither socketcan nor virtualcan plugin is available. Available plugins:" << availablePlugins;
    }
    #endif

    QList<QCanBusDevice*> devices;
    QList<QCanBusDevice*> virtualDevices;

    for (const CanBusConfig &config : configs) {
        QString errorString;

        qDebug() << "Creating CAN device with plugin:" << config.plugin << "and interface:" << config.physicalInterface;
        QCanBusDevice *device = canBus->createDevice(config.plugin, config.physicalInterface, &errorString);
        if (!device) {
            qCritical() << "Could not create CAN device:" << errorString;
            continue;
        }

        // Set physical CAN device parameters
        for (auto it = config.parameters.begin(); it != config.parameters.end(); ++it) {
            if (it.key() != "host" && it.key() != "port" && it.key() != "name") {
                QCanBusDevice::ConfigurationKey key = parseConfigurationkey(it.key());
                if (key == QCanBusDevice::ConfigurationKey(-1)) {
                    qCritical() << "Invalid configuration key:" << it.key();
                    continue;
                }
                QVariant val= parseConfigurationValue(key, it.value());
                device->setConfigurationParameter(QCanBusDevice::ConfigurationKey(key), val);
                
                if (device->error() != QCanBusDevice::NoError) {
                    qCritical() << "Error setting parameter" << it.key() << ":" << device->errorString();
                }
            }
        }

        if (!device->connectDevice()) {
            qCritical() << "Could not connect to CAN device:" << device->errorString();
            continue;
        }

        // Hardcode the creation of the virtualcan device
        QString virtualInterface = QStringLiteral("tcp://%1:%2/%3")
            .arg(config.parameters.value("host", "localhost").toString())
            .arg(config.parameters.value("port", 35468).toInt())
            .arg(config.parameters.value("name", "can0").toString());

        std::cout << "ip: " << config.parameters.value("host").toString().toStdString() << "port: " << config.parameters.value("port").toString().toStdString() << "name: " << config.parameters.value("name").toString().toStdString()<< std::endl ;  
        std::cout << "Virtual interface:" << virtualInterface.toStdString() << std::endl;
        QUrl m_url=QUrl(virtualInterface);
        QString host = m_url.host();
        quint16 port = static_cast<quint16>(m_url.port());
        std::cout << "host: " << host.toStdString() << std::endl;
        std::cout << "port: " << port << std::endl;

        QString virtualDeviceName = config.parameters.value("name").toString();
        if (virtualDeviceName != "can0" && virtualDeviceName != "can1") {
            qCritical() << "Invalid virtual CAN device name:" << virtualDeviceName;
            continue;
        }

        QCanBusDevice *virtualDevice = QCanBus::instance()->createDevice(QStringLiteral("virtualcan"), virtualInterface);
        if (!virtualDevice) {
            qCritical() << "Could not create virtual CAN device:" << errorString;
            continue;
        }
        QCanBusDeviceInfo info= virtualDevice->deviceInfo();
        std::cout << std::endl << "alias: " << info.alias().toStdString() << std::endl;
        std::cout << "channel: " << info.channel() << std::endl;
        std::cout << "flexibelDR?: " << info.hasFlexibleDataRate() << std::endl;
        std::cout << "virtual?: " << info.isVirtual() << std::endl;
        std::cout << "name: " << info.name().toStdString() << std::endl;
        std::cout << "plugin: " << info.plugin().toStdString() << std::endl;
        std::cout << "serialNumber: " << info.serialNumber().toStdString() << std::endl<< std::endl;

        // Set virtual CAN device parameters
        for (auto it = config.parameters.begin(); it != config.parameters.end(); ++it) {
            if (it.key() != "host" && it.key() != "port") {
                virtualDevice->setConfigurationParameter(QCanBusDevice::ConfigurationKey(it.key().toInt()), it.value());
            }
        }

        if (!connectVirtualDevice(virtualDevice)) {
            qCritical() << "Could not connect to virtual CAN device after multiple attempts";
            continue;
        }

        QObject::connect(device, &QCanBusDevice::framesReceived, [device, virtualDevice]() {
            processReceivedFrames(device, virtualDevice);
        });
        QObject::connect(virtualDevice, &QCanBusDevice::framesReceived, [virtualDevice, device]() {
            processReceivedFrames(virtualDevice, device);
        });

        devices.append(device);
        virtualDevices.append(virtualDevice);

        if (device->state() == QCanBusDevice::ConnectedState) {
            qInfo() << "Successfully connected physical CAN device" << config.physicalInterface;
        }

        if (virtualDevice->state() == QCanBusDevice::ConnectedState) {
            qInfo() << "Successfully connected virtual CAN device" << virtualDeviceName;
        }
    }

    return app.exec();
}

[zeroflagnet]

additional i using Qt.6.8.2

[aha_1980]

Hi @zeroflagnet,

that's a really interesting setup, much more I had originally in mind when creating the virtual CAN plugin.

Unfortunately, the server is indeed limited to two busses, which can be changed but requires to recompile at least the plugin.

What could work is: use QCanBus::instance()->createDevice("virtualcan", "tcp://127.0.0.1:35468/can0"); for the first two busses and use another TCP port (e.g. 35469) for the third. That should start two servers with two resp. one CAN bus attached.

Regards

[1] https://doc.qt.io/qt-6/qtserialbus-virtualcan-overview.html

[zeroflagnet]

Hi @aha_1980
Thanks for your reply.

In my Setup i try to create a second Server on localhost. Qt trys to create and start it, but i think this part of the Libery will deny this try (even If the Server on localhost has an different Port, as done via the Parameters ):

if (!m_server->listen(QHostAddress::LocalHost, port)) {
    qCInfo(QT_CANBUS_PLUGINS_VIRTUALCAN,
           "Server [%p] could not be started, port %d is already in use.", this, port);
    m_server->deleteLater();
    m_server = nullptr;
    return;

I will add tomorrow the qserialbus Plugin debug Messages.
But as far as i remember, the Server was created but then closed instandly again with the qdebug Message above.

Do you have an good Tutorial for compiling only one Plugin?

Where ther resons (to much load in the Connection/Network, instabile Connections/to big latency) for constrainig it to 2?

Thanks in advance

[aha_1980]

Hi @zeroflagnet,

please test again, it should work. I've just started canbusutil twice on Linux: ./canbusutil virtualcan tcp://127.0.0.1:<port>/can0 -l and got the following output when connecting to the two servers via the can example:

./canbusutil virtualcan tcp://127.0.0.1:35468/can0 -l
qt.canbus.plugins.virtualcan: Server [0x7fdd9e22f520] started and listening on port 35468.
qt.canbus.plugins.virtualcan: Server [0x7fdd9e22f520] client connected.
qt.canbus.plugins.virtualcan: Client [0x5577461f6c00] socket connected.
qt.canbus.plugins.virtualcan: Server [0x7fdd9e22f520] client connected.
     456   [3]  98 76 54
     456   [3]  98 76 54

and

 ./canbusutil virtualcan tcp://127.0.0.1:35469/can0 -l
qt.canbus.plugins.virtualcan: Server [0x7fb97228c520] started and listening on port 35469.
qt.canbus.plugins.virtualcan: Server [0x7fb97228c520] client connected.
qt.canbus.plugins.virtualcan: Client [0x56534f1c1c00] socket connected.
qt.canbus.plugins.virtualcan: Server [0x7fb97228c520] client connected.
     123   [3]  45 67 89
     123   [3]  45 67 89

Do you have an good Tutorial for compiling only one Plugin?

You can just get the qtserialbus source [1] for your Qt version, open the CMakeLists.txt in QtCreator, assign the correct Kit and compile the module. Afterwards, copy the new plugin DLL into your Qt installation.

Where ther resons (to much load in the Connection/Network, instabile Connections/to big latency) for constrainig it to 2?

My goal was to have an universal virtual CAN plugin, and did not think it would be used in production.

Now that you ask for it, I've created a patch to change that: https://codereview.qt-project.org/c/qt/qtserialbus/+/632813

Are 10 channels enough for you?

Regards

[1] for example, from git: git://code.qt.io/qt/qtserialbus.git