Determine media file format based on codec

Saviz

I am currently experimenting with the new Qt Multimedia module version 6.6.1 on a 64-bit Windows 10 platform. To gather the list of supported audio and video codecs, I am utilizing the QMediaFormat::supportedAudioCodecs() and QMediaFormat::supportedVideoCodecs() methods, respectively with Conversion mode set to Encoding, because I intend to use it in conjunction with a QMediaRecorder to record audio or video or even both. The intention is to populate a QComboBox with these codec options for the user to select.

The challenge that I am facing is as follows:

It seems that there is no direct method in the QMediaFormat class for the purpose of setting the file format(e.g, mp4, mp3, ...) based on the codecs passed to it. My question is whether there is a method within this class that I may be overlooking, or if there is an alternative approach to ascertain the correct file format corresponding to the selected codec. Any guidance on this matter would be greatly appreciated.

ChrisW67

@Saviz The example does not come copy-n-paste from the docs, it comes from reading the docs.

This will list all the combinations that are possible on the running platform concerned:

#include <QCoreApplication>
#include <QDebug>
#include <QMediaFormat>


int main(int argc, char **argv) {
        QCoreApplication app(argc, argv);

        const QList<QMediaFormat::FileFormat> containers = { 
                        QMediaFormat::WMA, QMediaFormat::AAC, QMediaFormat::Matroska,
                        QMediaFormat::WMV, QMediaFormat::MP3, QMediaFormat::Wave,
                        QMediaFormat::Ogg, QMediaFormat::MPEG4, QMediaFormat::AVI,
                        QMediaFormat::QuickTime, QMediaFormat::WebM, QMediaFormat::Mpeg4Audio,
                        QMediaFormat::FLAC 
                };
                const QList<QMediaFormat::AudioCodec> audioCodecs = {
                        QMediaFormat::AudioCodec::WMA, QMediaFormat::AudioCodec::AC3,
                        QMediaFormat::AudioCodec::AAC, QMediaFormat::AudioCodec::ALAC,
                        QMediaFormat::AudioCodec::DolbyTrueHD, QMediaFormat::AudioCodec::EAC3,
                        QMediaFormat::AudioCodec::MP3, QMediaFormat::AudioCodec::Wave,
                        QMediaFormat::AudioCodec::Vorbis, QMediaFormat::AudioCodec::FLAC,
                        QMediaFormat::AudioCodec::Opus, QMediaFormat::AudioCodec::Unspecified
                };
                const QList<QMediaFormat::VideoCodec> videoCodecs = {
                        QMediaFormat::VideoCodec::VP8, QMediaFormat::VideoCodec::MPEG2,
                        QMediaFormat::VideoCodec::MPEG1, QMediaFormat::VideoCodec::WMV,
                        QMediaFormat::VideoCodec::H265, QMediaFormat::VideoCodec::H264,
                        QMediaFormat::VideoCodec::MPEG4, QMediaFormat::VideoCodec::AV1,
                        QMediaFormat::VideoCodec::MotionJPEG, QMediaFormat::VideoCodec::VP9,
                        QMediaFormat::VideoCodec::Theora, QMediaFormat::VideoCodec::Unspecified
                };
        
        
        for (const auto &c : containers) {
                QMediaFormat format(c);
                for (const auto &v : videoCodecs) {
                        format.setVideoCodec(v);
                        for (const auto &a : audioCodecs) {
                                format.setAudioCodec(a);
                                if (format.isSupported(QMediaFormat::Encode)) {
                                        qDebug() << QMediaFormat::fileFormatName(c) << "("
                                               << QMediaFormat::videoCodecName(v) << ", "
                                               << QMediaFormat::audioCodecName(a) << ")";
                                }
                        }
                }
        }

        return 0;
}

If you only want to support "closely aligned" combinations then only check those.

There's no direct connection between the container format and the name of the file it is contained in. This is done purely by convention. If you want to put an MPEG4 container with audio-only in a file called blah.acc then go right ahead. Most players, I assume, only use this as a hint as to the file content and actually read the content to determine what is sane to do.

ChrisW67

@Saviz The codec for video or audio is not directly related to the container format of a file that might contain some of these encoded streams. Many of the supported containers (i.e. QMediaFormat::FileFormat) can contain audio, video, or other content that has be encoded in any of a range of formats (i.e codecs). There are common assumptions made on the basis of file extensions, but you cannot really know what codecs are present in a container file until you read it or query the library to see what it can manage.

The docs imply that something like this should tell you whether a combination of video and audio codecs would be acceptable to the Qt library code for a given target container:

QMediaFormat format(QMediaFormat::Matroska);
format.setVideoCodec(QMediaFormat::VideoCodec::H264);
format.setAudioCodec(QMediaFormat::AudioCodec::AAC);;
if (format.isSupported(QMediaFormat::Encoding)) {
  // Happy, happy, joy, joy
}

So, you should be able to iterate over the container types for the video/audio options selected to see what should go in the format list.

Saviz

@ChrisW67 Thank you for the reply. But, there are problems.

@ChrisW67 said in Determine media file format based on codec:

Many of the supported containers (i.e. QMediaFormat::FileFormat) can contain audio, video, or other content that has be encoded in any of a range of formats (i.e codecs).

I acknowledge that a single file container type has the capability to accommodate various codec variations. However, my preference is for the container to closely align with the chosen codec. For instance, suppose a user opts for the AAC codec for audio and intends to record solely an audio file. While it is technically possible to store the codec in an .mp4 file featuring a black screen and audio only, the more intuitive approach would be to use an .acc file. I aim for the file format to dynamically adjust based on the selected codec to optimize compatibility. As another example, if a user selects the QMediaFormat::VideoCodec::MPEG4 as the video codec, even though the container file format is set to QMediaFormat::Matroska for an .mkv file, it would be preferable for it to be stored in a format suitable for an .mp4 file, given that this is the more natural assumption

@ChrisW67 said in Determine media file format based on codec:

QMediaFormat format(QMediaFormat::Matroska);
format.setVideoCodec(QMediaFormat::VideoCodec::H264);
format.setAudioCodec(QMediaFormat::AudioCodec::AAC);;
if (format.isSupported(QMediaFormat::Encoding)) {
// Happy, happy, joy, joy
}

I don't think the QMediaFormat::isSupported(QMediaFormat::ConversionMode mode) method is going to help me in this case since the QMediaFormat::supportedAudioCodecs() and QMediaFormat::supportedVideoCodecs() already check if the codecs are supported..

And one more question, where did you find this:

@ChrisW67 said in Determine media file format based on codec:

The docs imply that something like this should tell you whether a combination of video and audio codecs would be acceptable to the Qt library code for a given target container:

I am looking at the docs for 6.6.1 and cannot find this example anywhere: https://doc.qt.io/qt-6/qmediaformat.html#isSupported

ChrisW67

@Saviz The example does not come copy-n-paste from the docs, it comes from reading the docs.

This will list all the combinations that are possible on the running platform concerned:

#include <QCoreApplication>
#include <QDebug>
#include <QMediaFormat>


int main(int argc, char **argv) {
        QCoreApplication app(argc, argv);

        const QList<QMediaFormat::FileFormat> containers = { 
                        QMediaFormat::WMA, QMediaFormat::AAC, QMediaFormat::Matroska,
                        QMediaFormat::WMV, QMediaFormat::MP3, QMediaFormat::Wave,
                        QMediaFormat::Ogg, QMediaFormat::MPEG4, QMediaFormat::AVI,
                        QMediaFormat::QuickTime, QMediaFormat::WebM, QMediaFormat::Mpeg4Audio,
                        QMediaFormat::FLAC 
                };
                const QList<QMediaFormat::AudioCodec> audioCodecs = {
                        QMediaFormat::AudioCodec::WMA, QMediaFormat::AudioCodec::AC3,
                        QMediaFormat::AudioCodec::AAC, QMediaFormat::AudioCodec::ALAC,
                        QMediaFormat::AudioCodec::DolbyTrueHD, QMediaFormat::AudioCodec::EAC3,
                        QMediaFormat::AudioCodec::MP3, QMediaFormat::AudioCodec::Wave,
                        QMediaFormat::AudioCodec::Vorbis, QMediaFormat::AudioCodec::FLAC,
                        QMediaFormat::AudioCodec::Opus, QMediaFormat::AudioCodec::Unspecified
                };
                const QList<QMediaFormat::VideoCodec> videoCodecs = {
                        QMediaFormat::VideoCodec::VP8, QMediaFormat::VideoCodec::MPEG2,
                        QMediaFormat::VideoCodec::MPEG1, QMediaFormat::VideoCodec::WMV,
                        QMediaFormat::VideoCodec::H265, QMediaFormat::VideoCodec::H264,
                        QMediaFormat::VideoCodec::MPEG4, QMediaFormat::VideoCodec::AV1,
                        QMediaFormat::VideoCodec::MotionJPEG, QMediaFormat::VideoCodec::VP9,
                        QMediaFormat::VideoCodec::Theora, QMediaFormat::VideoCodec::Unspecified
                };
        
        
        for (const auto &c : containers) {
                QMediaFormat format(c);
                for (const auto &v : videoCodecs) {
                        format.setVideoCodec(v);
                        for (const auto &a : audioCodecs) {
                                format.setAudioCodec(a);
                                if (format.isSupported(QMediaFormat::Encode)) {
                                        qDebug() << QMediaFormat::fileFormatName(c) << "("
                                               << QMediaFormat::videoCodecName(v) << ", "
                                               << QMediaFormat::audioCodecName(a) << ")";
                                }
                        }
                }
        }

        return 0;
}

If you only want to support "closely aligned" combinations then only check those.

There's no direct connection between the container format and the name of the file it is contained in. This is done purely by convention. If you want to put an MPEG4 container with audio-only in a file called blah.acc then go right ahead. Most players, I assume, only use this as a hint as to the file content and actually read the content to determine what is sane to do.