Clicking sounds when playing Raw Audio.

scottnat · 22 Dec 2017, 04:10

Hi all, I am writing a simple software that plays audio data from my microphone array with some C++ API. I have formatted the QAudioOutput and have started writing data to the QIODevice, of about period size byte chunks.

Problem 1: Audio is heard but its in short bursts of "tsss"..."tsss"..."tsss". Could anyone advise me on this problem?

My QAudioOutput object->bytesFree(), indicate that only 3 bytes are used each round. The audio data is obtained from the API function, "CcmRead..()", and it comes in blocks (64samples) of 32bit integers. I have to stream this 32bit integers into a QByteArray, and let the QIODevice write this QByteArray.

Question 1: If my QAudioFormat::sampleType is 32bits, would writing 32 bit integers to a QByteArray, be read as 4 bytes as one sample?

Question 2: It appears to me that notify interval increases the rate of "tss.." sounds being played, and from the docs, it says "Sets the interval for notify() signal to be emitted. This is based on the ms of audio data processed". How do I determine what notify interval ms to use?

Header

#ifndef MAINWINDOW_H
#define MAINWINDOW_H

#include <QMainWindow>
#include <QAudioOutput>
#include <QByteArray>
#include <QIODevice>

namespace Ui {
class MainWindow;
}

class MainWindow : public QMainWindow
{
    Q_OBJECT

public:
    explicit MainWindow(QWidget *parent = 0);
    ~MainWindow();
    void setupVariables();
    bool validateVars();
    bool open();
    bool close();
    bool setDomain();
    bool setSampleRate();
    bool setBlockTakeSkip();
    void setTime_BTS();
    void setFreq_BTS();
    void init_audio_format();
    void init_audio_output();
    void read_frames();
    void write_frames();

private slots:
    void on_startButton_clicked();
    void slot_writedata();

private:
    Ui::MainWindow *ui;
    QAudioOutput* audio;
    QByteArray* m_data;
    QByteArray m_bytearray;
    QIODevice* m_audiodevice;
    QAudioFormat m_format;
    QAudioDeviceInfo m_device;
    //setupFile *myObject;

};

#endif // MAINWINDOW_H

Source

#include "mainwindow.h"
#include "ui_mainwindow.h"
#include "CcmApi.h"
#include "CcmErrors.h"
#include <QDebug>
#include <QAudioFormat>
#include <QAudioOutput>
#include <QAudioDeviceInfo>
#include <QDataStream>


QString mDomain;
int mSampleRate, mBlockSize, mTake, mSkip;
bool isOpen = false;


MainWindow::MainWindow(QWidget *parent) :
    QMainWindow(parent),
    ui(new Ui::MainWindow)
{
    ui->setupUi(this);
    //setting up UI buttons/controls/user input
    connect(ui->comboBox,SIGNAL(currentIndexChanged(int)),ui->stackedWidget,SLOT(setCurrentIndex(int))); //to change page
}


MainWindow::~MainWindow()
{
    if(isOpen){
        close();
        qDebug()<<"Closed confirm";
    }
    delete ui;
}


void MainWindow::on_startButton_clicked()
{
    if(validateVars()&&open()&&setDomain()&&setSampleRate()&&setBlockTakeSkip()){
        qDebug()<<"Variables validated, domain, sample rate and take/skip set.";
        init_audio_format();
        init_audio_output();
        read_frames();
    }
}

void MainWindow::init_audio_format()
{
    qDebug()<<"Initialising Audio 1";

    m_format.setSampleRate(10000);
    m_format.setByteOrder(QAudioFormat::LittleEndian);
    m_format.setChannelCount(1);
    m_format.setCodec("audio/pcm");
    m_format.setSampleSize(32);
    m_format.setSampleType(QAudioFormat::UnSignedInt);

    m_device = QAudioDeviceInfo::defaultOutputDevice();
    QAudioDeviceInfo info(m_device);
    if (!info.isFormatSupported(m_format)) {
        qWarning() << "Raw audio format not supported by backend, cannot play audio.";
        return;
    }
    qDebug()<<"Initialising Audio 2";

}

void MainWindow::init_audio_output()
{
    audio = new QAudioOutput(m_device,m_format,this);
    audio->setBufferSize(32768);
    audio->setNotifyInterval(40);
    m_audiodevice = audio->start();
    connect(audio,SIGNAL(notify()),this,SLOT(slot_writedata()));
}

void MainWindow::slot_writedata()
{
    read_frames();
    qDebug()<<"The bytes free are:" <<audio->bytesFree();
    write_frames();
}

void MainWindow::read_frames()
{
    int* buffer;
    int frameSize, byteCount=0;
    DWORD tdFrames, fdFrames, micsNum;
    CcmGetMicrophoneCount(&micsNum);
    quint32 value;

    frameSize = (int) micsNum * mBlockSize;

    qDebug()<<"The period size: "<< audio->periodSize();

    if(CcmStartInput()!=CCM_ERR_NONE){
        qDebug()<<"CcmStartInput Error";
    }

    QDataStream mstream(&m_bytearray,QIODevice::ReadWrite);
    int periodBytes = audio->periodSize();
    buffer = new int[frameSize];
    while( byteCount<periodBytes ){
        CcmReadFrames(buffer,NULL,frameSize,0,&tdFrames,&fdFrames,NULL,CCM_WAIT);
        if(tdFrames==0){
            break;
        }
        byteCount+=(4*mBlockSize);

        for(int x=0;x<mBlockSize;x++){
            value = (quint32) buffer[x];
            mstream << value;
        }
    }
    if(CcmStopInput()!=CCM_ERR_NONE){
        qDebug()<<"CcmStopInput Error";
    }
    qDebug()<<"The length of the byte array: "<< m_bytearray.length();
    
    //    delete [] buffer;
}

void MainWindow::write_frames()
{
    const qint64 len = audio->periodSize();
    m_audiodevice->write(m_bytearray.data(),m_bytearray.length());
    qDebug()<<"Write attempt";

}

bool MainWindow::validateVars(){

    //initialise variables domain,samplerate,blocksize,take,skip
    mDomain = ui->domainBox->currentText();
    mSampleRate = ui->samplerateBox->currentText().toInt();
    mBlockSize = ui->blocksizeBox->currentText().toInt();

    int tStr, sStr;
    bool ok1, ok2; //check if strings are both integers
    tStr = ui->takeframes_Edit->text().toInt(&ok1,10);
    sStr = ui->skipframes_Edit->text().toInt(&ok2,10);

    if ((tStr>0) && (sStr>=0) && ok1 && ok2){
        mTake = tStr;
        mSkip = sStr;
        qDebug()<<"Validate ok";

        qDebug()<<mDomain;
        qDebug()<<mSampleRate;
        qDebug()<<mBlockSize;
        qDebug()<<mTake;
        qDebug()<<mSkip;


       return true;
    }
    else{return false;};
}

bool MainWindow::open()
{
    DWORD openVal, err;

    char er[100];
    openVal = CcmOpen();

    if(openVal != CCM_ERR_NONE){ //error opening
        CcmGetLastErrorString(er,100);
        err = CcmClose();
        if (err != CCM_ERR_NONE){ //close it first
            CcmGetLastErrorString(er,100);
            qDebug()<< er;
        }
        else{ //close successful
            isOpen = false;
            open(); //retry opening again;
        }
    }
    else{
        isOpen = true;
        qDebug()<<"Open no problem";
        return true;
    }
return false;
}

bool MainWindow::close()
{
   if (isOpen){
       if(CcmClose()!=CCM_ERR_NONE){
            return false;
       }
   }
   return true;
}

bool MainWindow::setDomain()
{

    if (mDomain=="Time"){
        qDebug()<<"Time";

        CcmSetTimeDomainEnabled(CCM_ENABLED);
        CcmSetFrequencyDomainEnabled(CCM_DISABLED);
    }
    else{
        qDebug()<<"Freq";
        CcmSetTimeDomainEnabled(CCM_DISABLED);
        CcmSetFrequencyDomainEnabled(CCM_ENABLED);
    }

    return true;
}


//Domain independent
bool MainWindow::setSampleRate()
{   int index;
    index = ui->samplerateBox->currentIndex();
    qDebug()<<index;

    switch(ui->samplerateBox->currentIndex()){

    case 0:
        CcmSetSampleRate(CCM_SAMPLERATE_10000);
        break;
    case 1:
        CcmSetSampleRate(CCM_SAMPLERATE_12500);
        break;
    case 2:
        CcmSetSampleRate(CCM_SAMPLERATE_20000);
        break;
    case 3:
        CcmSetSampleRate(CCM_SAMPLERATE_25000);
        break;
    case 4:
        CcmSetSampleRate(CCM_SAMPLERATE_40000);
        break;
    case 5:
        CcmSetSampleRate(CCM_SAMPLERATE_50000);
        break;

    default:
        CcmSetSampleRate(CCM_SAMPLERATE_10000);
    }

    return true;
}

bool MainWindow::setBlockTakeSkip()
{
    if(mDomain=="Time"){
        setTime_BTS();
    }
    else{
        setFreq_BTS();
    }
    return true;
}


//for time domain block take skip settings
void MainWindow::setTime_BTS()
{
    int choice;
    choice = ui->blocksizeBox->currentIndex();
    switch(choice){
    case 0: CcmSetTimeDomainBlockSize(CCM_BLOCKSIZE_64);
        break;
    case 1: CcmSetTimeDomainBlockSize(CCM_BLOCKSIZE_128);
        break;
    case 2: CcmSetTimeDomainBlockSize(CCM_BLOCKSIZE_256);
        break;
    case 3: CcmSetTimeDomainBlockSize(CCM_BLOCKSIZE_512);
        break;
    case 4: CcmSetTimeDomainBlockSize(CCM_BLOCKSIZE_1024);
        break;
    case 5: CcmSetTimeDomainBlockSize(CCM_BLOCKSIZE_2048);
        break;
    default:
        CcmSetTimeDomainBlockSize(CCM_BLOCKSIZE_64);
    }

    CcmSetTimeDomainSkipFrames((DWORD)mSkip);
    CcmSetTimeDomainTakeFrames((DWORD)mTake);
}

//for freq domain block take skip settings
void MainWindow::setFreq_BTS()
{
    int choice;
    choice = ui->blocksizeBox->currentIndex();
    switch(choice){
    case 0: CcmSetFrequencyDomainBlockSize(CCM_BLOCKSIZE_64);
        break;
    case 1: CcmSetFrequencyDomainBlockSize(CCM_BLOCKSIZE_128);
        break;
    case 2: CcmSetFrequencyDomainBlockSize(CCM_BLOCKSIZE_256);
        break;
    case 3: CcmSetFrequencyDomainBlockSize(CCM_BLOCKSIZE_512);
        break;
    case 4: CcmSetFrequencyDomainBlockSize(CCM_BLOCKSIZE_1024);
        break;
    case 5: CcmSetFrequencyDomainBlockSize(CCM_BLOCKSIZE_2048);
        break;
    default:
        CcmSetFrequencyDomainBlockSize(CCM_BLOCKSIZE_64);
    }

    CcmSetFrequencyDomainSkipFrames((DWORD)mSkip);
    CcmSetFrequencyDomainTakeFrames((DWORD)mTake);

}

Tirupathi Korla · 22 Dec 2017, 06:05

@scottnat
Make sure your header format is correct...

scottnat · 22 Dec 2017, 10:23

@Tirupathi-Korla Hi, thanks for the reply. What do you mean Header format is correct?

Tirupathi Korla · wrote on 22 Dec 2017, 10:23

@scottnat
Channels
Sample rate etc...

SGaist · 23 Dec 2017, 05:53

Hi,

Since it seems you are getting audio from a microphone, then why not use QAudioInput ?

scottnat · 24 Dec 2017, 04:27

@SGaist Hi, because I have an API function that stores the recorded data from the microphone into an array. I don't need to record the audio on my own. What i am doing is streaming the data from the array into a QByteArray, and writing that QByteArray to the QIODevice of QAudioOutput.

Tirupathi Korla · 25 Dec 2017, 09:49

@scottnat

Is that recorded file playing properly when it is played in other media players??
Save the buffer into a file and try to play..
Can u share a sample audio file... or first 128 bytes of buffer...

scottnat · wrote on 25 Dec 2017, 09:49

@Tirupathi-Korla

Basically this is the code that I have obtained from my C++ API and saved to a file.
[0_1514195321157_myfile_fs40kHz.txt](Uploading 100%)

I played with Audacity. Imported Raw Audio played at 64bit float. It sounds muffled.

int* buffer;
DWORD tdFrames, fdFrames;
 int framesize, counter=0;
 DWORD blocksize;
  CcmGetTimeDomainBlockSize(&blocksize);
  framesize = 40 * (int)blocksize;
  buffer = new int[framesize];

  sourceFile = new QFile("myfile_fs40kHz.txt");
  sourceFile->open(QIODevice::ReadWrite);
  QDataStream mDstream(sourceFile);

while(counter<200){
        CcmReadFrames(buffer,NULL,framesize,0,&tdFrames,&fdFrames,NULL,CCM_WAIT);
        for(int x=0; x<(int) blocksize; x++){
            mDstream << static_cast<quint64>(buffer[x]) << endl;
        }
    counter++;
 }
    sourceFile->close();

scottnat · 26 Dec 2017, 07:02

@Tirupathi-Korla
https://drive.google.com/open?id=1VDOrLLduLW5kWpPHVZMzzYd2BYqnp5_o

Tirupathi Korla · wrote on 26 Dec 2017, 07:02

@scottnat
When i tried to play your audio with sampleRate as 10000 and sampleSize 32, i don't hear any thing.. I changed SampleSize to 8 and i hear a sine wave audio and after few seconds i observed some noise along sine wave audio.. Is it the file you shared??

scottnat · wrote on 30 Dec 2017, 14:57

Hi all, thanks for the assistance. The issue has been fixed. Initially i was attempting to read and write data sequentially to achieve Real Time audio playback. However, this probably caused the issue of buffer underrun. I decided to save all my incoming data into a qbytearray and then buffer qiodevice to write.