reading file line by line crashes
-
Hello everyone,
I've a program that open a file (380 Mo) into a QTextStream and read data line by line with the readLine function.
Recenly a have encoutred a crash problem when reading data and I've tried some manipulation by the crash persists.
Note that every line contains some big data and lines ends with \n\r.Below my code :
QFile srfile(mScriptFileName); if (!srfile.open(QIODevice::ReadOnly | QIODevice::Text)) { mExistMajData = false; mIsError = true; return ; } QTextStream in(&srfile); while (!in.atEnd() && !mIsError) { qApp->processEvents(QEventLoop::WaitForMoreEvents); //lecture des données lignes par ligne try { line_data = in.readLine(); // it crashes here (eg. PJ) record_data = line_data.split("<|>"); qDebug() << "Fin lecture d'une ligne n°" << QString::number(++nb_cmd) << record_data[3] << QTime::currentTime().toString("hh:mm") << endl ; } catch ( const std::bad_alloc & ) { std::cerr << "Erreur : mémoire insuffisante.\n"; } catch ( const std::out_of_range & ) { std::cerr << "Erreur : débordement de mémoire.\n"; } catch ( const std::exception & e ) { std::cerr << e.what(); }The call stack indicated the following :
msvcr80d.dll!memcpy(unsigned char * dst=0x28446f80, unsigned char * src=0x01b26ffa, unsigned long count=32768) Ligne 188 Asm QtCored4.dll!QString::append(const QString & str={...}) Ligne 1244 + 0x2d octets C++ QtCored4.dll!QString::operator+=(const QString & s={...}) Ligne 225 + 0x13 octets C++ QtCored4.dll!QTextStreamPrivate::fillReadBuffer(__int64 maxBytes=-1) Ligne 568 + 0x5b octets C++ QtCored4.dll!QTextStreamPrivate::scan(const QChar * * ptr=0x0014a42c, int * length=0x0014a434, int maxlen=0, QTextStreamPrivate::TokenDelimiter delimiter=EndOfLine) Ligne 730 + 0x2a octets C++ QtCored4.dll!QTextStream::readLine(__int64 maxlen=0) Ligne 1588 + 0x16 octets C++ > TheriaqueUpdater.exe!TheriaUpdater::NewUpdateBase() Ligne 3377 + 0x14 octets C++ TheriaqueUpdater.exe!TheriaUpdater::start() Ligne 1479 C++ TheriaqueUpdater.exe!TheriaUpdater::Update() Ligne 1199 C++ TheriaqueUpdater.exe!TheriaUpdater::qt_metacall(QMetaObject::Call _c=InvokeMetaMethod, int _id=8, void * * _a=0x0014b154) Ligne 96 + 0x8 octets C++ QtCored4.dll!QMetaObject::activate(QObject * sender=0x01aa4888, int from_signal_index=29, int to_signal_index=30, void * * argv=0x0014b154) Ligne 3063 + 0x46 octets C++ QtCored4.dll!QMetaObject::activate(QObject * sender=0x01aa4888, const QMetaObject * m=0x657a2374, int from_local_signal_index=2, int to_local_signal_index=3, void * * argv=0x0014b154) Ligne 3142 + 0x15 octets C++ QtGuid4.dll!QAbstractButton::clicked(bool _t1=false) Ligne 180 + 0x17 octets C++ QtGuid4.dll!QAbstractButtonPrivate::emitClicked() Ligne 517 C++ QtGuid4.dll!QAbstractButtonPrivate::click() Ligne 509 C++ QtGuid4.dll!QAbstractButton::mouseReleaseEvent(QMouseEvent * e=0x0014b788) Ligne 1086 C++ QtGuid4.dll!QWidget::event(QEvent * event=0x0014b788) Ligne 6064 C++ QtGuid4.dll!QAbstractButton::event(QEvent * e=0x0014b788) Ligne 1048 C++ QtGuid4.dll!QPushButton::event(QEvent * e=0x0014b788) Ligne 651 C++ QtGuid4.dll!QApplicationPrivate::notify_helper(QObject * receiver=0x01aa4888, QEvent * e=0x0014b788) Ligne 3541 + 0x11 octets C++ QtGuid4.dll!QApplication::notify(QObject * receiver=0x01aa4888, QEvent * e=0x0014b788) Ligne 3240 + 0x2f octets C++ QtCored4.dll!QCoreApplication::notifyInternal(QObject * receiver=0x01aa4888, QEvent * event=0x0014b788) Ligne 516 C++ QtCored4.dll!QCoreApplication::sendSpontaneousEvent(QObject * receiver=0x01aa4888, QEvent * event=0x0014b788) Ligne 188 + 0x38 octets C++I've reinstalled the VC redistribuable. Rebuil all the project. Search if I can free QTextStream buffer but the problem persists
Can anyone tell me what wrong et what I must to do to resolve this. Some
Thanks in advance
@mourad_bilog said in reading file line by line crashes:
crash problem
What kind of crash is it? Out of memory? Or something else?
-
Hello everyone,
I've a program that open a file (380 Mo) into a QTextStream and read data line by line with the readLine function.
Recenly a have encoutred a crash problem when reading data and I've tried some manipulation by the crash persists.
Note that every line contains some big data and lines ends with \n\r.Below my code :
QFile srfile(mScriptFileName); if (!srfile.open(QIODevice::ReadOnly | QIODevice::Text)) { mExistMajData = false; mIsError = true; return ; } QTextStream in(&srfile); while (!in.atEnd() && !mIsError) { qApp->processEvents(QEventLoop::WaitForMoreEvents); //lecture des données lignes par ligne try { line_data = in.readLine(); // it crashes here (eg. PJ) record_data = line_data.split("<|>"); qDebug() << "Fin lecture d'une ligne n°" << QString::number(++nb_cmd) << record_data[3] << QTime::currentTime().toString("hh:mm") << endl ; } catch ( const std::bad_alloc & ) { std::cerr << "Erreur : mémoire insuffisante.\n"; } catch ( const std::out_of_range & ) { std::cerr << "Erreur : débordement de mémoire.\n"; } catch ( const std::exception & e ) { std::cerr << e.what(); }The call stack indicated the following :
msvcr80d.dll!memcpy(unsigned char * dst=0x28446f80, unsigned char * src=0x01b26ffa, unsigned long count=32768) Ligne 188 Asm QtCored4.dll!QString::append(const QString & str={...}) Ligne 1244 + 0x2d octets C++ QtCored4.dll!QString::operator+=(const QString & s={...}) Ligne 225 + 0x13 octets C++ QtCored4.dll!QTextStreamPrivate::fillReadBuffer(__int64 maxBytes=-1) Ligne 568 + 0x5b octets C++ QtCored4.dll!QTextStreamPrivate::scan(const QChar * * ptr=0x0014a42c, int * length=0x0014a434, int maxlen=0, QTextStreamPrivate::TokenDelimiter delimiter=EndOfLine) Ligne 730 + 0x2a octets C++ QtCored4.dll!QTextStream::readLine(__int64 maxlen=0) Ligne 1588 + 0x16 octets C++ > TheriaqueUpdater.exe!TheriaUpdater::NewUpdateBase() Ligne 3377 + 0x14 octets C++ TheriaqueUpdater.exe!TheriaUpdater::start() Ligne 1479 C++ TheriaqueUpdater.exe!TheriaUpdater::Update() Ligne 1199 C++ TheriaqueUpdater.exe!TheriaUpdater::qt_metacall(QMetaObject::Call _c=InvokeMetaMethod, int _id=8, void * * _a=0x0014b154) Ligne 96 + 0x8 octets C++ QtCored4.dll!QMetaObject::activate(QObject * sender=0x01aa4888, int from_signal_index=29, int to_signal_index=30, void * * argv=0x0014b154) Ligne 3063 + 0x46 octets C++ QtCored4.dll!QMetaObject::activate(QObject * sender=0x01aa4888, const QMetaObject * m=0x657a2374, int from_local_signal_index=2, int to_local_signal_index=3, void * * argv=0x0014b154) Ligne 3142 + 0x15 octets C++ QtGuid4.dll!QAbstractButton::clicked(bool _t1=false) Ligne 180 + 0x17 octets C++ QtGuid4.dll!QAbstractButtonPrivate::emitClicked() Ligne 517 C++ QtGuid4.dll!QAbstractButtonPrivate::click() Ligne 509 C++ QtGuid4.dll!QAbstractButton::mouseReleaseEvent(QMouseEvent * e=0x0014b788) Ligne 1086 C++ QtGuid4.dll!QWidget::event(QEvent * event=0x0014b788) Ligne 6064 C++ QtGuid4.dll!QAbstractButton::event(QEvent * e=0x0014b788) Ligne 1048 C++ QtGuid4.dll!QPushButton::event(QEvent * e=0x0014b788) Ligne 651 C++ QtGuid4.dll!QApplicationPrivate::notify_helper(QObject * receiver=0x01aa4888, QEvent * e=0x0014b788) Ligne 3541 + 0x11 octets C++ QtGuid4.dll!QApplication::notify(QObject * receiver=0x01aa4888, QEvent * e=0x0014b788) Ligne 3240 + 0x2f octets C++ QtCored4.dll!QCoreApplication::notifyInternal(QObject * receiver=0x01aa4888, QEvent * event=0x0014b788) Ligne 516 C++ QtCored4.dll!QCoreApplication::sendSpontaneousEvent(QObject * receiver=0x01aa4888, QEvent * event=0x0014b788) Ligne 188 + 0x38 octets C++I've reinstalled the VC redistribuable. Rebuil all the project. Search if I can free QTextStream buffer but the problem persists
Can anyone tell me what wrong et what I must to do to resolve this. Some
Thanks in advance
Even if this doesn't crash your application, you should check if recordData has indeed more than 3 entries!
qDebug() << "Fin lecture d'une ligne n°" << QString::number(++nb_cmd) << record_data[3] << QTime::currentTime().toString("hh:mm") << endl ;
besides that, I see this
qApp->processEvents(QEventLoop::WaitForMoreEvents);
potentially causing serious issues, do not spin the event loop manually!
-
wrote on 11 Mar 2019, 13:42 last edited by
First, Thanks for reply.
@jsulm it's a memory crash. Below the error occured :
Exception de première chance à 0x0fcbd5be (msvcr80d.dll) dans TheriaqueUpdater.exe : 0xC0000005: Violation d'accès lors de l'écriture à l'emplacement 0x2804e000. Exception non gérée à 0x0fcbd5be (msvcr80d.dll) dans TheriaqueUpdater.exe : 0xC0000005: Violation d'accès lors de l'écriture à l'emplacement 0x2804e000.@J-Hilk the crash occus on in.readLine. The recod data has 6 entries and it's shown evey time.
I've commented the eventLoop but the application still crashes.Note that line data lengths can be around 1 milion carachtars.
-
First, Thanks for reply.
@jsulm it's a memory crash. Below the error occured :
Exception de première chance à 0x0fcbd5be (msvcr80d.dll) dans TheriaqueUpdater.exe : 0xC0000005: Violation d'accès lors de l'écriture à l'emplacement 0x2804e000. Exception non gérée à 0x0fcbd5be (msvcr80d.dll) dans TheriaqueUpdater.exe : 0xC0000005: Violation d'accès lors de l'écriture à l'emplacement 0x2804e000.@J-Hilk the crash occus on in.readLine. The recod data has 6 entries and it's shown evey time.
I've commented the eventLoop but the application still crashes.Note that line data lengths can be around 1 milion carachtars.
@mourad_bilog said in reading file line by line crashes:
Note that line data lengths can be around 1 milion carachtars
How long is the line you're reading when the app is crashing?
-
First, Thanks for reply.
@jsulm it's a memory crash. Below the error occured :
Exception de première chance à 0x0fcbd5be (msvcr80d.dll) dans TheriaqueUpdater.exe : 0xC0000005: Violation d'accès lors de l'écriture à l'emplacement 0x2804e000. Exception non gérée à 0x0fcbd5be (msvcr80d.dll) dans TheriaqueUpdater.exe : 0xC0000005: Violation d'accès lors de l'écriture à l'emplacement 0x2804e000.@J-Hilk the crash occus on in.readLine. The recod data has 6 entries and it's shown evey time.
I've commented the eventLoop but the application still crashes.Note that line data lengths can be around 1 milion carachtars.
@mourad_bilog said in reading file line by line crashes:
@J-Hilk the crash occus on in.readLine. The recod data has 6 entries and it's shown evey time.
do not base this solely on faith:
if(record_data.size()> 3) qDebug() << "Fin lecture d'une ligne n°" << QString::number(++nb_cmd) << record_data[3] << QTime::currentTime().toString("hh:mm") << endl ; else qDebug() << "would have crashed"; -
wrote on 11 Mar 2019, 14:35 last edited by
@jsulm said in reading file line by line crashes:
How long is the line you're reading when the app is crashing?
About 1 200 000 charactars.
-
wrote on 11 Mar 2019, 14:36 last edited by
@J-Hilk
I've commented that line but the same occurs. -
@J-Hilk
I've commented that line but the same occurs.@mourad_bilog alight thanks thumbs up
question , is this Qt4?
and is this a multi threaded project, and is do you read and write to that file from different places ? -
@mourad_bilog alight thanks thumbs up
question , is this Qt4?
and is this a multi threaded project, and is do you read and write to that file from different places ?wrote on 11 Mar 2019, 14:44 last edited by@J.Hilk
question , is this Qt4? yes. but the program runs correctly for some years.and is this a multi threaded project, : No, it's one thread.
and is do you read and write to that file from different places : No, I just read from that file. I just open it in "ReadOnly" mode.
-
wrote on 11 Mar 2019, 14:51 last edited by
I've read in a post how has similar problem that he has solved the problem using file.readLine no QTextStream.readLine(). What is the difference between the 2 functions ?
-
-
wrote on 11 Mar 2019, 15:24 last edited by
@Christian-Ehrlicher there's many lines witch everyone exceed 1 milion charactars.
Is this a problem ? That can exceed what QTextStream buffer contains ? -
At least I would consider this - it's a 2.4MB huge QString, should be ok but who knows. I would simply try QFile::readLine(), split the QByteArray and convert to QString as late as possible (if needed at all)
-
wrote on 12 Mar 2019, 11:11 last edited by
Checking Windows events journal, it indicates that msvcr causes this crash, that's true ?
Nom de l’application défaillante Updater.exe, version : 2.7.4.0, horodatage : 0x5a018f88 Nom du module défaillant : MSVCR80.dll, version : 8.0.50727.9445, horodatage : 0x5a7bc74c Code d’exception : 0xc0000005 Décalage d’erreur : 0x0001508e ID du processus défaillant : 0x19a0 Heure de début de l’application défaillante : 0x01d4d8a5bb99dd52 Chemin d’accès du module défaillant: C:\WINDOWS\WinSxS\x86_microsoft.vc80.crt_1fc8b3b9a1e18e3b_8.0.50727.9445_none_d08c58b4442ba54f\MSVCR80.dll ID de rapport : 6c470b7e-a3b2-4584-b404-3d7957545ead Nom complet du package défaillant : ID de l’application relative au package défaillant :I've re-installed Visual redistribuable 2005 but the problem persists. Note that I dont found msvcr80 in the system32 not like other versions msvcr71, msvcr10, etc.
Does this the problem ?
-
Hi,
Did you monitor your application RAM usage ?
-
wrote on 22 Mar 2019, 06:24 last edited by
Finally I changed reading mode to read directly from the data file. It's most faster and just I was care to encode data when reading to keep special charcatars.
Thanks all for your ideas
