Reading a tab delimited file into a vector of doubles.

[Stephen]

Hello,

I am trying to write a program that will read a text file of numbers
from a data acquisition system into three vectors. The file produced
looks like this:

+0.0000000e+0 +2.7645134e+1 +2.7745625e+1

+0.4100041e-1 +2.7637787e+1 +2.7731047e+1

+0.0820008e+0 +2.7645134e+1 +2.7750483e+1
...

Once I open the file I try to collect the numbers like this:

vector<double> timeVec;
vector<double> ch1Vec;
vector<double> ch2Vec;
double n = 0;

while ( !inFile.eof() ) {
inFile >> n;
timeVec.push_back(n);
inFile >> n;
ch1Vec.push_back(n);
inFile >> n;
ch2Vec.push_back(n);
}

However, this gives me errors such as "*** glibc detected *** ./
response.o: double free or corruption" or fragmentation error. I have
not been able to find a way to read these numbers that works.

Thanks

[Neil Butterworth]

Post by Stephen
I am trying to write a program that will read a text file of numbers
from a data acquisition system into three vectors. The file produced
+0.0000000e+0 +2.7645134e+1 +2.7745625e+1
+0.4100041e-1 +2.7637787e+1 +2.7731047e+1
+0.0820008e+0 +2.7645134e+1 +2.7750483e+1
vector<double> timeVec;
vector<double> ch1Vec;
vector<double> ch2Vec;
double n = 0;
while ( !inFile.eof() ) {
inFile >> n;
timeVec.push_back(n);
inFile >> n;
ch1Vec.push_back(n);
inFile >> n;
ch2Vec.push_back(n);
}
However, this gives me errors such as "*** glibc detected *** ./
response.o: double free or corruption" or fragmentation error. I have
not been able to find a way to read these numbers that works.
while ( !inFile.eof() ) {

will not work because the eof bit is only set _after_ input, so your
logic results in using a double which contains rubbish. The
followingcoe works as expected:

#include <vector>
#include <iostream>
using namespace std;

vector <double> timeVec;
vector <double> ch1Vec;
vector <double> ch2Vec;

void Dump( ostream & os, const vector <double> & dv ) {
for ( unsigned int i = 0; i < dv.size(); i++ ) {
os << "[" << dv[i] << "]" ;
}
os << endl;
}

bool ReadLine( istream & is ) {
double t, c1, c2;
bool ok = (is >> t) && (is >>c1) && (is >> c2 );
if ( ok ) {
timeVec.push_back( t );
ch1Vec.push_back( c1 );
ch2Vec.push_back( c2 );
}
return ok;
}

int main() {
while ( ReadLine( cin ) ) {
}
Dump( cout, timeVec );
}

Neil Butterworth

[Nikola Smiljanić]

Post by Stephen
Hello,
I am trying to write a program that will read a text file of numbers
from a data acquisition system into three vectors. The file produced
+0.0000000e+0 +2.7645134e+1 +2.7745625e+1
+0.4100041e-1 +2.7637787e+1 +2.7731047e+1
+0.0820008e+0 +2.7645134e+1 +2.7750483e+1
...
vector<double> timeVec;
vector<double> ch1Vec;
vector<double> ch2Vec;
double n = 0;
while ( !inFile.eof() ) {
inFile >> n;
timeVec.push_back(n);
inFile >> n;
ch1Vec.push_back(n);
inFile >> n;
ch2Vec.push_back(n);
}
However, this gives me errors such as "*** glibc detected *** ./
response.o: double free or corruption" or fragmentation error. I have
not been able to find a way to read these numbers that works.

{ edits: sig & banner removed. please don't quote extraneous material. -mod }

This code works on MSVC9, the file contains values that you gave.

std::ifstream file("file.txt");
std::istream_iterator<double> start(file), end;
std::vector<double> vec(start, end);

[Frank Birbacher]

Hi!

Post by Nikola Smiljanić
This code works on MSVC9, the file contains values that you gave.
std::ifstream file("file.txt");
std::istream_iterator<double> start(file), end;
std::vector<double> vec(start, end);

So, what about the three vectors? I guess they are just "parallel
arrays", meaning the values from three different vectors at the same
index are logically together. In this case the following might be a more
usable solution:

#include <fstream>
#include <istream>
#include <iterator>
#include <vector>

struct Entry {
double first, second, third;
};

// overload operator >> to read in "Entry"s from a stream
std::istream& operator >> (std::istream& stream, Entry& entry)
{ return stream >> entry.first >> entry.second >> entry.third; }

int main()
{
std::ifstream file("file.txt");
std::istream_iterator<Entry> start(file), end;
std::vector<Entry> entries(start, end);

//use entries here
}

This avoids having vectors of different length.

Frank

[Michael DOUBEZ]

Post by Stephen
I am trying to write a program that will read a text file of numbers
from a data acquisition system into three vectors. The file produced
+0.0000000e+0 +2.7645134e+1 +2.7745625e+1
+0.4100041e-1 +2.7637787e+1 +2.7731047e+1
+0.0820008e+0 +2.7645134e+1 +2.7750483e+1
...
vector<double> timeVec;
vector<double> ch1Vec;
vector<double> ch2Vec;
double n = 0;
while ( !inFile.eof() ) {

As you are doing it, you may have an infinite loop if your file is
corrupted: if fail bit is raised at one point, you won't be able to read
and you will never reach the end of file.

If you are pretty sure of your file, you may want to activate exceptions
upon read failure.

inFile.exceptions( ifstream::failbit | ifstream::badbit );


Otherwise, you should check the result of input:
if(nFile>>n)xxx.push_back(n);

Post by Stephen
inFile >> n;
timeVec.push_back(n);
inFile >> n;
ch1Vec.push_back(n);
inFile >> n;
ch2Vec.push_back(n);

Personally, I'd rather use three temporaries to express I am reading a line:
double time,ch1,ch2;
if(inFile>>time>>ch1>>ch2)
{
//insert value
}

Post by Stephen
}

Finally, note that this kind of parsing is made as if all data were on
one line only which means that is the format change or is corrupted
(more or less entries per line or missing data), this is not detected
and may even go unnoticed if the total number of data entries is a
multiple of three.

One way to add line semantic is:

//get line
std::string line;
while(std::getline(inFile,linestr,'\n'))
{ //could read line - get data
double time,ch1,ch2;
//interpret line
std::istringstream is(line);
if(!is>>time>>ch1>>ch2)
{ //handle error
return false;
}

//insert values
//note: if one insertion fails, vector sizes may be different
timeVec.push_back(time);
ch1Vec.push_back(ch1);
ch2Vec.push_back(ch2);
}

Post by Stephen
However, this gives me errors such as "*** glibc detected *** ./
response.o: double free or corruption" or fragmentation error.

I doubt this is related to the code you posted. This usually happen when
you have an object destructed twice or when tour toolchain is not
correctly installed (or corrupted).

If you are sure, you are right, you can disable the warning with:
export MALLOC_CHECK_=0

Post by Stephen
I have not been able to find a way to read these numbers that works.

[Bart van Ingen Schenau]

Post by Michael DOUBEZ

Post by Stephen
I am trying to write a program that will read a text file of numbers
from a data acquisition system into three vectors. The file produced
+0.0000000e+0 +2.7645134e+1 +2.7745625e+1
+0.4100041e-1 +2.7637787e+1 +2.7731047e+1
+0.0820008e+0 +2.7645134e+1 +2.7750483e+1
...
vector<double> timeVec;
vector<double> ch1Vec;
vector<double> ch2Vec;
double n = 0;
while ( !inFile.eof() ) {

As you are doing it, you may have an infinite loop if your file is
corrupted: if fail bit is raised at one point, you won't be able to
read and you will never reach the end of file.
If you are pretty sure of your file, you may want to activate
exceptions upon read failure.
inFile.exceptions( ifstream::failbit | ifstream::badbit );

Even if you are really confident about your file, enabling exceptions is
very likely to give false-positive results.
For example, if your file ends with a whitespace character (like a
newline), then most strategies for reading the file to the end will set
*both* ios_base::failbit and ios_base::eofbit. With the exceptions
enabled, reaching EOF will also result in an exception being thrown.

<snip - good advise on reading from a formatted file>

Bart v Ingen Schenau