binary data buffers: vector<char> or std::string ?

[viki]

What should I prefer for binary data buffers, std::string or
vector<char> ?
Like data buffers that are written to and read from binary files.

Thanks
Viki

[Neil Butterworth]

Post by viki
What should I prefer for binary data buffers, std::string or
vector<char> ?
Like data buffers that are written to and read from binary files.

I would use std::vector, because it has a natural constructor for
creating buffers:

const int BUFSIZE = 1024; // or whatever
std::vector <char> buffer( BUFSIZE );


Neil Butterworth

[vdk]

Post by Neil Butterworth
I would use std::vector, because it has a natural constructor for
const int BUFSIZE = 1024; // or whatever
std::vector <char> buffer( BUFSIZE );

Just be warned that there are compilers (MSVC.NET 2003 for one) that
do not have the std::vector<> template specialized for char (or
similar) thus the constructor call above will ultimately lead to a
loop initializing the vector's components one by one(!).

[Bo Persson]

Post by vdk

Post by Neil Butterworth
I would use std::vector, because it has a natural constructor for
const int BUFSIZE = 1024; // or whatever
std::vector <char> buffer( BUFSIZE );

Just be warned that there are compilers (MSVC.NET 2003 for one) that
do not have the std::vector<> template specialized for char (or
similar) thus the constructor call above will ultimately lead to a
loop initializing the vector's components one by one(!).

This might of course be because there are compilers that are smart
enough to optimize this properly anyway, so that they don't need
specializations for POD types.


Bo Persson

[vdk]

Post by Bo Persson

Post by vdk
Just be warned that there are compilers (MSVC.NET 2003 for one) that
do not have the std::vector<> template specialized for char (or
similar) thus the constructor call above will ultimately lead to a
loop initializing the vector's components one by one(!).

This might of course be because there are compilers that are smart
enough to optimize this properly anyway, so that they don't need
specializations for POD types.

That could be, I never checked the resulting (optimized) code. Still,
I'd wonder why then the std::basic_string<char> specialization uses
std::memset() in its constructor?

Vlado Klimovsky

[red floyd]

Post by viki
What should I prefer for binary data buffers, std::string or
vector<char> ?
Like data buffers that are written to and read from binary files.

Neither. You should prefer vector<unsigned char>

[Hakusa]

Post by red floyd

Post by viki
What should I prefer for binary data buffers, std::string or
vector<char> ?
Like data buffers that are written to and read from binary files.

Neither. You should prefer vector<unsigned char>

Why? And why not std::basic_string<unsigned char>?

[Francis Glassborow]

Post by viki
What should I prefer for binary data buffers, std::string or
vector<char> ?
Like data buffers that are written to and read from binary files.

neither :)

vector<unsigned char>

might do at a stretch if you need an extensible buffer. However I would
normally prefer a buffer to have fixed locality.

[Neil Butterworth]

On Feb 24, 7:01 pm, Francis Glassborow

Post by red floyd
vector<unsigned char>
might do at a stretch if you need an extensible buffer. However I would
normally prefer a buffer to have fixed locality.

How are you going to arrange that in a way that vector <unsigned char>
cannot?

Neil Butterworth

[Francis Glassborow]

Post by Neil Butterworth
On Feb 24, 7:01 pm, Francis Glassborow

Post by red floyd
vector<unsigned char>
might do at a stretch if you need an extensible buffer. However I would
normally prefer a buffer to have fixed locality.

How are you going to arrange that in a way that vector <unsigned char>
cannot?
Neil Butterworth

A vector moves its contents when it expands.

[Jeff Schwab]

Post by Francis Glassborow

Post by Neil Butterworth
On Feb 24, 7:01 pm, Francis Glassborow

Post by red floyd
vector<unsigned char>
might do at a stretch if you need an extensible buffer. However I would
normally prefer a buffer to have fixed locality.

How are you going to arrange that in a way that vector <unsigned char>
cannot?

A vector moves its contents when it expands.

Wouldn't the same be true of a hand-rolled buffer?

[Francis Glassborow]

Post by Jeff Schwab

Post by Francis Glassborow

Post by Neil Butterworth
On Feb 24, 7:01 pm, Francis Glassborow

Post by red floyd
vector<unsigned char>
might do at a stretch if you need an extensible buffer. However I would
normally prefer a buffer to have fixed locality.

How are you going to arrange that in a way that vector <unsigned char>
cannot?

A vector moves its contents when it expands.

Wouldn't the same be true of a hand-rolled buffer?

Perhaps we have a communication problem. I prefer a buffer to have fixed
location and that implies that it needs to be of fixed size (we could
use a deque though).

[Jeff Schwab]

Post by Francis Glassborow

Post by Jeff Schwab

Post by Francis Glassborow

Post by Neil Butterworth
On Feb 24, 7:01 pm, Francis Glassborow

Post by red floyd
vector<unsigned char>
might do at a stretch if you need an extensible buffer. However I would
normally prefer a buffer to have fixed locality.

How are you going to arrange that in a way that vector <unsigned char>
cannot?

A vector moves its contents when it expands.

Wouldn't the same be true of a hand-rolled buffer?

Perhaps we have a communication problem. I prefer a buffer to have fixed
location and that implies that it needs to be of fixed size (we could
use a deque though).

Sorry, then what's the problem with std::vector? I thought your point
was that if it grew, you would lose locality. (I suggested Boost or TR1
array elsethread, but I often use std::vector myself.)

[Francis Glassborow]

Post by Jeff Schwab

Post by Francis Glassborow
Perhaps we have a communication problem. I prefer a buffer to have fixed
location and that implies that it needs to be of fixed size (we could
use a deque though).

Sorry, then what's the problem with std::vector? I thought your point
was that if it grew, you would lose locality.

Yes, but std::deque does not, elements stay where you put them and do
not suddenly get copied to somewhere entirely different.

(I suggested Boost or TR1

Post by Jeff Schwab
array elsethread, but I often use std::vector myself.)

Yes tr1::array is fine but so is a raw array.

Perhaps we have different concepts of buffers.

[Jeff Schwab]

Post by Francis Glassborow

Post by Jeff Schwab

Post by Francis Glassborow
I prefer a buffer to have fixed
location and that implies that it needs to be of fixed size (we could
use a deque though).

Sorry, then what's the problem with std::vector? I thought your point
was that if it grew, you would lose locality.

Yes, but std::deque does not, elements stay where you put them and do
not suddenly get copied to somewhere entirely different.

Why do you say that? Both push_front and push_back on a std::deque
invalidate any oustanding iterators to it.

Post by Francis Glassborow

Post by Jeff Schwab
(I suggested Boost or TR1
array elsethread, but I often use std::vector myself.)

Yes tr1::array is fine but so is a raw array.

Well, that depends on the definition of "fine." We're certainly allowed
to disagree on this. :)

Post by Francis Glassborow
Perhaps we have different concepts of buffers.

Probably. In this context, I would expect the buffer's elements to be
stored contiguously.

--
[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]

[Bo Persson]

Post by Jeff Schwab

Post by Francis Glassborow

Post by Jeff Schwab

Post by Francis Glassborow
I prefer a buffer to have fixed
location and that implies that it needs to be of fixed size (we
could use a deque though).

Sorry, then what's the problem with std::vector? I thought your
point was that if it grew, you would lose locality.

Yes, but std::deque does not, elements stay where you put them and
do not suddenly get copied to somewhere entirely different.

Why do you say that? Both push_front and push_back on a std::deque
invalidate any oustanding iterators to it.

But the elements themselves do not move. If you keep a pointer or a
reference to the element, that will still be valid.


Bo Persson

[Kai-Uwe Bux]

Post by Jeff Schwab

Post by Francis Glassborow

Post by Jeff Schwab

Post by Francis Glassborow
I prefer a buffer to have fixed
location and that implies that it needs to be of fixed size (we could
use a deque though).

Sorry, then what's the problem with std::vector? I thought your point
was that if it grew, you would lose locality.

Yes, but std::deque does not, elements stay where you put them and do
not suddenly get copied to somewhere entirely different.

Why do you say that? Both push_front and push_back on a std::deque
invalidate any oustanding iterators to it.

Those calls do invalidate iterators. But that is not due to objects moving
about in memory; it is an artifact of the re-indexing. The important
provision with regard to memory locality is that neither push_back() nor
push_front() invalidates references. (The standard does not mention
pointers.)

[snip]


Best

Kai-Uwe Bux

[Neil Butterworth]

On Feb 25, 7:38 pm, Francis Glassborow

Post by Francis Glassborow

Post by Neil Butterworth

Post by red floyd
vector<unsigned char>
might do at a stretch if you need an extensible buffer. However I would
normally prefer a buffer to have fixed locality.

How are you going to arrange that in a way that vector <unsigned char>
cannot?

A vector moves its contents when it expands.

Only if you actually expand it. My original suggestion was:

vector <char> buffer( BUFSIZE );

allocating a buffer not intended to be used with push_back() et al.
What's your alternative?

Neil Butterworth

[Francis Glassborow]

Post by Neil Butterworth
On Feb 25, 7:38 pm, Francis Glassborow

Post by Francis Glassborow

Post by Neil Butterworth

Post by red floyd
vector<unsigned char>
might do at a stretch if you need an extensible buffer. However I would
normally prefer a buffer to have fixed locality.

How are you going to arrange that in a way that vector <unsigned char>
cannot?

A vector moves its contents when it expands.

vector <char> buffer( BUFSIZE );
allocating a buffer not intended to be used with push_back() et al.
What's your alternative?

char buffer[BUFSIZE]

Now you cannot use features that would change its size or location.

[Neil Butterworth]

On Feb 27, 11:56 pm, Francis Glassborow

Post by Francis Glassborow

Post by Neil Butterworth
On Feb 25, 7:38 pm, Francis Glassborow

Post by Francis Glassborow

Post by Neil Butterworth

Post by red floyd
vector<unsigned char>
might do at a stretch if you need an extensible buffer. However I would
normally prefer a buffer to have fixed locality.

How are you going to arrange that in a way that vector <unsigned char>
cannot?

A vector moves its contents when it expands.

vector <char> buffer( BUFSIZE );
allocating a buffer not intended to be used with push_back() et al.
What's your alternative?

char buffer[BUFSIZE]
Now you cannot use features that would change its size or location.

You also cannot create a very large buffer without risking stack
overflow. What would you do if you needed a buffer to hold, say, a
large image?

Neil Butterworth

[Francis Glassborow]

Post by Neil Butterworth

Post by Francis Glassborow
char buffer[BUFSIZE]
Now you cannot use features that would change its size or location.

You also cannot create a very large buffer without risking stack
overflow. What would you do if you needed a buffer to hold, say, a
large image?

use a tr1 array<>

[Martin T.]

Post by Francis Glassborow

Post by Neil Butterworth

Post by Francis Glassborow
char buffer[BUFSIZE]
Now you cannot use features that would change its size or location.

You also cannot create a very large buffer without risking stack
overflow. What would you do if you needed a buffer to hold, say, a
large image?

use a tr1 array<>

W? That will still "allocate" the memory on the stack and as far as I
was able to check it will still cause UB by stack overflow ...

br,
Martin

[Jeff Schwab]

Post by viki
What should I prefer for binary data buffers, std::string or
vector<char> ?
Like data buffers that are written to and read from binary files.

boost::array, or tr1::array if your compiler has it.

[Alexander Gutenev]

Post by viki
What should I prefer for binary data buffers, std::string or
vector<char> ?

scoped_array or shared_array from boost:: or std::tr1::

[Martin T.]

Post by viki
What should I prefer for binary data buffers, std::string or
vector<char> ?

Why? Given that vector<> is already in the standard and every compiler
has it ... why would you generally recommend a feature that not everyone
has available at the moment?

br,
Martin