Dumping memory int std::map, std::vector

[deca5423]

I'm working on a fairly large c++ project. Now I'm stopped dead in my tracks trying to dump memory from std::vector and std::map types. I don't understand why this appears to be so impossible, but I can't find a single thing that has worked.

For instance I have a vector like: (all of my vectors contain nearly 1,000 elements) so memory usage is quite significant

std::vector<std::string> name_vec;

I've tried:

name_vec.clear(); - you would assumed since you want to clear it, you don't care about the data, apparently this isn't the case... then I try after
name_vec.reserve( 0 ); - now it's size is 0. yay! but memory usage hasn't dropped

Another try:

name_vec.~vector(); this instanly appears to do what I did above, vector is empty and allocated size is 0. No memory change.

Maps turn out to be unusable after you call ~map() unless its a pointer and you reallocate it using new. The map is the largest memory source of my program, calling ~map() doesn't do anything to memory usage.

I've also tried iterating through the map and deleting the contents of first and second by hand. Still no change.

This problem appeared when I was testing time and mem usage vs different compilers of my program and I realized that through every iteration and re-reading the files that the memory wasn't dumping at all. And my program would waste 20+mb of memory after some iterations. This is baffling me, if anyone could offer some insight...

[Alvaro]

The fact that clearing the vector doesn't release memory should not be a problem. If the vector is not going out of scope, you are probably going to repopulate it at some point, so there is no need to release the memory.

If your program's memory usage expands over time, you have a leak somewhere. Perhaps you have a true memory leak (some dynamically allocated block of memory that nothing points to), or perhaps you are not removing old elements from some container.

In the case of std::map, memory *is* released when you clear the container. I don't know how you are monitoring memory usage, but it is usually the case that once a process has asked the OS for some memory, it will never release it.

[MXP]

What are you using to measure memory usage?

[deca5423]

I've narrowed it down to the map. You're right, it must be leaking something. I wrote a test project using a similar map type and the memory stays static through tons of iterations. Should've done this before posting here. *sigh*. Now I feel dumb.

I was just watching the memory in the windows task manager, the memory would jump up 200k every iteration. Its going to be a pain to find

[Alvaro]

There are tools that can help you find this type of problems. We use one called "Purify" at work, and there is a free one called "Valgrind", at least for Linux. I believe Visual C++ has its own tool to find memory leaks, but I am not familiar with it.

There are also debugging malloc implementations that can help in these cases. Now that I think about it, I've only ever used one of those in C, never in C++. If your operator new happens to use malloc under the hood, it should still work...

Good luck.

[deca5423]

Thanks, I'll have to look into Purify. I found the issue was I had a vector holding pointers.

std::vector<WadInfo*> wad_files_vector;

I was under the impression that wad_files_vector.~vector(); would call the destructor of all its elements. As stated here:

http://www.cplusplus.com/reference/stl/vector/~vector/

However, it seems the destructor is not properly called. However if I do:

for( int i=0; i<wad_files_vector.size(); i++ )
delete wad_files_vector.at(i);

My memory leak issue is mostly resolved. By the definition of ~vector I thought this would happen. But at least I've mostly resolved the issue.

New question. I can see that my program now breaks into my WadInfo destructor. It has member: char* open_file allocated as such:

open_file = new char[strlen(wad_file+1)];
strcpy(open_file,wad_file);

Do I have to delete this myself in the destructor? delete[] open_file? If I do this I get a heap corruption error..

[deca5423]

Apparently this was my problem all along. I have to iterate through myself and physically delete the pointers. I'm thinking that ~vector() or ~map() isn't guaranteed to call the destructors of their elements? At least if they're pointers? I had to do this for my map:

nameMapType::iterator map_iter;

while( !lumps_map->empty() ) {

map_iter = lumps_map->begin();
delete map_iter->second;
lumps_map->erase(map_iter);

}

While lumps_map->~map(); or delete lumps_map;, does not actually deallocate the pointers memory it contains like the loop above.

[Alvaro]

Every time you use new, you need to do delete yourself. You didn't understand what vector::~vector and map::~map do correctly: They will call the destructor on each element, but if your elements are pointers, they have trivial destructors. That's one reason why you should generally prefer having containers of objects, not pointers. If you do need a container of pointers (for instance because you need polymorphic behavior), look into using a smart pointer or a pointer container from Boost. Those will do the right thing in the destructor.

[deca5423]

Thanks Alvaro, that makes sense. Could you point me in the right direction as to why I can't delete this without getting heap corruption? I have a class (used a pointer only) with:

It has member: char* open_file allocated as such:

open_file = new char[strlen(wad_file+1)];
strcpy(open_file,wad_file);

In its destructor I assumed I would have to: delete[] open_file; because it was allocated with new but this corrupts the heap. I believe this would be my last memory leak if I do indeed have to delete it. Because the heap is corrupting it leads to believe I would not have to delete it myself because after my destructor executes its attempting to delete the string itself, which I already deleted myself.

[Alvaro]

Things would be much more clear if you could post a tiny program that shows the problem. Make it as small as possible, removing anything that is not related to the problem. My guess is that you are accidentally making shallow copies of the object, and then several destructors will delete[] the pointer. In other words: You haven't followed the "rule of three": http://en.wikipedia.org/wiki/Rule_of_three_(C%2B%2B_programming).

This is a tricky part of C++ programming, but it's worth making the effort to understand it well. Then you can use STL containers most of the time and occasionally shared_ptr, and for the most part you won't see these problems again.

[girishog]

I am working with STL I find many memory leak in my code in vector and map. I have a vector inside a vector and vector inside a map. for exm
std::vector<class1> test

class class1
{

}

[girishog]

I am working with STL I find many memory leak in my code in vector and map. I have a vector inside a vector and vector inside a map. for exmple
std::vector<class1> test
class class1
{
int a;
vector<class2> obj1;
}

class2{
CString str1;
CString str2;
}

...
..
insterting data to obj1 ... and then adding those object to vector test.

when I try to clear the vector test.clear() will the destructor of str1and str2 will be called or not. Do I need to call the destructor explicitly.

Thanks in Advance,
Girish

[Bahnyen]

Things would be much more clear if you could post a tiny program that shows the problem. Make it as small as possible, removing anything that is not related to the problem. My guess is that you are accidentally making shallow copies of the object, and then several destructors will delete[] the pointer. In other words: You haven't followed the "rule of three": http://en.wikipedia.org/wiki/Rule_of_three_(C%2B%2B_programming).

This is a tricky part of C++ programming, but it's worth making the effort to understand it well. Then you can use STL containers most of the time and occasionally shared_ptr, and for the most part you won't see these problems again.

Wow this really tricky, and I found a lot of thing that I should learn more about it. Very useful and can go more deep to the hardware controller. Thank you very much for your guide. I will come back with different questions when I get stuck in the deeper level.

[nancy19]

Thanks for letting me know about other good stuff! It helped me a lot as well!

[Fahrenheit]

name_vec.clear(); - you would assumed since you want to clear it, you don't care about the data, apparently this isn't the case... then I try after
name_vec.reserve( 0 ); - now it's size is 0. yay! but memory usage hasn't dropped

I just solved similar problem in my own project.
Here is what actually helped me:
std::vector<std::string>().swap(name_vec);

Basically we create empty temporary vector that does not hold anything, and swap it with our own.
The reason why it works lies in following facts:
1) Vector stores a buffer to allocated items outside of the main object (as a chunk of dynamically allocated memory)
2) It releases this buffer only when it is destroyed
3) swap() implementation is efficient - it does not copy data but just swaps buffers

Hope this helps!