Valhalla Legends Forums Archive | Battle.net Bot Development | BNCSutil Memory Problems?

AuthorMessageTime
Archangel
Well, actually, when connecting with a single cdkey product (not war3) memory goes up to 2,000 kb,  when connecting using warcraft3 memory goes up to like 10,000 kb.

Each time i "reconnect" (SEXP) using BNCSUtil memory never goes down, and goes up by 2,000 kb. Its a problem im causing or a BNCSUtil.dll problem?
March 28, 2005, 12:38 AM
The-FooL
I just tested with my bot(newly implemented BNCS util code).

Connecting via WAR3 causes the jump of over 10k mem vs connection with BNLS.  Every connection with sc/d2 causes a jump of 2,000k (these are cumulative.  Connecting a few times doubles the memory usage).

I am using the NLS_Free and kd_free method, and calling them as soon as the decoding is done/login sequence is finished.  I also noticed that I am getting a handle of zero for the decoder(though it still works).  What is the issue here?
March 28, 2005, 1:02 AM
Archangel
Ok, StealthBot Beta got same problem, so i suppose it must be the library.
March 28, 2005, 1:17 AM
UserLoser.
Does BNCSUtil cache all the file data for War3 game files?  Those are quite large, and may be the cause
March 28, 2005, 1:21 AM
The-FooL
It must be some sort of memory leak, because the increase is cumulative.
March 28, 2005, 1:28 AM
Networks
[quote author=The-FooL link=topic=11069.msg105827#msg105827 date=1111971723]
I just tested with my bot(newly implemented BNCS util code).

Connecting via WAR3 causes the jump of over 10k mem vs connection with BNLS.  Every connection with sc/d2 causes a jump of 2,000k (these are cumulative.  Connecting a few times doubles the memory usage).

I am using the NLS_Free and kd_free method, and calling them as soon as the decoding is done/login sequence is finished.  I also noticed that I am getting a handle of zero for the decoder(though it still works).  What is the issue here?
[/quote]

Oh thank god, I thought I was just a poor programmer. whew, yes Archangel found this out in my moderation bot. I posted about this earlier and thought it got fixed but I guess I just connected using a different client than WAR3. I guess my topic is unresolved then. What ever occurs happens to me.
March 28, 2005, 2:54 AM
Archangel
I made test with 3 bots, Vanquish Bot, StealthBot and with one im currently developing. Im waiting for Cloaked answer or fix.
March 28, 2005, 3:08 AM
Kp
Rebuild with debugging symbols (dwarf-2 preferred), and run it in valgrind with --tool=memcheck.  Report what line(s) are leaking the memory.
March 28, 2005, 3:28 AM
Networks
[quote author=Kp link=topic=11069.msg105875#msg105875 date=1111980507]
Rebuild with debugging symbols (dwarf-2 preferred), and run it in valgrind with --tool=memcheck.  Report what line(s) are leaking the memory.
[/quote]

Wow...I am sorry but I didn't really comprehend that but ok.
March 28, 2005, 1:55 PM
Myndfyr
[quote author=Networks link=topic=11069.msg105941#msg105941 date=1112018118]
[quote author=Kp link=topic=11069.msg105875#msg105875 date=1111980507]
Rebuild with debugging symbols (dwarf-2 preferred), and run it in valgrind with --tool=memcheck.  Report what line(s) are leaking the memory.
[/quote]

Wow...I am sorry but I didn't really comprehend that but ok.
[/quote]

What's hard about that statement?

Rebuild the library.  Make sure the compiler/linker emits debugging symbols (a debug build) into the PE file.
Get the utility called "valgrind" and run the program with that utility, using the command line option "--tool=memcheck".

It's English.  Do you speak it?
March 28, 2005, 5:36 PM
Arta
Relax. For someone not familiar with those tools and techniques, that advice would probably be rather difficult to follow.

No point in assuming that everyone's an expert :P
March 28, 2005, 7:23 PM
Archangel
[quote author=MyndFyre link=topic=11069.msg105976#msg105976

What's hard about that statement?

Rebuild the library.  Make sure the compiler/linker emits debugging symbols (a debug build) into the PE file.
Get the utility called "valgrind" and run the program with that utility, using the command line option "--tool=memcheck".

It's English.  Do you speak it?
[quote][/quote]

Ok calm down, alot of people dont understand alot of stuff in here, so why being so damn agressive?
March 28, 2005, 7:32 PM
Myndfyr
[quote author=Archangel link=topic=11069.msg105997#msg105997 date=1112038323]
Ok calm down, alot of people dont understand alot of stuff in here, so why being so damn agressive?
[/quote]
[quote author=Arta[vL] link=topic=11069.msg105993#msg105993 date=1112037815]
Relax. For someone not familiar with those tools and techniques, that advice would probably be rather difficult to follow.
[/quote]
To clarify my last statement and then question:

I'm not familiar with the tool "valgrind".  I've never heard of it, nor do I know what it does.  My point was that there is a lot of information in the statement Kp made that you can read and understand if you try to think about the statement, and then there is a lot of implied information.

It's like the other day when I came home early to help my mom out with a party she was getting ready to have.  I told her that I came home to help.  Her response, "Help?  What do you mean, 'help'?"  Well wtf do you think I mean?  If she had thought about the context of the situation and my statement, it would have been pretty clear.  It's hardly an exercise in critical thinking.

Similarly, it's reasonable to expect that Networks would analyze the situation in terms of what Kp was suggesting; I understand if someone doesn't know what debugging symbols are.  However, the rest is quite clear: valgrind is a tool to obtain, and it uses the binary and debugging symbols file to figure out what lines in the source code are causing memory leaks.

Appropriate questions would have been "How do I build with debugging symbols?  What are dwarf-2 symbols, and how are they different?  Where do I get valgrind?"

The question was not meant to be aggressive, and for that, I apologize.  It's just frustrating when people don't try to figure something out.
March 28, 2005, 8:05 PM
iago
I grant you what you said, but you have to admit that Kp goes out of his way to make what he says difficult to understand :P

And, to post a bit back on topic, I'll talk to the guy who wrote BNCSutil tonight and point him to this thread.
March 28, 2005, 9:05 PM
Networks
[quote author=MyndFyre link=topic=11069.msg105999#msg105999 date=1112040339]
[quote author=Archangel link=topic=11069.msg105997#msg105997 date=1112038323]
Ok calm down, alot of people dont understand alot of stuff in here, so why being so damn agressive?
[/quote]
[quote author=Arta[vL] link=topic=11069.msg105993#msg105993 date=1112037815]
Relax. For someone not familiar with those tools and techniques, that advice would probably be rather difficult to follow.
[/quote]
To clarify my last statement and then question:

I'm not familiar with the tool "valgrind".  I've never heard of it, nor do I know what it does.  My point was that there is a lot of information in the statement Kp made that you can read and understand if you try to think about the statement, and then there is a lot of implied information.

It's like the other day when I came home early to help my mom out with a party she was getting ready to have.  I told her that I came home to help.  Her response, "Help?  What do you mean, 'help'?"  Well wtf do you think I mean?  If she had thought about the context of the situation and my statement, it would have been pretty clear.  It's hardly an exercise in critical thinking.

Similarly, it's reasonable to expect that Networks would analyze the situation in terms of what Kp was suggesting; I understand if someone doesn't know what debugging symbols are.  However, the rest is quite clear: valgrind is a tool to obtain, and it uses the binary and debugging symbols file to figure out what lines in the source code are causing memory leaks.

Appropriate questions would have been "How do I build with debugging symbols?  What are dwarf-2 symbols, and how are they different?  Where do I get valgrind?"

The question was not meant to be aggressive, and for that, I apologize.  It's just frustrating when people don't try to figure something out.
[/quote]

Rebuild with debugging symbols (dwarf-2 preferred), and run it in valgrind with --tool=memcheck.  Report what line(s) are leaking the memory.

In bold are the only real lines I can fully comprehend. I understood what he was saying. Similarily when your mom says "help!", I understood that this was to help us find where the memory leaks are occuring. The method in which he is explaining is hard to comprehend.

Doing what Kp said isn't go to help us, it's up to Cloaked (ShadyPalm88) to fix it and he probably will be able to understand what Kp stated. =)
March 28, 2005, 9:56 PM
Kp
Wow.  It's been a while since I've caused so much activity with so little effort. :)  Remarks, in no particular order:

Networks: if you don't understand something, a vague "I don't get it", isn't very helpful.  With such a non-specific problem, I doubt I would've been motivated to address your confusion.  Also, although shadypalm is the primary distributor at this point, it's not his obligation to fix it.  I suspect he'd be grateful for a patch instead of an "it doesn't work; fix it!" message.  That is, you should try to fix it yourself and send him the patch if it works.  See below about valgrind.

Arta: sure, not everyone's an expert.  I was a bit time pressed, and felt it better to toss out some concise instructions that an expert could interpret (and hope for the best).  If the recipient wasn't an expert, he would have the side joy of learning about valgrind and its functionality as a consequence of my message. :)

iago: not at all.  I only make it difficult to understand if it's not a lot of effort for me. :)  Jokes just aren't worth it if I have to spend days setting them up. ;)

For those who are not familiar with valgrind: it's a tool to perform dynamic instrumentation of a binary for purposes of profiling, correctness checking, or just about anything else you want to write a tool to do.  Binaries built with debugging symbols produce more informative reports than those which are stripped ( = have no debugging symbols).  The tool memcheck would detect any out of bounds reads/writes, as well as detect memory leaks when the process terminates.  Caveat: valgrind is Linux specific.  There's some rumors that they might try to port it to other Unices, but Microsoft Windows is entirely out of the question for the foreseeable future.  MS-Windows simply has far too many system calls to consider writing wrappers, since valgrind cannot follow execution into kernel mode, and thus must wrap system calls to update its post-call view of process state.  BNCSutil builds on Linux systems, so that's not a problem in this case.

There, that wasn't so unclear, was it? ;)
March 29, 2005, 2:16 AM
Null
or.. you couldve just said that originally instead of being a hardass on a forum!.

Lowest level of life = forum hardasses
March 29, 2005, 3:21 AM
BaDDBLooD
[quote author=effect link=topic=11069.msg106055#msg106055 date=1112066518]
or.. you couldve just said that originally instead of being a hardass on a forum!.

Lowest level of life = forum hardasses
[/quote]

Keep that to Private Messages please.

EDIT: Thanks KP, that will really come in handy in the future.
March 29, 2005, 3:26 AM
Myndfyr
[off-topic]
[quote author=Kp link=topic=11069.msg106044#msg106044 date=1112062576]
Caveat: valgrind is Linux specific.  There's some rumors that they might try to port it to other Unices, but Microsoft Windows is entirely out of the question for the foreseeable future.[/quote]
Hahaha.  I never thought of the plural of either of those being "Unices" nor "Linuces," although I suppose they are accurate.  Nice. :)
[/off-topic]
March 29, 2005, 3:29 AM
warz
[quote author=MyndFyre link=topic=11069.msg105999#msg105999 date=1112040339]
[quote author=Archangel link=topic=11069.msg105997#msg105997 date=1112038323]
Ok calm down, alot of people dont understand alot of stuff in here, so why being so damn agressive?
[/quote]
[quote author=Arta[vL] link=topic=11069.msg105993#msg105993 date=1112037815]
Relax. For someone not familiar with those tools and techniques, that advice would probably be rather difficult to follow.
[/quote]
To clarify my last statement and then question:

I'm not familiar with the tool "valgrind".  I've never heard of it, nor do I know what it does.  My point was that there is a lot of information in the statement Kp made that you can read and understand if you try to think about the statement, and then there is a lot of implied information.

It's like the other day when I came home early to help my mom out with a party she was getting ready to have.  I told her that I came home to help.  Her response, "Help?  What do you mean, 'help'?"  Well wtf do you think I mean?  If she had thought about the context of the situation and my statement, it would have been pretty clear.  It's hardly an exercise in critical thinking.

Similarly, it's reasonable to expect that Networks would analyze the situation in terms of what Kp was suggesting; I understand if someone doesn't know what debugging symbols are.  However, the rest is quite clear: valgrind is a tool to obtain, and it uses the binary and debugging symbols file to figure out what lines in the source code are causing memory leaks.

Appropriate questions would have been "How do I build with debugging symbols?  What are dwarf-2 symbols, and how are they different?  Where do I get valgrind?"

The question was not meant to be aggressive, and for that, I apologize.  It's just frustrating when people don't try to figure something out.
[/quote]

Dang, your mom is dumb. My mom is definantly better. Sorry.
March 29, 2005, 5:27 AM
Kp
[quote author=effect link=topic=11069.msg106055#msg106055 date=1112066518]or.. you couldve just said that originally instead of being a hardass on a forum!.

Lowest level of life = forum hardasses[/quote]

Like I said (which you'd know if you'd actually read my post), I was time pressed and felt it was better to give him some hints about where to start, rather than just ignore him.  Besides which, I hardly think that my original message constitutes "being a hardass".  It could've been more helpful if I'd had more time and inclination when crafting it, but it was by no means rude.  That said, don't provoke the moderators.
March 29, 2005, 6:29 AM
Networks
Isn't Cloaked the main distributor? It's his dll, his open-source project. I don't know as much C++ as I'd like to know to probably fix it.
March 29, 2005, 5:00 PM
iago
[quote author=Networks link=topic=11069.msg106115#msg106115 date=1112115659]
Isn't Cloaked the main distributor? It's his dll, his open-source project. I don't know as much C++ as I'd like to know to probably fix it.
[/quote]

Yes, he is.  But it's opensource, so if I find the problem I can send him a .patch file and he can apply it, then everybody is happy.  If he has made other changes since then, a .patch will merge with them.  It's very handy.
March 29, 2005, 5:19 PM
shadypalm88
Hi everybody.  When I heard about the memory usage I figured there was only one thing that could possibly use that much: CheckRevision.  On Windows systems this uses memory mapping.  Although the memory handle was being closed, the memory wasn't actually being unmapped.  This was fixed and should end the memory troubles.

Edit: I also fixed a (much smaller) leak in WarCraft III decoding (thanks for the ValGrind tip), and it compiles on *NIX "out of the box" now.
April 4, 2005, 2:55 AM
UserLoser.
[quote author=UserLoser link=topic=11069.msg105833#msg105833 date=1111972899]
Does BNCSUtil cache all the file data for War3 game files?  Those are quite large, and may be the cause
[/quote]
April 4, 2005, 3:05 AM
iago
[quote author=UserLoser link=topic=11069.msg107248#msg107248 date=1112583943]
[quote author=UserLoser link=topic=11069.msg105833#msg105833 date=1111972899]
Does BNCSUtil cache all the file data for War3 game files?  Those are quite large, and may be the cause
[/quote]
[/quote]

Wow, you're cool!  If you knew the problem, why didn't you fix it while shadypalm88 was gone?
April 4, 2005, 3:48 AM
UserLoser.
[quote author=iago link=topic=11069.msg107254#msg107254 date=1112586513]
[quote author=UserLoser link=topic=11069.msg107248#msg107248 date=1112583943]
[quote author=UserLoser link=topic=11069.msg105833#msg105833 date=1111972899]
Does BNCSUtil cache all the file data for War3 game files?  Those are quite large, and may be the cause
[/quote]
[/quote]

Wow, you're cool!  If you knew the problem, why didn't you fix it while shadypalm88 was gone?
[/quote]

If you are going to say something that would probably get you punched in the mouth in person, do not say it.
April 4, 2005, 4:35 AM
Maddox
Then shut up.
April 4, 2005, 9:18 AM
iago
[quote author=Maddox link=topic=11069.msg107274#msg107274 date=1112606280]
Then shut up.
[/quote]

Haha, exactly. 
April 4, 2005, 1:40 PM

Search