How should one handle a malloc failure? Should the program terminate or should it keep running and try to avoid doing things that will call malloc? Would it be safe to write out an error file still or how should one inform the user of the error? My goal is to try and gracefully handle malloc failure when the system may be out of resources and such.

Usually applications print an error message, and depending on whether that allocated memory was essential to the application or not, it'll quit or keep running and avoid using the memory.

I suppose it depends on what you feel like doing.

BC 3.xx has a global malloc/new failure handler that tries to free up some memory (e.g. clear chat history, disable icons, ...) and retry the request before giving up. BC 4.xx just fails the current operation.

Usually when my functions are doing malloc I return 0 from the function if malloc is successful, after doing whatever i was supposed to do. But 0 immediately after the failed allocation if unsuccessful.

[quote author=Krush[LM] link=board=30;threadid=7463;start=0#msg68155 date=1088731538]
Usually when my functions are doing malloc I return 0 from the function if malloc is successful, after doing whatever i was supposed to do. But 0 immediately after the failed allocation if unsuccessful.
[/quote]Not a good idea. How will the caller know something went wrong?

[quote author=Eibro[yL] link=board=30;threadid=7463;start=0#msg68463 date=1088954046]
[quote author=Krush[LM] link=board=30;threadid=7463;start=0#msg68155 date=1088731538]
Usually when my functions are doing malloc I return 0 from the function if malloc is successful, after doing whatever i was supposed to do. But 0 immediately after the failed allocation if unsuccessful.
[/quote]Not a good idea. How will the caller know something went wrong?
[/quote]

You're not paying attention! If something went wrong, he IMMEDIATELY returns 0, rather than returning 0 on success AFTER doing whatever. *smirk*

Raise an exception if that's available to you (malloc is C, so I don't know if you do or not). You can either catch it on the spot if it's critical to your system's operation, or you can let it bubble up to the global exception handler, in which case you can msgbox and quit.

Skywing's BC comment was interesting. Perhaps you could create a myMalloc function which wraps malloc. If malloc fails, attempt to delete some old unimportant pointers (as he said, chat history etc), and try again. If it fails again, raise an exception.

Why not use the new/delete operators and use the bad_alloc exception to do the error handling? Failing to allocate enough memory isn't something that is going to happen often, so the overhead of an exception call isn't going to be something you should have to worry about. If you're not going to use the bad_alloc exception, usually you can do one of two things (or be even more creative!):

1 - Check the return and determine what happened from there and notify the user. From that point you can execute some type of action (perhas a call to atexit/exit or terminating the specific operation) while notifying the user of what happened based on the return.

2 - Make some kind of malloc error wrapper where you call dynamic memory allocation functions/operators and do your error checking there.

Most C++ programmers that I know use the standard exception of bad_malloc. Note however, not all compilers support this exception and will still use the return convention.

Also see std::set_new_handler()