in my program i have one thread which handles incoming data from the socket and one other thread which processes the data. the problem is that the incoming data thread is going faster than the processing thread and therefore calls the processing function before it has a chance to finish.. that results in the outcome to be shit. i need advice on how to optimally model the threading system. If i should implement a semaphore to only call the other thread when it gives an ok signal, or create another thread to process the data, or process the data in the same thread as where i call recv() on the socket.

Or you could just not use multiple threads at all, hook the socket and call that function on the callback message for receving data.

lol

that's also not the question though, what if i want to run multiple connections, not just one.

[quote author=l2k-Shadow link=topic=16834.msg170591#msg170591 date=1183229552]
in my program i have one thread which handles incoming data from the socket and one other thread which processes the data. the problem is that the incoming data thread is going faster than the processing thread and therefore calls the processing function before it has a chance to finish.. that results in the outcome to be shit. i need advice on how to optimally model the threading system. If i should implement a semaphore to only call the other thread when it gives an ok signal, or create another thread to process the data, or process the data in the same thread as where i call recv() on the socket.
[/quote]

Well, in any data handling situation you'll need to know when it's alright to process received data. Create a method of storing unprocessed data in either the receiving thread, or the processing thread, and when you know it's fine to process a complete message, pass the message to the processing function, and clean up the unprocessed data container as necessary. This is the same logic used in just about every current client emulator for bnet - multi-threaded or not.

This avoids one thread sending data to another thread that is trying to read the data as it's received.

yeah, i created a queue system, but i'm not sure why it's not working properly. i'll look through it again. i just wanted ideas.

EDIT: yeah it was an error in my handling of the queue, thx.

Why not use a pool for incoming packets?

Doing so allows you the benefit of not allocating and deallocating memory a lot, but you can flag them to be done or re-insert them into queues for reuse and dequeue when necessary.  Then all you need to worry about is synchronous access to your global queue/linked-list, and that's much easier to manage with things like critical sections and spinlocks.

i/o thread to read, buffer, and parse your data. threadsafe queue (with an atomic get() operation) to pass messages from the i/o thread to the logic threads.

If the problem is that if the data takes considerable time (CPU wise) to process, creating a seperate thread to process it is not going to change a thing. Most likely it will make things worse because you'll have to queue data.

The only thing I can think of is have the processing queue multitask between received data.

For example, I have the same sort thing setup with my D2GS. One thread manages processing incomming messages, one thread manages collision detection/game movement mechanics.

[quote author=Win32 link=topic=16834.msg171303#msg171303 date=1185928491]
If the problem is that if the data takes considerable time (CPU wise) to process, creating a seperate thread to process it is not going to change a thing. Most likely it will make things worse because you'll have to queue data.
[/quote]

That's not true.  If you have a single thread receiving the data and running it through the logic and updating the UI, you're going to take a lot of overhead in processing each packet.

Queueing is an exceptionally lightweight task.  It's a linked list.  Woooo....