mailbox_t::recv is not thread-safe, however, there are several threads may call it!

Description

I read the code of version 3.2.2. And I found in mailbox.hpp there is a comment at line 58: " There's only one thread receiving from the mailbox, but there is arbitrary number of threads sending. "
However, mailbox_t::recv() in socket_base_t:rocess_commands() would be called in both socket_base_t::send() and socket_base_t::recv().
When we use one thread to send data and another thread to receive data, there is odd of error.
Here is the code:

int zmq::mailbox_t::recv (command_t *cmd_, int timeout_)
{
// Try to get the command straight away.
if (active) {
bool ok = cpipe.read (cmd_);
if (ok)
return 0;

// If there are no more commands available, switch into passive state.
active = false;
signaler.recv ();
}

// Wait for signal from the command sender.
int rc = signaler.wait (timeout_);
if (rc != 0 && (errno == EAGAIN || errno == EINTR))
return -1;

// We've got the signal. Now we can switch into active state.
active = true;
// Get a command.
errno_assert (rc == 0);
//<< thread1 runs to here, and thread2 gains cpu to go through
bool ok = cpipe.read (cmd_);
//<< thread 2 fetched the command, and return ok as true
//<< While, when thread 1 tried to fectch the command, false would be returned, and an assertion, " Assertion failed: false mailbox_t:line 79" , would be thrown.
zmq_assert (ok);
return 0;
}

Environment

linux X86_64

Activity

Show:
Christian Kamm
June 27, 2013, 6:37 AM

Thanks for checking, that sounds odd. Did you log the mailbox instance as well as the calling thread? The mailboxes are used to communicate among threads so there will be several active at the same time. Maybe attach your test code and debug output? I'll try to reproduce later.

starcheng
June 28, 2013, 1:47 AM

I'm Sorry for my misunderstanding about socket's mailbox_t::recv running in an io_thread. Actually mailbox_t::recv did not run in an io_thread, while mailbox_t::send sometimes did.
This problem is just because of sending data in an individual thread and receiving data in another.
But, for the bidirectional socket, such as ZMQ_ROUTER, it may be inefficient to send/recv data in the same thread especially if data throughput is huge.
Thanks for your following!

Christian Kamm
June 28, 2013, 6:33 AM

I'd close this issue now, but the close button just pops up a JIRA error.

You could provide your own synchronization around socket access. It may be nicer to use zmq sockets to communicate with your worker threads though, the guide has some examples of this.

THOMAS Yan
July 8, 2013, 12:27 PM

That seems identical has and

starcheng
September 25, 2013, 3:54 AM

It is supposed to call mailbox_t::recv always in a thread, while socket_base_t:rocess_command who calls mailbox_t::recv runs in multi-threads called by socket_base_t::send & socket_base_t::recv.
User has to ensure send and recv in a thread.

Won't Fix

Assignee

Unassigned

Reporter

starcheng

Labels

Components

Affects versions

Priority

Critical