Assertion failed: No buffer space available (tcp_socket.cpp:91) - only on Windows XP x86
Description
Environment
Activity
Martin Sustrik April 23, 2011 at 12:18 PM
Let's keep it open for now.
JamesDunne April 22, 2011 at 10:23 PM
That makes sense. Thanks. I'll leave the issue's open/close status to your discretion.
Martin Sustrik April 21, 2011 at 9:05 PM
Definitely a bug. ENOBUFS is used for UDP and similar transports. It should not happen with TCP.
Setting the buffers to 0 doesn't seem to be a good idea. AFAIU it means that send function then blocks until the data is actually pushed to the network. I.e. it block any other processing going on in the same I/O thread.
In theory, we can set a limit on amount of data written to the socket at a time. Not sure this would help though.
JamesDunne April 19, 2011 at 4:01 PM
Setting SNDBUF or RCVBUF to 0 appears to work correctly now.
JamesDunne April 19, 2011 at 3:52 PM
It seems this is a bug in WinXP's networking stack... http://support.microsoft.com/kb/201213
Workarounds:
Use the socket in nonblocking or asynchronous mode.
Break large-size data blocks into small ones and specify a relatively small buffer in send for blocking sockets, preferably no larger than 64K.
Set the SO_SNDBUF socket option to 0 (zero) to allow the stack to send from your application buffer directly.
I think I'll try setting the SO_SNDBUF to 0 since that's all I have available from my application's perspective to change.
Could we perhaps accommodate the other workarounds in libzmq code if the OS is WinXP to prevent other users from encountering this bug? Using overlapped I/O could be another possibility.
Details
Details
Assignee
Reporter
JamesDunne
I get this issue on WinXP x86 SP3. Testing on Win7 x64 has never produced this problem so far.
Sorry, I'm using C# and a custom-hacked clrzmq2 binding to do my work so posting an example would be useless here, but the problem is very simple in nature.
The issue occurs on a PUB socket with a SNDHWM = 128 and a SndBuf = 1048576 * 128 * 4 (or some other ridiculously large-enough buffer size to accommodate the hwm limit). I send messages of approximately 1MB in size. I've also seen the same error on the SUB socket with a high RCVHWM, like 128. If I back down the HWM values to like 4 the error does not occur. Basically I'm just trying to send as much data as possible which will eventually be written to disk, hence the high HWMs. I want to queue all the received messages up in memory and flush to a much slower disk writer PUSH socket with its PULL on a separate thread with its own HWM.
The error is a WSAENOBUFS code returned from WinSock2's ::send() which is unhandled by the tcp_socket.cpp code near line 91.
Should I temporarily add this error code to the known error cases and have the function return -1 instead of asserting out? Will zmq recover and continue to function properly if I do this? I'm not too familiar with the zmq code base here but I'm willing to try it.