push/pull + HWM + slow puller + zmq_close = lost message

I have two processes performing a simple push/pull flow of data from one to pusher to the puller. The puller has a HWM set as it is io bound on disk writes and the number and size of messages handled would exceed memory limits if there was no HWM set. When the pusher is finished sending all the data messages it sends a message that tells the puller there is no more data. Once this termination message is sent the pusher is finished and can exit. The puller continues to process messages until it receives the termination message whereon it too exits.

If the HWM is reached on the pull side when the pusher sends the termination message and calls zmq_close on it's end of the socket the termination message is lost and the puller blocks indefinitely expecting more messages. Note that this happens at zmq_close time (as demonstrated by flipping the WAIT_FOR_CHILD_BEFORE_TERM define in the example code) and not zmq_term.

This would seem to contradict the documentation for zmq_close and zmq_term. I would have expected the final message to be buffered after the zmq_close and only dropped by zmq_term according to the ZMQ_LINGER setting (The default of -1 is used in the example code but explicitly setting it to -1 has also been tried).

It is also worth noting that setting the pushers HWM to 1 and waiting for ZMQ_EVENTS to say a message can be sent without blocking does not help in this situation either.

Also note that this behavior is exhibited with both tcp and ipc.

Which side of the PUSH/PULL binds the end point also makes no difference (although this is not easily switched in the attached example code).

If this is expected behavior/not a bug, is there a known work around to termination messages? I could have the puller ack back to the sender that it has received the message via another socket but it would seem the same issue would exist with the puller shutting down before the pusher received the ACK. I guess this behavior could be avoided by not having a HWM on the ack socket. Doubling the number of sockets and greatly increasing the complexity of a simple termination method seems to intuitively suggest this is the wrong approach though (eg: unix pipes would be extremely frustrating and far less used with out EOF).

The attached example code creates two processes via fork, connects them via a PUSH/PULL request. There are #defines at the top t control key params and the endpoint used is passed as a command line argument. The puller/child emulates a slow receiver by doing a one second sleep between receives. The current defines are the minimum values needed to always produce this behavior on my laptop.

Thanks