Uploaded image for project: 'libzmq'
  1. libzmq
  2. LIBZMQ-273

O_CLOEXEC flag used in ip.cpp:192 is supported only on Linux kernels 2.6.27+

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.1.10
    • Fix Version/s: None
    • Component/s: build system
    • Labels:
      None

      Description

      When building ZeroMQ versions 2.1.10+ build system have to check whether the underlying kernel version is actually 2.6.27 or newer (or maybe check it on runtime), even if the O_CLOEXEC flag is defined. Some virtualization environment, like OpenVZ, allow to run newer Linux systems (like Lucid) on older host systems (like Hardy), which will end up in new libc dev headers with O_CLOEXEC defined running on older kernel which doesn't support it, which in order will block all TCP communications due to an inability to bind a socket at all.

        Gliffy Diagrams

          Attachments

            Activity

            Hide
            {annotationCollection: [{}], annotations: [{}], collectionInheritableAnnotations: [{}], declaredAnnotations: [{}], description: Annotated value: mikko; Annotations: [@com.atlassian.velocity.htmlsafe.HtmlSafe()]} Mikko Koppanen added a comment -

            Can you test the attached patch?

            Show
            {annotationCollection: [{}], annotations: [{}], collectionInheritableAnnotations: [{}], declaredAnnotations: [{}], description: Annotated value: mikko; Annotations: [@com.atlassian.velocity.htmlsafe.HtmlSafe()]} Mikko Koppanen added a comment - Can you test the attached patch?
            Hide
            {annotationCollection: [{}], annotations: [{}], collectionInheritableAnnotations: [{}], declaredAnnotations: [{}], description: Annotated value: kobolog; Annotations: [@com.atlassian.velocity.htmlsafe.HtmlSafe()]} Andrey Sibiryov added a comment -

            Yeah, it does work.

            Show
            {annotationCollection: [{}], annotations: [{}], collectionInheritableAnnotations: [{}], declaredAnnotations: [{}], description: Annotated value: kobolog; Annotations: [@com.atlassian.velocity.htmlsafe.HtmlSafe()]} Andrey Sibiryov added a comment - Yeah, it does work.
            Hide
            {annotationCollection: [{}], annotations: [{}], collectionInheritableAnnotations: [{}], declaredAnnotations: [{}], description: Annotated value: sustrik; Annotations: [@com.atlassian.velocity.htmlsafe.HtmlSafe()]} Martin Sustrik added a comment -

            I've pushed the patch to the master. Also sent it to the mailing list.

            Show
            {annotationCollection: [{}], annotations: [{}], collectionInheritableAnnotations: [{}], declaredAnnotations: [{}], description: Annotated value: sustrik; Annotations: [@com.atlassian.velocity.htmlsafe.HtmlSafe()]} Martin Sustrik added a comment - I've pushed the patch to the master. Also sent it to the mailing list.
            Hide
            {annotationCollection: [{}], annotations: [{}], collectionInheritableAnnotations: [{}], declaredAnnotations: [{}], description: Annotated value: jtaylor; Annotations: [@com.atlassian.velocity.htmlsafe.HtmlSafe()]} Julian Taylor added a comment - - edited

            can this check be done at runtime instead of compile time?

            On ubuntu the build machines run kernel 2.6.24, but the package is built for releases with much newer kernels.
            So we would either have to disable the feature for all users, reintroducing issue 218, or disable all testsuites of the zmq ecosystem.

            Show
            {annotationCollection: [{}], annotations: [{}], collectionInheritableAnnotations: [{}], declaredAnnotations: [{}], description: Annotated value: jtaylor; Annotations: [@com.atlassian.velocity.htmlsafe.HtmlSafe()]} Julian Taylor added a comment - - edited can this check be done at runtime instead of compile time? On ubuntu the build machines run kernel 2.6.24, but the package is built for releases with much newer kernels. So we would either have to disable the feature for all users, reintroducing issue 218, or disable all testsuites of the zmq ecosystem.
            Hide
            {annotationCollection: [{}], annotations: [{}], collectionInheritableAnnotations: [{}], declaredAnnotations: [{}], description: Annotated value: kobolog; Annotations: [@com.atlassian.velocity.htmlsafe.HtmlSafe()]} Andrey Sibiryov added a comment -

            Yeah, the corresponding call will return an error if an incorrect combination of flags are passed to it, so one can check for the error code, I think.

            Show
            {annotationCollection: [{}], annotations: [{}], collectionInheritableAnnotations: [{}], declaredAnnotations: [{}], description: Annotated value: kobolog; Annotations: [@com.atlassian.velocity.htmlsafe.HtmlSafe()]} Andrey Sibiryov added a comment - Yeah, the corresponding call will return an error if an incorrect combination of flags are passed to it, so one can check for the error code, I think.
            Hide
            {annotationCollection: [{}], annotations: [{}], collectionInheritableAnnotations: [{}], declaredAnnotations: [{}], description: Annotated value: jtaylor; Annotations: [@com.atlassian.velocity.htmlsafe.HtmlSafe()]} Julian Taylor added a comment -

            this is what one gets on the ubuntu build machines (zmq 2.1.10):
            socket(PF_INET, SOCK_STREAM|SOCK_CLOEXEC, IPPROTO_TCP) = -1 EINVAL (Invalid argument)

            catching that and try once again without CLOEXEC before aborting should work.

            Show
            {annotationCollection: [{}], annotations: [{}], collectionInheritableAnnotations: [{}], declaredAnnotations: [{}], description: Annotated value: jtaylor; Annotations: [@com.atlassian.velocity.htmlsafe.HtmlSafe()]} Julian Taylor added a comment - this is what one gets on the ubuntu build machines (zmq 2.1.10): socket(PF_INET, SOCK_STREAM|SOCK_CLOEXEC, IPPROTO_TCP) = -1 EINVAL (Invalid argument) catching that and try once again without CLOEXEC before aborting should work.
            Hide
            {annotationCollection: [{}], annotations: [{}], collectionInheritableAnnotations: [{}], declaredAnnotations: [{}], description: Annotated value: sustrik; Annotations: [@com.atlassian.velocity.htmlsafe.HtmlSafe()]} Martin Sustrik added a comment -

            I think this one is up to build guys to reply to. Is messing with the constants unknown to the OS safe? I think it boils down to the particular kernel's (Linux in this case) forward compatibility guarantees.

            Show
            {annotationCollection: [{}], annotations: [{}], collectionInheritableAnnotations: [{}], declaredAnnotations: [{}], description: Annotated value: sustrik; Annotations: [@com.atlassian.velocity.htmlsafe.HtmlSafe()]} Martin Sustrik added a comment - I think this one is up to build guys to reply to. Is messing with the constants unknown to the OS safe? I think it boils down to the particular kernel's (Linux in this case) forward compatibility guarantees.
            Hide
            {annotationCollection: [{}], annotations: [{}], collectionInheritableAnnotations: [{}], declaredAnnotations: [{}], description: Annotated value: mato; Annotations: [@com.atlassian.velocity.htmlsafe.HtmlSafe()]} Martin Lucina added a comment -

            Julian,

            yours is quite a special case; just to confirm that I understand it correctly, you are building libzmq against a glibc version which supports SOCK_CLOEXEC, running it against that same (or newer) glibc, but with a kernel which does not support SOCK_CLOEXEC.

            Am I right?

            So, the question here is, can we safely assume, on Linux only, that a socket(..., SOCK_CLOEXEC, ...) call on any kernel version not supporting SOCK_CLOEXEC will always fail with EINVAL? If yes, then this check is safe, if not, then we can't use this approach safely as a default for mainline libzmq.

            Note that the SOCK_CLOEXEC functionality has a security-related impact so we should not change this lightly.

            Another approach I can think of is that Ubuntu use the approach you've described, and you keep this as your own private patch to libzmq.

            Show
            {annotationCollection: [{}], annotations: [{}], collectionInheritableAnnotations: [{}], declaredAnnotations: [{}], description: Annotated value: mato; Annotations: [@com.atlassian.velocity.htmlsafe.HtmlSafe()]} Martin Lucina added a comment - Julian, yours is quite a special case; just to confirm that I understand it correctly, you are building libzmq against a glibc version which supports SOCK_CLOEXEC, running it against that same (or newer) glibc, but with a kernel which does not support SOCK_CLOEXEC. Am I right? So, the question here is, can we safely assume, on Linux only, that a socket(..., SOCK_CLOEXEC, ...) call on any kernel version not supporting SOCK_CLOEXEC will always fail with EINVAL? If yes, then this check is safe, if not, then we can't use this approach safely as a default for mainline libzmq. Note that the SOCK_CLOEXEC functionality has a security-related impact so we should not change this lightly. Another approach I can think of is that Ubuntu use the approach you've described, and you keep this as your own private patch to libzmq.
            Hide
            {annotationCollection: [{}], annotations: [{}], collectionInheritableAnnotations: [{}], declaredAnnotations: [{}], description: Annotated value: jtaylor; Annotations: [@com.atlassian.velocity.htmlsafe.HtmlSafe()]} Julian Taylor added a comment -

            >yours is quite a special case; just to confirm that I understand it correctly, you are building libzmq >against a glibc version which supports SOCK_CLOEXEC, running it against that same (or newer) glibc, but >with a kernel which does not support SOCK_CLOEXEC.

            this is correct. The built package is then distributed to releases which run newer kernels too.
            The only issue are the build time testsuites of packages using zmq running on the build machines with the old kernel.

            >So, the question here is, can we safely assume, on Linux only, that a socket(..., SOCK_CLOEXEC, ...) call >on any kernel version not supporting SOCK_CLOEXEC will always fail with EINVAL? If yes, then this check
            > is safe, if not, then we can't use this approach safely as a default for mainline libzmq.

            Sorry I can't give any input on this.

            >Another approach I can think of is that Ubuntu use the approach you've described, and you keep this as >your own private patch to libzmq.

            this is possible, probably its the safer choice as the patch can easily be removed again when the build machines are updated (likely 2013)

            Show
            {annotationCollection: [{}], annotations: [{}], collectionInheritableAnnotations: [{}], declaredAnnotations: [{}], description: Annotated value: jtaylor; Annotations: [@com.atlassian.velocity.htmlsafe.HtmlSafe()]} Julian Taylor added a comment - >yours is quite a special case; just to confirm that I understand it correctly, you are building libzmq >against a glibc version which supports SOCK_CLOEXEC, running it against that same (or newer) glibc, but >with a kernel which does not support SOCK_CLOEXEC. this is correct. The built package is then distributed to releases which run newer kernels too. The only issue are the build time testsuites of packages using zmq running on the build machines with the old kernel. >So, the question here is, can we safely assume, on Linux only, that a socket(..., SOCK_CLOEXEC, ...) call >on any kernel version not supporting SOCK_CLOEXEC will always fail with EINVAL? If yes, then this check > is safe, if not, then we can't use this approach safely as a default for mainline libzmq. Sorry I can't give any input on this. >Another approach I can think of is that Ubuntu use the approach you've described, and you keep this as >your own private patch to libzmq. this is possible, probably its the safer choice as the patch can easily be removed again when the build machines are updated (likely 2013)
            Hide
            {annotationCollection: [{}], annotations: [{}], collectionInheritableAnnotations: [{}], declaredAnnotations: [{}], description: Annotated value: mato; Annotations: [@com.atlassian.velocity.htmlsafe.HtmlSafe()]} Martin Lucina added a comment -

            >this is possible, probably its the safer choice as the patch can easily be removed again when the build machines are updated (likely 2013)

            Ack, can we close this as "fixed" then?

            Show
            {annotationCollection: [{}], annotations: [{}], collectionInheritableAnnotations: [{}], declaredAnnotations: [{}], description: Annotated value: mato; Annotations: [@com.atlassian.velocity.htmlsafe.HtmlSafe()]} Martin Lucina added a comment - >this is possible, probably its the safer choice as the patch can easily be removed again when the build machines are updated (likely 2013) Ack, can we close this as "fixed" then?
            Hide
            {annotationCollection: [{}], annotations: [{}], collectionInheritableAnnotations: [{}], declaredAnnotations: [{}], description: Annotated value: jtaylor; Annotations: [@com.atlassian.velocity.htmlsafe.HtmlSafe()]} Julian Taylor added a comment -

            runtime SOCK_CLOEXEC check, based on 2.1.10

            Show
            {annotationCollection: [{}], annotations: [{}], collectionInheritableAnnotations: [{}], declaredAnnotations: [{}], description: Annotated value: jtaylor; Annotations: [@com.atlassian.velocity.htmlsafe.HtmlSafe()]} Julian Taylor added a comment - runtime SOCK_CLOEXEC check, based on 2.1.10
            Hide
            {annotationCollection: [{}], annotations: [{}], collectionInheritableAnnotations: [{}], declaredAnnotations: [{}], description: Annotated value: jtaylor; Annotations: [@com.atlassian.velocity.htmlsafe.HtmlSafe()]} Julian Taylor added a comment -

            yes you can close it fixed if you don't want to use a runtime check.
            I have attached the patch I'm going to apply to ubuntu (based on 2.1.10)

            Show
            {annotationCollection: [{}], annotations: [{}], collectionInheritableAnnotations: [{}], declaredAnnotations: [{}], description: Annotated value: jtaylor; Annotations: [@com.atlassian.velocity.htmlsafe.HtmlSafe()]} Julian Taylor added a comment - yes you can close it fixed if you don't want to use a runtime check. I have attached the patch I'm going to apply to ubuntu (based on 2.1.10)
            Hide
            {annotationCollection: [{}], annotations: [{}], collectionInheritableAnnotations: [{}], declaredAnnotations: [{}], description: Annotated value: mato; Annotations: [@com.atlassian.velocity.htmlsafe.HtmlSafe()]} Martin Lucina added a comment -

            Thanks for the patch, are you sure you want this commented out?

            @@ -201,7 +207,8 @@ int zmq::open_socket (int domain_, int t
            // between socket creation and this point).
            #if !defined SOCK_CLOEXEC && defined FD_CLOEXEC
            int rc = fcntl (s, F_SETFD, FD_CLOEXEC);

            • errno_assert (rc != -1);
              + /* ignore to be able to run build testsuites on hardy builders
              + errno_assert (rc != -1); */
              #endif
            Show
            {annotationCollection: [{}], annotations: [{}], collectionInheritableAnnotations: [{}], declaredAnnotations: [{}], description: Annotated value: mato; Annotations: [@com.atlassian.velocity.htmlsafe.HtmlSafe()]} Martin Lucina added a comment - Thanks for the patch, are you sure you want this commented out? @@ -201,7 +207,8 @@ int zmq::open_socket (int domain_, int t // between socket creation and this point). #if !defined SOCK_CLOEXEC && defined FD_CLOEXEC int rc = fcntl (s, F_SETFD, FD_CLOEXEC); errno_assert (rc != -1); + /* ignore to be able to run build testsuites on hardy builders + errno_assert (rc != -1); */ #endif
            Hide
            {annotationCollection: [{}], annotations: [{}], collectionInheritableAnnotations: [{}], declaredAnnotations: [{}], description: Annotated value: jtaylor; Annotations: [@com.atlassian.velocity.htmlsafe.HtmlSafe()]} Julian Taylor added a comment -

            there is nothing one can further do when there is no CLOEXEC. A socket without CLOEXEC is better than none at all.
            The #ifdef will never be true on ubuntu systems with zeromq anyway.

            Show
            {annotationCollection: [{}], annotations: [{}], collectionInheritableAnnotations: [{}], declaredAnnotations: [{}], description: Annotated value: jtaylor; Annotations: [@com.atlassian.velocity.htmlsafe.HtmlSafe()]} Julian Taylor added a comment - there is nothing one can further do when there is no CLOEXEC. A socket without CLOEXEC is better than none at all. The #ifdef will never be true on ubuntu systems with zeromq anyway.
            Hide
            {annotationCollection: [{}], annotations: [{}], collectionInheritableAnnotations: [{}], declaredAnnotations: [{}], description: Annotated value: mato; Annotations: [@com.atlassian.velocity.htmlsafe.HtmlSafe()]} Martin Lucina added a comment -

            Closing as fixed since Ubuntu guys will use their own downstream patch

            Show
            {annotationCollection: [{}], annotations: [{}], collectionInheritableAnnotations: [{}], declaredAnnotations: [{}], description: Annotated value: mato; Annotations: [@com.atlassian.velocity.htmlsafe.HtmlSafe()]} Martin Lucina added a comment - Closing as fixed since Ubuntu guys will use their own downstream patch

              People

              • Assignee:
                {annotationCollection: [{}], annotations: [{}], collectionInheritableAnnotations: [{}], declaredAnnotations: [{}], description: Annotated value: sustrik; Annotations: [@com.atlassian.velocity.htmlsafe.HtmlSafe()]} Martin Sustrik
                Reporter:
                {annotationCollection: [{}], annotations: [{}], collectionInheritableAnnotations: [{}], declaredAnnotations: [{}], description: Annotated value: kobolog; Annotations: [@com.atlassian.velocity.htmlsafe.HtmlSafe()]} Andrey Sibiryov
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: