Hello,
There seems to be a race condition in Makefile.libs that occurs sporadically when doing parallel builds (make -j). I believe the culprit part is this:
$(NAME): $(LIB_RUNTIME_NAME) $(LIB_LINK_NAME) $(LIBINAME_F)
$(LIB_RUNTIME_NAME): -@ln -s $(LIB_NAME) $(LIB_RUNTIME_NAME)
$(LIB_LINK_NAME): ifeq ($(OS), freebsd) -@ln -s $(LIB_RUNTIME_NAME) $(LIB_LINK_NAME) else -@ln -s $(LIB_NAME) $(LIB_LINK_NAME) endif
The problem is that dependencies (the ln -s commands) get built first, and the actual target ($(NAME)) last. The result is that for a while, the symlinks already exist while the lib itself is not fully built yet. In the meantime, other make children build something else that depends on the lib, see that the symlink already exists and hence think the target has already been built, which then results in various funny errors when the linker tries to read a non-existent or half-finished .so file. At least I think that's what happens.
Unfortunately I have no idea how to fix this within the current makefile framework. I tried reordering the dependencies (making the symlink targets depend on the lib instead of the other way around) but those all resulted in build failures.
cheers
Hello,
haven't looked at it yet, just wondering if the symlink cannot be created as extra command in the make target building the object.
Cheers, Daniel
On 1/26/13 12:30 AM, Richard Fuchs wrote:
Hello,
There seems to be a race condition in Makefile.libs that occurs sporadically when doing parallel builds (make -j). I believe the culprit part is this:
$(NAME): $(LIB_RUNTIME_NAME) $(LIB_LINK_NAME) $(LIBINAME_F)
$(LIB_RUNTIME_NAME): -@ln -s $(LIB_NAME) $(LIB_RUNTIME_NAME)
$(LIB_LINK_NAME): ifeq ($(OS), freebsd) -@ln -s $(LIB_RUNTIME_NAME) $(LIB_LINK_NAME) else -@ln -s $(LIB_NAME) $(LIB_LINK_NAME) endif
The problem is that dependencies (the ln -s commands) get built first, and the actual target ($(NAME)) last. The result is that for a while, the symlinks already exist while the lib itself is not fully built yet. In the meantime, other make children build something else that depends on the lib, see that the symlink already exists and hence think the target has already been built, which then results in various funny errors when the linker tries to read a non-existent or half-finished .so file. At least I think that's what happens.
Unfortunately I have no idea how to fix this within the current makefile framework. I tried reordering the dependencies (making the symlink targets depend on the lib instead of the other way around) but those all resulted in build failures.
cheers
sr-dev mailing list sr-dev@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-dev
Hi,
Actually I've looked into this further in the meantime and it seems that the actual cause of the problem is elsewhere. It seems to be a basic problem of how modules are made to depend on libs.
For example, avpops and lcr both depend on kcore. Their makefiles specify this through:
SER_LIBS+=$(SERLIBPATH)/kcore/kcore
which Makefile.rules (line 164) translates into
SER_LIBS_DEPS=$(SERLIBPATH)/kcore/libkcore.so
which in turn gets the following rule and recipe a few lines below:
$(SER_LIBS_DEPS): FORCE @$(MAKE) -wC $(dir $@) ..........
So what happens during a parallel build is this: Make's thread A starts compiling avpops while make's thread B starts compiling lcr at the same time. Both make threads see the dependency on libkcore.so and thus execute the sub-make to build it. Which means that you now have two make children running at the same time in the same directory, trying to compile the same target. Obviously they get into each others way.
I'm not sure if the make children are supposed to communicate with each other and if one is supposed to know that the other one is compiling kcore already or not. Taking out the FORCE dependency seems to help a little, but it's still ending up with multiple makes trying to build the same targets.
I'm still playing this this, trying to figure out if there's some make flags that will fix it or if it's maybe a fundamental problem of make and/or the way the makefiles are made.
cheers
On 01/30/13 07:15, Daniel-Constantin Mierla wrote:
Hello,
haven't looked at it yet, just wondering if the symlink cannot be created as extra command in the make target building the object.
Cheers, Daniel
On 1/26/13 12:30 AM, Richard Fuchs wrote:
Hello,
There seems to be a race condition in Makefile.libs that occurs sporadically when doing parallel builds (make -j). I believe the culprit part is this:
$(NAME): $(LIB_RUNTIME_NAME) $(LIB_LINK_NAME) $(LIBINAME_F)
$(LIB_RUNTIME_NAME): -@ln -s $(LIB_NAME) $(LIB_RUNTIME_NAME)
$(LIB_LINK_NAME): ifeq ($(OS), freebsd) -@ln -s $(LIB_RUNTIME_NAME) $(LIB_LINK_NAME) else -@ln -s $(LIB_NAME) $(LIB_LINK_NAME) endif
The problem is that dependencies (the ln -s commands) get built first, and the actual target ($(NAME)) last. The result is that for a while, the symlinks already exist while the lib itself is not fully built yet. In the meantime, other make children build something else that depends on the lib, see that the symlink already exists and hence think the target has already been built, which then results in various funny errors when the linker tries to read a non-existent or half-finished .so file. At least I think that's what happens.
Unfortunately I have no idea how to fix this within the current makefile framework. I tried reordering the dependencies (making the symlink targets depend on the lib instead of the other way around) but those all resulted in build failures.
cheers
sr-dev mailing list sr-dev@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-dev
-- Daniel-Constantin Mierla - http://www.asipto.com http://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda Kamailio World Conference, April 16-17, 2013, Berlin
sr-dev mailing list sr-dev@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-dev
-
Daniel-Constantin Mierla -
Richard Fuchs