gnulib-tool: conditional dependencies calculation

[Dmitry Selyutin]

During implementing support for --conditional-dependencies option I've found a
strange behavior, which makes me thinking that I don't quite get how module's
conditional state is being calculated. Let's consider the following gnulib-
tool invocation:

gnulib-tool --with-tests --conditional-dependencies --no-changelog --aux-dir
build-aux --doc-base doc --lib libgnu --m4-base m4/ --source-base lib/ --
tests-base tests --local-dir gl --makefile-name=gnulib.mk --libtool --import
fnmatch vasnprintf

Upon executing the same command with pygnulib.py from "python" branch, I
clearly see the difference between way how modules are treat as conditional.
For example, module configmake is considered to be conditional by pygnulib.py
and unconditional by gnulib-tool. Here are all the possible paths which lead
to configmake module, collected via TransitiveClosure.paths("configmake")
invocation):

('configmake', 'localcharset', 'mbrtowc', 'mbsinit-tests')
('configmake', 'localcharset', 'mbrtowc', 'mbsrtowcs-tests')
('configmake', 'localcharset', 'mbrtowc', 'mbsinit', 'mbrtowc-tests')
('configmake', 'localcharset', 'mbrtowc', 'mbsinit', 'fnmatch')
('configmake', 'localcharset', 'mbrtowc', 'mbsinit', 'mbsrtowcs-tests')
('configmake', 'localcharset', 'mbrtowc', 'mbsinit', 'wcrtomb', 'wctomb',
'wctob', 'mbrtowc-tests')
('configmake', 'localcharset', 'mbrtowc', 'mbsinit', 'wcrtomb', 'wctomb',
'wctob', 'mbsrtowcs-tests')
('configmake', 'localcharset', 'mbrtowc', 'mbsrtowcs', 'fnmatch')
('configmake', 'localcharset', 'mbrtowc', 'mbtowc', 'btowc', 'wcrtomb-tests')

The relationship between configmake and localcharset is an unconditional
dependency; however, localcharset is listed as a conditional dependency in
mbrtowc "Depends-on" section, and thus it seems reasonable that not only
localcharset is a conditional dependency, but all modules required by it.

No other module seem to require localcharset except of mbrtowc.
1) mbrtowc is the only one module which requires localcharset;
2) mbrtowc requires localcharset conditionally;
3) all the dependencies of localcharset shall be considered conditional.

Is the following assumption correct and I've found a bug in gnulib-tool, or am
I misunderstanding the whole concept of conditional dependencies? I'd be very
grateful if someone could help me with this issue, currently it is the last
part of the support for --conditional-dependencies in Python.

Thank you!

[Bruno Haible]

Hi Dmitry,

Post by Dmitry Selyutin
gnulib-tool --with-tests --conditional-dependencies --no-changelog --aux-dir
build-aux --doc-base doc --lib libgnu --m4-base m4/ --source-base lib/ --
tests-base tests --local-dir gl --makefile-name=gnulib.mk --libtool --import
fnmatch vasnprintf
... module configmake is considered to be conditional by pygnulib.py
and unconditional by gnulib-tool.

When I attempt to reproduce it, the module 'configmake' is considered conditional
by gnulib-tool - just like you and I would expect.

What I did:
1. Unpack the attached hello-c.tar.gz (a trivial Autoconf/Automake package).
2. Ran
$ ../gnulib-git/gnulib-tool --with-tests --conditional-dependencies --no-changelog --aux-dir build-aux --doc-base doc --lib libgnu --m4-base m4/ --source-base lib/ --tests-base tests --local-dir gl --makefile-name=gnulib.mk --libtool --import fnmatch vasnprintf
3. Looked at m4/gnulib-comp.m4.

It contains

func_gl_gnulib_m4code_configmake ()
{
if ! $gl_gnulib_enabled_configmake; then
gl_CONFIGMAKE_PREP
gl_gnulib_enabled_configmake=true
fi
}
...
func_gl_gnulib_m4code_localcharset ()
{
if ! $gl_gnulib_enabled_localcharset; then
gl_LOCALCHARSET
LOCALCHARSET_TESTS_ENVIRONMENT="CHARSETALIASDIR=\"\$(abs_top_builddir)/$gl_source_base\""
AC_SUBST([LOCALCHARSET_TESTS_ENVIRONMENT])
gl_gnulib_enabled_localcharset=true
func_gl_gnulib_m4code_configmake
fi
}

I'm using bash 4.3.48 and sed 4.2.2.

So, it could be a gnulib-tool bug, that occurs only in your environment and not in mine.

Bruno

[Dmitry Selyutin]

Hi Bruno,

thank you for answer! It seems that I was confused by the following entry in
`diff -ur dir1 dir2` output:

@@ -123,8 +124,8 @@

BUILT_SOURCES += configmake.h
CLEANFILES += configmake.h configmake.h-t
-endif

+endif

I've totally missed the last line and noticed only that line with endif
disappeared, but from what I see it is present, just with an additional new
line before it. Sorry for the noise on it; this example was simply incorrect.

However, there are other discrepancies which I observe. Let's consider another
example which were run on GNU hello:

./gnulib/pygnulib.py --with-tests --conditional-dependencies --no-changelog
--aux-dir build-aux --doc-base doc --lib libgnu --m4-base m4/ --source-base
lib/ --tests-base tests --local-dir gl --makefile-name=gnulib.mk --libtool --
import getaddrinfo

./gnulib/gnulib-tool --with-tests --conditional-dependencies --no-changelog
--aux-dir build-aux --doc-base doc --lib libgnu --m4-base m4/ --source-base
lib/ --tests-base tests --local-dir gl --makefile-name=gnulib.mk --libtool --
import getaddrinfo

Here is the supposed chain of dependencies for "float" module (conditions are
below, each condition represents a condition between two modules):

('float', 'vasnprintf', 'snprintf', 'getaddrinfo')
[None, 'test $ac_cv_func_snprintf = no || test $REPLACE_SNPRINTF = 1',
'test $HAVE_GETADDRINFO = 0']

From my perspective it seems that "float" module should have been considered
conditional; but this is not the case, running `diff -ur hello-py hello-sh`
yields the following:

## begin gnulib module float

-if gl_GNULIB_ENABLED_float
BUILT_SOURCES += $(FLOAT_H)

# We need the following in order to create <float.h> when the system
@@ -180,8 +178,9 @@
rm -f $@
endif
MOSTLYCLEANFILES += float.h float.h-t
-endif
+
EXTRA_DIST += float.c float.in.h itold.c
+
EXTRA_libgnu_la_SOURCES += float.c itold.c

## end gnulib module float


There are also other modules, which appear to be unconditional in gnulib-tool,
but conditional in pygnulib; the other way incompatibilities also occur. I've
listed the supposed dependencies below. The [C] means that module was supposed
to be conditional by pygnulib, but not by gnulib-tool; [U] means that module
was supposed to be unconditional by pygnulib, but gnulib-tool decided that it
is conditional.

alloca-opt [C]
('alloca-opt', 'vasnprintf', 'snprintf', 'getaddrinfo')
[None, 'test $ac_cv_func_snprintf = no || test $REPLACE_SNPRINTF = 1', 'test
$HAVE_GETADDRINFO = 0']

float [C]
('float', 'vasnprintf', 'snprintf', 'getaddrinfo')
[None, 'test $ac_cv_func_snprintf = no || test $REPLACE_SNPRINTF = 1', 'test
$HAVE_GETADDRINFO = 0']

msvc-inval [U]
('msvc-inval', 'fwrite-tests', 'stdio-tests')
[None, None]
('msvc-inval', 'fread-tests', 'stdio-tests')
[None, None]
('msvc-inval', 'fgetc-tests', 'stdio-tests')
[None, None]
('msvc-inval', 'fputc-tests', 'stdio-tests')
[None, None]
('msvc-inval', 'msvc-nothrow', 'sockets', 'getaddrinfo')
[None, None, 'test $HAVE_GETADDRINFO = 0']
('msvc-inval', 'fdopen', 'fwrite-tests', 'stdio-tests')
['test $REPLACE_FDOPEN = 1', None, None]
('msvc-inval', 'fdopen', 'fread-tests', 'stdio-tests')
['test $REPLACE_FDOPEN = 1', None, None]
('msvc-inval', 'fdopen', 'fgetc-tests', 'stdio-tests')
['test $REPLACE_FDOPEN = 1', None, None]
('msvc-inval', 'fdopen', 'fputc-tests', 'stdio-tests')
['test $REPLACE_FDOPEN = 1', None, None]

stdio [U]
('stdio', 'snprintf', 'getaddrinfo')
[None, 'test $HAVE_GETADDRINFO = 0']
('stdio', 'fdopen', 'fputc-tests', 'stdio-tests')
[None, None, None]
('stdio', 'fdopen', 'fread-tests', 'stdio-tests')
[None, None, None]
('stdio', 'fdopen', 'fgetc-tests', 'stdio-tests')
[None, None, None]
('stdio', 'fdopen', 'fwrite-tests', 'stdio-tests')
[None, None, None]

unistd [U]
('unistd', 'fgetc-tests', 'stdio-tests')
[None, None]
('unistd', 'fread-tests', 'stdio-tests')
[None, None]
('unistd', 'fwrite-tests', 'stdio-tests')
[None, None]
('unistd', 'fputc-tests', 'stdio-tests')
[None, None]
('unistd', 'getpagesize', 'memchr-tests')
[None, None]
('unistd', 'fd-hook', 'sockets', 'getaddrinfo')
[None, None, 'test $HAVE_GETADDRINFO = 0']


I guess that the problem lies somewhere inside module.py, in __init__ method
of TransitiveClosure class, but I cannot understand what's wrong. I suspect
that the whole issue is somehow related to handling of the "-tests"-related
modules.
Could you please take a look on it? I've been dealing with this issue for
several evenings and yet no result. :-( Thank you!