parse-datetime.c generated in the source (instead of build) directory

[Kamil Dudka]

parse-datetime.c generated out of parse-datetime.y ends up in the source
directory, instead of the build directory as one would expect. This was
introduced by the following commit:

http://git.savannah.gnu.org/gitweb/?p=gnulib.git;a=commitdiff;h=6c680191

Neither the comment, nor the change log entry explains why it is desired.
As far as I know, the build should work even if the source directory is
on a read-only file system.

Moreover, this oddity seems to cause problems to the debug info extractor
used by rpmbuild in Fedora:

https://bugzilla.redhat.com/1555079

Kamil

[Vivien Kraus]

Hello,

Distributing the generated sources is the default behaviour to expect
from automake; I don't really understand why it is needed to override
the .y.c rule... This behaviour permits you to configure, build,
install, check... the package without having bison installed. Since
the generated .c is distributed and does not depend on the
configuration (this is the definition of a source file for automake),it will never change if the source directory is read-only.

You can check that this is actually the case by running "make
distcheck" in an automake project: it will make a source distribution,
extract it in a read-only directory, and build it. If bison needs to
write in the read-only directory, then it means the source distribution
is broken.

Sorry for my english, I hope it can help.

Vivien

Post by Kamil Dudka
parse-datetime.c generated out of parse-datetime.y ends up in the
source 
directory, instead of the build directory as one would expect.  This
was 
http://git.savannah.gnu.org/gitweb/?p=gnulib.git;a=commitdiff;h=6c680
191
Neither the comment, nor the change log entry explains why it is desired.
As far as I know, the build should work even if the source directory is
on a read-only file system.
Moreover, this oddity seems to cause problems to the debug info extractor
https://bugzilla.redhat.com/1555079
Kamil

[Vivien Kraus]

Sorry, I have just checked and automake does not distribute the
generated source by default, thus the modified .y.c rule is required.

Post by Vivien Kraus
Hello,
Distributing the generated sources is the default behaviour to expect
from automake; I don't really understand why it is needed to override
the .y.c rule...  This behaviour permits you to configure, build,
install, check... the package without having bison installed.  Since
the generated .c is distributed and does not depend on the
configuration (this is the definition of a source file for
automake),it will never change if the source directory is read-
only.  
You can check that this is actually the case by running "make
distcheck" in an automake project: it will make a source
distribution,
extract it in a read-only directory, and build it.  If bison needs to
write in the read-only directory, then it means the source
distribution
is broken.
Sorry for my english, I hope it can help.
Vivien

Post by Kamil Dudka
parse-datetime.c generated out of parse-datetime.y ends up in the
source 
directory, instead of the build directory as one would
expect.  This
was 
http://git.savannah.gnu.org/gitweb/?p=gnulib.git;a=commitdiff;h=6c6
80
191
Neither the comment, nor the change log entry explains why it is desired.
As far as I know, the build should work even if the source
directory
is
on a read-only file system.
Moreover, this oddity seems to cause problems to the debug info extractor
https://bugzilla.redhat.com/1555079
Kamil

[Paul Eggert]

Post by Kamil Dudka
the build should work even if the source directory is
on a read-only file system.

That's the first I've heard of that constraint, at least for the case where one
is building from the Git repository. (The problem you describe should not occur
when building from a distribution tarball.)

[Bruno Haible]

Post by Kamil Dudka
parse-datetime.c generated out of parse-datetime.y ends up in the source
directory, instead of the build directory as one would expect. This was
http://git.savannah.gnu.org/gitweb/?p=gnulib.git;a=commitdiff;h=6c680191
Neither the comment, nor the change log entry explains why it is desired.

For gnulib, you sometimes need to look up in the mailing list archive to get
the complete discussion. In this case, the essential mail was
https://lists.gnu.org/archive/html/bug-gnulib/2010-04/msg00488.html
which referenced
https://lists.gnu.org/archive/html/bug-gnulib/2009-05/msg00061.html

Post by Kamil Dudka
As far as I know, the build should work even if the source directory is
on a read-only file system.

... *only* if the source directory contains a complete, unmodified tarballs
contents. But when a .y.c rule needs to be run, you must have modified the
.y file, or the timestamps are wrong. You can't build off a read-only source
directory in this case.

The most important use-case of Makefiles are the developer's working directory.
The use-case with a read-only source directory is only a corner case; in case
of conflict with the important use-case, it's the important use-case which wins.

Post by Kamil Dudka
Moreover, this oddity seems to cause problems to the debug info extractor
https://bugzilla.redhat.com/1555079

I don't know how the debug info extractor works, but you make it sound like
it would ignore source files in $srcdir - which would not be right.

Bruno

[Kamil Dudka]

Post by Bruno Haible

Post by Kamil Dudka
parse-datetime.c generated out of parse-datetime.y ends up in the source
directory, instead of the build directory as one would expect. This was
http://git.savannah.gnu.org/gitweb/?p=gnulib.git;a=commitdiff;h=6c680191
Neither the comment, nor the change log entry explains why it is desired.

For gnulib, you sometimes need to look up in the mailing list archive to get
the complete discussion.

Ideally yes. But the commit contains no reference to that discussion and the
discussion contains no reference to that commit. Are you expecting people to
look it up by date and/or by names of the people who suggested it?

Post by Bruno Haible
In this case, the essential mail was
https://lists.gnu.org/archive/html/bug-gnulib/2010-04/msg00488.html
which referenced
https://lists.gnu.org/archive/html/bug-gnulib/2009-05/msg00061.html

Post by Kamil Dudka
As far as I know, the build should work even if the source directory is
on a read-only file system.

... *only* if the source directory contains a complete, unmodified tarballs
contents. But when a .y.c rule needs to be run, you must have modified the
.y file, or the timestamps are wrong. You can't build off a read-only source
directory in this case.

OK. I did not realise that the distribution tarball already contained bison's
output file. In fact, bison does not run during the build of coreutils in
Fedora. But even if I touch lib/parse-datetime.y to force bison to run, it
has no effect on the missing debuginfo.

Post by Bruno Haible
The most important use-case of Makefiles are the developer's working
directory. The use-case with a read-only source directory is only a corner
case; in case of conflict with the important use-case, it's the important
use-case which wins.

Post by Kamil Dudka
Moreover, this oddity seems to cause problems to the debug info extractor
https://bugzilla.redhat.com/1555079

I don't know how the debug info extractor works,

Nor do I.

Post by Bruno Haible
but you make it sound like
it would ignore source files in $srcdir - which would not be right.

Other source files are captured just fine. The only problem is that it
expects parse-datetime.{c,y} to be in build directories, instead of the
source directory. It could be confused by the #line directives that bison
produces.

A full build log is available at:
https://kojipkgs.fedoraproject.org//packages/coreutils/8.29/6.fc29/data/logs/x86_64/build.log

The related part looks like this:

+ /usr/lib/rpm/find-debuginfo.sh -j6 --strict-build-id -m -i --build-id-seed 8.29-6.fc29 --unique-debug-suffix -8.29-6.fc29.x86_64 --unique-debug-src-base coreutils-8.29-6.fc29.x86_64 --run-dwz --dwz-low-mem-die-limit 10000000 --dwz-max-die-limit 110000000 -S debugsourcefiles.list /builddir/build/BUILD/coreutils-8.29
extracting debug info from /builddir/build/BUILDROOT/coreutils-8.29-6.fc29.x86_64/usr/sbin/chroot
extracting debug info from /builddir/build/BUILDROOT/coreutils-8.29-6.fc29.x86_64/usr/bin/coreutils.single
extracting debug info from /builddir/build/BUILDROOT/coreutils-8.29-6.fc29.x86_64/usr/libexec/coreutils/libstdbuf.so
extracting debug info from /builddir/build/BUILDROOT/coreutils-8.29-6.fc29.x86_64/usr/bin/stty
extracting debug info from /builddir/build/BUILDROOT/coreutils-8.29-6.fc29.x86_64/usr/libexec/coreutils/libstdbuf.so.single
[...]
extracting debug info from /builddir/build/BUILDROOT/coreutils-8.29-6.fc29.x86_64/usr/bin/expand
/usr/lib/rpm/sepdebugcrcfix: Updated 107 CRC32s, 0 CRC32s did match.
cpio: separate/parse-datetime.c: Cannot stat: No such file or directory
cpio: separate/parse-datetime.y: Cannot stat: No such file or directory
cpio: single/parse-datetime.c: Cannot stat: No such file or directory
cpio: single/parse-datetime.y: Cannot stat: No such file or directory
10371 blocks


If I hard-link parse-datetime.{c,y} from the source directory to the build
directories before the debug info extractor is started, it seems to work as
expected. Those files are included in the debuginfo package and gdb is able
to find them when the debuginfo package is installed.

Kamil

Post by Bruno Haible
Bruno

[Bruno Haible]

Post by Kamil Dudka

Post by Bruno Haible
For gnulib, you sometimes need to look up in the mailing list archive to get
the complete discussion.

Ideally yes. But the commit contains no reference to that discussion and the
discussion contains no reference to that commit. Are you expecting people to
look it up by date and/or by names of the people who suggested it?

I usually do the lookup by date. The date of the discussion is usually the
date of the commit (minus up to 3 days).

I'm not a fan of putting discussion references into git commits, because they
may change each time the infrastructure changes. For example, references to
gmane threads became useless last year, and references to sourceforge mailing
list posts also were invalidated at some point in the past.

Bruno

[Kamil Dudka]

Post by Bruno Haible

Post by Kamil Dudka

Post by Bruno Haible
For gnulib, you sometimes need to look up in the mailing list archive to
get the complete discussion.

Ideally yes. But the commit contains no reference to that discussion and
the discussion contains no reference to that commit. Are you expecting
people to look it up by date and/or by names of the people who suggested
it?

I usually do the lookup by date. The date of the discussion is usually the
date of the commit (minus up to 3 days).
I'm not a fan of putting discussion references into git commits, because
they may change each time the infrastructure changes. For example,
references to gmane threads became useless last year, and references to
sourceforge mailing list posts also were invalidated at some point in the
past.
Bruno

For me it would be invaluably useful if the commit message told me _why_ the
change was made (with or without reference). It was pushed to fix something,
wasn't it?

The fact that you are adding a rule for getdate.c of modules/getdate to
Makefile.am and augmenting MOSTLYCLEANFILES is already obvious from the
corresponding diff. Duplicating the info in the commit message is, in my
opinion, less useful than referencing discussion with a temporarily valid
URL.

Kamil

[Bruno Haible]

Post by Kamil Dudka
Other source files are captured just fine. The only problem is that it
expects parse-datetime.{c,y} to be in build directories, instead of the
source directory. It could be confused by the #line directives that bison
produces.

Indeed, that could be. OTOH, tools like bison and gperf should *not* put
relative directories into #line directives, because such relative directories
would often depend on whether it's a VPATH build or not, and when the tool
was run on a Windows system, the relative directory might contain backslashes
as directory separator characters.

So, I think that tools that interpret debugging information should have a
way to specify a set of source directories. Like gdb does.

Bruno