Gnulib on Windows (native / mingw32) / VMS / etc.

[Paul Smith]

Hi all. I spent a bit of time this weekend looking into what it would
take to update GNU make to use the glob/fnmatch implementations from
the current gnulib, rather than the ancient (and buggy) implementation
that GNU make has embedded, more or less unchanged, for at least 25
years.

This, unfortunately, turns out to be quite a daunting task.

Gnulib glob/fnmatch rely on a huge number of other gnulib packages so
instead of the simple two-file implementation GNU make has now it
expands to a lib directory containing 20+ files (I didn't actually
count but it's quite a lot).

This matters for a few reasons: first, GNU make provides a
bootstrapping script that will let you compile make on systems which
don't already have make... that means that I need to be able to build
all these extra files without the assistance of automake (I do run
configure). I'm not sure how easy that will be.

More worrying, GNU make is compile-able on Windows and does not require
Cygwin: it can be built using native MSVC and/or MingW32. For
bootstrapping GNU make provides a "build_w32.bat" file that compiles
make using either gcc or MSVC. In this setup I have a hard-coded
config.h for Windows that I install since I don't run configure.
Making this work with the many packages added to lib seems complex.

Finally, GNU make can also be compiled on other less common
architectures, such as VMS and these ports are quite active. Is there
any information about all these different glob/fnmatch requirements in
gnulib, insofar as portability to other environments like VMS?

I would really like to use native gnulib implementations but the
overhead involved has me considering whether it would be feasible to
try to extract the latest implementation and somehow fencing it in to
avoid all these extra dependencies.

Any comments anyone has would be welcome, cheers!

[Paul Eggert]

This post might be inappropriate. Click to display it.

[Eli Zaretskii]

Date: Wed, 11 Apr 2018 13:22:32 -0700
Except for VMS, GNU Emacs is in a boat similar to GNU Make. It uses
Gnulib (but not Automake), it is buildable on MS-Windows and does not
need Cygwin, it has its own configuration batch script for MS-Windows
(and MS-DOS!).

Actually, Emacs on Windows uses the same configure script used on
Posix hosts, it stopped using its own batch file several years ago.
(The MS-DOS build does still use a batch file and a bunch of Sed
scripts.) For that, the Windows build requires a working installation
of MSYS, which is a fork of Cygwin specifically tailored to building
MinGW programs using the Posix configury.

So I would look to Emacs for inspiration here. I'm mostly
responsible for the Emacs Gnulib usage and can help and/or answer
questions in this area.

First, the job of switching Emacs on Windows to using the normal
configury was nowhere near trivial, it required some adaptations,
mainly due to the fact that Emacs provides its own emulations of Posix
APIs, which the configure script cannot detect, for obvious reasons.
I'm guessing this will be a much smaller job in the case of Make, but
it still will need to be done.

More importantly, Make is somewhat special: it is one of a couple of
"basic" utilities which need to be built without a functional Posix
environment, in order to "bootstrap" the development environment.
This is why Make has simple build scripts for all supported systems,
not just on Windows. It's just that on Windows no one bothered to
provide or support the alternative of using Autoconf, that's all.
Since Make is a much smaller program than, say, Emacs, and its
development rate is much slower, the batch file is "good enough": it
and the related config.h file need changes only rarely.

And I don't think MSYS supports MSVC, at least not out of the box.

For these reasons, I wouldn't suggest switching to MSYS/configure kind
of build unless all the other alternatives are much worse. Make is a
different package, with different needs and very different scale, so
the Emacs experience is not necessarily exactly applicable to Make.

I didn't try comparing Gnulib's glob.c and fnmatch.c with what's now
in GNU Make -- are the differences really so significant as to
disallow any attempt of a simple incorporation of changes from Gnulib
in just those 2 files? If not, such a "merge" could do the same job
much easier, without requiring Paul Smith to import all the Gnulib
dependencies, switch to MSYS for the MS-Windows build, and do
something else for VMS.

[Paul Eggert]

Post by Eli Zaretskii
I didn't try comparing Gnulib's glob.c and fnmatch.c with what's now
in GNU Make -- are the differences really so significant as to
disallow any attempt of a simple incorporation of changes from Gnulib
in just those 2 files?

Certainly that sort of cherry-picking could be done. However, I expect
it'd be less work for everybody in the long run for GNU Make to use
Gnulib's code instead of continuing to maintain its old version. This
might require changing Gnulib to make it easier to decouple its glob.c
and fnmatch.c from the rest of Gnulib. We've done that for some of the
Gnulib modules that Emacs uses, and I expect we could do that for glob.c
and fnmatch.c too.

[Eli Zaretskii]

Date: Thu, 12 Apr 2018 13:18:28 -0700

Post by Eli Zaretskii
I didn't try comparing Gnulib's glob.c and fnmatch.c with what's now
in GNU Make -- are the differences really so significant as to
disallow any attempt of a simple incorporation of changes from Gnulib
in just those 2 files?

Certainly that sort of cherry-picking could be done. However, I expect
it'd be less work for everybody in the long run for GNU Make to use
Gnulib's code instead of continuing to maintain its old version.

I agree, but switching to MSYS build of Make on Windows definitely
doesn't qualify as less work for everybody in my book.

This might require changing Gnulib to make it easier to decouple its
glob.c and fnmatch.c from the rest of Gnulib. We've done that for
some of the Gnulib modules that Emacs uses, and I expect we could do
that for glob.c and fnmatch.c too.

I didn't realize such an option was on the table. If that's possible,
it is certainly a much better way forward.

Thanks.

[Paul Smith]

Post by Eli Zaretskii

This might require changing Gnulib to make it easier to decouple its
glob.c and fnmatch.c from the rest of Gnulib. We've done that for
some of the Gnulib modules that Emacs uses, and I expect we could do
that for glob.c and fnmatch.c too.

I didn't realize such an option was on the table. If that's
possible, it is certainly a much better way forward.

As a starting position I've modified the GNU make repository to change
the "src/glob" directory to be a top-level "lib" directory, and I've
imported alloca and getloadavg modules from gnulib to replace the old
versions previously provided with GNU make.

Things mostly still work even on Windows, with one small change to
getloadavg that I'll send under separate cover.

glob/fnmatch is a different story and I'm not sure quite how to tackle
it yet. So far I've just moved the original glob/fnmatch (old and
buggy) into lib/ to keep things working.

Here's what I find extremely difficult to manage regarding gnulib's
current design:

- Heavy reliance on .in files that need to be transformed.

On systems that aren't POSIX and don't have ready access to sed or
similar tools, this is very annoying to deal with.

The original design of autoconf, where there was a single config.h
which defined a ton of #define values that were then tested by other
files, was far simpler for non-traditional systems: you merely had to
create a pre-defined config.h file for such a target (most of the time
these targets were homogeneous enough across releases that they didn't
require autoconf anyway) and arrange to have the file renamed to start
the build process.

However with gnulib's current design for the replacement system headers
(at least) instead of using the old-style '#if !HAVE_BOOL' (as on
example from stdbool.in.h), they use:

# if !@HAVE__BOOL@

This means I can't just define HAVE_BOOL in config.h and then use this
header file, I need to come up with some way of emulating sed behavior
on all my supported platforms (Windows, VMS, etc.) so I can replace
these values. Or else I have to create per-system instances of each of
these files, of which I already have 5 just for alloca and getloadavg
and if I do take on glob/fnmatch that number will balloon.

For glob.in.h in gnulib it's even worse; even the header guards require
replacement!!

#ifndef ***@GUARD_PREFIX@_GLOB_H

I understand that the goal is to have versions of these standard header
files which can be used without config.h, but the GNU coding standards
suggest that config.h must be included first in each compilation unit,
before even system headers, so to me that desire doesn't outweigh the
downside of using this method for non-traditional systems.

I'm interested in anyone's opinion on how best to deal with this issue.

Cheers!

[Paul Eggert]

Post by Paul Smith
I understand that the goal is to have versions of these standard header
files which can be used without config.h, but the GNU coding standards
suggest that config.h must be included first in each compilation unit,
before even system headers

As I understand it, the Gnulib .h files in question are intended to be
useful even to programs that do not follow the GNU coding standards and
do not necessarily even have a config.h.

Here's a thought. Perhaps we could replace each instance of "@" in our
FOO.h.in files with "_aT_", and adjust all our other scripts
accordingly, except that (1) on systems lacking 'sed' the makefiles just
use 'cp' when creating FOO.h from FOO.h.in, and (2) we define the
corresponding *_aT_* symbols in config.h. That way, you'd get the
behavior you prefer even on development platforms lacking 'sed', and
people running on  platforms with 'sed' will continue to get .h files
that are useful to programs that do not follow the GNU coding standards.

[Bruno Haible]

Post by Paul Smith
Here's what I find extremely difficult to manage regarding gnulib's
- Heavy reliance on .in files that need to be transformed.
On systems that aren't POSIX and don't have ready access to sed or
similar tools, this is very annoying to deal with.
The original design of autoconf, where there was a single config.h
which defined a ton of #define values that were then tested by other
files, was far simpler for non-traditional systems: you merely had to
create a pre-defined config.h file for such a target (most of the time
these targets were homogeneous enough across releases that they didn't
require autoconf anyway) and arrange to have the file renamed to start
the build process.
However with gnulib's current design for the replacement system headers
(at least) instead of using the old-style '#if !HAVE_BOOL' (as on

I think you are barking up the wrong tree.

In the other mail I've explained how you can reduce the number of build
systems you use from 3 (Autoconf+Automake, Windows nmake, VMS nms) to
one (Autoconf+Automake).

The step that you underestimate here is "you merely had to
create a pre-defined config.h file for such a target". Gnulib-generated
header files (such as glob.h) use many more autoconf tests (such as
whether a function exists, is declared, has specific bugs). If someone
tries to create a config.h file by hand for an exotic platform, not
only will it be time-consuming (the gettext-tools config.h.in for example
has more than 400 results of configure findings), but also mistakes will
show up as compilation errors. (This is the downside of the many features
gnulib has:
- C++ support,
- support for many platforms,
- using the function name 'rpl_foo' if and only if 'foo' would collide
with system-provided 'foo'.
The downside is that wrong guesses for the HAVE_* symbols lead to
compilation failures more quickly.)

Really, the approach of maintaining a config.h for a particular platform
by hand is outdated.

Even if gnulib would accommodate your wish to have a special copy of
glob.in.h that uses HAVE_FOO instead of @HAVE_FOO@, that would not solve
the problem how to maintain the config.h for specific platforms.

Post by Paul Smith
On systems that aren't POSIX and don't have ready access to sed or
similar tools

On such systems the essential step is to create the set of tools that
are needed to be able to run configure scripts. This includes a POSIX-like
shell, the coreutils, grep, sed, and maybe awk.

I would argue that porters who are "active" but have not yet achieved this
step should do this first, instead of you (and GNU) spend time on supporting
them better by adding a second or third build system to your package.

I mean, nowadays, we are running DOS inside a browser, compiling C to
JavaScript, emulating m68k on x86, and so on. The task of making the
POSIX build tools work on a non-POSIX system is much smaller and very
achievable, especially given enough time (like for VMS).

And when new OSes appear, with a roughly ANSI C compliant libc, they
typically
- either have the POSIX build tools already ported,
- or are supported as a cross-compilation target.

Bruno

[Bruno Haible]

Post by Bruno Haible
The task of making the
POSIX build tools work on a non-POSIX system is much smaller and very
achievable, especially given enough time (like for VMS).

The VMS people apparently actually achieved this task; the binaries of
these tools are for download at https://sourceforge.net/projects/gnv/ .

So, the same approach as for porting to native Windows should also
work for VMS.

Bruno

[Bruno Haible]

Post by Bruno Haible
- C++ support,
- support for many platforms,
- using the function name 'rpl_foo' if and only if 'foo' would collide
with system-provided 'foo'.
The downside is that wrong guesses for the HAVE_* symbols lead to
compilation failures more quickly.)

All this is true, but Paul is concerned only by a few specific
platforms, and about a single programming language. So the above
features are mostly irrelevant in the case in point.

But we don't have two gnulibs: one with many features, and a separate one
with as few HAVE_* symbols as possible.

Post by Bruno Haible
Really, the approach of maintaining a config.h for a particular platform
by hand is outdated.

Nevertheless, GNU Make uses it to this day.

However, gnulib modules can add new HAVE_* symbols and configuration tests
at any moment, and without notice in the NEWS file.

If Paul is OK to update (or let his porters update) the config.h files
for native Windows, VMS, and so on, each time he upgrades to a newer
version of gnulib, then fine. If not, he either shouldn't use gnulib,
or he should change the way he works with config.h.

Bruno

[Eli Zaretskii]

Date: Tue, 15 May 2018 18:31:05 -0400
Here's what I find extremely difficult to manage regarding gnulib's
- Heavy reliance on .in files that need to be transformed.
On systems that aren't POSIX and don't have ready access to sed or
similar tools, this is very annoying to deal with.
The original design of autoconf, where there was a single config.h
which defined a ton of #define values that were then tested by other
files, was far simpler for non-traditional systems: you merely had to
create a pre-defined config.h file for such a target (most of the time
these targets were homogeneous enough across releases that they didn't
require autoconf anyway) and arrange to have the file renamed to start
the build process.
However with gnulib's current design for the replacement system headers
(at least) instead of using the old-style '#if !HAVE_BOOL' (as on
This means I can't just define HAVE_BOOL in config.h and then use this
header file, I need to come up with some way of emulating sed behavior
on all my supported platforms (Windows, VMS, etc.) so I can replace
these values. Or else I have to create per-system instances of each of
these files, of which I already have 5 just for alloca and getloadavg
and if I do take on glob/fnmatch that number will balloon.

For the record: what are those 5 systems? MS-Windows is one, but what
are the others?

For glob.in.h in gnulib it's even worse; even the header guards require
replacement!!
I understand that the goal is to have versions of these standard header
files which can be used without config.h, but the GNU coding standards
suggest that config.h must be included first in each compilation unit,
before even system headers, so to me that desire doesn't outweigh the
downside of using this method for non-traditional systems.
I'm interested in anyone's opinion on how best to deal with this issue.

Here's an (100% untested) idea:

. We ignore all the Gnulib headers that work around problems in
standard headers, such as stdbool.h, stdint.h, etc. We assume the
corresponding system headers are "good enough". Where that
happens to be false, we will add the necessary stuff to the
config.h templates.

. For the other Gnulib headers and for config.h, we run the
configure script once, after importing those Gnulib headers, on
systems where the tools for that are available, be that a
cross-compiling environment or a native one. (I can volunteer to
do that for MS-Windows' MinGW port.) This only matters for Gnulib
headers that are processed at configure time; AFAICT, only glob.h
and fnmatch.h need that, since all the other Gnulib headers are
provided ready to be used, i.e. not as *.in.h templates.

. We then keep the produced config.h and other generated Gnulib
headers in the Make repository, for each affected system, in the
same way we did with config.h templates until now.

. The C sources of glob.c and fnmatch.c we import from Gnulib and
try not to make local changes, so that these sources will not need
any manual maintenance outside of the Gnulib project.

This idea assumes that glob.in.h and fnmatch.in.h will not change
significantly with time. Whenever they do change, we will need to
update the templates; hopefully the changes will be minor and we will
be able to do that manually, without resorting to the full configure
run.

WDYT?

[Paul Smith]

Post by Eli Zaretskii

Post by Paul Smith
these values. Or else I have to create per-system instances of
each of these files, of which I already have 5 just for alloca and
getloadavg and if I do take on glob/fnmatch that number will
balloon.

For the record: what are those 5 systems? MS-Windows is one, but
what are the others?

Sorry for not being clear: I meant 5 replacement header files that
contain @-tokens, not 5 supported target systems.

The systems we ostensibly support are POSIX/UNIX, Windows, VMS, MS-DOS
(?), and AmigaOS (?). I don't know if anyone actually uses either of
the two latter platforms anymore, but the last time I asked about
ditching Amiga support (for example) there were people who objected.
That was some years ago now though. I've never been entirely clear on
the difference between the DOS and Windows ports.

Regarding your suggestion, that might work. I'll look into it.

[Eli Zaretskii]

Date: Wed, 16 May 2018 12:53:22 -0400

Post by Eli Zaretskii

Post by Paul Smith
these values. Or else I have to create per-system instances of
each of these files, of which I already have 5 just for alloca and
getloadavg and if I do take on glob/fnmatch that number will
balloon.

For the record: what are those 5 systems? MS-Windows is one, but
what are the others?

Sorry for not being clear: I meant 5 replacement header files that

Ah, thanks for clarifications.

The systems we ostensibly support are POSIX/UNIX, Windows, VMS, MS-DOS
(?), and AmigaOS (?).

MS-DOS has glob and fnmatch in its library, so it probably doesn't
need those from Gnulib.

I don't know if anyone actually uses either of the two latter
platforms anymore, but the last time I asked about ditching Amiga
support (for example) there were people who objected. That was some
years ago now though.

"MS-DOS" is a misnomer in this case: it really means the DJGPP tools
(www.delorie.com/djgpp). And yes, that is still used, albeit by a
very small group of people. In particular, I use the DJGPP port of
GNU Make (and other DJGPP ports of GNU software) to build the "MS-DOS"
port of Emacs.

I've never been entirely clear on the difference between the DOS and
Windows ports.

They target 2 different operating systems. Because Windows versions
up to XP included either DOS itself or its good emulation, executables
that needed DOS services could run on Windows, and I believe that is
the source of your confusion. But since modern versions of Windows
are 64-bit, they no longer support running MS-DOS executables, and the
quality of MS-DOS subsystems on 32-bit versions of modern Windows
systems consistently deteriorates, so soon one will need a true DOS
system or a true emulator to run those.

[Bruno Haible]

Post by Eli Zaretskii
...
This idea assumes that glob.in.h and fnmatch.in.h will not change
significantly with time. Whenever they do change, we will need to
update the templates; hopefully the changes will be minor and we will
be able to do that manually

This sounds like a good compromise between Paul's requirements and the
wish to have little manual work.

When you implement it, take a look at the gnulib-tool options:
--avoid to avoid a module despite a dependency,
--copy-file to copy a single file if you don't want to slurp the
entire module it belongs to.

When you upgrade to a newer version of gnulib, you should look not only
at the changes that were made in glob.in.h but also those in glob.m4.

Bruno

[Bruno Haible]

Paul Smith wrote in

Post by Paul Smith
first, GNU make provides a
bootstrapping script that will let you compile make on systems which
don't already have make... that means that I need to be able to build
all these extra files without the assistance of automake (I do run
configure). I'm not sure how easy that will be.

To cope for this case, I would suggest to generate a "build all at once"
script from the generated Makefiles and config.status.
The user would then
- run "./configure --disable-dependency-tracking"
- execute the "build all at once" script generator,
- execute the resulting script.

The output of "make -n" on the GNU make directory is quite simple.
In the case of make-4.2.1, it is:
$ make -n
[lots of automake junk]
gcc -DLOCALEDIR=\"/usr/local/share/locale\" -DLIBDIR=\"/usr/local/lib\" -DINCLUDEDIR=\"/usr/local/include\" -DHAVE_CONFIG_H -I. -g -O2 -c -o ar.o ar.c
gcc -DLOCALEDIR=\"/usr/local/share/locale\" -DLIBDIR=\"/usr/local/lib\" -DINCLUDEDIR=\"/usr/local/include\" -DHAVE_CONFIG_H -I. -g -O2 -c -o arscan.o arscan.c
gcc -DLOCALEDIR=\"/usr/local/share/locale\" -DLIBDIR=\"/usr/local/lib\" -DINCLUDEDIR=\"/usr/local/include\" -DHAVE_CONFIG_H -I. -g -O2 -c -o commands.o commands.c
gcc -DLOCALEDIR=\"/usr/local/share/locale\" -DLIBDIR=\"/usr/local/lib\" -DINCLUDEDIR=\"/usr/local/include\" -DHAVE_CONFIG_H -I. -g -O2 -c -o default.o default.c
gcc -DLOCALEDIR=\"/usr/local/share/locale\" -DLIBDIR=\"/usr/local/lib\" -DINCLUDEDIR=\"/usr/local/include\" -DHAVE_CONFIG_H -I. -g -O2 -c -o dir.o dir.c
gcc -DLOCALEDIR=\"/usr/local/share/locale\" -DLIBDIR=\"/usr/local/lib\" -DINCLUDEDIR=\"/usr/local/include\" -DHAVE_CONFIG_H -I. -g -O2 -c -o expand.o expand.c
gcc -DLOCALEDIR=\"/usr/local/share/locale\" -DLIBDIR=\"/usr/local/lib\" -DINCLUDEDIR=\"/usr/local/include\" -DHAVE_CONFIG_H -I. -g -O2 -c -o file.o file.c
gcc -DLOCALEDIR=\"/usr/local/share/locale\" -DLIBDIR=\"/usr/local/lib\" -DINCLUDEDIR=\"/usr/local/include\" -DHAVE_CONFIG_H -I. -g -O2 -c -o function.o function.c
gcc -DLOCALEDIR=\"/usr/local/share/locale\" -DLIBDIR=\"/usr/local/lib\" -DINCLUDEDIR=\"/usr/local/include\" -DHAVE_CONFIG_H -I. -g -O2 -c -o getopt.o getopt.c
gcc -DLOCALEDIR=\"/usr/local/share/locale\" -DLIBDIR=\"/usr/local/lib\" -DINCLUDEDIR=\"/usr/local/include\" -DHAVE_CONFIG_H -I. -g -O2 -c -o getopt1.o getopt1.c
gcc -DLOCALEDIR=\"/usr/local/share/locale\" -DLIBDIR=\"/usr/local/lib\" -DINCLUDEDIR=\"/usr/local/include\" -DHAVE_CONFIG_H -I. -g -O2 -c -o guile.o guile.c
gcc -DLOCALEDIR=\"/usr/local/share/locale\" -DLIBDIR=\"/usr/local/lib\" -DINCLUDEDIR=\"/usr/local/include\" -DHAVE_CONFIG_H -I. -g -O2 -c -o implicit.o implicit.c
gcc -DLOCALEDIR=\"/usr/local/share/locale\" -DLIBDIR=\"/usr/local/lib\" -DINCLUDEDIR=\"/usr/local/include\" -DHAVE_CONFIG_H -I. -g -O2 -c -o job.o job.c
gcc -DLOCALEDIR=\"/usr/local/share/locale\" -DLIBDIR=\"/usr/local/lib\" -DINCLUDEDIR=\"/usr/local/include\" -DHAVE_CONFIG_H -I. -g -O2 -c -o load.o load.c
gcc -DLOCALEDIR=\"/usr/local/share/locale\" -DLIBDIR=\"/usr/local/lib\" -DINCLUDEDIR=\"/usr/local/include\" -DHAVE_CONFIG_H -I. -g -O2 -c -o loadapi.o loadapi.c
gcc -DLOCALEDIR=\"/usr/local/share/locale\" -DLIBDIR=\"/usr/local/lib\" -DINCLUDEDIR=\"/usr/local/include\" -DHAVE_CONFIG_H -I. -g -O2 -c -o main.o main.c
gcc -DLOCALEDIR=\"/usr/local/share/locale\" -DLIBDIR=\"/usr/local/lib\" -DINCLUDEDIR=\"/usr/local/include\" -DHAVE_CONFIG_H -I. -g -O2 -c -o misc.o misc.c
gcc -DLOCALEDIR=\"/usr/local/share/locale\" -DLIBDIR=\"/usr/local/lib\" -DINCLUDEDIR=\"/usr/local/include\" -DHAVE_CONFIG_H -I. -g -O2 -c -o posixos.o posixos.c
gcc -DLOCALEDIR=\"/usr/local/share/locale\" -DLIBDIR=\"/usr/local/lib\" -DINCLUDEDIR=\"/usr/local/include\" -DHAVE_CONFIG_H -I. -g -O2 -c -o output.o output.c
gcc -DLOCALEDIR=\"/usr/local/share/locale\" -DLIBDIR=\"/usr/local/lib\" -DINCLUDEDIR=\"/usr/local/include\" -DHAVE_CONFIG_H -I. -g -O2 -c -o read.o read.c
gcc -DLOCALEDIR=\"/usr/local/share/locale\" -DLIBDIR=\"/usr/local/lib\" -DINCLUDEDIR=\"/usr/local/include\" -DHAVE_CONFIG_H -I. -g -O2 -c -o remake.o remake.c
gcc -DLOCALEDIR=\"/usr/local/share/locale\" -DLIBDIR=\"/usr/local/lib\" -DINCLUDEDIR=\"/usr/local/include\" -DHAVE_CONFIG_H -I. -g -O2 -c -o rule.o rule.c
gcc -DLOCALEDIR=\"/usr/local/share/locale\" -DLIBDIR=\"/usr/local/lib\" -DINCLUDEDIR=\"/usr/local/include\" -DHAVE_CONFIG_H -I. -g -O2 -c -o signame.o signame.c
gcc -DLOCALEDIR=\"/usr/local/share/locale\" -DLIBDIR=\"/usr/local/lib\" -DINCLUDEDIR=\"/usr/local/include\" -DHAVE_CONFIG_H -I. -g -O2 -c -o strcache.o strcache.c
gcc -DLOCALEDIR=\"/usr/local/share/locale\" -DLIBDIR=\"/usr/local/lib\" -DINCLUDEDIR=\"/usr/local/include\" -DHAVE_CONFIG_H -I. -g -O2 -c -o variable.o variable.c
gcc -DLOCALEDIR=\"/usr/local/share/locale\" -DLIBDIR=\"/usr/local/lib\" -DINCLUDEDIR=\"/usr/local/include\" -DHAVE_CONFIG_H -I. -g -O2 -c -o version.o version.c
gcc -DLOCALEDIR=\"/usr/local/share/locale\" -DLIBDIR=\"/usr/local/lib\" -DINCLUDEDIR=\"/usr/local/include\" -DHAVE_CONFIG_H -I. -g -O2 -c -o vpath.o vpath.c
gcc -DLOCALEDIR=\"/usr/local/share/locale\" -DLIBDIR=\"/usr/local/lib\" -DINCLUDEDIR=\"/usr/local/include\" -DHAVE_CONFIG_H -I. -g -O2 -c -o hash.o hash.c
gcc -DLOCALEDIR=\"/usr/local/share/locale\" -DLIBDIR=\"/usr/local/lib\" -DINCLUDEDIR=\"/usr/local/include\" -DHAVE_CONFIG_H -I. -g -O2 -c -o remote-stub.o remote-stub.c
rm -f make
gcc -g -O2 -Wl,--export-dynamic -o make ar.o arscan.o commands.o default.o dir.o expand.o file.o function.o getopt.o getopt1.o guile.o implicit.o job.o load.o loadapi.o main.o misc.o posixos.o output.o read.o remake.o rule.o signame.o strcache.o variable.o version.o vpath.o hash.o remote-stub.o -ldl
[a bit more of automake junk]

So really this job can be done in a shell script that
- extracts the relevant file lists from config.status and the Makefiles
- uses them to write out the gcc commands in a loop.

Post by Paul Smith
More worrying, GNU make is compile-able on Windows and does not require
Cygwin: it can be built using native MSVC and/or MingW32. For
bootstrapping GNU make provides a "build_w32.bat" file that compiles
make using either gcc or MSVC. In this setup I have a hard-coded
config.h for Windows that I install since I don't run configure.
Making this work with the many packages added to lib seems complex.

This approach with "build_w32.bat" is outdated. The modern approach is
to use the Autoconf and Automake generated configure and Makefile.in
files without modifications.

More in detail: About ca. 10 years ago, Automake started to include
wrapper scripts (shell scripts) that make the MSVC compiler and archiver
('cl' and 'lib') be usable with the usual command-line options of a
'cc' and 'ar' program. From this point on, compilation for MSVC could
be done with just a Cygwin or MSYS build environment (that includes
bash, coreutils, grep, sed, and make).

The advantages for the package maintainer are big:
- No more need to care about two different build systems
(automake on one side, nmake on the other side).
- No more need to write .bat files. Shell scripts are universally
executable.
- In some places, no more need to worry about slash vs. backslash
as directory name separator.

The person who builds Windows binaries has more effort, though:
- Need use install and get familiar with Cygwin or MSYS first.
- Build times are higher, because of the wrapper script invocations
(but this is mitigated by the faster machines that we have today).

This approach is described in detail in
http://git.savannah.gnu.org/gitweb/?p=gettext.git;a=blob;f=INSTALL.windows

You can copy this file literally to other packages (I use the same file in
gperf, gettext, libunistring, libffcall, etc.).

Post by Paul Smith
Finally, GNU make can also be compiled on other less common
architectures, such as VMS and these ports are quite active.

I would suggest that these maintainers of these ports follow the approach
described above for Windows:
- Package a build environment with bash, grep, sed, make, that can be
easily installed.
- Create wrapper scripts for 'cc' and 'ar'.

Really, no one has time to maintain a 'Makefile.vms' like I did for some
time for GNU gperf:
http://git.savannah.gnu.org/gitweb/?p=gperf.git;a=blob;f=Makefile.vms

You really want to reduce the number of redundant Makefiles you have to
maintain to 1, as soon as possible.

Bruno

[Paul Smith]

Post by Bruno Haible
This approach with "build_w32.bat" is outdated. The modern approach is
to use the Autoconf and Automake generated configure and Makefile.in
files without modifications.
More in detail: About ca. 10 years ago, Automake started to include
wrapper scripts (shell scripts) that make the MSVC compiler and archiver
('cl' and 'lib') be usable with the usual command-line options of a
'cc' and 'ar' program. From this point on, compilation for MSVC could
be done with just a Cygwin or MSYS build environment (that includes
bash, coreutils, grep, sed, and make).

Unfortunately (perhaps only for me) this isn't sufficient.

GNU make is a foundational tool: you need it before you can build any
other package. It is also widely used by itself in environments where
no other POSIX-like or GNU tools are used (so, no MSYS etc.)

One could argue that since an exe for GNU make already exists for
Windows it's OK to rely on it to extract yourself from a catch-22
situation, but I'm not willing to relax that requirement.

My position continues to be that the build of GNU make on Windows (or
other environments) needs to be possible with only native tools: on
Windows this means command.com and a C compiler. No Cygwin, no MSYS,
no bash, no sed, no grep, and obviously no make (since that's what
we're trying to build).

Post by Bruno Haible
On such systems the essential step is to create the set of tools that
are needed to be able to run configure scripts. This includes a
POSIX-like shell, the coreutils, grep, sed, and maybe awk.

And how does one do this, without make itself?

Post by Bruno Haible
- No more need to care about two different build systems
(automake on one side, nmake on the other side).

GNU make doesn't (or won't, in the next release) support nmake builds,
or any other variation of make. They weren't used, so they have likely
been broken for a long time anyway.

Post by Bruno Haible
You really want to reduce the number of redundant Makefiles you have
to maintain to 1, as soon as possible.

The only makefiles the GNU make distribution will support will be
standard make makefiles. The model is:

If you have a configure-capable system, use autotools to compile GNU
make.

If you don't have a configure-capable system, use the provided
bootstrap script (or create your own) to build GNU make.

The build of GNU make is straightforward enough that using a script
isn't a big problem.

[Paul Eggert]

Post by Paul Smith
The only makefiles the GNU make distribution will support will be
If you have a configure-capable system, use autotools to compile GNU
make.
If you don't have a configure-capable system, use the provided
bootstrap script (or create your own) to build GNU make.

Such a bootstrap script should be able to use the approach I mentioned
earlier, by copying FOO.h.in files to FOO.h and replacing "@" with
"_GLat" or something similar, and then by augmenting config.h with the
appropriate #defines. If you can't assume 'sed', you can compile and run
a simple program that does the "@" replacement.

Alternatively, we could change Gnulib to use "_GLat" instead of "@"
uniformly, so that you don't need to do the "@" replacement in GNU Make;
but before doing that I'd like to know that the approach actually works
for GNU Make.

[Bruno Haible]

Post by Paul Eggert
If you can't assume 'sed', you can compile and run

Yes, given that the uses of 'sed' in gnulib are nearly all of the form
sed -e 's/@VAR1@/VALUE1/g' ... -e 's/@VARn@/VALUEn/g' FILE
you could write a program 'atsubst' such that
atsubst VAR1 VALUE1 ... VARn VALUEn FILE
is equivalent to
sed -e 's/@VAR1@/VALUE1/g' ... -e 's/@VARn@/VALUEn/g' FILE

This 'atsubst' program is simple enough that it fits in a single file
(like 'envsubst' in GNU gettext); it can be a prerequisite single command
for anyone who builds 'make'.
I don't think this would fully work. Regarding @INCLUDE_NEXT@ for example:
You can't have a C macro that expands to either #include or #include_next.

So, Paul would probably have to generate foo.halfsubsted.h from foo.in.h
where @GUARD_PREFIX@, @INCLUDE_NEXT@, @PRAGMA_SYSTEM_HEADER@, @PRAGMA_COLUMNS@
and @NEXT_FOO_H@ have been already substituted, and use C preprocessor
expansion (i.e. HAVE_FOO instead of @HAVE_FOO@) for the other symbols.

I would see this machinery as being in GNU make only (no modifications in
gnulib), unless/until the GNU bash, coreutils, grep, gawk, sed projects want
to use it as well.

Bruno

[Bruno Haible]

Hi Paul,

Post by Paul Smith

Post by Bruno Haible
You really want to reduce the number of redundant Makefiles you have
to maintain to 1, as soon as possible.

The only makefiles the GNU make distribution will support will be
If you have a configure-capable system, use autotools to compile GNU
make.
If you don't have a configure-capable system, use the provided
bootstrap script (or create your own) to build GNU make.
The build of GNU make is straightforward enough that using a script
isn't a big problem.

Using a script to compile the various .c files files in turn is easy.

The problem is to maintain the config.h file, if you don't want to
assume a configure-capable system.

Post by Paul Smith
My position continues to be that the build of GNU make on Windows (or
other environments) needs to be possible with only native tools: on
Windows this means command.com and a C compiler. No Cygwin, no MSYS,
no bash, no sed, no grep, and obviously no make (since that's what
we're trying to build).

What are the reasons for this position?

- You want users of Windows ME (from 2000) to be able to build 'make'?

- You want the process of building 'make' from source to be as simple
as possible, the least effort for the person who does it?

- You care about reproducible builds and bootstrapping, as in [1]?

- Other?

Bruno

[1] https://reproducible-builds.org/events/athens2015/bootstrapping/