gnulib-tool.py

[Tim Rühsen]

Hi,

just tried out gnulib-tool, taking the very naive approach:

- (in gnulib/) git checkout pygnulib
- made a commit with this change
- edited project's bootstrap to rename gnulib-tool into gnulib-tool.py
- tried './bootstrap'.

./bootstrap: 920: ./bootstrap: gnulib/gnulib-tool.py: Permission denied

This is easily fixed by 'chmod a+x gnulib/gnulib-tool.py'

Second try resulted in:
./bootstrap: gnulib/gnulib-tool.py --import --no-changelog --aux-dir build-
aux --doc-base doc --lib libgnu --m4-base m4/ --source-base lib/ --tests-base
tests --local-dir gl --makefile-name=gnulib.mk --libtool --import ...
usage: gnulib-tool.py --help
gnulib/gnulib-tool.py: error: argument -i/--import: expected at least one
argument
Traceback (most recent call last):
File "gnulib/gnulib-tool.py", line 927, in <module>
errno = error.errno
AttributeError: 'exceptions.SystemExit' object has no attribute 'errno'
./bootstrap: gnulib-tool failed

Maybe I did something basically wrong !?

Attached bootstrap.conf

[Dmitry Selyutin]

Hi Tim,

that's exactly what I've done today. I've just pushed some changes into
repository; could you please try the latest version?
I didn't have time to integrate --no-changelog support yet (it seems it was
either unimplemented before or appeared after project was completed).
Thank you for trying the Python version; please let me know if it works for
you or if there are some other errors to be fixed.

P.S. I've started cleaning the entire code base, also rewriting some slow
paths, but there is a lot of work to be done; stay tuned. :-)

Hi Dmitry,
I tried to remove some other options as well (as possible), but whatever
options there are, gnulib-tool.py doesn't like them.
The bootstrap.conf that I sent you is from project GNU Wget2.
Simply clone it (plus gnulib submodule) via
git clone --recursive https://gitlab.com/gnuwget/wget2.git
And change to the pygnulib branch via
cd gnulib
git checkout pygnulib
cd ..
git commit -a -m "Use pygnulib"
Now edit ./bootstrap to use gnulib-tool.py and
./bootstrap
The ./bootstrap script (also from gnulib) uses/loade ./bootstrap.conf.
Maybe that can help you with debugging (sorry, but I am absolutely not into
python).
Regards, Tim

Hi Tim,
thank you for reporting it. I strongly suspect that such behavior could

be

caused by invalid option processing. I've noticed that there are two
"--import" directive, and the first one appears to have no arguments;

could

you please try omitting the very first "--import" directive?
Hi,
- (in gnulib/) git checkout pygnulib
- made a commit with this change
- edited project's bootstrap to rename gnulib-tool into gnulib-tool.py
- tried './bootstrap'.
./bootstrap: 920: ./bootstrap: gnulib/gnulib-tool.py: Permission denied
This is easily fixed by 'chmod a+x gnulib/gnulib-tool.py'
./bootstrap: gnulib/gnulib-tool.py --import --no-changelog --aux-dir

build-

aux --doc-base doc --lib libgnu --m4-base m4/ --source-base lib/ --tests-base
tests --local-dir gl --makefile-name=gnulib.mk --libtool --import ...
usage: gnulib-tool.py --help
gnulib/gnulib-tool.py: error: argument -i/--import: expected at least one
argument
File "gnulib/gnulib-tool.py", line 927, in <module>
errno = error.errno
AttributeError: 'exceptions.SystemExit' object has no attribute 'errno'
./bootstrap: gnulib-tool failed
Maybe I did something basically wrong !?
Attached bootstrap.conf

[Tim Rühsen]

Post by Dmitry Selyutin
Hi Tim,
that's exactly what I've done today. I've just pushed some changes into
repository; could you please try the latest version?
I didn't have time to integrate --no-changelog support yet (it seems it was
either unimplemented before or appeared after project was completed).
Thank you for trying the Python version; please let me know if it works for
you or if there are some other errors to be fixed.
P.S. I've started cleaning the entire code base, also rewriting some slow
paths, but there is a lot of work to be done; stay tuned. :-)

Awesome, Dmitry ! \o/

Works like a charm on a quick test (will do some more testing the next days).

That's really *much* faster than before.
Now:
real 0m34,265s
user 0m31,524s
sys 0m2,279s

Before:
real 1m20,295s
user 1m16,460s
sys 0m16,050s

The bottleneck now is autoreconf followed by libtoolize :-)

If you remove the gnulib-tool from branch 'pygnulib' and instead add a link
gnulib-tool -> gnulib-tool.py then one wouldn't have to edit ./bootstrap.

I just can't say how much I appreciate your work !

Regards, Tim

[Dmitry Selyutin]

Hi Tim,

thank you for your kind words!
As for merging into master, I tried to do it, but didn't succeed: git
complains a lot on whitespaces and similar stuff. I suspect it may be
caused by custom git hook. Bruno, Paul, could you help me with merging? I
suppose that right now almost everything shall be merged, except for the
files I've added recently:

pygnulib/error.py
pygnulib/generator.py
pygnulib/config.py
pygnulib/filesystem.py

These files represent the development branch; I'm trying to cleanup and
mostly rewrite the API. Could you help me with merge, please? If necessary,
I can try using some kind of tool which formats the code automatically.

Post by Tim Rühsen

Post by Dmitry Selyutin
Hi Tim,
that's exactly what I've done today. I've just pushed some changes into
repository; could you please try the latest version?
I didn't have time to integrate --no-changelog support yet (it seems it

was

Post by Dmitry Selyutin
either unimplemented before or appeared after project was completed).
Thank you for trying the Python version; please let me know if it works

for

Post by Dmitry Selyutin
you or if there are some other errors to be fixed.
P.S. I've started cleaning the entire code base, also rewriting some slow
paths, but there is a lot of work to be done; stay tuned. :-)

Awesome, Dmitry ! \o/
Works like a charm on a quick test (will do some more testing the next days).
That's really *much* faster than before.
real 0m34,265s
user 0m31,524s
sys 0m2,279s
real 1m20,295s
user 1m16,460s
sys 0m16,050s
The bottleneck now is autoreconf followed by libtoolize :-)
If you remove the gnulib-tool from branch 'pygnulib' and instead add a link
gnulib-tool -> gnulib-tool.py then one wouldn't have to edit ./bootstrap.
I just can't say how much I appreciate your work !
Regards, Tim

[Bruno Haible]

Hi Dmitry,

Post by Dmitry Selyutin
As for merging into master, I tried to do it, but didn't succeed: git
complains a lot on whitespaces and similar stuff. I suspect it may be
caused by custom git hook.

Yes, the custom git hook can emit errors and warnings. The messages about
whitespace are only warnings, as far as I know. The error message(s) must
be different.

Post by Dmitry Selyutin
Bruno, Paul, could you help me with merging?

I might help you better if you show the error message(s) you got - and,
obviously, the command sequence you tried.

Bruno

[Dmitry Selyutin]

Hi Bruno,

below is the sequence of commands.

# creating a new branch on commit before pushing gnulib-tool
$ git branch pygnulib-stable ec2d72558

# checking out branch and rebasing on the latest master changes
$ git checkout pygnulib-stable
$ git rebase master

# cherry-picking only commits which touch original implementation
$ git cherry-pick eb41868f9 6edef0c50 f9364747f d1a39ef33

# checking out master and merging, trying to push
$ git checkout master
$ git merge pygnulib-stable
$ git push origin master:master

I've attached the log. All I can see is a lot of complaints on whitespaces
and EOF.
Does that just mean that I need to make the code a bit cleaner?
Or there may be another reason for hook to reject the commit?

Hi Dmitry,

Post by Dmitry Selyutin
As for merging into master, I tried to do it, but didn't succeed: git
complains a lot on whitespaces and similar stuff. I suspect it may be
caused by custom git hook.

Yes, the custom git hook can emit errors and warnings. The messages about
whitespace are only warnings, as far as I know. The error message(s) must
be different.

Post by Dmitry Selyutin
Bruno, Paul, could you help me with merging?

I might help you better if you show the error message(s) you got - and,
obviously, the command sequence you tried.
Bruno

[Darshit Shah]

Post by Dmitry Selyutin
Hi Bruno,
below is the sequence of commands.
# creating a new branch on commit before pushing gnulib-tool
$ git branch pygnulib-stable ec2d72558
# checking out branch and rebasing on the latest master changes
$ git checkout pygnulib-stable
$ git rebase master
# cherry-picking only commits which touch original implementation
$ git cherry-pick eb41868f9 6edef0c50 f9364747f d1a39ef33
# checking out master and merging, trying to push
$ git checkout master
$ git merge pygnulib-stable
$ git push origin master:master
I've attached the log. All I can see is a lot of complaints on whitespaces
and EOF.
Does that just mean that I need to make the code a bit cleaner?
Or there may be another reason for hook to reject the commit?

Hi Dmitry,

Post by Dmitry Selyutin
As for merging into master, I tried to do it, but didn't succeed: git
complains a lot on whitespaces and similar stuff. I suspect it may be
caused by custom git hook.

Yes, the custom git hook can emit errors and warnings. The messages about
whitespace are only warnings, as far as I know. The error message(s) must
be different.

Post by Dmitry Selyutin
Bruno, Paul, could you help me with merging?

I might help you better if you show the error message(s) you got - and,
obviously, the command sequence you tried.
Bruno

--
With best regards,
Dmitry Selyutin

Hi Dmitry,

I think you forgot to attach the log files. I don't see anything
attached to your mail. Could you please check it once?

Since I don't know the exact warnings, I can't speak about those.
However, from what I see, gnulib-tool.py has a lot of trailing
whitespaces. That is lines that end with a whitespace character. This is
usually discouraged, and my guess is exactly what the commit hooks are
catching on to. You could try to set your editor to highlight such
spaces or even to delete them on saving a file. Or you could use a regex
like: `s/\s\+$//g` to remove all the trailing whitespaces

[Dmitry Selyutin]

Yes, whitespaces and EOF's were the real cause. I've run the autopep8 tool
and it solved the issue.
Tim, I've managed to build wget2 against the latest master (though I had to
change $gnulib_tool and checkout master inside submodule).

Post by Dmitry Selyutin

Post by Dmitry Selyutin
Hi Bruno,
below is the sequence of commands.
# creating a new branch on commit before pushing gnulib-tool
$ git branch pygnulib-stable ec2d72558
# checking out branch and rebasing on the latest master changes
$ git checkout pygnulib-stable
$ git rebase master
# cherry-picking only commits which touch original implementation
$ git cherry-pick eb41868f9 6edef0c50 f9364747f d1a39ef33
# checking out master and merging, trying to push
$ git checkout master
$ git merge pygnulib-stable
$ git push origin master:master
I've attached the log. All I can see is a lot of complaints on

whitespaces

Post by Dmitry Selyutin
and EOF.
Does that just mean that I need to make the code a bit cleaner?
Or there may be another reason for hook to reject the commit?

Hi Dmitry,

Post by Dmitry Selyutin
As for merging into master, I tried to do it, but didn't succeed: git
complains a lot on whitespaces and similar stuff. I suspect it may be
caused by custom git hook.

Yes, the custom git hook can emit errors and warnings. The messages

about

Post by Dmitry Selyutin

whitespace are only warnings, as far as I know. The error message(s)

must

Post by Dmitry Selyutin

be different.

Post by Dmitry Selyutin
Bruno, Paul, could you help me with merging?

I might help you better if you show the error message(s) you got - and,
obviously, the command sequence you tried.
Bruno

--
With best regards,
Dmitry Selyutin

Hi Dmitry,
I think you forgot to attach the log files. I don't see anything
attached to your mail. Could you please check it once?
Since I don't know the exact warnings, I can't speak about those.
However, from what I see, gnulib-tool.py has a lot of trailing
whitespaces. That is lines that end with a whitespace character. This is
usually discouraged, and my guess is exactly what the commit hooks are
catching on to. You could try to set your editor to highlight such
spaces or even to delete them on saving a file. Or you could use a regex
like: `s/\s\+$//g` to remove all the trailing whitespaces
--
Thanking You,
Darshit Shah
PGP Fingerprint: 7845 120B 07CB D8D6 ECE5 FF2B 2A17 43ED A91A 35B6

[Tim Rühsen]

Post by Dmitry Selyutin
Yes, whitespaces and EOF's were the real cause. I've run the autopep8 tool
and it solved the issue.
Tim, I've managed to build wget2 against the latest master (though I had to
change $gnulib_tool and checkout master inside submodule).

Thanks, it now successfully works with all our Gitlab CI runners :-)

Regards, Tim

[Dmitry Selyutin]

Hi Tim,

this is a kind reminder that it is still unstable and buggy, so I hope that
you still maintain runners with the original gnilub-tool script. Still this
news sounds great!