Change the python interpretor to use /usr/bin/env python3

[Darshit Shah]

Hi,

On a fedora machine, Python 3 is installed by default, but not Python 2. And
the only binary available is /usr/bin/python3. This means that despite having
all the requirements to run gnulib-tool.py, one would still need to install
python to get the right binary.

However, simply changing the shebang line to `/usr/bin/env python3`, this can
be eliminated. I don't mind if you would like to keep it as `/usr/bin/env python` to use
the python2 interpretor by default either.

[Paul Eggert]

Could you please clarify this? Which Fedora release and architecture, and what
What is the output of 'ls -l /usr/bin/python*' and of 'dnf list --installed
'python*''? That sort of thing. I don't understand why '#!/usr/bin/python' would
fail at the same time that '#!/usr/bin/env python' worked. Thanks.

[Dmitry Selyutin]

Hello,

thank you for investigating this. I'm joining to questions asked by Paul;
moreover, as far as I know specifying python is usually the most correct
way to make python work regardless of version (i.e. it is supposed that one
day there will be no distinction between python and python3).
FWIW, the future reimplementation will drop python 2 entirely. And "python"
branch uses /usr/bin/python too.

Post by Paul Eggert
Could you please clarify this? Which Fedora release and architecture, and what
What is the output of 'ls -l /usr/bin/python*' and of 'dnf list --installed
'python*''? That sort of thing. I don't understand why '#!/usr/bin/python' would
fail at the same time that '#!/usr/bin/env python' worked. Thanks.

[Darshit Shah]

Hi,

Sorry for the sparse details. I was confused the first time I saw the issue as
well, but then I forgot to mention all the details.

I was trying to test building GNU Wget2 in a Fedora 27 docker container.
So, the I am talking about Fedora 27 on x86-64. But not the full version, just
what is available in the Docker container.

lrwxrwxrwx 1 root root 9 Feb 8 14:46 /usr/bin/python3 -> python3.6
-rwxr-xr-x 1 root root 11240 Feb 8 14:46 /usr/bin/python3.6
-rwxr-xr-x 1 root root 11240 Feb 8 14:46 /usr/bin/python3.6m
Installed Packages

As you can see here, only Python 3 is installed in the base system. However,
contrary to expectations, python3 does not provide /usr/bin/python. That binary
is provided by the `python2` package on Fedora. Since python2 is not installed
by default in the docker container, this means that only the "/usr/bin/python3"
binary is available on that system.

However, I was mistaken about the call to `#!/usr/bin/env python` working. Only
`#!/usr/bin/env python3` works. Since Python 3 is the version that is being
specifically targeted, could we please change it to reflect that? As far as I'm
aware, all systems with Python 3 installed always have a python3 binary
available in $PATH. However, a /usr/bin/python may point to python 2 or 3.

Hello,
thank you for investigating this. I'm joining to questions asked by Paul;
moreover, as far as I know specifying python is usually the most correct
way to make python work regardless of version (i.e. it is supposed that one
day there will be no distinction between python and python3).
FWIW, the future reimplementation will drop python 2 entirely. And "python"
branch uses /usr/bin/python too.

Post by Paul Eggert
Could you please clarify this? Which Fedora release and architecture, and what
What is the output of 'ls -l /usr/bin/python*' and of 'dnf list --installed
'python*''? That sort of thing. I don't understand why '#!/usr/bin/python' would
fail at the same time that '#!/usr/bin/env python' worked. Thanks.

[Paul Eggert]

Post by Darshit Shah
Since Python 3 is the version that is being
specifically targeted, could we please change it to reflect that?

That module doesn't target Python 3 in particular, as far as I can tell; it
should run on either Python 2 or 3.

I dunno, it's pretty weird to have a system that has Python but for which the
command 'python' does not work.

[Bernhard Voelker]

Post by Darshit Shah
As you can see here, only Python 3 is installed in the base system. However,
contrary to expectations, python3 does not provide /usr/bin/python. That binary
is provided by the `python2` package on Fedora. Since python2 is not installed
by default in the docker container, this means that only the "/usr/bin/python3"
binary is available on that system.
However, I was mistaken about the call to `#!/usr/bin/env python` working. Only
`#!/usr/bin/env python3` works. Since Python 3 is the version that is being
specifically targeted, could we please change it to reflect that? As far as I'm
aware, all systems with Python 3 installed always have a python3 binary
available in $PATH. However, a /usr/bin/python may point to python 2 or 3.

FWIW there are discussions [1] in the opposite direction, i.e., to avoid
/usr/bin/env in the shebang where possible. E.g. in openSUSE there is a
check for RPM builds which leads to:

pyrenamer.noarch: E: env-script-interpreter (Badness: 9) /usr/bin/pyrenamer /usr/bin/env python
This script uses 'env' as an interpreter. For the rpm runtime dependency
detection to work, the shebang #!/usr/bin/env python needs to be patched into
#!/usr/bin/python otherwise the package dependency generator merely adds a
dependency on /usr/bin/env rather than the actual interpreter /usr/bin/python.
Alternatively, if the file should not be executed, then ensure that it is not
marked as executable or don't install it in a path that is reserved for
executables.

[1]
https://lists.opensuse.org/opensuse-packaging/2018-03/msg00017.html

Have a nice day,
Berny

[Bruno Haible]

Post by Bernhard Voelker
FWIW there are discussions [1] in the opposite direction, i.e., to avoid
/usr/bin/env in the shebang where possible. E.g. in openSUSE there is a
pyrenamer.noarch: E: env-script-interpreter (Badness: 9) /usr/bin/pyrenamer /usr/bin/env python
This script uses 'env' as an interpreter. For the rpm runtime dependency
detection to work, the shebang #!/usr/bin/env python needs to be patched into
#!/usr/bin/python otherwise the package dependency generator merely adds a
dependency on /usr/bin/env rather than the actual interpreter /usr/bin/python.
Alternatively, if the file should not be executed, then ensure that it is not
marked as executable or don't install it in a path that is reserved for
executables.

It would make sense for the "rpm runtime dependency detection" to recognize
the /usr/bin/env case as well. In fact, there's already a TODO item in the
source code about this [2].

Additionally, this discussion what rpm does or does not do is not related to
gnulib. What rpm does has an impact on the difference between a tarball/source
code meant to be installed by users _from_source_ and a package/binaries meant
to be installed _from_a_distro's_repositories_. This is what rpm is about.

Whereas gnulib is always installed from a tarball/source code. gnulib does not
have distro-package maintainers.

Therefore, in gnulib, the most general solution is the best one:
#!/usr/bin/env python
or
#!/usr/bin/env python3

The user can then create a symlink as they wish:
ln -s /usr/bin/python3 $HOME/bin/python
Or rely on /etc/alternatives/...

Bruno

[2] https://github.com/rpm-software-management/rpm/blob/master/scripts/script.req

[Bernhard Voelker]

Post by Bruno Haible
It would make sense for the "rpm runtime dependency detection" to recognize
the /usr/bin/env case as well. In fact, there's already a TODO item in the
source code about this [2].
Additionally, this discussion what rpm does or does not do is not related to
gnulib. What rpm does has an impact on the difference between a tarball/source
code meant to be installed by users _from_source_ and a package/binaries meant
to be installed _from_a_distro's_repositories_. This is what rpm is about.
Whereas gnulib is always installed from a tarball/source code. gnulib does not
have distro-package maintainers.
#!/usr/bin/env python
or
#!/usr/bin/env python3
ln -s /usr/bin/python3 $HOME/bin/python
Or rely on /etc/alternatives/...
Bruno
[2] https://github.com/rpm-software-management/rpm/blob/master/scripts/script.req

Indeed, that sounded like a quirk. Thanks for the wrap-up.

Have a nice day,
Berny

[Bruno Haible]

Post by Darshit Shah
On a fedora machine, Python 3 is installed by default, but not Python 2. And
the only binary available is /usr/bin/python3.

It's not only on Fedora. The Python docs themselves ([1], footnote 1) say:

"On Unix, the Python 3.x interpreter is by default not installed with the
executable named 'python', so that it does not conflict with a
simultaneously installed Python 2.x executable."

Bruno

[1] https://docs.python.org/3/tutorial/interpreter.html