OpenVMS getdtablesize.c and getloadavg.c issues.

Discussion:

John E. Malmberg

2017-06-27 13:02:26 UTC

Hello,

The getdtablesize.c module will not compile on OpenVMS and is really not
needed.

The way that the an OpenVMS getrlimit() would be most likely implemented
to return the RLIMIT_NOFILE is to call the system supplied getdtablesize().

Is there someway for configure to be changed to not set the macros
causing it to be built when the platform is OpenVMS?

The dtablesize is fixed at the time that the image is run as it is the
lesser of the SYSGEN channelcnt setting and the user's fillm quota.

There is no API to adjust it for a running process.

The getloadavg.c OpenVMS code probably has not built in quite some time
and probably for now should be reduced to setting the ENOSYS errno and
returning -1.

It has two variants, one requires the long gone eunice product, and the
other requires a third party LOADAVERAGE driver that I can not find any
current download location for.

From OpenVMS 7.3 on, the sys$getrmi() system service is used to return
that information, and I will look at writing that code at a future time.

Regards,
-John

Bruno Haible

2017-06-27 15:13:03 UTC

Permalink

Post by John E. Malmberg
The getdtablesize.c module will not compile on OpenVMS and is really not
needed.
...
Is there someway for configure to be changed to not set the macros
causing it to be built when the platform is OpenVMS?

gnulib builds the module 'getdtablesize' because it's a glibc function:
https://www.gnu.org/software/gnulib/manual/html_node/getdtablesize.html

Are you saying that OpenVMS already has the function and that
m4/getdtablesize.m4 guesses wrong about its existence?

Post by John E. Malmberg
The dtablesize is fixed at the time that the image is run

Sure, no one expects to be able to change the number of available
file descriptors of a running process.

Post by John E. Malmberg
it is the
lesser of the SYSGEN channelcnt setting and the user's fillm quota.

Perfect: you already know how to implement the gnulib replacement for
getdtablesize() on OpenVMS.

Bruno

John E. Malmberg

2017-06-28 00:34:33 UTC

Permalink

Post by Bruno Haible

https://www.gnu.org/software/gnulib/manual/html_node/getdtablesize.html
Are you saying that OpenVMS already has the function and that
m4/getdtablesize.m4 guesses wrong about its existence?

OpenVMS has the function. Configure detects it exists, but the test for
it working is failing.

I missed that the test for getdtablesize working was failing before now
though.

The OpenVMS getdtablesize() is returning the maximum number of open
files that a process can have.

It is not returning the highest file descriptor + 1 that can be used in
the system. That number is 65535 for OpenVMS Alpha and Integrity.

That is why it is failing the configure test.

So what is the recommended solution?

If I have a replacement getdtablesize() return 65535, that could cause
programs to assume that they could actually open that many descriptors.

And if an OpenVMS getrlimit() is provided, what value should it provide
for RLIMIT_NOFILE?

Regards,
-John

Bruno Haible

2017-06-28 20:56:13 UTC

Permalink

Post by John E. Malmberg
OpenVMS has the function. Configure detects it exists, but the test for
it working is failing.
...
The OpenVMS getdtablesize() is returning the maximum number of open
files that a process can have.
It is not returning the highest file descriptor + 1 that can be used in
the system. That number is 65535 for OpenVMS Alpha and Integrity.
That is why it is failing the configure test.

There are two possibilities:
(a) change the configure test so that is duplicates
fd 0 to 1, 2, 3, ..., getdtablesize() and see if that fails.
(b) special-case OpenVMS.

I'm not in favour of (a) because it might allocate so many file descriptors
that the kernel gets problems.

So (b). Just add a few lines

case "$host_os" in
vms*) ...

at the appropriate place. Can you propose such a patch?

Post by John E. Malmberg
If I have a replacement getdtablesize() return 65535, that could cause
programs to assume that they could actually open that many descriptors.

This is only a theoretical risk. I know of no programs that are that
aggressive.

Post by John E. Malmberg
And if an OpenVMS getrlimit() is provided, what value should it provide
for RLIMIT_NOFILE?

Please try to approximate the wording of POSIX [1] as closely as possible.

Bruno

[1] http://pubs.opengroup.org/onlinepubs/9699919799/functions/getrlimit.html

Paul Eggert

2017-06-28 22:08:03 UTC

Permalink

Post by Bruno Haible

Post by John E. Malmberg
If I have a replacement getdtablesize() return 65535, that could cause
programs to assume that they could actually open that many descriptors.

This is only a theoretical risk. I know of no programs that are that
aggressive.

Although GNU 'sort' does something like that, it has code to deal with later
'open' failures due to EMFILE and/or ENFILE. Any program that allocates file
descriptors that aggressively really needs to deal with 'open' failures that can
occur regardless of what getdtablesize returns.

John E. Malmberg

2017-07-11 13:12:48 UTC

Permalink

Post by Bruno Haible

(a) change the configure test so that is duplicates
fd 0 to 1, 2, 3, ..., getdtablesize() and see if that fails.
(b) special-case OpenVMS.
I'm not in favour of (a) because it might allocate so many file descriptors
that the kernel gets problems.
So (b). Just add a few lines
case "$host_os" in
vms*) ...
at the appropriate place. Can you propose such a patch?
Please try to approximate the wording of POSIX [1] as closely as possible.
Bruno
[1] http://pubs.opengroup.org/onlinepubs/9699919799/functions/getrlimit.html

According to that link, a system is not required to fail a request for a
file descriptor to be created higher than RLIMIT_NOFILE.

Attached is a patch for the test to pass on OpenVMS.

Regards,
-John

Bruno Haible

2017-07-15 15:00:30 UTC

Permalink

Post by John E. Malmberg

Post by Bruno Haible
[1] http://pubs.opengroup.org/onlinepubs/9699919799/functions/getrlimit.html

According to that link, a system is not required to fail a request for a
file descriptor to be created higher than RLIMIT_NOFILE.

Not true. [1] refers to [2], and [2] as well as [3] say that getrlimit(RLIMIT_NOFILE) is
"one greater than the maximum value that the system may assign to a newly-created descriptor."
The other sentence is just an explanation of the effect of this definition.

But I admit that you may have been confused because the comments in gnulib
were misleading.

Post by John E. Malmberg
Attached is a patch for the test to pass on OpenVMS.

Not usable: Comments are wrong, and I prefer a 'case' statement over
a #ifdef that renders the test a dummy.

Here's what I'm pushing.

[1] http://pubs.opengroup.org/onlinepubs/7908799/xsh/getdtablesize.html
[2] http://pubs.opengroup.org/onlinepubs/7908799/xsh/getrlimit.html
[3] http://pubs.opengroup.org/onlinepubs/009695399/functions/getrlimit.html

2017-07-15 Bruno Haible <***@clisp.org>

getdtablesize: Add minimal support for OpenVMS.
Reported by John E. Malmberg <***@qsl.net>.
* modules/getdtablesize (Description): Fix.
* lib/getdtablesize.c: Fix comment.
* m4/getdtablesize.m4 (gl_FUNC_GETDTABLESIZE): Don't replace the
getdtablesize() function, even though the test fails.
* doc/glibc-functions/getdtablesize.texi: Reference SUSv2. Describe
limitation on OpenVMS.

diff --git a/doc/glibc-functions/getdtablesize.texi b/doc/glibc-functions/getdtablesize.texi
index 921c985..b7a16d1 100644
--- a/doc/glibc-functions/getdtablesize.texi
+++ b/doc/glibc-functions/getdtablesize.texi
@@ -2,6 +2,8 @@
@subsection @code{getdtablesize}
@findex getdtablesize

+SUSv2 specification: @url{http://pubs.opengroup.org/onlinepubs/7908799/xsh/getdtablesize.html}
+
Gnulib module: getdtablesize

Portability problems fixed by Gnulib:
@@ -22,4 +24,8 @@ Android LP32, Cygwin 1.7.25.

Portability problems not fixed by Gnulib:
@itemize
+@item
+On OpenVMS, this function returns the maximum number of open file descriptors
+in a process. The possible values of file descriptors are not constrained by
+this function.
@end itemize
diff --git a/lib/getdtablesize.c b/lib/getdtablesize.c
index c356cf4..a092863 100644
--- a/lib/getdtablesize.c
+++ b/lib/getdtablesize.c
@@ -1,4 +1,4 @@
-/* getdtablesize() function for platforms that don't have it.
+/* getdtablesize() function: Return maximum possible file descriptor value + 1.
Copyright (C) 2008-2017 Free Software Foundation, Inc.
Written by Bruno Haible <***@clisp.org>, 2008.

diff --git a/m4/getdtablesize.m4 b/m4/getdtablesize.m4
index 1af2a24..f1e4f5f 100644
--- a/m4/getdtablesize.m4
+++ b/m4/getdtablesize.m4
@@ -1,4 +1,4 @@
-# getdtablesize.m4 serial 6
+# getdtablesize.m4 serial 7
dnl Copyright (C) 2008-2017 Free Software Foundation, Inc.
dnl This file is free software; the Free Software Foundation
dnl gives unlimited permission to copy and/or distribute it,
@@ -12,29 +12,43 @@ AC_DEFUN([gl_FUNC_GETDTABLESIZE],
AC_CHECK_DECLS_ONCE([getdtablesize])
if test $ac_cv_func_getdtablesize = yes &&
test $ac_cv_have_decl_getdtablesize = yes; then
- # Cygwin 1.7.25 automatically increases the RLIMIT_NOFILE soft limit
- # up to an unchangeable hard limit; all other platforms correctly
- # require setrlimit before getdtablesize() can report a larger value.
AC_CACHE_CHECK([whether getdtablesize works],
[gl_cv_func_getdtablesize_works],
- [AC_RUN_IFELSE([
- AC_LANG_PROGRAM([[#include <unistd.h>]],
- [int size = getdtablesize();
- if (dup2 (0, getdtablesize()) != -1)
- return 1;
- if (size != getdtablesize())
- return 2;
- ])],
- [gl_cv_func_getdtablesize_works=yes],
- [gl_cv_func_getdtablesize_works=no],
- [case "$host_os" in
- cygwin*) # on cygwin 1.5.25, getdtablesize() automatically grows
- gl_cv_func_getdtablesize_works="guessing no" ;;
- *) gl_cv_func_getdtablesize_works="guessing yes" ;;
- esac])
+ [dnl There are two concepts: the "maximum possible file descriptor value + 1"
+ dnl and the "maximum number of open file descriptors in a process".
+ dnl Per SUSv2 and POSIX, getdtablesize() should return the first one.
+ dnl On most platforms, the first and the second concept are the same.
+ dnl On OpenVMS, however, they are different and getdtablesize() returns
+ dnl the second one; thus the test below fails. But we don't care
+ dnl because there's no good way to write a replacement getdtablesize().
+ case "$host_os" in
+ vms*) gl_cv_func_getdtablesize_works="no (limitation)" ;;
+ *)
+ dnl Cygwin 1.7.25 automatically increases the RLIMIT_NOFILE soft
+ dnl limit up to an unchangeable hard limit; all other platforms
+ dnl correctly require setrlimit before getdtablesize() can report
+ dnl a larger value.
+ AC_RUN_IFELSE([
+ AC_LANG_PROGRAM([[#include <unistd.h>]],
+ [int size = getdtablesize();
+ if (dup2 (0, getdtablesize()) != -1)
+ return 1;
+ if (size != getdtablesize())
+ return 2;
+ ])],
+ [gl_cv_func_getdtablesize_works=yes],
+ [gl_cv_func_getdtablesize_works=no],
+ [case "$host_os" in
+ cygwin*) # on cygwin 1.5.25, getdtablesize() automatically grows
+ gl_cv_func_getdtablesize_works="guessing no" ;;
+ *) gl_cv_func_getdtablesize_works="guessing yes" ;;
+ esac
+ ])
+ ;;
+ esac
])
case "$gl_cv_func_getdtablesize_works" in
- *yes) ;;
+ *yes | "no (limitation)") ;;
*) REPLACE_GETDTABLESIZE=1 ;;
esac
else
diff --git a/modules/getdtablesize b/modules/getdtablesize
index e458f3a..f33fcb5 100644
--- a/modules/getdtablesize
+++ b/modules/getdtablesize
@@ -1,5 +1,5 @@
Description:
-getdtablesize() function: return maximum number of file descriptors.
+getdtablesize() function: return tight upper bound for file descriptor values.

Files:
lib/getdtablesize.c