Discussion:
some questions
(too old to reply)
Tim Rühsen
2017-05-17 14:52:47 UTC
Permalink
Raw Message
Hi,

just a couple of questions...

How do I enable valgrind testing ?

What about automated / CI testing ?
We (the Wget maintainers) recently set up a group 'gnuwget' on Gitlab
with CI runners (Debian, CentOS, Fedora). I could add gnulib as a
project so that it can use these runners as well. The only thing to do
for the gnulib project is to add a .gitlab-ci.yml file for building and
testing (gcc, clang, valgrind, coverage reporting, ...).

What about fuzzing ?
I just work on OSS-Fuzz[1] integration for Wget2. Is There interest in a
'starter' for gnulib with a example fuzz code ?

[1]
https://opensource.googleblog.com/2017/05/oss-fuzz-five-months-later-and.html


With Best Regards, Tim
Bruno Haible
2017-05-17 17:09:05 UTC
Permalink
Raw Message
Hi Tim,
Post by Tim Rühsen
How do I enable valgrind testing ?
There is this doc [1] and this recent mail thread [2].

Bruno

[1] https://www.gnu.org/software/gnulib/manual/html_node/Running-self_002dtests-under-valgrind.html
[2] https://lists.gnu.org/archive/html/bug-gnulib/2017-04/msg00041.html
Tim Rühsen
2017-05-17 19:25:29 UTC
Permalink
Raw Message
Post by Bruno Haible
Hi Tim,
Post by Tim Rühsen
How do I enable valgrind testing ?
There is this doc [1] and this recent mail thread [2].
Ok, there seems to a misunderstanding. While I have no problems to setup/run
most projects to use valgrind, I have problems to run gnulib self-tests with
valgrind.

The TESTS_ENVIRONMENT trick failed - well, gnulib's gltest/Makefile.am
overwrites this variable. I also couldn't find any references in the code /
Makefiles / docs.
Post by Bruno Haible
Bruno
[1]
https://www.gnu.org/software/gnulib/manual/html_node/Running-self_002dtests
-under-valgrind.html [2]
https://lists.gnu.org/archive/html/bug-gnulib/2017-04/msg00041.html
Bruno Haible
2017-05-17 17:09:52 UTC
Permalink
Raw Message
Hi Tim,
Post by Tim Rühsen
What about automated / CI testing ?
We (the Wget maintainers) recently set up a group 'gnuwget' on Gitlab
with CI runners (Debian, CentOS, Fedora). I could add gnulib as a
project so that it can use these runners as well. The only thing to do
for the gnulib project is to add a .gitlab-ci.yml file for building and
testing (gcc, clang, valgrind, coverage reporting, ...).
Simon had an autobuild site [1][2], but it appears to be offline now.

From my perspective, glibc systems are the ones which pose the least problems,
and get reported quickly anyway. Therefore what is more valuable for gnulib
(IMO) is automated testing on 20 different platforms.

Bruno

[1] https://www.gnu.org/software/gnulib/manual/html_node/Build-robot-for-gnulib.html
[2] http://josefsson.org/autobuild/
Tim Rühsen
2017-05-17 19:20:33 UTC
Permalink
Raw Message
Post by Bruno Haible
Hi Tim,
Post by Tim Rühsen
What about automated / CI testing ?
We (the Wget maintainers) recently set up a group 'gnuwget' on Gitlab
with CI runners (Debian, CentOS, Fedora). I could add gnulib as a
project so that it can use these runners as well. The only thing to do
for the gnulib project is to add a .gitlab-ci.yml file for building and
testing (gcc, clang, valgrind, coverage reporting, ...).
Simon had an autobuild site [1][2], but it appears to be offline now.
From my perspective, glibc systems are the ones which pose the least
problems, and get reported quickly anyway. Therefore what is more valuable
for gnulib (IMO) is automated testing on 20 different platforms.
Of course everybody wishes to have a CI for 20 different platform :-)

But you have to start at some point, picking the low hanging fruits. And that
is Gitlab CI based on Docker. You can build, test and deploy with using
different Linux platforms (x86, amd64, different software release
environments). Also cross-platform builds can be done (ARM, MinGW, ...),
that's up to you. You can have a free OSX test environment using Travis CI -
that needs a mirror on Github. There is AppVeyor for Windows CI, but I have no
experience with that.
And there is the OpenCSW Solaris build platform...

For real VMs with other OSes running, you can
- set up your own VMs 'at home'
- pay for customizable VMs and CPU power
- find a sponsor to pay for VMs and CPU power
Post by Bruno Haible
Bruno
[1]
https://www.gnu.org/software/gnulib/manual/html_node/Build-robot-for-gnulib
.html [2] http://josefsson.org/autobuild/
With Best Regards, Tim
Assaf Gordon
2017-05-17 19:43:50 UTC
Permalink
Raw Message
Hello,
Post by Tim Rühsen
Post by Tim Rühsen
What about automated / CI testing ?
[...] Therefore what is more valuable
for gnulib (IMO) is automated testing on 20 different platforms.
Of course everybody wishes to have a CI for 20 different platform :-)
For real VMs with other OSes running, you can
- set up your own VMs 'at home'
- pay for customizable VMs and CPU power
- find a sponsor to pay for VMs and CPU power
<shameless plug>
I work on a project called PreTest ( http://pretest.nongnu.org/ )
which provides pre-configured virtual machines for many
operating-systems (all with same user/pw, 'sudo', and git/make/cc
installed - so you can boot it and start compiling immediately).

List of available VMs:
http://pretest.nongnu.org/downloads/

Usage example:
http://pretest.nongnu.org/command-line-qemu.html

And, when people post messages to 'platform-***@gnu.org',
I try to run against all these VMs, with results automatically
collected here:
https://pretest.housegordon.org/
</shameless plug>
Post by Tim Rühsen
Simon had an autobuild site [1][2], but it appears to be offline now.
[2] http://josefsson.org/autobuild/
My long-term plan is to setup something like that myself
(time and resources permitting). I hope to get it working
towards the end of this year.

regards,
- assaf
Tim Rühsen
2017-05-18 07:52:38 UTC
Permalink
Raw Message
Tim Rühsen
2017-05-18 09:26:04 UTC
Permalink
Raw Message
Post by Assaf Gordon
Hello,
Post by Tim Rühsen
Post by Tim Rühsen
What about automated / CI testing ?
[...] Therefore what is more valuable
for gnulib (IMO) is automated testing on 20 different platforms.
Of course everybody wishes to have a CI for 20 different platform :-)
For real VMs with other OSes running, you can
- set up your own VMs 'at home'
- pay for customizable VMs and CPU power
- find a sponsor to pay for VMs and CPU power
<shameless plug>
I work on a project called PreTest ( http://pretest.nongnu.org/ )
which provides pre-configured virtual machines for many
operating-systems (all with same user/pw, 'sudo', and git/make/cc
installed - so you can boot it and start compiling immediately).
http://pretest.nongnu.org/downloads/
http://pretest.nongnu.org/command-line-qemu.html
I try to run against all these VMs, with results automatically
https://pretest.housegordon.org/
</shameless plug>
Assaf, that.is.so.cool !

Really, your work is a great help to any project striving for
portability (VM setup and 'suggested command line').
Post by Assaf Gordon
Post by Tim Rühsen
Simon had an autobuild site [1][2], but it appears to be offline now.
[2] http://josefsson.org/autobuild/
My long-term plan is to setup something like that myself
(time and resources permitting). I hope to get it working
towards the end of this year.
You mean letting projects use the VMs as a CI service ?
That would be much more valuable then docker based CIs.
Maybe with Gitlab integration ? I guess, you would find a sponsor for
CPU and bandwidth...

Would you collect project-based 'build recipes' ?


Thanks for that information.
I'll test your VMs soon :-)
Post by Assaf Gordon
regards,
- assaf
With Best Regards, Tim

Bruno Haible
2017-05-17 17:10:40 UTC
Permalink
Raw Message
Hi Tim,
Post by Tim Rühsen
What about fuzzing ?
I just work on OSS-Fuzz[1] integration for Wget2. Is There interest in a
'starter' for gnulib with a example fuzz code ?
Gnulib by itself does not provide functionality that consumes input.

But Gnulib provides common coding patterns for packages that use the GNU Build
System, and some such packages (gettext, grep, guile, gzip, sed, tar, texinfo
maybe?) could probably make use of an OSS-FUZZ integration. If you have a
Makefile.am pattern for this, it would be interesting for Gnulib.

Bruno

[1] https://opensource.googleblog.com/2017/05/oss-fuzz-five-months-later-and.html
Tim Rühsen
2017-05-17 19:47:28 UTC
Permalink
Raw Message
Post by Bruno Haible
Hi Tim,
Post by Tim Rühsen
What about fuzzing ?
I just work on OSS-Fuzz[1] integration for Wget2. Is There interest in a
'starter' for gnulib with a example fuzz code ?
Gnulib by itself does not provide functionality that consumes input.
You can fuzz each function provided by gnulib. You start with test data that
covers as much of the codes paths as possible. The fuzzer also detects new
code paths, outputs data to cover these (to use it in unit tests) and also is
able to minimize this test data.

It likely doesn't make sense to fuzz *all* functions. Maybe one should start
with the most complex functions.
Post by Bruno Haible
But Gnulib provides common coding patterns for packages that use the GNU
Build System, and some such packages (gettext, grep, guile, gzip, sed, tar,
texinfo maybe?) could probably make use of an OSS-FUZZ integration. If you
have a Makefile.am pattern for this, it would be interesting for Gnulib.
Perhaps that is possible. Let me first finish my fuzzer work for Wget2 (not even
pushed yet) and then we'll examine it.
Post by Bruno Haible
Bruno
[1]
https://opensource.googleblog.com/2017/05/oss-fuzz-five-months-later-and.ht
ml
With Best Regards, Tim
Loading...