[SCM] Samba Shared Repository - branch master updated
Douglas Bagnall
dbagnall at samba.org
Thu Oct 22 14:11:04 UTC 2020
The branch, master has been updated
via c03a265030f oss-fuzz: standardise on RPATH for the static-ish binaries
via 048725080b6 fuzzing: Improve robustness and documentation of the ldd-base library copy
from 7b52c2db264 s3:param:service - ensure registry shares loaded before home check
https://git.samba.org/?p=samba.git;a=shortlog;h=master
- Log -----------------------------------------------------------------
commit c03a265030fefb1652469ff98a8b895b8f61b771
Author: Andrew Bartlett <abartlet at samba.org>
Date: Thu Oct 22 15:39:50 2020 +1300
oss-fuzz: standardise on RPATH for the static-ish binaries
This includes a revert of commit e60df214998afc145ca482cab184691b3ddc3bb2.
We strictly require RPATH, not the modern RUNPATH for the behaviour
we need in oss-fuzz, which is that not just the first line of dependencies
but the full set of libraries used by the program are looked for in the
'$ORIGIN/lib' directory.
Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Autobuild-User(master): Douglas Bagnall <dbagnall at samba.org>
Autobuild-Date(master): Thu Oct 22 14:10:04 UTC 2020 on sn-devel-184
commit 048725080b608d0d714f133a15e09dda6b6dd8ca
Author: Andrew Bartlett <abartlet at samba.org>
Date: Thu Oct 22 12:05:29 2020 +1300
fuzzing: Improve robustness and documentation of the ldd-base library copy
This tries to make progress towards understanding why we sometime see errors like
Step #6: Error occured while running fuzz_reg_parse:
Step #6: /workspace/out/coverage/fuzz_reg_parse: error while loading shared libraries: libavahi-common.so.3: cannot open shared object file: No such file or directory
in the previously failing coverage builds.
Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
-----------------------------------------------------------------------
Summary of changes:
lib/fuzzing/oss-fuzz/build_samba.sh | 40 +++++++++++++++++++++++--------------
lib/fuzzing/oss-fuzz/check_build.sh | 13 ++++++++++--
script/autobuild.py | 2 +-
3 files changed, 37 insertions(+), 18 deletions(-)
Changeset truncated at 500 lines:
diff --git a/lib/fuzzing/oss-fuzz/build_samba.sh b/lib/fuzzing/oss-fuzz/build_samba.sh
index 5980f88ad02..28c37a793cc 100755
--- a/lib/fuzzing/oss-fuzz/build_samba.sh
+++ b/lib/fuzzing/oss-fuzz/build_samba.sh
@@ -63,7 +63,7 @@ case "$SANITIZER" in
# cc style options, so we can just set ADDITIONAL_LDFLAGS
# to ensure the coverage build is done, despite waf splitting
# the compile and link phases.
- ADDITIONAL_LDFLAGS="$COVERAGE_FLAGS"
+ ADDITIONAL_LDFLAGS="${ADDITIONAL_LDFLAGS:-} $COVERAGE_FLAGS"
export ADDITIONAL_LDFLAGS
SANITIZER_ARG=''
@@ -94,25 +94,35 @@ mkdir -p $OUT/lib
for x in bin/fuzz_*
do
+ # Copy any system libraries needed by this fuzzer to $OUT/lib.
+
+ # We run ldd on $x, the fuzz_binary in bin/ which has not yet had
+ # the RUNPATH altered. This is clearer for debugging in local
+ # development builds as $OUT is not cleaned between runs.
+ #
+ # Otherwise trying to re-run this can see cp can fail with:
+ # cp: '/out/lib/libgcc_s.so.1' and '/out/lib/libgcc_s.so.1' are the same file
+ # which is really confusing!
+
+ # The cut for ( and ' ' removes the special case references to:
+ # linux-vdso.so.1 => (0x00007ffe8f2b2000)
+ # /lib64/ld-linux-x86-64.so.2 (0x00007fc63ea6f000)
+
+ ldd $x | cut -f 2 -d '>' | cut -f 1 -d \( | cut -f 2 -d ' ' | xargs -i cp \{\} $OUT/lib/
+
cp $x $OUT/
bin=`basename $x`
- # Copy any system libraries needed by this fuzzer to $OUT/lib
- ldd $OUT/$bin | cut -f 2 -d '>' | cut -f 1 -d \( | cut -f 2 -d ' ' | xargs -i cp \{\} $OUT/lib/
-
- # Change any RPATH to RUNPATH.
- #
- # We use ld.bfd for the coverage builds, rather than the faster ld.gold.
+ # Changing RPATH (not RUNPATH, but we can't tell here which was
+ # set) is critical, otherwise libraries used by libraries won't be
+ # found on the oss-fuzz target host. Sadly this is only possible
+ # with clang or ld.bfd on Ubuntu 16.04 (this script is only run on
+ # that).
#
- # On Ubuntu 16.04, used for the oss-fuzz build, when linking with
- # ld.bfd the binaries get a RPATH, but builds in Ubuntu 18.04
- # ld.bfd and those using ld.gold get a RUNPATH.
+ # chrpath --convert only allows RPATH to be changed to RUNPATH,
+ # not the other way around, and we really don't want RUNPATH.
#
- # Just convert them all to RUNPATH to make the check_build.sh test
- # easier.
- chrpath -c $OUT/$bin
- # Change RUNPATH so that the copied libraries are found on the
- # runner
+ # This means the copied libraries are found on the runner
chrpath -r '$ORIGIN/lib' $OUT/$bin
# Truncate the original binary to save space
diff --git a/lib/fuzzing/oss-fuzz/check_build.sh b/lib/fuzzing/oss-fuzz/check_build.sh
index b971d2c1bb0..501c2c813fc 100755
--- a/lib/fuzzing/oss-fuzz/check_build.sh
+++ b/lib/fuzzing/oss-fuzz/check_build.sh
@@ -23,12 +23,21 @@ do
continue
fi
# Confirm that the chrpath was reset to lib/ in the same directory
- # as the binary
- chrpath -l $bin | grep 'RUNPATH=$ORIGIN/lib'
+ # as the binary. RPATH (not RUNPATH) is critical, otherwise
+ # libraries used by libraries won't be found on the oss-fuzz
+ # target host, but is only possible with clang or ld.bfd on Ubuntu
+ # 16.04 (this script is only run on that).
+ chrpath -l $bin | grep 'RPATH=$ORIGIN/lib'
# Confirm that we link to at least some libraries in this
# directory (shows that the libraries were found and copied).
ldd $bin | grep "$OUT/lib"
+ num_libs=$(ldd $bin | grep -v ld-linux | grep -v linux-vdso | grep -v "$OUT/lib"| wc -l)
+
+ if [ 0$num_libs -ne 0 ]; then
+ echo "some libraries not linked to $ORIGIN/lib, oss-fuzz will fail!"
+ exit 1
+ fi
if [ -f ${bin}_seed_corpus.zip ]; then
seeds_found=yes
diff --git a/script/autobuild.py b/script/autobuild.py
index 0bf80d5e821..00ed3a47022 100755
--- a/script/autobuild.py
+++ b/script/autobuild.py
@@ -651,7 +651,7 @@ tasks = {
"samba-fuzz": [
# build the fuzzers (static) via the oss-fuzz script
("fuzzers-mkdir-prefix", "mkdir -p ${PREFIX_DIR}"),
- ("fuzzers-build", "OUT=${PREFIX_DIR} LIB_FUZZING_ENGINE= SANITIZER=address CXX= CFLAGS= ./lib/fuzzing/oss-fuzz/build_samba.sh --enable-afl"),
+ ("fuzzers-build", "OUT=${PREFIX_DIR} LIB_FUZZING_ENGINE= SANITIZER=address CXX= CFLAGS= ADDITIONAL_LDFLAGS='-fuse-ld=bfd' ./lib/fuzzing/oss-fuzz/build_samba.sh --enable-afl"),
("fuzzers-check", "./lib/fuzzing/oss-fuzz/check_build.sh ${PREFIX_DIR}")
],
--
Samba Shared Repository
More information about the samba-cvs
mailing list