[SCM] Samba Shared Repository - branch master updated
Douglas Bagnall
dbagnall at samba.org
Thu Nov 21 00:46:02 UTC 2019
The branch, master has been updated
via f8947538b5e lib/fuzzing: Add oss-fuzz info to README.md
via cc128c78856 lib/fuzzing/oss-fuzz: copy required libraries to the build target
via 4946811eb67 lib/fuzzing/oss-fuzz: Install chrpath as we use it in the build.sh script to set -rpath
via fbb2377d51f lib/fuzzing/oss-fuzz: Add build_image.sh using Samba's bootstrap tools
via ec4f6f8fd36 lib/fuzzing/oss-fuzz: add stub build.sh that will not change often
via f57c0238e91 lib/fuzzing/oss-fuzz: Add build_samba.sh for oss-fuzz
via 92ee6478583 lib/fuzzing: Use --fuzz-target-ldflags if specified
via cb240510977 build: Set fuzzer=True on fuzzer binaries
from 9e51bb77f82 s4:rpc_server: Return the status code from dcesrv_transport_session_key()
https://git.samba.org/?p=samba.git;a=shortlog;h=master
- Log -----------------------------------------------------------------
commit f8947538b5e445ab9c1931d1f7826771ee582385
Author: Andrew Bartlett <abartlet at samba.org>
Date: Thu Nov 21 10:21:54 2019 +1300
lib/fuzzing: Add oss-fuzz info to README.md
Note that Samba has not been accepted yet, but will be soon once some requirements
are addressed per:
https://github.com/google/oss-fuzz/pull/2993
Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Uri Simchoni <uri at samba.org>
Autobuild-User(master): Douglas Bagnall <dbagnall at samba.org>
Autobuild-Date(master): Thu Nov 21 00:45:33 UTC 2019 on sn-devel-184
commit cc128c788562d6d86c24cbcb784e1e8fca1f06a1
Author: Andrew Bartlett <abartlet at samba.org>
Date: Thu Nov 7 14:22:07 2019 +1300
lib/fuzzing/oss-fuzz: copy required libraries to the build target
This is an alternative to static linking as we do not have static source
libraries for all the things we depend on.
Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Uri Simchoni <uri at samba.org>
commit 4946811eb67c6e4e6abdfd64a552e7a2686feeec
Author: Andrew Bartlett <abartlet at samba.org>
Date: Thu Nov 7 17:17:40 2019 +1300
lib/fuzzing/oss-fuzz: Install chrpath as we use it in the build.sh script to set -rpath
Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Uri Simchoni <uri at samba.org>
commit fbb2377d51f00617e9065a8a3daecc8f5c340ca0
Author: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Date: Thu Nov 21 09:59:24 2019 +1300
lib/fuzzing/oss-fuzz: Add build_image.sh using Samba's bootstrap tools
Google's oss-fuzz environment is Ubuntu 16.04 based so we can
just use the maintained bootstrap system rather than a manual
package list here that will get out of date.
Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Uri Simchoni <uri at samba.org>
Pair-programmed-by: Andrew Bartlett <abartlet at samba.org>
commit ec4f6f8fd36edfda8b949dcf7451c153f6599427
Author: Andrew Bartlett <abartlet at samba.org>
Date: Thu Nov 21 10:29:57 2019 +1300
lib/fuzzing/oss-fuzz: add stub build.sh that will not change often
This makes local development of build_samba.sh easier as it will remain in the source tree.
Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Uri Simchoni <uri at samba.org>
commit f57c0238e91061698787e4a4b18fb549cc2c86f2
Author: Andrew Bartlett <abartlet at samba.org>
Date: Wed Nov 20 14:03:13 2019 +1300
lib/fuzzing/oss-fuzz: Add build_samba.sh for oss-fuzz
We work hard to put the primary logic for oss-fuzz here, and
where possible into waf, so that only a tiny stub needs to
be maintained in the Google oss-fuzz repo.
This will be called by build.sh (not copied directly because
it is too easy to forget to copy in an updated version when
doing development in the docker image).
Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Uri Simchoni <uri at samba.org>
commit 92ee647858331db012db06314d98e08fe1e5eb5b
Author: Andrew Bartlett <abartlet at samba.org>
Date: Wed Nov 6 12:24:18 2019 +1300
lib/fuzzing: Use --fuzz-target-ldflags if specified
This makes integration with oss-fuzz possible. Only the fuzzer binaries should be
linked with libFuzzer, not things like asn1_compile, so this can not be done via
the global ADDITIONAL_LDFLAGS.
Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Uri Simchoni <uri at samba.org>
commit cb240510977e60fd46ef0f35b3da77f00733829a
Author: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Date: Fri Nov 1 17:18:24 2019 +1300
build: Set fuzzer=True on fuzzer binaries
This ensures that the binaries are the only binaries built
when configured for fuzzing.
Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Uri Simchoni <uri at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
-----------------------------------------------------------------------
Summary of changes:
buildtools/wafsamba/wafsamba.py | 15 +++++++++++-
buildtools/wafsamba/wscript | 10 +++++++-
lib/fuzzing/README.md | 26 +++++++++++++++++++++
lib/fuzzing/oss-fuzz/build.sh | 6 +++++
lib/fuzzing/oss-fuzz/build_image.sh | 11 +++++++++
lib/fuzzing/oss-fuzz/build_samba.sh | 46 +++++++++++++++++++++++++++++++++++++
lib/fuzzing/wscript_build | 29 +++++++----------------
7 files changed, 120 insertions(+), 23 deletions(-)
create mode 100644 lib/fuzzing/oss-fuzz/build.sh
create mode 100755 lib/fuzzing/oss-fuzz/build_image.sh
create mode 100755 lib/fuzzing/oss-fuzz/build_samba.sh
Changeset truncated at 500 lines:
diff --git a/buildtools/wafsamba/wafsamba.py b/buildtools/wafsamba/wafsamba.py
index 7081f382eaa..7460fbae020 100644
--- a/buildtools/wafsamba/wafsamba.py
+++ b/buildtools/wafsamba/wafsamba.py
@@ -360,13 +360,26 @@ def SAMBA_BINARY(bld, binname, source,
subdir=None,
install=True,
install_path=None,
- enabled=True):
+ enabled=True,
+ fuzzer=False):
'''define a Samba binary'''
if not enabled:
SET_TARGET_TYPE(bld, binname, 'DISABLED')
return
+ # Fuzzing builds do not build normal binaries
+ # however we must build asn1compile etc
+
+ if not use_hostcc and bld.env.enable_libfuzzer != fuzzer:
+ SET_TARGET_TYPE(bld, binname, 'DISABLED')
+ return
+
+ if fuzzer:
+ install = False
+ if ldflags is None:
+ ldflags = bld.env['FUZZ_TARGET_LDFLAGS']
+
if not SET_TARGET_TYPE(bld, binname, 'BINARY'):
return
diff --git a/buildtools/wafsamba/wscript b/buildtools/wafsamba/wscript
index b601dd2596d..9987c6e4fcc 100644
--- a/buildtools/wafsamba/wscript
+++ b/buildtools/wafsamba/wscript
@@ -130,9 +130,16 @@ def options(opt):
dest='undefined_sanitizer',
default=False)
gr.add_option('--enable-libfuzzer',
- help=("Build fuzzing binaries (requires compiler options for libFuzzer or compiler wrapper such as honggfuzz/hfuzz-cc)"),
+ help=("Build fuzzing binaries (use ADDITIONAL_CFLAGS to specify compiler options for libFuzzer or use a compiler wrapper such as honggfuzz/hfuzz-cc)"),
action="store_true", dest='enable_libfuzzer', default=False)
+ # Fuzz targets may need additional LDFLAGS that we can't use on
+ # internal binaries like asn1_compile
+
+ gr.add_option('--fuzz-target-ldflags',
+ help=("Linker flags to be used when building fuzz targets"),
+ action="store", dest='FUZZ_TARGET_LDFLAGS', default='')
+
gr.add_option('--abi-check',
help=("Check ABI signatures for libraries"),
action='store_true', dest='ABI_CHECK', default=False)
@@ -596,6 +603,7 @@ struct foo bar = { .y = 'X', .x = 1 };
conf.env.enable_libfuzzer = Options.options.enable_libfuzzer
if conf.env.enable_libfuzzer:
conf.DEFINE('ENABLE_LIBFUZZER', 1)
+ conf.env.FUZZ_TARGET_LDFLAGS = Options.options.FUZZ_TARGET_LDFLAGS
conf.load('clang_compilation_database')
diff --git a/lib/fuzzing/README.md b/lib/fuzzing/README.md
index 3848838ba02..97b49ed0fb1 100644
--- a/lib/fuzzing/README.md
+++ b/lib/fuzzing/README.md
@@ -32,4 +32,30 @@ buildtools/bin/waf --targets=fuzz_tiniparser build && \
--rlimit_rss 100 -f .../tiniparser-corpus -- bin/fuzz_tiniparser
```
+# oss-fuzz
+
+Samba can be fuzzed by Google's oss-fuzz system. Assuming you have an
+oss-fuzz checkout from https://github.com/google/oss-fuzz with Samba's
+metadata in projects/samba, the following guides will help:
+
+## Testing locally
+
+https://google.github.io/oss-fuzz/getting-started/new-project-guide/#testing-locally
+
+## Debugging oss-fuzz
+
+See https://google.github.io/oss-fuzz/advanced-topics/debugging/
+
+## Samba-specific hints
+
+A typical debugging workflow is:
+
+oss-fuzz$ python infra/helper.py shell samba
+git fetch $REMOTE $BRANCH
+git checkout FETCH_HEAD
+lib/fuzzing/oss-fuzz/build_image.sh
+compile
+
+This will pull in any new Samba deps and build Samba's fuzzers.
+
# vim: set sw=8 sts=8 ts=8 tw=79 :
diff --git a/lib/fuzzing/oss-fuzz/build.sh b/lib/fuzzing/oss-fuzz/build.sh
new file mode 100644
index 00000000000..14ba6ee83b2
--- /dev/null
+++ b/lib/fuzzing/oss-fuzz/build.sh
@@ -0,0 +1,6 @@
+#!/bin/sh -e
+
+# This will be copied into $SRC, being the root of the source tree by
+# build_image.sh
+
+exec lib/fuzzing/oss-fuzz/build_samba.sh
diff --git a/lib/fuzzing/oss-fuzz/build_image.sh b/lib/fuzzing/oss-fuzz/build_image.sh
new file mode 100755
index 00000000000..fa899effc37
--- /dev/null
+++ b/lib/fuzzing/oss-fuzz/build_image.sh
@@ -0,0 +1,11 @@
+#!/bin/sh -e
+
+DIST=ubuntu1604
+SCRIPT_DIR=`dirname $0`
+
+$SCRIPT_DIR/../../../bootstrap/generated-dists/$DIST/bootstrap.sh
+$SCRIPT_DIR/../../../bootstrap/generated-dists/$DIST/locale.sh
+
+apt-get install chrpath
+
+cp $SCRIPT_DIR/build.sh $SRC/
diff --git a/lib/fuzzing/oss-fuzz/build_samba.sh b/lib/fuzzing/oss-fuzz/build_samba.sh
new file mode 100755
index 00000000000..aa8e223b0a1
--- /dev/null
+++ b/lib/fuzzing/oss-fuzz/build_samba.sh
@@ -0,0 +1,46 @@
+#!/bin/sh -e
+#
+# This is not a general-purpose build script, but instead one specific to the Google oss-fuzz compile environment.
+#
+# https://google.github.io/oss-fuzz/getting-started/new-project-guide/#Requirements
+#
+# https://github.com/google/oss-fuzz/blob/master/infra/base-images/base-builder/README.md#provided-environment-variables
+#
+# We have to push to oss-fuzz CFLAGS into the waf ADDITIONAL_CFLAGS
+# as otherwise waf's configure fails linking the first test binary
+#
+# CFLAGS are supplied by the caller, eg the oss-fuzz compile command
+#
+ADDITIONAL_CFLAGS="$CFLAGS"
+export ADDITIONAL_CFLAGS
+CFLAGS=""
+export CFLAGS
+LD="$CXX"
+export LD
+
+# $LIB_FUZZING_ENGINE is provided by the oss-fuzz "compile" command
+#
+
+./configure -C --without-gettext --enable-debug --enable-developer \
+ --address-sanitizer --enable-libfuzzer \
+ --disable-warnings-as-errors \
+ --abi-check-disable \
+ --fuzz-target-ldflags="$LIB_FUZZING_ENGINE" \
+ --nonshared-binary=ALL LINK_CC="$CXX"
+
+make -j
+
+# Make a directory for the system shared libraries to be copied into
+mkdir -p $OUT/lib
+
+# We can't static link to all the system libs with waf, so copy them
+# to $OUT/lib and set the rpath to point there. This is similar to how
+# firefox handles this.
+
+for x in bin/fuzz_*
+do
+ cp $x $OUT/
+ bin=`basename $x`
+ ldd $OUT/$bin | cut -f 2 -d '>' | cut -f 1 -d \( | cut -f 2 -d ' ' | xargs -i cp \{\} $OUT/lib/
+ chrpath -r '$ORIGIN/lib' $OUT/$bin
+done
diff --git a/lib/fuzzing/wscript_build b/lib/fuzzing/wscript_build
index 7305ce41262..386145c43b2 100644
--- a/lib/fuzzing/wscript_build
+++ b/lib/fuzzing/wscript_build
@@ -3,53 +3,40 @@
bld.SAMBA_SUBSYSTEM('fuzzing',
source='fuzzing.c',
deps='talloc',
- enabled=bld.env.enable_libfuzzer,
+ enabled=bld.env.enable_libfuzzer
)
bld.SAMBA_BINARY('fuzz_tiniparser',
source='fuzz_tiniparser.c',
deps='fuzzing tiniparser talloc',
- install=False,
- enabled=bld.env.enable_libfuzzer)
+ fuzzer=True)
bld.SAMBA_BINARY('fuzz_oLschema2ldif',
source='fuzz_oLschema2ldif.c',
deps='fuzzing oLschema2ldif-lib',
- install=False,
- enabled=bld.env.enable_libfuzzer,
- )
+ fuzzer=True)
bld.SAMBA_BINARY('fuzz_reg_parse',
source='fuzz_reg_parse.c',
deps='fuzzing samba3-util smbconf REGFIO',
- install=False,
- enabled=bld.env.enable_libfuzzer,
- )
+ fuzzer=True)
bld.SAMBA_BINARY('fuzz_regfio',
source='fuzz_regfio.c',
deps='fuzzing samba3-util smbconf REGFIO',
- install=False,
- enabled=bld.env.enable_libfuzzer,
- )
+ fuzzer=True)
bld.SAMBA_BINARY('fuzz_lzxpress',
source='fuzz_lzxpress.c',
deps='fuzzing LZXPRESS',
- install=False,
- enabled=bld.env.enable_libfuzzer,
- )
+ fuzzer=True)
bld.SAMBA_BINARY('fuzz_ldap_decode',
source='fuzz_ldap_decode.c',
deps='fuzzing cli-ldap',
- install=False,
- enabled=bld.env.enable_libfuzzer,
- )
+ fuzzer=True)
bld.SAMBA_BINARY('fuzz_ldb_parse_tree',
source='fuzz_ldb_parse_tree.c',
deps='fuzzing ldb',
- install=False,
- enabled=bld.env.enable_libfuzzer
- )
+ fuzzer=True)
--
Samba Shared Repository
More information about the samba-cvs
mailing list