[SCM] Samba Shared Repository - branch master updated
Jeremy Allison
jra at samba.org
Thu Aug 11 19:24:02 UTC 2022
The branch, master has been updated
via 23988f19e7c s4:torture/smb2: add smb2.bench.echo
via 8ee783c4803 s4:torture/smb2: teach smb2.bench.path-contention-shared about --option="torture:qdepth=4"
via 72caffbe111 s4:param: add --option="libsmb:client_guid=6112f7d3-9528-4a2a-8861-0ca129aae6c4" support...
via cd01f513469 s3:g_lock: use TDB_VOLATILE to avoid fcntl locks
via a0a97d27f7a smbd: avoid calling SMB_VFS_FGET_NT_ACL() if do_not_check_mask already covers all
via 8c7e8c5f80f s3:include: remove unused update_stat_ex_file_id() prototype
from 1b470aaa67b s3:passdb: Consolidate error checking in fetch_ldap_pw()
https://git.samba.org/?p=samba.git;a=shortlog;h=master
- Log -----------------------------------------------------------------
commit 23988f19e7cc2823d6c0c0f40af0195d0a3b81bf
Author: Stefan Metzmacher <metze at samba.org>
Date: Wed Aug 10 13:14:52 2022 +0000
s4:torture/smb2: add smb2.bench.echo
This test calls SMB2_Echo in a loop per connection.
For 4 connections with 2 parallel loops use this:
time smbtorture //127.0.0.1/m -Uroot%test smb2.bench.echo \
--option="torture:timelimit=600" \
--option="torture:nprocs=1" \
--option="torture:qdepth=2"
Sometimes the bottleneck is the smbtorture process.
In order to bring the smbd process to 100% cpu, you can use
'--option="libsmb:client_guid=6112f7d3-9528-4a2a-8861-0ca129aae6c4"'
and run multiple instances of the test at the same time,
which both talk to the same smbd process.
This is a very useful test to show how many requests are possible
at the raw SMB2 layer.
Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Thu Aug 11 19:23:37 UTC 2022 on sn-devel-184
commit 8ee783c4803d28cccc39144afa7b78c4b9e0cc2e
Author: Stefan Metzmacher <metze at samba.org>
Date: Wed Aug 10 11:43:20 2022 +0000
s4:torture/smb2: teach smb2.bench.path-contention-shared about --option="torture:qdepth=4"
This can now test more than one open/close loop per connection.
time smbtorture //127.0.0.1/m -Uroot%test \
smb2.create.bench-path-contention-shared \
--option='torture:bench_path=' \
--option="torture:timelimit=60" \
--option="torture:nprocs=1" \
--option="torture:qdepth=4"
The default is still 1, but it's very useful for tests.
Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
commit 72caffbe1115c57ad38270eaeb951f6b97bf62b3
Author: Stefan Metzmacher <metze at samba.org>
Date: Wed Aug 10 13:15:45 2022 +0000
s4:param: add --option="libsmb:client_guid=6112f7d3-9528-4a2a-8861-0ca129aae6c4" support...
We already handle this in the source3/libsmb code, but it's good to
have this also for torture tests.
Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
commit cd01f5134696f7789fbc2933629ac2606feb0b5e
Author: Stefan Metzmacher <metze at samba.org>
Date: Wed Aug 10 13:16:14 2022 +0000
s3:g_lock: use TDB_VOLATILE to avoid fcntl locks
This improves 'time smbtorture3 //foo/bar -U% local-g-lock-ping-pong -o 50000000'
from ~1.400.000 to ~3.400.000 operations per second any a testsystem.
As we also use TDB_VOLATILE for locking.tdb, this is a much more
realistic test now.
Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
commit a0a97d27f7a60dbd86317b51bec0ece2476e8c8d
Author: Stefan Metzmacher <metze at samba.org>
Date: Tue Aug 9 14:07:12 2022 +0000
smbd: avoid calling SMB_VFS_FGET_NT_ACL() if do_not_check_mask already covers all
This is inspired by 0d4cb5a641e1fea2d369bdc66470a580321366c2,
which avoids SMB_VFS_FGET_NT_ACL() for the root user again.
Opens with just FILE_READ_ATTRIBUTES are very common, so it's worth
optimizing for it.
Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
commit 8c7e8c5f80f1488456f9dd6225020d29f74458d2
Author: Stefan Metzmacher <metze at samba.org>
Date: Wed Aug 10 16:45:26 2022 +0200
s3:include: remove unused update_stat_ex_file_id() prototype
It was removed by commit 643da37fd139413651a6198fb0f6e550f7de6584
Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
-----------------------------------------------------------------------
Summary of changes:
source3/include/proto.h | 1 -
source3/lib/g_lock.c | 2 +-
source3/smbd/open.c | 69 +++--
source4/param/loadparm.c | 11 +-
source4/torture/smb2/create.c | 591 ++++++++++++++++++++++++++++++++++--------
5 files changed, 538 insertions(+), 136 deletions(-)
Changeset truncated at 500 lines:
diff --git a/source3/include/proto.h b/source3/include/proto.h
index aebbd7f9c47..6a6edc36dfb 100644
--- a/source3/include/proto.h
+++ b/source3/include/proto.h
@@ -171,7 +171,6 @@ int sys_fcntl_long(int fd, int cmd, long arg);
int sys_fcntl_int(int fd, int cmd, int arg);
void update_stat_ex_mtime(struct stat_ex *dst, struct timespec write_ts);
void update_stat_ex_create_time(struct stat_ex *dst, struct timespec create_time);
-void update_stat_ex_file_id(struct stat_ex *dst, uint64_t file_id);
void update_stat_ex_from_saved_stat(struct stat_ex *dst,
const struct stat_ex *src);
int sys_stat(const char *fname, SMB_STRUCT_STAT *sbuf,
diff --git a/source3/lib/g_lock.c b/source3/lib/g_lock.c
index 52a1cf9fa46..d683c4ddee6 100644
--- a/source3/lib/g_lock.c
+++ b/source3/lib/g_lock.c
@@ -236,7 +236,7 @@ struct g_lock_ctx *g_lock_ctx_init(TALLOC_CTX *mem_ctx,
mem_ctx,
db_path,
0,
- TDB_CLEAR_IF_FIRST|TDB_INCOMPATIBLE_HASH,
+ TDB_CLEAR_IF_FIRST|TDB_INCOMPATIBLE_HASH|TDB_VOLATILE,
O_RDWR|O_CREAT,
0600,
DBWRAP_LOCK_ORDER_3,
diff --git a/source3/smbd/open.c b/source3/smbd/open.c
index 6d3dee9ce18..fa3be0a6e3b 100644
--- a/source3/smbd/open.c
+++ b/source3/smbd/open.c
@@ -99,9 +99,11 @@ static NTSTATUS smbd_check_access_rights_fname(
struct connection_struct *conn,
const struct smb_filename *smb_fname,
bool use_privs,
- uint32_t access_mask)
+ uint32_t access_mask,
+ uint32_t do_not_check_mask)
{
uint32_t rejected_share_access;
+ uint32_t effective_access;
rejected_share_access = access_mask & ~(conn->share_access);
@@ -114,6 +116,14 @@ static NTSTATUS smbd_check_access_rights_fname(
return NT_STATUS_ACCESS_DENIED;
}
+ effective_access = access_mask & ~do_not_check_mask;
+ if (effective_access == 0) {
+ DBG_DEBUG("do_not_check_mask override on %s. Granting 0x%x for free.\n",
+ smb_fname_str_dbg(smb_fname),
+ (unsigned int)access_mask);
+ return NT_STATUS_OK;
+ }
+
if (!use_privs && get_current_uid(conn) == (uid_t)0) {
/* I'm sorry sir, I didn't know you were root... */
DBG_DEBUG("root override on %s. Granting 0x%x\n",
@@ -151,39 +161,16 @@ static NTSTATUS smbd_check_access_rights_sd(
const struct smb_filename *smb_fname,
struct security_descriptor *sd,
bool use_privs,
- uint32_t access_mask)
+ uint32_t access_mask,
+ uint32_t do_not_check_mask)
{
uint32_t rejected_mask = access_mask;
- uint32_t do_not_check_mask = 0;
NTSTATUS status;
if (sd == NULL) {
goto access_denied;
}
- /*
- * If we can access the path to this file, by
- * default we have FILE_READ_ATTRIBUTES from the
- * containing directory. See the section:
- * "Algorithm to Check Access to an Existing File"
- * in MS-FSA.pdf.
- *
- * se_file_access_check() also takes care of
- * owner WRITE_DAC and READ_CONTROL.
- */
- do_not_check_mask = FILE_READ_ATTRIBUTES;
-
- /*
- * Samba 3.6 and earlier granted execute access even
- * if the ACL did not contain execute rights.
- * Samba 4.0 is more correct and checks it.
- * The compatibilty mode allows one to skip this check
- * to smoothen upgrades.
- */
- if (lp_acl_allow_execute_always(SNUM(conn))) {
- do_not_check_mask |= FILE_EXECUTE;
- }
-
status = se_file_access_check(sd,
get_current_nttok(conn),
use_privs,
@@ -263,6 +250,7 @@ NTSTATUS smbd_check_access_rights_fsp(struct files_struct *dirfsp,
uint32_t access_mask)
{
struct security_descriptor *sd = NULL;
+ uint32_t do_not_check_mask = 0;
NTSTATUS status;
/* Cope with fake/printer fsp's. */
@@ -288,10 +276,34 @@ NTSTATUS smbd_check_access_rights_fsp(struct files_struct *dirfsp,
return NT_STATUS_OK;
}
+ /*
+ * If we can access the path to this file, by
+ * default we have FILE_READ_ATTRIBUTES from the
+ * containing directory. See the section:
+ * "Algorithm to Check Access to an Existing File"
+ * in MS-FSA.pdf.
+ *
+ * se_file_access_check() also takes care of
+ * owner WRITE_DAC and READ_CONTROL.
+ */
+ do_not_check_mask = FILE_READ_ATTRIBUTES;
+
+ /*
+ * Samba 3.6 and earlier granted execute access even
+ * if the ACL did not contain execute rights.
+ * Samba 4.0 is more correct and checks it.
+ * The compatibilty mode allows one to skip this check
+ * to smoothen upgrades.
+ */
+ if (lp_acl_allow_execute_always(SNUM(fsp->conn))) {
+ do_not_check_mask |= FILE_EXECUTE;
+ }
+
status = smbd_check_access_rights_fname(fsp->conn,
fsp->fsp_name,
use_privs,
- access_mask);
+ access_mask,
+ do_not_check_mask);
if (!NT_STATUS_EQUAL(status, NT_STATUS_MORE_PROCESSING_REQUIRED)) {
return status;
}
@@ -314,7 +326,8 @@ NTSTATUS smbd_check_access_rights_fsp(struct files_struct *dirfsp,
fsp->fsp_name,
sd,
use_privs,
- access_mask);
+ access_mask,
+ do_not_check_mask);
}
/*
diff --git a/source4/param/loadparm.c b/source4/param/loadparm.c
index 81892d540ef..7d3891dc0b9 100644
--- a/source4/param/loadparm.c
+++ b/source4/param/loadparm.c
@@ -35,6 +35,15 @@
void lpcfg_smbcli_options(struct loadparm_context *lp_ctx,
struct smbcli_options *options)
{
+ struct GUID client_guid;
+ const char *str = NULL;
+
+ str = lpcfg_parm_string(lp_ctx, NULL, "libsmb", "client_guid");
+ if (str != NULL) {
+ GUID_from_string(str, &client_guid);
+ } else {
+ client_guid = GUID_random();
+ }
*options = (struct smbcli_options) {
.max_xmit = lpcfg_max_xmit(lp_ctx),
.max_mux = lpcfg_max_mux(lp_ctx),
@@ -48,7 +57,7 @@ void lpcfg_smbcli_options(struct loadparm_context *lp_ctx,
.use_oplocks = true,
.use_level2_oplocks = true,
.smb2_capabilities = SMB2_CAP_ALL,
- .client_guid = GUID_random(),
+ .client_guid = client_guid,
.max_credits = WINDOWS_CLIENT_PURE_SMB2_NEGPROT_INITIAL_CREDIT_ASK,
.smb3_capabilities = smb311_capabilities_parse("client",
lpcfg_client_smb3_signing_algorithms(lp_ctx),
diff --git a/source4/torture/smb2/create.c b/source4/torture/smb2/create.c
index 8f1e94b5773..9277488e9d7 100644
--- a/source4/torture/smb2/create.c
+++ b/source4/torture/smb2/create.c
@@ -22,6 +22,7 @@
#include "includes.h"
#include "libcli/smb2/smb2.h"
#include "libcli/smb2/smb2_calls.h"
+#include "libcli/smb/smbXcli_base.h"
#include "torture/torture.h"
#include "torture/util.h"
#include "torture/smb2/proto.h"
@@ -126,6 +127,333 @@
__location__, (unsigned int)(sattrib), fname); \
}} while (0)
+/*
+ stress testing keepalive iops
+ */
+
+struct test_smb2_bench_echo_conn;
+struct test_smb2_bench_echo_loop;
+
+struct test_smb2_bench_echo_state {
+ struct torture_context *tctx;
+ size_t num_conns;
+ struct test_smb2_bench_echo_conn *conns;
+ size_t num_loops;
+ struct test_smb2_bench_echo_loop *loops;
+ struct timeval starttime;
+ int timecount;
+ int timelimit;
+ uint64_t num_finished;
+ double total_latency;
+ double min_latency;
+ double max_latency;
+ bool ok;
+ bool stop;
+};
+
+struct test_smb2_bench_echo_conn {
+ struct test_smb2_bench_echo_state *state;
+ int idx;
+ struct smb2_tree *tree;
+};
+
+struct test_smb2_bench_echo_loop {
+ struct test_smb2_bench_echo_state *state;
+ struct test_smb2_bench_echo_conn *conn;
+ int idx;
+ struct tevent_immediate *im;
+ struct tevent_req *req;
+ struct timeval starttime;
+ uint64_t num_started;
+ uint64_t num_finished;
+ double total_latency;
+ double min_latency;
+ double max_latency;
+ NTSTATUS error;
+};
+
+static void test_smb2_bench_echo_loop_do(
+ struct test_smb2_bench_echo_loop *loop);
+
+static void test_smb2_bench_echo_loop_start(struct tevent_context *ctx,
+ struct tevent_immediate *im,
+ void *private_data)
+{
+ struct test_smb2_bench_echo_loop *loop =
+ (struct test_smb2_bench_echo_loop *)
+ private_data;
+
+ test_smb2_bench_echo_loop_do(loop);
+}
+
+static void test_smb2_bench_echo_loop_done(struct tevent_req *req);
+
+static void test_smb2_bench_echo_loop_do(
+ struct test_smb2_bench_echo_loop *loop)
+{
+ struct test_smb2_bench_echo_state *state = loop->state;
+
+ loop->num_started += 1;
+ loop->starttime = timeval_current();
+ loop->req = smb2cli_echo_send(state->loops,
+ state->tctx->ev,
+ loop->conn->tree->session->transport->conn,
+ 1000);
+ torture_assert_goto(state->tctx, loop->req != NULL,
+ state->ok, asserted, "smb2_create_send");
+
+ tevent_req_set_callback(loop->req,
+ test_smb2_bench_echo_loop_done,
+ loop);
+ return;
+asserted:
+ state->stop = true;
+}
+
+static void test_smb2_bench_echo_loop_done(struct tevent_req *req)
+{
+ struct test_smb2_bench_echo_loop *loop =
+ (struct test_smb2_bench_echo_loop *)
+ _tevent_req_callback_data(req);
+ struct test_smb2_bench_echo_state *state = loop->state;
+ double latency = timeval_elapsed(&loop->starttime);
+ TALLOC_CTX *frame = talloc_stackframe();
+
+ torture_assert_goto(state->tctx, loop->req == req,
+ state->ok, asserted, __location__);
+ loop->error = smb2cli_echo_recv(req);
+ torture_assert_ntstatus_ok_goto(state->tctx, loop->error,
+ state->ok, asserted, __location__);
+ SMB_ASSERT(latency >= 0.000001);
+
+ if (loop->num_finished == 0) {
+ /* first round */
+ loop->min_latency = latency;
+ loop->max_latency = latency;
+ }
+
+ loop->num_finished += 1;
+ loop->total_latency += latency;
+
+ if (latency < loop->min_latency) {
+ loop->min_latency = latency;
+ }
+
+ if (latency > loop->max_latency) {
+ loop->max_latency = latency;
+ }
+
+ TALLOC_FREE(frame);
+ test_smb2_bench_echo_loop_do(loop);
+ return;
+asserted:
+ state->stop = true;
+ TALLOC_FREE(frame);
+}
+
+static void test_smb2_bench_echo_progress(struct tevent_context *ev,
+ struct tevent_timer *te,
+ struct timeval current_time,
+ void *private_data)
+{
+ struct test_smb2_bench_echo_state *state =
+ (struct test_smb2_bench_echo_state *)private_data;
+ uint64_t num_echos = 0;
+ double total_echo_latency = 0;
+ double min_echo_latency = 0;
+ double max_echo_latency = 0;
+ double avs_echo_latency = 0;
+ size_t i;
+
+ state->timecount += 1;
+
+ for (i=0;i<state->num_loops;i++) {
+ struct test_smb2_bench_echo_loop *loop =
+ &state->loops[i];
+
+ num_echos += loop->num_finished;
+ total_echo_latency += loop->total_latency;
+ if (min_echo_latency == 0.0 && loop->min_latency != 0.0) {
+ min_echo_latency = loop->min_latency;
+ }
+ if (loop->min_latency < min_echo_latency) {
+ min_echo_latency = loop->min_latency;
+ }
+ if (max_echo_latency == 0.0) {
+ max_echo_latency = loop->max_latency;
+ }
+ if (loop->max_latency > max_echo_latency) {
+ max_echo_latency = loop->max_latency;
+ }
+ loop->num_finished = 0;
+ loop->total_latency = 0.0;
+ }
+
+ state->num_finished += num_echos;
+ state->total_latency += total_echo_latency;
+ if (state->min_latency == 0.0 && min_echo_latency != 0.0) {
+ state->min_latency = min_echo_latency;
+ }
+ if (min_echo_latency < state->min_latency) {
+ state->min_latency = min_echo_latency;
+ }
+ if (state->max_latency == 0.0) {
+ state->max_latency = max_echo_latency;
+ }
+ if (max_echo_latency > state->max_latency) {
+ state->max_latency = max_echo_latency;
+ }
+
+ if (state->timecount < state->timelimit) {
+ te = tevent_add_timer(state->tctx->ev,
+ state,
+ timeval_current_ofs(1, 0),
+ test_smb2_bench_echo_progress,
+ state);
+ torture_assert_goto(state->tctx, te != NULL,
+ state->ok, asserted, "tevent_add_timer");
+
+ if (!torture_setting_bool(state->tctx, "progress", true)) {
+ return;
+ }
+
+ avs_echo_latency = total_echo_latency / num_echos;
+
+ torture_comment(state->tctx,
+ "%.2f second: "
+ "echo[num/s=%llu,avslat=%.6f,minlat=%.6f,maxlat=%.6f] \r",
+ timeval_elapsed(&state->starttime),
+ (unsigned long long)num_echos,
+ avs_echo_latency,
+ min_echo_latency,
+ max_echo_latency);
+ return;
+ }
+
+ avs_echo_latency = state->total_latency / state->num_finished;
+ num_echos = state->num_finished / state->timelimit;
+
+ torture_comment(state->tctx,
+ "%.2f second: "
+ "echo[num/s=%llu,avslat=%.6f,minlat=%.6f,maxlat=%.6f]\n",
+ timeval_elapsed(&state->starttime),
+ (unsigned long long)num_echos,
+ avs_echo_latency,
+ state->min_latency,
+ state->max_latency);
+
+asserted:
+ state->stop = true;
+}
+
+static bool test_smb2_bench_echo(struct torture_context *tctx,
+ struct smb2_tree *tree)
+{
+ struct test_smb2_bench_echo_state *state = NULL;
+ bool ret = true;
+ int torture_nprocs = torture_setting_int(tctx, "nprocs", 4);
+ int torture_qdepth = torture_setting_int(tctx, "qdepth", 1);
+ size_t i;
+ size_t li = 0;
+ int timelimit = torture_setting_int(tctx, "timelimit", 10);
+ struct tevent_timer *te = NULL;
+ uint32_t timeout_msec;
+
+ state = talloc_zero(tctx, struct test_smb2_bench_echo_state);
+ torture_assert(tctx, state != NULL, __location__);
+ state->tctx = tctx;
+ state->num_conns = torture_nprocs;
+ state->conns = talloc_zero_array(state,
+ struct test_smb2_bench_echo_conn,
+ state->num_conns);
+ torture_assert(tctx, state->conns != NULL, __location__);
+ state->num_loops = torture_nprocs * torture_qdepth;
+ state->loops = talloc_zero_array(state,
+ struct test_smb2_bench_echo_loop,
+ state->num_loops);
+ torture_assert(tctx, state->loops != NULL, __location__);
+ state->ok = true;
+ state->timelimit = MAX(timelimit, 1);
+
+ timeout_msec = tree->session->transport->options.request_timeout * 1000;
+
+ torture_comment(tctx, "Opening %zu connections\n", state->num_conns);
+
+ for (i=0;i<state->num_conns;i++) {
+ struct smb2_tree *ct = NULL;
+ DATA_BLOB out_input_buffer = data_blob_null;
+ DATA_BLOB out_output_buffer = data_blob_null;
+ size_t pcli;
+
+ state->conns[i].state = state;
+ state->conns[i].idx = i;
+
+ if (!torture_smb2_connection(tctx, &ct)) {
+ torture_comment(tctx, "Failed opening %zu/%zu connections\n", i, state->num_conns);
+ return false;
+ }
+ state->conns[i].tree = talloc_steal(state->conns, ct);
+
+ smb2cli_conn_set_max_credits(ct->session->transport->conn, 8192);
+ smb2cli_ioctl(ct->session->transport->conn,
+ timeout_msec,
+ ct->session->smbXcli,
+ ct->smbXcli,
+ UINT64_MAX, /* in_fid_persistent */
+ UINT64_MAX, /* in_fid_volatile */
+ UINT32_MAX,
+ 0, /* in_max_input_length */
+ NULL, /* in_input_buffer */
+ 1, /* in_max_output_length */
+ NULL, /* in_output_buffer */
+ SMB2_IOCTL_FLAG_IS_FSCTL,
+ ct,
+ &out_input_buffer,
+ &out_output_buffer);
+ torture_assert(tctx,
+ smbXcli_conn_is_connected(ct->session->transport->conn),
+ "smbXcli_conn_is_connected");
+
+ for (pcli = 0; pcli < torture_qdepth; pcli++) {
+ struct test_smb2_bench_echo_loop *loop = &state->loops[li];
+
+ loop->idx = li++;
+ loop->state = state;
+ loop->conn = &state->conns[i];
+ loop->im = tevent_create_immediate(state->loops);
+ torture_assert(tctx, loop->im != NULL, __location__);
+
+ tevent_schedule_immediate(loop->im,
+ tctx->ev,
+ test_smb2_bench_echo_loop_start,
+ loop);
+ }
+ }
--
Samba Shared Repository
More information about the samba-cvs
mailing list