[SCM] Samba Shared Repository - branch master updated
Jeremy Allison
jra at samba.org
Wed Sep 6 20:32:01 UTC 2023
The branch, master has been updated
via 3fc35827569 smb2_server: move struct msghdr to smbd_smb2_send_queue
via 02df6bda490 smb2_server: split out smbd_smb2_flush_with_sendmsg() out of smbd_smb2_flush_send_queue()
via 0ca825c4bc2 smb2_server: split out smbd_smb2_advance_send_queue() out of smbd_smb2_flush_send_queue()
via 153323efc55 smb2_server: simplify smbd_smb2_advance_incoming() recvfile logic
via 72d86d49894 smb2_server: change smbd_smb2_advance_incoming() to use iov_advance()
via a9c53b9eff6 lib/util: inline iov_{buflen,buf,advance}()
via 76f9a41fa67 smb2_server: split smbd_smb2_advance_incoming() out of smbd_smb2_io_handler()
via 02e0ba710db smb2_server: remove state->hdr.done and always set state->vector first
via 948d19b09a7 smb2_server: move struct msghdr to smbd_smb2_request_read_state
via 16f46601b84 smb2_server: avoid ZERO_STRUCT*() in the core code
from f3c632e74ba testprogs: Add net offlinejoin composeodj tests
https://git.samba.org/?p=samba.git;a=shortlog;h=master
- Log -----------------------------------------------------------------
commit 3fc358275693a1dc44f2d03b43b68d1ee7f1ec14
Author: Stefan Metzmacher <metze at samba.org>
Date: Wed Sep 30 23:42:48 2020 +0200
smb2_server: move struct msghdr to smbd_smb2_send_queue
The main reason is the preparation of io_uring support,
as it can't be on the stack for async operations.
Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Wed Sep 6 20:31:04 UTC 2023 on atb-devel-224
commit 02df6bda4902524d9fbcc961287779b3e770be2f
Author: Stefan Metzmacher <metze at samba.org>
Date: Wed Sep 30 23:21:04 2020 +0200
smb2_server: split out smbd_smb2_flush_with_sendmsg() out of smbd_smb2_flush_send_queue()
We'll have an smbd_smb2_flush_with_io_uring() later...
Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
commit 0ca825c4bc25a7843b8200ea0126fc3320f8724a
Author: Stefan Metzmacher <metze at samba.org>
Date: Wed Sep 30 14:01:25 2020 +0200
smb2_server: split out smbd_smb2_advance_send_queue() out of smbd_smb2_flush_send_queue()
The logic in smbd_smb2_advance_send_queue() will be reused for io_uring.
Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
commit 153323efc55bef47a4694645bb2dbfb3334776a3
Author: Stefan Metzmacher <metze at samba.org>
Date: Tue Sep 14 02:22:09 2021 +0200
smb2_server: simplify smbd_smb2_advance_incoming() recvfile logic
This will make further changes easier...
Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
commit 72d86d498945f5c7bff2d1e5d75ed2fd2ba646a8
Author: Stefan Metzmacher <metze at samba.org>
Date: Thu Apr 15 19:56:59 2021 +0000
smb2_server: change smbd_smb2_advance_incoming() to use iov_advance()
In future we may use vectors with more elements, so we convert to
a single element array now...
Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
commit a9c53b9eff605d67880ae1a333b95d2f6f921d1a
Author: Stefan Metzmacher <metze at samba.org>
Date: Tue Apr 25 17:55:16 2023 +0000
lib/util: inline iov_{buflen,buf,advance}()
The main reason for this change was the use of
iov_advance() in the next commits in
source3/smbd/smb2_server.c
And the function calls to iov_advance() showed up
in profiling with callgrind.
While there iov_buf() and iov_buflen() are moved as
well, as they are also used there.
Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
commit 76f9a41fa67da9a9c761855e6a745700a1115506
Author: Stefan Metzmacher <metze at samba.org>
Date: Thu Apr 15 10:23:37 2021 +0200
smb2_server: split smbd_smb2_advance_incoming() out of smbd_smb2_io_handler()
The logic in smbd_smb2_advance_incoming() will be reused for io_uring.
Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
commit 02e0ba710db5f1819fb53161d1ddcdac8e01bb6c
Author: Stefan Metzmacher <metze at samba.org>
Date: Thu Apr 15 10:23:37 2021 +0200
smb2_server: remove state->hdr.done and always set state->vector first
Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
commit 948d19b09a75101ac6c050ad83cce401de324f2a
Author: Stefan Metzmacher <metze at samba.org>
Date: Thu Apr 15 09:53:03 2021 +0200
smb2_server: move struct msghdr to smbd_smb2_request_read_state
This makes the code a little bit faster, but the main reason
is the preparation of io_uring support, as it can't be on the
stack for async operations.
Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
commit 16f46601b84428d86dde894d1c1179f96c74ace8
Author: Stefan Metzmacher <metze at samba.org>
Date: Tue Apr 25 17:44:49 2023 +0000
smb2_server: avoid ZERO_STRUCT*() in the core code
We should avoid calling memset_s() in the core smbd processing,
we can use struct initializers instead.
This reduces the overhead...
Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
-----------------------------------------------------------------------
Summary of changes:
lib/util/iov_buf.c | 70 ---------
lib/util/iov_buf.h | 75 +++++++++-
source3/smbd/globals.h | 8 +-
source3/smbd/smb2_server.c | 352 +++++++++++++++++++++++++++------------------
4 files changed, 293 insertions(+), 212 deletions(-)
Changeset truncated at 500 lines:
diff --git a/lib/util/iov_buf.c b/lib/util/iov_buf.c
index a9224164068..0c184669721 100644
--- a/lib/util/iov_buf.c
+++ b/lib/util/iov_buf.c
@@ -22,76 +22,6 @@
#include "iov_buf.h"
#include <talloc.h>
-ssize_t iov_buflen(const struct iovec *iov, int iovcnt)
-{
- return iov_buf(iov, iovcnt, NULL, 0);
-}
-
-ssize_t iov_buf(const struct iovec *iov, int iovcnt,
- uint8_t *buf, size_t buflen)
-{
- size_t needed = 0;
- uint8_t *p = buf;
- int i;
-
- for (i=0; i<iovcnt; i++) {
- size_t thislen = iov[i].iov_len;
- size_t tmp;
-
- tmp = needed + thislen;
-
- if (tmp < needed) {
- /* wrap */
- return -1;
- }
- needed = tmp;
-
- if ((p != NULL) && needed <= buflen && thislen > 0) {
- memcpy(p, iov[i].iov_base, thislen);
- p += thislen;
- }
- }
-
- return needed;
-}
-
-bool iov_advance(struct iovec **iov, int *iovcnt, size_t n)
-{
- struct iovec *v = *iov;
- int cnt = *iovcnt;
-
- while (n > 0) {
- if (cnt == 0) {
- return false;
- }
- if (n < v->iov_len) {
- v->iov_base = (char *)v->iov_base + n;
- v->iov_len -= n;
- break;
- }
- n -= v->iov_len;
- v += 1;
- cnt -= 1;
- }
-
- /*
- * Skip 0-length iovec's
- *
- * There might be empty buffers at the end of iov. Next time we do a
- * readv/writev based on this iov would give 0 transferred bytes, also
- * known as EPIPE. So we need to be careful discarding them.
- */
-
- while ((cnt > 0) && (v->iov_len == 0)) {
- v += 1;
- cnt -= 1;
- }
-
- *iov = v;
- *iovcnt = cnt;
- return true;
-}
-
uint8_t *iov_concat(TALLOC_CTX *mem_ctx, const struct iovec *iov, int count)
{
ssize_t buflen;
diff --git a/lib/util/iov_buf.h b/lib/util/iov_buf.h
index 07e08e1650b..cb330dea07a 100644
--- a/lib/util/iov_buf.h
+++ b/lib/util/iov_buf.h
@@ -24,10 +24,79 @@
#include <talloc.h>
#include "system/filesys.h"
-ssize_t iov_buflen(const struct iovec *iov, int iovlen);
+static inline
ssize_t iov_buf(const struct iovec *iov, int iovcnt,
- uint8_t *buf, size_t buflen);
-bool iov_advance(struct iovec **iov, int *iovcnt, size_t n);
+ uint8_t *buf, size_t buflen)
+{
+ size_t needed = 0;
+ uint8_t *p = buf;
+ int i;
+
+ for (i=0; i<iovcnt; i++) {
+ size_t thislen = iov[i].iov_len;
+ size_t tmp;
+
+ tmp = needed + thislen;
+
+ if (tmp < needed) {
+ /* wrap */
+ return -1;
+ }
+ needed = tmp;
+
+ if ((p != NULL) && needed <= buflen && thislen > 0) {
+ memcpy(p, iov[i].iov_base, thislen);
+ p += thislen;
+ }
+ }
+
+ return needed;
+}
+
+static inline
+ssize_t iov_buflen(const struct iovec *iov, int iovcnt)
+{
+ return iov_buf(iov, iovcnt, NULL, 0);
+}
+
+static inline
+bool iov_advance(struct iovec **iov, int *iovcnt, size_t n)
+{
+ struct iovec *v = *iov;
+ int cnt = *iovcnt;
+
+ while (n > 0) {
+ if (cnt == 0) {
+ return false;
+ }
+ if (n < v->iov_len) {
+ v->iov_base = (char *)v->iov_base + n;
+ v->iov_len -= n;
+ break;
+ }
+ n -= v->iov_len;
+ v += 1;
+ cnt -= 1;
+ }
+
+ /*
+ * Skip 0-length iovec's
+ *
+ * There might be empty buffers at the end of iov. Next time we do a
+ * readv/writev based on this iov would give 0 transferred bytes, also
+ * known as EPIPE. So we need to be careful discarding them.
+ */
+
+ while ((cnt > 0) && (v->iov_len == 0)) {
+ v += 1;
+ cnt -= 1;
+ }
+
+ *iov = v;
+ *iovcnt = cnt;
+ return true;
+}
+
uint8_t *iov_concat(TALLOC_CTX *mem_ctx, const struct iovec *iov, int count);
#endif
diff --git a/source3/smbd/globals.h b/source3/smbd/globals.h
index 69023fcc50a..f58e3176f31 100644
--- a/source3/smbd/globals.h
+++ b/source3/smbd/globals.h
@@ -460,9 +460,11 @@ struct smbXsrv_connection {
struct smbd_smb2_request *req;
struct {
uint8_t nbt[NBT_HDR_SIZE];
- bool done;
} hdr;
- struct iovec vector;
+ struct iovec _vector[1];
+ struct iovec *vector;
+ int count;
+ struct msghdr msg;
bool doing_receivefile;
size_t min_recv_size;
size_t pktfull;
@@ -686,6 +688,8 @@ struct smbd_smb2_send_queue {
DATA_BLOB *sendfile_header;
uint32_t sendfile_body_size;
NTSTATUS *sendfile_status;
+
+ struct msghdr msg;
struct iovec *vector;
int count;
diff --git a/source3/smbd/smb2_server.c b/source3/smbd/smb2_server.c
index 5a595313cd0..55b383072e6 100644
--- a/source3/smbd/smb2_server.c
+++ b/source3/smbd/smb2_server.c
@@ -385,7 +385,7 @@ void smb2_request_set_async_internal(struct smbd_smb2_request *req,
req->async_internal = async_internal;
}
-static struct smbd_smb2_request *smbd_smb2_request_allocate(TALLOC_CTX *mem_ctx)
+static struct smbd_smb2_request *smbd_smb2_request_allocate(struct smbXsrv_connection *xconn)
{
TALLOC_CTX *mem_pool;
struct smbd_smb2_request *req;
@@ -400,18 +400,21 @@ static struct smbd_smb2_request *smbd_smb2_request_allocate(TALLOC_CTX *mem_ctx)
return NULL;
}
- req = talloc_zero(mem_pool, struct smbd_smb2_request);
+ req = talloc(mem_pool, struct smbd_smb2_request);
if (req == NULL) {
talloc_free(mem_pool);
return NULL;
}
- talloc_reparent(mem_pool, mem_ctx, req);
+ talloc_reparent(mem_pool, xconn, req);
#if 0
TALLOC_FREE(mem_pool);
#endif
-
- req->last_session_id = UINT64_MAX;
- req->last_tid = UINT32_MAX;
+ *req = (struct smbd_smb2_request) {
+ .sconn = xconn->client->sconn,
+ .xconn = xconn,
+ .last_session_id = UINT64_MAX,
+ .last_tid = UINT32_MAX,
+ };
talloc_set_destructor(req, smbd_smb2_request_destructor);
@@ -650,7 +653,6 @@ static NTSTATUS smbd_smb2_request_create(struct smbXsrv_connection *xconn,
const uint8_t *_inpdu, size_t size,
struct smbd_smb2_request **_req)
{
- struct smbd_server_connection *sconn = xconn->client->sconn;
struct smbd_smb2_request *req;
uint32_t protocol_version;
uint8_t *inpdu = NULL;
@@ -692,8 +694,6 @@ static NTSTATUS smbd_smb2_request_create(struct smbXsrv_connection *xconn,
if (req == NULL) {
return NT_STATUS_NO_MEMORY;
}
- req->sconn = sconn;
- req->xconn = xconn;
inpdu = talloc_memdup(req, _inpdu, size);
if (inpdu == NULL) {
@@ -1924,8 +1924,6 @@ static struct smbd_smb2_request *dup_smb2_req(const struct smbd_smb2_request *re
return NULL;
}
- newreq->sconn = req->sconn;
- newreq->xconn = req->xconn;
newreq->session = req->session;
newreq->do_encryption = req->do_encryption;
newreq->do_signing = req->do_signing;
@@ -4551,8 +4549,8 @@ static bool is_smb2_recvfile_write(struct smbd_smb2_request_read_state *state)
static NTSTATUS smbd_smb2_request_next_incoming(struct smbXsrv_connection *xconn)
{
- struct smbd_server_connection *sconn = xconn->client->sconn;
struct smbd_smb2_request_read_state *state = &xconn->smb2.request_read_state;
+ struct smbd_smb2_request *req = NULL;
size_t max_send_queue_len;
size_t cur_send_queue_len;
@@ -4584,14 +4582,22 @@ static NTSTATUS smbd_smb2_request_next_incoming(struct smbXsrv_connection *xconn
}
/* ask for the next request */
- ZERO_STRUCTP(state);
- state->req = smbd_smb2_request_allocate(xconn);
- if (state->req == NULL) {
+ req = smbd_smb2_request_allocate(xconn);
+ if (req == NULL) {
return NT_STATUS_NO_MEMORY;
}
- state->req->sconn = sconn;
- state->req->xconn = xconn;
- state->min_recv_size = lp_min_receive_file_size();
+ *state = (struct smbd_smb2_request_read_state) {
+ .req = req,
+ .min_recv_size = lp_min_receive_file_size(),
+ ._vector = {
+ [0] = (struct iovec) {
+ .iov_base = (void *)state->hdr.nbt,
+ .iov_len = NBT_HDR_SIZE,
+ },
+ },
+ .vector = state->_vector,
+ .count = 1,
+ };
TEVENT_FD_READABLE(xconn->transport.fde);
@@ -4726,7 +4732,40 @@ static int socket_error_from_errno(int ret,
return sys_errno;
}
-static NTSTATUS smbd_smb2_flush_send_queue(struct smbXsrv_connection *xconn)
+static NTSTATUS smbd_smb2_advance_send_queue(struct smbXsrv_connection *xconn,
+ struct smbd_smb2_send_queue **_e,
+ size_t n)
+{
+ struct smbd_smb2_send_queue *e = *_e;
+ bool ok;
+
+ xconn->ack.unacked_bytes += n;
+
+ ok = iov_advance(&e->vector, &e->count, n);
+ if (!ok) {
+ return NT_STATUS_INTERNAL_ERROR;
+ }
+
+ if (e->count > 0) {
+ return NT_STATUS_RETRY;
+ }
+
+ xconn->smb2.send_queue_len--;
+ DLIST_REMOVE(xconn->smb2.send_queue, e);
+
+ if (e->ack.req == NULL) {
+ *_e = NULL;
+ talloc_free(e->mem_ctx);
+ return NT_STATUS_OK;
+ }
+
+ e->ack.required_acked_bytes = xconn->ack.unacked_bytes;
+ DLIST_ADD_END(xconn->ack.queue, e);
+
+ return NT_STATUS_OK;
+}
+
+static NTSTATUS smbd_smb2_flush_with_sendmsg(struct smbXsrv_connection *xconn)
{
int ret;
int err;
@@ -4741,8 +4780,6 @@ static NTSTATUS smbd_smb2_flush_send_queue(struct smbXsrv_connection *xconn)
while (xconn->smb2.send_queue != NULL) {
struct smbd_smb2_send_queue *e = xconn->smb2.send_queue;
unsigned sendmsg_flags = 0;
- bool ok;
- struct msghdr msg;
if (!NT_STATUS_IS_OK(xconn->transport.status)) {
/*
@@ -4809,7 +4846,7 @@ static NTSTATUS smbd_smb2_flush_send_queue(struct smbXsrv_connection *xconn)
continue;
}
- msg = (struct msghdr) {
+ e->msg = (struct msghdr) {
.msg_iov = e->vector,
.msg_iovlen = e->count,
};
@@ -4821,7 +4858,7 @@ static NTSTATUS smbd_smb2_flush_send_queue(struct smbXsrv_connection *xconn)
sendmsg_flags |= MSG_DONTWAIT;
#endif
- ret = sendmsg(xconn->transport.sock, &msg, sendmsg_flags);
+ ret = sendmsg(xconn->transport.sock, &e->msg, sendmsg_flags);
if (ret == 0) {
/* propagate end of file */
return NT_STATUS_INTERNAL_ERROR;
@@ -4839,29 +4876,29 @@ static NTSTATUS smbd_smb2_flush_send_queue(struct smbXsrv_connection *xconn)
return status;
}
- xconn->ack.unacked_bytes += ret;
-
- ok = iov_advance(&e->vector, &e->count, ret);
- if (!ok) {
- return NT_STATUS_INTERNAL_ERROR;
- }
-
- if (e->count > 0) {
- /* we have more to write */
+ status = smbd_smb2_advance_send_queue(xconn, &e, ret);
+ if (NT_STATUS_EQUAL(status, NT_STATUS_RETRY)) {
+ /* retry later */
TEVENT_FD_WRITEABLE(xconn->transport.fde);
return NT_STATUS_OK;
}
+ if (!NT_STATUS_IS_OK(status)) {
+ smbXsrv_connection_disconnect_transport(xconn,
+ status);
+ return status;
+ }
+ }
- xconn->smb2.send_queue_len--;
- DLIST_REMOVE(xconn->smb2.send_queue, e);
+ return NT_STATUS_MORE_PROCESSING_REQUIRED;
+}
- if (e->ack.req == NULL) {
- talloc_free(e->mem_ctx);
- continue;
- }
+static NTSTATUS smbd_smb2_flush_send_queue(struct smbXsrv_connection *xconn)
+{
+ NTSTATUS status;
- e->ack.required_acked_bytes = xconn->ack.unacked_bytes;
- DLIST_ADD_END(xconn->ack.queue, e);
+ status = smbd_smb2_flush_with_sendmsg(xconn);
+ if (!NT_STATUS_EQUAL(status, NT_STATUS_MORE_PROCESSING_REQUIRED)) {
+ return status;
}
/*
@@ -4877,100 +4914,36 @@ static NTSTATUS smbd_smb2_flush_send_queue(struct smbXsrv_connection *xconn)
return NT_STATUS_OK;
}
-static NTSTATUS smbd_smb2_io_handler(struct smbXsrv_connection *xconn,
- uint16_t fde_flags)
+static NTSTATUS smbd_smb2_advance_incoming(struct smbXsrv_connection *xconn, size_t n)
{
struct smbd_server_connection *sconn = xconn->client->sconn;
struct smbd_smb2_request_read_state *state = &xconn->smb2.request_read_state;
struct smbd_smb2_request *req = NULL;
size_t min_recvfile_size = UINT32_MAX;
- unsigned recvmsg_flags = 0;
- int ret;
- int err;
- bool retry;
NTSTATUS status;
NTTIME now;
- struct msghdr msg;
-
- if (!NT_STATUS_IS_OK(xconn->transport.status)) {
- /*
- * we're not supposed to do any io
- */
- TEVENT_FD_NOT_READABLE(xconn->transport.fde);
- TEVENT_FD_NOT_WRITEABLE(xconn->transport.fde);
- return NT_STATUS_OK;
- }
-
- if (fde_flags & TEVENT_FD_WRITE) {
- status = smbd_smb2_flush_send_queue(xconn);
- if (!NT_STATUS_IS_OK(status)) {
- return status;
- }
- }
-
- if (!(fde_flags & TEVENT_FD_READ)) {
- return NT_STATUS_OK;
- }
+ bool ok;
- if (state->req == NULL) {
- TEVENT_FD_NOT_READABLE(xconn->transport.fde);
- return NT_STATUS_OK;
+ ok = iov_advance(&state->vector, &state->count, n);
+ if (!ok) {
+ return NT_STATUS_INTERNAL_ERROR;
}
-again:
- if (!state->hdr.done) {
- state->hdr.done = true;
-
- state->vector.iov_base = (void *)state->hdr.nbt;
- state->vector.iov_len = NBT_HDR_SIZE;
+ if (state->count > 0) {
+ return NT_STATUS_PENDING;
}
- msg = (struct msghdr) {
- .msg_iov = &state->vector,
- .msg_iovlen = 1,
- };
-
-#ifdef MSG_NOSIGNAL
- recvmsg_flags |= MSG_NOSIGNAL;
-#endif
-#ifdef MSG_DONTWAIT
- recvmsg_flags |= MSG_DONTWAIT;
-#endif
-
- ret = recvmsg(xconn->transport.sock, &msg, recvmsg_flags);
- if (ret == 0) {
- /* propagate end of file */
- status = NT_STATUS_END_OF_FILE;
- smbXsrv_connection_disconnect_transport(xconn,
- status);
- return status;
- }
- err = socket_error_from_errno(ret, errno, &retry);
- if (retry) {
- /* retry later */
- TEVENT_FD_READABLE(xconn->transport.fde);
- return NT_STATUS_OK;
--
Samba Shared Repository
More information about the samba-cvs
mailing list