[PATCH] messaging: Fix queueing on FreeBSD

[Jeremy Allison]

On Tue, Jun 20, 2017 at 04:55:48PM +0200, Volker Lendecke via samba-technical wrote:
> Hi!
> 
> Without this the local.messaging tests fail on FreeBSD, in particular
> the overflow one.
> 
> Review appreciated!

Nice catch (and fix :-). Reviewed-by: Jeremy Allison <jra at samba.org>

> -- 
> SerNet GmbH, Bahnhofsallee 1b, 37081 Göttingen
> phone: +49-551-370000-0, fax: +49-551-370000-9
> AG Göttingen, HRB 2816, GF: Dr. Johannes Loxen
> http://www.sernet.de, mailto:kontakt at sernet.de

> From 8f190bb68c96e20b569666861c35fbb291816c16 Mon Sep 17 00:00:00 2001
> From: Volker Lendecke <vl at samba.org>
> Date: Wed, 23 Nov 2016 16:51:25 +0100
> Subject: [PATCH] messaging: Fix queueing on FreeBSD
> 
> FreeBSD does not do the nice blocking send that Linux does. Instead,
> it returns ENOBUFS if the dst socket is full. According to the
> manpage you have to do polling. Try with exponential backoff, at
> the end try once a second forever.
> 
> Signed-off-by: Volker Lendecke <vl at samba.org>
> ---
>  source3/lib/messages_dgm.c | 37 ++++++++++++++++++++++++++++++++++++-
>  1 file changed, 36 insertions(+), 1 deletion(-)
> 
> diff --git a/source3/lib/messages_dgm.c b/source3/lib/messages_dgm.c
> index 095a319..f29180d 100644
> --- a/source3/lib/messages_dgm.c
> +++ b/source3/lib/messages_dgm.c
> @@ -541,9 +541,35 @@ static void messaging_dgm_out_threaded_job(void *private_data)
>  	struct iovec iov = { .iov_base = state->buf,
>  			     .iov_len = talloc_get_size(state->buf) };
>  	size_t num_fds = talloc_array_length(state->fds);
> +	int msec = 1;
>  
> -	state->sent = messaging_dgm_sendmsg(state->sock, &iov, 1,
> +	while (true) {
> +		int ret;
> +
> +		state->sent = messaging_dgm_sendmsg(state->sock, &iov, 1,
>  					    state->fds, num_fds, &state->err);
> +
> +		if (state->sent != -1) {
> +			return;
> +		}
> +		if (errno != ENOBUFS) {
> +			return;
> +		}
> +
> +		/*
> +		 * ENOBUFS is the FreeBSD way of saying "Try
> +		 * again". We have to do polling.
> +		 */
> +		do {
> +			ret = poll(NULL, 0, msec);
> +		} while ((ret == -1) && (errno == EINTR));
> +
> +		/*
> +		 * Exponential backoff up to once a second
> +		 */
> +		msec *= 2;
> +		msec = MIN(msec, 1000);
> +	}
>  }
>  
>  /*
> @@ -619,6 +645,15 @@ static int messaging_dgm_out_send_fragment(
>  			return 0;
>  		}
>  
> +		if (err == ENOBUFS) {
> +			/*
> +			 * FreeBSD's way of telling us the dst socket
> +			 * is full. EWOULDBLOCK makes us spawn a
> +			 * polling helper thread.
> +			 */
> +			err = EWOULDBLOCK;
> +		}
> +
>  		if (err != EWOULDBLOCK) {
>  			return err;
>  		}
> -- 
> 2.1.4
>