[SCM] The rsync repository. - branch master updated
Rsync CVS commit messages
rsync-cvs at lists.samba.org
Wed Apr 13 06:43:20 UTC 2022
The branch, master has been updated
via 85c56b26 The latest news.
via 10aeb75c Add debugging comment about read_buf_().
via d41bb98c systemd: restart daemon on-failure (#302)
via 2fda5169 Specify log format to avoid malfunctions and unexpected errors. (#305)
via 1de71e8a Fix for CVE-2018-25032 in zlib (#306)
via 60dd42be Handle linking with a zlib with external read_buf.
from d821e4cb Preparing for release of 3.2.4pre4
https://git.samba.org/?p=rsync.git;a=shortlog;h=master
- Log -----------------------------------------------------------------
commit 85c56b2603d97c225889175797ffff6745a4d305
Author: Wayne Davison <wayne at opencoder.net>
Date: Mon Apr 11 09:46:38 2022 -0700
The latest news.
commit 10aeb75cea7be5cd01145919c38f85c199b8c32a
Author: Wayne Davison <wayne at opencoder.net>
Date: Mon Apr 11 09:46:32 2022 -0700
Add debugging comment about read_buf_().
commit d41bb98c09bf0b999c4eee4e2125c7e5d0747ec4
Author: Simon Deziel <simon.deziel at gmail.com>
Date: Mon Apr 11 12:08:11 2022 -0400
systemd: restart daemon on-failure (#302)
man 5 systemd.service:
> Setting this to on-failure is the recommended choice for long-running services
Partial fix for https://bugzilla.samba.org/show_bug.cgi?id=13463
Signed-off-by: Simon Deziel <simon at sdeziel.info>
commit 2fda51692be7a6268e97237f7cf43f7ac16023f7
Author: Yoichi NAKAYAMA <yoichi.nakayama at gmail.com>
Date: Tue Apr 12 00:57:19 2022 +0900
Specify log format to avoid malfunctions and unexpected errors. (#305)
Solve the following problems:
* mishandling of commit message lines similar to committer lines
* UnicodeDecodeError with commit messages that cannot be interpreted
as utf-8
commit 1de71e8a7870fa1be29f562df61f4c5c4685818f
Author: Michal Ruprich <michalruprich at gmail.com>
Date: Mon Apr 11 17:50:50 2022 +0200
Fix for CVE-2018-25032 in zlib (#306)
commit 60dd42be603a79cd57cec076fe1680e9037be774
Author: Wayne Davison <wayne at opencoder.net>
Date: Mon Apr 11 08:29:54 2022 -0700
Handle linking with a zlib with external read_buf.
-----------------------------------------------------------------------
Summary of changes:
NEWS.md | 24 ++++++++++---
io.c | 1 +
packaging/systemd/rsync.service | 1 +
rsync.h | 4 +++
support/git-set-file-times | 4 +--
zlib/deflate.c | 74 ++++++++++++++++++++++++++++++-----------
zlib/deflate.h | 25 ++++++--------
zlib/trees.c | 50 ++++++++--------------------
8 files changed, 106 insertions(+), 77 deletions(-)
Changeset truncated at 500 lines:
diff --git a/NEWS.md b/NEWS.md
index e66b3895..72a62780 100644
--- a/NEWS.md
+++ b/NEWS.md
@@ -20,13 +20,13 @@
If your rsync script depends on the old arg-splitting behavior, either run
it with the [`--old-args`](rsync.1#opt) option or `export RSYNC_OLD_ARGS=1`
in the script's environment. See also the [ADVANCED USAGE](rsync.1#)
- section of rsync's manpage.
+ section of rsync's manpage for how to use a more modern arg style.
- A long-standing bug was preventing rsync from figuring out the current
locale's decimal point character, which made rsync always output numbers
using the "C" locale. Since this is now fixed in 3.2.4, a script that
- parses rsync's decimal numbers (e.g. from the verbose footer) should be sure
- to setup the environment in a way that the output continues to be in the C
+ parses rsync's decimal numbers (e.g. from the verbose footer) may want to
+ setup the environment in a way that the output continues to be in the C
locale. For instance, one of the following should work fine:
```shell
@@ -44,6 +44,10 @@
export LC_NUMERIC=C.UTF-8
```
+### SECURITY FIXES:
+
+ - A fix for CVE-2018-25032 in the bundled zlib (memory corruption issue).
+
### BUG FIXES:
- Fixed a bug with [`--inplace`](rsync.1#opt) + [`--sparse`](rsync.1#opt) (and
@@ -103,6 +107,12 @@
- Silence some chmod warnings about symlinks when it looks like we have a
function to set their permissions but they can't really be set.
+ - Fixed a potential issue in git-set-file-times when handling commits with
+ high-bit characters in the description & when handling a description that
+ might mimick the git raw-commit deliniators. (See the support dir.)
+
+ - The bundled systemd/rsync.service file now includes `Restart=on-failure`.
+
### ENHANCEMENTS:
- Use openssl's `-verify_hostname` option in the rsync-ssl script.
@@ -139,7 +149,7 @@
that is being refused due to the Linux fs.protected_regular sysctl setting.
- When [`--chown`](rsync.1#opt), [`--usermap`](rsync.1#opt), or
- [`--groupmap`](rsync.1#opt), is specified, rsync now makes sure that the
+ [`--groupmap`](rsync.1#opt) is specified, rsync now makes sure that the
appropriate [`--owner`](rsync.1#opt) and/or [`--group`](rsync.1#opt) options
are enabled.
@@ -212,7 +222,8 @@
- Renamed configure's `--enable-asm` option to `--enable-md5-asm` to avoid
confusion with the asm option for the rolling checksum. It is also honored
even when openssl crypto is in use. This allows: normal MD4 & MD5, normal
- MD4 + asm MD5, openssl MD4 & MD5, or openssl MD4 + asm MD5.
+ MD4 + asm MD5, openssl MD4 & MD5, or openssl MD4 + asm MD5 depending on the
+ configure options selected.
- Made SIMD & asm configure checks default to "no" on non-Linux hosts due to
various reports of problems on NetBSD & macOS hosts. These were also
@@ -227,6 +238,9 @@
get warned if a static initialization overflows in the future (among other
things).
+ - When linking with an external zlib, rsync renames its `read_buf()` function
+ to `read_buf_()` to avoid a symbol clash on an unpatched zlib.
+
- Added a SECURITY.md file.
### DEVELOPER RELATED:
diff --git a/io.c b/io.c
index 8ee48f93..cf94cee7 100644
--- a/io.c
+++ b/io.c
@@ -1843,6 +1843,7 @@ int64 read_longint(int f)
#endif
}
+/* Debugging note: this will be named read_buf_() when using an external zlib. */
void read_buf(int f, char *buf, size_t len)
{
if (f != iobuf.in_fd) {
diff --git a/packaging/systemd/rsync.service b/packaging/systemd/rsync.service
index 8a0b5820..8a867ca6 100644
--- a/packaging/systemd/rsync.service
+++ b/packaging/systemd/rsync.service
@@ -7,6 +7,7 @@ Documentation=man:rsync(1) man:rsyncd.conf(5)
[Service]
ExecStart=/usr/bin/rsync --daemon --no-detach
RestartSec=1
+Restart=on-failure
# Citing README.md:
#
diff --git a/rsync.h b/rsync.h
index 4b30570b..e5aacd25 100644
--- a/rsync.h
+++ b/rsync.h
@@ -1172,6 +1172,10 @@ struct name_num_obj {
struct name_num_item list[10]; /* we'll get a compile error/warning if this is ever too small */
};
+#ifdef EXTERNAL_ZLIB
+#define read_buf read_buf_
+#endif
+
#ifndef __cplusplus
#include "proto.h"
#endif
diff --git a/support/git-set-file-times b/support/git-set-file-times
index 24b3fde5..e06f0737 100755
--- a/support/git-set-file-times
+++ b/support/git-set-file-times
@@ -38,7 +38,7 @@ def main():
print_line(fn, mtime, mtime)
ls.discard(fn)
- cmd = git + 'log -r --name-only --no-color --pretty=raw --no-renames -z'.split()
+ cmd = git + 'log -r --name-only --format=%x00commit%x20%H%n%x00commit_time%x20%ct%n --no-renames -z'.split()
if args.tree:
cmd.append(args.tree)
cmd += ['--'] + args.files
@@ -46,7 +46,7 @@ def main():
proc = subprocess.Popen(cmd, stdout=subprocess.PIPE, encoding='utf-8')
for line in proc.stdout:
line = line.strip()
- m = re.match(r'^committer .*? (\d+) [-+]\d+$', line)
+ m = re.match(r'^\0commit_time (\d+)$', line)
if m:
commit_time = int(m[1])
elif NULL_COMMIT_RE.search(line):
diff --git a/zlib/deflate.c b/zlib/deflate.c
index 529c5e8d..2cbc4fc2 100644
--- a/zlib/deflate.c
+++ b/zlib/deflate.c
@@ -227,11 +227,6 @@ int ZEXPORT deflateInit2_(strm, level, method, windowBits, memLevel, strategy,
int wrap = 1;
static const char my_version[] = ZLIB_VERSION;
- ushf *overlay;
- /* We overlay pending_buf and d_buf+l_buf. This works since the average
- * output size for (length,distance) codes is <= 24 bits.
- */
-
if (version == Z_NULL || version[0] != my_version[0] ||
stream_size != sizeof(z_stream)) {
return Z_VERSION_ERROR;
@@ -300,9 +295,47 @@ int ZEXPORT deflateInit2_(strm, level, method, windowBits, memLevel, strategy,
s->lit_bufsize = 1 << (memLevel + 6); /* 16K elements by default */
- overlay = (ushf *) ZALLOC(strm, s->lit_bufsize, sizeof(ush)+2);
- s->pending_buf = (uchf *) overlay;
- s->pending_buf_size = (ulg)s->lit_bufsize * (sizeof(ush)+2L);
+ /* We overlay pending_buf and sym_buf. This works since the average size
+ * for length/distance pairs over any compressed block is assured to be 31
+ * bits or less.
+ *
+ * Analysis: The longest fixed codes are a length code of 8 bits plus 5
+ * extra bits, for lengths 131 to 257. The longest fixed distance codes are
+ * 5 bits plus 13 extra bits, for distances 16385 to 32768. The longest
+ * possible fixed-codes length/distance pair is then 31 bits total.
+ *
+ * sym_buf starts one-fourth of the way into pending_buf. So there are
+ * three bytes in sym_buf for every four bytes in pending_buf. Each symbol
+ * in sym_buf is three bytes -- two for the distance and one for the
+ * literal/length. As each symbol is consumed, the pointer to the next
+ * sym_buf value to read moves forward three bytes. From that symbol, up to
+ * 31 bits are written to pending_buf. The closest the written pending_buf
+ * bits gets to the next sym_buf symbol to read is just before the last
+ * code is written. At that time, 31*(n-2) bits have been written, just
+ * after 24*(n-2) bits have been consumed from sym_buf. sym_buf starts at
+ * 8*n bits into pending_buf. (Note that the symbol buffer fills when n-1
+ * symbols are written.) The closest the writing gets to what is unread is
+ * then n+14 bits. Here n is lit_bufsize, which is 16384 by default, and
+ * can range from 128 to 32768.
+ *
+ * Therefore, at a minimum, there are 142 bits of space between what is
+ * written and what is read in the overlain buffers, so the symbols cannot
+ * be overwritten by the compressed data. That space is actually 139 bits,
+ * due to the three-bit fixed-code block header.
+ *
+ * That covers the case where either Z_FIXED is specified, forcing fixed
+ * codes, or when the use of fixed codes is chosen, because that choice
+ * results in a smaller compressed block than dynamic codes. That latter
+ * condition then assures that the above analysis also covers all dynamic
+ * blocks. A dynamic-code block will only be chosen to be emitted if it has
+ * fewer bits than a fixed-code block would for the same set of symbols.
+ * Therefore its average symbol length is assured to be less than 31. So
+ * the compressed data for a dynamic block also cannot overwrite the
+ * symbols from which it is being constructed.
+ */
+
+ s->pending_buf = (uchf *) ZALLOC(strm, s->lit_bufsize, 4);
+ s->pending_buf_size = (ulg)s->lit_bufsize * 4;
if (s->window == Z_NULL || s->prev == Z_NULL || s->head == Z_NULL ||
s->pending_buf == Z_NULL) {
@@ -311,8 +344,12 @@ int ZEXPORT deflateInit2_(strm, level, method, windowBits, memLevel, strategy,
deflateEnd (strm);
return Z_MEM_ERROR;
}
- s->d_buf = overlay + s->lit_bufsize/sizeof(ush);
- s->l_buf = s->pending_buf + (1+sizeof(ush))*s->lit_bufsize;
+ s->sym_buf = s->pending_buf + s->lit_bufsize;
+ s->sym_end = (s->lit_bufsize - 1) * 3;
+ /* We avoid equality with lit_bufsize*3 because of wraparound at 64K
+ * on 16 bit machines and because stored blocks are restricted to
+ * 64K-1 bytes.
+ */
s->level = level;
s->strategy = strategy;
@@ -473,7 +510,7 @@ int ZEXPORT deflatePrime (strm, bits, value)
if (strm == Z_NULL || strm->state == Z_NULL) return Z_STREAM_ERROR;
s = strm->state;
- if ((Bytef *)(s->d_buf) < s->pending_out + ((Buf_size + 7) >> 3))
+ if (s->sym_buf < s->pending_out + ((Buf_size + 7) >> 3))
return Z_BUF_ERROR;
do {
put = Buf_size - s->bi_valid;
@@ -1022,7 +1059,6 @@ int ZEXPORT deflateCopy (dest, source)
#else
deflate_state *ds;
deflate_state *ss;
- ushf *overlay;
if (source == Z_NULL || dest == Z_NULL || source->state == Z_NULL) {
@@ -1042,8 +1078,7 @@ int ZEXPORT deflateCopy (dest, source)
ds->window = (Bytef *) ZALLOC(dest, ds->w_size, 2*sizeof(Byte));
ds->prev = (Posf *) ZALLOC(dest, ds->w_size, sizeof(Pos));
ds->head = (Posf *) ZALLOC(dest, ds->hash_size, sizeof(Pos));
- overlay = (ushf *) ZALLOC(dest, ds->lit_bufsize, sizeof(ush)+2);
- ds->pending_buf = (uchf *) overlay;
+ ds->pending_buf = (uchf *) ZALLOC(dest, ds->lit_bufsize, 4);
if (ds->window == Z_NULL || ds->prev == Z_NULL || ds->head == Z_NULL ||
ds->pending_buf == Z_NULL) {
@@ -1057,8 +1092,7 @@ int ZEXPORT deflateCopy (dest, source)
zmemcpy(ds->pending_buf, ss->pending_buf, (uInt)ds->pending_buf_size);
ds->pending_out = ds->pending_buf + (ss->pending_out - ss->pending_buf);
- ds->d_buf = overlay + ds->lit_bufsize/sizeof(ush);
- ds->l_buf = ds->pending_buf + (1+sizeof(ush))*ds->lit_bufsize;
+ ds->sym_buf = ds->pending_buf + ds->lit_bufsize;
ds->l_desc.dyn_tree = ds->dyn_ltree;
ds->d_desc.dyn_tree = ds->dyn_dtree;
@@ -1737,7 +1771,7 @@ local block_state deflate_fast(s, flush)
FLUSH_BLOCK(s, 1);
return finish_done;
}
- if (s->last_lit)
+ if (s->sym_next)
FLUSH_BLOCK(s, 0);
return block_done;
}
@@ -1878,7 +1912,7 @@ local block_state deflate_slow(s, flush)
FLUSH_BLOCK(s, 1);
return finish_done;
}
- if (s->last_lit)
+ if (s->sym_next)
FLUSH_BLOCK(s, 0);
return block_done;
}
@@ -1953,7 +1987,7 @@ local block_state deflate_rle(s, flush)
FLUSH_BLOCK(s, 1);
return finish_done;
}
- if (s->last_lit)
+ if (s->sym_next)
FLUSH_BLOCK(s, 0);
return block_done;
}
@@ -1992,7 +2026,7 @@ local block_state deflate_huff(s, flush)
FLUSH_BLOCK(s, 1);
return finish_done;
}
- if (s->last_lit)
+ if (s->sym_next)
FLUSH_BLOCK(s, 0);
return block_done;
}
diff --git a/zlib/deflate.h b/zlib/deflate.h
index ce0299ed..a300a28d 100644
--- a/zlib/deflate.h
+++ b/zlib/deflate.h
@@ -214,7 +214,7 @@ typedef struct internal_state {
/* Depth of each subtree used as tie breaker for trees of equal frequency
*/
- uchf *l_buf; /* buffer for literals or lengths */
+ uchf *sym_buf; /* buffer for distances and literals/lengths */
uInt lit_bufsize;
/* Size of match buffer for literals/lengths. There are 4 reasons for
@@ -236,13 +236,8 @@ typedef struct internal_state {
* - I can't count above 4
*/
- uInt last_lit; /* running index in l_buf */
-
- ushf *d_buf;
- /* Buffer for distances. To simplify the code, d_buf and l_buf have
- * the same number of elements. To use different lengths, an extra flag
- * array would be necessary.
- */
+ uInt sym_next; /* running index in sym_buf */
+ uInt sym_end; /* symbol table full when sym_next reaches this */
ulg opt_len; /* bit length of current block with optimal trees */
ulg static_len; /* bit length of current block with static trees */
@@ -322,20 +317,22 @@ void ZLIB_INTERNAL _tr_stored_block OF((deflate_state *s, charf *buf,
# define _tr_tally_lit(s, c, flush) \
{ uch cc = (c); \
- s->d_buf[s->last_lit] = 0; \
- s->l_buf[s->last_lit++] = cc; \
+ s->sym_buf[s->sym_next++] = 0; \
+ s->sym_buf[s->sym_next++] = 0; \
+ s->sym_buf[s->sym_next++] = cc; \
s->dyn_ltree[cc].Freq++; \
- flush = (s->last_lit == s->lit_bufsize-1); \
+ flush = (s->sym_next == s->sym_end); \
}
# define _tr_tally_dist(s, distance, length, flush) \
{ uch len = (length); \
ush dist = (distance); \
- s->d_buf[s->last_lit] = dist; \
- s->l_buf[s->last_lit++] = len; \
+ s->sym_buf[s->sym_next++] = dist; \
+ s->sym_buf[s->sym_next++] = dist >> 8; \
+ s->sym_buf[s->sym_next++] = len; \
dist--; \
s->dyn_ltree[_length_code[len]+LITERALS+1].Freq++; \
s->dyn_dtree[d_code(dist)].Freq++; \
- flush = (s->last_lit == s->lit_bufsize-1); \
+ flush = (s->sym_next == s->sym_end); \
}
#else
# define _tr_tally_lit(s, c, flush) flush = _tr_tally(s, 0, c)
diff --git a/zlib/trees.c b/zlib/trees.c
index 1fd7759e..9c667702 100644
--- a/zlib/trees.c
+++ b/zlib/trees.c
@@ -418,7 +418,7 @@ local void init_block(s)
s->dyn_ltree[END_BLOCK].Freq = 1;
s->opt_len = s->static_len = 0L;
- s->last_lit = s->matches = 0;
+ s->sym_next = s->matches = 0;
}
#define SMALLEST 1
@@ -943,7 +943,7 @@ void ZLIB_INTERNAL _tr_flush_block(s, buf, stored_len, last)
Tracev((stderr, "\nopt %lu(%lu) stat %lu(%lu) stored %lu lit %u ",
opt_lenb, s->opt_len, static_lenb, s->static_len, stored_len,
- s->last_lit));
+ s->sym_next / 3));
if (static_lenb <= opt_lenb) opt_lenb = static_lenb;
@@ -1012,8 +1012,9 @@ int ZLIB_INTERNAL _tr_tally (s, dist, lc)
unsigned dist; /* distance of matched string */
unsigned lc; /* match length-MIN_MATCH or unmatched char (if dist==0) */
{
- s->d_buf[s->last_lit] = (ush)dist;
- s->l_buf[s->last_lit++] = (uch)lc;
+ s->sym_buf[s->sym_next++] = dist;
+ s->sym_buf[s->sym_next++] = dist >> 8;
+ s->sym_buf[s->sym_next++] = lc;
if (dist == 0) {
/* lc is the unmatched char */
s->dyn_ltree[lc].Freq++;
@@ -1028,30 +1029,7 @@ int ZLIB_INTERNAL _tr_tally (s, dist, lc)
s->dyn_ltree[_length_code[lc]+LITERALS+1].Freq++;
s->dyn_dtree[d_code(dist)].Freq++;
}
-
-#ifdef TRUNCATE_BLOCK
- /* Try to guess if it is profitable to stop the current block here */
- if ((s->last_lit & 0x1fff) == 0 && s->level > 2) {
- /* Compute an upper bound for the compressed length */
- ulg out_length = (ulg)s->last_lit*8L;
- ulg in_length = (ulg)((long)s->strstart - s->block_start);
- int dcode;
- for (dcode = 0; dcode < D_CODES; dcode++) {
- out_length += (ulg)s->dyn_dtree[dcode].Freq *
- (5L+extra_dbits[dcode]);
- }
- out_length >>= 3;
- Tracev((stderr,"\nlast_lit %u, in %ld, out ~%ld(%ld%%) ",
- s->last_lit, in_length, out_length,
- 100L - out_length*100L/in_length));
- if (s->matches < s->last_lit/2 && out_length < in_length/2) return 1;
- }
-#endif
- return (s->last_lit == s->lit_bufsize-1);
- /* We avoid equality with lit_bufsize because of wraparound at 64K
- * on 16 bit machines and because stored blocks are restricted to
- * 64K-1 bytes.
- */
+ return (s->sym_next == s->sym_end);
}
/* ===========================================================================
@@ -1064,13 +1042,14 @@ local void compress_block(s, ltree, dtree)
{
unsigned dist; /* distance of matched string */
int lc; /* match length or unmatched char (if dist == 0) */
- unsigned lx = 0; /* running index in l_buf */
+ unsigned sx = 0; /* running index in sym_buf */
unsigned code; /* the code to send */
int extra; /* number of extra bits to send */
- if (s->last_lit != 0) do {
- dist = s->d_buf[lx];
- lc = s->l_buf[lx++];
+ if (s->sym_next != 0) do {
+ dist = s->sym_buf[sx++] & 0xff;
+ dist += (unsigned)(s->sym_buf[sx++] & 0xff) << 8;
+ lc = s->sym_buf[sx++];
if (dist == 0) {
send_code(s, lc, ltree); /* send a literal byte */
Tracecv(isgraph(lc), (stderr," '%c' ", lc));
@@ -1095,11 +1074,10 @@ local void compress_block(s, ltree, dtree)
}
} /* literal or match pair ? */
- /* Check that the overlay between pending_buf and d_buf+l_buf is ok: */
- Assert((uInt)(s->pending) < s->lit_bufsize + 2*lx,
- "pendingBuf overflow");
+ /* Check that the overlay between pending_buf and sym_buf is ok: */
+ Assert(s->pending < s->lit_bufsize + sx, "pendingBuf overflow");
- } while (lx < s->last_lit);
+ } while (sx < s->sym_next);
send_code(s, END_BLOCK, ltree);
}
--
The rsync repository.
More information about the rsync-cvs
mailing list