[patch] add "--ignore" option
Wayne Davison
wayned at samba.org
Sat Feb 5 01:09:16 GMT 2005
On Tue, Feb 01, 2005 at 05:16:59PM +0000, Matthew Kirkwood wrote:
> On Mon, 31 Jan 2005, Wayne Davison wrote:
> > -f "-s *.no_send" -f "-r *.no_delete" -f "- *.no_send+no_delete"
>
> Does that give any additional useful behaviour?
Sure. Having a rule be server-side only allows you to mark just some of
the rules with a --delete-excluded behavior and make it the default when
--delete is specified.
> The syntax seems a little painful.
Perhaps. Of course, the rules would normally be used in a file with
other rules instead of using a bunch of --filter rules on the command-
line. However, it is also easy to make more intuitive rules, such as
"H pattern" to hide server files (i.e. providing no delete protection on
the receiver) and "P pattern" to protect receiver files from being
deleted. For instance:
- *.o
H hide-this-dir/
# Send new *.c files & update old ones, but don't delete.
P *.c
:r .per-dir-filter-protections-on-receiver
The reason for wanting this as a filter syntax is that filter rules are
a superset of all possible ways to pattern match files. Now that the
filter option is present, I don't really want to see other new pattern-
matching options added, so I would choose to omit the --ignore-from and
--ignore.
> Additionally, isn't it the case that, until now, the sender never
> needed to pass options to the receiver?
If by options you mean include/exclude patterns, rsync has a way to send
the excludes to the correct side of the transfer where they are needed.
(FYI, this is one area where your patch was deficient because it would
only work if the user were pulling files -- if the user pushed files and
specified --delete-excluded, no ignore patterns would have been sent to
the remote receiving side.)
It is arguable if the receiver-side exclude should be ignored by
--delete-excluded or not. On the one hand, with the ability to turn any
rule into a server-side rule, --delete-excluded behavior can be applied
to any rule without specifying the option, so perhaps --delete-excluded
should be made to ignore all filter rules, including "protect" rules. I
have chosen to err on the side of safety and made it so that the
--delete-excluded option only affects normal +/- rules.
One other comment on your implementation: I don't think that we need a
3-state return from the exclude functions. I think a better approach is
to simply remove rules that don't apply to the current side. So, to
protect files from deletion without affecting the sending of files,
rsync would simply need to remove the receiver-side rules from (or not
transfer them to) the sending side. This does mean that any "+" rules
that were added to override receiver-side "-" rules would also need to
be marked as receiver-only rules.
I've appended a patch that applies to the latest CVS source and
implements the idioms I mentioned above. It has been minimally tested.
Note that there has been some significant check-ins to improve the CVS-
exclude code and some other aspects of the filter system, so this patch
will not apply to anything older than the latest "nightly" tar file.
Let me know what you think. Feel free to suggest alternatives if you
think something should be done a better way.
..wayne..
-------------- next part --------------
Index: compat.c
--- compat.c 31 Jan 2005 19:13:19 -0000 1.25
+++ compat.c 5 Feb 2005 00:38:10 -0000
@@ -31,6 +31,7 @@ extern int verbose;
extern int am_server;
extern int am_sender;
extern int read_batch;
+extern int delete_excluded;
extern int checksum_seed;
extern int protocol_version;
@@ -74,6 +75,12 @@ void setup_protocol(int f_out,int f_in)
exit_cleanup(RERR_PROTOCOL);
}
+ /* In newer protocols, --delete-excluded does not prevent the exclude list
+ * from getting sent to the receiver, so mark a modern --delete-excluded
+ * conversation with a 2 instead of a 1. */
+ if (protocol_version >= 29 && delete_excluded)
+ delete_excluded = 2;
+
if (am_server) {
if (!checksum_seed)
checksum_seed = time(NULL);
Index: exclude.c
--- exclude.c 4 Feb 2005 22:32:17 -0000 1.103
+++ exclude.c 5 Feb 2005 00:38:13 -0000
@@ -53,7 +53,8 @@ struct filter_list_struct server_filter_
#define MAX_RULE_PREFIX (16)
#define MODIFIERS_MERGE_FILE "-+Cenw"
-#define MODIFIERS_INCL_EXCL "/!C"
+#define MODIFIERS_INCL_EXCL "/!Crs"
+#define MODIFIERS_HIDE_PROTECT "/!"
/* The dirbuf is set by push_local_filters() to the current subdirectory
* relative to curr_dir that is being processed. The path always has a
@@ -667,6 +668,14 @@ static const char *parse_rule_tok(const
case '-':
mods = MODIFIERS_INCL_EXCL;
break;
+ case 'H':
+ new_mflags |= MATCHFLG_SENDER_SIDE;
+ mods = MODIFIERS_HIDE_PROTECT;
+ break;
+ case 'P':
+ new_mflags |= MATCHFLG_RECEIVER_SIDE;
+ mods = MODIFIERS_HIDE_PROTECT;
+ break;
case '!':
new_mflags |= MATCHFLG_CLEAR_LIST;
mods = NULL;
@@ -719,6 +728,12 @@ static const char *parse_rule_tok(const
case 'n':
new_mflags |= MATCHFLG_NO_INHERIT;
break;
+ case 'r':
+ new_mflags |= MATCHFLG_RECEIVER_SIDE;
+ break;
+ case 's':
+ new_mflags |= MATCHFLG_SENDER_SIDE;
+ break;
case 'w':
new_mflags |= MATCHFLG_WORD_SPLIT;
break;
@@ -973,6 +988,11 @@ char *get_rule_prefix(int match_flags, c
}
if (match_flags & MATCHFLG_EXCLUDE_SELF)
*op++ = 'e';
+ if (match_flags & MATCHFLG_SENDER_SIDE && !sending)
+ *op++ = 's';
+ if (match_flags & MATCHFLG_RECEIVER_SIDE
+ && (!sending || delete_excluded))
+ *op++ = 'r';
if (op - buf > legal_len)
return NULL;
if (legal_len)
@@ -987,19 +1007,37 @@ char *get_rule_prefix(int match_flags, c
static void send_rules(int f_out, struct filter_list_struct *flp)
{
- struct filter_struct *ent;
+ struct filter_struct *ent, *prev = NULL;
for (ent = flp->head; ent; ent = ent->next) {
unsigned int len, plen, dlen;
+ int elide = 0;
char *p;
+ if (ent->match_flags & MATCHFLG_SENDER_SIDE)
+ elide = am_sender ? 1 : -1;
+ if (ent->match_flags & MATCHFLG_RECEIVER_SIDE)
+ elide = elide ? 0 : am_sender ? -1 : 1;
+ else if (delete_excluded)
+ elide = am_sender ? 1 : -1;
+ if (elide < 0) {
+ if (prev)
+ prev->next = ent->next;
+ else
+ flp->head = ent->next;
+ } else
+ prev = ent;
+ if (elide > 0)
+ continue;
if (ent->match_flags & MATCHFLG_CVS_IGNORE
&& !(ent->match_flags & MATCHFLG_MERGE_FILE)) {
- if (am_sender || protocol_version < 29) {
- send_rules(f_out, &cvs_filter_list);
+ int f = am_sender || protocol_version < 29 ? f_out : -1;
+ send_rules(f, &cvs_filter_list);
+ if (f >= 0)
continue;
- }
}
+ if (f_out < 0)
+ continue;
p = get_rule_prefix(ent->match_flags, ent->pattern, 1, &plen);
if (!p) {
rprintf(FERROR,
@@ -1017,12 +1055,15 @@ static void send_rules(int f_out, struct
if (dlen)
write_byte(f_out, '/');
}
+ flp->tail = prev;
}
/* This is only called by the client. */
void send_filter_list(int f_out)
{
- if (local_server || (am_sender && (!delete_mode || delete_excluded)))
+ int receiver_wants_list = delete_mode && delete_excluded != 1;
+
+ if (local_server || (am_sender && !receiver_wants_list))
f_out = -1;
if (cvs_exclude && am_sender) {
if (protocol_version >= 29)
@@ -1035,10 +1076,10 @@ void send_filter_list(int f_out)
if (list_only == 1 && !recurse)
parse_rule(&filter_list, "/*/*", MATCHFLG_NO_PREFIXES, 0);
- if (f_out >= 0) {
- send_rules(f_out, &filter_list);
+ send_rules(f_out, &filter_list);
+
+ if (f_out >= 0)
write_int(f_out, 0);
- }
if (cvs_exclude) {
if (!am_sender || protocol_version < 29)
@@ -1054,8 +1095,9 @@ void recv_filter_list(int f_in)
char line[MAXPATHLEN+MAX_RULE_PREFIX+1]; /* +1 for trailing slash. */
int xflags = protocol_version >= 29 ? 0 : XFLG_OLD_PREFIXES;
unsigned int len;
+ int receiver_wants_list = delete_mode && delete_excluded != 1;
- if (!local_server && (am_sender || (delete_mode && !delete_excluded))) {
+ if (!local_server && (am_sender || receiver_wants_list)) {
while ((len = read_int(f_in)) != 0) {
if (len >= sizeof line)
overflow("recv_rules");
@@ -1070,4 +1112,7 @@ void recv_filter_list(int f_in)
if (local_server || am_sender)
parse_rule(&filter_list, "-C", 0, 0);
}
+
+ if (local_server) /* filter out any rules that aren't for us. */
+ send_rules(-1, &filter_list);
}
Index: flist.c
--- flist.c 3 Feb 2005 19:19:39 -0000 1.263
+++ flist.c 5 Feb 2005 00:38:13 -0000
@@ -979,7 +979,7 @@ void send_file_name(int f, struct file_l
/* f is set to -1 when calculating deletion file list */
file = make_file(fname, flist,
- f == -1 && delete_excluded? SERVER_FILTERS : ALL_FILTERS);
+ f == -1 && delete_excluded == 1 ? SERVER_FILTERS : ALL_FILTERS);
if (!file)
return;
Index: rsync.h
--- rsync.h 4 Feb 2005 21:13:12 -0000 1.240
+++ rsync.h 5 Feb 2005 00:38:15 -0000
@@ -565,9 +565,12 @@ struct map_struct {
#define MATCHFLG_FINISH_SETUP (1<<13)/* per-dir merge file needs setup */
#define MATCHFLG_NEGATE (1<<14)/* rule matches when pattern does not */
#define MATCHFLG_CVS_IGNORE (1<<15)/* rule was -C or :C */
+#define MATCHFLG_SENDER_SIDE (1<<16)/* rule doesn't affect receiver */
+#define MATCHFLG_RECEIVER_SIDE (1<<17)/* rule doesn't affect sender */
#define MATCHFLGS_FROM_CONTAINER (MATCHFLG_ABS_PATH | MATCHFLG_INCLUDE \
- | MATCHFLG_DIRECTORY | MATCHFLG_NEGATE)
+ | MATCHFLG_DIRECTORY | MATCHFLG_SENDER_SIDE \
+ | MATCHFLG_NEGATE | MATCHFLG_RECEIVER_SIDE)
struct filter_struct {
struct filter_struct *next;
Index: rsync.yo
--- rsync.yo 5 Feb 2005 00:03:46 -0000 1.228
+++ rsync.yo 5 Feb 2005 00:38:16 -0000
@@ -678,7 +678,9 @@ send the whole directory (e.g. "dir" or
for the directory's contents (e.g. "dir/*") since the wildcard is expanded
by the shell and rsync thus gets a request to transfer individual files, not
the files' parent directory. Files that are excluded from transfer are
-excluded from being deleted unless you use bf(--delete-excluded).
+also excluded from being deleted unless you use the bf(--delete-excluded)
+option or mark the rules as only matching on the sending side (see the
+include/exclude modifiers in the FILTER RULES section).
This option has no effect unless directory recursion is enabled.
@@ -725,6 +727,9 @@ See bf(--delete) (which is implied) for
dit(bf(--delete-excluded)) In addition to deleting the files on the
receiving side that are not on the sending side, this tells rsync to also
delete any files on the receiving side that are excluded (see bf(--exclude)).
+See the FILTER RULES section for a way to make individual exclusions behave
+this way on the receiver, and for a way to protect files from
+bf(--delete-excluded).
See bf(--delete) (which is implied) for more details on file-deletion.
dit(bf(--ignore-errors)) Tells bf(--delete) to go ahead and delete files
@@ -1241,6 +1246,8 @@ bf(-) specifies an exclude pattern. nl()
bf(+) specifies an include pattern. nl()
bf(.) specifies a merge-file to read for more rules. nl()
bf(:) specifies a per-directory merge-file. nl()
+bf(H) specifies a pattern for hiding files from the transfer. nl()
+bf(P) specifies a pattern for protecting files from deletion. nl()
bf(!) clears the current include/exclude list (takes no arg) nl()
)
@@ -1263,10 +1270,17 @@ comment lines that start with a "#".
manpagesection(INCLUDE/EXCLUDE PATTERN RULES)
-You can include and exclude files by specifying patterns using the "+" and
-"-" filter rules (as introduced in the FILTER RULES section above). These
-rules specify a pattern that is matched against the names of the files
-that are going to be transferred. These patterns can take several forms:
+You can include and exclude files by specifying patterns using the "+",
+"-", "H", and "P" filter rules (as introduced in the FILTER RULES section
+above).
+Note that the "H" (hide) rule is just a more intuitive way to specify a "-"
+rule with an "s" modifier (a sender-only exclusion) and "P" (protect) is
+just a more intuitive way to specify a "-" rule with an "r" modifier (a
+receiver-only exclusion). See the modifiers below for more information.
+
+The include/exclude rules each specify a pattern that is matched against
+the names of the files that are going to be transferred. These patterns
+can take several forms:
itemize(
it() if the pattern starts with a / then it is anchored to a
@@ -1398,6 +1412,9 @@ itemize(
space that separates the prefix from the rule is treated specially, so
"- foo + bar" is parsed as two rules (assuming that bf(-) or bf(+) was not
specified to turn off the parsing of prefixes).
+ it() You may also specify any of the modifiers for "+" or "-" to have the
+ rules that are read-in default to having that option set. For instance,
+ ":s_.excl" would make all the rules in .excl server-side only.
)
The following modifiers are accepted after a "+" or "-":
@@ -1413,6 +1430,16 @@ itemize(
it() A bf(C) is used to indicate that all the global CVS-exclude rules
should be inserted as excludes in place of the "-C". No arg should
follow.
+ it() An bf(s) is used to indicate that the rule applies to the sending
+ side. Sender-only exclusions hide files from being transferred without
+ protecting them from being deleted. This makes the associated rules
+ behave as if --delete-excluded had been applied just for them. See also
+ the "P" rule.
+ it() An bf(r) is used to indicate that the rule applies to the receiving
+ side. Receiver-only exclusions protect files from being deleted by
+ --delete and also --delete-excluded. Note that "s" may be combined with
+ "r" if you want a rule that applies to both sides, but can't be ignored
+ by --delete-excluded. See also the "H" rule.
)
Per-directory rules are inherited in all subdirectories of the directory
@@ -1706,10 +1733,10 @@ error.
When reading a batch file, rsync will force the value of certain options
to match the data in the batch file if you didn't set them to the same
as the batch-writing command. Other options can (and should) be changed.
-For instance
-bf(--write-batch) changes to bf(--read-batch), bf(--files-from) is dropped, and the
-bf(--filter)/bf(--include)/bf(--exclude) options are not needed unless one of the
-bf(--delete) options is specified without bf(--delete-excluded).
+For instance bf(--write-batch) changes to bf(--read-batch),
+bf(--files-from) is dropped, and the
+bf(--filter)/bf(--include)/bf(--exclude) options are not needed unless
+one of the bf(--delete) options is specified.
The code that creates the BATCH.sh file transforms any filter/include/exclude
options into a single list that is appended as a "here" document to the
More information about the rsync
mailing list