[SCM] Samba Shared Repository - branch master updated
Jeremy Allison
jra at samba.org
Thu Aug 8 21:44:02 UTC 2019
The branch, master has been updated
via 64f182412a5 s3:mdssvc: fix slrpc_fetch_attributes() when CNID is not known
via b2bf13ecf7b s3:mdssvc: close mdssvc rpc command must return in handle
via a5e705504bf s3:mdssvc: failing the RPC request if the mdssvc policy handle is not found
via 63366996879 s3:mdssvc: the open command must work on shares with Spotlight disabled
via 520175e2275 s3:mdssvc: use an early return if spotlight is not enabled.
via 68a9d1205e5 s3:mdssvc: update a few debug macros to modern style
via 940c3b31dd1 s3:mdssvv: don't fail the RPC request if the share name is unknown
via fef3a4c75f9 s3:mdssvc: initialize the returned share_path with the empty string
via c4b2dc95d5d s3:mdssvc: macOS returns the client values in the open command
via a3cb4fca864 s3:mdssvc: macOS returns UINT64_MAX, not 0 for closeQueryForContext mdscmd
via 5750f38ca78 s3:mdssvc: fix error handling of mdssvc RPC requests
via 017af5d5831 torture: beginning of a mdssvc RPC service test-suite
via b45a09f0e1a rpcclient: add some simple commands for Spotlight RPC
via 2cb978b6498 rpcclient: domain_sid is only needed for SAMR
via 4162ba78f81 lib:torture: add torture_assert_u32_[not_]equal[_goto] macros
via b34fd5b9978 s3:mdssvc: fix unmarshalling of empty CNID array
via 29ef0306241 s3:mdssvc: marshalling: fix unpacking empty CNID structure
via d67c98184a1 s3:mdssvc: marshalling: fix unpacking empty filemeta structure
via ed37f2d3c47 s3:mdssvc: marshalling: add a newline to a DEBUG message
via c282d76d556 torture: start of a mdssvc packet (un)marshalling testsuite
via 9d6794df728 librpc: add RPC_NDR_MDSSVC
via 12ef7b38437 s3:wscript: enable Spotlight by default
via 25c5012c53f s3:mdssvc: avoid strncpy when marshalling strings
via 9e706ee5737 unittest: workaround dependency problem in test_lib_util_modules
via 0b1d9d40c24 s3:mdssvc: supposed status field is in fact a fragment indicator
via c742ab7a4c7 s3:mdssvc: add noindex backend
via e5a4114bb56 s3:mdssvc: log inode number in ino_path_map_destr_cb()
via 2d98fb57e05 s3:mdssvc: move some code to a subsystem
via f58026b30b2 s3:mdssvc: mds_dalloc_dump() -> dalloc_dump()
via b5731a94bbc s3:mdssvc: add Unicode normalisation
via 1b897df4f74 s3:mdssvc: new option "spotlight backend"
via 66b8d86d715 s3:mdssvc: pass down sharename and SNUM
via 1ef96e09071 s3-mdssvc: factor out Tracker backend logic
via f80f8c5f99f s3:mdssvc: move debug macro to header file
via 17f29fa8677 s3:mdssvc: README.Coding fixes, long lines, modernize debug macros
via 9e0b6f667f0 s3:rpc_server: factor out rpc_mdssvc_sources
from 3bf915846f7 tfork: Free memory in torture test to satisfy sanitizer
https://git.samba.org/?p=samba.git;a=shortlog;h=master
- Log -----------------------------------------------------------------
commit 64f182412a5b7bdb1f8eccffaee296e7c8554a95
Author: Ralph Boehme <slow at samba.org>
Date: Mon May 6 14:19:31 2019 +0200
s3:mdssvc: fix slrpc_fetch_attributes() when CNID is not known
Samba currenlty fails the whole RPC request, macOS returns returns a nil entry
for the requested CNID:
DALLOC_CTX(#1): {
sl_array_t(#3): {
uint64_t: 0x0000
CNIDs: unkn1: 0xfec, unkn2: 0x6b000020
DALLOC_CTX(#1): {
uint64_t: 0xe4bbf314c03b1e
}
sl_filemeta_t(#1): {
sl_array_t(#2): {
nil
nil
}
}
}
}
Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Thu Aug 8 21:43:14 UTC 2019 on sn-devel-184
commit b2bf13ecf7b5406665f27487cb892e4f15fb28c4
Author: Ralph Boehme <slow at samba.org>
Date: Mon May 6 14:18:27 2019 +0200
s3:mdssvc: close mdssvc rpc command must return in handle
Checked against macOS mdssvc.
Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
commit a5e705504bf5a9cade2e57a18b068f654fa27a32
Author: Ralph Boehme <slow at samba.org>
Date: Mon May 6 14:14:26 2019 +0200
s3:mdssvc: failing the RPC request if the mdssvc policy handle is not found
Turns out macOS mdssvc doesn't fail the RPC request if the policy handle is all
zero. Also, if it fails with a non-all-zero handle, it returns a different RPC
error, namely DCERPC_NCA_S_PROTO_ERROR, not DCERPC_FAULT_CONTEXT_MISMATCH (or
rather their mapped NT_STATUS codes).
Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
commit 63366996879fab5b1729de6d6224ca167bdb05b0
Author: Ralph Boehme <slow at samba.org>
Date: Mon May 6 14:11:31 2019 +0200
s3:mdssvc: the open command must work on shares with Spotlight disabled
Move the implementation of this setting down to the actual search query
processing. macOS has no notion of "spotlight = false" at the DCERPC layer and
the open request will always succeed even on all shares.
When later the client issues search requests on such shares, we ensure we use
the noindex backend.
Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
commit 520175e2275abfabfdb5a62e192c4936a6c74d35
Author: Ralph Boehme <slow at samba.org>
Date: Tue Apr 16 19:32:52 2019 +0200
s3:mdssvc: use an early return if spotlight is not enabled.
No change in behaviour, best viewed with git show -w.
Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
commit 68a9d1205e54d2f447baad5481ff20dfb184dfe6
Author: Ralph Boehme <slow at samba.org>
Date: Wed Apr 17 07:38:53 2019 +0200
s3:mdssvc: update a few debug macros to modern style
Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
commit 940c3b31dd1fd68c9d7c9f4e9989d725dc77af4a
Author: Ralph Boehme <slow at samba.org>
Date: Mon May 6 14:07:53 2019 +0200
s3:mdssvv: don't fail the RPC request if the share name is unknown
Taken from macOS. We have to return an empty share_path and an empty policy
handle, but not fail the RPC request.
Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
commit fef3a4c75f9cad1a81201257f0208b241a93f6c9
Author: Ralph Boehme <slow at samba.org>
Date: Tue Apr 16 19:25:09 2019 +0200
s3:mdssvc: initialize the returned share_path with the empty string
macOS returns the empty path for an unknown share. This paves the way for that
change. Currently we still fail the RPC request if the share is not known with
DCERPC_FAULT_CANT_PERFORM, but this is wrong and is going to be changed in the
next commit.
Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
commit c4b2dc95d5dfb4e875ea58ff6568b89dacf8b916
Author: Ralph Boehme <slow at samba.org>
Date: Tue Apr 16 19:23:25 2019 +0200
s3:mdssvc: macOS returns the client values in the open command
Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
commit a3cb4fca864b6e872f453d60b5e58591da3624b1
Author: Ralph Boehme <slow at samba.org>
Date: Tue Apr 16 19:14:39 2019 +0200
s3:mdssvc: macOS returns UINT64_MAX, not 0 for closeQueryForContext mdscmd
Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
commit 5750f38ca7834254268a93193d72184d03990c1b
Author: Ralph Boehme <slow at samba.org>
Date: Tue Apr 16 14:17:11 2019 +0200
s3:mdssvc: fix error handling of mdssvc RPC requests
It seems for certain error cases macOS just sends an empty response
blob. So if our mdssvc request processing fails, we should just return an empty
response blob, but not fail the mdssvc request at the DCERPC layer.
Example, passing "xxx" as sharename which does not exist at the server:
$ bin/rpcclient -U slow%pass macmini -c "fetch_attributes xxx /foo/bar 123" -d 10
....
Got pdu len 56, data_len 32
rpc_api_pipe: got frag len of 56 at offset 0: NT_STATUS_OK
rpc_api_pipe: host macmini returned 32 bytes.
mdssvc_cmd: struct mdssvc_cmd
out: struct mdssvc_cmd
fragment : *
fragment : 0x00000000 (0)
response_blob : *
response_blob: struct mdssvc_blob
length : 0x00000000 (0)
size : 0x00010000 (65536)
spotlight_blob : *
spotlight_blob: ARRAY(0)
unkn9 : *
unkn9 : 0x00000000 (0)
...
Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
commit 017af5d583175ff9f15c64fa7cc631eb4512367b
Author: Ralph Boehme <slow at samba.org>
Date: Tue Apr 16 16:54:21 2019 +0200
torture: beginning of a mdssvc RPC service test-suite
Yikes! Most tests fail atm.
Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
commit b45a09f0e1ada1b5fa4a16fb652871f9ae973e2e
Author: Ralph Boehme <slow at samba.org>
Date: Sun Mar 8 07:57:57 2015 +0100
rpcclient: add some simple commands for Spotlight RPC
Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
commit 2cb978b6498c42947377a50fa8a5748f596d9114
Author: Ralph Boehme <slow at samba.org>
Date: Fri Nov 16 16:07:13 2018 +0100
rpcclient: domain_sid is only needed for SAMR
This allows using rpcclient commands with servers that don't implement LSA
QueryInfoPolicy.
Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
commit 4162ba78f8146ad9b39d0749a36fab674197c78e
Author: Ralph Boehme <slow at samba.org>
Date: Tue Apr 16 16:46:43 2019 +0200
lib:torture: add torture_assert_u32_[not_]equal[_goto] macros
Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
commit b34fd5b9978904ef0b316ba11564691fdac5316c
Author: Ralph Boehme <slow at samba.org>
Date: Wed Aug 7 14:02:12 2019 +0200
s3:mdssvc: fix unmarshalling of empty CNID array
len=0 is invalid, len=8 is an empty array, len>8 is an array with members, so
for the len=8 case we must add the empty cnid array.
Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
commit 29ef0306241da38356ffa1ee927e1b343564426e
Author: Ralph Boehme <slow at samba.org>
Date: Fri May 3 22:52:33 2019 +0200
s3:mdssvc: marshalling: fix unpacking empty CNID structure
Pass the correct tag member tag.size to sl_unpack_CNID(), not
tag.length. tag.size is the size actually used in a buffer of size tag.length.
Cf other users of tag.size that already do this correctly, this was only wrong
in this place.
Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
commit d67c98184a176805c0406ea4d6d56bcde78bb26d
Author: Ralph Boehme <slow at samba.org>
Date: Fri May 3 22:52:56 2019 +0200
s3:mdssvc: marshalling: fix unpacking empty filemeta structure
This is how a correct dump of an empty sl_filemeta_t container should look like:
DALLOC_CTX(#1): {
sl_array_t(#3): {
uint64_t: 0x0023
CNIDs: unkn1: 0x0, unkn2: 0x0
DALLOC_CTX(#0): {
}
sl_filemeta_t(#0): {
}
}
}
This is basically the response from macOS mdssvc for a query that yields no
results: sl_filemeta_t is empty, the CNIDs array as well.
Looking at the raw packet data, the empty sl_filemeta_t container as a size of 8
bytes which fails the following check in sl_unpack_cpx():
case SQ_CPX_TYPE_FILEMETA:
...
if (tag.size < 16) {
*boom*
}
Only tag.size=0 is invalid, tag.size=8 denotes an empty container and tag.size>=16
denotes a sl_filemeta_t container with actual content must be unpacked by
calling sl_unpack(). Note that size is always a muliple of 8.
Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
commit ed37f2d3c478da34dd9b80171ec603be3c23e37b
Author: Ralph Boehme <slow at samba.org>
Date: Sat May 4 11:10:37 2019 +0200
s3:mdssvc: marshalling: add a newline to a DEBUG message
Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
commit c282d76d556ae3744cae3f113afeda746b93949f
Author: Ralph Boehme <slow at samba.org>
Date: Sat May 4 08:53:48 2019 +0200
torture: start of a mdssvc packet (un)marshalling testsuite
Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
commit 9d6794df728d772e6bae611f782d70ed97e5c5aa
Author: Ralph Boehme <slow at samba.org>
Date: Mon Jul 15 16:44:08 2019 +0200
librpc: add RPC_NDR_MDSSVC
Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
commit 12ef7b38437f4a4211fa3dce3e3c1e76a3d3df78
Author: Ralph Boehme <slow at samba.org>
Date: Wed Apr 17 16:42:20 2019 +0200
s3:wscript: enable Spotlight by default
Now that we have a no-op backend that is always available, we can compile mdssvc
by default.
The new behaviour is:
option not used Default: build mdsvc with available backends
from autodetection
--disable-spotlight Do not build mdssvc
--enable-spotlight Build mdssvc and require a real backend
(currently Tracker, in the future also Elasticsearch)
Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
commit 25c5012c53f4d8efbe04ee8d3c8af256fec592a7
Author: Ralph Boehme <slow at samba.org>
Date: Tue Jul 30 15:39:38 2019 +0200
s3:mdssvc: avoid strncpy when marshalling strings
Avoids failure when at O3 level:
[2082/4232] Compiling source3/rpc_server/mdssvc/marshalling.c
==> /builds/samba-team/devel/samba/samba-o3.stderr <==
In file included from /usr/include/string.h:494,
from /usr/include/bsd/string.h:30,
from ../../lib/tevent/../replace/replace.h:164,
from ../../source3/include/includes.h:23,
from ../../source3/rpc_server/mdssvc/marshalling.c:21:
In function ‘strncpy’,
inlined from ‘sl_pack_string’ at ../../source3/rpc_server/mdssvc/marshalling.c:493:2,
inlined from ‘sl_pack_loop’ at ../../source3/rpc_server/mdssvc/marshalling.c:607:13:
/usr/include/bits/string_fortified.h:106:10: error: ‘__builtin_strncpy’ output
truncated before terminating nul copying as many bytes from a string as its
length [-Werror=stringop-truncation]
106 | return __builtin___strncpy_chk (__dest, __src, __len, __bos (__dest));
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../../source3/rpc_server/mdssvc/marshalling.c: In function ‘sl_pack_loop’:
../../source3/rpc_server/mdssvc/marshalling.c:458:8: note: length computed here
458 | len = strlen(s);
| ^~~~~~~~~
cc1: all warnings being treated as errors
Marshalled strings are not 0 terminated.
Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
commit 9e706ee5737bffd6f0a699ce68a75bbd0f7fd7dd
Author: Ralph Boehme <slow at samba.org>
Date: Sun Apr 21 08:38:23 2019 +0200
unittest: workaround dependency problem in test_lib_util_modules
waf somehow screws the dependencies and the module ends up with a bunch of
missing RPC related symbols once an RPC service has special dependencies like
the mdssvc RPC service.
$ bin/test_lib_util_modules
test: test_samba_module_probe
success: test_samba_module_probe
test: test_samba_module_probe_dummy
Error loading module '/home/samba/src/bin/modules/rpc/test_dummy_module.so':
/home/samba/src/bin/modules/rpc/test_dummy_module.so: undefined symbol:
rpc_service_mode
failure: test_samba_module_probe_dummy [
NT_STATUS_IS_OK(status)
../../testsuite/unittests/test_lib_util_modules.c:39: error: Failure!
]
test: test_samba_module_probe_slash
success: test_samba_module_probe_slash
This is currently not noticed as mdssvc is not enabled in selftest, but that is
going to change with a subsequent commit.
Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
commit 0b1d9d40c249ca7f893cbcc54587ae8fc93f15ee
Author: Ralph Boehme <slow at samba.org>
Date: Tue Apr 16 14:04:16 2019 +0200
s3:mdssvc: supposed status field is in fact a fragment indicator
Spotted this in mdssvc response that containied many results for a search
request: if the mdssvc response blob is larger then ~32k, the server fragments
the response in 32k fragments and sets the "fragment" field to 1.
Note that mdssvc implemenets result set "fragmentation" at the result set layer,
not at the marshalled response buffer layer. Therefor mdssvc always sets this
field to 0.
Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
commit c742ab7a4c78db4101499d15ada1a6635d5cc35b
Author: Ralph Boehme <slow at samba.org>
Date: Wed Apr 17 11:00:52 2019 +0200
s3:mdssvc: add noindex backend
Add a new default backend that, while allowing mdsvc RPC and search queries from
clients, always returns no results.
Shares using this backend will behave the same way as shares on a macOS SMB
server where indexing is disabled.
This change will later also allow us to compile the Spotlight RPC service by
default which is a big step in the direction of adding tests to CI.
Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
commit e5a4114bb563b1ebf8ce7cad2a335bf24dbbdd32
Author: Ralph Boehme <slow at samba.org>
Date: Tue Apr 16 11:24:02 2019 +0200
s3:mdssvc: log inode number in ino_path_map_destr_cb()
Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
commit 2d98fb57e05ebe16c3a3d8256281830435320607
Author: Ralph Boehme <slow at samba.org>
Date: Sun Mar 8 08:01:06 2015 +0100
s3:mdssvc: move some code to a subsystem
We need this in a later patchset when adding a mdssvc client library and for
unit tests.
Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
commit f58026b30b2bf9d60222d39947d07ee858ee5756
Author: Ralph Boehme <slow at samba.org>
Date: Fri Nov 16 14:24:35 2018 +0100
s3:mdssvc: mds_dalloc_dump() -> dalloc_dump()
Move mds_dalloc_dump() to dalloc and rename it to dalloc_dump().
Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
commit b5731a94bbc95fa001059d488a2c16a988127619
Author: Ralph Boehme <slow at samba.org>
Date: Tue Apr 9 11:19:43 2019 +0200
s3:mdssvc: add Unicode normalisation
Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
commit 1b897df4f7489c6ac7663ca4830dc4c129dd2457
Author: Ralph Boehme <slow at samba.org>
Date: Mon May 6 15:03:02 2019 +0200
s3:mdssvc: new option "spotlight backend"
Currently there's only the tracker backend, but subsequent commits will add
other backends.
Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
commit 66b8d86d71551ab8c615428211294ca110608645
Author: Ralph Boehme <slow at samba.org>
Date: Fri Apr 12 12:15:56 2019 +0200
s3:mdssvc: pass down sharename and SNUM
Not used for now, will be needed in the upcoming Elasticsearch backend.
Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
commit 1ef96e09071458322845b4323fa59cf4e7475783
Author: Ralph Boehme <slow at samba.org>
Date: Thu Mar 14 07:38:20 2019 +0100
s3-mdssvc: factor out Tracker backend logic
This moves all Tracker backend logic into a modularized component.
This should not result in any change in behaviour, it just paves the way
for adding additional backends. Currently the only available backend is
Gnome Tracker.
slq_destroy_send/recv is not needed anymore as the problem is solved now by
correctly checking if an async Tracker request was cancelled and we got
G_IO_ERROR_CANCELLED in tracker_con_cb() or tracker_query_cb() and avoid using
user_data in that the case.
Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
commit f80f8c5f99f6b5949d14ee62261a7dd448e9b0fe
Author: Ralph Boehme <slow at samba.org>
Date: Wed Aug 7 11:59:39 2019 +0200
s3:mdssvc: move debug macro to header file
This is going to be used from other parts of the code soon.
Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
commit 17f29fa86772a3fe9c03ecee6813c377d85dde6c
Author: Ralph Boehme <slow at samba.org>
Date: Wed Aug 7 11:55:44 2019 +0200
s3:mdssvc: README.Coding fixes, long lines, modernize debug macros
Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
commit 9e0b6f667f055ab74389759471c0b7181aaf4973
Author: Ralph Boehme <slow at samba.org>
Date: Wed Mar 13 15:42:51 2019 +0100
s3:rpc_server: factor out rpc_mdssvc_sources
Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
-----------------------------------------------------------------------
Summary of changes:
docs-xml/smbdotconf/misc/spotlight_backend.xml | 26 +
lib/param/loadparm.c | 2 +
lib/param/loadparm.h | 6 +
lib/param/param_table.c | 6 +
lib/torture/torture.h | 48 ++
librpc/idl/mdssvc.idl | 2 +-
librpc/wscript_build | 5 +
selftest/target/Samba3.pm | 9 +
source3/param/loadparm.c | 1 +
source3/rpc_server/mdssvc/dalloc.c | 171 +++++
source3/rpc_server/mdssvc/dalloc.h | 2 +
source3/rpc_server/mdssvc/marshalling.c | 32 +-
source3/rpc_server/mdssvc/mdssvc.c | 765 +++++--------------
source3/rpc_server/mdssvc/mdssvc.h | 64 +-
source3/rpc_server/mdssvc/mdssvc_noindex.c | 57 ++
.../mdssvc/mdssvc_noindex.h} | 13 +-
source3/rpc_server/mdssvc/mdssvc_tracker.c | 491 ++++++++++++
source3/rpc_server/mdssvc/mdssvc_tracker.h | 62 ++
source3/rpc_server/mdssvc/sparql_parser.y | 9 +-
source3/rpc_server/mdssvc/sparql_parser_test.c | 11 +-
source3/rpc_server/mdssvc/srv_mdssvc_nt.c | 100 ++-
source3/rpc_server/wscript_build | 34 +-
source3/rpcclient/cmd_spotlight.c | 460 +++++++++++
source3/rpcclient/rpcclient.c | 15 +-
source3/rpcclient/wscript_build | 3 +
source3/selftest/tests.py | 3 +
source3/wscript | 38 +-
source3/wscript_build | 2 +-
source4/selftest/tests.py | 7 +-
source4/torture/local/local.c | 1 +
source4/torture/local/mdspkt.c | 104 +++
source4/torture/local/wscript_build | 1 +
source4/torture/rpc/mdssvc.c | 845 +++++++++++++++++++++
source4/torture/rpc/rpc.c | 81 ++
source4/torture/rpc/torture_rpc.h | 24 +-
source4/torture/wscript_build | 3 +
testsuite/unittests/wscript | 2 +-
37 files changed, 2841 insertions(+), 664 deletions(-)
create mode 100644 docs-xml/smbdotconf/misc/spotlight_backend.xml
create mode 100644 source3/rpc_server/mdssvc/mdssvc_noindex.c
copy source3/{lib/util_macstreams.h => rpc_server/mdssvc/mdssvc_noindex.h} (76%)
create mode 100644 source3/rpc_server/mdssvc/mdssvc_tracker.c
create mode 100644 source3/rpc_server/mdssvc/mdssvc_tracker.h
create mode 100644 source3/rpcclient/cmd_spotlight.c
create mode 100644 source4/torture/local/mdspkt.c
create mode 100644 source4/torture/rpc/mdssvc.c
Changeset truncated at 500 lines:
diff --git a/docs-xml/smbdotconf/misc/spotlight_backend.xml b/docs-xml/smbdotconf/misc/spotlight_backend.xml
new file mode 100644
index 00000000000..6d224b81e5f
--- /dev/null
+++ b/docs-xml/smbdotconf/misc/spotlight_backend.xml
@@ -0,0 +1,26 @@
+<samba:parameter name="spotlight backend"
+ context="S"
+ type="enum"
+ enumlist="enum_spotlight_backend"
+ xmlns:samba="http://www.samba.org/samba/DTD/samba-doc">
+<description>
+ <para>
+ Spotlight search backend. Available backends:
+
+ <itemizedlist>
+ <listitem><para><constant>noindex</constant> -
+ a backend that returns no results.
+ </para></listitem>
+
+ </itemizedlist>
+
+ <itemizedlist>
+ <listitem><para><constant>tracker</constant> -
+ Gnome Tracker.
+ </para></listitem>
+
+ </itemizedlist>
+ </para>
+</description>
+<value type="default">noindex</value>
+</samba:parameter>
diff --git a/lib/param/loadparm.c b/lib/param/loadparm.c
index 883d4167bf4..13e8b677e30 100644
--- a/lib/param/loadparm.c
+++ b/lib/param/loadparm.c
@@ -3025,6 +3025,8 @@ struct loadparm_context *loadparm_init(TALLOC_CTX *mem_ctx)
lpcfg_do_global_parameter(lp_ctx, "debug encryption", "no");
+ lpcfg_do_global_parameter(lp_ctx, "spotlight backend", "noindex");
+
for (i = 0; parm_table[i].label; i++) {
if (!(lp_ctx->flags[i] & FLAG_CMDLINE)) {
lp_ctx->flags[i] |= FLAG_DEFAULT;
diff --git a/lib/param/loadparm.h b/lib/param/loadparm.h
index 0b2e302d2a9..5c5b1cd7cd4 100644
--- a/lib/param/loadparm.h
+++ b/lib/param/loadparm.h
@@ -248,6 +248,12 @@ enum inheritowner_options {
/* mangled names options */
enum mangled_names_options {MANGLED_NAMES_NO, MANGLED_NAMES_YES, MANGLED_NAMES_ILLEGAL};
+/* Spotlight backend options */
+enum spotlight_backend_options {
+ SPOTLIGHT_BACKEND_NOINDEX,
+ SPOTLIGHT_BACKEND_TRACKER,
+};
+
/*
* Default passwd chat script.
*/
diff --git a/lib/param/param_table.c b/lib/param/param_table.c
index f9d3b55adf2..82dc5cd6cda 100644
--- a/lib/param/param_table.c
+++ b/lib/param/param_table.c
@@ -351,6 +351,12 @@ static const struct enum_list enum_ntlm_auth[] = {
{-1, NULL}
};
+static const struct enum_list enum_spotlight_backend[] = {
+ {SPOTLIGHT_BACKEND_NOINDEX, "noindex"},
+ {SPOTLIGHT_BACKEND_TRACKER, "tracker"},
+ {-1, NULL}
+};
+
/* Note: We do not initialise the defaults union - it is not allowed in ANSI C
*
* NOTE: Handling of duplicated (synonym) parameters:
diff --git a/lib/torture/torture.h b/lib/torture/torture.h
index 9f55c164ac1..3507740967c 100644
--- a/lib/torture/torture.h
+++ b/lib/torture/torture.h
@@ -520,6 +520,54 @@ static inline void torture_dump_data_str_cb(const char *buf, void *private_data)
} \
} while(0)
+#define torture_assert_u32_equal(torture_ctx,got,expected,cmt)\
+ do { uint32_t __got = (got), __expected = (expected); \
+ if (__got != __expected) { \
+ torture_result(torture_ctx, TORTURE_FAIL, \
+ __location__": "#got" was %ju (0x%jX), expected %ju (0x%jX): %s", \
+ (uintmax_t)__got, (uintmax_t)__got, \
+ (uintmax_t)__expected, (uintmax_t)__expected, \
+ cmt); \
+ return false; \
+ } \
+ } while(0)
+
+#define torture_assert_u32_equal_goto(torture_ctx,got,expected,ret,label,cmt)\
+ do { uint32_t __got = (got), __expected = (expected); \
+ if (__got != __expected) { \
+ torture_result(torture_ctx, TORTURE_FAIL, \
+ __location__": "#got" was %ju (0x%jX), expected %ju (0x%jX): %s", \
+ (uintmax_t)__got, (uintmax_t)__got, \
+ (uintmax_t)__expected, (uintmax_t)__expected, \
+ cmt); \
+ ret = false; \
+ goto label; \
+ } \
+ } while(0)
+
+#define torture_assert_u32_not_equal(torture_ctx,got,not_expected,cmt)\
+ do { uint32_t __got = (got), __not_expected = (not_expected); \
+ if (__got == __not_expected) { \
+ torture_result(torture_ctx, TORTURE_FAIL, \
+ __location__": "#got" was %ju (0x%jX), expected a different number: %s", \
+ (uintmax_t)__got, (uintmax_t)__got, \
+ cmt); \
+ return false; \
+ } \
+ } while(0)
+
+#define torture_assert_u32_not_equal_goto(torture_ctx,got,not_expected,ret,label,cmt)\
+ do { uint32_t __got = (got), __not_expected = (not_expected); \
+ if (__got == __not_expected) { \
+ torture_result(torture_ctx, TORTURE_FAIL, \
+ __location__": "#got" was %ju (0x%jX), expected a different number: %s", \
+ (uintmax_t)__got, (uintmax_t)__got, \
+ cmt); \
+ ret = false; \
+ goto label; \
+ } \
+ } while(0)
+
#define torture_assert_u64_equal(torture_ctx,got,expected,cmt)\
do { uint64_t __got = (got), __expected = (expected); \
if (__got != __expected) { \
diff --git a/librpc/idl/mdssvc.idl b/librpc/idl/mdssvc.idl
index bfb27c4a795..43447961b0d 100644
--- a/librpc/idl/mdssvc.idl
+++ b/librpc/idl/mdssvc.idl
@@ -51,7 +51,7 @@ interface mdssvc
[in] uint32 max_fragment_size2,
[in] uint32 unkn7, /* always 0 ? */
[in] uint32 unkn8, /* always 0 ? */
- [out,ref] uint32 *status,
+ [out,ref] uint32 *fragment,
[out,ref] mdssvc_blob *response_blob,
[out,ref] uint32 *unkn9 /* always 0 ? */
);
diff --git a/librpc/wscript_build b/librpc/wscript_build
index 70fe8c2f7fe..d72592b52cc 100644
--- a/librpc/wscript_build
+++ b/librpc/wscript_build
@@ -597,6 +597,11 @@ bld.SAMBA_SUBSYSTEM('RPC_NDR_CLUSAPI',
public_deps='dcerpc-binding NDR_CLUSAPI'
)
+bld.SAMBA_SUBSYSTEM('RPC_NDR_MDSSVC',
+ source='gen_ndr/ndr_mdssvc_c.c',
+ public_deps='dcerpc-binding NDR_MDSSVC'
+ )
+
# a grouping library for NDR subsystems that may be used by more than one target
bld.SAMBA_LIBRARY('ndr-samba',
source=[],
diff --git a/selftest/target/Samba3.pm b/selftest/target/Samba3.pm
index 5c327cab543..2587ea31253 100755
--- a/selftest/target/Samba3.pm
+++ b/selftest/target/Samba3.pm
@@ -939,6 +939,7 @@ sub setup_fileserver
my $fileserver_options = "
kernel change notify = yes
+ rpc_server:mdssvc = embedded
usershare path = $usershare_dir
usershare max shares = 10
@@ -947,6 +948,14 @@ sub setup_fileserver
get quota command = $prefix_abs/getset_quota.py
set quota command = $prefix_abs/getset_quota.py
+[spotlight]
+ path = $share_dir
+ spotlight = yes
+ read only = no
+[no_spotlight]
+ path = $share_dir
+ spotlight = no
+ read only = no
[lowercase]
path = $lower_case_share_dir
comment = smb username is [%U]
diff --git a/source3/param/loadparm.c b/source3/param/loadparm.c
index 9e97d719f4e..0daac0e44cb 100644
--- a/source3/param/loadparm.c
+++ b/source3/param/loadparm.c
@@ -247,6 +247,7 @@ static const struct loadparm_service _sDefault =
.param_opt = NULL,
.smbd_search_ask_sharemode = true,
.smbd_getinfo_ask_sharemode = true,
+ .spotlight_backend = SPOTLIGHT_BACKEND_NOINDEX,
.dummy = ""
};
diff --git a/source3/rpc_server/mdssvc/dalloc.c b/source3/rpc_server/mdssvc/dalloc.c
index 28944b809f1..23193fee70a 100644
--- a/source3/rpc_server/mdssvc/dalloc.c
+++ b/source3/rpc_server/mdssvc/dalloc.c
@@ -18,6 +18,10 @@
#include "replace.h"
#include <talloc.h>
#include "dalloc.h"
+#include "marshalling.h"
+#include "lib/util/charset/charset.h"
+#include "lib/util/talloc_stack.h"
+#include "system/time.h"
/**
* Dynamic Datastore
@@ -230,3 +234,170 @@ int dalloc_stradd(DALLOC_CTX *d, const char *string)
return 0;
}
+
+static char *tab_level(TALLOC_CTX *mem_ctx, int level)
+{
+ int i;
+ char *string = talloc_array(mem_ctx, char, level + 1);
+
+ for (i = 0; i < level; i++) {
+ string[i] = '\t';
+ }
+
+ string[i] = '\0';
+ return string;
+}
+
+char *dalloc_dump(DALLOC_CTX *dd, int nestinglevel)
+{
+ const char *type;
+ int n, result;
+ uint64_t i;
+ sl_bool_t bl;
+ sl_time_t t;
+ struct tm *tm;
+ char datestring[256];
+ sl_cnids_t cnids;
+ char *logstring, *nested_logstring;
+ char *tab_string1, *tab_string2;
+ void *p;
+ bool ok;
+ char *utf8string;
+ size_t utf8len;
+
+ tab_string1 = tab_level(dd, nestinglevel);
+ if (tab_string1 == NULL) {
+ return NULL;
+ }
+ tab_string2 = tab_level(dd, nestinglevel + 1);
+ if (tab_string2 == NULL) {
+ return NULL;
+ }
+
+ logstring = talloc_asprintf(dd,
+ "%s%s(#%lu): {\n",
+ tab_string1,
+ talloc_get_name(dd),
+ dalloc_size(dd));
+ if (logstring == NULL) {
+ return NULL;
+ }
+
+ for (n = 0; n < dalloc_size(dd); n++) {
+ type = dalloc_get_name(dd, n);
+ if (type == NULL) {
+ return NULL;
+ }
+ p = dalloc_get_object(dd, n);
+ if (p == NULL) {
+ return NULL;
+ }
+ if (strcmp(type, "DALLOC_CTX") == 0
+ || strcmp(type, "sl_array_t") == 0
+ || strcmp(type, "sl_filemeta_t") == 0
+ || strcmp(type, "sl_dict_t") == 0) {
+ nested_logstring = dalloc_dump(p, nestinglevel + 1);
+ if (nested_logstring == NULL) {
+ return NULL;
+ }
+ logstring = talloc_strdup_append(logstring,
+ nested_logstring);
+ } else if (strcmp(type, "uint64_t") == 0) {
+ memcpy(&i, p, sizeof(uint64_t));
+ logstring = talloc_asprintf_append(
+ logstring,
+ "%suint64_t: 0x%04jx\n",
+ tab_string2, (uintmax_t)i);
+ } else if (strcmp(type, "char *") == 0) {
+ logstring = talloc_asprintf_append(
+ logstring,
+ "%sstring: %s\n",
+ tab_string2,
+ (char *)p);
+ } else if (strcmp(type, "smb_ucs2_t *") == 0) {
+ ok = convert_string_talloc(talloc_tos(),
+ CH_UTF16LE,
+ CH_UTF8,
+ p,
+ talloc_get_size(p),
+ &utf8string,
+ &utf8len);
+ if (!ok) {
+ return NULL;
+ }
+ logstring = talloc_asprintf_append(
+ logstring,
+ "%sUTF16-string: %s\n",
+ tab_string2,
+ utf8string);
+ TALLOC_FREE(utf8string);
+ } else if (strcmp(type, "sl_bool_t") == 0) {
+ memcpy(&bl, p, sizeof(sl_bool_t));
+ logstring = talloc_asprintf_append(
+ logstring,
+ "%sbool: %s\n",
+ tab_string2,
+ bl ? "true" : "false");
+ } else if (strcmp(type, "sl_nil_t") == 0) {
+ logstring = talloc_asprintf_append(
+ logstring,
+ "%snil\n",
+ tab_string2);
+ } else if (strcmp(type, "sl_time_t") == 0) {
+ memcpy(&t, p, sizeof(sl_time_t));
+ tm = localtime(&t.tv_sec);
+ if (tm == NULL) {
+ return NULL;
+ }
+ result = strftime(datestring,
+ sizeof(datestring),
+ "%Y-%m-%d %H:%M:%S", tm);
+ if (result == 0) {
+ return NULL;
+ }
+ logstring = talloc_asprintf_append(
+ logstring,
+ "%ssl_time_t: %s.%06lu\n",
+ tab_string2,
+ datestring,
+ (unsigned long)t.tv_usec);
+ } else if (strcmp(type, "sl_cnids_t") == 0) {
+ memcpy(&cnids, p, sizeof(sl_cnids_t));
+ logstring = talloc_asprintf_append(
+ logstring,
+ "%sCNIDs: unkn1: 0x%" PRIx16 ", unkn2: 0x%" PRIx32 "\n",
+ tab_string2,
+ cnids.ca_unkn1,
+ cnids.ca_context);
+ if (logstring == NULL) {
+ return NULL;
+ }
+ if (cnids.ca_cnids) {
+ nested_logstring = dalloc_dump(
+ cnids.ca_cnids,
+ nestinglevel + 2);
+ if (!nested_logstring) {
+ return NULL;
+ }
+ logstring = talloc_strdup_append(logstring,
+ nested_logstring);
+ }
+ } else {
+ logstring = talloc_asprintf_append(
+ logstring,
+ "%stype: %s\n",
+ tab_string2,
+ type);
+ }
+ if (logstring == NULL) {
+ return NULL;
+ }
+ }
+ logstring = talloc_asprintf_append(logstring,
+ "%s}\n",
+ tab_string1);
+ if (logstring == NULL) {
+ return NULL;
+ }
+ return logstring;
+}
diff --git a/source3/rpc_server/mdssvc/dalloc.h b/source3/rpc_server/mdssvc/dalloc.h
index b268893160b..69650b8a7a0 100644
--- a/source3/rpc_server/mdssvc/dalloc.h
+++ b/source3/rpc_server/mdssvc/dalloc.h
@@ -160,4 +160,6 @@ extern int dalloc_stradd(DALLOC_CTX *d, const char *string);
extern void *_dalloc_new(TALLOC_CTX *mem_ctx, const char *type);
extern int _dalloc_add_talloc_chunk(DALLOC_CTX *d, void *obj, const char *type, size_t size);
+extern char *dalloc_dump(DALLOC_CTX *dd, int nestinglevel);
+
#endif /* DALLOC_H */
diff --git a/source3/rpc_server/mdssvc/marshalling.c b/source3/rpc_server/mdssvc/marshalling.c
index b5931c7f060..1aa750413cd 100644
--- a/source3/rpc_server/mdssvc/marshalling.c
+++ b/source3/rpc_server/mdssvc/marshalling.c
@@ -490,7 +490,7 @@ static ssize_t sl_pack_string(char *s, char *buf, ssize_t offset, size_t bufsize
}
memset(buf + offset, 0, octets * 8);
- strncpy(buf + offset, s, len);
+ memcpy(buf + offset, s, len);
offset += octets * 8;
return offset;
@@ -847,12 +847,17 @@ static int sl_unpack_CNID(DALLOC_CTX *query,
return -1;
}
- if (length <= 16) {
+ if (length < 8) {
+ return -1;
+ }
+ if (length == 8) {
/*
- * That's permitted, iirc length = 16 is an empty
- * array, so anything lesser then 16 should probably
- * be treated as an error, but I'm not quite sure.
+ * That's permitted, length=8 is an empty CNID array.
*/
+ result = dalloc_add(query, cnids, sl_cnids_t);
+ if (result != 0) {
+ return -1;
+ }
return 0;
}
@@ -1005,8 +1010,8 @@ static ssize_t sl_unpack_cpx(DALLOC_CTX *query,
if (offset == -1) {
return -1;
}
- if (tag.size < 16) {
- DEBUG(1, ("%s: size too mall: %zu", __func__, tag.size));
+ if (tag.size < 8) {
+ DBG_WARNING("size too mall: %zu\n", tag.size);
return -1;
}
@@ -1014,9 +1019,14 @@ static ssize_t sl_unpack_cpx(DALLOC_CTX *query,
if (sl_fm == NULL) {
return -1;
}
- result = sl_unpack(sl_fm, buf + offset, bufsize - offset );
- if (result == -1) {
- return -1;
+
+ if (tag.size >= 16) {
+ result = sl_unpack(sl_fm,
+ buf + offset,
+ bufsize - offset );
+ if (result == -1) {
+ return -1;
+ }
}
result = dalloc_add(query, sl_fm, sl_filemeta_t);
if (result != 0) {
@@ -1032,7 +1042,7 @@ static ssize_t sl_unpack_cpx(DALLOC_CTX *query,
}
result = sl_unpack_CNID(query, buf, offset, bufsize,
- tag.length, encoding);
+ tag.size, encoding);
if (result == -1) {
return -1;
}
diff --git a/source3/rpc_server/mdssvc/mdssvc.c b/source3/rpc_server/mdssvc/mdssvc.c
index 58a219bedb0..24183329301 100644
--- a/source3/rpc_server/mdssvc/mdssvc.c
+++ b/source3/rpc_server/mdssvc/mdssvc.c
@@ -27,32 +27,14 @@
#include "lib/dbwrap/dbwrap_rbt.h"
#include "libcli/security/dom_sid.h"
#include "mdssvc.h"
-#include "rpc_server/mdssvc/sparql_parser.tab.h"
-#include "lib/tevent_glib_glue.h"
+#include "mdssvc_noindex.h"
+#ifdef HAVE_SPOTLIGHT_BACKEND_TRACKER
+#include "mdssvc_tracker.h"
+#endif
#undef DBGC_CLASS
#define DBGC_CLASS DBGC_RPC_SRV
-#define SLQ_DEBUG(lvl, _slq, state) do { if (CHECK_DEBUGLVL(lvl)) { \
- const struct sl_query *__slq = _slq; \
- struct timeval_buf start_buf; \
- const char *start; \
- struct timeval_buf last_used_buf; \
- const char *last_used; \
--
Samba Shared Repository
More information about the samba-cvs
mailing list