WIP WSP patches
Noel Power
nopower at suse.com
Fri Jan 6 13:56:29 UTC 2017
Hello List
Another gentle reminder about this stuff (and a little update)
I finally got around to adding support for 32/64 bit msg negotiation so
that the client/server can operate in either mode. Both the WSP server
and wspsearch cli client now use a version id based on the build time
arch (and thus use either a 32 or 64 bit version id as appropriate)
which they send on connection requests. I cleaned up the client code
quite a bit (and tweaked some of the Row handling code in the server
too) I am sure there is plenty of room for further improvement.
If you want to try it out I've created a repository [1] that you can use
with a suse system (leap or sle)[2] that you can download rpms from [2]
This repo contains 2 packages of interest, the first obviously is samba
(which has the WSP patches enabled) and secondly a package called
system-tracker, system-tracker sets up all the systemd related foo
(including creating a dedicated user called 'tracker') based on what's
described at https://wiki.samba.org/index.php/Spotlight#Setup. So with a
desktop suse system (e.g. Leap42.2) just a few steps are needed to get
it going
1. install system-tracker & samba packages from [1]
2. start tracker as described
"https://wiki.samba.org/index.php/Spotlight#Start_Tracker_as_a_special_System_Service_with_systemd"
3. configure tracker as described
"https://wiki.samba.org/index.php/Spotlight#Configuring_which_Paths_to_index_in_Tracker"
NOTE: remember to use the the username 'tracker' instead of 'mdssvc'
4. configure smb.conf
[global]
....
rpc_daemon:wspd = fork
....
[share]
path = /some/path
wsp = yes
5. restart smbd and you should be good to go
Once again, please let me know what I can do to move this along......
I've attached patches that correspond to branch WSP-WIP-CLIENT-ONLY &
WSP-WIP-NO_RAWPIPE from my github samba repo
https://github.com/noelpower/samba
Noel
[1] http://download.opensuse.org/repositories/home:/npower:/WSP
[2] although the build service supports creating packages for other
distros like Fedora etc I haven't enabled those because I lack any real
packaging skills, I barely got it working for the suse stuff :-) I'd
happily add any patches to support other distros if anyone cares to
help. Otherwise I will in time try to get around to enabling some other
vendor specific repos
On 25/11/16 14:27, Noel Power wrote:
> Hi,
>
> I've updated my github repo
>
> On 23/09/16 09:17, Noel Power wrote:
>> On 23/09/16 05:00, Ralph Böhme wrote:
> [...]
>>> nevermind, looks like this is because we don't do a pipe wait before
>>> the pipe open. Adding support for this to libsmb atm...
>> interesting, seems like a nice catch, I never encountered that, I have
>> only run the client against a 2k12 server and the pipe was probably
>> already setup from previous testing.
> I didn't see a patch come through and since I still need all the
> practise I can get to understand this samba/SMB stuff I had a crack at
> it myself (but I'll quite happily replace my patches with whatever is
> else available if needed)
>> Have to say though I only added the client for testing purposes
>> really, it needs some love I think to make it more generally useful.
>> E.g. to make binary queries more flexible, maybe limit results
>> returned and I was thinking maybe some optional shell interface to
>> help create more complex queries.... but that's all just thoughts
>> right now.
> Well thoughts sometimes turn into reality so I've updated the branch
> WSP-WIP-CLIENT-ONLY (and the others) with some new client & server
> functionality
>
> Firstly the wspsearch tool has some new options
>
> o --limit=number which allows you to limit the search results
> returned, the following example will limit the results returned to 500,
> note the default value for this option is 0 (meaning unlimited)
>
> wspsearch -U$(USER)%$(PASSWD) --kind=Picture
> --limit=500 //$(SERVER)/$(SHARE)
>
> o --query=$QUERY where $QUERY is something more or less resembling an
> AQS (advanced query syntax) query. I was looking for a way to specify
> more complex queries and came across
> https://msdn.microsoft.com/en-us/library/windows/desktop/bb266512(v=vs.85).aspx#examples)
> I decided to implement the syntax described there (not slavishly but
> lets say a subset) This at least allows dynamic and expressive queries
> to be built (great for testing the server/protocol and useful for actual
> searches too!!)
>
>
> Probably best to describe what is supported by some simple examples;
>
> The basic syntax is supported e.g. a query is built of a sequence of
> queries connected by AND, OR and NOT where the query elements are
> essentially restrictions defined by a property. There are some
> limitations on the operators supported[1] and additionally some things
> like enumerated ranges are not supported at all, range values and not
> delimited as specified [2]. Some special cases that you see in the UI
> are exceptions [3] which are handled more or less as keywords
>
> Examples
> --------
>
> The following are all exactly the same query just expressed using
> different variations of the syntax
>
> wspsearch -Ufoo%bar --query='SALL:($<p403 OR $<p404) AND
> System.Kind:picture AND Scope:"FILE://somemachine/someshare" AND
> System.Size:10241-102401' //somemachine
>
> wspsearch -Ufoo%bar --query='ALL:$<p403 OR ALL:$<p404 AND
> System.Kind:picture AND Scope:"FILE://somemachine/someshare" AND
> System.Size:>=10241 AND System.Size:<102401' //somemachine
>
>
> wspsearch -Ufoo%bar --query='ALL:$<p403 OR ALL:$<p404 AND
> System.Kind:picture AND Scope:"FILE://somemachine/someshare" AND
> System.Size:small' //somemachine
>
> The queries above by default select the property System.ItemUrl as the
> one and only column returned, the query parameter however accepts a
> variation to the AQS like syntax to allow arbitrary columns to be
> selected e.g.
>
> ./bin/wspsearch -Ufoo%bar --query='SELECT System.ItemName,
> System.ItemURL, System.Size WHERE ALL:$<p403 OR ALL:$<p404 AND
> System.Kind:picture AND Scope:"FILE://somemachine/someshare" AND
> System.Size:small' //somemachine
>
> Secondly the server implementation has been modified a little too,
> besides some shuffling around of code to share between the cmdline tools
> wsp-to-tracker, wspsearch and the server implementation and some
> cleaning up of the code (more to do there :-)) support for some new
> smb.conf options has been added
>
> wsp result limit (G) like the client this limits the number of results
> that the server processes, currently the default value is 0 meaning
> unlimited results returned
> wsp (S) This allows the wsp searching to be disabled on a per share
> basis. Behaviour here is a little odd (but matches windows) on a machine
> with indexing enabled where a share was previously indexed (but has the
> indexing turned off (but index service still runs)) it takes quite a
> while even up to 30mins or more for the client to switch over to
> searching using SMB commands for that share, in the time there can be 0
> results returned for searches that otherwise would return results
> (strange but true).
>
>
>
> Noel
>
> ps once again, please let me know what I can do or change to help this along
>
> [1] supported operators
> -------------------
> = Equals
> != Not Equals
> > Greater than
> < Less than
> >= Greater than or equals
> <= Less than or equals
>
> $= equals
> $< starts with
>
> Note: The binary CPMCREATEQUERY message allow many many different
> restriction types to be used, the query syntax described above only
> produces 2 types of restrictions either RTPROPERTY or RTCONTENT. Which
> type of restriction is created is determined by the operators above
> where using the last 2 will result in a RTCONTENT restriction being used
> or RTPROPERTY otherwise.
>
> [2] ranges are specified as value-value instead of value..value (seems
> my flex/bison skills are not good enough and couldn't get that to work
> yet, I'll keep trying)
>
> [3] The UI has shortcut ranges (presumably the enumerated ranges
> mentioned earlier) providingdate ranges like 'today', 'tomorrow',
> 'lastweek' etc. and similarly sizes like "empty, tiny, small, large..."
> These are supported (but implemented as keywords)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: WSP-WIP-CLIENT-ONLY.patch
Type: text/x-diff
Size: 437391 bytes
Desc: not available
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20170106/d70d91b4/WSP-WIP-CLIENT-ONLY-0001.diff>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: WSP-WIP-NO_RAWPIPE.patch
Type: text/x-diff
Size: 757875 bytes
Desc: not available
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20170106/d70d91b4/WSP-WIP-NO_RAWPIPE-0001.diff>
More information about the samba-technical
mailing list