WIP WSP patches

Noel Power nopower at suse.com
Fri Nov 25 14:27:55 UTC 2016


Hi,

I've updated my github repo

On 23/09/16 09:17, Noel Power wrote:
> On 23/09/16 05:00, Ralph Böhme wrote: 
[...]
>> nevermind, looks like this is because we don't do a pipe wait before
>> the pipe open. Adding support for this to libsmb atm... 
> interesting, seems like a nice catch, I never encountered that, I have
> only run the client against a 2k12 server and the pipe was probably
> already setup from previous testing.
I didn't see a patch come through and since I still need all the
practise I can get to understand this samba/SMB stuff I had a crack at
it myself (but I'll quite happily replace my patches with whatever is
else available if needed)
> Have to say though I only added the client for testing purposes
> really, it needs some love I think to make it more generally useful.
> E.g. to make binary queries more flexible, maybe limit results
> returned and I was thinking maybe some optional shell interface to
> help create more complex queries.... but that's all just thoughts
> right now.

Well thoughts sometimes turn into reality so I've updated the branch
WSP-WIP-CLIENT-ONLY (and the others) with some new client & server
functionality

Firstly the wspsearch tool has some new options

  o --limit=number which allows you to limit the search results
returned, the following example will limit the results returned to 500,
note the default value for this option is 0 (meaning unlimited)
   
wspsearch -U$(USER)%$(PASSWD) --kind=Picture
--limit=500 //$(SERVER)/$(SHARE)

  o --query=$QUERY where $QUERY is something more or less resembling an
AQS (advanced query syntax) query. I was looking for a way to specify
more complex queries and came across
https://msdn.microsoft.com/en-us/library/windows/desktop/bb266512(v=vs.85).aspx#examples)
I decided to implement the syntax described there (not slavishly but
lets say a subset) This at least allows dynamic and expressive queries
to be built (great for testing the server/protocol and useful for actual
searches too!!)


Probably best to describe what is supported by some simple examples;

The basic syntax is supported e.g. a query is built of a sequence of
queries connected by AND, OR and NOT where the query elements are
essentially restrictions defined by a property. There are some
limitations on the operators supported[1] and additionally some things
like enumerated ranges are not supported at all, range values and not
delimited as specified [2]. Some special cases that you see in the UI
are exceptions [3] which are handled more or less as keywords

Examples
--------

The following are all exactly the same query just expressed using
different variations of the syntax

 wspsearch -Ufoo%bar --query='SALL:($<p403 OR $<p404) AND
System.Kind:picture AND Scope:"FILE://somemachine/someshare" AND
System.Size:10241-102401' //somemachine

 wspsearch -Ufoo%bar --query='ALL:$<p403 OR ALL:$<p404 AND
System.Kind:picture AND Scope:"FILE://somemachine/someshare" AND
System.Size:>=10241 AND System.Size:<102401' //somemachine


 wspsearch -Ufoo%bar --query='ALL:$<p403 OR ALL:$<p404 AND
System.Kind:picture AND Scope:"FILE://somemachine/someshare" AND
System.Size:small' //somemachine

The queries above by default select the property System.ItemUrl as the
one and only column returned, the query parameter however accepts a
variation to the AQS like syntax to allow arbitrary columns to be
selected e.g.

./bin/wspsearch -Ufoo%bar --query='SELECT System.ItemName,
System.ItemURL, System.Size  WHERE ALL:$<p403 OR ALL:$<p404 AND
System.Kind:picture AND Scope:"FILE://somemachine/someshare" AND
System.Size:small' //somemachine

Secondly the server implementation has been modified a little too,
besides some shuffling around of code to share between the cmdline tools
wsp-to-tracker, wspsearch and the server implementation and some
cleaning up of the code (more to do there :-)) support for some new
smb.conf options has been added

  wsp result limit (G) like the client this limits the number of results
that the server processes, currently the default value is 0 meaning
unlimited results returned
  wsp (S) This allows the wsp searching to be disabled on a per share
basis. Behaviour here is a little odd (but matches windows) on a machine
with indexing enabled where a share was previously indexed (but has the
indexing turned off (but index service still runs)) it takes quite a
while even up to 30mins or more for the client to switch over to
searching using SMB commands for that share, in the time there can be 0
results returned for searches that otherwise would return results
(strange but true).



Noel

ps once again, please let me know what I can do or change to help this along

[1] supported operators
    -------------------
    =    Equals
    !=   Not Equals
    >    Greater than
    <    Less than
    >=   Greater than or equals
    <=   Less than or equals

    $=   equals
    $<   starts with

Note: The binary CPMCREATEQUERY message allow many many different
restriction types to be used, the query syntax described above only
produces 2 types of restrictions either RTPROPERTY or RTCONTENT. Which
type of restriction is created is determined by the operators above
where using the last 2 will result in a RTCONTENT restriction being used
or RTPROPERTY otherwise.

[2] ranges are specified as value-value instead of value..value (seems
my flex/bison skills are not good enough and couldn't get that to work
yet, I'll keep trying)

[3] The UI has shortcut ranges (presumably the enumerated ranges
mentioned earlier) providingdate ranges like 'today', 'tomorrow',
'lastweek' etc. and similarly sizes like "empty, tiny, small, large..."
These are supported (but implemented as keywords)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: WSP-WIP-CLIENT-ONLY.patch
Type: text/x-patch
Size: 251210 bytes
Desc: not available
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20161125/eb395de7/WSP-WIP-CLIENT-ONLY-0001.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: WSP-WIP-NO_RAWPIPE.patch
Type: text/x-patch
Size: 568212 bytes
Desc: not available
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20161125/eb395de7/WSP-WIP-NO_RAWPIPE-0001.bin>


More information about the samba-technical mailing list