[SCM] Samba Shared Repository - branch master updated
Noel Power
npower at samba.org
Thu Sep 3 14:34:02 UTC 2026
The branch, master has been updated
via 8fd1e3196c4 selftest: add new wsp python bindings/wsp message test
via dc1307203b8 selftest: Add new test harness for wsp elasticsearch backend
via 5b949010bc4 selftest: add some tests to parse and verify wsp -> elasticsearch query
via 51a017dfe51 selftest: update client.conf for elasticsearch mappings
via 302b97bd038 selftest: testenv fileserver setup wsp elastic mappings
via 9f849889c4d s3/wscript add and build py_wsp.c (python wsp client api bindings)
via da5af8e3f59 s3/librpc: Build generated python bindings into library
via 23fddd35148 librpc/idl/wsp.idl Add dummy uuid to wsp.idl
via d224c8bfafb wsp-to: initialise the elasticsearch backend
via 95aa924c90a s3/utils: add wsp-to
via 32d5ccb0755 s3:rpc_server: add npsd_wsp_server.c
via e5a2ff8f885 s3/rpc_server/wsp: elasticsearch gss abstract interfaces
via edcb9680a2b wsp: add Elasticsearch conversion functions
via c9e691464b0 s3/rpc_server/wsp: start of implementation of elasticsearch bld_mapper
via 4b504283420 codespell: Add ignore for cnat variable name which triggers codespell err.
via ba71e3f4a96 docs-xml: Document elasticsearch:wsp use fscrawler folders
via 5907a0d0e6a docs-xml: Add elasticsearch smbconf constant for wsp backend
via 4c0f8dedc6d s3: configure check for Elasticsearch WSP backend
via dd9f836b8c0 s3/rpc_server/wsp: add gss implementation
via cbe7bce8720 s4/scripting: Add HRESULT_FROM_NT() and HRESULT_TO_NT()
via 78058c34a20 lib/util: add tevent_req helpers to deal with hresult and errors
via d941492e726 s3/rpc_server/wsp: store entryid -> docid fwd/reverse mappings
via 66ad9182358 lib/util: Add new cache types to cache entryid -> docid (and reverse)
via af7147bbd30 docs-xml/smbdotconf: add some new smb.conf params for WSP
via abd2d648670 s3/rpc_server: Add a WSP backend registry
via a5aacf1ac16 s3/rpc_server: abstract interface for WSP operations
via f451a1e2a96 mdssvc: use create_conn_struct() and conn_wrap_chdir()
via 4ba9118d49f smbd: add conn_wrap_chdir()
via 4395cde6015 smbd: add create_conn_struct()
via a6b8e27ff52 smbd: split out create_conn_struct_chdir_internal()
via b2ca4c3f325 libcli/http: Allow cli to sent http GET
via 6c5b5afd8b1 librpc/idl: ADD DB_S_BLOCKLIMITEDROWS to idl
via e6eac0e6d97 librpc/idl: Add Priority enum
via dbd8b206b8e librpc: Add backend constants to idl
via b004b59210a s3/rpcs_server/mdssvc: Use new common escape_str routines
via 425e0971e48 s3/lib: add elastic_util elastic utility functions
from 5012cc95a3d s3:utils: fix wrong deregister in do_profilelevel()
https://git.samba.org/?p=samba.git;a=shortlog;h=master
- Log -----------------------------------------------------------------
commit 8fd1e3196c45b6863514eeb406f585ae87789526
Author: Noel Power <noel.power at suse.com>
Date: Sat Apr 17 11:30:42 2021 +0100
selftest: add new wsp python bindings/wsp message test
Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Ralph Boehme <slow at samba.org>
Autobuild-User(master): Noel Power <npower at samba.org>
Autobuild-Date(master): Thu Sep 3 14:33:08 UTC 2026 on atb-devel-224
commit dc1307203b8a7932c4dcc6a5918d1b75a3aeb1a3
Author: Noel Power <noel.power at suse.com>
Date: Thu Mar 25 11:48:08 2021 +0000
selftest: Add new test harness for wsp elasticsearch backend
Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Ralph Boehme <slow at samba.org>
commit 5b949010bc4b19033bad7b1c99036904da401c79
Author: Noel Power <noel.power at suse.com>
Date: Wed Mar 24 18:19:48 2021 +0000
selftest: add some tests to parse and verify wsp -> elasticsearch query
wsp-to tool accepts simple query notation which is in turn converted
to wsp binary format and then passed into the elasticsearch query
generation code. These tests verify the end to end translation of some
basic queries.
Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Ralph Boehme <slow at samba.org>
commit 51a017dfe51bdb34b4181f717bf1d3850f0e7a09
Author: Noel Power <noel.power at suse.com>
Date: Thu Mar 14 15:54:24 2024 +0000
selftest: update client.conf for elasticsearch mappings
Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Ralph Boehme <slow at samba.org>
commit 302b97bd038fbd65d9ae2dd235052740ca67108d
Author: Noel Power <noel.power at suse.com>
Date: Wed Mar 24 15:05:13 2021 +0000
selftest: testenv fileserver setup wsp elastic mappings
Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Ralph Boehme <slow at samba.org>
commit 9f849889c4df649a40c151f0ea5bfc65cd50cb4b
Author: Noel Power <noel.power at suse.com>
Date: Tue Nov 21 20:33:15 2023 +0000
s3/wscript add and build py_wsp.c (python wsp client api bindings)
Simple python api to provide ability to write some python tests
Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Ralph Boehme <slow at samba.org>
commit da5af8e3f59943ce714f22c5f4114be7df860ef1
Author: Noel Power <noel.power at suse.com>
Date: Tue Apr 13 12:14:33 2021 +0100
s3/librpc: Build generated python bindings into library
generated python needs to be compiled to 2 shared libs
we need a shared lib each for wsp and wsp_data
Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Ralph Boehme <slow at samba.org>
commit 23fddd35148e1f9b4478a92568d517b7f5132833
Author: Noel Power <noel.power at suse.com>
Date: Tue Jun 20 10:44:00 2023 +0100
librpc/idl/wsp.idl Add dummy uuid to wsp.idl
This just ensures with server idl flags the endpoint related
code and tables get generated
Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Ralph Boehme <slow at samba.org>
commit d224c8bfafb7e5e15370b6258bc893198a3faa15
Author: Noel Power <noel.power at suse.com>
Date: Wed Jun 5 12:32:15 2024 +0100
wsp-to: initialise the elasticsearch backend
Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Ralph Boehme <slow at samba.org>
commit 95aa924c90a9c80d62c4dc7cbc57e1dcaf124056
Author: Noel Power <noel.power at suse.com>
Date: Wed Jul 27 15:03:00 2016 +0100
s3/utils: add wsp-to
'wsp-to' is a developer tool that can be used introspect the
expression tree of the binary CPMCREATEQUERY message and convert it to
the backend query that the WSP server/daemon will use.
Has:
'--input' switch to specify the binary message file
'--query' switch which allow an an advanced query
like string to be used (see wspsearch) to be converted into the
specific backend query. It does this in 2 stages,
first it parses the aqs string and creates a populated cpmcreatequery
message structure (as if it were pulled from the wire) and then passes
it on the the backend query conversion logic.
Note: This initial version only supports the 'none' backend (which as
it's name suggests) does nothing and so is non functional
Pair-Programmed-With: Ralph Boehme <slow at samba.org>
Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Ralph Boehme <slow at samba.org>
commit 32d5ccb07557fc97f7287136f2613b9f5d0e06ef
Author: Stefan Metzmacher <metze at samba.org>
Date: Wed Jun 10 11:39:02 2026 +0200
s3:rpc_server: add npsd_wsp_server.c
This implements the wsp service, which starts on demand
by samba-dcerpcd, if "wsp backend" is not "none".
A multi-worker model for the wsp server requires some co-ordination
an data to be shared across the workers. At this time we don't
support that, start simpler.. need to get it to work properly
in a single worker model first
Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Noel Power <noel.power at suse.com>
commit e5a2ff8f8852b49898fa256bd0bfb77629e50049
Author: Noel Power <noel.power at suse.com>
Date: Fri Dec 2 14:31:00 2022 +0000
s3/rpc_server/wsp: elasticsearch gss abstract interfaces
Pair-Programmed-With: Ralph Boehme <slow at samba.org>
Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Ralph Boehme <slow at samba.org>
commit edcb9680a2b79fff18aedb61e7a1bd871e3184b8
Author: Noel Power <noel.power at suse.com>
Date: Mon Aug 3 10:51:46 2026 +0200
wsp: add Elasticsearch conversion functions
Pair-Programmed-With: Ralph Boehme <slow at samba.org>
Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Ralph Boehme <slow at samba.org>
commit c9e691464b01c0d22784886539ecdc60f872e730
Author: Noel Power <noel.power at suse.com>
Date: Mon Nov 14 16:20:00 2022 +0000
s3/rpc_server/wsp: start of implementation of elasticsearch bld_mapper
Creates an initial implementation of the backend query_conv_ops
stucture with dummy implementation of bld_mapper/build_es_mapper function.
Currently non functional this commit just creates the stub function
and initialise one of the maps which will be used in populating the
'binding_result_mapper' structure
Pair-Programmed-With: Ralph Boehme <slow at samba.org>
Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Ralph Boehme <slow at samba.org>
commit 4b504283420c7efb5b21667b1d587c693fa87191
Author: Noel Power <noel.power at suse.com>
Date: Tue Jun 4 16:42:37 2024 +0100
codespell: Add ignore for cnat variable name which triggers codespell err.
Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Ralph Boehme <slow at samba.org>
commit ba71e3f4a966894a904acc75ff28f2600268e315
Author: Noel Power <noel.power at suse.com>
Date: Wed Aug 13 11:07:22 2025 +0100
docs-xml: Document elasticsearch:wsp use fscrawler folders
Setting to also fscrawler folder index to be included in
searches so folder results are also included
Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Ralph Boehme <slow at samba.org>
commit 5907a0d0e6a3091abc1c5eb9e8d2e92c6a2f3650
Author: Noel Power <noel.power at suse.com>
Date: Fri Jan 22 12:09:59 2021 +0000
docs-xml: Add elasticsearch smbconf constant for wsp backend
Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Ralph Boehme <slow at samba.org>
commit 4c0f8dedc6d14a310c90ba83ea5e2853982edf31
Author: Noel Power <noel.power at suse.com>
Date: Wed Jun 5 11:52:55 2024 +0100
s3: configure check for Elasticsearch WSP backend
Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Ralph Boehme <slow at samba.org>
commit dd9f836b8c02c76ff447195439f294599b0a2bac
Author: Noel Power <noel.power at suse.com>
Date: Sun Aug 9 16:18:13 2026 +0200
s3/rpc_server/wsp: add gss implementation
This allows a gss (represtenting the server search service) to be
instantiated. See MS-WSP 1.1 Glossary which describes the GSS as follows:
Generic Search Service (GSS): A service that implements the back-end
database functionality needed to provide results to search queries.
The Windows Search Service is a Generic Search Service instance. The
abstract interfaces that need to be implemented by a Generic Search
Service are described in Local Events. These interfaces are called by
the server in order to obtain appropriate responses to Windows Search
Protocol messages.
* Add message processing handler infrastruction
This adds the main entry point for the GSS which handles the incoming
messages. A map of message types (and associated handlers) is added
here (currently empty). Subsequent commits will fill out how processing and
sequencing of various messages are handled
* Handle CPMCONNECT message (MS-WSP 3.1.5.2.1)
* Enforce message sequencing rules
* Handle receiving CPMCREATEQUERY message (MS-WSP 3.1.5.2.2)
* Handle receiving CPMGETQUERYSTATUS message (MS-WSP 3.1.5.2.3)
* Handle receiving CPMGETQUERYSTATUSEX message (MS-WSP 3.1.5.2.4)
* Handle receiving CCPMRATIOFINISHED message (MS-WSP 3.1.5.2.5)
* Save query row related data:
Saves information about the row results associated with a query. Additionally
it adds a method for retrieving that information which will be used in future
commit where row data will be manipulated.
* Handle receiving CPMGETROWS message (MS-WSP 3.1.5.2.6)
* Handle receiving CPMFETCHVALUE message (MS-WSP 3.1.5.2.7)
* Handle receiving CPMSETBINDINGSIN message (MS-WSP 3.1.5.2.8)
* Handle receiving CPMGETNOTIFY message (MS-WSP 3.1.5.2.9)
Windows 10 returns E_NOTIMPL for CPMGETNOTIFY so in this case the gss just
ensures same behariour
* Handle receiving CPMGETAPPROXIMATEPOSITION message (MS-WSP 3.1.5.2.10)
* Handle receiving CPMCOMPAREBMK message (MS-WSP 3.1.5.2.11)
* Handle receiving CPMRESTARTPOSITIONIN message (MS-WSP 3.1.5.2.12)
This message has no real implementation, it just returns success.
* Handle receiving CPMFREECURSOR message (MS-WSP 3.1.5.2.13)
* Handle receiving CPMDISCONNECT message (MS-WSP 3.1.5.2.14)
* Handle receiving CPMFINDINDICES message (MS-WSP 3.1.5.2.15)
This message is not handled, it returns an UNHANDLEED_EXCEPTION error in the
response header.
* Handle receiving CPMGETROWSETNOTIFY message (MS-WSP 3.1.5.2.16)
* Handle receiving CPMGETSCOPESTATISTICS message (MS-WSP 3.1.5.2.17)
* Handle receiving CPMSETSCOPEPRIORITIZATION message (MS-WSP 3.1.5.2.18)
* Handle receiving CPMCISTATEOUT message (MS-WSP 3.1.5.1.1)
Adds some utility methods to access client handle and connection info
* Add wsp_gss_state_create method for creating gss state
* Add method for creating gss client state
* Add method to handle free of resources when client disconnects
Pair-Programmed-With: Ralph Boehme <slow at samba.org>
Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Ralph Boehme <slow at samba.org>
commit cbe7bce87206859e803d0ae111bee031e116c9c9
Author: Ralph Boehme <slow at samba.org>
Date: Wed Aug 12 17:51:26 2026 +0200
s4/scripting: Add HRESULT_FROM_NT() and HRESULT_TO_NT()
Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Noel Power <noel.power at suse.com>
commit 78058c34a20f5553e97ed0241c1a70d57e9a2bc9
Author: Ralph Boehme <slow at samba.org>
Date: Tue Aug 4 14:47:56 2026 +0200
lib/util: add tevent_req helpers to deal with hresult and errors
Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Noel Power <noel.power at suse.com>
commit d941492e72624d8f72417965fd9ea660feec91d1
Author: Noel Power <noel.power at suse.com>
Date: Wed Apr 26 16:04:40 2023 +0100
s3/rpc_server/wsp: store entryid -> docid fwd/reverse mappings
The idea is that a session specific memcache provides a first level
in memory cache that stores copies of the id -> path & path -> id
mappings assoicated with the index specific memcache.
When searching for a value we search in the memory cache first,
if it doesn't exist we search in the index cache, if it doesn't exist
in the index cache we create the mapping (and the reverse one),
if the value already exists in the index memcache we just populate the
memory cache foward and reverse mappings (and increment a refcount
associated with the records) The refcount is basically a count of
how many unique sessions actually have had a specific id -> path (and
reverse) item returned to it as part of a query result. Even if the
item is no longer used as long as the session (and the associated
client) exists we hold onto the cached id.
When all session caches that reference a id -> path (and reverse) are
gone (e.g. the refcount for that id -> path cache item goes to 0) then
that item is removed from the index cache it belongs to.
We create memory cache(s) for each SMB session we are handling to
minimise memory use, we free those memory caches when all connections
using the associated session are gone.
Pair-Programmed-With: Ralph Boehme <slow at samba.org>
Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Ralph Boehme <slow at samba.org>
commit 66ad9182358a58ee106b6f8f31856907e51db9e3
Author: Noel Power <noel.power at suse.com>
Date: Fri Aug 14 11:21:12 2020 +0100
lib/util: Add new cache types to cache entryid -> docid (and reverse)
We define 2 new talloc based types here to allow clients to cache in
memory partial data (just the key/values they are using) held in a global tdb.
This gives us is the ability to have many clients point to cached
values owned/stored in the global tdb, when all clients pointing to a
specific key/value are freed then the forward and reverse mappings
in the global tdb have their reference counts decremented and depending
on the value of the reference count maybe are also removed.
Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Ralph Boehme <slow at samba.org>
commit af7147bbd30cfce42cace50d7905f6530784f482
Author: Noel Power <noel.power at suse.com>
Date: Thu Jan 21 09:59:39 2021 +0000
docs-xml/smbdotconf: add some new smb.conf params for WSP
wsp backend (g)
wsp enable (s)
Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Ralph Boehme <slow at samba.org>
commit abd2d648670e08cd7bc9c1c1663b79569093f55a
Author: Noel Power <noel.power at suse.com>
Date: Fri Nov 11 14:48:03 2022 +0000
s3/rpc_server: Add a WSP backend registry
The backend registry allows both a concrete implementation of
the abstract interface and additionally functions to create and
convert a query from the WSP to backend domain to be registered.
Pair-Programmed-With: Ralph Boehme <slow at samba.org>
Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Ralph Boehme <slow at samba.org>
commit a5aacf1ac16db7ae4e2049765241213d8a87f44c
Author: Noel Power <noel.power at suse.com>
Date: Fri Nov 11 15:19:26 2022 +0000
s3/rpc_server: abstract interface for WSP operations
Creates an abstract interface based on the information and interfaces
described in MS-WSP 3.1.7
Most abstract interface functions have been adapted to _send/_recv
variants to reflect potential async behaviour of the indexer
implementations to be used as WSP backends to provide the wsp service.
Other simpler functions are assumed for simplicity to be synchronous.
Each interface function is additionally passed a struct wspd_client_state
instance (which contains both global, client and backend specific data)
the global state (struct wsp_abstract_state) wrapped in
struct wspd_client_state is created from the abstract init method
(not defined in the MS-WSP spec) Note: The init method called when the
backend is created and allows backend specific data to be stored.
Pair-Programmed-With: Ralph Boehme <slow at samba.org>
Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Ralph Boehme <slow at samba.org>
commit f451a1e2a96e457ba1d8c1ccfcf91b2804366884
Author: Ralph Boehme <slow at samba.org>
Date: Wed Aug 19 18:25:30 2026 +0200
mdssvc: use create_conn_struct() and conn_wrap_chdir()
Ensures two things:
1. For every VFS call make sure we're in the root directory of the conn.
2. When done with doing VFS operations, go back to "/".
Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Noel Power <noel.power at suse.com>
commit 4ba9118d49f5e9befed25407e91f526128620809
Author: Ralph Boehme <slow at samba.org>
Date: Wed Aug 19 18:22:18 2026 +0200
smbd: add conn_wrap_chdir()
Undoes the chdir() to the conn root directory via a destructor which chdir()s
back to "/".
Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Noel Power <noel.power at suse.com>
commit 4395cde601552ecc4c7ed47acd84edb3f7b1ea74
Author: Ralph Boehme <slow at samba.org>
Date: Wed Aug 19 18:04:06 2026 +0200
smbd: add create_conn_struct()
Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Noel Power <noel.power at suse.com>
commit a6b8e27ff52ff3e99dcc0625ad745d275ecc8d0f
Author: Ralph Boehme <slow at samba.org>
Date: Wed Aug 19 17:58:27 2026 +0200
smbd: split out create_conn_struct_chdir_internal()
Allows reusing the logic for a version of create_conn_struct_chdir() that
doesn't chdir().
Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Noel Power <noel.power at suse.com>
commit b2ca4c3f3251b24fcd113f0a0803b3894c6956ec
Author: Noel Power <noel.power at suse.com>
Date: Thu Apr 2 17:08:18 2020 +0100
libcli/http: Allow cli to sent http GET
Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Ralph Boehme <slow at samba.org>
commit 6c5b5afd8b19897a21041e0477382e41d01c4e4f
Author: Noel Power <noel.power at suse.com>
Date: Wed Jun 26 19:18:23 2024 +0100
librpc/idl: ADD DB_S_BLOCKLIMITEDROWS to idl
Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Ralph Boehme <slow at samba.org>
commit e6eac0e6d9707819f96e033297580318462c1e2d
Author: Noel Power <noel.power at suse.com>
Date: Sat Feb 3 13:38:10 2024 +0000
librpc/idl: Add Priority enum
Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Ralph Boehme <slow at samba.org>
commit dbd8b206b8e04a5df790c0691242c299b6231cf8
Author: Noel Power <noel.power at suse.com>
Date: Fri Oct 6 11:36:41 2023 +0100
librpc: Add backend constants to idl
Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Ralph Boehme <slow at samba.org>
commit b004b59210a7396558effb5422402a1329685924
Author: Noel Power <noel.power at suse.com>
Date: Sat Jun 29 15:23:52 2024 +0100
s3/rpcs_server/mdssvc: Use new common escape_str routines
Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Ralph Boehme <slow at samba.org>
commit 425e0971e48539095b1593f73c45e78aa3d0ee78
Author: Noel Power <noel.power at suse.com>
Date: Thu Jun 6 15:14:46 2024 +0100
s3/lib: add elastic_util elastic utility functions
Add code (that could be shared between wsp and mdssvc) for
escaping strings to be used with elasticsearch/opensearch
A later commit will remove the duplicate code in mdssvc.
Pair-Programmed-With: Ralph Boehme <slow at samba.org>
Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Ralph Boehme <slow at samba.org>
-----------------------------------------------------------------------
Summary of changes:
.codespellignore | 1 +
.../misc/elasticsearchwspusefscrawlerfolders.xml | 20 +
docs-xml/smbdotconf/misc/wsp_backend.xml | 22 +
docs-xml/smbdotconf/misc/wspenable.xml | 43 +
lib/param/loadparm.h | 5 +
lib/param/param_table.c | 4 +
lib/util/memcache.c | 2 +
lib/util/memcache.h | 2 +
lib/util/tevent_hresult.c | 81 +
.../test_headers.c => lib/util/tevent_hresult.h | 42 +-
lib/util/wscript_build | 3 +-
libcli/http/http.c | 3 +
librpc/idl/wsp.idl | 1 +
librpc/idl/wsp_data.idl | 19 +
python/samba/tests/blackbox/wsp_query_generator.py | 132 +
python/samba/tests/blackbox/wsp_test.py | 266 ++
python/samba/tests/wsp_test.py | 459 ++
selftest/selftest.pl | 1 +
selftest/target/Samba3.pm | 7 +
.../mdssvc/es_mapping.c => lib/elastic_util.c} | 149 +-
.../wsp/wsp_helper.h => source3/lib/elastic_util.h | 16 +-
source3/param/loadparm.c | 1 +
source3/rpc_client/py_wsp.c | 293 ++
source3/rpc_server/mdssvc/es_mapping.c | 129 +-
source3/rpc_server/mdssvc/es_mapping.h | 3 -
source3/rpc_server/mdssvc/es_parser.y | 8 +-
source3/rpc_server/mdssvc/mdssvc.c | 57 +-
source3/rpc_server/npsd_wsp_server.c | 468 ++
source3/rpc_server/wscript_build | 34 +-
source3/rpc_server/wsp/elasticsearch_mappings.json | 233 +
source3/rpc_server/wsp/wsp_abs_if.h | 376 ++
source3/rpc_server/wsp/wsp_backend.c | 152 +
source3/rpc_server/wsp/wsp_backend.h | 203 +
source3/rpc_server/wsp/wsp_es_abs_if.c | 4601 ++++++++++++++++++++
.../rpc_server/wsp/wsp_es_abs_if.h | 11 +-
source3/rpc_server/wsp/wsp_es_conv.c | 3360 ++++++++++++++
source3/rpc_server/wsp/wsp_es_conv.h | 104 +
source3/rpc_server/wsp/wsp_gss.c | 4185 ++++++++++++++++++
source3/rpc_server/wsp/wsp_gss.h | 55 +
source3/rpc_server/wsp/wspd_db.c | 480 ++
source3/rpc_server/wsp/wspd_db.h | 60 +
source3/smbd/msdfs.c | 105 +-
source3/smbd/proto.h | 7 +
source3/utils/wsp-to.c | 432 ++
source3/wscript | 11 +
source3/wscript_build | 31 +-
source4/librpc/wscript_build | 14 +
source4/scripting/bin/gen_hresult.py | 5 +
source4/selftest/tests.py | 6 +
49 files changed, 16374 insertions(+), 328 deletions(-)
create mode 100644 docs-xml/smbdotconf/misc/elasticsearchwspusefscrawlerfolders.xml
create mode 100644 docs-xml/smbdotconf/misc/wsp_backend.xml
create mode 100644 docs-xml/smbdotconf/misc/wspenable.xml
create mode 100644 lib/util/tevent_hresult.c
copy testsuite/headers/test_headers.c => lib/util/tevent_hresult.h (58%)
create mode 100644 python/samba/tests/blackbox/wsp_query_generator.py
create mode 100644 python/samba/tests/blackbox/wsp_test.py
create mode 100644 python/samba/tests/wsp_test.py
copy source3/{rpc_server/mdssvc/es_mapping.c => lib/elastic_util.c} (51%)
copy librpc/wsp/wsp_helper.h => source3/lib/elastic_util.h (73%)
create mode 100644 source3/rpc_client/py_wsp.c
create mode 100644 source3/rpc_server/npsd_wsp_server.c
create mode 100644 source3/rpc_server/wsp/elasticsearch_mappings.json
create mode 100644 source3/rpc_server/wsp/wsp_abs_if.h
create mode 100644 source3/rpc_server/wsp/wsp_backend.c
create mode 100644 source3/rpc_server/wsp/wsp_backend.h
create mode 100644 source3/rpc_server/wsp/wsp_es_abs_if.c
copy librpc/wsp/wsp_helper.h => source3/rpc_server/wsp/wsp_es_abs_if.h (79%)
create mode 100644 source3/rpc_server/wsp/wsp_es_conv.c
create mode 100644 source3/rpc_server/wsp/wsp_es_conv.h
create mode 100644 source3/rpc_server/wsp/wsp_gss.c
create mode 100644 source3/rpc_server/wsp/wsp_gss.h
create mode 100644 source3/rpc_server/wsp/wspd_db.c
create mode 100644 source3/rpc_server/wsp/wspd_db.h
create mode 100644 source3/utils/wsp-to.c
Changeset truncated at 500 lines:
diff --git a/.codespellignore b/.codespellignore
index 08ae8c6cfca..91a89457c19 100644
--- a/.codespellignore
+++ b/.codespellignore
@@ -10,6 +10,7 @@ bre
bu
childres
clen
+cnat
crate
creat
connectin
diff --git a/docs-xml/smbdotconf/misc/elasticsearchwspusefscrawlerfolders.xml b/docs-xml/smbdotconf/misc/elasticsearchwspusefscrawlerfolders.xml
new file mode 100644
index 00000000000..36785b9da7e
--- /dev/null
+++ b/docs-xml/smbdotconf/misc/elasticsearchwspusefscrawlerfolders.xml
@@ -0,0 +1,20 @@
+<samba:parameter name="elasticsearch:wsp use fscrawler folders"
+ context="S"
+ type="string"
+ xmlns:samba="http://www.samba.org/samba/DTD/samba-doc">
+ <description>
+ <para>
+ When enabled the wsp server will additionally query the
+ associated fscrawler folder index (assumed to be named
+ 'indexname_folder') where indexname
+ is the index name defined by
+ <smbconfoption name="elasticsearch:index"/>
+ Searching the folder index allow folders to be found,
+ without this setting enabled searching specifically for
+ 'Folder' kind or finding results that are directories is
+ not possible.
+ </para>
+ </description>
+ <value type="default">false</value>
+
+</samba:parameter>
diff --git a/docs-xml/smbdotconf/misc/wsp_backend.xml b/docs-xml/smbdotconf/misc/wsp_backend.xml
new file mode 100644
index 00000000000..47089c46cec
--- /dev/null
+++ b/docs-xml/smbdotconf/misc/wsp_backend.xml
@@ -0,0 +1,22 @@
+<samba:parameter name="wsp backend"
+ context="G"
+ type="enum"
+ enumlist="enum_wsp_backend"
+ xmlns:samba="http://www.samba.org/samba/DTD/samba-doc">
+<description>
+ <para>
+ WSP search backend. Available backends:
+
+ <itemizedlist>
+ <listitem><para><constant>none</constant> -
+ </para></listitem>
+ <listitem><para><constant>elasticsearch</constant> -
+ a backend that uses JSON and REST over HTTP(s) to query an
+ Elasticsearch or Opensearch server.
+ </para></listitem>
+
+ </itemizedlist>
+ </para>
+</description>
+<value type="default">none</value>
+</samba:parameter>
diff --git a/docs-xml/smbdotconf/misc/wspenable.xml b/docs-xml/smbdotconf/misc/wspenable.xml
new file mode 100644
index 00000000000..f95665596de
--- /dev/null
+++ b/docs-xml/smbdotconf/misc/wspenable.xml
@@ -0,0 +1,43 @@
+<samba:parameter name="wsp"
+ context="S"
+ type="boolean"
+ xmlns:samba="http://www.samba.org/samba/DTD/samba-doc">
+<description>
+ <para>
+ This parameter controls whether Samba allows WSP
+ queries on a share. For controlling indexing of filesystems
+ you may also have to use the associated indexer's own
+ configuration system or tools.
+ </para>
+
+ <para>
+ WSP has several prerequisites:
+ </para>
+
+ <itemizedlist>
+ <listitem><para>
+ Samba must be configured and built with WSP support.
+ </para></listitem>
+
+ <listitem><para>
+ The <emphasis>wsp</emphasis> service must be
+ enabled, see below.
+ </para></listitem>
+
+ <listitem><para> The associated supported <emphasis>wsp backend
+ </emphasis> must be setup and the share content
+ indexed.</para></listitem>
+ </itemizedlist>
+
+ <para>For a detailed set of instructions please see <ulink
+ url="https://wiki.samba.org/index.php/WSP">https://wiki.samba.org/index.php/WSP</ulink>.
+ </para>
+
+ <para>
+ The WSP service is started automatically if global 'wsp backend'
+ is set to a supported backend (other than the default 'none')
+ </para>
+
+</description>
+<value type="default">no</value>
+</samba:parameter>
diff --git a/lib/param/loadparm.h b/lib/param/loadparm.h
index a979a8ac8f6..981555c0d4b 100644
--- a/lib/param/loadparm.h
+++ b/lib/param/loadparm.h
@@ -268,6 +268,11 @@ enum acl_claims_evaluation {
ACL_CLAIMS_EVALUATION_AD_DC_ONLY,
ACL_CLAIMS_EVALUATION_NEVER
};
+/* WSP backend options */
+enum wsp_backend_options {
+ WSP_BACKEND_NONE,
+ WSP_BACKEND_ELASTIC,
+};
/* Controlling the protocol for netlogon ping */
enum client_netlogon_ping_protocol {
diff --git a/lib/param/param_table.c b/lib/param/param_table.c
index d23046b6365..777b78241df 100644
--- a/lib/param/param_table.c
+++ b/lib/param/param_table.c
@@ -469,6 +469,10 @@ static const struct enum_list enum_ad_functional_level[] = {
static const struct enum_list enum_acl_claims_evaluation[] = {
{ACL_CLAIMS_EVALUATION_AD_DC_ONLY, "AD DC only"},
{ACL_CLAIMS_EVALUATION_NEVER, "never"},
+};
+static const struct enum_list enum_wsp_backend[] = {
+ {WSP_BACKEND_NONE, "none"},
+ {WSP_BACKEND_ELASTIC, "elasticsearch"},
{-1, NULL}
};
diff --git a/lib/util/memcache.c b/lib/util/memcache.c
index a9f814ff65c..e725c3206d1 100644
--- a/lib/util/memcache.c
+++ b/lib/util/memcache.c
@@ -60,6 +60,8 @@ static bool memcache_is_talloc(enum memcache_number n)
case SINGLETON_CACHE_TALLOC:
case SHARE_MODE_LOCK_CACHE:
case VIRUSFILTER_SCAN_RESULTS_CACHE_TALLOC:
+ case WSP_ENTRYID_UID_TALLOC:
+ case WSP_UID_ENTRYID_TALLOC:
result = true;
break;
default:
diff --git a/lib/util/memcache.h b/lib/util/memcache.h
index fca9d3b4440..f603c04d96b 100644
--- a/lib/util/memcache.h
+++ b/lib/util/memcache.h
@@ -46,6 +46,8 @@ enum memcache_number {
SHARE_MODE_LOCK_CACHE, /* talloc */
VIRUSFILTER_SCAN_RESULTS_CACHE_TALLOC, /* talloc */
DFREE_CACHE,
+ WSP_ENTRYID_UID_TALLOC, /* talloc */
+ WSP_UID_ENTRYID_TALLOC, /* talloc */
};
/*
diff --git a/lib/util/tevent_hresult.c b/lib/util/tevent_hresult.c
new file mode 100644
index 00000000000..1025f0e0cba
--- /dev/null
+++ b/lib/util/tevent_hresult.c
@@ -0,0 +1,81 @@
+/*
+ Unix SMB/CIFS implementation.
+ Wrap HRESULT errors around tevent_req
+ Copyright (C) Ralph Boehme 2026
+
+ This program is free software; you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation; either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>.
+*/
+
+#include "lib/replace/replace.h"
+#include "tevent_hresult.h"
+#include "libcli/util/error.h"
+#include "libcli/util/ntstatus_gen.h"
+
+#define TEVENT_HERROR_MAGIC (UINT64_C(0x917b5ace) << 32)
+#define TEVENT_HERROR_MAGIC_MASK (UINT64_C(0xffffffff) << 32)
+
+bool _tevent_req_herror(struct tevent_req *req,
+ HRESULT hres,
+ const char *location)
+{
+ uint64_t err;
+
+ if (HRES_IS_OK(hres)) {
+ return false;
+ }
+
+ err = TEVENT_HERROR_MAGIC | HRES_ERROR_V(hres);
+
+ return _tevent_req_error(req, err, location);
+}
+
+bool tevent_req_is_herror(struct tevent_req *req, HRESULT *hres)
+{
+ enum tevent_req_state state;
+ uint64_t err;
+
+ if (!tevent_req_is_error(req, &state, &err)) {
+ return false;
+ }
+ switch (state) {
+ case TEVENT_REQ_TIMED_OUT:
+ *hres = HRES_RPC_E_TIMEOUT;
+ break;
+ case TEVENT_REQ_NO_MEMORY:
+ *hres = HRES_E_OUTOFMEMORY;
+ break;
+ case TEVENT_REQ_USER_ERROR:
+ if ((err & TEVENT_HERROR_MAGIC_MASK) != TEVENT_HERROR_MAGIC) {
+ abort();
+ }
+ *hres = HRES_ERROR(err & ~(TEVENT_HERROR_MAGIC_MASK));
+ break;
+ default:
+ *hres = HRES_E_FAIL;
+ break;
+ }
+ return true;
+}
+
+HRESULT tevent_req_simple_recv_hresult(struct tevent_req *req)
+{
+ HRESULT hres;
+
+ if (tevent_req_is_herror(req, &hres)) {
+ tevent_req_received(req);
+ return hres;
+ }
+ tevent_req_received(req);
+ return HRES_ERROR(0);
+}
diff --git a/testsuite/headers/test_headers.c b/lib/util/tevent_hresult.h
similarity index 58%
copy from testsuite/headers/test_headers.c
copy to lib/util/tevent_hresult.h
index 4e63e998995..428c2559b76 100644
--- a/testsuite/headers/test_headers.c
+++ b/lib/util/tevent_hresult.h
@@ -1,7 +1,7 @@
/*
Unix SMB/CIFS implementation.
-
- Copyright (C) Andrew Tridgell 2011
+ Wrap HRESULT errors around tevent_req
+ Copyright (C) Ralph Boehme 2026
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
@@ -17,29 +17,21 @@
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
-/*
- this tests that all of our public headers will build
- */
-
-#define _GNU_SOURCE 1
+#ifndef TEVENT_HRESULT_H
+#define TEVENT_HRESULT_H
-#ifdef HAVE_PYTHON_H
-# include <Python.h>
-#endif
-#include <stdio.h>
-#include <unistd.h>
-#include <stdlib.h>
#include <stdint.h>
#include <stdbool.h>
-
-/* pre-include some of the public headers to avoid ordering issues */
-#include "core/ntstatus.h"
-
-/* include all our public headers */
-#include "test_headers.h"
-
-int main(void)
-{
- printf("All OK\n");
- return 0;
-}
+#include <tevent.h>
+#include "libcli/util/ntstatus.h"
+#include "libcli/util/hresult.h"
+
+bool _tevent_req_herror(struct tevent_req *req,
+ HRESULT hres,
+ const char *location);
+#define tevent_req_herror(req, hres) \
+ _tevent_req_herror(req, hres, __location__)
+
+bool tevent_req_is_herror(struct tevent_req *req, HRESULT *hres);
+HRESULT tevent_req_simple_recv_hresult(struct tevent_req *req);
+#endif
diff --git a/lib/util/wscript_build b/lib/util/wscript_build
index 2acf6ca2409..de1cf879576 100644
--- a/lib/util/wscript_build
+++ b/lib/util/wscript_build
@@ -283,11 +283,12 @@ else:
tevent_unix.c
tevent_ntstatus.c
tevent_werror.c
+ tevent_hresult.c
tevent_req_profile.c
''',
local_include=False,
public_deps='tevent time-basic samba-errors',
- public_headers='tevent_ntstatus.h tevent_unix.h tevent_werror.h',
+ public_headers='tevent_ntstatus.h tevent_unix.h tevent_werror.h tevent_hresult.h',
header_path=[ ('*', 'util') ],
pc_files=[],
vnum='0.0.1'
diff --git a/libcli/http/http.c b/libcli/http/http.c
index 6f22214f706..0b303e61fed 100644
--- a/libcli/http/http.c
+++ b/libcli/http/http.c
@@ -926,6 +926,9 @@ static const char *http_method_str(enum http_cmd_type type)
const char *method;
switch (type) {
+ case HTTP_REQ_GET:
+ method = "GET";
+ break;
case HTTP_REQ_POST:
method = "POST";
break;
diff --git a/librpc/idl/wsp.idl b/librpc/idl/wsp.idl
index fceaf0a85f2..eeabd945ea7 100644
--- a/librpc/idl/wsp.idl
+++ b/librpc/idl/wsp.idl
@@ -3,6 +3,7 @@ import "wsp_data.idl";
import "misc.idl";
[
+ uuid("3e0ad41d-0d34-4fd2-8a1c-c767d013a1be"),
version(1.0),
endpoint("ncacn_np:[\\pipe\\MsFteWds]"),
helpstring("Windows Search WSP Protocol"),
diff --git a/librpc/idl/wsp_data.idl b/librpc/idl/wsp_data.idl
index fde754aef81..8d5c88a5955 100644
--- a/librpc/idl/wsp_data.idl
+++ b/librpc/idl/wsp_data.idl
@@ -16,6 +16,24 @@ interface constants
* see MS-WSP 2.2.3.11
*/
const uint32_t MAX_ROW_BUFF_SIZE = 0x0004000;
+ /*
+ * Priority - see CPMSetScopePrioritizationIn MS-WSP 2.2.3.31
+ */
+ typedef enum {
+ PRIORITY_LEVEL_FOREGROUND = 0,
+ PRIORITY_LEVEL_HIGH = 1,
+ PRIORITY_LEVEL_LOW = 2,
+ PRIORITY_LEVEL_DEFAULT = 3
+ } priority;
+
+ typedef enum {
+ BACKEND_NULL = 0,
+ BACKEND_STRING = 1,
+ BACKEND_INTEGER = 2,
+ BACKEND_BOOLEAN = 3,
+ BACKEND_DOUBLE = 4,
+ BACKEND_DEFERRED = 5
+ } backend_type;
/* values for guidPropertySet */
const char* DBPROPSET_FSCIFRMWRK_EXT = "A9BD1526-6A80-11D0-8C9D-0020AF1D740E";
@@ -303,6 +321,7 @@ interface constants
const uint32 STORESTATUSNULL = 0x00000002;
const uint32 DB_S_ENDOFROWSET = 0x00040EC6;
+ const uint32 DB_S_BLOCKLIMITEDROWS = 0x00040ECD;
const uint32 XOR_CONST = 0x59533959;
const uint32 E_UNEXPECTED = 0x8000FFFF;
diff --git a/python/samba/tests/blackbox/wsp_query_generator.py b/python/samba/tests/blackbox/wsp_query_generator.py
new file mode 100644
index 00000000000..0610534e0bd
--- /dev/null
+++ b/python/samba/tests/blackbox/wsp_query_generator.py
@@ -0,0 +1,132 @@
+# Blackbox tests for WSP
+#
+# Copyright (C) Noel Power noel.power at suse.com
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+#
+from samba.tests import BlackboxTestCase, BlackboxProcessError
+import os
+import json
+
+class WspToTest(BlackboxTestCase):
+
+ def setUp(self):
+ super(BlackboxTestCase, self).setUp()
+ self.server = os.environ["SERVER"]
+ self.dflt_cmd = ["wsp-to", "elastic"]
+ # parse the output from end of wsp-to
+ # e.g.
+ # Col[0] System.FileName is mapped/converted from elastic col[0]
+ # file.filename to a simple WSP property to elastic property map
+ def parse_cols(self, col_lines):
+ cols = {}
+ for col in col_lines.split('\n'):
+ if not len(col):
+ continue
+ key = col.split()[1]
+ value = col.split()[-1]
+ cols[key] = value
+ return cols
+
+ def normalize_query(self, query_in):
+ query_out = query_in
+ temp = query_in[2:-2]
+ jquery = json.loads(temp).get("query")
+ return json.dumps(jquery)
+
+ def check_wsp_to_output(self, cmd_args):
+ result = self.check_output(cmd_args)
+ # parse query
+ query_including_cols = result.decode().split("query is:")[1]
+ query = query_including_cols.split("selected columns:")[0]
+ query = self.normalize_query(query)
+ cols_section = query_including_cols.split("selected columns:")[1]
+ cols = self.parse_cols(cols_section)
+ return query, cols;
+
+ def test_simple_folder_query(self):
+ query_args = ["--query", "SELECT System.FileName WHERE System.Kind:Folder"]
+ expected_query = '{"query_string": {"query": "file.content_type:(text\\\\/directory)"}}'
+ expected_cols = {'System.FileName': 'file.filename'}
+ try:
+ cmd = self.dflt_cmd
+ cmd.extend(query_args)
+ query_result, cols = self.check_wsp_to_output(cmd)
+ except BlackboxProcessError as e:
+ self.fail(str(e))
+ # test the query part
+ self.assertEqual(query_result, expected_query)
+ # test the requested columns
+ self.assertEqual(expected_cols, cols)
+
+ def test_simple_picture_query(self):
+ query_args = ["--query", "SELECT System.FileName WHERE System.Kind:Picture"]
+ expected_query = '{"query_string": {"query": "file.content_type:(image\\\\/*)"}}'
+ expected_cols = {'System.FileName': 'file.filename'}
+ try:
+ cmd = self.dflt_cmd
+ cmd.extend(query_args)
+ query_result, cols = self.check_wsp_to_output(cmd)
+ except BlackboxProcessError as e:
+ self.fail(str(e))
+ # test the query part
+ self.assertEqual(query_result, expected_query)
+ # test the requested columns
+ self.assertEqual(expected_cols, cols)
+
+ def test_simple_music_query(self):
+ query_args = ["--query", "SELECT System.FileName WHERE System.Kind:Music"]
+ expected_query = '{"query_string": {"query": "file.content_type:(audio\\\\/*)"}}'
+ expected_cols = {'System.FileName': 'file.filename'}
+ try:
--
Samba Shared Repository
More information about the samba-cvs
mailing list