[SCM] Samba Shared Repository - branch master updated
Ralph Böhme
slow at samba.org
Wed Oct 9 15:53:03 UTC 2019
The branch, master has been updated
via c9b6298249c mdssvc.idl: pass policy_handle as pointer
via 63cbeaf0e74 s3:mdssvc: fix service startup in deamon mode
via 78a2d955113 s3:mdssvc: use a helper variable for the service type
via 91f397bec0d mdssd: fix a debug message
via bf02a935083 s3: rpc_server: enable mdssvc by default
via fa4c3fd9341 selftest: add mdfind blackbox test
via c85cd5c4311 s3:utils: add mdfind
via da7dec0a50f selftest: add end-to-end tests for mdssvc with a fake HTTP server
via f0e8dd1a086 selftest: split fileserver testenv into simpleserver and fileserver
via f479e010961 s3:mdssvc: add unit tests for the Spotlight to Elasticsearch parser
via f5510d7db38 s3:mdssvc: add Elasticsearch backend
via c338bdf5a4a s3:mdssvc: add missing mds_ctx deallocation
via eb740b571d1 s3:mdssvc: fix a long line
via ba899694e45 s3:mdssvc: modernize a few DEBUG macros
via 08cb82d26f8 s3:mdssvc: remove unused snum from struct sl_query
via afc90321ecd build: add SAMBA_DATADIR as "samba" subdirectory of DATADIR
via 213ca6e4cc9 s3: add mdscli Python bindings
via b27d9afa294 s3:rpc_client: add a mdssvc client library
via 873aa1e6c84 librpc: add Python mdssvc bindings
from 459b43e5776 testprogs: Add test for 'net ads join createcomputer='
https://git.samba.org/?p=samba.git;a=shortlog;h=master
- Log -----------------------------------------------------------------
commit c9b6298249cd2f670e6953c9c3ef2e977f8bc565
Author: Ralph Boehme <slow at samba.org>
Date: Fri Aug 23 18:43:02 2019 +0200
mdssvc.idl: pass policy_handle as pointer
No change in behaviour, this just changes all functions to take the
policy_handle argument as pointer instead of passing it by value.
This is how all other IDLs pass it.
Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Noel Power <noel.power at suse.com>
Autobuild-User(master): Ralph Böhme <slow at samba.org>
Autobuild-Date(master): Wed Oct 9 15:52:55 UTC 2019 on sn-devel-184
commit 63cbeaf0e7472a7321f27327091217c909864100
Author: Ralph Boehme <slow at samba.org>
Date: Tue Aug 20 17:00:49 2019 +0200
s3:mdssvc: fix service startup in deamon mode
Changes:
* Don't initialize the RPC service by calling setup_rpc_module() in the parent
mdssd. This is not needed in the parent, only in the worker childs.
* In the worker childs call setup_rpc_module() instead of init_rpc_module()
which ensures rpc_mdssvc_init() is called with the mdssvc callback which is
needed to initialize mdssvc via mdssvc_init_cb() -> init_service_mdssvc()
* Finally rpc_setup_mdssvc() is adjusted to be a noop if mdssvc is configured to
as external and when called by the main parent smbd via dcesrv_ep_setup() ->
setup_rpc_modules()
I've manually tested all 4 combinations of external=yes|no X module=yes|no with
the new mdfind command.
Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Noel Power <noel.power at suse.com>
commit 78a2d95511357a5ea34473e6a6d98515b4344fb5
Author: Ralph Boehme <slow at samba.org>
Date: Tue Aug 20 16:50:52 2019 +0200
s3:mdssvc: use a helper variable for the service type
No change in behaviour. Simplifies a subsequent logical change.
Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Noel Power <noel.power at suse.com>
commit 91f397bec0dbc24b60f74e80ee413642d955d7bc
Author: Ralph Boehme <slow at samba.org>
Date: Tue Aug 20 17:00:12 2019 +0200
mdssd: fix a debug message
Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Noel Power <noel.power at suse.com>
commit bf02a935083c858490b9aecd8949f8a57918615a
Author: Ralph Boehme <slow at samba.org>
Date: Mon Aug 19 15:50:36 2019 +0200
s3: rpc_server: enable mdssvc by default
Now that mdssvc is built by default and also tested in CI, enable it by default,
running as embedded service.
Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Noel Power <noel.power at suse.com>
commit fa4c3fd9341412a9fc4a53923bf52f7433c08079
Author: Ralph Boehme <slow at samba.org>
Date: Sat Aug 17 11:36:55 2019 +0200
selftest: add mdfind blackbox test
Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Noel Power <noel.power at suse.com>
commit c85cd5c4311512fff18b04267408eb60d7c8ecdf
Author: Ralph Boehme <slow at samba.org>
Date: Thu May 2 21:33:46 2019 +0200
s3:utils: add mdfind
A small command line tool to run macOS Spotlight searches against an SMB server
that runs the Spotlight mdssvc RPC service, including macOS and Samba.
Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Noel Power <noel.power at suse.com>
commit da7dec0a50fa5d9f6b92403d8703cd6b7a42159c
Author: Ralph Boehme <slow at samba.org>
Date: Sun Jul 28 15:25:54 2019 +0200
selftest: add end-to-end tests for mdssvc with a fake HTTP server
Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Noel Power <noel.power at suse.com>
commit f0e8dd1a08698884209873bb84002d7b34db016c
Author: Ralph Boehme <slow at samba.org>
Date: Tue Jul 30 12:00:18 2019 +0200
selftest: split fileserver testenv into simpleserver and fileserver
The simpleserver testenv continues to be built with minimal
dependencies. fileserver otoh will be built with bells and whistles including
JSON which I need for using the env as target for Spotlight tests.
Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Noel Power <noel.power at suse.com>
commit f479e010961e652230c7e33876058c8fb82e9019
Author: Ralph Boehme <slow at samba.org>
Date: Tue Jul 30 12:47:03 2019 +0200
s3:mdssvc: add unit tests for the Spotlight to Elasticsearch parser
Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Noel Power <noel.power at suse.com>
commit f5510d7db3816c6a18dc333e1ae7f505fca69815
Author: Ralph Boehme <slow at samba.org>
Date: Mon Aug 5 16:25:01 2019 +0200
s3:mdssvc: add Elasticsearch backend
Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Noel Power <noel.power at suse.com>
commit c338bdf5a4aef16cb3048b18c77a0095f3fc3e53
Author: Ralph Boehme <slow at samba.org>
Date: Tue Aug 13 14:16:07 2019 +0200
s3:mdssvc: add missing mds_ctx deallocation
The mds_ctx object was created in _mdssvc_open() as a talloc child of the pipe
which means as long as the pipe is connected it's not freed.
To ensure we do proper rundown of all resources including backend connections
and pending queries, we must free the mds_ctx object.
Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Noel Power <noel.power at suse.com>
commit eb740b571d1ed5d55619c059160f236126351bd5
Author: Ralph Boehme <slow at samba.org>
Date: Tue Aug 13 14:13:20 2019 +0200
s3:mdssvc: fix a long line
Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Noel Power <noel.power at suse.com>
commit ba899694e45aa2e691cf44098846fc0ddf3e79fd
Author: Ralph Boehme <slow at samba.org>
Date: Tue Aug 13 14:11:23 2019 +0200
s3:mdssvc: modernize a few DEBUG macros
Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Noel Power <noel.power at suse.com>
commit 08cb82d26f8f1a0dc66fadfcdee4f487ea06e010
Author: Ralph Boehme <slow at samba.org>
Date: Tue Aug 13 12:55:33 2019 +0200
s3:mdssvc: remove unused snum from struct sl_query
Looks like this was never used, it's also available via mds_ctx->snum.
Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Noel Power <noel.power at suse.com>
commit afc90321ecd47ec1a8b6196c9b4aa15c2c654d41
Author: Ralph Boehme <slow at samba.org>
Date: Fri May 10 18:49:20 2019 +0200
build: add SAMBA_DATADIR as "samba" subdirectory of DATADIR
DATADIR should have been set to this path from the beginning, too late to change
that now as ut's used as parent for two other directory varialbles: SETUPDIR and
CODEPAGEDIR.
From <https://www.gnu.org/prep/standards/html_node/Directory-Variables.html>:
datadir
The directory for installing idiosyncratic read-only
architecture-independent data files for this program. This is usually the
same place as ‘datarootdir’, but we use the two separate variables so that
you can move these program-specific files without altering the location for
Info files, man pages, etc.
This should normally be /usr/local/share, but write it as
$(datarootdir). (If you are using Autoconf, write it as ‘@datadir@’.)
The definition of ‘datadir’ is the same for all packages, so you should
install your data in a subdirectory thereof. Most packages install their
data under $(datadir)/package-name/.
Currently Samba doesn't install any application specific data files, but I'm
going to do just that in a subsequent commit.
Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Noel Power <noel.power at suse.com>
commit 213ca6e4cc981b4a9550042ce927c677883810be
Author: Ralph Boehme <slow at samba.org>
Date: Sun Jul 28 15:08:29 2019 +0200
s3: add mdscli Python bindings
Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Noel Power <noel.power at suse.com>
commit b27d9afa29435b7af6335f228b8a315701b2607a
Author: Ralph Boehme <slow at samba.org>
Date: Thu May 2 07:45:39 2019 +0200
s3:rpc_client: add a mdssvc client library
Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Noel Power <noel.power at suse.com>
commit 873aa1e6c8495c04eb21cfc38fc5a4ad401f80fd
Author: Ralph Boehme <slow at samba.org>
Date: Sun Jul 28 15:05:13 2019 +0200
librpc: add Python mdssvc bindings
Not used for now, but soon.
Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Noel Power <noel.power at suse.com>
-----------------------------------------------------------------------
Summary of changes:
.gitlab-ci.yml | 3 +
buildtools/wafsamba/samba_patterns.py | 2 +
docs-xml/manpages/mdfind.1.xml | 148 +++
docs-xml/smbdotconf/generate-file-list.sh | 1 +
docs-xml/smbdotconf/misc/elasticsearchaddress.xml | 14 +
docs-xml/smbdotconf/misc/elasticsearchindex.xml | 16 +
docs-xml/smbdotconf/misc/elasticsearchmappings.xml | 14 +
.../smbdotconf/misc/elasticsearchmaxresults.xml | 15 +
docs-xml/smbdotconf/misc/elasticsearchport.xml | 14 +
docs-xml/smbdotconf/misc/elasticsearchusetls.xml | 14 +
docs-xml/smbdotconf/misc/spotlight_backend.xml | 4 +
docs-xml/wscript_build | 1 +
dynconfig/dynconfig.c | 1 +
dynconfig/dynconfig.h | 1 +
dynconfig/wscript | 5 +
lib/param/loadparm.h | 1 +
lib/param/param_table.c | 1 +
librpc/idl/mdssvc.idl | 6 +-
python/samba/tests/blackbox/mdfind.py | 130 +++
python/samba/tests/dcerpc/mdssvc.py | 194 ++++
script/autobuild.py | 14 +-
selftest/selftest.pl | 1 +
selftest/target/Samba3.pm | 4 +
selftest/tests.py | 2 +
source3/rpc_client/cli_mdssvc.c | 1039 ++++++++++++++++++++
source3/rpc_client/cli_mdssvc.h | 97 ++
source3/rpc_client/cli_mdssvc_private.h | 70 ++
source3/rpc_client/cli_mdssvc_util.c | 531 ++++++++++
source3/rpc_client/cli_mdssvc_util.h | 41 +
source3/rpc_client/py_mdscli.c | 567 +++++++++++
source3/rpc_server/mdssd.c | 9 +-
.../rpc_server/mdssvc/elasticsearch_mappings.json | 142 +++
source3/rpc_server/mdssvc/es_lexer.l | 92 ++
source3/rpc_server/mdssvc/es_mapping.c | 241 +++++
source3/rpc_server/mdssvc/es_mapping.h | 49 +
source3/rpc_server/mdssvc/es_parser.y | 625 ++++++++++++
source3/rpc_server/mdssvc/es_parser_test.c | 99 ++
source3/rpc_server/mdssvc/mdssvc.c | 27 +
source3/rpc_server/mdssvc/mdssvc.h | 1 -
source3/rpc_server/mdssvc/mdssvc_es.c | 835 ++++++++++++++++
source3/rpc_server/mdssvc/mdssvc_es.h | 108 ++
source3/rpc_server/mdssvc/srv_mdssvc_nt.c | 34 +-
source3/rpc_server/mdssvc/test_mdsparser_es.c | 233 +++++
source3/rpc_server/rpc_config.c | 2 +-
source3/rpc_server/wscript_build | 12 +
source3/rpcclient/cmd_spotlight.c | 8 +-
source3/utils/mdfind.c | 290 ++++++
source3/utils/wscript_build | 14 +
source3/wscript | 20 +-
source3/wscript_build | 40 +
source4/librpc/wscript_build | 7 +
source4/selftest/tests.py | 4 +
source4/torture/rpc/mdssvc.c | 20 +-
53 files changed, 5821 insertions(+), 42 deletions(-)
create mode 100644 docs-xml/manpages/mdfind.1.xml
create mode 100644 docs-xml/smbdotconf/misc/elasticsearchaddress.xml
create mode 100644 docs-xml/smbdotconf/misc/elasticsearchindex.xml
create mode 100644 docs-xml/smbdotconf/misc/elasticsearchmappings.xml
create mode 100644 docs-xml/smbdotconf/misc/elasticsearchmaxresults.xml
create mode 100644 docs-xml/smbdotconf/misc/elasticsearchport.xml
create mode 100644 docs-xml/smbdotconf/misc/elasticsearchusetls.xml
create mode 100644 python/samba/tests/blackbox/mdfind.py
create mode 100644 python/samba/tests/dcerpc/mdssvc.py
create mode 100644 source3/rpc_client/cli_mdssvc.c
create mode 100644 source3/rpc_client/cli_mdssvc.h
create mode 100644 source3/rpc_client/cli_mdssvc_private.h
create mode 100644 source3/rpc_client/cli_mdssvc_util.c
create mode 100644 source3/rpc_client/cli_mdssvc_util.h
create mode 100644 source3/rpc_client/py_mdscli.c
create mode 100644 source3/rpc_server/mdssvc/elasticsearch_mappings.json
create mode 100644 source3/rpc_server/mdssvc/es_lexer.l
create mode 100644 source3/rpc_server/mdssvc/es_mapping.c
create mode 100644 source3/rpc_server/mdssvc/es_mapping.h
create mode 100644 source3/rpc_server/mdssvc/es_parser.y
create mode 100644 source3/rpc_server/mdssvc/es_parser_test.c
create mode 100644 source3/rpc_server/mdssvc/mdssvc_es.c
create mode 100644 source3/rpc_server/mdssvc/mdssvc_es.h
create mode 100644 source3/rpc_server/mdssvc/test_mdsparser_es.c
create mode 100644 source3/utils/mdfind.c
Changeset truncated at 500 lines:
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 01838b93d72..62a3c7fcb5d 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -174,6 +174,9 @@ samba-ad-dc-ntvfs:
samba-ad-dc-backup:
extends: .private_template
+samba-simpleserver:
+ extends: .private_template
+
samba-fileserver:
extends: .private_template
diff --git a/buildtools/wafsamba/samba_patterns.py b/buildtools/wafsamba/samba_patterns.py
index d0fe965c730..43422a71f99 100644
--- a/buildtools/wafsamba/samba_patterns.py
+++ b/buildtools/wafsamba/samba_patterns.py
@@ -101,6 +101,8 @@ def write_build_options_header(fp):
fp.write(" output(screen,\" LOGFILEBASE: %s\\n\", get_dyn_LOGFILEBASE());\n")
fp.write(" output(screen,\" LMHOSTSFILE: %s\\n\",get_dyn_LMHOSTSFILE());\n")
fp.write(" output(screen,\" LIBDIR: %s\\n\",get_dyn_LIBDIR());\n")
+ fp.write(" output(screen,\" DATADIR: %s\\n\",get_dyn_DATADIR());\n")
+ fp.write(" output(screen,\" SAMBA_DATADIR: %s\\n\",get_dyn_SAMBA_DATADIR());\n")
fp.write(" output(screen,\" MODULESDIR: %s\\n\",get_dyn_MODULESDIR());\n")
fp.write(" output(screen,\" SHLIBEXT: %s\\n\",get_dyn_SHLIBEXT());\n")
fp.write(" output(screen,\" LOCKDIR: %s\\n\",get_dyn_LOCKDIR());\n")
diff --git a/docs-xml/manpages/mdfind.1.xml b/docs-xml/manpages/mdfind.1.xml
new file mode 100644
index 00000000000..0deef066059
--- /dev/null
+++ b/docs-xml/manpages/mdfind.1.xml
@@ -0,0 +1,148 @@
+<?xml version="1.0" encoding="iso-8859-1"?>
+<!DOCTYPE refentry PUBLIC "-//Samba-Team//DTD DocBook V4.2-Based Variant V1.0//EN" "http://www.samba.org/samba/DTD/samba-doc">
+<refentry id="mdfind.1">
+
+ <refmeta>
+ <refentrytitle>mdfind</refentrytitle>
+ <manvolnum>1</manvolnum>
+ <refmiscinfo class="source">Samba</refmiscinfo>
+ <refmiscinfo class="manual">User Commands</refmiscinfo>
+ <refmiscinfo class="version">&doc.version;</refmiscinfo>
+ </refmeta>
+
+ <refnamediv>
+ <refname>mdfind</refname>
+ <refpurpose>Run Spotlight searches against an SMB server</refpurpose>
+ </refnamediv>
+
+ <refsynopsisdiv>
+ <cmdsynopsis>
+ <command>mvxattr</command>
+ <arg choice="req">server</arg>
+ <arg choice="req">sharename</arg>
+ <arg choice="req">query</arg>
+ <arg choice="opt">-p, --path</arg>
+ <arg choice="opt">-L, --live</arg>
+ </cmdsynopsis>
+ </refsynopsisdiv>
+
+ <refsect1>
+ <title>DESCRIPTION</title>
+
+ <para>This tool is part of the <citerefentry><refentrytitle>samba</refentrytitle>
+ <manvolnum>1</manvolnum></citerefentry> suite.</para>
+
+ <para>mdfind is a simple utility to run Spotlight searches against an SMB server
+ that runs the Spotlight <emphasis>mdssvc</emphasis> RPC service.</para>
+ </refsect1>
+
+ <refsect1>
+ <title>OPTIONS</title>
+
+ <variablelist>
+ <varlistentry>
+ <term>server</term>
+ <listitem>
+ <para>The SMB server name or IP address to connect to.</para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>sharename</term>
+ <listitem>
+ <para>The name of a share on the server.</para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>query</term>
+ <listitem>
+ <para>The query expression syntax is a simplified form of filename
+ globbing familiar to shell users. Queries have the following
+ format:</para>
+
+ <para>attribute=="value"</para>
+
+ <para>For queries against a Samba server with Spotlight enabled using
+ the Elasticsearch backend, the list of supported metadata attributes
+ is given by the JSON attribute mapping file, typically installed at
+ <filename>/usr/share/samba/mdssvc/elasticsearch_mappings.json</filename>
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>-p PATH, --path=PATH</term>
+ <listitem>
+ <para>Server side path to search, defaults to
+ <emphasis>"/"</emphasis></para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>-L, --live</term>
+ <listitem><para>Query remains running.</para></listitem>
+ </varlistentry>
+
+ </variablelist>
+ </refsect1>
+
+ <refsect1>
+ <title>EXAMPLES</title>
+
+ <para>Search all indexed metadata attributes, exact match:</para>
+ <programlisting>
+ '*=="Samba"'
+ </programlisting>
+
+ <para>Search all indexed metadata attributes, prefix match:</para>
+ <programlisting>
+ '*=="Samba*"'
+ </programlisting>
+
+ <para>Search by filename:</para>
+ <programlisting>
+ 'kMDItemFSName=="Samba*"'
+ </programlisting>
+
+ <para>Search by date:</para>
+ <programlisting>
+ 'kMDItemFSContentChangeDate<$time.iso(2018-10-01T10:00:00Z)'
+ </programlisting>
+
+ <para>Search files's content:</para>
+ <programlisting>
+ 'kMDItemTextContent=="Samba*"'
+ </programlisting>
+
+ <para>Expressions:</para>
+ <programlisting>
+ kMDItemFSName=="Samba*"||kMDItemTextContent=="Tango*"'
+ </programlisting>
+ </refsect1>
+
+ <refsect1>
+ <title>SEE ALSO</title>
+
+ <para>File Metadata Search Programming Guide
+ <ulink url="https://developer.apple.com/library/archive/documentation/Carbon/Conceptual/SpotlightQuery/Concepts/Introduction.html">
+ https://developer.apple.com/library/archive/documentation/Carbon/Conceptual/SpotlightQuery/Concepts/Introduction.html</ulink>
+ </para>
+ </refsect1>
+
+ <refsect1>
+ <title>VERSION</title>
+
+ <para>This man page is part of version &doc.version; of the Samba suite.</para>
+ </refsect1>
+
+ <refsect1>
+ <title>AUTHOR</title>
+
+ <para>The original Samba software and related utilities were created by
+ Andrew Tridgell. Samba is now developed by the Samba Team as an Open
+ Source project similar to the way the Linux kernel is developed.</para>
+
+ <para>The mdfind manpage was written by Ralph Boehme.</para>
+ </refsect1>
+</refentry>
diff --git a/docs-xml/smbdotconf/generate-file-list.sh b/docs-xml/smbdotconf/generate-file-list.sh
index 7ab1b7caf76..a86bac5c289 100755
--- a/docs-xml/smbdotconf/generate-file-list.sh
+++ b/docs-xml/smbdotconf/generate-file-list.sh
@@ -17,6 +17,7 @@ echo "<!DOCTYPE section [
<!ENTITY pathconfig.CACHEDIR '\${prefix}/var/cache'>
<!ENTITY pathconfig.NTP_SIGND_SOCKET_DIR '\${prefix}/var/lib/ntp_signd'>
<!ENTITY pathconfig.MITKDCPATH '\${prefix}/sbin/krb5kdc'>
+<!ENTITY pathconfig.SAMBA_DATADIR '\${prefix}/var/samba'>
]>"
DIR=.
diff --git a/docs-xml/smbdotconf/misc/elasticsearchaddress.xml b/docs-xml/smbdotconf/misc/elasticsearchaddress.xml
new file mode 100644
index 00000000000..61125461a91
--- /dev/null
+++ b/docs-xml/smbdotconf/misc/elasticsearchaddress.xml
@@ -0,0 +1,14 @@
+<samba:parameter name="elasticsearch:address"
+ context="S"
+ type="string"
+ xmlns:samba="http://www.samba.org/samba/DTD/samba-doc">
+ <description>
+ <para>
+ Specifies the name of the Elasticsearch server to use for Spotlight
+ queries when using the Elasticsearch backend.
+ </para>
+ </description>
+
+ <value type="default">localhost</value>
+ <value type="example">needle.haystack.samba.org</value>
+</samba:parameter>
diff --git a/docs-xml/smbdotconf/misc/elasticsearchindex.xml b/docs-xml/smbdotconf/misc/elasticsearchindex.xml
new file mode 100644
index 00000000000..7f394b264f8
--- /dev/null
+++ b/docs-xml/smbdotconf/misc/elasticsearchindex.xml
@@ -0,0 +1,16 @@
+<samba:parameter name="elasticsearch:index"
+ context="S"
+ type="string"
+ xmlns:samba="http://www.samba.org/samba/DTD/samba-doc">
+ <description>
+ <para>
+ Specifies the name of the Elasticsearch index to use for Spotlight queries
+ when using the Elasticsearch backend. The default value of "_all" is a
+ special Elasticsearch value that performs the search operation on all
+ indices.
+ </para>
+ </description>
+
+ <value type="default">_all</value>
+ <value type="example">spotlight</value>
+</samba:parameter>
diff --git a/docs-xml/smbdotconf/misc/elasticsearchmappings.xml b/docs-xml/smbdotconf/misc/elasticsearchmappings.xml
new file mode 100644
index 00000000000..d2502a6fc3b
--- /dev/null
+++ b/docs-xml/smbdotconf/misc/elasticsearchmappings.xml
@@ -0,0 +1,14 @@
+<samba:parameter name="elasticsearch:mappings"
+ context="G"
+ type="string"
+ xmlns:samba="http://www.samba.org/samba/DTD/samba-doc">
+ <description>
+ <para>
+ Path to a file specifying metadata attribute mappings in JSON format. Use
+ by the Elasticsearch backend of the Spotlight RPC service.
+ </para>
+ </description>
+
+ <value type="default">&pathconfig.SAMBA_DATADIR;/elasticsearch_mappings.json</value>
+ <value type="example">/usr/share/foo/mymappings.json</value>
+</samba:parameter>
diff --git a/docs-xml/smbdotconf/misc/elasticsearchmaxresults.xml b/docs-xml/smbdotconf/misc/elasticsearchmaxresults.xml
new file mode 100644
index 00000000000..1086b898ed8
--- /dev/null
+++ b/docs-xml/smbdotconf/misc/elasticsearchmaxresults.xml
@@ -0,0 +1,15 @@
+<samba:parameter name="elasticsearch:max results"
+ context="S"
+ type="integer"
+ xmlns:samba="http://www.samba.org/samba/DTD/samba-doc">
+ <description>
+ <para>
+ Path to a file specifying metadata attribute mappings in JSON format. Used
+ by the Elasticsearch backend of the Spotlight RPC service. A value of 0
+ means no limit.
+ </para>
+ </description>
+
+ <value type="default">100</value>
+ <value type="example">10</value>
+</samba:parameter>
diff --git a/docs-xml/smbdotconf/misc/elasticsearchport.xml b/docs-xml/smbdotconf/misc/elasticsearchport.xml
new file mode 100644
index 00000000000..ea87daabc28
--- /dev/null
+++ b/docs-xml/smbdotconf/misc/elasticsearchport.xml
@@ -0,0 +1,14 @@
+<samba:parameter name="elasticsearch:port"
+ context="S"
+ type="integer"
+ xmlns:samba="http://www.samba.org/samba/DTD/samba-doc">
+ <description>
+ <para>
+ Specifies the TCP port of the Elasticsearch server to use for Spotlight
+ queries when using the Elasticsearch backend.
+ </para>
+ </description>
+
+ <value type="default">9200</value>
+ <value type="example">9201</value>
+</samba:parameter>
diff --git a/docs-xml/smbdotconf/misc/elasticsearchusetls.xml b/docs-xml/smbdotconf/misc/elasticsearchusetls.xml
new file mode 100644
index 00000000000..e1aa8a31495
--- /dev/null
+++ b/docs-xml/smbdotconf/misc/elasticsearchusetls.xml
@@ -0,0 +1,14 @@
+<samba:parameter name="elasticsearch:use tls"
+ context="S"
+ type="boolean"
+ xmlns:samba="http://www.samba.org/samba/DTD/samba-doc">
+ <description>
+ <para>
+ Specifies whether to use HTTPS when talking to the Elasticsearch server
+ used for Spotlight queries when using the Elasticsearch backend.
+ </para>
+ </description>
+
+ <value type="default">no</value>
+ <value type="example">yes</value>
+</samba:parameter>
diff --git a/docs-xml/smbdotconf/misc/spotlight_backend.xml b/docs-xml/smbdotconf/misc/spotlight_backend.xml
index 6d224b81e5f..0643fc16cbd 100644
--- a/docs-xml/smbdotconf/misc/spotlight_backend.xml
+++ b/docs-xml/smbdotconf/misc/spotlight_backend.xml
@@ -19,6 +19,10 @@
Gnome Tracker.
</para></listitem>
+ <listitem><para><constant>elasticsearch</constant> -
+ a backend that uses JSON and REST over HTTP(s) to query an
+ Elasticsearch server.
+ </para></listitem>
</itemizedlist>
</para>
</description>
diff --git a/docs-xml/wscript_build b/docs-xml/wscript_build
index b85309d4ead..70f5b43dd33 100644
--- a/docs-xml/wscript_build
+++ b/docs-xml/wscript_build
@@ -19,6 +19,7 @@ manpages='''
manpages/libsmbclient.7
manpages/lmhosts.5
manpages/log2pcap.1
+ manpages/mdfind.1
manpages/mvxattr.1
manpages/net.8
manpages/nmbd.8
diff --git a/dynconfig/dynconfig.c b/dynconfig/dynconfig.c
index e70a10f8cfe..dd4de16e1d4 100644
--- a/dynconfig/dynconfig.c
+++ b/dynconfig/dynconfig.c
@@ -99,6 +99,7 @@ DEFINE_DYN_CONFIG_PARAM(BINDDNS_DIR)
DEFINE_DYN_CONFIG_PARAM(LOCALEDIR)
DEFINE_DYN_CONFIG_PARAM(NMBDSOCKETDIR)
DEFINE_DYN_CONFIG_PARAM(DATADIR)
+DEFINE_DYN_CONFIG_PARAM(SAMBA_DATADIR)
DEFINE_DYN_CONFIG_PARAM(SETUPDIR)
DEFINE_DYN_CONFIG_PARAM(WINBINDD_SOCKET_DIR) /* from winbind_struct_protocol.h in s3 autoconf */
DEFINE_DYN_CONFIG_PARAM(NTP_SIGND_SOCKET_DIR)
diff --git a/dynconfig/dynconfig.h b/dynconfig/dynconfig.h
index bdab2e8f242..156e63e38ac 100644
--- a/dynconfig/dynconfig.h
+++ b/dynconfig/dynconfig.h
@@ -50,6 +50,7 @@ DEFINE_DYN_CONFIG_PROTO(BINDDNS_DIR)
DEFINE_DYN_CONFIG_PROTO(LOCALEDIR)
DEFINE_DYN_CONFIG_PROTO(NMBDSOCKETDIR)
DEFINE_DYN_CONFIG_PROTO(DATADIR)
+DEFINE_DYN_CONFIG_PROTO(SAMBA_DATADIR)
DEFINE_DYN_CONFIG_PROTO(SETUPDIR)
DEFINE_DYN_CONFIG_PROTO(WINBINDD_SOCKET_DIR)
DEFINE_DYN_CONFIG_PROTO(NTP_SIGND_SOCKET_DIR)
diff --git a/dynconfig/wscript b/dynconfig/wscript
index e352712d284..5a0ce6d956a 100644
--- a/dynconfig/wscript
+++ b/dynconfig/wscript
@@ -100,6 +100,11 @@ dynconfig = {
'FHS-PATH': '${DATADIR}',
'OVERWRITE': True,
},
+ 'SAMBA_DATADIR' : {
+ 'STD-PATH': '${DATADIR}/samba',
+ 'FHS-PATH': '${DATADIR}/samba',
+ 'OVERWRITE': True,
+ },
'LOCALEDIR' : {
'STD-PATH': '${LOCALEDIR}',
'FHS-PATH': '${LOCALEDIR}',
diff --git a/lib/param/loadparm.h b/lib/param/loadparm.h
index 5c5b1cd7cd4..d0ce3d312e1 100644
--- a/lib/param/loadparm.h
+++ b/lib/param/loadparm.h
@@ -252,6 +252,7 @@ enum mangled_names_options {MANGLED_NAMES_NO, MANGLED_NAMES_YES, MANGLED_NAMES_I
enum spotlight_backend_options {
SPOTLIGHT_BACKEND_NOINDEX,
SPOTLIGHT_BACKEND_TRACKER,
+ SPOTLIGHT_BACKEND_ES,
};
/*
diff --git a/lib/param/param_table.c b/lib/param/param_table.c
index 82dc5cd6cda..2fd3361f996 100644
--- a/lib/param/param_table.c
+++ b/lib/param/param_table.c
@@ -354,6 +354,7 @@ static const struct enum_list enum_ntlm_auth[] = {
static const struct enum_list enum_spotlight_backend[] = {
{SPOTLIGHT_BACKEND_NOINDEX, "noindex"},
{SPOTLIGHT_BACKEND_TRACKER, "tracker"},
+ {SPOTLIGHT_BACKEND_ES, "elasticsearch"},
{-1, NULL}
};
diff --git a/librpc/idl/mdssvc.idl b/librpc/idl/mdssvc.idl
index 43447961b0d..e7004f07419 100644
--- a/librpc/idl/mdssvc.idl
+++ b/librpc/idl/mdssvc.idl
@@ -18,7 +18,7 @@ interface mdssvc
);
void mdssvc_unknown1(
- [in] policy_handle handle,
+ [in] policy_handle *handle,
[in] uint32 unkn1, /* always 0, some status ? */
[in] uint32 device_id,
[in] uint32 unkn3, /* = mdssvc_open.unkn2 ? */
@@ -37,7 +37,7 @@ interface mdssvc
} mdssvc_blob;
void mdssvc_cmd(
- [in] policy_handle handle,
+ [in] policy_handle *handle,
[in] uint32 unkn1, /* always 0, status ? */
[in] uint32 device_id,
[in] uint32 unkn3, /* = mdssvc_open.unkn2 ? */
@@ -57,7 +57,7 @@ interface mdssvc
);
void mdssvc_close(
- [in] policy_handle in_handle,
+ [in] policy_handle *in_handle,
[in] uint32 unkn1, /* always 0, some status ? */
[in] uint32 device_id,
[in] uint32 unkn2, /* = mdssvc_open.unkn2 ? */
diff --git a/python/samba/tests/blackbox/mdfind.py b/python/samba/tests/blackbox/mdfind.py
new file mode 100644
index 00000000000..44de51cf767
--- /dev/null
+++ b/python/samba/tests/blackbox/mdfind.py
@@ -0,0 +1,130 @@
+#
+# Blackbox tests for mdfind
+#
+# Copyright (C) Ralph Boehme 2019
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+#
+
+"""Blackbox test for mdfind"""
+
+import os
+import time
+import threading
+import logging
+import json
+from http.server import HTTPServer, BaseHTTPRequestHandler
+from samba.dcerpc import mdssvc
+from samba.tests import BlackboxTestCase
+from samba.samba3 import mdscli
+from samba.logger import get_samba_logger
+
+logger = get_samba_logger(name=__name__)
+
+testfiles = [
+ "foo",
+ "bar",
+]
+
+class MdssvcHTTPRequestHandler(BaseHTTPRequestHandler):
+ def do_POST(self):
+ content_length = int(self.headers['content-length'])
+ body = self.rfile.read(content_length)
+
+ actual_json = json.loads((body))
+ expected_json = json.loads(self.server.json_in)
+
+ if actual_json != expected_json:
+ logger.error("Bad request, expected:\n%s\nGot:\n%s\n" % (expected_json, actual_json))
+ self.send_error(400,
+ "Bad request",
+ "Expected: %s\n"
+ "Got: %s\n" %
+ (expected_json, actual_json))
+ return
+
--
Samba Shared Repository
More information about the samba-cvs
mailing list