[PATCH] Re: [WIP] Log database changes.

Andrew Bartlett abartlet at samba.org
Tue May 15 22:53:08 UTC 2018


On Fri, 2018-05-11 at 06:06 +1200, Andrew Bartlett via samba-technical
wrote:
> On Mon, 2018-05-07 at 18:05 +0200, Stefan Metzmacher via samba-
> technical wrote:
> > Hi Gary,
> > 
> > > Current state of this task.
> > > 
> > > Comments appreciated.
> > 
> > Most of the preparation like the session guid looks good.
> 
> I've reviewed and pushed the preparation patches patches that I could
> get past an autobuild on GitLab CI.  The new common audit/json code
> needs a talloc dep (fails to find talloc.h, a common trap for new
> isolated subsystems), and I'll fix that next and push some more of the
> prep code later.

Attached is the larger reviewed set of patches that I'm planning to put
into the next autobuild.

These are 
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Andrew Bartlett

-- 
Andrew Bartlett
https://samba.org/~abartlet/
Authentication Developer, Samba Team         https://samba.org
Samba Development and Support, Catalyst IT   
https://catalyst.net.nz/services/samba



-------------- next part --------------
From abff099c5762ca30dce9bb36a9f4e33e3be702e6 Mon Sep 17 00:00:00 2001
From: Gary Lockyer <gary at catalyst.net.nz>
Date: Tue, 10 Apr 2018 06:45:47 +1200
Subject: [PATCH 01/10] auth logging: Extract common audit logging code

Extract the common audit logging code into a library to allow it's
re-use in other logging modules.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
---
 lib/audit_logging/audit_logging.c            | 771 +++++++++++++++++++++++++++
 lib/audit_logging/audit_logging.h            |  89 ++++
 lib/audit_logging/tests/audit_logging_test.c | 557 +++++++++++++++++++
 lib/audit_logging/wscript_build              |  24 +
 source4/selftest/tests.py                    |   2 +
 wscript_build                                |   1 +
 6 files changed, 1444 insertions(+)
 create mode 100644 lib/audit_logging/audit_logging.c
 create mode 100644 lib/audit_logging/audit_logging.h
 create mode 100644 lib/audit_logging/tests/audit_logging_test.c
 create mode 100644 lib/audit_logging/wscript_build

diff --git a/lib/audit_logging/audit_logging.c b/lib/audit_logging/audit_logging.c
new file mode 100644
index 00000000000..7da161cd204
--- /dev/null
+++ b/lib/audit_logging/audit_logging.c
@@ -0,0 +1,771 @@
+/*
+   common routines for audit logging
+
+   Copyright (C) Andrew Bartlett <abartlet at samba.org> 2018
+
+   This program is free software; you can redistribute it and/or modify
+   it under the terms of the GNU General Public License as published by
+   the Free Software Foundation; either version 3 of the License, or
+   (at your option) any later version.
+
+   This program is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+   GNU General Public License for more details.
+
+   You should have received a copy of the GNU General Public License
+   along with this program.  If not, see <http://www.gnu.org/licenses/>.
+*/
+
+/*
+ * Error handling:
+ *
+ * The json_object structure contains a boolean 'error'.  This is set whenever
+ * an error is detected. All the library functions check this flag and return
+ * immediately if it is set.
+ *
+ *	if (object->error) {
+ *		return;
+ *	}
+ *
+ * This allows the operations to be sequenced naturally with out the clutter
+ * of error status checks.
+ *
+ *	audit = json_new_object();
+ *	json_add_version(&audit, OPERATION_MAJOR, OPERATION_MINOR);
+ *	json_add_int(&audit, "statusCode", ret);
+ *	json_add_string(&audit, "status", ldb_strerror(ret));
+ *	json_add_string(&audit, "operation", operation);
+ *	json_add_address(&audit, "remoteAddress", remote);
+ *	json_add_sid(&audit, "userSid", sid);
+ *	json_add_string(&audit, "dn", dn);
+ *	json_add_guid(&audit, "transactionId", &ac->transaction_guid);
+ *	json_add_guid(&audit, "sessionId", unique_session_token);
+ *
+ * The assumptions are that errors will be rare, and that the audit logging
+ * code should not cause failures. So errors are logged but processing
+ * continues on a best effort basis.
+ */
+
+#include "includes.h"
+
+#include "librpc/ndr/libndr.h"
+#include "lib/tsocket/tsocket.h"
+#include "libcli/security/dom_sid.h"
+#include "lib/messaging/messaging.h"
+#include "auth/common_auth.h"
+#include "audit_logging.h"
+
+/*
+ * @brief Get a human readable timestamp.
+ *
+ * Returns the current time formatted as
+ *  "Tue, 14 Mar 2017 08:38:42.209028 NZDT"
+ *
+ * The returned string is allocated by talloc in the supplied context.
+ * It is the callers responsibility to free it.
+ *
+ * @param mem_ctx talloc memory context that owns the returned string.
+ *
+ * @return a human readable time stamp.
+ *
+ */
+char* audit_get_timestamp(TALLOC_CTX *frame)
+{
+	char buffer[40];	/* formatted time less usec and timezone */
+	char tz[10];		/* formatted time zone			 */
+	struct tm* tm_info;	/* current local time			 */
+	struct timeval tv;	/* current system time			 */
+	int r;			/* response code from gettimeofday	 */
+	char * ts;		/* formatted time stamp			 */
+
+	r = gettimeofday(&tv, NULL);
+	if (r) {
+		DBG_ERR("Unable to get time of day: (%d) %s\n",
+			errno,
+			strerror(errno));
+		return NULL;
+	}
+
+	tm_info = localtime(&tv.tv_sec);
+	if (tm_info == NULL) {
+		DBG_ERR("Unable to determine local time\n");
+		return NULL;
+	}
+
+	strftime(buffer, sizeof(buffer)-1, "%a, %d %b %Y %H:%M:%S", tm_info);
+	strftime(tz, sizeof(tz)-1, "%Z", tm_info);
+	ts = talloc_asprintf(frame, "%s.%06ld %s", buffer, tv.tv_usec, tz);
+	if (ts == NULL) {
+		DBG_ERR("Out of memory formatting time stamp\n");
+	}
+	return ts;
+}
+
+#ifdef HAVE_JANSSON
+
+#include "system/time.h"
+
+/*
+ * @brief get a connection to the messaging event server.
+ *
+ * Get a connection to the messaging event server registered by server_name.
+ *
+ * @param msg_ctx a valid imessaging_context.
+ * @param server_name name of messaging event server to connect to.
+ * @param server_id The event server details to populate
+ *
+ * @return NTSTATUS
+ */
+static NTSTATUS get_event_server(
+	struct imessaging_context *msg_ctx,
+	const char *server_name,
+	struct server_id *event_server)
+{
+	NTSTATUS status;
+	TALLOC_CTX *frame = talloc_stackframe();
+	unsigned num_servers, i;
+	struct server_id *servers;
+
+	status = irpc_servers_byname(
+		msg_ctx,
+		frame,
+		server_name,
+		&num_servers,
+		&servers);
+
+	if (!NT_STATUS_IS_OK(status)) {
+		DBG_NOTICE(
+			"Failed to find '%s' registered on the message bus to "
+			"send audit events to: %s\n",
+			server_name,
+			nt_errstr(status));
+		TALLOC_FREE(frame);
+		return status;
+	}
+
+	/*
+	 * Select the first server that is listening, because we get
+	 * connection refused as NT_STATUS_OBJECT_NAME_NOT_FOUND
+	 * without waiting
+	 */
+	for (i = 0; i < num_servers; i++) {
+		status = imessaging_send(
+			msg_ctx,
+			servers[i],
+			MSG_PING,
+			&data_blob_null);
+		if (NT_STATUS_IS_OK(status)) {
+			*event_server = servers[i];
+			TALLOC_FREE(frame);
+			return NT_STATUS_OK;
+		}
+	}
+	DBG_NOTICE(
+		"Failed to find '%s' registered on the message bus to "
+		"send audit events to: %s\n",
+		server_name,
+		nt_errstr(status));
+	TALLOC_FREE(frame);
+	return NT_STATUS_OBJECT_NAME_NOT_FOUND;
+}
+
+/*
+ * @brief send an audit message to a messaging event server.
+ *
+ * Send the message to a registered and listening event server.
+ * Note: Any errors are logged, and the message is not sent.  This is to ensure
+ *       that a poorly behaved event server does not impact Samba.
+ *
+ *       As it is possible to lose messages, especially during server
+ *       shut down, currently this function is primarily intended for use
+ *       in integration tests.
+ *
+ * @param msg_ctx an imessaging_context, can be NULL in which case no message
+ *                will be sent.
+ * @param server_name the naname of the event server to send the message to.
+ * @param messag_type A message type defined in librpc/idl/messaging.idl
+ * @param message The message to send.
+ *
+ */
+void audit_message_send(
+	struct imessaging_context *msg_ctx,
+	const char *server_name,
+	uint32_t message_type,
+	const char *message)
+{
+	struct server_id event_server;
+	NTSTATUS status;
+	DATA_BLOB message_blob = data_blob_string_const(message);
+
+	if (msg_ctx == NULL) {
+		DBG_DEBUG("No messaging context\n");
+		return;
+	}
+
+	/* Need to refetch the address each time as the destination server may
+	 * have disconnected and reconnected in the interim, in which case
+	 * messages may get lost
+	 */
+	status = get_event_server(msg_ctx, server_name, &event_server);
+	if (!NT_STATUS_IS_OK(status) &&
+	    !NT_STATUS_EQUAL(status, NT_STATUS_OBJECT_NAME_NOT_FOUND)) {
+		DBG_ERR("get_event_server for %s returned (%s)\n",
+			server_name,
+			nt_errstr(status));
+		return;
+	}
+
+	status = imessaging_send(
+		msg_ctx,
+		event_server,
+		message_type,
+		&message_blob);
+
+	/*
+	 * If the server crashed, try to find it again
+	 */
+	if (NT_STATUS_EQUAL(status, NT_STATUS_OBJECT_NAME_NOT_FOUND)) {
+		status = get_event_server(msg_ctx, server_name, &event_server);
+		if (!NT_STATUS_IS_OK(status) &&
+		    !NT_STATUS_EQUAL(status, NT_STATUS_OBJECT_NAME_NOT_FOUND)) {
+			DBG_ERR("get_event_server for %s returned (%s)\n",
+				server_name,
+				nt_errstr(status));
+			return;
+		}
+		imessaging_send(
+			msg_ctx,
+			event_server,
+			message_type,
+			&message_blob);
+	}
+}
+
+/*
+ * @brief Create a new struct json_object, wrapping a JSON Object.
+ *
+ * Create a new json object, the json_object wraps the underlying json
+ * implementations JSON Object representation.
+ *
+ * Free with a call to json_free_object, note that the jansson inplementation
+ * allocates memory with malloc and not talloc.
+ *
+ * @return a struct json_object, error will be set to true if the object
+ *         could not be created.
+ *
+ */
+struct json_object json_new_object(void) {
+
+	struct json_object object;
+	object.error = false;
+
+	object.root = json_object();
+	if (object.root == NULL) {
+		object.error = true;
+		DBG_ERR("Unable to create json_object\n");
+	}
+	return object;
+}
+
+/*
+ * @brief Create a new struct json_object wrapping a JSON Array.
+ *
+ * Create a new json object, the json_object wraps the underlying json
+ * implementations JSON Array representation.
+ *
+ * Free with a call to json_free_object, note that the jansson inplementation
+ * allocates memory with malloc and not talloc.
+ *
+ * @return a struct json_object, error will be set to true if the array
+ *         could not be created.
+ *
+ */
+struct json_object json_new_array(void) {
+
+	struct json_object array;
+	array.error = false;
+
+	array.root = json_array();
+	if (array.root == NULL) {
+		array.error = true;
+		DBG_ERR("Unable to create json_array\n");
+	}
+	return array;
+}
+
+
+/*
+ * @brief free and invalidate a previously created JSON object.
+ *
+ * Release any resources owned by a json_object, and then mark the structure
+ * as invalid.  It is safe to call this multiple times on an object.
+ *
+ */
+void json_free(struct json_object *object)
+{
+	if (object->root != NULL) {
+		json_decref(object->root);
+	}
+	object->root = NULL;
+	object->error = true;
+}
+
+/*
+ * @brief is the current JSON object invalid?
+ *
+ * Check the state of the object to determine if it is invalid.
+ *
+ * @return is the object valid?
+ *
+ */
+bool json_is_invalid(struct json_object *object)
+{
+	return object->error;
+}
+
+/*
+ * @brief Add an integer value to a JSON object.
+ *
+ * Add an integer value named 'name' to the json object.
+ * In the event of an error object will be invalidated.
+ *
+ * @param object the JSON object to be updated.
+ * @param name the name of the value.
+ * @param value the value.
+ *
+ */
+void json_add_int(struct json_object *object,
+		  const char* name,
+		  const int value)
+{
+	int rc = 0;
+
+	if (object->error) {
+		return;
+	}
+
+	rc = json_object_set_new(object->root, name, json_integer(value));
+	if (rc) {
+		DBG_ERR("Unable to set name [%s] value [%d]\n", name, value);
+		object->error = true;
+	}
+}
+
+/*
+ * @brief Add a boolean value to a JSON object.
+ *
+ * Add a boolean value named 'name' to the json object.
+ * In the event of an error object will be invalidated.
+ *
+ * @param object the JSON object to be updated.
+ * @param name the name.
+ * @param value the value.
+ *
+ */
+void json_add_bool(struct json_object *object,
+		   const char* name,
+		   const bool value)
+{
+	int rc = 0;
+
+	if (object->error) {
+		return;
+	}
+
+	rc = json_object_set_new(object->root, name, json_boolean(value));
+	if (rc) {
+		DBG_ERR("Unable to set name [%s] value [%d]\n", name, value);
+		object->error = true;
+	}
+
+}
+
+/*
+ * @brief Add a string value to a JSON object.
+ *
+ * Add a string value named 'name' to the json object.
+ * In the event of an error object will be invalidated.
+ *
+ * @param object the JSON object to be updated.
+ * @param name the name.
+ * @param value the value.
+ *
+ */
+void json_add_string(struct json_object *object,
+		     const char* name,
+		     const char* value)
+{
+	int rc = 0;
+
+	if (object->error) {
+		return;
+	}
+
+	if (value) {
+		rc = json_object_set_new(
+			object->root,
+			name,
+			json_string(value));
+	} else {
+		rc = json_object_set_new(object->root, name, json_null());
+	}
+	if (rc) {
+		DBG_ERR("Unable to set name [%s] value [%s]\n", name, value);
+		object->error = true;
+	}
+}
+
+/*
+ * @brief Assert that the current JSON object is an array.
+ *
+ * Check that the current object is a JSON array, and if not
+ * invalidate the object. We also log an error message as this indicates
+ * bug in the calling code.
+ *
+ * @param object the JSON object to be validated.
+ */
+void json_assert_is_array(struct json_object *array) {
+
+	if (array->error) {
+		return;
+	}
+
+	if (json_is_array(array->root) == false) {
+		DBG_ERR("JSON object is not an array\n");
+		array->error = true;
+		return;
+	}
+}
+
+/*
+ * @brief Add a JSON object to a JSON object.
+ *
+ * Add a JSON object named 'name' to the json object.
+ * In the event of an error object will be invalidated.
+ *
+ * @param object the JSON object to be updated.
+ * @param name the name.
+ * @param value the value.
+ *
+ */
+void json_add_object(struct json_object *object,
+		const char* name,
+		struct json_object *value)
+{
+	int rc = 0;
+	json_t *jv = NULL;
+
+	if (object->error) {
+		return;
+	}
+
+	if (value != NULL && value->error) {
+		object->error = true;
+		return;
+	}
+
+	jv = value == NULL ? json_null() : value->root;
+
+	if (json_is_array(object->root)) {
+		rc = json_array_append_new(object->root, jv);
+	} else if (json_is_object(object->root)) {
+		rc = json_object_set_new(object->root, name,  jv);
+	} else {
+		DBG_ERR("Invalid JSON object type\n");
+		object->error = true;
+	}
+	if (rc) {
+		DBG_ERR("Unable to add object [%s]\n", name);
+		object->error = true;
+	}
+}
+
+/*
+ * @brief Add a string to a JSON object, truncating if necessary.
+ *
+ *
+ * Add a string value named 'name' to the json object, the string will be
+ * truncated if it is more than len characters long. If len is 0 the value
+ * is encoded as a JSON null.
+ *
+ * In the event of an error object will be invalidated.
+ *
+ * @param object the JSON object to be updated.
+ * @param name the name.
+ * @param value the value.
+ * @param len the maximum number of characters to be copied.
+ *
+ */
+void json_add_stringn(
+	struct json_object *object,
+	const char *name,
+	const char *value,
+	const size_t len)
+{
+
+	int rc = 0;
+	if (object->error) {
+		return;
+	}
+
+	if (value != NULL && len > 0) {
+		char buffer[len+1];
+		strncpy(buffer, value, len);
+		buffer[len] = '\0';
+		rc = json_object_set_new(object->root,
+					 name,
+					 json_string(buffer));
+	} else {
+		rc = json_object_set_new(object->root, name, json_null());
+	}
+	if (rc) {
+		DBG_ERR("Unable to set name [%s] value [%s]\n", name, value);
+		object->error = true;
+	}
+}
+
+/*
+ * @brief Add a version object to a JSON object
+ *
+ * Add a version object to the JSON object
+ * 	"version":{"major":1, "minor":0}
+ *
+ * The version tag is intended to aid the processing of the JSON messages
+ * The major version number should change when an attribute is:
+ *  - renamed
+ *  - removed
+ *  - its meaning changes
+ *  - its contents change format
+ * The minor version should change whenever a new attribute is added and for
+ * minor bug fixes to an attributes content.
+ *
+ * In the event of an error object will be invalidated.
+ *
+ * @param object the JSON object to be updated.
+ * @param major the major version number
+ * @param minor the minor version number
+ */
+void json_add_version(struct json_object *object, int major, int minor)
+{
+	struct json_object version = json_new_object();
+	json_add_int(&version, "major", major);
+	json_add_int(&version, "minor", minor);
+	json_add_object(object, "version", &version);
+}
+
+/*
+ * @brief add an ISO 8601 timestamp to the object.
+ *
+ * Add the current date and time as a timestamp in ISO 8601 format
+ * to a JSON object
+ *
+ * "timestamp":"2017-03-06T17:18:04.455081+1300"
+ *
+ * In the event of an error object will be invalidated.
+ *
+ * @param object the JSON object to be updated.
+ */
+void json_add_timestamp(struct json_object *object)
+{
+	char buffer[40];	/* formatted time less usec and timezone */
+	char timestamp[50];	/* the formatted ISO 8601 time stamp	 */
+	char tz[10];		/* formatted time zone			 */
+	struct tm* tm_info;	/* current local time			 */
+	struct timeval tv;	/* current system time			 */
+	int r;			/* response code from gettimeofday	 */
+
+	if (object->error) {
+		return;
+	}
+
+	r = gettimeofday(&tv, NULL);
+	if (r) {
+		DBG_ERR("Unable to get time of day: (%d) %s\n",
+			errno,
+			strerror(errno));
+		object->error = true;
+		return;
+	}
+
+	tm_info = localtime(&tv.tv_sec);
+	if (tm_info == NULL) {
+		DBG_ERR("Unable to determine local time\n");
+		object->error = true;
+		return;
+	}
+
+	strftime(buffer, sizeof(buffer)-1, "%Y-%m-%dT%T", tm_info);
+	strftime(tz, sizeof(tz)-1, "%z", tm_info);
+	snprintf(
+		timestamp,
+		sizeof(timestamp),
+		"%s.%06ld%s",
+		buffer,
+		tv.tv_usec,
+		tz);
+	json_add_string(object, "timestamp", timestamp);
+}
+
+
+/*
+ *@brief Add a tsocket_address to a JSON object
+ *
+ * Add the string representation of a Samba tsocket_address to the object.
+ *
+ * "localAddress":"ipv6::::0"
+ *
+ * In the event of an error object will be invalidated.
+ *
+ * @param object the JSON object to be updated.
+ * @param name the name.
+ * @param address the tsocket_address.
+ *
+ */
+void json_add_address(
+	struct json_object *object,
+	const char *name,
+	const struct tsocket_address *address)
+{
+
+	if (object->error) {
+		return;
+	}
+	if (address == NULL) {
+		int rc = json_object_set_new(object->root, name, json_null());
+		if (rc) {
+			DBG_ERR("Unable to set address [%s] to null\n", name);
+			object->error = true;
+		}
+	} else {
+		TALLOC_CTX *ctx = talloc_new(NULL);
+		char *s = NULL;
+
+		s = tsocket_address_string(address, ctx);
+		json_add_string(object, name, s);
+		TALLOC_FREE(ctx);
+	}
+}
+
+/*
+ * @brief Add a formatted string representation of a sid to a json object.
+ *
+ * Add the string representation of a Samba sid to the object.
+ *
+ * "sid":"S-1-5-18"
+ *
+ * In the event of an error object will be invalidated.
+ *
+ * @param object the JSON object to be updated.
+ * @param name the name.
+ * @param sid the sid
+ *
+ */
+void json_add_sid(
+	struct json_object *object,
+	const char *name,
+	const struct dom_sid *sid)
+{
+
+	if (object->error) {
+		return;
+	}
+	if (sid == NULL) {
+		int rc = json_object_set_new(object->root, name, json_null());
+		if (rc) {
+			DBG_ERR("Unable to set SID [%s] to null\n", name);
+			object->error = true;
+		}
+	} else {
+		char sid_buf[DOM_SID_STR_BUFLEN];
+
+		dom_sid_string_buf(sid, sid_buf, sizeof(sid_buf));
+		json_add_string(object, name, sid_buf);
+	}
+}
+
+/*
+ * @brief Add a formatted string representation of a guid to a json object.
+ *
+ * Add the string representation of a Samba GUID to the object.
+ *
+ * "guid":"1fb9f2ee-2a4d-4bf8-af8b-cb9d4529a9ab"
+ *
+ * In the event of an error object will be invalidated.
+ *
+ * @param object the JSON object to be updated.
+ * @param name the name.
+ * @param guid the guid.
+ *
+ *
+ */
+void json_add_guid(
+	struct json_object *object,
+	const char *name,
+	const struct GUID *guid)
+{
+
+
+	if (object->error) {
+		return;
+	}
+	if (guid == NULL) {
+		int rc = json_object_set_new(object->root, name, json_null());
+		if (rc) {
+			DBG_ERR("Unable to set GUID [%s] to null\n", name);
+			object->error = true;
+		}
+	} else {
+		char *guid_str;
+		struct GUID_txt_buf guid_buff;
+
+		guid_str = GUID_buf_string(guid, &guid_buff);
+		json_add_string(object, name, guid_str);
+	}
+}
+
+
+/*
+ * @brief Convert a JSON object into a string
+ *
+ * Convert the jsom object into a string suitable for printing on a log line,
+ * i.e. with no embedded line breaks.
+ *
+ * If the object is invalid it returns NULL.
+ *
+ * @param mem_ctx the talloc memory context owning the returned string
+ * @param object the json object.
+ *
+ * @return A string representation of the object or NULL if the object
+ *         is invalid.
+ */
+char *json_to_string(TALLOC_CTX *mem_ctx, struct json_object *object)
+{
+	char *json = NULL;
+	char *json_string = NULL;
+
+	if (object->error) {
+		return NULL;
+	}
+
+	/*
+	 * json_dumps uses malloc, so need to call free(json) to release
+	 * the memory
+	 */
+	json = json_dumps(object->root, 0);
+	if (json == NULL) {
+		DBG_ERR("Unable to convert JSON object to string\n");
+		return NULL;
+	}
+
+	json_string = talloc_strdup(mem_ctx, json);
+	if (json_string == NULL) {
+		free(json);
+		DBG_ERR("Unable to copy JSON object string to talloc string\n");
+		return NULL;
+	}
+	free(json);
+
+	return json_string;
+}
+#endif
diff --git a/lib/audit_logging/audit_logging.h b/lib/audit_logging/audit_logging.h
new file mode 100644
index 00000000000..763f3ed5d9c
--- /dev/null
+++ b/lib/audit_logging/audit_logging.h
@@ -0,0 +1,89 @@
+/*
+   common routines for audit logging
+
+   Copyright (C) Andrew Bartlett <abartlet at samba.org> 2018
+
+   This program is free software; you can redistribute it and/or modify
+   it under the terms of the GNU General Public License as published by
+   the Free Software Foundation; either version 3 of the License, or
+   (at your option) any later version.
+
+   This program is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+   GNU General Public License for more details.
+
+   You should have received a copy of the GNU General Public License
+   along with this program.  If not, see <http://www.gnu.org/licenses/>.
+*/
+
+#include <talloc.h>
+#include "lib/messaging/irpc.h"
+#include "lib/tsocket/tsocket.h"
+
+char* audit_get_timestamp(
+	TALLOC_CTX *frame);
+
+void audit_message_send(
+	struct imessaging_context *msg_ctx,
+	const char *server_name,
+	uint32_t message_type,
+	const char *message);
+#ifdef HAVE_JANSSON
+#include <jansson.h>
+/*
+ * Wrapper for jannson JSON object
+ *
+ */
+struct json_object {
+	json_t *root;
+	bool error;
+};
+
+struct json_object json_new_object(void);
+struct json_object json_new_array(void);
+void json_free(struct json_object *object);
+void json_assert_is_array(struct json_object *array);
+bool json_is_invalid(struct json_object *object);
+
+void json_add_int(
+	struct json_object *object,
+	const char* name,
+	const int value);
+void json_add_bool(
+	struct json_object *object,
+	const char* name,
+	const bool value);
+void json_add_string(
+	struct json_object *object,
+	const char* name,
+	const char* value);
+void json_add_object(
+	struct json_object *object,
+	const char* name,
+	struct json_object *value);
+void json_add_stringn(
+	struct json_object *object,
+	const char *name,
+	const char *value,
+	const size_t len);
+void json_add_version(
+	struct json_object *object,
+	int major,
+	int minor);
+void json_add_timestamp(struct json_object *object);
+void json_add_address(
+	struct json_object *object,
+	const char *name,
+	const struct tsocket_address *address);
+void json_add_sid(
+	struct json_object *object,
+	const char *name,
+	const struct dom_sid *sid);
+void json_add_guid(
+	struct json_object *object,
+	const char *name,
+	const struct GUID *guid);
+
+char *json_to_string(TALLOC_CTX *mem_ctx, struct json_object *object);
+#endif
diff --git a/lib/audit_logging/tests/audit_logging_test.c b/lib/audit_logging/tests/audit_logging_test.c
new file mode 100644
index 00000000000..7481df53ae0
--- /dev/null
+++ b/lib/audit_logging/tests/audit_logging_test.c
@@ -0,0 +1,557 @@
+/*
+ * Unit tests for the audit_logging library.
+ *
+ *  Copyright (C) Andrew Bartlett <abartlet at samba.org> 2018
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+
+/*
+ * from cmocka.c:
+ * These headers or their equivalents should be included prior to
+ * including
+ * this header file.
+ *
+ * #include <stdarg.h>
+ * #include <stddef.h>
+ * #include <setjmp.h>
+ *
+ * This allows test applications to use custom definitions of C standard
+ * library functions and types.
+ *
+ */
+
+/*
+ * Note that the messaging routines (audit_message_send and get_event_server)
+ * are not tested by these unit tests.  Currently they are for integration
+ * test support, and as such are exercised by the integration tests.
+ */
+#include <stdarg.h>
+#include <stddef.h>
+#include <setjmp.h>
+#include <cmocka.h>
+
+#include <string.h>
+#include <time.h>
+#include <tevent.h>
+#include <config.h>
+#include <talloc.h>
+#include "lib/util/talloc_stack.h"
+
+#include "lib/util/data_blob.h"
+#include "lib/util/time.h"
+#include "libcli/util/werror.h"
+#include "lib/param/loadparm.h"
+#include "libcli/security/dom_sid.h"
+#include "librpc/ndr/libndr.h"
+
+#include "lib/audit_logging/audit_logging.h"
+
+#ifdef HAVE_JANSSON
+static void test_json_add_int(void **state)
+{
+	struct json_object object;
+	struct json_t *value = NULL;
+	double n;
+
+	object = json_new_object();
+	json_add_int(&object, "positive_one", 1);
+	json_add_int(&object, "zero", 0);
+	json_add_int(&object, "negative_one", -1);
+
+
+	assert_int_equal(3, json_object_size(object.root));
+
+	value = json_object_get(object.root, "positive_one");
+	assert_true(json_is_integer(value));
+	n = json_number_value(value);
+	assert_true(n == 1.0);
+
+	value = json_object_get(object.root, "zero");
+	assert_true(json_is_integer(value));
+	n = json_number_value(value);
+	assert_true(n == 0.0);
+
+	value = json_object_get(object.root, "negative_one");
+	assert_true(json_is_integer(value));
+	n = json_number_value(value);
+	assert_true(n == -1.0);
+
+	json_free(&object);
+}
+
+static void test_json_add_bool(void **state)
+{
+	struct json_object object;
+	struct json_t *value = NULL;
+
+	object = json_new_object();
+	json_add_bool(&object, "true", true);
+	json_add_bool(&object, "false", false);
+
+
+	assert_int_equal(2, json_object_size(object.root));
+
+	value = json_object_get(object.root, "true");
+	assert_true(json_is_boolean(value));
+	assert_true(value == json_true());
+
+	value = json_object_get(object.root, "false");
+	assert_true(json_is_boolean(value));
+	assert_true(value == json_false());
+
+	json_free(&object);
+}
+
+static void test_json_add_string(void **state)
+{
+	struct json_object object;
+	struct json_t *value = NULL;
+	const char *s = NULL;
+
+	object = json_new_object();
+	json_add_string(&object, "null", NULL);
+	json_add_string(&object, "empty", "");
+	json_add_string(&object, "name", "value");
+
+
+
+	assert_int_equal(3, json_object_size(object.root));
+
+	value = json_object_get(object.root, "null");
+	assert_true(json_is_null(value));
+
+	value = json_object_get(object.root, "empty");
+	assert_true(json_is_string(value));
+	s = json_string_value(value);
+	assert_string_equal("", s);
+
+	value = json_object_get(object.root, "name");
+	assert_true(json_is_string(value));
+	s = json_string_value(value);
+	assert_string_equal("value", s);
+	json_free(&object);
+}
+
+static void test_json_add_object(void **state)
+{
+	struct json_object object;
+	struct json_object other;
+	struct json_t *value = NULL;
+
+	object = json_new_object();
+	other  = json_new_object();
+	json_add_object(&object, "null", NULL);
+	json_add_object(&object, "other", &other);
+
+
+
+	assert_int_equal(2, json_object_size(object.root));
+
+	value = json_object_get(object.root, "null");
+	assert_true(json_is_null(value));
+
+	value = json_object_get(object.root, "other");
+	assert_true(json_is_object(value));
+	assert_ptr_equal(other.root, value);
+
+	json_free(&object);
+}
+
+static void test_json_add_to_array(void **state)
+{
+	struct json_object array;
+	struct json_object o1;
+	struct json_object o2;
+	struct json_object o3;
+	struct json_t *value = NULL;
+
+	array = json_new_array();
+	assert_true(json_is_array(array.root));
+
+	o1 = json_new_object();
+	o2 = json_new_object();
+	o3 = json_new_object();
+
+	json_add_object(&array, NULL, &o3);
+	json_add_object(&array, "", &o2);
+	json_add_object(&array, "will-be-ignored", &o1);
+	json_add_object(&array, NULL, NULL);
+
+	assert_int_equal(4, json_array_size(array.root));
+
+	value = json_array_get(array.root, 0);
+	assert_ptr_equal(o3.root, value);
+
+	value = json_array_get(array.root, 1);
+	assert_ptr_equal(o2.root, value);
+
+	value = json_array_get(array.root, 2);
+	assert_ptr_equal(o1.root, value);
+
+	value = json_array_get(array.root, 3);
+	assert_true(json_is_null(value));
+
+	json_free(&array);
+
+}
+
+static void test_json_add_timestamp(void **state)
+{
+	struct json_object object;
+	struct json_t *ts = NULL;
+	const char *t = NULL;
+	int rc;
+	int usec, tz;
+	char c[2];
+	struct tm tm;
+	time_t before;
+	time_t after;
+	time_t actual;
+
+
+	object = json_new_object();
+	before = time(NULL);
+	json_add_timestamp(&object);
+	after = time(NULL);
+
+	ts = json_object_get(object.root, "timestamp");
+	assert_true(json_is_string(ts));
+
+	/*
+	 * Convert the returned ISO 8601 timestamp into a time_t
+	 * Note for convenience we ignore the value of the microsecond
+	 * part of the time stamp.
+	 */
+	t = json_string_value(ts);
+ 	rc = sscanf(
+		t,
+		"%4d-%2d-%2dT%2d:%2d:%2d.%6d%1c%4d",
+		&tm.tm_year,
+		&tm.tm_mon,
+		&tm.tm_mday,
+		&tm.tm_hour,
+		&tm.tm_min,
+		&tm.tm_sec,
+		&usec,
+		c,
+		&tz);
+	assert_int_equal(9, rc);
+	tm.tm_year = tm.tm_year - 1900;
+	tm.tm_mon = tm.tm_mon - 1;
+	tm.tm_isdst = -1;
+	actual = mktime(&tm);
+
+	/*
+	 * The timestamp should be before <= actual <= after
+	 */
+	assert_true(difftime(actual, before) >= 0);
+	assert_true(difftime(after, actual) >= 0);
+
+	json_free(&object);
+}
+
+static void test_json_add_stringn(void **state)
+{
+	struct json_object object;
+	struct json_t *value = NULL;
+	const char *s = NULL;
+
+	object = json_new_object();
+	json_add_stringn(&object, "null", NULL, 10);
+	json_add_stringn(&object, "null-zero-len", NULL, 0);
+	json_add_stringn(&object, "empty", "", 1);
+	json_add_stringn(&object, "empty-zero-len", "", 0);
+	json_add_stringn(&object, "value-less-than-len", "123456", 7);
+	json_add_stringn(&object, "value-greater-than-len", "abcd", 3);
+	json_add_stringn(&object, "value-equal-len", "ZYX", 3);
+	json_add_stringn(&object, "value-len-is-zero", "this will be null", 0);
+
+
+	assert_int_equal(8, json_object_size(object.root));
+
+	value = json_object_get(object.root, "null");
+	assert_true(json_is_null(value));
+
+	value = json_object_get(object.root, "null-zero-len");
+	assert_true(json_is_null(value));
+
+	value = json_object_get(object.root, "empty");
+	assert_true(json_is_string(value));
+	s = json_string_value(value);
+	assert_string_equal("", s);
+
+	value = json_object_get(object.root, "empty-zero-len");
+	assert_true(json_is_null(value));
+
+	value = json_object_get(object.root, "value-greater-than-len");
+	assert_true(json_is_string(value));
+	s = json_string_value(value);
+	assert_string_equal("abc", s);
+	assert_int_equal(3, strlen(s));
+
+	value = json_object_get(object.root, "value-equal-len");
+	assert_true(json_is_string(value));
+	s = json_string_value(value);
+	assert_string_equal("ZYX", s);
+	assert_int_equal(3, strlen(s));
+
+	value = json_object_get(object.root, "value-len-is-zero");
+	assert_true(json_is_null(value));
+
+	json_free(&object);
+}
+
+static void test_json_add_version(void **state)
+{
+	struct json_object object;
+	struct json_t *version = NULL;
+	struct json_t *v = NULL;
+	double n;
+
+	object = json_new_object();
+	json_add_version(&object, 3, 1);
+
+	assert_int_equal(1, json_object_size(object.root));
+
+	version = json_object_get(object.root, "version");
+	assert_true(json_is_object(version));
+	assert_int_equal(2, json_object_size(version));
+
+	v = json_object_get(version, "major");
+	assert_true(json_is_integer(v));
+	n = json_number_value(v);
+	assert_true(n == 3.0);
+
+	v = json_object_get(version, "minor");
+	assert_true(json_is_integer(v));
+	n = json_number_value(v);
+	assert_true(n == 1.0);
+
+	json_free(&object);
+}
+
+static void test_json_add_address(void **state)
+{
+	struct json_object object;
+	struct json_t *value = NULL;
+	struct tsocket_address *ip4  = NULL;
+	struct tsocket_address *ip6  = NULL;
+	struct tsocket_address *pipe = NULL;
+	const char *s = NULL;
+	int rc;
+
+	TALLOC_CTX *ctx = talloc_new(NULL);
+
+	object = json_new_object();
+
+	json_add_address(&object, "null", NULL);
+
+	rc = tsocket_address_inet_from_strings(
+		ctx,
+		"ip",
+		"127.0.0.1",
+		21,
+		&ip4);
+	assert_int_equal(0, rc);
+	json_add_address(&object, "ip4", ip4);
+
+	rc = tsocket_address_inet_from_strings(
+		ctx,
+		"ip",
+		"2001:db8:0:0:1:0:0:1",
+		42,
+		&ip6);
+	assert_int_equal(0, rc);
+	json_add_address(&object, "ip6", ip6);
+
+	rc = tsocket_address_unix_from_path(ctx, "/samba/pipe", &pipe);
+	assert_int_equal(0, rc);
+	json_add_address(&object, "pipe", pipe);
+
+	assert_int_equal(4, json_object_size(object.root));
+
+	value = json_object_get(object.root, "null");
+	assert_true(json_is_null(value));
+
+	value = json_object_get(object.root, "ip4");
+	assert_true(json_is_string(value));
+	s = json_string_value(value);
+	assert_string_equal("ipv4:127.0.0.1:21", s);
+
+	value = json_object_get(object.root, "ip6");
+	assert_true(json_is_string(value));
+	s = json_string_value(value);
+	assert_string_equal("ipv6:2001:db8::1:0:0:1:42", s);
+
+	value = json_object_get(object.root, "pipe");
+	assert_true(json_is_string(value));
+	s = json_string_value(value);
+	assert_string_equal("unix:/samba/pipe", s);
+
+	json_free(&object);
+	TALLOC_FREE(ctx);
+}
+
+static void test_json_add_sid(void **state)
+{
+	struct json_object object;
+	struct json_t *value = NULL;
+	const char *SID = "S-1-5-21-2470180966-3899876309-2637894779";
+	struct dom_sid sid;
+	const char *s = NULL;
+
+
+	object = json_new_object();
+
+	json_add_sid(&object, "null", NULL);
+
+	assert_true(string_to_sid(&sid, SID));
+	json_add_sid(&object, "sid", &sid);
+
+	assert_int_equal(2, json_object_size(object.root));
+
+	value = json_object_get(object.root, "null");
+	assert_true(json_is_null(value));
+
+	value = json_object_get(object.root, "sid");
+	assert_true(json_is_string(value));
+	s = json_string_value(value);
+	assert_string_equal(SID, s);
+	json_free(&object);
+}
+
+static void test_json_add_guid(void **state)
+{
+	struct json_object object;
+	struct json_t *value = NULL;
+	const char *GUID = "3ab88633-1e57-4c1a-856c-d1bc4b15bbb1";
+	struct GUID guid;
+	const char *s = NULL;
+	NTSTATUS status;
+
+
+	object = json_new_object();
+
+	json_add_guid(&object, "null", NULL);
+
+	status = GUID_from_string(GUID, &guid);
+	assert_true(NT_STATUS_IS_OK(status));
+	json_add_guid(&object, "guid", &guid);
+
+	assert_int_equal(2, json_object_size(object.root));
+
+	value = json_object_get(object.root, "null");
+	assert_true(json_is_null(value));
+
+	value = json_object_get(object.root, "guid");
+	assert_true(json_is_string(value));
+	s = json_string_value(value);
+	assert_string_equal(GUID, s);
+
+	json_free(&object);
+}
+
+static void test_json_to_string(void **state)
+{
+	struct json_object object;
+	char *s = NULL;
+
+	TALLOC_CTX *ctx = talloc_new(NULL);
+
+	object = json_new_object();
+	object.error = true;
+
+	s = json_to_string(ctx, &object);
+	assert_null(s);
+
+	object.error = false;
+	s = json_to_string(ctx, &object);
+	assert_string_equal("{}", s);
+	TALLOC_FREE(s);
+
+	json_add_string(&object, "name", "value");
+	s = json_to_string(ctx, &object);
+	assert_string_equal("{\"name\": \"value\"}", s);
+	TALLOC_FREE(s);
+
+	json_free(&object);
+	TALLOC_FREE(ctx);
+}
+#endif
+
+static void test_audit_get_timestamp(void **state)
+{
+	const char *t = NULL;
+	char *c;
+	struct tm tm;
+	time_t before;
+	time_t after;
+	time_t actual;
+
+	TALLOC_CTX *ctx = talloc_new(NULL);
+
+	before = time(NULL);
+	t = audit_get_timestamp(ctx);
+	after = time(NULL);
+
+
+	c = strptime(t, "%a, %d %b %Y %H:%M:%S", &tm);
+	tm.tm_isdst = -1;
+	if (c != NULL && *c == '.') {
+		char *e;
+		strtod(c, &e);
+		c = e;
+	}
+	if (c != NULL && *c == ' ') {
+		struct tm tz;
+		c = strptime(c, " %Z", &tz);
+	}
+	assert_int_equal(0, strlen(c));
+
+	actual = mktime(&tm);
+
+	/*
+	 * The timestamp should be before <= actual <= after
+	 */
+	assert_true(difftime(actual, before) >= 0);
+	assert_true(difftime(after, actual) >= 0);
+
+	TALLOC_FREE(ctx);
+}
+
+int main(int argc, const char **argv)
+{
+	const struct CMUnitTest tests[] = {
+#ifdef HAVE_JANSSON
+		cmocka_unit_test(test_json_add_int),
+		cmocka_unit_test(test_json_add_bool),
+		cmocka_unit_test(test_json_add_string),
+		cmocka_unit_test(test_json_add_object),
+		cmocka_unit_test(test_json_add_to_array),
+		cmocka_unit_test(test_json_add_timestamp),
+		cmocka_unit_test(test_json_add_stringn),
+		cmocka_unit_test(test_json_add_version),
+		cmocka_unit_test(test_json_add_address),
+		cmocka_unit_test(test_json_add_sid),
+		cmocka_unit_test(test_json_add_guid),
+		cmocka_unit_test(test_json_to_string),
+#endif
+		cmocka_unit_test(test_audit_get_timestamp),
+	};
+
+	cmocka_set_message_output(CM_OUTPUT_SUBUNIT);
+	return cmocka_run_group_tests(tests, NULL, NULL);
+}
diff --git a/lib/audit_logging/wscript_build b/lib/audit_logging/wscript_build
new file mode 100644
index 00000000000..4022d9031f6
--- /dev/null
+++ b/lib/audit_logging/wscript_build
@@ -0,0 +1,24 @@
+#!/usr/bin/env python
+
+bld.SAMBA_SUBSYSTEM(
+    'audit_logging',
+    deps='''MESSAGING_SEND
+            jansson
+            samba-debug
+            LIBTSOCKET''',
+    source='audit_logging.c'
+)
+
+bld.SAMBA_BINARY(
+    'audit_logging_test',
+    source='tests/audit_logging_test.c',
+    deps='''
+        audit_logging
+        jansson
+        cmocka
+        talloc
+        samba-util
+        LIBTSOCKET
+    ''',
+    install=False
+)
diff --git a/source4/selftest/tests.py b/source4/selftest/tests.py
index 97401180388..b1d91ef9935 100755
--- a/source4/selftest/tests.py
+++ b/source4/selftest/tests.py
@@ -1065,3 +1065,5 @@ plantestsuite("samba4.dsdb.samdb.ldb_modules.unique_object_sids" , "none",
               [os.path.join(bindir(), "test_unique_object_sids")])
 plantestsuite("samba4.dsdb.samdb.ldb_modules.encrypted_secrets", "none",
                   [os.path.join(bindir(), "test_encrypted_secrets")])
+plantestsuite("lib.audit_logging.audit_logging", "none",
+                  [os.path.join(bindir(), "audit_logging_test")])
diff --git a/wscript_build b/wscript_build
index 8a59bdf6cab..54548769b19 100644
--- a/wscript_build
+++ b/wscript_build
@@ -46,6 +46,7 @@ bld.RECURSE('lib/texpect')
 bld.RECURSE('lib/addns')
 bld.RECURSE('lib/ldb')
 bld.RECURSE('lib/param')
+bld.RECURSE('lib/audit_logging')
 bld.RECURSE('dynconfig')
 bld.RECURSE('lib/util/charset')
 bld.RECURSE('python')
-- 
2.11.0


From db0abddc867bc25ef6dfd4c2879f890b106337a9 Mon Sep 17 00:00:00 2001
From: Gary Lockyer <gary at catalyst.net.nz>
Date: Wed, 4 Apr 2018 11:56:30 +1200
Subject: [PATCH 02/10] logging: add ldb audit classes

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
---
 docs-xml/smbdotconf/logging/loglevel.xml | 28 +++++++++++++++++++++++++---
 lib/util/debug.c                         |  6 ++++++
 lib/util/debug.h                         |  6 ++++++
 3 files changed, 37 insertions(+), 3 deletions(-)

diff --git a/docs-xml/smbdotconf/logging/loglevel.xml b/docs-xml/smbdotconf/logging/loglevel.xml
index fae5c7b079e..4e9efc03990 100644
--- a/docs-xml/smbdotconf/logging/loglevel.xml
+++ b/docs-xml/smbdotconf/logging/loglevel.xml
@@ -46,6 +46,12 @@
         <listitem><para><parameter moreinfo="none">auth_audit</parameter></para></listitem>
         <listitem><para><parameter moreinfo="none">auth_json_audit</parameter></para></listitem>
         <listitem><para><parameter moreinfo="none">kerberos</parameter></para></listitem>
+        <listitem><para><parameter moreinfo="none">samdb_audit</parameter></para></listitem>
+        <listitem><para><parameter moreinfo="none">samdb_json_audit</parameter></para></listitem>
+        <listitem><para><parameter moreinfo="none">password_audit</parameter></para></listitem>
+        <listitem><para><parameter moreinfo="none">password_json_audit</parameter></para></listitem>
+        <listitem><para><parameter moreinfo="none">transaction_audit</parameter></para></listitem>
+        <listitem><para><parameter moreinfo="none">transaction_json_audit</parameter></para></listitem>
     </itemizedlist>
 
     <para>Authentication and authorization audit information is logged
@@ -58,7 +64,7 @@
     as well as the implicit authentication in password changes.  In
     the file server, NTLM authentication, SMB and RPC authorization is
     covered.</para>
-    
+
     <para>Log levels for auth_audit and auth_audit_json are:</para>
     <itemizedlist>
 	<listitem><para>2: Authentication Failure</para></listitem>
@@ -66,9 +72,25 @@
 	<listitem><para>4: Authorization Success</para></listitem>
 	<listitem><para>5: Anonymous Authentication and Authorization Success</para></listitem>
     </itemizedlist>
-    
 
-    
+    <para>Changes to the samdb database are logged
+    under the samdb_audit, and if Samba is compiled against the jansson
+    JSON library, a JSON representation is logged under
+    samdb_json_audit.</para>
+
+    <para>Password changes and Password resets are logged under
+    the password_audit, and if Samba is compiled against the jansson
+    JSON library, a JSON representation is logged under the
+    password_json_audit.</para>
+
+    <para>Transaction rollbacks and prepare commit failures are logged under
+    the transaction_audit the password_audit, and if Samba is compiled against
+    the jansson JSON library, a JSON representation is logged under the
+    password_json_audit. Logging the transaction details allows the
+    identification of password and samdb operations that have been rolled
+    back.</para>
+
+
 </description>
 <value type="default">0</value>
 <value type="example">3 passdb:5 auth:10 winbind:2</value>
diff --git a/lib/util/debug.c b/lib/util/debug.c
index d010b724203..db4f71a1501 100644
--- a/lib/util/debug.c
+++ b/lib/util/debug.c
@@ -543,6 +543,12 @@ static const char *default_classname_table[] = {
 	[DBGC_DRS_REPL] =       "drs_repl",
 	[DBGC_SMB2] =           "smb2",
 	[DBGC_SMB2_CREDITS] =   "smb2_credits",
+	[DBGC_SAMDB_AUDIT]  =	"samdb_audit",
+	[DBGC_SAMDB_AUDIT_JSON] = "samdb_json_audit",
+	[DBGC_PWD_AUDIT]  =	"password_audit",
+	[DBGC_PWD_AUDIT_JSON] = "password_json_audit",
+	[DBGC_TRN_AUDIT]  =	"transaction_audit",
+	[DBGC_TRN_AUDIT_JSON] = "transaction_json_audit",
 };
 
 /*
diff --git a/lib/util/debug.h b/lib/util/debug.h
index 1e184b47de9..4e425a0e0f8 100644
--- a/lib/util/debug.h
+++ b/lib/util/debug.h
@@ -95,6 +95,12 @@ bool dbghdr( int level, const char *location, const char *func);
 #define DBGC_DRS_REPL           27
 #define DBGC_SMB2               28
 #define DBGC_SMB2_CREDITS       29
+#define DBGC_SAMDB_AUDIT	30
+#define DBGC_SAMDB_AUDIT_JSON	31
+#define DBGC_PWD_AUDIT		32
+#define DBGC_PWD_AUDIT_JSON	33
+#define DBGC_TRN_AUDIT		34
+#define DBGC_TRN_AUDIT_JSON	35
 
 /* So you can define DBGC_CLASS before including debug.h */
 #ifndef DBGC_CLASS
-- 
2.11.0


From aa80a829f4e4b78d6160f67922dbf5c6cd4b77f9 Mon Sep 17 00:00:00 2001
From: Gary Lockyer <gary at catalyst.net.nz>
Date: Thu, 12 Apr 2018 10:19:16 +1200
Subject: [PATCH 03/10] smb conf: Add samdb event notification parameter

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
---
 .../smbdotconf/misc/passwordeventnotification.xml  | 27 ++++++++++++++++++++++
 .../smbdotconf/misc/samdbeventnotification.xml     | 27 ++++++++++++++++++++++
 2 files changed, 54 insertions(+)
 create mode 100644 docs-xml/smbdotconf/misc/passwordeventnotification.xml
 create mode 100644 docs-xml/smbdotconf/misc/samdbeventnotification.xml

diff --git a/docs-xml/smbdotconf/misc/passwordeventnotification.xml b/docs-xml/smbdotconf/misc/passwordeventnotification.xml
new file mode 100644
index 00000000000..e83f05d80f0
--- /dev/null
+++ b/docs-xml/smbdotconf/misc/passwordeventnotification.xml
@@ -0,0 +1,27 @@
+<samba:parameter name="password event notification"
+                 context="G"
+                 type="boolean"
+                 xmlns:samba="http://www.samba.org/samba/DTD/samba-doc">
+<description>
+	<para>When enabled, this option causes Samba (acting as an
+	Active Directory Domain Controller) to stream password change
+	and reset events across the internal message bus.
+	Scripts built using Samba's python bindings can listen to these
+	events by registering as the service
+	<filename moreinfo="none">password_event</filename>.</para>
+
+	<para>This should be considered a developer option (it assists
+	in the Samba testsuite) rather than a facility for external
+	auditing, as message delivery is not guaranteed (a feature
+	that the testsuite works around).  Additionally Samba must be
+	compiled with the jansson support for this option to be
+	effective.</para>
+
+	<para>The password events are also logged via the normal
+	logging methods when the <smbconfoption name="log level"/> is
+	set appropriately.</para>
+
+</description>
+
+<value type="default">no</value>
+</samba:parameter>
diff --git a/docs-xml/smbdotconf/misc/samdbeventnotification.xml b/docs-xml/smbdotconf/misc/samdbeventnotification.xml
new file mode 100644
index 00000000000..a1d7aca074c
--- /dev/null
+++ b/docs-xml/smbdotconf/misc/samdbeventnotification.xml
@@ -0,0 +1,27 @@
+<samba:parameter name="samdb event notification"
+                 context="G"
+                 type="boolean"
+                 xmlns:samba="http://www.samba.org/samba/DTD/samba-doc">
+<description>
+	<para>When enabled, this option causes Samba (acting as an
+	Active Directory Domain Controller) to stream Samba database
+	events across the internal message bus.  Scripts built using
+	Samba's python bindings can listen to these events by
+	registering as the service
+	<filename moreinfo="none">samdb_event</filename>.</para>
+
+	<para>This should be considered a developer option (it assists
+	in the Samba testsuite) rather than a facility for external
+	auditing, as message delivery is not guaranteed (a feature
+	that the testsuite works around).  Additionally Samba must be
+	compiled with the jansson support for this option to be
+	effective.</para>
+
+	<para>The Samba database events are also logged via the normal
+	logging methods when the <smbconfoption name="log level"/> is
+	set appropriately.</para>
+
+</description>
+
+<value type="default">no</value>
+</samba:parameter>
-- 
2.11.0


From 0770e739a0a2f7128160970d9c6502be6a82a1c0 Mon Sep 17 00:00:00 2001
From: Gary Lockyer <gary at catalyst.net.nz>
Date: Thu, 12 Apr 2018 13:19:16 +1200
Subject: [PATCH 04/10] idl messaging: Add Samdb and Password events and
 message types

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
---
 librpc/idl/messaging.idl | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/librpc/idl/messaging.idl b/librpc/idl/messaging.idl
index 14a6f92d583..b7454404d50 100644
--- a/librpc/idl/messaging.idl
+++ b/librpc/idl/messaging.idl
@@ -140,8 +140,13 @@ interface messaging
 		MSG_NTVFS_OPLOCK_BREAK          = 0x0703,
 		MSG_DREPL_ALLOCATE_RID          = 0x0704,
 
-		/* Called during authentication and authorization to allow out-of- */
+		/*
+		 * Audit, Authentication and Authorisation event
+		 * messages
+		 */
 		MSG_AUTH_LOG                    = 0x0800,
+		MSG_SAMDB_LOG                   = 0x0801,
+		MSG_PWD_LOG                     = 0x0802,
 
 		/* dbwrap messages 4001-4999 (0x0FA0 - 0x1387) */
 		/* MSG_DBWRAP_TDB2_CHANGES		= 4001, */
@@ -178,5 +183,7 @@ interface messaging
 	} messaging_reclog;
 
         /* This allows this well known service name to be referenced in python and C */
-        const string AUTH_EVENT_NAME = "auth_event";
+        const string AUTH_EVENT_NAME  = "auth_event";
+	const string SAMDB_EVENT_NAME = "samdb_event";
+	const string PWD_EVENT_NAME   = "password_event";
 }
-- 
2.11.0


From d413d1f6efe7b2781c237d3dcc4f129d43640d3f Mon Sep 17 00:00:00 2001
From: Gary Lockyer <gary at catalyst.net.nz>
Date: Tue, 10 Apr 2018 11:45:32 +1200
Subject: [PATCH 05/10] auth_log: Use common code from audit_logging

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
---
 auth/auth_log.c    | 507 ++++++++++-------------------------------------------
 auth/wscript_build |   2 +-
 2 files changed, 94 insertions(+), 415 deletions(-)

diff --git a/auth/auth_log.c b/auth/auth_log.c
index 97b65371bdc..ab44e7d1803 100644
--- a/auth/auth_log.c
+++ b/auth/auth_log.c
@@ -57,48 +57,7 @@
 #include "lib/util/server_id_db.h"
 #include "lib/param/param.h"
 #include "librpc/ndr/libndr.h"
-
-/*
- * Get a human readable timestamp.
- *
- * Returns the current time formatted as
- *  "Tue, 14 Mar 2017 08:38:42.209028 NZDT"
- *
- * The returned string is allocated by talloc in the supplied context.
- * It is the callers responsibility to free it.
- *
- */
-static const char* get_timestamp(TALLOC_CTX *frame)
-{
-	char buffer[40];	/* formatted time less usec and timezone */
-	char tz[10];		/* formatted time zone			 */
-	struct tm* tm_info;	/* current local time			 */
-	struct timeval tv;	/* current system time			 */
-	int r;			/* response code from gettimeofday	 */
-	const char * ts;	/* formatted time stamp			 */
-
-	r = gettimeofday(&tv, NULL);
-	if (r) {
-		DBG_ERR("Unable to get time of day: (%d) %s\n",
-			errno,
-			strerror(errno));
-		return NULL;
-	}
-
-	tm_info = localtime(&tv.tv_sec);
-	if (tm_info == NULL) {
-		DBG_ERR("Unable to determine local time\n");
-		return NULL;
-	}
-
-	strftime(buffer, sizeof(buffer)-1, "%a, %d %b %Y %H:%M:%S", tm_info);
-	strftime(tz, sizeof(tz)-1, "%Z", tm_info);
-	ts = talloc_asprintf(frame, "%s.%06ld %s", buffer, tv.tv_usec, tz);
-	if (ts == NULL) {
-		DBG_ERR("Out of memory formatting time stamp\n");
-	}
-	return ts;
-}
+#include "lib/audit_logging/audit_logging.h"
 
 /*
  * Determine the type of the password supplied for the
@@ -113,97 +72,12 @@ static const char* get_password_type(const struct auth_usersupplied_info *ui);
 #include "system/time.h"
 
 /*
- * Context required by the JSON generation
- *  routines
- *
- */
-struct json_context {
-	json_t *root;
-	bool error;
-};
-
-static NTSTATUS get_auth_event_server(struct imessaging_context *msg_ctx,
-				      struct server_id *auth_event_server)
-{
-	NTSTATUS status;
-	TALLOC_CTX *frame = talloc_stackframe();
-	unsigned num_servers, i;
-	struct server_id *servers;
-
-	status = irpc_servers_byname(msg_ctx, frame,
-				     AUTH_EVENT_NAME,
-				     &num_servers, &servers);
-
-	if (!NT_STATUS_IS_OK(status)) {
-		DBG_NOTICE("Failed to find 'auth_event' registered on the "
-			   "message bus to send JSON authentication events to: %s\n",
-			   nt_errstr(status));
-		TALLOC_FREE(frame);
-		return status;
-	}
-
-	/*
-	 * Select the first server that is listening, because
-	 * we get connection refused as
-	 * NT_STATUS_OBJECT_NAME_NOT_FOUND without waiting
-	 */
-	for (i = 0; i < num_servers; i++) {
-		status = imessaging_send(msg_ctx, servers[i], MSG_PING,
-					 &data_blob_null);
-		if (NT_STATUS_IS_OK(status)) {
-			*auth_event_server = servers[i];
-			TALLOC_FREE(frame);
-			return NT_STATUS_OK;
-		}
-	}
-	DBG_NOTICE("Failed to find a running 'auth_event' server "
-		   "registered on the message bus to send JSON "
-		   "authentication events to\n");
-	TALLOC_FREE(frame);
-	return NT_STATUS_OBJECT_NAME_NOT_FOUND;
-}
-
-static void auth_message_send(struct imessaging_context *msg_ctx,
-			      const char *json)
-{
-	struct server_id auth_event_server;
-	NTSTATUS status;
-	DATA_BLOB json_blob = data_blob_string_const(json);
-	if (msg_ctx == NULL) {
-		return;
-	}
-
-	/* Need to refetch the address each time as the destination server may
-	 * have disconnected and reconnected in the interim, in which case
-	 * messages may get lost, manifests in the auth_log tests
-	 */
-	status = get_auth_event_server(msg_ctx, &auth_event_server);
-	if (!NT_STATUS_IS_OK(status)) {
-		return;
-	}
-
-	status = imessaging_send(msg_ctx, auth_event_server, MSG_AUTH_LOG,
-				 &json_blob);
-
-	/* If the server crashed, try to find it again */
-	if (NT_STATUS_EQUAL(status, NT_STATUS_OBJECT_NAME_NOT_FOUND)) {
-		status = get_auth_event_server(msg_ctx, &auth_event_server);
-		if (!NT_STATUS_IS_OK(status)) {
-			return;
-		}
-		imessaging_send(msg_ctx, auth_event_server, MSG_AUTH_LOG,
-				&json_blob);
-
-	}
-}
-
-/*
  * Write the json object to the debug logs.
  *
  */
 static void log_json(struct imessaging_context *msg_ctx,
 		     struct loadparm_context *lp_ctx,
-		     struct json_context *context,
+		     struct json_object *context,
 		     const char *type, int debug_class, int debug_level)
 {
 	char* json = NULL;
@@ -221,7 +95,10 @@ static void log_json(struct imessaging_context *msg_ctx,
 
 	DEBUGC(debug_class, debug_level, ("JSON %s: %s\n", type, json));
 	if (msg_ctx && lp_ctx && lpcfg_auth_event_notification(lp_ctx)) {
-		auth_message_send(msg_ctx, json);
+		audit_message_send(msg_ctx,
+				   AUTH_EVENT_NAME,
+				   MSG_AUTH_LOG,
+				   json);
 	}
 
 	if (json) {
@@ -231,227 +108,6 @@ static void log_json(struct imessaging_context *msg_ctx,
 }
 
 /*
- * Create a new json logging context.
- *
- * Free with a call to free_json_context
- *
- */
-static struct json_context get_json_context(void) {
-
-	struct json_context context;
-	context.error = false;
-
-	context.root = json_object();
-	if (context.root == NULL) {
-		context.error = true;
-		DBG_ERR("Unable to create json_object\n");
-	}
-	return context;
-}
-
-/*
- * free a previously created json_context
- *
- */
-static void free_json_context(struct json_context *context)
-{
-	if (context->root) {
-		json_decref(context->root);
-	}
-}
-
-/*
- * Output a JSON pair with name name and integer value value
- *
- */
-static void add_int(struct json_context *context,
-		    const char* name,
-		    const int value)
-{
-	int rc = 0;
-
-	if (context->error) {
-		return;
-	}
-
-	rc = json_object_set_new(context->root, name, json_integer(value));
-	if (rc) {
-		DBG_ERR("Unable to set name [%s] value [%d]\n", name, value);
-		context->error = true;
-	}
-
-}
-
-/*
- * Output a JSON pair with name name and string value value
- *
- */
-static void add_string(struct json_context *context,
-		       const char* name,
-		       const char* value)
-{
-	int rc = 0;
-
-	if (context->error) {
-		return;
-	}
-
-	if (value) {
-		rc = json_object_set_new(context->root, name, json_string(value));
-	} else {
-		rc = json_object_set_new(context->root, name, json_null());
-	}
-	if (rc) {
-		DBG_ERR("Unable to set name [%s] value [%s]\n", name, value);
-		context->error = true;
-	}
-}
-
-
-/*
- * Output a JSON pair with name name and object value
- *
- */
-static void add_object(struct json_context *context,
-		       const char* name,
-		       struct json_context *value)
-{
-	int rc = 0;
-
-	if (value->error) {
-		context->error = true;
-	}
-	if (context->error) {
-		return;
-	}
-	rc = json_object_set_new(context->root, name, value->root);
-	if (rc) {
-		DBG_ERR("Unable to add object [%s]\n", name);
-		context->error = true;
-	}
-}
-
-/*
- * Output a version object
- *
- * "version":{"major":1,"minor":0}
- *
- */
-static void add_version(struct json_context *context, int major, int minor)
-{
-	struct json_context version = get_json_context();
-	add_int(&version, "major", major);
-	add_int(&version, "minor", minor);
-	add_object(context, "version", &version);
-}
-
-/*
- * Output the current date and time as a timestamp in ISO 8601 format
- *
- * "timestamp":"2017-03-06T17:18:04.455081+1300"
- *
- */
-static void add_timestamp(struct json_context *context)
-{
-	char buffer[40];	/* formatted time less usec and timezone */
-	char timestamp[50];	/* the formatted ISO 8601 time stamp	 */
-	char tz[10];		/* formatted time zone			 */
-	struct tm* tm_info;	/* current local time			 */
-	struct timeval tv;	/* current system time			 */
-	int r;			/* response code from gettimeofday	 */
-
-	if (context->error) {
-		return;
-	}
-
-	r = gettimeofday(&tv, NULL);
-	if (r) {
-		DBG_ERR("Unable to get time of day: (%d) %s\n",
-			errno,
-			strerror(errno));
-		context->error = true;
-		return;
-	}
-
-	tm_info = localtime(&tv.tv_sec);
-	if (tm_info == NULL) {
-		DBG_ERR("Unable to determine local time\n");
-		context->error = true;
-		return;
-	}
-
-	strftime(buffer, sizeof(buffer)-1, "%Y-%m-%dT%T", tm_info);
-	strftime(tz, sizeof(tz)-1, "%z", tm_info);
-	snprintf(timestamp, sizeof(timestamp),"%s.%06ld%s",
-		 buffer, tv.tv_usec, tz);
-	add_string(context,"timestamp", timestamp);
-}
-
-
-/*
- * Output an address pair, with name name.
- *
- * "localAddress":"ipv6::::0"
- *
- */
-static void add_address(struct json_context *context,
-			const char *name,
-			const struct tsocket_address *address)
-{
-	char *s = NULL;
-	TALLOC_CTX *frame = talloc_stackframe();
-
-	if (context->error) {
-		return;
-	}
-
-	s = tsocket_address_string(address, frame);
-	add_string(context, name, s);
-	talloc_free(frame);
-
-}
-
-/*
- * Output a SID with name name
- *
- * "sid":"S-1-5-18"
- *
- */
-static void add_sid(struct json_context *context,
-		    const char *name,
-		    const struct dom_sid *sid)
-{
-	char sid_buf[DOM_SID_STR_BUFLEN];
-
-	if (context->error) {
-		return;
-	}
-
-	dom_sid_string_buf(sid, sid_buf, sizeof(sid_buf));
-	add_string(context, name, sid_buf);
-}
-
-/*
- * Add a formatted string representation of a GUID to a json object.
- *
- */
-static void add_guid(struct json_context *context,
-		     const char *name,
-		     struct GUID *guid)
-{
-
-	char *guid_str;
-	struct GUID_txt_buf guid_buff;
-
-	if (context->error) {
-		return;
-	}
-
-	guid_str = GUID_buf_string(guid, &guid_buff);
-	add_string(context, name, guid_str);
-}
-
-/*
  * Write a machine parsable json formatted authentication log entry.
  *
  * IF removing or changing the format/meaning of a field please update the
@@ -482,49 +138,63 @@ static void log_authentication_event_json(
 			struct dom_sid *sid,
 			int debug_level)
 {
-	struct json_context context = get_json_context();
-	struct json_context authentication;
+	struct json_object context = json_new_object();
+	struct json_object authentication;
 	char negotiate_flags[11];
 
-	add_timestamp(&context);
-	add_string(&context, "type", AUTH_JSON_TYPE);
-
-	authentication = get_json_context();
-	add_version(&authentication, AUTH_MAJOR, AUTH_MINOR);
-	add_string(&authentication, "status", nt_errstr(status));
-	add_address(&authentication, "localAddress", ui->local_host);
-	add_address(&authentication, "remoteAddress", ui->remote_host);
-	add_string(&authentication,
-		   "serviceDescription",
-		   ui->service_description);
-	add_string(&authentication, "authDescription", ui->auth_description);
-	add_string(&authentication, "clientDomain", ui->client.domain_name);
-	add_string(&authentication, "clientAccount", ui->client.account_name);
-	add_string(&authentication, "workstation", ui->workstation_name);
-	add_string(&authentication, "becameAccount", account_name);
-	add_string(&authentication, "becameDomain", domain_name);
-	add_sid(&authentication, "becameSid", sid);
-	add_string(&authentication, "mappedAccount", ui->mapped.account_name);
-	add_string(&authentication, "mappedDomain", ui->mapped.domain_name);
-	add_string(&authentication,
-		   "netlogonComputer",
-		   ui->netlogon_trust_account.computer_name);
-	add_string(&authentication,
-		   "netlogonTrustAccount",
-		   ui->netlogon_trust_account.account_name);
+	json_add_timestamp(&context);
+	json_add_string(&context, "type", AUTH_JSON_TYPE);
+
+	authentication = json_new_object();
+	json_add_version(&authentication, AUTH_MAJOR, AUTH_MINOR);
+	json_add_string(&authentication, "status", nt_errstr(status));
+	json_add_address(&authentication, "localAddress", ui->local_host);
+	json_add_address(&authentication, "remoteAddress", ui->remote_host);
+	json_add_string(&authentication,
+			"serviceDescription",
+			ui->service_description);
+	json_add_string(&authentication,
+			"authDescription",
+			ui->auth_description);
+	json_add_string(&authentication,
+			"clientDomain",
+			ui->client.domain_name);
+	json_add_string(&authentication,
+			"clientAccount",
+			ui->client.account_name);
+	json_add_string(&authentication,
+			"workstation",
+			ui->workstation_name);
+	json_add_string(&authentication, "becameAccount", account_name);
+	json_add_string(&authentication, "becameDomain", domain_name);
+	json_add_sid(&authentication, "becameSid", sid);
+	json_add_string(&authentication,
+			"mappedAccount",
+			ui->mapped.account_name);
+	json_add_string(&authentication,
+			"mappedDomain",
+			ui->mapped.domain_name);
+	json_add_string(&authentication,
+			"netlogonComputer",
+			ui->netlogon_trust_account.computer_name);
+	json_add_string(&authentication,
+			"netlogonTrustAccount",
+			ui->netlogon_trust_account.account_name);
 	snprintf(negotiate_flags,
 		 sizeof( negotiate_flags),
 		 "0x%08X",
 		 ui->netlogon_trust_account.negotiate_flags);
-	add_string(&authentication, "netlogonNegotiateFlags", negotiate_flags);
-	add_int(&authentication,
-		"netlogonSecureChannelType",
-		ui->netlogon_trust_account.secure_channel_type);
-	add_sid(&authentication,
-		"netlogonTrustAccountSid",
-		ui->netlogon_trust_account.sid);
-	add_string(&authentication, "passwordType", get_password_type(ui));
-	add_object(&context,AUTH_JSON_TYPE, &authentication);
+	json_add_string(&authentication,
+			"netlogonNegotiateFlags",
+			negotiate_flags);
+	json_add_int(&authentication,
+		     "netlogonSecureChannelType",
+		     ui->netlogon_trust_account.secure_channel_type);
+	json_add_sid(&authentication,
+		     "netlogonTrustAccountSid",
+		     ui->netlogon_trust_account.sid);
+	json_add_string(&authentication, "passwordType", get_password_type(ui));
+	json_add_object(&context,AUTH_JSON_TYPE, &authentication);
 
 	log_json(msg_ctx,
 		 lp_ctx,
@@ -532,7 +202,7 @@ static void log_authentication_event_json(
 		 AUTH_JSON_TYPE,
 		 DBGC_AUTH_AUDIT,
 		 debug_level);
-	free_json_context(&context);
+	json_free(&context);
 }
 
 /*
@@ -566,36 +236,45 @@ static void log_successful_authz_event_json(
 				struct auth_session_info *session_info,
 				int debug_level)
 {
-	struct json_context context = get_json_context();
-	struct json_context authorization;
+	struct json_object context = json_new_object();
+	struct json_object authorization;
 	char account_flags[11];
 
-	//start_object(&context, NULL);
-	add_timestamp(&context);
-	add_string(&context, "type", AUTHZ_JSON_TYPE);
-	authorization = get_json_context();
-	add_version(&authorization, AUTHZ_MAJOR, AUTHZ_MINOR);
-	add_address(&authorization, "localAddress", local);
-	add_address(&authorization, "remoteAddress", remote);
-	add_string(&authorization, "serviceDescription", service_description);
-	add_string(&authorization, "authType", auth_type);
-	add_string(&authorization, "domain", session_info->info->domain_name);
-	add_string(&authorization, "account", session_info->info->account_name);
-	add_sid(&authorization, "sid", &session_info->security_token->sids[0]);
-	add_guid(&authorization,
-		 "sessionId",
-		 &session_info->unique_session_token);
-	add_string(&authorization,
-		   "logonServer",
-		   session_info->info->logon_server);
-	add_string(&authorization, "transportProtection", transport_protection);
+	json_add_timestamp(&context);
+	json_add_string(&context, "type", AUTHZ_JSON_TYPE);
+	authorization = json_new_object();
+	json_add_version(&authorization, AUTHZ_MAJOR, AUTHZ_MINOR);
+	json_add_address(&authorization, "localAddress", local);
+	json_add_address(&authorization, "remoteAddress", remote);
+	json_add_string(&authorization,
+			"serviceDescription",
+			service_description);
+	json_add_string(&authorization, "authType", auth_type);
+	json_add_string(&authorization,
+			"domain",
+			session_info->info->domain_name);
+	json_add_string(&authorization,
+			"account",
+			session_info->info->account_name);
+	json_add_sid(&authorization,
+		     "sid",
+		     &session_info->security_token->sids[0]);
+	json_add_guid(&authorization,
+		      "sessionId",
+		      &session_info->unique_session_token);
+	json_add_string(&authorization,
+			"logonServer",
+			session_info->info->logon_server);
+	json_add_string(&authorization,
+			"transportProtection",
+			transport_protection);
 
 	snprintf(account_flags,
 		 sizeof(account_flags),
 		 "0x%08X",
 		 session_info->info->acct_flags);
-	add_string(&authorization, "accountFlags", account_flags);
-	add_object(&context,AUTHZ_JSON_TYPE, &authorization);
+	json_add_string(&authorization, "accountFlags", account_flags);
+	json_add_object(&context, AUTHZ_JSON_TYPE, &authorization);
 
 	log_json(msg_ctx,
 		 lp_ctx,
@@ -603,7 +282,7 @@ static void log_successful_authz_event_json(
 		 AUTHZ_JSON_TYPE,
 		 DBGC_AUTH_AUDIT,
 		 debug_level);
-	free_json_context(&context);
+	json_free(&context);
 }
 
 #else
@@ -738,7 +417,7 @@ static void log_authentication_event_human_readable(
 
 	password_type = get_password_type(ui);
 	/* Get the current time */
-        ts = get_timestamp(frame);
+        ts = audit_get_timestamp(frame);
 
 	/* Only log the NETLOGON details if they are present */
 	if (ui->netlogon_trust_account.computer_name ||
@@ -869,7 +548,7 @@ static void log_successful_authz_event_human_readable(
 	frame = talloc_stackframe();
 
 	/* Get the current time */
-        ts = get_timestamp(frame);
+        ts = audit_get_timestamp(frame);
 
 	remote_str = tsocket_address_string(remote, frame);
 	local_str = tsocket_address_string(local, frame);
diff --git a/auth/wscript_build b/auth/wscript_build
index 88e9a039314..e2e3d213f48 100644
--- a/auth/wscript_build
+++ b/auth/wscript_build
@@ -2,7 +2,7 @@
 
 bld.SAMBA_LIBRARY('common_auth',
                   source='auth_sam_reply.c wbc_auth_util.c auth_log.c',
-                  deps='talloc samba-security samba-util util_str_escape LIBTSOCKET jansson MESSAGING_SEND server_id_db ',
+                  deps='talloc samba-security samba-util util_str_escape LIBTSOCKET audit_logging jansson MESSAGING_SEND server_id_db ',
                   private_library=True)
 
 bld.RECURSE('gensec')
-- 
2.11.0


From e50853419963ee90f5b479f6c842645881428d48 Mon Sep 17 00:00:00 2001
From: Gary Lockyer <gary at catalyst.net.nz>
Date: Tue, 10 Apr 2018 11:57:41 +1200
Subject: [PATCH 06/10] auth_log: tidy up code formatting

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
---
 auth/auth_log.c | 146 +++++++++++++++++++++++++++++---------------------------
 1 file changed, 76 insertions(+), 70 deletions(-)

diff --git a/auth/auth_log.c b/auth/auth_log.c
index ab44e7d1803..403bcad166f 100644
--- a/auth/auth_log.c
+++ b/auth/auth_log.c
@@ -78,7 +78,9 @@ static const char* get_password_type(const struct auth_usersupplied_info *ui);
 static void log_json(struct imessaging_context *msg_ctx,
 		     struct loadparm_context *lp_ctx,
 		     struct json_object *context,
-		     const char *type, int debug_class, int debug_level)
+		     const char *type,
+		     int debug_class,
+		     int debug_level)
 {
 	char* json = NULL;
 
@@ -128,15 +130,15 @@ static void log_json(struct imessaging_context *msg_ctx,
  *           \t\(.Authentication.localAddress)"'
  */
 static void log_authentication_event_json(
-	                struct imessaging_context *msg_ctx,
-			struct loadparm_context *lp_ctx,
-			const struct auth_usersupplied_info *ui,
-			NTSTATUS status,
-			const char *domain_name,
-			const char *account_name,
-			const char *unix_username,
-			struct dom_sid *sid,
-			int debug_level)
+	struct imessaging_context *msg_ctx,
+	struct loadparm_context *lp_ctx,
+	const struct auth_usersupplied_info *ui,
+	NTSTATUS status,
+	const char *domain_name,
+	const char *account_name,
+	const char *unix_username,
+	struct dom_sid *sid,
+	int debug_level)
 {
 	struct json_object context = json_new_object();
 	struct json_object authentication;
@@ -226,15 +228,15 @@ static void log_authentication_event_json(
  *
  */
 static void log_successful_authz_event_json(
-				struct imessaging_context *msg_ctx,
-				struct loadparm_context *lp_ctx,
-				const struct tsocket_address *remote,
-				const struct tsocket_address *local,
-				const char *service_description,
-				const char *auth_type,
-				const char *transport_protection,
-				struct auth_session_info *session_info,
-				int debug_level)
+	struct imessaging_context *msg_ctx,
+	struct loadparm_context *lp_ctx,
+	const struct tsocket_address *remote,
+	const struct tsocket_address *local,
+	const char *service_description,
+	const char *auth_type,
+	const char *transport_protection,
+	struct auth_session_info *session_info,
+	int debug_level)
 {
 	struct json_object context = json_new_object();
 	struct json_object authorization;
@@ -294,13 +296,15 @@ static void log_no_json(struct imessaging_context *msg_ctx,
 		static bool auth_event_logged = false;
 		if (auth_event_logged == false) {
 			auth_event_logged = true;
-			DBG_ERR("auth event notification = true but Samba was not compiled with jansson\n");
+			DBG_ERR("auth event notification = true but Samba was "
+				"not compiled with jansson\n");
 		}
 	} else {
 		static bool json_logged = false;
 		if (json_logged == false) {
 			json_logged = true;
-			DBG_NOTICE("JSON auth logs not available unless compiled with jansson\n");
+			DBG_NOTICE("JSON auth logs not available unless "
+				   "compiled with jansson\n");
 		}
 	}
 
@@ -308,30 +312,30 @@ static void log_no_json(struct imessaging_context *msg_ctx,
 }
 
 static void log_authentication_event_json(
-	                struct imessaging_context *msg_ctx,
-			struct loadparm_context *lp_ctx,
-			const struct auth_usersupplied_info *ui,
-			NTSTATUS status,
-			const char *domain_name,
-			const char *account_name,
-			const char *unix_username,
-			struct dom_sid *sid,
-			int debug_level)
+	struct imessaging_context *msg_ctx,
+	struct loadparm_context *lp_ctx,
+	const struct auth_usersupplied_info *ui,
+	NTSTATUS status,
+	const char *domain_name,
+	const char *account_name,
+	const char *unix_username,
+	struct dom_sid *sid,
+	int debug_level)
 {
 	log_no_json(msg_ctx, lp_ctx);
 	return;
 }
 
 static void log_successful_authz_event_json(
-				struct imessaging_context *msg_ctx,
-				struct loadparm_context *lp_ctx,
-				const struct tsocket_address *remote,
-				const struct tsocket_address *local,
-				const char *service_description,
-				const char *auth_type,
-				const char *transport_protection,
-				struct auth_session_info *session_info,
-				int debug_level)
+	struct imessaging_context *msg_ctx,
+	struct loadparm_context *lp_ctx,
+	const struct tsocket_address *remote,
+	const struct tsocket_address *local,
+	const char *service_description,
+	const char *auth_type,
+	const char *transport_protection,
+	struct auth_session_info *session_info,
+	int debug_level)
 {
 	log_no_json(msg_ctx, lp_ctx);
 	return;
@@ -394,13 +398,13 @@ static const char* get_password_type(const struct auth_usersupplied_info *ui)
  *
  */
 static void log_authentication_event_human_readable(
-			const struct auth_usersupplied_info *ui,
-			NTSTATUS status,
-			const char *domain_name,
-			const char *account_name,
-			const char *unix_username,
-			struct dom_sid *sid,
-			int debug_level)
+	const struct auth_usersupplied_info *ui,
+	NTSTATUS status,
+	const char *domain_name,
+	const char *account_name,
+	const char *unix_username,
+	struct dom_sid *sid,
+	int debug_level)
 {
 	TALLOC_CTX *frame = NULL;
 
@@ -469,7 +473,7 @@ static void log_authentication_event_human_readable(
 		logon_line,
 		local,
 		nl ? nl : ""
-	       ));
+	));
 
 	talloc_free(frame);
 }
@@ -481,14 +485,15 @@ static void log_authentication_event_human_readable(
  * NOTE: msg_ctx and lp_ctx is optional, but when supplied allows streaming the
  * authentication events over the message bus.
  */
-void log_authentication_event(struct imessaging_context *msg_ctx,
-			      struct loadparm_context *lp_ctx,
-			      const struct auth_usersupplied_info *ui,
-			      NTSTATUS status,
-			      const char *domain_name,
-			      const char *account_name,
-			      const char *unix_username,
-			      struct dom_sid *sid)
+void log_authentication_event(
+	struct imessaging_context *msg_ctx,
+	struct loadparm_context *lp_ctx,
+	const struct auth_usersupplied_info *ui,
+	NTSTATUS status,
+	const char *domain_name,
+	const char *account_name,
+	const char *unix_username,
+	struct dom_sid *sid)
 {
 	/* set the log level */
 	int debug_level = AUTH_FAILURE_LEVEL;
@@ -530,13 +535,13 @@ void log_authentication_event(struct imessaging_context *msg_ctx,
  *
  */
 static void log_successful_authz_event_human_readable(
-				const struct tsocket_address *remote,
-				const struct tsocket_address *local,
-				const char *service_description,
-				const char *auth_type,
-				const char *transport_protection,
-				struct auth_session_info *session_info,
-				int debug_level)
+	const struct tsocket_address *remote,
+	const struct tsocket_address *local,
+	const char *service_description,
+	const char *auth_type,
+	const char *transport_protection,
+	struct auth_session_info *session_info,
+	int debug_level)
 {
 	TALLOC_CTX *frame = NULL;
 
@@ -586,14 +591,15 @@ static void log_successful_authz_event_human_readable(
  * NOTE: msg_ctx and lp_ctx is optional, but when supplied allows streaming the
  * authentication events over the message bus.
  */
-void log_successful_authz_event(struct imessaging_context *msg_ctx,
-				struct loadparm_context *lp_ctx,
-				const struct tsocket_address *remote,
-				const struct tsocket_address *local,
-				const char *service_description,
-				const char *auth_type,
-				const char *transport_protection,
-				struct auth_session_info *session_info)
+void log_successful_authz_event(
+	struct imessaging_context *msg_ctx,
+	struct loadparm_context *lp_ctx,
+	const struct tsocket_address *remote,
+	const struct tsocket_address *local,
+	const char *service_description,
+	const char *auth_type,
+	const char *transport_protection,
+	struct auth_session_info *session_info)
 {
 	int debug_level = AUTHZ_SUCCESS_LEVEL;
 
-- 
2.11.0


From ec4c0a71be6f8f9e111484e0c81f5fa664562e07 Mon Sep 17 00:00:00 2001
From: Gary Lockyer <gary at catalyst.net.nz>
Date: Mon, 16 Apr 2018 09:29:04 +1200
Subject: [PATCH 07/10] auth_log: Rename the json variables

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
---
 auth/auth_log.c | 32 ++++++++++++++++----------------
 1 file changed, 16 insertions(+), 16 deletions(-)

diff --git a/auth/auth_log.c b/auth/auth_log.c
index 403bcad166f..87daf2f110f 100644
--- a/auth/auth_log.c
+++ b/auth/auth_log.c
@@ -77,21 +77,21 @@ static const char* get_password_type(const struct auth_usersupplied_info *ui);
  */
 static void log_json(struct imessaging_context *msg_ctx,
 		     struct loadparm_context *lp_ctx,
-		     struct json_object *context,
+		     struct json_object *object,
 		     const char *type,
 		     int debug_class,
 		     int debug_level)
 {
 	char* json = NULL;
 
-	if (context->error) {
+	if (object->error) {
 		return;
 	}
 
-	json = json_dumps(context->root, 0);
+	json = json_dumps(object->root, 0);
 	if (json == NULL) {
 		DBG_ERR("Unable to convert JSON object to string\n");
-		context->error = true;
+		object->error = true;
 		return;
 	}
 
@@ -140,12 +140,12 @@ static void log_authentication_event_json(
 	struct dom_sid *sid,
 	int debug_level)
 {
-	struct json_object context = json_new_object();
+	struct json_object wrapper = json_new_object();
 	struct json_object authentication;
 	char negotiate_flags[11];
 
-	json_add_timestamp(&context);
-	json_add_string(&context, "type", AUTH_JSON_TYPE);
+	json_add_timestamp(&wrapper);
+	json_add_string(&wrapper, "type", AUTH_JSON_TYPE);
 
 	authentication = json_new_object();
 	json_add_version(&authentication, AUTH_MAJOR, AUTH_MINOR);
@@ -196,15 +196,15 @@ static void log_authentication_event_json(
 		     "netlogonTrustAccountSid",
 		     ui->netlogon_trust_account.sid);
 	json_add_string(&authentication, "passwordType", get_password_type(ui));
-	json_add_object(&context,AUTH_JSON_TYPE, &authentication);
+	json_add_object(&wrapper, AUTH_JSON_TYPE, &authentication);
 
 	log_json(msg_ctx,
 		 lp_ctx,
-		 &context,
+		 &wrapper,
 		 AUTH_JSON_TYPE,
 		 DBGC_AUTH_AUDIT,
 		 debug_level);
-	json_free(&context);
+	json_free(&wrapper);
 }
 
 /*
@@ -238,12 +238,12 @@ static void log_successful_authz_event_json(
 	struct auth_session_info *session_info,
 	int debug_level)
 {
-	struct json_object context = json_new_object();
+	struct json_object wrapper = json_new_object();
 	struct json_object authorization;
 	char account_flags[11];
 
-	json_add_timestamp(&context);
-	json_add_string(&context, "type", AUTHZ_JSON_TYPE);
+	json_add_timestamp(&wrapper);
+	json_add_string(&wrapper, "type", AUTHZ_JSON_TYPE);
 	authorization = json_new_object();
 	json_add_version(&authorization, AUTHZ_MAJOR, AUTHZ_MINOR);
 	json_add_address(&authorization, "localAddress", local);
@@ -276,15 +276,15 @@ static void log_successful_authz_event_json(
 		 "0x%08X",
 		 session_info->info->acct_flags);
 	json_add_string(&authorization, "accountFlags", account_flags);
-	json_add_object(&context, AUTHZ_JSON_TYPE, &authorization);
+	json_add_object(&wrapper, AUTHZ_JSON_TYPE, &authorization);
 
 	log_json(msg_ctx,
 		 lp_ctx,
-		 &context,
+		 &wrapper,
 		 AUTHZ_JSON_TYPE,
 		 DBGC_AUTH_AUDIT,
 		 debug_level);
-	json_free(&context);
+	json_free(&wrapper);
 }
 
 #else
-- 
2.11.0


From 04b6ebde16f0e465c1ce6b9dc91c41bb8d5df08a Mon Sep 17 00:00:00 2001
From: Gary Lockyer <gary at catalyst.net.nz>
Date: Mon, 23 Apr 2018 08:49:26 +1200
Subject: [PATCH 08/10] messaging idl add group membersip events

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
---
 librpc/idl/messaging.idl | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/librpc/idl/messaging.idl b/librpc/idl/messaging.idl
index b7454404d50..f6c1693ab23 100644
--- a/librpc/idl/messaging.idl
+++ b/librpc/idl/messaging.idl
@@ -147,6 +147,7 @@ interface messaging
 		MSG_AUTH_LOG                    = 0x0800,
 		MSG_SAMDB_LOG                   = 0x0801,
 		MSG_PWD_LOG                     = 0x0802,
+		MSG_GROUP_LOG                   = 0x0803,
 
 		/* dbwrap messages 4001-4999 (0x0FA0 - 0x1387) */
 		/* MSG_DBWRAP_TDB2_CHANGES		= 4001, */
@@ -186,4 +187,5 @@ interface messaging
         const string AUTH_EVENT_NAME  = "auth_event";
 	const string SAMDB_EVENT_NAME = "samdb_event";
 	const string PWD_EVENT_NAME   = "password_event";
+	const string GROUP_EVENT_NAME = "group_event";
 }
-- 
2.11.0


From 660d7c0f31faa20524303500760464113cc3c3e1 Mon Sep 17 00:00:00 2001
From: Gary Lockyer <gary at catalyst.net.nz>
Date: Mon, 23 Apr 2018 09:00:54 +1200
Subject: [PATCH 09/10] smb.conf: Add group change notification parameter

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
---
 .../smbdotconf/misc/groupchangenotification.xml    | 27 ++++++++++++++++++++++
 1 file changed, 27 insertions(+)
 create mode 100644 docs-xml/smbdotconf/misc/groupchangenotification.xml

diff --git a/docs-xml/smbdotconf/misc/groupchangenotification.xml b/docs-xml/smbdotconf/misc/groupchangenotification.xml
new file mode 100644
index 00000000000..09278ccd7dc
--- /dev/null
+++ b/docs-xml/smbdotconf/misc/groupchangenotification.xml
@@ -0,0 +1,27 @@
+<samba:parameter name="group change notification"
+                 context="G"
+                 type="boolean"
+                 xmlns:samba="http://www.samba.org/samba/DTD/samba-doc">
+<description>
+	<para>When enabled, this option causes Samba (acting as an
+	Active Directory Domain Controller) to stream group membership change
+	events across the internal message bus.  Scripts built using
+	Samba's python bindings can listen to these events by
+	registering as the service
+	<filename moreinfo="none">group_event</filename>.</para>
+
+	<para>This should be considered a developer option (it assists
+	in the Samba testsuite) rather than a facility for external
+	auditing, as message delivery is not guaranteed (a feature
+	that the testsuite works around).  Additionally Samba must be
+	compiled with the jansson support for this option to be
+	effective.</para>
+
+	<para>The group events are also logged via the normal
+	logging methods when the <smbconfoption name="log level"/> is
+	set appropriately.</para>
+
+</description>
+
+<value type="default">no</value>
+</samba:parameter>
-- 
2.11.0


From c50167f484452ae6053cae41d483cb8f8b19fc24 Mon Sep 17 00:00:00 2001
From: Gary Lockyer <gary at catalyst.net.nz>
Date: Mon, 23 Apr 2018 12:24:34 +1200
Subject: [PATCH 10/10] debug: Add group logging classes

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
---
 lib/util/debug.c | 2 ++
 lib/util/debug.h | 2 ++
 2 files changed, 4 insertions(+)

diff --git a/lib/util/debug.c b/lib/util/debug.c
index db4f71a1501..9e6b5d8087d 100644
--- a/lib/util/debug.c
+++ b/lib/util/debug.c
@@ -549,6 +549,8 @@ static const char *default_classname_table[] = {
 	[DBGC_PWD_AUDIT_JSON] = "password_json_audit",
 	[DBGC_TRN_AUDIT]  =	"transaction_audit",
 	[DBGC_TRN_AUDIT_JSON] = "transaction_json_audit",
+	[DBGC_GROUP_AUDIT] =	"group_audit",
+	[DBGC_GROUP_AUDIT_JSON] = "group_json_audit",
 };
 
 /*
diff --git a/lib/util/debug.h b/lib/util/debug.h
index 4e425a0e0f8..67b737eaba3 100644
--- a/lib/util/debug.h
+++ b/lib/util/debug.h
@@ -101,6 +101,8 @@ bool dbghdr( int level, const char *location, const char *func);
 #define DBGC_PWD_AUDIT_JSON	33
 #define DBGC_TRN_AUDIT		34
 #define DBGC_TRN_AUDIT_JSON	35
+#define DBGC_GROUP_AUDIT	36
+#define DBGC_GROUP_AUDIT_JSON	37
 
 /* So you can define DBGC_CLASS before including debug.h */
 #ifndef DBGC_CLASS
-- 
2.11.0



More information about the samba-technical mailing list