vfs_full_audit annoyances on major version upgrades

[Andrew Bartlett]

On Fri, 2021-03-26 at 16:28 +0100, Anders Östling via samba-technical
wrote:
> Hi
> 
> Newbie here, but I would like to comment todays post by Andrew Walker’s  regarding audit logging.
> 
> C systems programming is not new to me, although it is about 15 years since I wrote anything serious. The proposed patch looks fine to me, but I believe that a better way is to burp and inform the user about the change in keywords/syntax, at least when there are behind-the-scene changes like this one describes. Or maybe “testparm” should look at the config lines a little more in detail. Maybe both proposed solutions could be combined.
> 
> Another point is that the logging format from the AD-DC modules seems to be completely different from what SMBD uses. A common format would IMO make it much much easier to consolidate, parse and analyse logs in an automated way. Maybe that’s a pipers dream, but still … here is an example of what I mean
> 
> AD-DC (Global config for formatting  full_audit:prefix = IP=%I | USER=%u | MACHINE=%m | VOLUME=%S - (seems to be ignored))
> 
> [2021/03/26 16:09:27.394208,  3] ../../auth/auth_log.c:653(log_authentication_event_human_readable)
>   Auth: [Kerberos KDC,ENC-TS Pre-authentication] user [(null)]\[anders at HOGANAS-PLATSLAGAREN.SE] at [Fri, 26 Mar 2021 16:09:27.394194 CET] with [aes256-cts-hmac-sha1-96] status [NT_STATUS_OK] workstation [(null)] remote host [ipv4:10.0.100.14:39820] became [HPAB]\[anders] [S-1-5-21-1399469354-1941875790-2784827883-1601]. local host [NULL]

Yes, we regret adding this logging format.  We added JSON formatted
logs at the same time and they were enthusiastically received, compared
to the work it took to write and test both, we should have just done
the JSON.

> SMBD (Global config for formatting full_audit:prefix = IP=%I | USER=%u | MACHINE=%m | VOLUME=%S (used as specified)
> 
> Mar 26 16:10:45 fs1a smbd_audit[62673]: IP=10.0.100.14 | USER=HPAB\anders | MACHINE=10.0.100.14 | VOLUME=Dokument|get_nt_acl_at|ok|/share/documents
> Mar 26 16:10:45 fs1a smbd_audit[62673]: IP=10.0.100.14 | USER=HPAB\anders | MACHINE=10.0.100.14 | VOLUME=Dokument|close|ok|/share/documents
> 
> I have a few more thoughts, but I will save them for later. I will download the sources myself and see if I can understand hos logging is structured. Who knows, maybe this old fart can contribute with something later on!

I think the general mood is that we should move to JSON for structured
logging, as that is what modern log ingestion tools prefer.  

I wrote the following to another developer who was at the time
interested in reworking the vfs_full_audit to output JSON:


Most important hint is to remember to tie the entries to each other -
so include:

      rc = json_add_guid(
            &authorization, "sessionId", &session_info->
unique_session_token);

        if (rc != 0) {
                goto failure;
        }

This helps the administrator map the audit entry back to the
authorization in a deterministic manner. 

For inspiration the other place to look is the audit code in the DSDB
module stack. 

source4/dsdb/samdb/ldb_modules/audit_log.c

This logs things quite similar to what you will be targeting, so should
be good inspiration.

Testing is best using python or cmocka (for unit tests).  The python
tests listen on a messaging socket and get a stream of JSON audit
entries over this connection.

git ls-files| grep audit| grep dsdb

Back on the design question, just note how Gary added a major and minor
version to the JSON output, so that any consumer can know that the JSON
formatting has changed.  

I hope this is of assistance.

Andrew Bartlett
-- 
Andrew Bartlett (he/him)        https://samba.org/~abartlet/
Samba Team Member (since 2001)  https://samba.org
Samba Developer, Catalyst IT    https://catalyst.net.nz/services/samba