libsmb2, meet samba

Tue Apr 25 23:26:48 UTC 2017

Update. Start of unstructured brain dump:

Libsmb2 is nearing its first official release. Which means it has all
the features that QEMU and KODI (main consumers) need.
Please have a look at my github repo.

Libsmb2 is fully async and non-blocking.
It does zero-copy for both READ and WRITE commands, in the sense that
it reads/writes the application buffer directly to/from the socket
without any memcpy() being done in the library itself.  True zero-copy
will have to wait until RDMA support comes.

It also supports compounded requests making the common pattern of
CREATE/do-something/CLOSE execute in a single roundtrip.
This should make a lot of metadata operations fast-ish.

Not bloated yet, almost fitting in memory of a Sinclair Spectrum:
$ ls -l lib/.libs/libsmb2.so.1.0.0
-rwxrwxr-x. 1 sahlberg sahlberg 51816 Apr 25 16:05 lib/.libs/libsmb2.so.1.0.0

No external dependencies outside from MIT and basic libc making it,
very portable.
As I am not fluent in Visual Studio I would welcome any help I can get
to set up a proper VS project file.

At this point I think the library is in a usable state and the API
should be declared stable.
Some features should still be added but as far as main consumers, i.e.
QEMU and KODI, are concerned we
are in a releasable state for 0.1 feature wise.

I was chatting with Jeremy about features that would be nice to add
after the 0.1 release and I would be happy to work with anyone
that wants to contribute.
My idea of such post 0.1 features would be :
* krb5 support. Currently only gss-ntlmssp is supported but adding
krb5 support should be trivial.
* sing and seal.
* visual studio 1* project files and support.
* SET_INFO support
* a simple cifs fuse module.
* RDMA (this is probably not a near term goal)

Now, at the end, I just want to show what the compound API ended up
looking like.
Not too shabby me thinks.

Example of using the compound request API:
==================================
        /* CREATE command */
        memset(&cr_req, 0, sizeof(struct smb2_create_request));
        cr_req.requested_oplock_level = SMB2_OPLOCK_LEVEL_NONE;
        cr_req.impersonation_level = SMB2_IMPERSONATION_IMPERSONATION;
        cr_req.desired_access = SMB2_READ_CONTROL;
        cr_req.file_attributes = 0;
        cr_req.share_access = SMB2_FILE_SHARE_READ | SMB2_FILE_SHARE_WRITE;
        cr_req.create_disposition = SMB2_FILE_OPEN;
        cr_req.create_options = 0;
        cr_req.name = path;

        pdu = smb2_cmd_create_async(smb2, &cr_req, stat_cb_1, stat_data);
        if (pdu == NULL) {
                fprintf(stderr, "Failed to create create command");
                free(stat_data);
                return -1;
        }

        /* QUERY INFO command */
        memset(&qi_req, 0, sizeof(struct smb2_query_info_request));
        qi_req.info_type = SMB2_0_INFO_SECURITY;
        qi_req.output_buffer_length = 65535;
        qi_req.additional_information =
                SMB2_OWNER_SECURITY_INFORMATION |
                SMB2_GROUP_SECURITY_INFORMATION |
                SMB2_DACL_SECURITY_INFORMATION;
        memcpy(qi_req.file_id, compound_file_id, SMB2_FD_SIZE);

        next_pdu = smb2_cmd_query_info_async(smb2, &qi_req,
                                             stat_cb_2, stat_data);
        if (next_pdu == NULL) {
                fprintf(stderr, "Failed to create query command");
                free(stat_data);
                smb2_free_pdu(smb2, pdu);
                return -1;
        }
        smb2_add_compound_pdu(smb2, pdu, next_pdu);

        /* CLOSE command */
        memset(&cl_req, 0, sizeof(struct smb2_close_request));
        cl_req.flags = SMB2_CLOSE_FLAG_POSTQUERY_ATTRIB;
        memcpy(cl_req.file_id, compound_file_id, SMB2_FD_SIZE);

        next_pdu = smb2_cmd_close_async(smb2, &cl_req, stat_cb_3, stat_data);
        if (next_pdu == NULL) {
                stat_data->cb(smb2, -1, NULL, stat_data->cb_data);
                free(stat_data);
                smb2_free_pdu(smb2, pdu);
                return -1;
        }
        smb2_add_compound_pdu(smb2, pdu, next_pdu);

        smb2_queue_pdu(smb2, pdu);

regards
ronnie sahlberg

On Tue, Jan 24, 2017 at 7:31 PM, ronnie sahlberg
<ronniesahlberg at gmail.com> wrote:
> samba, please meet libsmb2
> libsmb2 please meet samba
>
>
> As you know already I am am for various reasons, to benefit linux, working on a
> smb2 client library.
>
> https://github.com/sahlberg/libsmb2
>
>
> Why, and what requirements?
> ----------------------------------------
> I want there to be a highly portable and free smb2 client that any/all
> applications can use.
>
> It should be small and nimble. Currently clocks in at ~50kb compiled
> (though that excludes MIT kerberos).
>
> $ ls -l lib/.libs/libsmb2.a -rw-rw-r-- 1 sahlberg sahlberg 54188 Jan
> 24 17:40 lib/.libs/libsmb2.a
>
> It should need very few external dependencies. Right now I target
> basic posix libc plus MIT. Nothing else.
>
> It should be so portable that it compiles native on both Win32/64 as
> well as Amiga Kickstart 1.3.
>
> It should provide an API similar to libnfs so that porting between nfs
> and smb wrappers should be as painless as possible.
>
> SMB2+ only. No smb1 support. RDMA support will be nice to add in the future.
>
> Zero-copy for [p]read/[p]write.
>
>
> Also, I have been a bit bored recently and what better reason for
> building a lightweight SMB2 client? Boredom is the fuel for
> innovation.
>
>
> Licence
> ----------
> For licence, gnu licences lgplv2.1 only or lgpl2.1-plus are
> acceptable. QMU licence compatibility is not negotiable. (Make me
> angry and I will consider 2-clause BSD, or worse :-) )
> Nah lgpl2.1 is fine :-)
>
> Kodi support is kind of not negotiable as well, but they are
> compatible with a lot mode different licences so that should not be an
> issue.
>
>
>
> Current status
> -------------------
> The codebase is in hue flux and will need a major refactoring and also
> audits over the next few weeks until the implementation as well as the
> APIs will settle down.
>
> Mimicing the libnfs APIs it provides fair coverage of the common posix
> file api already:
> smb2_close
> smb2_close_async
> smb2_closedir
> smb2_cmd_close_async
> smb2_cmd_create_async
> smb2_cmd_echo_async
> smb2_cmd_logoff_async
> smb2_cmd_negotiate_async
> smb2_cmd_query_directory_async
> smb2_cmd_query_info_async
> smb2_cmd_session_setup_async
> smb2_cmd_tree_connect_async
> smb2_connect_async
> smb2_connect_share
> smb2_connect_share_async
> smb2_destroy_context
> smb2_destroy_url
> smb2_fstat
> smb2_fstat_async
> smb2_get_client_guid
> smb2_get_error
> smb2_get_fd
> smb2_init_context
> smb2_mkdir
> smb2_mkdir_async
> smb2_open
> smb2_open_async
> smb2_opendir
> smb2_opendir_async
> smb2_parse_url
> smb2_pread
> smb2_pread_async
> smb2_pwrite
> smb2_pwrite_async
> smb2_read
> smb2_read_async
> smb2_readdir
> smb2_rewinddir
> smb2_rmdir
> smb2_rmdir_async
> smb2_seek
> smb2_seekdir
> smb2_service
> smb2_set_security_mode
> smb2_stat
> smb2_stat_async
> smb2_telldir
> smb2_unlink
> smb2_unlink_async
> smb2_which_events
> smb2_write
> smb2_write_async
>
> Not complete yet but sufficient for building a filemanager or something similar.
>
> It uses MIT + simo's GSS-NLTMSSP for authentication. It does not yet
> do actual sign or seal
> but I think that is low priority for the initial version. Patches welcome.
>
> It leaks memory from the gss contexts that are used during session
> setup. Again patches welcome as I am completely ignorant about MIT and
> its GSS layer, and wish to remain so as I am too old and lazy to learn
> new stuff.
>
>
> These funcitons work pretty well and even run valgrind clean (aside
> from the two leaks form the mis-use of the MIT library).
> Woohoo for that.
>
>
> Qemu and Kodi and all the other users of libnfs
> ---------------------------------------------------------------
> It supports all the posix fs functions that any of those apps use
> today for libnfs so it should be fairly easy for those folks to take
> theur libnfs wrappers, do some sed-magic and end up with an almost
> functional smb2 wrapper.
>
> Easy porting from libnfs to libsmb2 is another requirement that I just made up.
>
>
> Examples
> -------------
> There are a bunch of examples in the examples subdirectory to show the
> async as well as the sync api. Please have look.
>
>
> Fire extinguisher
> ----------------------
> This is not a replacement for libsmbclient. This is aimed as a minimal
> library that is
> easy to port and use. With limited functionality but an alternative to
> those apps that just need the most basic smb2 support and where
> libsmb2 client is overkill.
>
> Well, on amiga Kickstart 1.3 it might be a replacement as I doubt
> anyone wants to port non-extremely-ancient versions of libsmbclient to
> the amiga.
>
>
>
> Interested?
> ---------------
> It is functional right now. More posix function emulation is needed.
> It probably need a re-write and re-factor as well as the internal APIs
> settle down.
>
> Want to help out? Contact me and check out the github.
> Contacting me first before starting to prepare any pull requests. I
> expect that the code base will
> be in major flux for the next few weeks until things stabilize.
>
> There is a lot of work that still needs doing.
>
>
> Where to host?
> --------------------
> I kind of like github/sahlberg as your on-stop-shop for everything
> related to network storage client software.
> However, hosting this on samba.org at some stage might be a better choice.
>
> You open for that idea?
>
>
>
>
>
> regards, your friend in the file/block client space
> ronnie sahlberg