DCERPC requirements for DFS-R

[Samuel Cabrero]

Hi Garming,

I have done some progress and I was close to sharing it once I solved a
couple of things. I will try to give a quick summary about the status.

The client runs as a task ('dfsrsrv') and can handle generic replica
sets containing multiple content sets. It establish connections to all
replica set members following the replica set topology, but this is not
completely tested yet.

Each connection runs multiple sessions, one per content set inside the
replica set. The sequence number in the AsyncPoll response is used to
find the session the response is for. If any session is terminated for
any reason, it is restarted by the 'dfsrsrv' periodic handler.

The sessions run the state machine described [MS-FRS2] 3.3.1.2, and
retrieve the updates following the state machine described in [MS-FRS2] 
3.3.4.6.1. For the moment the updates are not queued neither processed
in a separate loop; When a batch of updates are retrieved calling
RequestUpdates, they are downloaded and installed sequentially. This is
sub-optimal, because a later update can supersede a former one.

The updates are fully downloaded to the staging directory. The DCERPC
async pipe and RDC compression stuff can be avoided using
RawGetFileData, I have tested with files of hundreds of megabytes.

Once the update is fully downloaded to the staging directory, it has to
be decompressed, unmarshalled and installed into the final location. To
install it, it has to go through the SMB VFS layer, so I have spawned a
new smbd child process (DFS-R meet module). When the staged file is
fully downloaded, the 'dfsrsrv' service sends an IRPC message to the
meet module to install it, waiting for the response. 

The meet module decompress the staged file (LZ77 + Huffman) and
unmarshall the decompressed stream. The marshaling format is specified
in [MS-FRS2] 3.2.4.1.14.1. It contains the metadata, the security
descriptor and the NT backup file stream ([MS-BKUP] 2.1)

Once the file is in its final location and the security descriptor set,
the meet module inserts the update in a TDB database, indexed by the
file_id. This database is also used to rebuild the absolute path, as
each update only references its parent update and does not know nothing
about folder hierarchy.

At the moment I am working in retrieving from the database the known
version vectors when the dfsrsrv service starts up. The next step is
to  handle rename and delete updates, and handle the MS-BKUP header,
now it is ignored when installing the updates.

Cheers,

Samuel

On Tue, 2017-11-21 at 11:45 +1300, Garming Sam via samba-technical
wrote:
> Hi Samuel,
> 
> I noticed that you've been doing more work on your dfs-r branch,
> which
> is nice to see. I was wondering if you might give a status update on
> what you've actually implemented (and what parts of the protocol
> you're
> trying to get working).
> 
> 
> Cheers,
> 
> Garming
> 
> 
> On 07/09/17 20:19, Samuel Cabrero wrote:
> > Hi,
> > 
> > I am also interested on this because I have started to work on
> > implement the missing parts on the initial Matthieu's work to get a
> > working client for the general case, not only for sysvol (multiple
> > content sets per replica group and arbitrary replication topology)
> > [1].
> > 
> > I am working on top of current master and I haven't found any
> > problem
> > related to DCERPC, so is it insufficient only for the server side
> > but
> > enough for the client?
> > 
> > [1] https://github.com/kernevil/samba/tree/dfs-r
> > 
> 
>