[cifs-protocol] RE: LZ77 + DIRECT2 compression MS-OXCRPC

[Tom Devey]

Matthieu,

Let me dig into this.  Also note there is a problem in the current documentation for [MS-OXCRPC] 3.1.7.2.2.4 Match Length for decoding.

The feedback to the resolution was posted below:

<forum source="http://forums.msdn.microsoft.com/en-US/os_exchangeprotocols/thread/b4e16062-0ecf-40fa-879a-380bfbac0028">
The [MS-OXCRPC] document says that if the match length is greater than 9 then we have to read an additional byte and use high-order nibble as an additive length. The low-order nibble should be kept and used when the next time the match length is greater than nine. As it turned out we should do quite opposite: use LOW-order nibble the first time and HIGH-order nibble the next time.
</forum>

-----Original Message-----
From: Matthieu Suiche [mailto:msuiche at gmail.com]
Sent: Thursday, August 28, 2008 4:37 PM
To: Interoperability Documentation Help
Cc: pfif at tridgell.net; cifs-protocol at samba.org; Stefan (metze) Metzmacher
Subject: LZ77 + DIRECT2 compression MS-OXCRPC

As a PFIF subcontractor, I am requesting correction assistance:

In MS-OXCRPC, 3.1.7.2.1.2 is claims "2. Find the longest match in the
window for the lookahead buffer." in the "Using the compression
algorithm".

But it looks Microsoft implementation has an unexpected behavior
regarding the above line. Actually, I've implemented both
decompression and compression of (LZ77+DIRECT2) algorithm and it works
well. But the problem I have is my compression function doesn't return
the same output has MSFT does. The routine I've implemented to find
the longest match in the window for the lookahead buffer returns a
longer length than MSFT implementation.

My compression routine returns:
string found at uncompressed_position 0x00000052
[ 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ]
 comp[00000035] = 00BF = Metadata (len,offset) = (22, 00000017) -
uncomp_pos: 00000052
 bestlen < 25
 comp[00000037] = 0C (shared byte)

Comparaison of the compressed chunk:
comp_chunk_by_microsoft[00000035] 32 != BF mycomp[00000035]
comp_chunk_by_microsoft[00000036] 00 == 00 mycomp[00000036]
comp_chunk_by_microsoft[00000037] 07 != DC mycomp[00000037]

As you can see, MSFT implementation encode a 0x0032 metadata at offset
0x35 which means size of 5 (2 + 3)  at position -6, and my
implementation encode a three bytes metadata at offset 0x35.

Here, http://www.msuiche.net/samba/debug_lzxpress.zip, you can found
debug outputs and concerned file. All files concern only
one chunk.

Could you please describe the exact process to retrieve the longest
match in the window for the lookahead buffer?

Regards,
--
Matthieu Suiche