[cifs-protocol] [EXTERNAL] RE: [REG:120060621000848] [MS-XCA] L77+Huffman negative huffman symbol update

[Jeff McCashland]

Hi Aurélien,

We're still working on this. In the meantime, have you considered the code changes since the published version which are covered by the Errata document?

An additional paragraph has been added right before the compression pseudocode:
"During the beginning of processing each block for decompression, an implementation MUST check for EOF. An implementation can do this by comparing the block size against the required space for a Huffman table — if this condition is met and all output has been written, then processing stops and success is returned. Alternately, an implementation can explicitly examine the input buffer using the Huffman table from the previous block."

The current pseudocode including errata (table construction is the same):
        Loop until a decompression terminating condition
            Check for EOF
            Build the decoding table
            CurrentPosition += 256    // start at the end of the Huffman table
            NextBits = Read16Bits(InputBuffer + CurrentPosition)
            CurrentPosition += 2
            NextBits <<= 16
            NextBits |= Read16Bits(InputBuffer + CurrentPosition)
            CurrentPosition += 2
            ExtraBits = 16
            BlockEnd = OutputPosition + 65536
        
            Loop until a block terminating condition
                Loop until a literal processing terminating condition
                    If OutputPosition >= BlockEnd then terminate block processing
                        Next15Bits = NextBits >> (32 – 15)
                        HuffmanSymbol = DecodingTable[Next15Bits]
                        HuffmanSymbolBitLength = the bit length of HuffmanSymbol, from the table in
                                             the input buffer
                        If HuffmanSymbol <= 0
                            NextBits <<= HuffmanSymbolBitLength
                            ExtraBits -= HuffmanSymbolBitLength
        
                            Do
                                HuffmanSymbol = - HuffmanSymbol
                                HuffmanSymbol += (NextBits >> 31)
                                NextBits *= 2
                                ExtraBits = ExtraBits - 1
                                HuffmanSymbol = DecodingTable[HuffmanSymbol]
                            While HuffmanSymbol <= 0
                        Else
                            DecodedBitCount = HuffmanSymbol & 15        
                            NextBits <<= DecodedBitCount
                            ExtraBits -= DedcodedBitCount

                        HuffmanSymbol >>= 4 // Shift by 4 bits to get the symbol value
                                            // (the lower 4 bits are the bit length of the symbol)
                        HuffmanSymbol -= 256
                        If ExtraBits < 0
                            NextBits |= Read16Bits(InputBuffer + CurrentPosition) << (-ExtraBits)
                            ExtraBits += 16
                            CurrentPosition += 2
                        If HuffmanSymbol >= 0
                            If HuffmanSymbol == 0
                                If the entire input buffer has been read and
                                the expected decompressed size has been written to the output buffer
                                    Decompression is complete.  Return with success.
                            Terminate literal processing
                        Else
                            Output the byte value of HuffmanSymbol to the output stream
                End of literal processing Loop
           
                MatchLength = HuffmanSymbol mod 16
                MatchOffsetBitLength = HuffmanSymbol / 16
                If MatchLength == 15
                    MatchLength = ReadByte(InputBuffer + CurrentPosition)
                    CurrentPosition += 1
                    If MatchLength == 255
                        MatchLength = Read16Bits(InputBuffer + CurrentPosition)
                        CurrentPosition += 2
                        If MatchLength < 15
                            The compressed data is invalid. Return error.
                        MatchLength = MatchLength - 15
                    MatchLength = MatchLength + 15
                MatchLength = MatchLength + 3
                MatchOffset = NextBits >> (32 – MatchOffsetBitLength)
                MatchOffset += (1 << MatchOffsetBitLength)
                NextBits <<= MatchOffsetBitLength
                ExtraBits -= MatchOffsetBitLength
                If ExtraBits < 0
                    NextBits |= Read16Bits(InputBuffer + CurrentPosition) << (-ExtraBits)
                    ExtraBits += 16
                    CurrentPosition += 2
                For i = 0 to MatchLength - 1
                    Output OutputBuffer[OutputPosition – MatchOffset + i]
             End of block loop
        End of decoding loop

I will let you know as we have more information. 

Best regards,
Jeff McCashland | Senior Escalation Engineer | Microsoft Protocol Open Specifications Team 
Phone: +1 (425) 703-8300 x38300 | Hours: 9am-5pm | Time zone: (UTC-08:00) Pacific Time (US and Canada)
Local country phone number found here: http://support.microsoft.com/globalenglish | Extension 1138300
We value your feedback.  My manager is Natesha Morrison (namorri), +1 (704) 430-4292

-----Original Message-----
From: Jeff McCashland 
Sent: Tuesday, June 23, 2020 6:47 AM
To: Aurélien Aptel <aaptel at suse.com>; cifs-protocol at lists.samba.org
Cc: support <support at mail.support.microsoft.com>
Subject: RE: [EXTERNAL] RE: [REG:120060621000848] [MS-XCA] L77+Huffman negative huffman symbol update

Apologies! I see that now. 

-----Original Message-----
From: Aurélien Aptel <aaptel at suse.com> 
Sent: Tuesday, June 23, 2020 2:16 AM
To: Jeff McCashland <jeffm at microsoft.com>; cifs-protocol at lists.samba.org
Cc: support <support at mail.support.microsoft.com>
Subject: RE: [EXTERNAL] RE: [REG:120060621000848] [MS-XCA] L77+Huffman negative huffman symbol update

Hi Jeff,

Jeff McCashland <jeffm at microsoft.com> writes:
> Are there any specific parts that are particularly confusing? How far do you get with it? 

Hm yes, I put the details in my previous email after the quote. The table is filled with values ranging from 0 to 511. This doesn't touch the highest bit of a 16bit number so it cannot be negative.

Thanks,
--
Aurélien Aptel / SUSE Labs Samba Team
GPG: 1839 CB5F 9F5B FB9B AA97  8C99 03C8 A49B 521B D5D3 SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, 90409 Nürnberg, DE
GF: Felix Imendörffer, Mary Higgins, Sri Rasiah HRB 247165 (AG München)