[cifs-protocol] [EXTERNAL] [MS-XCA] LZ77+ Huffman: questions about blocks - TrackingID#2210140040006030

[Jeff McCashland (He/him)]

Hi Douglas,

As I understand, each 64k block is processed separately. In other words, the first 64k block is LZ77 compressed, then the Huffman codes are constructed based on symbol frequency in that 64k. If, in your example, DEF ends the 64k block, then the subsequent ghi... will be processed with the second 64k block and Huffman table, and not dropped. 

Am I understanding your question correctly? 

Best regards,
Jeff McCashland (He/him) | Senior Escalation Engineer | Microsoft Protocol Open Specifications Team 
Phone: +1 (425) 703-8300 x38300 | Hours: 9am-5pm | Time zone: (UTC-08:00) Pacific Time (US and Canada)
Local country phone number found here: http://support.microsoft.com/globalenglish | Extension 1138300

-----Original Message-----
From: Jeff McCashland (He/him) 
Sent: Friday, October 14, 2022 1:58 PM
To: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>; cifs-protocol at lists.samba.org; Samuel Cabrero (Samba) <scabrero at samba.org>
Cc: Microsoft Support <supportmail at microsoft.com>
Subject: RE: [EXTERNAL] [MS-XCA] LZ77+ Huffman: questions about blocks - TrackingID#2210140040006030 

[Tom to BCC]

Hi Douglas,

I'll research this question and let you know what I learn.

Best regards,
Jeff McCashland (He/him) | Senior Escalation Engineer | Microsoft Protocol Open Specifications Team
Phone: +1 (425) 703-8300 x38300 | Hours: 9am-5pm | Time zone: (UTC-08:00) Pacific Time (US and Canada) Local country phone number found here: http://support.microsoft.com/globalenglish | Extension 1138300

-----Original Message-----
From: Tom Jebo <tomjebo at microsoft.com>
Sent: Friday, October 14, 2022 9:45 AM
To: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>; cifs-protocol at lists.samba.org; Samuel Cabrero (Samba) <scabrero at samba.org>
Cc: Microsoft Support <supportmail at microsoft.com>
Subject: RE: [EXTERNAL] [MS-XCA] LZ77+ Huffman: questions about blocks - TrackingID#2210140040006030 

[dochelp to bcc]
[casemail cc]

Hi Douglas, 

Thank you for your request. One of the Open Specifications team will respond to start working with you. I have created case 2210140040006030 and added the number to the subject of this email. Please refer to this case number in future communications regarding this issue.

Best regards,
Tom Jebo
Sr Escalation Engineer
Microsoft Open Specifications

-----Original Message-----
From: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Sent: Thursday, October 13, 2022 9:57 PM
To: Interoperability Documentation Help <dochelp at microsoft.com>; cifs-protocol at lists.samba.org; Samuel Cabrero (Samba) <scabrero at samba.org>
Subject: [EXTERNAL] [MS-XCA] LZ77+ Huffman: questions about blocks

hi Dochelp,


Does the beginning of the second and subsequent blocks break the bitstream, starting again at a byte boundary after the new Huffman table?

The question is best explained by analogy to the way long lengths are handled in matches. Suppose we have a match symbol in the middle of a bitstream, and the match is a long one, requiring the reading of an extra byte:

    ijklmnop  abcDEFgh [distance] qrs...
                 |
                 [match 1, 15]

Here abc, ghi.. are the sequence of bits in the stream around the match DEF, which is read in alternating bytes by little-endian rules, and the distance is plonked in the middle of the stream as an individual byte. The stream just flows around it, so gh-ijklmnop are interpreted after [distance].

Now, if DEF instead ended the block:

    ijklmnop  abcDEFgh [new Huffman table] qrs...
                 |
                 [ends the block (64k)]


would the bits gh-jklmnop be interpreted using the new Huffman table, as part of the new block, or would those bits be dropped?

Multi-block examples would of course be helpful.


Douglas