Possible bug in opcZipRawInflateBuffer

Coordinator
Feb 13 at 5:09 PM
Hi,

I think I've uncovered a bug in opcZipRawInflateBuffer() which will occur when the compressed size is just over 4096 bytes.

In my case, I have a ppt/presentation.xml which starts at 23,251 bytes and is compressed by two passes through deflate to 4097 bytes. This is done using two writes of 4096 bytes (full buffer) and 1 byte.

Later, when the same document inflated, the first call to zlib inflate() will create the buffer of 23,251 bytes using the first block of 4096 compressed bytes and returns Z_OK (note, it has not yet returned Z_STREAM_END!!).

The loop on zip.p # 487 exits b/c bof_ofs now == buf_len but the inflate() sequence is not yet done.

This later leads to failure in opcZipCleanupInflateState where it checks and finds that
state->stream.total_in!=compressed_size
i.e.
(4096 != 4097)

I think the fix is to change the loop in zip.c#487 and I'm trying that now but thought I'd post here in case others have already addressed this and have the solution in hand. flr??

John
Coordinator
Feb 13 at 7:34 PM
To follow up with my own findings.

I do believe that there is a bug in opcZipRawInflateBuffer but I cannot figure out the proper loop conditions to know when there is more of the stream available. I tried:
  • Loop until Z_STREAM_END. This works for the problematic segment but error is returned from inflate for other segments on the second pass through. It's not clear why it doesn't return Z_STREAM_END for the initial or subsequent calls there.
  • Changed byte count condition from ... && buf_ofs<buf_len) to ... && state->compressed_size > stream.total_in). Again this works for the 4097 byte segment but over-reads for other segments resulting in a stream error.
After spending three days debugging this, I'm resigned that my resolution will be to increase the deflate buffer from 4k to 128k as advised in the zlib how-to here:
http://zlib.net/zlib_how.html
#define OPC_DEFLATE_BUFFER_SIZE 128*1024 // Was: 4096
I do not anticipate any segments will require more than 128k of compressed bytes so I hope to keep the processing to one deflate buffer and try to avoid the bug.. Just beware, there is definitely a zlib buffer handling bug in the libopc code.

John