There seems to be a lot of confusion about zsync's custom embedded version of zlib. I had not documented the exact reason for the patches very well; so I have now committed an explanation of the changes to my own repo. As I may not make another release for a little while yet, I am posting the explanation here also.

I have started a discussion with the zlib maintainer about what sort of API changes could be made such that I could use the standard zlib, but so far no-one other than me understands the requirements and I'm not actually bothered about it (it's the Fedora people who are worked up about it). So, absent anyone else stepping in to do the work, it may take me a while to get to it.

Local changes to zlib used by zsync

There are two different modes of operation that zsync supports that these patches are designed to support:

Changes to the deflate code: Compressing a file in a way that is optimised for zsync's block-based rsync algorithm ‒ starting a new zlib block for each 1024 byte (for example) block in the source file. cf http://zsync.moria.org.uk/paper/ch03s04.html . This is used by makegz.c in the zsync source.

Changes to the inflate code: Working with files compressed with the standard gzip(1). To enable people to get started with zsync, I want it to work with existing compressed content. To achieve optimal results with standard gzip files, I made zsync capable of starting decompression in the middle of a block. In these cases it has to download the block header, then skips forward to the part of the block that gives it the data that it wants. cf http://zsync.moria.org.uk/paper/ch03s02.html

Contrary to some internet discussion, the changes are not related to rsync's changes nor to rsync compatibility (zsync isn't compatible with rsync - whatever that would mean ‒ nor do these changes relate in any way to the rsyncable gzip patch).

Changes to the deflate code

Essentially, I hijacked Z_PARTIAL_FLUSH to mean something new ‒ I want to start a new zlib block, but unlike Z_PARTIAL_FLUSH I don't need to emit a whole byte between blocks, so I took that out. Correctly this ought to be implemented by adding a Z_NEWBLOCKONLY_FLUSH or something like that instead of repurposing an existing state.

(If this were the only issue preventing the use of a standard zlib, distros could change it to use Z_PARTIAL_FLUSH with only a slight loss of compression efficiency.)

Changes to the inflate code

zsync uses the rsync algorithm to construct a desired file from an (e.g.) older local version of the file and then downloading any new/needed blocks from a server; the aim being to minimise the amount of data downloaded to construct the target file. It supports downloading those blocks from a gzipped version of the file on the server. If I want e.g. bytes 4096-8192 of the file from inside the gzipped file, I could download the whole zlib block (using a map of the compressed file that I construct beforehand and is downloaded first) containing the range 4096-8192 (zsync 0.1.0 used this method); but it can do better (fewer bytes downloaded) than that, by downloading just the block header and then downloading the bytes within that compressed block that correspond to bytes 4096-8192 of the contained data.

To do that, I need: a) to be able to start inflating at the start of "any length/literal/eob code in any dynamic or fixed block, or at any stored byte in a stored block.". That is what additional function inflate_advance() and the export of updatewindow() allow me to do.

b) make a map of the gzip file that lets me know what points I should start downloading at in order to inflate particular byte ranges of the contained content. To do this, I can decompress each byte range into a buffer of that size and then quiz zlib for the position in the stream; but I need to know that the position in the stream does correspond to the start of a code or the middle of a stored block (not, e.g., that we have just read a backref and the backref expands to span the boundary; in that case, I would need to know that position where the backref started and the lib doesn't give me a way to find that out).

This is given by inflateSafePoint(), by the modification to cause the inflator to return to the caller at each code in a dynamic block (the LENDO change), and the implicit guarantee provided by using my own copy of the library that I know how the library behaves around internal states and stream position (I need a guarantee that the library won't read ahead more than it needs to, and I need to access certain member variables directly to get the bit position in the stream).

I also removed inflate_fast as I did not want to spend the time working out if it was compatible with these changes.