Browse zlib specs (55 of 62)
Proofs / zlib / Earlier, longer properties
x_deflate_roundtrip
One inflate call gives back exactly what deflate compressed, for any parameters, tuning, dictionary and gzip header (which comes back too) and any sequence of deflate (flush up to Z_BLOCK) and deflateParams calls with any split (data up to 1 MiB, output space up to 1 MiB per call, 4 MiB of compressed output).
- spec
- 84 lines of C
- with its headers
- 473 lines
- status
- Proved
- 55 of 62
On this page
The spec
The spec is the C program harness/x_deflate_roundtrip.c, 84 lines, shown whole as it is in the repository at 37189f4. It includes harness/deflate_common.h (234 lines) and harness/zlib_common.h (155 lines). Its first comment states the property in full, with every bound.
// Law deflate_roundtrip: inflate gives back exactly what deflate compressed, for every parameter and call sequence of
// deflate. The compression run of harness/deflate_common.h: deflateInit2 with ANY level, method, windowBits,
// memLevel, strategy (any ints); optionally deflateTune with ANY four ints, a preset dictionary of ANY dlen <= 2^15
// bytes, a gzip header with ANY fields; ANY data of n <= 2^20 bytes, compressed by ANY sequence of deflate() calls
// with ANY flush values (Z_NO_FLUSH .. Z_BLOCK) and deflateParams calls with ANY level and strategy, ANY split of the
// data and of the output space (at most 2^20 bytes per call, objects of exactly avail_in and avail_out bytes); the
// first Z_FINISH ends the data (data[0..nf)). When deflateInit2 accepted the parameters and the run reached
// Z_STREAM_END, the output is inflated in one call (inflateInit2 with the matching windowBits: -w for raw, 0 for
// zlib, 32 for gzip; the dictionary set when inflate asks for it (zlib, Z_NEED_DICT with strm->adler = the
// dictionary's Adler-32) or before (raw); inflateGetHeader with buffers of exactly the header fields' sizes), its
// input exactly the compressed bytes, its output space exactly nf bytes, flush Z_FINISH. Properties:
// - inflate returns Z_STREAM_END, having consumed all the compressed bytes and produced exactly nf bytes, which are
// data[0..nf);
// - with a gzip header: inflate reports it complete (done = 1) with the text flag, time, OS, header-CRC flag, extra
// field, name and comment that were given to deflateSetHeader;
// - deflateEnd returns Z_OK.
// (Allocation failures, Z_MEM_ERROR, end the run.) Bounds: n <= 2^20, dlen <= 2^15, header fields <= 2^12 bytes,
// at most 2^22 bytes of compressed output in total.
#include "deflate_common.h"
int main(void)
{
struct drun R;
z_stream s;
if (!deflate_run(&R, &s, 0) || R.init != Z_OK)
return 0;
gp_assert(deflateEnd(&s) == Z_OK);
z_stream t;
stream_defaults(&t);
const int iw = R.wrap == 0 ? -R.wb : R.wrap == 1 ? 0 : 32;
int r = inflateInit2(&t, iw);
if (r != Z_OK)
return 0;
gz_header ih;
if (R.head_set) {
ih.extra = malloc(R.xlen);
ih.extra_max = R.xlen;
ih.name = malloc(R.nl + 1);
ih.name_max = R.nl + 1;
ih.comment = malloc(R.cl + 1);
ih.comm_max = R.cl + 1;
if (ih.extra == NULL || ih.name == NULL || ih.comment == NULL)
return 0;
gp_assert(inflateGetHeader(&t, &ih) == Z_OK);
}
if (R.wrap == 0 && R.dict_set) {
r = inflateSetDictionary(&t, R.dict, R.dlen);
if (r == Z_MEM_ERROR)
return 0;
gp_assert(r == Z_OK);
}
unsigned char *ib = exact_copy(R.comp, (unsigned) R.clen);
unsigned char *ob = malloc(R.nf);
if (ib == NULL || ob == NULL)
return 0;
t.next_in = ib;
t.avail_in = (unsigned) R.clen;
t.next_out = ob;
t.avail_out = R.nf;
r = inflate(&t, Z_FINISH);
if (R.wrap == 1 && R.dict_set) {
gp_assert(r == Z_NEED_DICT);
gp_assert(t.adler == ref_adler32(1, R.dict, R.dlen));
r = inflateSetDictionary(&t, R.dict, R.dlen);
if (r == Z_MEM_ERROR)
return 0;
gp_assert(r == Z_OK);
r = inflate(&t, Z_FINISH);
}
if (r == Z_MEM_ERROR)
return 0;
gp_assert(r == Z_STREAM_END);
gp_assert(t.avail_in == 0 && t.avail_out == 0);
gp_assert(bytes_same(ob, R.data, R.nf));
if (R.head_set) {
gp_assert(ih.done == 1);
gp_assert(ih.text == R.h.text && ih.time == R.h.time && ih.os == R.h.os && ih.hcrc == R.h.hcrc);
gp_assert(ih.extra_len == R.xlen && bytes_same(ih.extra, R.xf, R.xlen));
gp_assert(bytes_same(ih.name, R.nf_, R.nl + 1));
gp_assert(bytes_same(ih.comment, R.cf, R.cl + 1));
}
inflateEnd(&t);
return 0;
}harness/deflate_common.h, 234 lines
// Shared by the deflate laws (deflate_roundtrip, deflate_api): a compression run with ANY parameters and ANY sequence
// of calls, recorded so that the laws can check it. See harness/zlib_common.h for the conventions (draws, exact
// buffers).
//
// The run (deflate_run):
// - deflateInit2 with ANY level, method, windowBits, memLevel and strategy (any int each);
// - at the caller's choice: deflateTune with ANY four ints; deflateSetDictionary with ANY dictionary of dlen <= 2^15
// bytes; deflateSetHeader with a gzip header of ANY text flag, time, OS, header-CRC flag, an extra field of ANY
// xlen <= 2^12 bytes, a name and a comment of ANY length <= 2^12 of ANY non-zero bytes (each with its zero byte);
// - then ANY number of calls until deflate returns Z_STREAM_END: deflate() with ANY flush value of zlib.h
// (Z_NO_FLUSH .. Z_BLOCK), or deflateParams with ANY level and strategy, each offered the next ANY number of not yet
// consumed bytes of the data (n <= 2^20 bytes, an object of exactly that many per call) and ANY output space (at
// most 2^20 bytes, an object of exactly that many per call); the first Z_FINISH ends the data: the data is what has
// been offered up to that call, and every later call is deflate(Z_FINISH) offered the rest of it (zlib.h: "If
// deflate returns with avail_out == 0, this function must be called again with Z_FINISH and more output space ...
// but no more input data").
// Bound: the total output (at most 2^22 bytes) is kept for the laws.
#ifndef ZLIB_HARNESS_DEFLATE_COMMON_H
#define ZLIB_HARNESS_DEFLATE_COMMON_H
#include "zlib_common.h"
#define GP_DATA_MAX (1u << 20)
#define GP_DICT_MAX (1u << 15)
#define GP_HSTR_MAX (1u << 12)
#define GP_DOUT_MAX (1u << 20)
#define GP_COMP_MAX (1u << 22)
// deflateInit2's parameter check (zlib.h, deflateInit2): accepted values
static int deflate_params_ok(int level, int method, int wbits, int mem, int strategy)
{
int wrap = 1;
if (level == Z_DEFAULT_COMPRESSION)
level = 6;
if (wbits < 0) {
if (wbits < -15)
return 0;
wrap = 0;
wbits = -wbits;
} else if (wbits > 15) {
wrap = 2;
wbits -= 16;
}
return mem >= 1 && mem <= 9 && method == Z_DEFLATED && wbits >= 8 && wbits <= 15 && level >= 0 && level <= 9 &&
strategy >= 0 && strategy <= Z_FIXED && (wbits != 8 || wrap == 1);
}
struct drun {
int level, method, wbits, mem, strategy;
int init; // deflateInit2's result
int wrap; // 0 raw, 1 zlib, 2 gzip
int wb; // the window bits (8..15; 8 means 9 for deflate)
unsigned char *data; // the data: data[0..n), of which the first nf were compressed
unsigned n, nf;
unsigned char *dict; // the dictionary (dlen bytes) when set (dict_set)
unsigned dlen;
int dict_set; // a non-empty dictionary was set
int head_set; // a gzip header was set
gz_header h;
unsigned char *xf, *nf_, *cf; // the header's extra, name, comment fields
unsigned xlen, nl, cl;
unsigned char *comp; // the compressed output, clen bytes
unsigned long clen;
int ended; // deflate returned Z_STREAM_END
int api_ok; // every call's API check held (deflate_api)
};
// 1 when every byte of buf[0..n) is non-zero
static void draw_nonzero_bytes(unsigned char *buf, unsigned n)
{
nondet_bytes(buf, n);
for (unsigned i = 0; i < n; ++i)
gp_assume(buf[i] != 0);
}
// the run; returns 0 when the harness's own allocations fail (the run ends there), else 1
static int deflate_run(struct drun *R, z_stream *s, int api)
{
R->level = (int) nondet_u32();
const int mval = (int) nondet_u32();
R->wbits = (int) nondet_u32();
R->mem = (int) nondet_u32();
R->strategy = (int) nondet_u32();
R->n = nondet_u32();
R->dlen = nondet_u32();
const int t1 = (int) nondet_u32(), t2 = (int) nondet_u32(), t3 = (int) nondet_u32(), t4 = (int) nondet_u32();
R->xlen = nondet_u32();
R->nl = nondet_u32();
R->cl = nondet_u32();
const unsigned long mtime = nondet_u32();
const unsigned char msel = nondet_u8();
const unsigned char tsel = nondet_u8();
const unsigned char dsel = nondet_u8();
const unsigned char hsel = nondet_u8();
const unsigned char text = nondet_u8();
const unsigned char os = nondet_u8();
const unsigned char hcrc = nondet_u8();
gp_assume(R->n <= GP_DATA_MAX && R->dlen <= GP_DICT_MAX);
gp_assume(R->xlen <= GP_HSTR_MAX && R->nl <= GP_HSTR_MAX && R->cl <= GP_HSTR_MAX);
R->method = msel == 0 ? mval : Z_DEFLATED;
R->data = malloc(R->n);
R->dict = malloc(R->dlen);
R->xf = malloc(R->xlen);
R->nf_ = malloc(R->nl + 1);
R->cf = malloc(R->cl + 1);
R->comp = malloc(GP_COMP_MAX);
if (R->data == NULL || R->dict == NULL || R->xf == NULL || R->nf_ == NULL || R->cf == NULL || R->comp == NULL)
return 0;
nondet_bytes(R->data, R->n);
nondet_bytes(R->dict, R->dlen);
nondet_bytes(R->xf, R->xlen);
draw_nonzero_bytes(R->nf_, R->nl);
R->nf_[R->nl] = 0;
draw_nonzero_bytes(R->cf, R->cl);
R->cf[R->cl] = 0;
R->clen = 0;
R->ended = 0;
R->api_ok = 1;
R->dict_set = 0;
R->head_set = 0;
R->nf = R->n;
stream_defaults(s);
R->init = deflateInit2(s, R->level, R->method, R->wbits, R->mem, R->strategy);
if (api) {
gp_assert(R->init == Z_OK || R->init == Z_STREAM_ERROR || R->init == Z_MEM_ERROR);
gp_assert((R->init == Z_STREAM_ERROR) == !deflate_params_ok(R->level, R->method, R->wbits, R->mem, R->strategy));
}
if (R->init != Z_OK)
return 1;
R->wrap = R->wbits < 0 ? 0 : R->wbits > 15 ? 2 : 1;
R->wb = R->wbits < 0 ? -R->wbits : R->wbits > 15 ? R->wbits - 16 : R->wbits;
int r;
if (tsel == 0) {
r = deflateTune(s, t1, t2, t3, t4);
if (api)
gp_assert(r == Z_OK);
}
if (dsel == 0) {
r = deflateSetDictionary(s, R->dict, R->dlen);
if (api)
gp_assert((r == Z_STREAM_ERROR) == (R->wrap == 2) && (r == Z_OK || r == Z_STREAM_ERROR));
R->dict_set = r == Z_OK && R->dlen > 0; // an empty dictionary is no dictionary (no FDICT)
}
if (hsel == 0) {
R->h.text = text & 1;
R->h.time = mtime;
R->h.xflags = 0;
R->h.os = os;
R->h.extra = R->xf;
R->h.extra_len = R->xlen;
R->h.extra_max = 0;
R->h.name = R->nf_;
R->h.name_max = 0;
R->h.comment = R->cf;
R->h.comm_max = 0;
R->h.hcrc = hcrc & 1;
R->h.done = 0;
r = deflateSetHeader(s, &R->h);
if (api)
gp_assert((r == Z_OK) == (R->wrap == 2) && (r == Z_OK || r == Z_STREAM_ERROR));
R->head_set = r == Z_OK;
}
unsigned pos = 0, lim = 0;
int finishing = 0;
for (;;) {
const unsigned nlim = nondet_u32();
const unsigned aout = nondet_u32();
const int fsel = (int) nondet_u32();
const int plev = (int) nondet_u32();
const int pstrat = (int) nondet_u32();
const unsigned char psel = nondet_u8();
gp_assume(aout <= GP_DOUT_MAX && aout <= GP_COMP_MAX - R->clen);
int flush = fsel;
if (finishing) {
flush = Z_FINISH;
lim = R->nf;
} else {
gp_assume(pos <= nlim && nlim <= R->n);
gp_assume(flush >= Z_NO_FLUSH && flush <= Z_BLOCK);
lim = nlim;
if (flush == Z_FINISH) {
finishing = 1;
R->nf = lim;
}
}
const unsigned ain = lim - pos;
unsigned char *ib = exact_copy(R->data + pos, ain);
unsigned char *osnap;
unsigned char *ob = exact_space(aout, &osnap);
if (ib == NULL || ob == NULL)
return 0;
s->next_in = ib;
s->avail_in = ain;
s->next_out = ob;
s->avail_out = aout;
const unsigned long tin = s->total_in, tout = s->total_out;
const int params = !finishing && psel == 0;
if (params)
r = deflateParams(s, plev, pstrat);
else
r = deflate(s, flush);
const unsigned used = ain - s->avail_in;
const unsigned made = aout - s->avail_out;
if (api) {
gp_assert(s->avail_in <= ain && s->avail_out <= aout);
gp_assert(s->next_in == ib + used && s->next_out == ob + made);
gp_assert(s->total_in == tin + used && s->total_out == tout + made);
gp_assert(bytes_same(ib, R->data + pos, ain));
gp_assert(bytes_same(ob + made, osnap + made, aout - made));
if (params) {
const int pok = plev >= -1 && plev <= 9 && pstrat >= 0 && pstrat <= Z_FIXED;
gp_assert(r == Z_OK || r == Z_BUF_ERROR || r == Z_STREAM_ERROR);
gp_assert((r == Z_STREAM_ERROR) == !pok);
} else {
gp_assert(r == Z_OK || r == Z_STREAM_END || r == Z_BUF_ERROR);
if (r == Z_STREAM_END)
gp_assert(flush == Z_FINISH && pos + used == R->nf && s->avail_in == 0);
if (r == Z_BUF_ERROR)
gp_assert(used == 0 && made == 0);
}
}
bytes_copy(R->comp + R->clen, ob, made);
R->clen += made;
pos += used;
free(ib);
free(ob);
free(osnap);
if (!params && r == Z_STREAM_END)
break;
}
R->ended = 1;
return 1;
}
#endifharness/zlib_common.h, 155 lines
// Shared by the zlib harnesses (zlib 1.3.2 core: adler32.c, crc32.c, deflate.c, trees.c, inflate.c, inffast.c,
// inftrees.c, compress.c, uncompr.c, zutil.c).
//
// A harness draws every input from quickproof.h (nondet_*, pick), restricts it with gp_assume, and states each property
// with gp_assert (undefined behaviour when the property fails). Buffers have exactly the size the claim allows. Every
// call of inflate() or deflate() gets its input and its output space as objects of their own, allocated with exactly
// avail_in and avail_out bytes (exact_copy, exact_space): an access before next_in, after next_in + avail_in, before
// next_out or after next_out + avail_out is then outside an object, i.e. undefined behaviour in the semantics of the
// theorem, so "inflate reads only the input it is offered and writes only the space it is offered" is part of every
// law's "no undefined behaviour".
//
// The reference facts, written from the definitions, not from zlib:
// - Adler-32 (RFC 1950 8.2): s1 = 1 + the sum of the bytes, s2 = the sum of the successive values of s1, both mod
// 65521; the checksum is s2 * 65536 + s1. ref_adler32 updates a checksum byte by byte;
// - CRC-32 (RFC 1952 8, ISO 3309): the reflected polynomial 0xedb88320, register preset to all ones and inverted at
// the end. ref_crc32 updates a checksum bit by bit;
// - the zlib wrapper (RFC 1950): 2 header bytes, an optional 4-byte dictionary id (big-endian Adler-32 of the
// dictionary), the deflate data, then the big-endian Adler-32 of the uncompressed data;
// - the gzip wrapper (RFC 1952): a header of at least 10 bytes (1f 8b 08 FLG MTIME(4) XFL OS, then the optional
// fields), the deflate data, then the little-endian CRC-32 and the little-endian length mod 2^32 of the data;
// - the inflate windowBits of zlib.h: 8..15 (zlib wrapper, 2^windowBits window), 0 (zlib wrapper, the window size of
// the header), -8..-15 (raw deflate), + 16 (gzip wrapper only; 16 itself: the window size of the header), + 32
// (zlib or gzip, detected); everything else is rejected with Z_STREAM_ERROR.
#ifndef ZLIB_HARNESS_COMMON_H
#define ZLIB_HARNESS_COMMON_H
#include <stddef.h>
#include <stdint.h>
#include <stdlib.h>
#include "quickproof.h"
#include "zlib.h"
#define GP_BASE 65521ul // the Adler-32 modulus (RFC 1950)
#define GP_POLY 0xedb88320ul // the CRC-32 polynomial, reflected (RFC 1952)
// Adler-32 of buf[0..n-1] continued from the checksum a (both halves < 65521 for a checksum).
static unsigned long ref_adler32(unsigned long a, const unsigned char *buf, unsigned long n)
{
unsigned long s1 = a & 0xffff;
unsigned long s2 = (a >> 16) & 0xffff;
for (unsigned long i = 0; i < n; ++i) {
s1 = (s1 + buf[i]) % GP_BASE;
s2 = (s2 + s1) % GP_BASE;
}
return (s2 << 16) | s1;
}
// The CRC-32 register update by one byte, without pre- or post-conditioning (the register is the 32 low bits).
static unsigned long ref_crc_byte(unsigned long c, unsigned char b)
{
c ^= b;
for (int k = 0; k < 8; ++k) {
if (c & 1)
c = (c >> 1) ^ GP_POLY;
else
c = c >> 1;
}
return c;
}
// CRC-32 of buf[0..n-1] continued from the checksum c (its 32 low bits).
static unsigned long ref_crc32(unsigned long c, const unsigned char *buf, unsigned long n)
{
c = (~c) & 0xffffffff;
for (unsigned long i = 0; i < n; ++i)
c = ref_crc_byte(c, buf[i]);
return (~c) & 0xffffffff;
}
static unsigned long be32(const unsigned char *p)
{
return ((unsigned long) p[0] << 24) | ((unsigned long) p[1] << 16) | ((unsigned long) p[2] << 8) | p[3];
}
static unsigned long le32(const unsigned char *p)
{
return ((unsigned long) p[3] << 24) | ((unsigned long) p[2] << 16) | ((unsigned long) p[1] << 8) | p[0];
}
static int bytes_same(const unsigned char *a, const unsigned char *b, unsigned long n)
{
for (unsigned long i = 0; i < n; ++i)
if (a[i] != b[i])
return 0;
return 1;
}
static void bytes_copy(unsigned char *d, const unsigned char *s, unsigned long n)
{
for (unsigned long i = 0; i < n; ++i)
d[i] = s[i];
}
// A new object of exactly n bytes holding src[0..n-1] (NULL when the allocation fails).
static unsigned char *exact_copy(const unsigned char *src, unsigned n)
{
unsigned char *p = malloc(n);
if (p != NULL)
bytes_copy(p, src, n);
return p;
}
// A new object of exactly n bytes with any contents, and a snapshot of them in *snap (NULL when an allocation fails).
static unsigned char *exact_space(unsigned n, unsigned char **snap)
{
unsigned char *p = malloc(n);
unsigned char *q = malloc(n);
if (p == NULL || q == NULL) {
free(p);
free(q);
*snap = NULL;
return NULL;
}
nondet_bytes(p, n);
bytes_copy(q, p, n);
*snap = q;
return p;
}
// inflateInit2's windowBits check (zlib.h): accepted values.
static int inflate_wbits_ok(int w)
{
if (w < 0)
return w >= -15 && w <= -8;
if (w >= 48)
return 0;
w &= 15;
return w == 0 || (w >= 8 && w <= 15);
}
// Is a value one of inflate()'s documented return codes (zlib.h)?
static int inflate_code_ok(int r)
{
return r == Z_OK || r == Z_STREAM_END || r == Z_NEED_DICT || r == Z_DATA_ERROR || r == Z_STREAM_ERROR ||
r == Z_MEM_ERROR || r == Z_BUF_ERROR;
}
// A z_stream with the default allocator (zalloc, zfree, opaque all Z_NULL: zlib uses zcalloc/zcfree, i.e. malloc and
// free) and any values in the fields the caller does not have to set.
static void stream_defaults(z_stream *s)
{
s->zalloc = Z_NULL;
s->zfree = Z_NULL;
s->opaque = Z_NULL;
s->next_in = Z_NULL;
s->avail_in = 0;
s->next_out = Z_NULL;
s->avail_out = 0;
pick(s->total_in);
pick(s->total_out);
pick(s->adler);
pick(s->data_type);
pick(s->reserved);
}
#endifThe theorem
QuickProof generated this statement from the spec, in src/contracts/x_deflate_roundtrip.v:
Theorem x_deflate_roundtrip : harness_safe whole_prog.src/contracts/laws.v states it on the spec's own program, whole_prog (the spec linked with the code under test), with its proof:
Theorem x_deflate_roundtrip : harness_safe Zlib.contracts.x_deflate_roundtrip.whole_prog.
Proof. exact (proj1 (proj2 (proj2 (proj2 (proj2 (proj2 (proj2 (proj2 (proj2 Zlib.contracts.suite.suite_x))))))))). Qed.Read as a statement about runs, it says that no run of the program has undefined behaviour or a failing check, for every value the spec can draw. That reading rests on VST's adequacy theorem, which is not yet instantiated in Rocq (formerly Coq) for QuickProof's environment.
The C files (units) its program links
adler32.ccrc32.cdeflate.ctrees.cinflate.cinffast.cinftrees.czutil.cquickproof/libc/libc.c
What it leaves out
- What the first comment of the spec excludes, and its bounds.
- Like every spec of this proof: termination (a run that never returns is not a counterexample), and binaries built by a compiler (the theorem is about the code as CompCert defines C for x86_64, LP64), one thread.
- Where zlib reports
Z_MEM_ERROR, most specs end the run and claim nothing about it; the first comment of the spec says so. - The edits to zlib's code, listed in the repository, are tested, not proved.
The summary page says what the proof as a whole covers.
How it was proved
AI agents wrote the proof, and Rocq checks it. qp verify passes on this repository: this statement is the one generated from the spec at the contract commit, and its proof uses only Rocq's standard axioms and VST's abstract malloc library.