Compression with node:zlib

Compression takes repeated text and turns it into a shorter sequence of bytes, and the reverse turns it back into exactly the same text. node:zlib has both directions for the three formats that matter on the web: gzip, deflate and brotli. The first two are the same algorithm with and without a small header, and brotli is the newer one that a browser asks for when it can.

The lesson compresses a fixed text and reads it back, compares the three formats by size, compresses a file through a stream and ends with an archive that is not one. The sizes come from the runs on this page and from a fixed input, so a reader can repeat every number in a terminal.

gzip and gunzip

gzipSync(buffer) takes bytes and answers with bytes, and gunzipSync goes the other way. The two are the simple form for a value that is already in memory; the asynchronous versions exist for the same work without blocking the thread. A gzip archive starts with two fixed bytes, which is how a program recognises one without trusting the name of the file.

The program compresses ten copies of a three-line text, unpacks them and checks both the text and the first two bytes.

The file gzip.mjs:

import { gzipSync, gunzipSync } from 'node:zlib'; const text = 'alpha\nbeta\ngamma\n'.repeat(10); const packed = gzipSync(Buffer.from(text)); const unpacked = gunzipSync(packed).toString('utf8'); console.log('the text is', text.length, 'bytes'); console.log('compressed it is', packed.length, 'bytes'); console.log('the round trip is exact:', unpacked === text); console.log('it starts with the gzip magic bytes:', packed[0] === 0x1f && packed[1] === 0x8b);

node gzip.mjs

the text is 170 bytes
compressed it is 40 bytes
the round trip is exact: true
it starts with the gzip magic bytes: true

A hundred and seventy bytes became forty, and the round trip gave the original text back character for character. The repetition is why: ten copies of the same three lines are a pattern the algorithm can describe once and repeat, while a text with no repetition shrinks much less or grows a little.

Three formats and their sizes

The same input gives different sizes for the three formats, and the difference is the trade they make. Deflate has no header of its own and is the smallest of the three here; gzip adds a header and a checksum, which cost twelve bytes on this input; brotli compresses text better than gzip in general, but it has a slower mode and its dictionary is tuned for the web. A program that stores an archive cares about the size, and a program that answers a browser cares about the time as well.

The program compresses the same text with each format and prints the three sizes next to the size of the input.

The file compare.mjs:

import { brotliCompressSync, deflateSync, gzipSync } from 'node:zlib'; const text = 'alpha\nbeta\ngamma\n'.repeat(10); console.log('the original:', text.length, 'bytes'); console.log('gzip:', gzipSync(text).length); console.log('deflate:', deflateSync(text).length); console.log('brotli:', brotliCompressSync(text).length);

node compare.mjs

the original: 170 bytes
gzip: 40
deflate: 28
brotli: 36

Deflate is the smallest because it carries no wrapper, and gzip is twelve bytes larger for the header and the checksum that make it a file a reader can recognise, which is the whole difference on a short text. Brotli sits between the two on this input; its advantage grows with a larger text, and its cost is a slower compression, which is a decision for the tool that produces the file and not for the one that reads it.

Compressing a file

A gzip stream is a transform: it takes chunks in and gives compressed chunks out, so a file is compressed by a pipeline of a read stream, createGzip and a write stream. The same pipeline with createGunzip in the middle reads the archive back, and the lesson on streams explains why the promise form of the pipeline is the one to use.

The program writes a text into a temporary folder, compresses it into a second file, unpacks that into a third and prints the three sizes and the comparison of the first and last file.

The file stream.mjs:

import { createReadStream, createWriteStream } from 'node:fs'; import { mkdtemp, readFile, rm, stat, writeFile } from 'node:fs/promises'; import { tmpdir } from 'node:os'; import { join } from 'node:path'; import { pipeline } from 'node:stream/promises'; import { createGunzip, createGzip } from 'node:zlib'; const dir = await mkdtemp(join(tmpdir(), 'zip-')); const file = join(dir, 'notes.txt'); const packed = join(dir, 'notes.txt.gz'); const unpacked = join(dir, 'copy.txt'); const text = 'alpha\nbeta\ngamma\n'.repeat(10); await writeFile(file, text); await pipeline(createReadStream(file), createGzip(), createWriteStream(packed)); await pipeline(createReadStream(packed), createGunzip(), createWriteStream(unpacked)); console.log('the original:', (await stat(file)).size, 'bytes'); console.log('the archive:', (await stat(packed)).size, 'bytes'); console.log('the copy is the same:', (await readFile(unpacked, 'utf8')) === text); await rm(dir, { force: true, recursive: true });

node stream.mjs

the original: 170 bytes
the archive: 40 bytes
the copy is the same: true

The archive has the size the in-memory version gave for the same text, and the copy is identical to the original. This is the shape of a backup tool: read, transform, write, and a comparison at the end that does not need the whole file in memory at once.

Bad input

gunzipSync on bytes that are not a gzip archive throws, and the error carries the code Z_DATA_ERROR and a message from the compression library. The code is the part to match: a program that reads an archive from a user should answer with a message of its own and an exit status, not with the name of a library function.

The program feeds a line of text to the unpacker and prints what came back.

The file error.mjs:

import { gunzipSync } from 'node:zlib'; try { gunzipSync(Buffer.from('this is not a gzip archive')); } catch (error) { console.log('the name of the error:', error.name); console.log('the code:', error.code); console.log('the message:', error.message); }

node error.mjs

the name of the error: Error
the code: Z_DATA_ERROR
the message: incorrect header check

The check of the header failed before any output was produced, and the message says so in three words. A pipeline has the same failure in the form of a rejection, which is what a tool that unpacks a folder of archives catches and reports per file. The rest of the part is now behind the reader: files, processes, the network, hashing and compression are the tools a small service is built from.

Exercises

  1. Compress an empty buffer. How many bytes does gzip add to nothing?
  2. Compress the same text twice and compare the two archives byte by byte. Are they the same?
  3. Remove one byte from a gzip archive and unpack it. Which code does the error carry?
  4. Compress a JSON string and print the ratio of the two sizes. Does a longer text with the same keys compress better?
  5. Why does brotli usually beat gzip on text, and what does a program pay for the difference?

Questions

A new set of questions in a new order every time. Choose one answer for each, then check.

  1. What does gzipSync return?

    Answer

    A Buffer with the archive

    The synchronous form works on bytes and gives bytes back, so the caller writes them or measures them.

  2. What does this program print? import { gzipSync, gunzipSync } from "node:zlib"; const text = "alpha\nbeta\n".repeat(5); console.log(gunzipSync(gzipSync(text)).toString("utf8") === text);

    Answer

    true

    The round trip gives the original text back, so the comparison is true.

  3. What does gunzipSync need?

    Answer

    A gzip archive

    The unpacker checks the gzip header and throws Z_DATA_ERROR on anything else.

  4. What does this program print? import { gzipSync } from "node:zlib"; console.log(gzipSync("alpha\nbeta\ngamma\n".repeat(10)).length);

    Answer

    40

    The archive of the fixed text is forty bytes on the machine of this page.

  5. Which format is the smallest for the text of this lesson?

    Answer

    deflate

    Deflate carries no header or checksum, and on a short text that wrapper is the whole difference.

  6. What does this program print? import { gunzipSync } from "node:zlib"; try { gunzipSync(Buffer.from("not gzip")); } catch (error) { console.log(error.code); }

    Answer

    Z_DATA_ERROR

    The header check fails, and the code names the compression library as the source.

  7. What are the first two bytes of a gzip archive?

    Answer

    1f 8b

    Those two bytes are the magic number that identifies the format.

  8. What does this program print? import { brotliCompressSync, deflateSync } from "node:zlib"; const text = "alpha\nbeta\ngamma\n".repeat(10); console.log(deflateSync(text).length, brotliCompressSync(text).length);

    Answer

    28 36

    The two sizes come from the run on a fixed text, in the order the calls are printed.

  9. Why compress a file through a stream?

    Answer

    The file never has to fit in memory

    Read, transform and write in chunks is what keeps memory flat for a large file.

  10. What happens when a program unpacks data that was never compressed?

    Answer

    The call throws an error with a code

    The format has a header, and a check of it fails before any output exists.

The code on this page was run with Node.js 24.22.1 and linted with ESLint 10.14.0 on Alpine Linux 3.24.3 (x86_64) in a Docker container.

Article author: Arthur Isaev

Banner image