Why Do We Compress Images? The Story of Making Pictures Smaller

A photograph on a screen looks like one thing. To a computer, it is millions of numbers.

Consider an ordinary 1920 × 1080 RGB image. It contains 2,073,600 pixels. If each pixel stores 8 bits for red, 8 for green and 8 for blue, the raw pixel data comes to 6,220,800 bytes, or about 5.93 MiB.

That is before a file format has added anything of its own. It is simply the cost of writing down every pixel, one after another.

The strange part is that a photograph contains a lot of information that does not need to be written down so literally. A blue sky usually does not jump from one completely unrelated colour to another from pixel to pixel. A wall may occupy thousands of pixels that are nearly identical. Edges, textures and repeated patterns have structure.

Compression takes advantage of that structure. The trick is knowing which information can be represented more efficiently, and, for lossy formats, which information can be sacrificed without making the picture look wrong.

The history does not begin with JPEG. In fact, it does not begin with pictures at all.

In 1948, Claude Shannon published A Mathematical Theory of Communication in the Bell System Technical Journal. Shannon was working at Bell Labs, where engineers were dealing with a very different problem: how to represent and transmit information efficiently over communication systems.

His work gave engineers a mathematical language for talking about information, uncertainty and redundancy. A message could contain patterns, and those patterns could be used when deciding how much information actually needed to be transmitted.

A few years later, David Huffman's 1952 paper described a method for constructing minimum-redundancy codes. Huffman coding became one of the standard tools of data compression and would eventually find its way into image and multimedia formats.

Neither Shannon nor Huffman invented a modern photo compressor. Their work was more fundamental than that. They helped establish the mathematical ideas that would make efficient digital representation possible.

There was a second problem to solve. Even if an image contains redundancy, how should an encoder find it?

Looking at pixels one at a time is not always the best way to describe a photograph. Engineers began looking for mathematical transformations that could describe the patterns inside a signal in a more useful form.

One important result appeared in 1974, when Nasir Ahmed, T. Raj Natarajan and K. R. Rao published their paper on the Discrete Cosine Transform in IEEE Transactions on Computers.

The paper did not set out to create JPEG. It described the DCT as a mathematical transform and examined its use in digital processing. What made the technique valuable for compression was the way it could concentrate much of the important information in relatively few coefficients.

In simpler terms, instead of asking what every pixel is doing individually, the transform gives the encoder another description of the same data. Smooth changes, broad shapes and fine detail show up differently. That gives a compression system somewhere to make decisions about precision.

By the 1980s, digital images were becoming more important, but the hardware and networks of the time made large files expensive to store and move. A photograph could be useful only if computers had a practical way to handle it.

The Joint Photographic Experts Group began its standardization work in 1986. JPEG 1 was created in 1992, with the latest version of the standard published in 1994.

JPEG was aimed at continuous-tone still images, the kind of pictures represented by photographs. Its encoder combines several techniques rather than relying on one magic compression trick. The image is transformed, the resulting values are quantized, and the remaining information is encoded efficiently.

The quantization step is where much of JPEG's characteristic size reduction comes from. Values are rounded with different levels of precision. Some details survive almost unchanged. Others are represented more coarsely.

That makes JPEG lossy. Once information has been discarded during quantization, decoding cannot reconstruct the original pixels exactly.

A typical baseline JPEG starts by working on small blocks of the image. The familiar 8 × 8 block is important because the transform and quantization stages operate on these groups of pixels.

The DCT turns each block from a collection of pixel values into coefficients describing spatial frequencies. The low-frequency coefficients describe broad changes. Higher-frequency coefficients describe increasingly fine detail.

Quantization then reduces the precision of those coefficients. The encoder can be much less careful with information that is difficult to see than with information that defines the major shapes and brightness changes in the photograph.

The result is a file that can be dramatically smaller than the raw pixel data. Turn the compression too far, however, and the mathematics becomes visible. Fine textures disappear. Edges develop ringing. The boundaries of the 8 × 8 blocks can start to show.

A JPEG therefore contains a compromise. It is not a smaller copy of the original photograph. It is a new representation of it, designed to preserve enough of what people notice while spending fewer bits on everything else.

There is a small piece of JPEG history hiding in plain sight on almost every computer.

JPEG describes the compression standard. JFIF, the JPEG File Interchange Format, describes a practical way of exchanging JPEG-encoded images between programs and platforms.

The first JFIF version was released to a small group in 1991. Version 1.01 was publicly released in 1992, followed shortly by version 1.02. JFIF became a de facto standard for exchanging JPEG images.

That distinction matters because a compression algorithm is only useful if software can reliably store, recognize and exchange what it produces. JPEG provided the coding technology. JFIF helped make that technology portable.

It is one reason the history of image formats is not simply a list of better compression algorithms. File formats also have to survive contact with operating systems, applications, browsers and millions of unrelated pieces of software.

JPEG worked beautifully for photographs because photographs contain small variations in colour and detail that people are often willing to lose.

A screenshot is different.

Take a page of black text on a white background. The letters have sharp boundaries. A diagram may have one-pixel lines. A logo may depend on a precise edge. Throwing away a little information around those boundaries can make the result visibly worse.

Transparency creates another problem. A graphic might need to sit over different backgrounds while retaining a clean edge.

For those images, exact reconstruction can be more valuable than aggressive size reduction. That is where lossless compression becomes important.

PNG was created in December 1994 as a patent-free replacement for GIF. The format was designed for lossless, portable graphics and for the practical demands of the growing World Wide Web.

The W3C made PNG a Recommendation on October 1, 1996. The specification supported features that mattered to web graphics, including transparency, colour information and progressive display.

The distinction between PNG and JPEG was clear even in the original W3C material. PNG was intended for lossless images and was not presented as a replacement for JPEG's role in lossy photographic compression.

That division of labour is still familiar. A photograph often gets much smaller as a JPEG. A screenshot, diagram or graphic with sharp edges may be better kept lossless.

There was never going to be one compression method that was ideal for every kind of picture. The images themselves were too different.

Before PNG, there was GIF.

CompuServe introduced GIF87a on June 15, 1987 as a colour image format for its online service. At the time, downloading an image was a very different experience from opening one today. A connection could be slow enough that the number of bytes in an image mattered immediately.

GIF used a palette of up to 256 colours and LZW compression. That made it particularly useful for graphics with relatively simple colour palettes. It was not designed to preserve the full range of colours in a photographic image.

GIF's success was partly about compression, but it was also about portability. A graphics format was useful because different machines and software could agree on what the file meant.

The web would eventually make that lesson much more important. Images were no longer files people occasionally downloaded. They were becoming part of almost every page people visited.

For decades, compression had been closely associated with storage and transmission. Then the web turned every image into a network request.

In Google's 2010 announcement of WebP, the company said images and photographs accounted for about 65 percent of the bytes transmitted per web page at the time. The argument for a new image format was straightforward: if the same picture could be delivered using fewer bytes, the page could load faster.

WebP was built around that problem. Its first release focused on lossy compression for photographs and used technology based on the VP8 codec. Later versions added lossless compression and transparency.

The significance of WebP was not simply that it produced another file extension. It reflected a change in what engineers were optimizing for. The cost of an image was increasingly measured in milliseconds and megabytes transferred to someone's browser, not only in space on a hard drive.

The next major step came from the world of video compression.

AVIF, the AV1 Image File Format, packages AV1 image data inside the HEIF file format. Version 1.0.0 of the specification was published on February 19, 2019.

AVIF brought modern codec technology to still images and supports features including high bit depths, HDR, wide colour gamut and transparency. It can also represent image sequences.

The attraction is obvious for the web: modern coding techniques can produce small files while retaining a great deal of visual information. But AVIF also illustrates something that has always been true of compression. A smaller file is not automatically a better file.

Encoding and decoding have costs. Software support matters. The right format depends on the image, the device, the workflow and the reason the image is being delivered in the first place.

Return to that 1920 × 1080 RGB image. It has 2,073,600 pixels, and at 24 bits per pixel the raw pixel data occupies 6,220,800 bytes, about 5.93 MiB.

A compressed file does not need to write all of those pixel values independently. A lossless method can exploit patterns and statistical redundancy. A lossy method can go further by changing the representation and discarding information that has relatively little visual importance.

There is no fixed percentage that every image can be compressed by. A smooth photograph, a noisy photograph, a screenshot and a line drawing have very different structures.

That is why two files with exactly the same dimensions can end up with very different sizes. The dimensions tell you how many pixels exist. They do not tell you how difficult those pixels are to compress.

Storage became cheaper. Networks became faster. Cameras became dramatically better.

The files grew with them.

A modern phone can produce an image containing millions of pixels in seconds. A web page can request dozens of images. A messaging service can store enormous numbers of photographs. An email system can impose a surprisingly small attachment limit. A website may reject an upload that is only a few megabytes too large.

The constraint keeps moving. Sometimes it is bandwidth. Sometimes it is storage. Sometimes it is the time a user is willing to wait. Sometimes it is simply a file-size limit imposed by another system.

Compression remains useful because the amount of visual information we want to move around keeps increasing.

When someone says they want to compress an image, there is usually another question hiding underneath.

How small does it need to be?

If an upload must be below 100 KB, making the file 20 KB instead of 90 KB may accomplish nothing useful if the extra compression makes the image noticeably worse. If the image is being used as a thumbnail, the right answer may be to resize it first. If it is a screenshot containing text, lossless compression may be the better choice.

There is no universal setting because there is no universal purpose.

The engineers who worked on information theory, transforms and image standards were solving versions of the same practical problem: how much data do we really need to keep?

Today, a person can answer that question with a few clicks. The machinery underneath those clicks took decades to build.