A measurement study of JavaScript minification effectiveness

Steve Gribble -- gribble (at) cs (dot) washington.edu (dot) edu

Summary: All studied minifiers were successful at producing significant compression of JavaScript. As well, in virtually all cases, gzipping the minified JavaScript (as would happen transparently if your Web server and browser combination support gzipped content encoding) results in another factor of two or three savings, on average. Of the five studied minifiers, Google's closure compiler had a slight advantage over others when combined with gzip. In contrast, the packer 2 compressor fared the worst, largely because of fixed overhead outweighing benefits for small scripts. For our script corpus, the closure compiler combined with gzip reduced the combined size of these scripts from 1,371,059 bytes to just 262,493 bytes, a factor of five savings (a reduction factor of 0.19).

Introduction

This report examines the effectiveness of JavaScript minifiers and compares them to the effectiveness of gzip. As well, this report quantifies the degree to which the combination of gzipping and minification beats either individual technique. We examined five minifiers:

  1. Google's closure compiler, set to use SIMPLE_OPTIMIZATIONS. Though the ADVANCED_OPTIMIZATIONS option can result in better minification, we have found that without manual effort to export symbols that should be kept, its dead-code elimination removes code that breaks dependent scripts.

  2. Douglas Crockford's JSMin minifier, which eliminates whitespace and comments from scripts.

  3. The Perl version of Dean Edward's packer 2 compressor, written by Rob Seiler. This minifier eliminates whitespace and also shrinks variable identifiers.

  4. Mihai Bazon's UglifyJS compressor, set to use safe transformations only. In addition to eliminating whitespace, UglifyJS uses other optimizations, such as shortening variable identifiers, removing some brackets, joining consecutive variable declarations, and others.

  5. Yahoo's YUI compressor, which eliminates whitespace, shrinks local symbols, and performs other safe compression.

Method

To get a good sense of how well the minifiers worked, we looked at their performance across a diversity of pages, including JavaScript libraries contained within their own file and embedded scripts contained inside an HTML page. We also looked at a range of script sizes, though of course, embedded scripts tend to be small and libraries tend to be big.

We ran a broad, moderate scale crawl seeded from DMOZ to gather over 25,000,000 HTML pages, CSS files, and JavaScript libraries. From these, we selected 179 unique pieces of non-minified JavaScript, including 131 libraries and 48 embedded scripts, ranging in size from 34 bytes to 129,738 bytes. For a more detailed description of our data set, or to download or browse the JavaScript corpus we analyzed, visit this page:

JavaScript minification dataset
Given this corpus, we produced three sets of transformed data:

In some cases, a minifier will actually increase the size of a script, in particular with small scripts if the minifier has some constant overhead, such as might be encountered to process variable name tranformations. To avoid unnecessarily skewing our data with these cases, we emulated an intelligent minifier that tests to see whether the post-minified output is smaller than the base script, and if not, it emits the untransformed script. Similarly, for the analysis of the gzipped(minified) set, we emulated an intelligent tool that emits the smallest of (a) the untransformed script, (b) the minified script, (c) the gzipped script, or (d) the gzipped + minified script.

Results

How often does minification help?

closurejsminpackeruglifyyui
number of scripts
the minifier shrunk
177178119179179
percent of scripts
the minifier shrunk
99%99%66%100%100%
As described above, our emulated intelligent minifiers only emit transformed code if the minification results in smaller scripts. The table on the right shows how often minification helped.

For most minifiers, minification was effective across virtually all scripts. However, the packer minifier only deflated 66% of scripts; the other 34% grew larger. This is because packer emits a constant overhead preamble to minified scripts, and in the case of small (roughly 100 byte or less) scripts, the benefits from minification did not outweigh the cost of this constant overhead. Click here to explore an example.

How many bytes are saved using minification and gzipped(minification)?

We analyzed how many bytes would be transmitted to send (a) the entire corpus of JavaScript, uncompressed and unminified, (b) the minified JavaScript corpus, using the intelligent minification, and (c) the gzipped(minified) JavaScript corpus, also using intelligent minification. The following table and graph shows the results.

minifiernoneclosurejsminpackeruglifyyui
bytes sent1371059
(100%)
790254
(58%)
962618
(70%)
654934
(48%)
843003
(61%)
827042
(60%)
gzipped bytes sent397296
(29%)
262493
(19%)
289078
(21%)
319216
(23%)
286231
(21%)
271021
(20%)

gzipping alone was quite effective, compressing away 71% of bytes. Individual minifiers fare worse than gzip alone, saving between 48% and 70% of bytes, with packer being the most effective, in spite of its ineffectiveness for small scripts.

Interestingly, however, the combination of gzipping and minification does best, and Google's closure compiler seems to be most amenable to gzipping. closure has a slight advantage over the other minifiers when combined with gzip, saving 81% of transmitted bytes. gzipped(closured) scripts were reduced another 34% compared to gzipped scripts alone.

Overall, then, enabling gzip-compressed content encoding on web servers will produce substantial savings, but there are still substantial improvements that can be gained with gzipped minification.

How do the different minifiers stack up?

To answer this question, we calculated the distribution of reduction factors for each minifier across the pages of the corpus. If a script is X bytes long, and post-minification it is Y bytes, then we define the reduction factor to be Y/X. So, smaller numbers mean more effective minification, and a number less than one implies that minification reduced the script size. Since we're using our intelligent minifiers, the worst-case reduction factor we will see is 1.0.

The following charts show these distributions across the minifiers, and across the gzipped(minified) combinations. Most minifiers seem similar, except that packer fares badly on small scripts while closure and perhaps yui do best overall. As previously described, in the vast majority of cases gzip improves post-minification script sizes.

Related studies

For related studies, see: