Client-side zipping on the fly

11 min. read

Caution! This article is 6 years old. It may be obsolete or show old techniques. It may also still be relevant, and you may find it useful! So it has been marked as deprecated, just in case.

Recently I needed to find a solution to Zip some files on the fly (client-side) when the user clicked a download link on a static site.

Context

The site used Jekyll, was served through GitHub pages, and contained mostly tutorials. Some of those tutorials had example files to download. Since it was an open-source project with many contributors, these example files where spread all over GitHub in gists in the contributors personal accounts. This made it very difficult to update those example files when the tutorial was updated, as you can't submit a PR to a gist file.

Problem

  • The exercise files of the tutorials were in gists where they could not be updated or new files added.
  • We wanted to bring these files into the tutorials repository to version-control them together with the tutorial they belong to.
  • We also wanted the students to be able to keep downloading the files in a zip file from the tutorial page as they did before.

Constrains

  • Jekyll plugins are not allowed in GitHub pages.

    GitHub Pages cannot build sites using unsupported plugins. If you want to use unsupported plugins, generate your site locally and then push your site's static files to GitHub.

  • Version control of a zip file (or any binary file) is considered bad practice, as they make the repository bloated and humans can't make sense of the changes anyway.
  • Client-side JavaScript does not have read access to files in the server. So the zip has to be generated on the fly.
  • We can't use HTML attributes in tags. So we can only use the href attribute of the download link [*].
  • We prefer a simple solution, to be able to have a single commit that updates the instructions and example code together.
  • No fancy workflows, so we can push changes and have them served immediately through GitHub pages.

Possible solution

The solution we agreed was to move those gists to the main repo, alongside their tutorial, and zip them on the fly when the download link was clicked. After doing some research, I found JSZip.

So we used JSZip and some liquid to automatically generate a zip file containing the tutorial downloadable files, when clicking the download link.

The workflow looks like this:

  • You add a folder with your exercise files inside of the tutorial folder:
    
    javascript_tutorial/lesson3/
    ├── files/
    │   ├── index.html
    │   ├── jquery.js
    │   ├── script.js
    │   └── style.css
    └── tutorial.md
    
  • Then, add an array variable files to the frontmatter of the tutorial page, containing a list with the paths of the files you added, including folder name:
    
    ---
    layout: page
    title: 'Lesson 3: Introduction to jQuery'
    files:
      - files/index.html
      - files/jquery.js
      - files/script.js
      - files/style.css
    ---
    
  • In the markdown of the tutorial page, add a download link and make it point to download ("download" is just to differenciate download links from other links):
    
    Download the files that you will need to work through the example [here](download).

How it works

As I mentioned, we decided to use JSZip to zip the files on the fly. The way the JSZip library works is that you have to pass it the paths to the files you want to zip. For example, this code creates a text file called hello.txt with contents Hello World\n:


var zip = new JSZip();
zip.file('hello.txt', 'Hello World\n');

If we store the tutorial files alongside the markdown tutorial file, we can tell JavaScript the path and names of the files it has to zip while Jekyll is building the site.

In order to build the site, Jekyll parses the markdown files into HTML pages using the configuration in the frontmatter of said markdown files.

The templating engine (in this case Liquid), reads the files variable with the file paths in the frontmatter. Then it dumps them as a JavaScript array in a <script> tag at the bottom of the tutorial HTML page, so that the JSZip library can access them.


{% if page.files %}
{% assign pageurl = page.url | split: '/' | pop | join: '/' %}
{% capture files %}[{% for file in page.files %}
"{{ pageurl }}/{{ file }}"{% if forloop.last %}{% else %},{% endif %}{% endfor %}
]{% endcapture %}
{% endif %}

<script>
var Files = {
  all : {{ files }}
}
</script>

In JavaScript we add an event listener to links having an href of download that tells JSZip to generate the zip on the fly when the link is clicked. Files are downloaded asynchronously from the repo.

This is how we generate the zip on the fly client-side, after Jekyll generates the page:


var addFiles = function() {
  var filename;
  Files.all.map(function(fileurl) {
    filename = fileurl.replace(/.*\//g, '');
    Zipper.zip.file(filename, downloader.getFile(fileurl), { binary: true });
  });
};

var generateZip = function() {
  if (JSZip.support.blob) {
    addFiles();
    generateAsync();
  } else {
    console.log('Blob is not supported')
  }
}

var registerListener = function(downloadLink) {
  downloadLink.addEventListener('click', function(event) {
    event.preventDefault();
    createZip();
  }, false);
};

Then we can query all the download links:

var downloadLink = document.body.querySelector('a[href=download]');

The files are bundled to reduce the number of HTTP requests. Sadly, I couldn't find a CDN URL for the library, so at the moment it is bundled with the rest of the JavaScript [**].

Caveats

At the moment this solution allows only one download link per page. This means that tutorials with more than one set of downloadables should be split into two. This can be fixed easily though, but I wanted to go one step at a time.

Also, JSZip relies on blobs to work. The only problematic browsers that didn't have support for blobs seemed to be IE9 and Opera Mini.

Blob support
Blob support according to "Can I use..."

However, JSZip provides a IE version that can be added like this:


  <!--[if IE]>
  <script type="text/javascript" src="http://stuk.github.io/jszip-utils/dist/jszip-utils-ie.js"></script>
  <![endif]-->

Challenges

I had some challenges testing promises and asynchronous methods. The manual testing worked though.

Room for improvement

There is a lot of room for improvement. For example:

  • Rather than clicking a link, I think semantically speaking a <button> may be a better option 🤔
  • I couldn't test it widely in every browser and OS, just Ubuntu, Firefox and chrome. Other community members where able to make a couple more tests though.
  • At the moment it only does one download per page.
  • The tests of promises and asynchronous methods could be improved.
  • We need to port the rest of the tutorials.

[*] Jekyll parses Markdwon using Kramdown. According to the Kramdown docs, you can indeed add attributes to an element. For example, to add an id to a link: [a link with an id](http://foo.com) {: #link-id}

[**] I found it!!! there is a CDN link at Cloudflare!

Comments