When I first started hearing about static sites generators a few years ago, they seemed like a decent alternative to traditional database-backed CMS like WordPress for blogs or other simple sites, but not much more.

But over time, the more I’ve been using them, the more I’ve realized that “static” doesn’t have to mean “dumb”.

I previously talked about how we’re using Middleman on this site. Today, I’d like to talk about how we’re using it for our community translation project, and more specifically about:

  • Our internationalization setup.
  • How we’re calculating chapter-by-chapter translation progress.
  • How we created individual APIs for each translation instance.

(Note: I like to use Middleman, but this article applies equally to most static site generators.)

This is the third part in a three-part series on static sites (I know, pretty much the opposite of Meteor!). Previously:

Easy Internationalization

The first technique I’d like to talk about is internationalization (translating a site’s content in multiple languages).

My post about our translation workflow already explained how we’re using a single Middleman app in conjunction with GitHub, Codeship, and Heroku to create separate Heroku instances for each version of the book.

Having a single codebase was a huge time-saver, but it also meant that codebase (and especially the user interface) needed to be able adapt to each language.

One of Middleman’s great strength is its local data feature: basically, instead of storing data in a database, you just store it as YAML data in a plain text file.

We use this feature to store all our UI strings in a single strings.yml YAML file:

# English
- lang: 'en'
  repo: 'https://github.com/DiscoverMeteor/DiscoverMeteor_En'
  book_title: 'Discover Meteor'
  # ...

# Spanish
- lang: 'es'
  repo: 'https://github.com/DiscoverMeteor/DiscoverMeteor_Es'
  book_title: 'Descubriendo Meteor'
  # ...

# French
- lang: 'fr'
  repo: 'https://github.com/DiscoverMeteor/DiscoverMeteor_fr'
  book_title: 'Découvrir Meteor'
  # ...

  # etc.

By placing this file in Middleman’s /data directory, we then have access to its contents in the data.strings Ruby hash.

All that’s left is extracting the strings for the current language. My first implementation was quite simple:

@strings = data.strings.find{|s| s['lang'] == LANG}

I was now able to use this new @strings variable throughout the site:

  %h2= @strings.toc

But this triggers an error every time someone adds a new language and forgets to translate a string.

I wanted the strings to fall back to English in that event, so I used Ruby’s merge (thanks Christian!) to merge the current language’s strings into the English strings (giving priority to the current language’s strings), like so:

@strings = data.strings.find{|s| s['lang'] == 'en'}.merge(data.strings.find{|s| s['lang'] == LANG})

Now of course, our internationalization setup is quite basic: it doesn’t deal with plurals, gender, and all the other fancy stuff that “real” i18n software takes care of. But it’s quite sufficient for our needs, and fits in a single line of code!

Making Progress

I knew that once a language started having more than two or three contributors, figuring out what needed to be done would start to be a problem. We needed a way to display the translation progress of each chapter, as well as the progress of the overall translation.

The Chinese version’s chapter-by-chapter and overall progress.

We needed a representative metric to calculate progress, and I picked the number of translated paragraphs per chapter.

I took the original English version of the book and created an outline version with every single text paragraph replaced by ////. Each new translated version was then created as a fork of that outline repository.

Getting the progress of a chapter was now as simple as taking the total number of paragraphs in a chapter, and subtracting the number of occurrences of the string //// in that chapter’s text.

In other words: progress_percentage = (total_paragraphs - x * 100) / total_paragraphs, with x being how many times //// appeared in a chapter.

But how would I actually figure out x? I could’ve used JavaScript, but it made no sense to recalculate the same values over and over on each request. No, this needed to be done on the server.

Not So Static After All

A classic Ruby or PHP app can be said to be dynamic because it can react to various parameters on each request (for example, variables passed through the URL).

True, static HTML files can’t do that. But your static site generator can still take into account parameters during the build process. In other words, static sites are only static after they have been generated.

We are using Rack to serve our Middleman site on Heroku, so this means we have a window before Rack starts the Middleman build process to execute some code.

So we first use a simple bash script to loop through the book’s chapter files and grep the number of occurrences of the string ////, storing the result in a Ruby hash using the file name as a key.

to_go = {}
Dir.glob('source/chapters/'+LANG+'/*.md.erb').each do |file|
  chapter = file.match('/([^/-]*)-.*$')[1]
  to_go[chapter] = `grep -o '////' #{file} | wc -l`.to_i
end

We then convert the hash into a string:

to_go_str = to_go.map {|k,v| "#{k}:#{v}"}.join(',')

And pass that string to the middleman build command as an environment variable:

puts `TOGO='#{to_go_str}' bundle exec middleman build`

All that remains is to access the value of the progress hash from within Middleman’s config.rb config file:

@togo= ENV['TOGO']

And access the right element of the hash when looping through our chapters:

paragraphs_to_go = @togo.select{|key, value| key == chapter_code }[chapter_code].to_i()

This process might seem convoluted, but the main point is that it’s entirely possible to use dynamic data in a static site, as long as you do it before the HTML files are generated.

Static APIs

We could now show the progress of each translation on each individual translation site, but that wasn’t convenient enough: we needed a single dashboard compiling the progress of every translation in a single place.

The translations dashboard page.

The problem of course is that while we did have that progress data, it was currently spread out on 14 different sites. We needed to give each translation needed its own API in order to communicate back to the central dashboard.

Thankfully, creating an API with Middleman couldn’t be simpler thanks to the layouts feature. In addition to the default layout containing your <head>, <body>, and all that, Middleman also lets you define page-specific layouts in its config.rb configuration file:

page "api.html", :layout => :api_layout

And here’s what the api_layout layout looks like:

=yield

The net result of this bare-bones layout is that we’re only outputting the API’s content and nothing else. Here’s what the api.html.erb file looks like:


<%
total_done = 0
total_todo = 0
%>

{
  "chapters": [
      <% 
      # Loop through blog articles in reverse (chronological) order
      blog.articles.reverse.each_with_index do |post, index|

        # Some ugly code to figure out progress metrics
        chapter_code = post.data.number.to_s()
        paragraphs_to_go = @togo.select{|key, value| key == chapter_code }[chapter_code].to_i()
        paragraphs_done = post.data.paragraphs - paragraphs_to_go
        percentage = paragraphs_done*100/post.data.paragraphs
        total_done += paragraphs_done
        total_todo += post.data.paragraphs
        %>

        <% 
        # Only add coma after the first item of the array
        if index > 0 %>,<% end %>

        {
          "title": "<%=post.title%>",
          "paragraphs": <%=post.data.paragraphs%>,
          "paragraphs_done": <%=paragraphs_done%>,
          "percentage": <%=percentage%>,
          "number": "<%=post.data.number%>",
          "slug": "<%=post.data.slug%>"
        }
      <% end %>
  ],
  "total_paragraphs": <%=total_todo%>,
  "total_paragraphs_done": <%=total_done%>,
  "total_percentage": <%=total_done*100/total_todo%>
}

There’s a bunch of ugly code in there to calculate the translation progress of each chapter, but it works. For example, here’s the output of the Chinese version’s API:

{
  "chapters": [
    {
      "title": "简介",
      "paragraphs": 35,
      "paragraphs_done": 35,
      "percentage": 100,
      "number": "1",
      "slug": "introduction"
    },
    {
      "title": "开始",
      "paragraphs": 49,
      "paragraphs_done": 49,
      "percentage": 100,
      "number": "2",
      "slug": "getting-started"
    },
    //...
    {
      "title": "Meteor Vocabulary",
      "paragraphs": 24,
      "paragraphs_done": 0,
      "percentage": 0,
      "number": "14.5",
      "slug": "meteor-vocabulary"
    }
  ],
  "total_paragraphs": 1007,
  "total_paragraphs_done": 525,
  "total_percentage": 52
}

We aren’t quite done yet though. We need to address a common problem: when I tried testing out my new API locally, I ran into the dreaded CORS error:

XMLHttpRequest cannot load http://zh.discovermeteor.com/. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://0.0.0.0:9000' is therefore not allowed access. 

By default, AJAX requests from one domain to another are blocked unless the browser detects the correct header on the receiving end. Thankfully, it turns out that Rack (which we’re using to serve our Middleman site on Heroku) already has a module to take care of this problem.

In your Gemfile, add:

gem "rack-cors"

Then, in your config.ru Rack config file:

require 'rack/cors'

# allow all origins
use Rack::Cors do
  allow do
    origins '*'
    resource '*', 
        :headers => :any, 
        :methods => [:get, :post, :options]
  end
end

This will allow GET and POST requests to any page of the site, including /api.

Doing Even More

If you want to take things even further, Romain Dardour from Hull.io has a gist that shows how to basically run a regular Ruby app on top of your Middleman site.

Whether you use his technique, the tips I outlined here, or something else altogether, I just hope this article has helped show that static sites don’t need to be that static after all!