Reno, NV    +1 (800)0 621-0871

Go back

Internship

Internship Project: Speeding up Rubygems at AELOGICA

  |  July 21, 2015

Sitting and waiting for bundler to install different gems could take a while. With an intermittent Internet connection, it could take longer and block you from making any forward progress. During my last week as an intern, I started exploring different ways to speed up gem installations and this blog post gives a rundown of my search:

Setting up a Squid HTTP proxy

The first library I started to use was squid. It is a caching web proxy supporting different protocols like HTTP, FTP and more. There are also configuration files available online tailored for caching request to rubygems.org. I also tried setting up squid in my local machine. After that, I copied configuration files and reloaded squid to apply changes.

Next I tested the installation on my local system by using codetriage. My initial approach was to create a new gemset (using RVM) and timing bundle install without a squid cache. It took 26 minutes to download and install the gems. Then I created a new gemset, but this time ran bundle install using squid. This took 23 minutes as the cache is being warmed up. Lastly, I created a new gemset, ran bundle install for the second time with squid enabled and it took about 18 minutes. I think this is good enough, considering some gems in the project’s Gemfile took a while to install:

The major problem we encountered with using squid as a cache for gems is that we can’t easily cache SSL connections. However, using the HTTP protocol, the connection to RubyGems wouldn’t be secure and it would be difficult to confirm that the gem indeed came from rubygems.org. Squid has the squid-in-the-middle SSL Bump feature which allows inspection of SSL traffic. However, this would cause problems with the SSL certificate since RubyGems would see this a possible man-in-the-middle attack. By default, this feature is not available in Ubuntu and must be added manually by compiling Squid’s source code.

Private gem server using Geminabox

The next library I used was geminabox. This project allows you to host your own gems, much like running RubyGems locally. Geminabox has a very simple configuration file and can also be setup to pull gems it doesn’t have from rubygems.org. I was able to run Geminabox with WEBrick as its HTTP server. During testing, we were able to install a gem (bundler), but had problems with the index file (I later learned how to re-generate the local gem index).

Your mileage may vary

Squid worked great as proxy server to speed up request to RubyGems. This could be effective for small to medium-sized Rails shops. Also, you’ll have to decide if not having HTTPS for squid is worth the risk. You could use Geminabox as a simple gem cache, but you’ll need to seed the cache in bulk and also find a way to re-generate the index every time a new gem is added.