Speeding up Jekyll's LSI

Jekyll’s Latent Semantic Indexing (LSI) indexes your posts to create the “Related Posts” section you see at the bottom of the individual post pages. However, if you have more than a handful of posts, it is very slow, and it seems to get exponentially slower as the number of posts increase. I ran it on this site, with ~140 posts, and I finally killed it after 3 hours.

There’s a fix for this, though, buried in the README for classifier-reborn, the library used to do the post classification. Install GNU GSL and rb-gsl. On OS X with homebrew, it’s as easy as:

brew install gsl
gem install rb-gsl

Indexing time for me went from, well, 3-∞ hours to less than 5 seconds.