Announcing Ripper for Ruby 1.8.x

I have just released the “Ripper” library as a gem on Rubygems. It is available here and is meant for Ruby 1.8.x users only. Ruby 1.9 users already have Ripper packaged with their stdlib, so they don’t need to install this. This release is a backport of Ripper to Ruby 1.8.x and enables the library to work in pre-1.9 MRI. Ruby 1.8.x users can install it as:

$ [sudo] gem install ripper

An example usage of ripper is:

As always, the source is available on Github: http://github.com/lsegal/ripper18

Give it a shot and see if you can break it— err, like it! If you have issues, please open an issue, or just fork and pull request away 🙂

A Little Backstory

Ripper is a Ruby source parser (tokenization, AST generator) written by Minero Aoki that was introduced in 1.9.x and is now packaged in the stdlib. YARD has been using Ripper in 1.9 mode for the last 2 years now, so it has become an important part of YARD development. Unfortunately, being a 1.9.x-only lib meant we had to basically maintain two separate versions of YARD’s parser infrastructure for the last little while, in order to benefit from Ripper but also maintain 1.8.x compatibility. In addition, the 1.8.x parser was slow and much less functional than Ripper, which meant that 1.9 users were getting much more bang for their buck.

Over the holidays, when putting together a roadmap for the next release of YARD, I decided to tackle this “slowness” issue in 1.8.x, but I didn’t really know how I would do it. After a bit of investigating, I decided what we really need to do more than improve perf is unify the parser infrastructure. I figured why not hit two birds with one stone and attempt to backport Ripper. I had previously tried it before and failed miserably, but I did some more research and found a couple of clues that made it a lot more feasible.

So I spent the last week in the trenches of GCC ripping (for lack of better words) “ripper” out of 1.9.x. After a heck of a lot of debugging and brain-wracking, I finally got it to compile, and most importantly, work in a fairly consistent and stable manner. The port is basically a copy of the ripper source, the parse.y, and certain important header files from the latest Ruby 1.9.2 release. Minus a few very small adjustments, the code is nearly the same. The only “major” changes that had to be made was implementing String encoding related functionality, which were implemented as dummy methods (that call the regular 1.8.x String methods). For this reason, parsing source with non-ASCII encodings might blow things up— I haven’t tested it myself, but I warned you! Hopefully we can improve this in the future if it is indeed an issue. With such minor overall modifications, though, I think it would be easy to maintain this lib and keep it in sync with the 1.9 version.

Gem Names

I released the library as its original name, “ripper”, even though it is not exactly “ripper”. I tweeted this last week, asking for opinions on the gem name:

I decided to settle on “ripper” rather than adding a suffix like “ripper18”, because I figured it would be better for the library to “just work” when ‘require’d in code. Forcing library maintainers to require 'ripper18' if “ripper” was unavailable would probably cause more problems, and existing code would not be able to transparently take advantage of ripper. Of course, I also realize that shadowing the name of a library that is in the stdlib might cause confusion among users. Certainly if there are compatibility issues with ripper in 1.⁸⁄₁.9, or any other problems, I would rethink the library name. If you have issues with the naming, let me know.