lloyd.io is the personal website of Lloyd Hilaiel, a software engineer who works for Team Ozlo and lives in Denver.

All the stuff you'll find here is available under a CC BY-SA 3.0 license (use it and change it, just don't lie about who wrote it). Icons on this site are commercially available from steedicons.com. Fonts used are available in Google's Web Font directory, and I'm using Ubuntu and Lekton. Finally, Jekyll is used for site rendering.

Finally, Atul, Pascal, and Stephen inspired the site's design. And in case you're interested, this site's code is available on github.

Parse JSON in ruby in 1/4 the time of YAML
2009-09-03 00:00:00 -0700

Crazy claim, eh? I figure there's no better way to get this claim tested that by posting it as a truth!

We'll here's the story: Once upon a time, lloyd (that’s me) wanted a parser that didn’t suck that he could embed in a particular product, primarily because he was sick of patching other peoples code: solving problems like:

  1. crashes on certain malformed inputs
  2. the niggling tendency to scan from the begging of the input text every time a number was encountered
  3. the requirement that you had to hold the full json text in memory at one time (usally in addition to an in-memory representation).

To be fair, the project that I was previously using got a lot better, but it still lacked a couple key features that were extremely important to me: stream parsing, and representation independence. At the time (and actually still), we were using JSON as the data representation over our IPC channel (in that same particular product), and this meant that a lot of data was flowing. We had one of two options:

  1. throw away json and move to a binary format
  2. make a json parser that caused the IPC serialization overhead to fall off our our profiles…

Cause' I love transparency over the wire (binary still ain't that scrutable), and because JSON is the representation language of the web (hush now XML people) I decided to try to build this parser.

Here's the thing, I'm rather stupid and short sighted, and didn't realize that it's actually extremely well suited to high level language bindings for other languages, say, ruby or python. Luckily, Brian Lopez is not stupid, and he has brought yajl to ruby.

Initially Brian is reporting that it's about 4 times as fast as YAML parsing via syck, about 20% faster than the other native JSON parser for ruby](http://flori.github.com/json/) out there, and in general uses up to ½ the memory (now this claim especially is difficult given the exponential nature of the heap mechanism in ruby, so perhaps we need to retry this with some of them gc patches I wrote, or with 1.9).

I'm posting to re-iterate how awesome Brian and I are? No, not really. I'm posting to challenge skeptics to validate these findings, just cause' I was extremely pleased and surprised to hear these numbers from brian, and a little bit of that rigor that comes from public scrutiny couldn't hurt…

till the next, lloyd