April | 2015 | JoeKiller

I haven’t made much effort to blog recently so I figured I would do a catchup article on some of the projects I’ve worked on recently.

gor is a http traffic replay tool written in go. When I first found Gor, it was a great tool but there were several bugs that made it unstable or unreliable. When I had a little time, I submitted several enhancements.

https://github.com/buger/gor/pull/116 added dynamic http worker scaling which was key for throughput. I also provided documentation updates to help others discover and identify bottlenecks in their testing systems.
https://github.com/buger/gor/pull/94 sorted the TCP packets properly which was pretty crucial for replaying connections that had out of sequence packet arrival. Alejandro Martinez of Vivid Cortex helped me in spotting the problem.

moto is a mock AWS endpoint written in Python. We use AWS a ton at Room Key and needed to better test our internal Python tools. Moto had a good feature set but didn’t quite implement the more advanced things we were using so I added to the library over 11 different PRs. Here are a few of the biggest features I added:

https://github.com/spulec/moto/pull/235 implements some of the more advanced features of CloudFormation template processing such as the Fn::GetAtt command which can get attributes of any other resource created in the template.
https://github.com/spulec/moto/pull/224 implemented Amazon’s tagging API into moto. … all of it 🙂

logstash-kafka and jruby-kafka are my Ruby projects that I created to make the Kafka plugin for Logstash. The following link is where the Elasticsearch blog announces support for Kafka in Logstash and they credit my project as being the origin of the support, https://www.elastic.co/blog/logstash-1-5-0-beta1-released. Because 1.5 isn’t out yet, I still maintain support for my independent plugin and provide input and support otherwise for the official plugins logstash-output-kafka and logstash-input-kafka.

Sometimes other projects just aren’t getting enough attention for their PRs from their primary owner and I occasionally will volunteer to be a maintainer (as I want my PR’s merged). They are generally low volume projects that don’t need much love beyond a merge here and there combined with a release. These include:

s3_file, a Chef resource for downloading files from S3 for Chef recipes
grunt-aws-sdk, a wrapper for AWS’s Javascript SDK for Grunt tasks.

I’ve touched quite a few different projects over the last year or so. In many cases I consider these projects to be an act of standing on the shoulder of giants and adding a little extra to each. That being said, even as I contribute to these external projects, most of my work happens internally at Room Key. You can see the difference between my public activity graph versus my private graph: