HTTP Libraries – Your Best Friends

For a programmer like me, the web is a playground like no other. Not just the information that can be retrieved, but the sheer amount of services and APIs available online make getting structured, useful data a matter of reading a few documentation pages. From social data like Facebook and Twitter to data services like Xbox Music, the web is a repository of rich information that can be manipulated as per your whim.

So while the data is omnipresent on the web, the bigger task is pulling that data from the stores they reside in. This is where the HTTP protocol comes in. With its basic operations like GET, POST, PUT and DELETE, you have a handy mechanism to retrieve, push or manipulate the data on the servers. Now, the HTTP protocol is not as simple a protocol as this blog post makes it seem. Things like HTTP Headers, application types, authentication tokens and other similar things make the task of HTTP based retrieval slightly more complex than it must have been intended to be.

This is where HTTP libraries come in. They handle the most of the heavy lifting and let you focus on the kind of content you are dealing with on a per API basis while abstracting away the inner complexities of packet and protocol rules. As someone who has been doing a lot of development in C# and Python, the HttpClient library in the .NET framework and Python’s Requests libraries have helped me unleash the raw power of the web countless times. To give you a glimpse of this power, you can take a look at how the HttpClient library handles various forms of HTTP content in this Stackoverflow answer.

The simplicity of Python’s requests library makes rapid prototyping with new web APIs a cinch. Simple methods like requests.get() and requests.post() (which do exactly as the names suggest) let you examine in real time the data coming in. This knowledge can then be leveraged to build more robust languages either in Python or in enterprise level languages like C#. The HttpClient library brings amazing power in terms of its ability to not only do various kinds of HTTP requests, such as upload images programmatically, but also manipulate headers and metadata effectively, thus helping you quickly get off the ground with your application. Add to that C#’s asynchronous operation support and you’ll be developing .NET software full-time.

Now while I learn Ruby, I have already exposed myself to the Net::Http library of Ruby to take advantage of this newly discovered ability. I personally will definitely make it a point to master the HTTP libraries of whatever language I use along with the defining features of the language. Accordingly, especially if you are a new programmer, I urge you to do the same and reward yourself with this power and flexibility to leverage existing information to build upon, create cool things and make the world a whole better place!

Eviva!

The Sound Of Commits

Well, all I can say is that I’ve been busy. How busy you ask? Let this post answer it for you.

I had come across the Github API a while back and ever since I have been contemplating ways to use the data provided by the API in a meaningful way. After weeks of brain-wracking, I finally hit upon the idea that why not render the commits for a year long period as musical notes? If you have a Github account, you might have noticed in your “Repositories” tab there is a wonderful little visualizer that has blocks of varying green colorations, showing your Github activity over the past 1 year. Why not audiolize it?

So first things first, I head over to Github’s API docs. Now the beauty of Github’s API is that it to read user and public repository a.k.a. repo information, I don’t need to sign in. That enabled me to write the whole thing in client side Javascript and not worry about Cross Origin Resource Sharing (CORS) since the audiolization would be hosted on my website. This made my task much, much easier (and more hassle-free for the user). Some quick jQuery setup and custom wrappers to the AJAX calls had me pulling the data in a jiffy.

The next step was preparing the data, i.e. getting the commits from the last one year for a user across repos and grouping them by date. Github allowed me to limit the data returned to the past 1 year, but grouping it was still an issue. That is where moment.js comes in. This intuitive and feature-rich library gave me all that I needed to manipulate and compare the date-time signatures on each commit as well as taught me the standard for representing dates and times. Couple this with a few articles on Javascript promises and closures (Lisp throwback!!) and I have my data ready to be audiolized.

A small digression: For all my difficulty with JS and its weak-typing model, the fact that Lisp has inspired JS and that JS allows me to come from a functional programming school of thought, made my code really clean and easy to work with. I was strictly against creating a global variable to store and work on the final data in the JS file since side effects from functions could mess up the data and be really hard to track down and fix. Functional programming, which by design avoids side affects on your data, really helped me out and JS’ closures really sold me to the language.

Now for the music. Musical Instrument Digital Interface or MIDI is the de facto standard for rendering music and this is no different on the web. Here is where I found this amazing project called MIDI.js on, you guessed it, Github! I forked the repo so I had my own version to play around with, and after some digging around since the documentation was substandard, I had attached MIDI musical notes to my commit data.

After some quick HTML and CSS where I set up the web page, I finally published it to the web. Sharing with people via Facebook and Twitter helped get some good feedback on it. You can find it here. Be sure to try it out, especially if you have a Github account. Seeing all my Github activity rendered as music was a joy like no other and I was glad to be able to help others see that joy as well.

Let me play you a song of your commits. If you have any suggestions, do feel free to comment.