People who are really serious about software should at least write their own libraries

We have been using a lot of tools and libraries in our software, and have replaced a number of them with our code. Libraries makes it easy to get things done, and to ship early. But from my experience, having a third-party library or a tool dominate a core part of your software is not a good idea.

People who are really serious about software should make their own hardware. - Alan Kay

We have moved away from a number of libraries (and frameworks and platforms) over the last couple of years. This may sound like a lot of hate, but it is not so. We still love those libraries and use them on a lot of smaller projects. But when your product grows and you want to mold it the way you want, sometimes libraries stand in your way. Some of the decisions we made could be wrong because we didn’t understand the library properly. But we spent a lot of time trying to stick to those libraries before replacing them.

Writing your code instead of using libraries takes a lot of weight off the product. Most of these libraries are written by really good programmers to be used in a wide range of scenarios, and there is a pretty good chance that you won’t need them all. So getting inspiration from them and writing your own stuff will make the software lighter while making things work the way you want. A lot of programmers are likely to write their own code to replace libraries at some point, and that is probably why there are a number of libraries doing almost exact same thing in slightly different ways.

One of the traps you can fall into when writing our own code is that you might just end up reinventing the wheel. There is a chance that you replace a well written library with a small piece of code initially, but with time end up improving your code to do exactly the same thing the library you threw away did.

Libraries and platforms we’ve replaced with our code #

jqPlot #

Our first release of Forestpin used jqPlot for most of our charts. Some not-so-ordinary visualizations were made with protovis, a predecessor of D3.js. jqPlot helped us quickly develop a product (more like a prototype) to show potential customers, but introduced a lot of constraints. We made some changes to jqPlot to customize some of the charts, but it wasn’t enough. The next version of Forestpin used D3.js for all the visualizations[1], which gave us more control, and it was quite close to writing your own code. jqPlot is the only library we have not used again in any project.

Backbone.js #

Backbone was used at Forestpin as well as at nearby.lk. What triggered us to code a replacement for Backbone was that it didn’t save states in HTML5 history[2]. We weren’t using most of the features of Backbone too, so the replacement, Sweet.js, was much simpler. We plan on making Sweet.js independent from jQuery and Underscore.js, and also renaming it so that it doesn’t get confused with Mozilla’s Sweet.js

Database #

Forestpin Enterprise used a custom in-memory data store built ground up at Forestpin from the early days. The data store was a core part of the Forestpin product, and we gained a lot of performance by doing most of the calculations within the database itself.

jQuery mobile #

nearby.lk decided to drop jQuery mobile after an year with it. Adopting jQuery mobile was a bad decision. It was very heavy[3] and was not developed to be used for apps like nearby.lk. jQuery mobile is super easy to be implemented for a web site with server generated HTML pages, but ours had a lot of dynamically generated pages and we spent quite sometime getting jQuery mobile to work.

Google App Engine #

nearby.lk was hosted on Google App engine for an year and a half and we moved to Amazon [appengine]. There were some interesting discussion about it on Hacker News. [hackernews].

D3.js #

At Forestpin, we use d3 for all our visualizations and some tables - whenever data is connected to the DOM. We also introduced CoffeeScript helpers to simplify D3.js DOM manipulation code.

We came across some requirements which were hard to tackle with D3.js. One was adding DOM elements progressively. For example, when you are drawing a matrix with a large number of small rectangles. if you draw all if it at once, the user will see nothing for a while and everything will appear at once. But it would have been more user friendly, if elements were added progressively. Then the user will see sets of rectangles appearing in short intervals, as if it was an intended animation. The total time for all rectangles to appear might be slightly longer, but the user will feel otherwise. We couldn’t find a neat way to do this with D3.js.

Another problem was that we couldn’t move a particular DOM element across parent elements easily. For instance, if you want to show controls like the action links beneath the focused tweet in the twitter timeline, you should either have hidden action links on each tweet. or redraw action links when the focus changes, or have the action links on a different layer and move it. The first two options are not efficient and the third is tricky. The easiest is to have a DOM element removed and inserted to the focused tweet when the focus changes. This option, although probably not as efficient as the third option, is simpler and faster than the first two[4].

There were a few similar issues so we thought of going for native Javascript code for DOM manipulation, and wrote a small library with an interface similar to CoffeeScript helpers for D3.js. We will continue to use D3.js for scales, csv parsing, etc.

jQuery #

jQuery is used for selectors, events and Ajax. The dependency on jQuery is becoming less and less. Using pure Javascript is not that complicated and it is much faster[5][6].

Although we’ve been moving away from a lot of libraries and tools, there is still a number of libraries we use. We use them because they make the development process easier, but only as long as they don’t constraint us from building what we want to build. In other word, don’t let the tools constraint the design of the product.

 
32
Kudos
 
32
Kudos

Now read this

Weya.coffee

Weya.coffee is a lightweight library with no dependencies to generate DOM elements. We developed it to replace Coffeecup as a client side template engine. Because of its simplicity and performance, we are also using Weya to replace DOM... Continue →