I started working on Sweet.js about a month ago. It is inspired by Backbone.js. Sweet.js supports HTML5 states, so that you don’t have to go through work arounds like these. Sweet.js is not a MVC framework, but it has a views similar to Backbone.js, which supports inheritance without affecting events and initializations of super classes. And it’s written in Coffeescript.

Router

class Router extends Sweet.Router
 @routes You can add more routes from sub classes
  '': 'home'
  'search/:what': 'results'

 home: ->
  View.home()

 results: (what) ->
  console.log @getState() state
  View.results(what)

router = new Router()
router.start pushState: true

Emulate browser back button
router.back()

Whether we can go back without leaving the web app
router.canBack()

Views

A generic form class.

class Form extends Sweet.View
 @events
  'click .cancel': 'cancel'
  'click .submit': 'submit'

nearby.lk moved the servers from Google App engine to Amazon EC2 a couple of months back, and the backend is now built with nodejs with mongodb as the database.

Backend does a lot of pre-computations and caching which gives slower start up times and faster response times. The migration decision was based on a number of factors such as ease of development, performance and cost. There were a bunch of disadvantages of using the app engine, a several benefits of using NodeJS and some things we miss after the migration.

App Engine: the bad

Data store operations are slow

There were not many data store writes. Almost none on most days since data was uploaded in bulk. Still, the data store reads were significantly slow. To improve performance we kept most of the data in memory, which worked great until the web traffic and database size started growing.

Too many instances

App engine does...

Anyone who programs in C uses pre-processor macros to keep the code clean, maintainable and sometimes to improve run-time performance. You can use the same C pre-processor to include macros in Coffeescript or Javascript code, and it is easy!

The coffeescript file needs to be renamed with .c extension before running it thrrough gcc; otherwise, it gives a warning and doesn’t run the pre-processor. There may be some option to force it compile without checking the extension.

cp [source.coffe] [source.c]
gcc -E [source.c] > [processed.coffee]

Examples

Improve performance

This is the code with macros. It compares the time taken to calculate the sum of squares, using functions and macros.

square = (x) -> x * x

define SQUARE(x) (x * x)

total = 0
console.time "no macro"
for i in [0...100000]
 total += square i
console.timeEnd "no macro"
console.log "Total {total}"

total = 0

We are releasing Forestpin Lite, with a lot of improvements to our previous Forestpin Lite version released at the 24th Fraud Conference in June. The new version is packaged as a Google Chrome offline application and therefore runs on Windows, Mac and Linux platforms. Chethiya Abeysinghe was behind Forestpin Lite.

[Forestpin Lite](//vpj.github.io/images/posts/forestpin-lite/lite.png)

Forestpin Lite was developed since we felt that most auditors are using standard spreadsheet software and analyzing data manually, which is a waste of their precious time. One of the reasons to rely on spreadsheets is that auditors have access to them easily. Most risk management software are designed for large corporations and not for individual auditors/accountants; therefore those are expensive and purchase is a corporate level decision. Spreadsheets is a great tool but it is not developed with focus on...

We released the new version of Forestpin Enterprise last week. The new version is a complete rewrite of both the backend and UI. The backend was rewritten to be faster and to introduce a bunch of new features and analytics. The user interface was redesigned to be much more user friendly and also with focus on mobile devices such as tablets.

Teachings of visualizations experts such as Edward Tufte, Stephan Few and Naomi Robbins were followed during the user interface redesign. We have also changed the way we presented the results of analyses that existed in previous version as well. The images below are screenshots of the new version vs. the previous version and what made us make the changes.

Example 1 - Benford’s Law Analysis

[Benford’s Law Analysis - Old](//vpj.github.io/images/posts/forestpin-enterprise-redesign/2013-benford.png)
In the previous version you could only see the...

[Benford’s Law Analysis with Sparklines](//vpj.github.io/images/posts/benford-sparkline.png)
The image shows first-two-digit distribution compared with the expected values based on Benford’s Law. What is new here are the sparklines on the right most column which shows the timeline of number of transactions for each first-two-digit combination. For first-two-digits 46, 47, 48, and 49 you don’t see a large variation from Benford’s law, but you see spikes in the sparklines that moved from 49 to 48 to 47 to 46 with time - something to be investigated!.

Sparklines are “data-intense, design-simple, word-sized graphics” according to its inventor Edward Tufte. Before we had sparklines, we used to filter by first-two-digit combinations that deviated significantly from expected value to figure out if there’s something actually wrong. Intuitively, the first thing we examined in the filtered data...

jQuery Mobile lets you create slick mobile web applications, and that’s what we did with nearby.lk. However, when we wanted to add more content and customize it heavily, jQuery Mobile stood in our way. So we decided to move away from it.

The migration wasn’t hard. We replaced jQuery Mobile with responsive css and a mobile first design. Since we don’t have to stick to jQuery Mobile design patterns we have a lot of customizability. This lead to a cleaner design - the design that is visible and the design underneath.

We are still testing the new UI, and planning to release on nearby.lk in a few weeks time.

We first released a calendar view with Forestpin Lite early this year, where we visualized daily transaction volume. This was a simple analysis - simple to develop as well as understand.

Calendar view was something we neglected first, but it’s usefulness was realized as we saw different financial data sets visualized in calendar view. When you see a lot of information, covering the entire data set in a single view, you start seeing patterns that you could not see previously.

[Calendar view without annotations](//vpj.github.io/images/posts/calendar.png)
The image above shows two years of transactions visualized in a calendar view, and the image below shows the same with annotations.
[Calendar view with annotations](//vpj.github.io/images/posts/calendar-annot.png)

A few things you can notice,

1. This inactive period is festival season, most companies don’t work for about a week.
2. An...

[Mantissa](//vpj.github.io/images/posts/ftd-mantissa1.png)
At Forestpin we’ve been analysing financial data from various companies, and Benford’s Law analysis was one of the techniques used by our software to find irregularities and fraud in financial data.

If you are new to Benford’s Law, this article gives a nice introduction. And the book Benford’s Law: Applications for Forensic Accounting, Auditing, and Fraud Detection by Dr. Mark Nigrini is a great resource to learn about Benford’s Law and it’s applications in financial data analysis.

However, one of the things that is not much discussed and used in applying Benford’s law is the sum invariance. This characteristic was first observed by Nigrini in 1992.

[First Two Digit Distribution](//vpj.github.io/images/posts/ftd-distribution.png)
The basic use of Benford’s law is to take the first digit distribution or first two digit...

Using this technique you can pass file descriptors between processes using sendmsg() and recvmsg() functions using UNIX Domain Protocol. Any descriptor can be passed using this method not just a file descriptor.

This is quite useful when you want to balance load in a multi-core system. Other way of passing file descriptors is by forking the process, but in this case you can pass file descriptors between different processes at anytime.

Creating the UNIX Domain Protocol server

SOCKET_PATH was set to /tmp/unix_socket

int create_server() {
 struct sockaddr_un addr;
 int fd;

 if ((fd = socket(AF_LOCAL, SOCK_STREAM, 0)) < 0) {
  log_error("Failed to create server socket");
  return fd;
 }

 memset(&addr, 0, sizeof(addr));

 addr.sun_family = AF_LOCAL;
 unlink(SOCKET_PATH);
 strcpy(addr.sun_path, SOCKET_PATH);

 if (bind(fd, (struct sockaddr *) &(addr),