How we increased chat performance by 10x without writing any code.

There is a huge problem with the internet in 2014. Thanks to the advancements in technology, primarily by Facebook, everyone expects the web to work in real time. If your page loads in under 300ms, that's average. Everything past that is already perceived as slow. In my view, this is an incredible achievement, but it can also be quite challenging. For us, the challenge was in substantially increasing the speed of our chat system. We were able to solve this problem in a couple of hours and without major changes to code. Turns out, perception of speed was all that we needed. Here's how.

The problem

[This paragraph was updated in 2015] If you are not familiar with Scoutzie, we were the best way to hire premium iOS designers that were available to work right away. Not only did we connect designers and clients, but we also provided both parties with tools that guided them through the steps of a creative process. (Unfortunately, we shut that business down after all). But, the technical aspects of what we had built are pretty relevant to date.

Back to the story.

All of the parts within our system rely heavily on real-time messaging and the longer it takes to close a deal, the higher the chance that it won't go through. At the end of the day, time lost on messages translates directly to lost revenue.

In the past, as illustrated by the diagram below, when both a designer and a client were on Scoutzie, we subscribed them to the same Pusher channel. When one user created a message, that message was saved and pushed back to the channel, displaying simultaneously for both users. This worked really nicely and rarely failed, but there was always a delay between the time a message was posted and when it appeared on the screen.

A one second delay does not sound like a lot, but when that delay happens every time you send a message, that accounts to minutes of combined delay for each conversation. Not only is that frustrating, but once again, delays lead to lost sales, and that's just bad for business.

The solution

We could have started by trying to redesign how our messages are being delivered. Perhaps we don't need the Rails-Sidekiq-Pusher combo. Maybe we could send messages directly to pusher first, and then save them? Perhaps we could use a faster data store to keep the messages...etc; Maybe we could have switched to Firebase to handle all the messaging. It's all possible. There are many technical ways to make the system faster. However, being a small team, we like easy solutions that don't take very long to implement, and if at all possible, require no major code changes. Here's what we did.

First, we started to show 'The other user is typing...' message, while as you might guess, the other user was typing. Although the delay in delivering a message was still happening, this simple notice gave users a perception of an activity, and as long as something was happening before a message got delivered, it reduced the overall agony.

Second, we elimitated the delay on the sender's part. Not sure why it didn't occur to us right away, but when User1 posts a message, there is absolutely zero need to wait and see if that message gets posted correctly, until it's displayed in the channel back to User1. Moreso, since we know exactly what the message was and exactly what it should look like, we only need to use a Pusher channel for User2 !

Of course, the user on the "other side" of the chat (User2) will still need to wait for the message via Pusher, but for the current user, we can construct an HTML fragment and add it the message stream, immediately after a user clicks "send."

Under this new scenario, the person that is sending a message gets an impression that his message was posted instanteneously; no delay. And, in the worst case if the message was not delivered, we can always remove it from the stream and notify the sender of a problem.

You are probably thinking I am just stupid, for not realizing this earlier, but believe it or not, little things are hard to notice. Now that we have implemented this solution, I suspect that's exactly how other chats, like Facebook, are doing it.

All-in-all, this quick "hack" required only minor changes in our code, took a couple hours to get out of the door, and is now fully functional. The lesson learned here - there is always an easier solution to the problem at hand, but it's hiding under a pile of obvious ones.

-Kirill & Team Scoutzie

If you'd like to leave a comment, please do so here on Hacker News.

p.s. If you are interested to read more from my blog, please subscribe to my newsletter and I will let you know when a new blog post is out.