Technology behind Twitter

Twit­ter

I was inter­ested in the actual Web Tech­nolo­gies used to develop some of the big social media sites. So, I looked at Twit­ter and found some inter­est­ing infor­ma­tion. It was built using Ruby on Rails and did face cer­tain scal­ing issues which led to beliefs that Twit­ter would move off RoR or Rails in mid 2008. There was much con­tro­versy sur­round­ing blame on Ruby for Twitter’s scal­ing prob­lems and if you are inter­ested in nerdy details of how Twit­ter evolved (which I found extreeemely inter­est­ing), check this:

Update 5: Twit­ter on Scala. A Con­ver­sa­tion with Steve Jen­son, Alex Payne, and Robey Pointer by Bill Ven­ners. A fas­ci­nat­ing dis­cus­sion of why Twit­ter moved to the Java JVM for their server infra­struc­ture (long lived processes) and why they moved to Scala to pro­gram against it (high level lan­guage, sta­tic typ­ing, func­tional). Ruby is used on the front-end but wasn’t per­for­mant or reli­able enough for the back-end.

Update 4: Improv­ing Run­ning Com­po­nents at Twit­ter by Evan Weaver. Tells how Twit­ter changed their infra­struc­ture to go from han­dling 3 requests to 139 requests a sec­ond. They moved to a mes­sag­ing model, asyn­chro­nous process, 3 lev­els of cache, and moved their mid­dle­ware to a mix­ture C and Scala/JVM.

I’ve heard about RoR many a times as an extremely fast web devel­op­ment frame­work but never had a chance to use it directly. The Word­Press and user fac­ing inter­face of Face­book is built using PHP. I am very lit­tle famil­iar with other plat­forms such as Cold­fu­sion etc.

To gain some insight into how RoR was selected for Twit­ter, I found this post from AJ at IBM quite interesting:

JACK DORSEY: For exam­ple the use of Rails made the appli­ca­tion acces­si­ble to peo­ple with­ing 2 weeks! The fact that peo­ple saw the appli­ca­tion and reacted to it with such a short time was very impor­tant. Flo­rian Weber , One of the Rails core com­mit­ters was in the team (with close ties to DHH) and that also was entic­ing in choos­ing the frame­work.
The mis­take we did was putting it all out there. We did not fol­low a con­trolled expose method­ol­ogy and that is what came to bite us.
As for the tech­nol­ogy per­spec­tive I would not have changed any­thing! We would be cau­tious and expect explo­sive growth but the plat­form would have been the same.

AJ: These days I work with Rails every­day and given the com­plex­ity of the appli­ca­tion suite we (Max, Roy and me) are build­ing, if we did not have Ruby on Rails, we wouldn’t have done even half the stuff ! Its amaz­ingly pro­duc­tive and given a good rails tool set (I use Eclipse Aptana stu­dio — free ver­sion of course) it becomes a really good com­peti­tor to any Web devel­op­ment frame­work. And Jack just pol­ished the chrome mak­ing rails shine more

Hm, I am tempted to try a hand at RoR soon. I think the real ques­tion is between choos­ing some­thing that helps you launch faster vs some­thing that is robust and scal­able in long term. In the world of Inter­net, ‘launch­ing fast’ is a big advan­tage and since not all the ser­vices explode like Twit­ter, it gives you more time to redesign as your user base expands.

My inter­est was purely tech­ni­cal but here is where you can read how the idea behind Twit­ter was born.

Like

Unlike

No related posts.